Hide outline
Feedback
Guide
Was this lesson helpful?
Dive Deeper
Generate a follow-up sub-lesson on any aspect of this topic
Guide complete
Object And Blob Storage For System Design
Generate a follow-up sub-lesson on any aspect of this topic
Object And Blob Storage For System Design
TLDR;
- A single server with a local disk fails first on backups and capacity, not on HTTP throughput.
- The sharpest difference across block, file, and object storage is who owns the namespace and consistency.
- Listing behavior, not
GETbehavior, is what usually breaks beginner gallery and feed designs.
Storing original photos is simplest when the app server writes image bytes to its own local disk and stores rows in a relational database for users, captions, and the path to each file. That baseline looks fine at 2 TB of photos and 50k uploads per day until the disk fills and the first serious backup takes hours, because the backup now competes with uploads and can only go as fast as one machine’s I/O. Let’s ground that baseline architecture before we change it.
In that diagram, the app server owns both the database connection and the filesystem writes, so scaling storage means either moving bytes off the box or adding a shared storage layer that multiple app servers can agree on.
Block storage works when the application wants a mountable volume and is willing to manage filesystems and directories itself, like a database storing its own pages on an attached disk device. File storage works when multiple servers need shared folders with POSIX-like semantics, meaning paths, directories, and operations like rename have system-level meaning across clients.
Object storage works when the application can treat each photo as a single retrievable object over HTTP, and the storage system owns durability and scaling while the app owns naming through keys instead of directories.
This is the mental model shift that matters, since with object storage the app does not mount a disk and browse folders, it sends PUT and GET for a named object and usually cannot rely on directory operations for correctness.
Here is a side by side view of the tradeoffs.
The scan highlights that block storage scales by volume and IOPS, file storage scales by shared namespace coordination, and object storage scales by buckets and key partitions, which is why media and logs fit it so often.
An object or blob is just a byte array stored under a name, plus metadata the store returns with it, and you usually organize many objects inside a bucket which is a top-level container for keys and policy.
A key is the full object name inside the bucket, and in most object stores it is a flat namespace even if you embed slashes like users/123/original/abc.jpg for human readability.
For the photo app, the upload path becomes writing the image bytes to a bucket under a deterministic key, storing the key in the SQL table, and setting metadata like Content-Type so browsers and CDNs cache it correctly.
Immutability is typically by convention, meaning the app chooses keys that never change for originals, such as including a content hash in the key, so readers never see partial overwrites or surprising updates.
Let’s break a single object request into its parts.
The building blocks show that the bucket selects the logical container, the key selects exactly one object, the bytes are the payload, and metadata controls how downstream systems interpret and cache the response.
Durability and availability are the real reason teams move media off a single disk, because a local filesystem plus RAID still shares fate within one machine, rack, or operator mistake. When an object store advertises durability like , it is claiming the probability of losing an object is extremely low over a year, usually by storing redundant fragments across independent failure domains.
A simple intuition is replication versus erasure coding, where replication stores full copies and erasure coding stores data plus parity so the system can rebuild after losing a few parts. The design effect on our photo app is that backups shift from copying the entire 2 TB dataset nightly to instead backing up just the SQL metadata and being able to re-derive or re-fetch image bytes from the durable object store. We can tune the overhead versus durability tradeoff to see what changes.
The slider makes visible that higher durability targets generally require more redundancy, and the redundancy can be paid as extra full copies or as parity fragments depending on the store’s internal design.
Read heavy scale hurts when the same servers that accept uploads also serve every image view, since hot photos drive repeated reads and outbound bandwidth that do not need database logic. The common pattern is to write originals into object storage, then serve them through a CDN that caches at the edge, so most views do not hit the origin object store at all.
For the photo app, the app server stays in the control plane, meaning it authenticates uploads and writes SQL rows, while the data plane becomes CDN edge GET requests that fetch objects by key.
Cost follows the same split, since high CDN hit rate reduces origin egress and origin request charges, and it also improves latency because the edge is closer than the object store region.
Let’s estimate what views and cache hit rate do to origin load.
The estimate ties daily views, average size, and hit rate to a smaller origin reads per second and lower egress, and it separates edge latency from origin latency as two different user experiences.
Many object stores give strong read after write for a newly written object when you GET the exact key, but listings over a prefix can be eventually consistent or delayed under load.
That means a naive gallery that does list then read can miss a recent upload, since the LIST response is stale even though the object is present and readable by its key.
In the photo app this appears as a user uploading a photo and not seeing it in their gallery for seconds, which becomes confusing when the direct link works but the grid does not. Safer patterns are write known key and then insert a row in SQL that powers the gallery, or use retries with backoff for listings when you must list, and design keys with prefixes that reduce hot partitions. Here is the failure story and fixes in one flow.
The reveal shows that the break happens at LIST, not GET, and the fixes work by making SQL the source of truth for ordering and membership while object storage stays the source of truth for bytes.
Object storage is the right default for media originals, static assets, backups, and append-only logs, because the app can tolerate key-based access and wants cheap durability at large scale. It is the wrong default when the workload needs low latency random writes to small regions, atomic renames, or shared POSIX semantics, which is where block storage for databases and file storage for shared home directories tend to fit better.
In our photo app, thumbnails can live in object storage because they are immutable and cacheable, user home directories tend to want file storage semantics, and a database write ahead log belongs on block storage attached to the database host. A redesign is forced when a new constraint demands shared mutable files or strict directory operations for correctness, because that constraint moves the problem from storing bytes durably to coordinating a shared namespace. Let’s sanity check those choices against a few prompts.
If the checkpoint feels hard, it usually means the requirement is really about namespace and write patterns, not about how many gigabytes you need to store.