Storage Basics

Every project eventually has to put its data somewhere. Nearly always the choice is between block storage and object storage. From a distance they look like the same thing with different billing; once you start writing to them they behave nothing alike.

Block Storage

Block storage gives you a disk. Not a metaphor for a disk, an actual block device: a fixed-size range of addresses carved into chunks of 512 bytes or 4 KB, with no concept of a file. Something else has to supply that concept, and that something is a filesystem. You run mkfs once, ext4 or XFS or APFS writes its own bookkeeping into the first blocks, and from then on the kernel translates /data/results.csv into “blocks 88,412 through 88,530.”

On AWS this is EBS. On GCP it’s Persistent Disk. In a machine room it’s whatever the SAN hands out over iSCSI. The laptop you’re reading this on has one soldered to the board.

The detail that drives everything else is attachment. A block volume connects to one machine, the way an external drive does. That machine mounts it and treats it as a normal path. Detach it and you have an inert volume sitting in a single availability zone, reachable by nothing.

A single virtual machine attached to one block volume made of fixed-size blocks Virtual Machine filesystem (ext4) mounted at /data attached 1 volume : 1 host Block Volume (fixed size) edit One byte changes, one 4 KB block gets rewritten

What it’s good at

  • Speed. The volume sits on the host or one hop away on a dedicated storage network. A gp3 volume starts at 3,000 IOPS and 125 MB/s and can be dialed up to 16,000 IOPS. Latency is measured in microseconds and low milliseconds, not round trips.
  • Editing in place. Change four bytes in the middle of a 40 GB file and the drive rewrites one block. Everything transactional depends on this. Postgres writes 8 KB pages and flushes a write-ahead log on every commit; if each of those commits meant rewriting the whole database file, the database would not work.
  • Programs already expect it. File locks, seek(), mmap, atomic renames, permissions, symlinks. SQLite needs locking. Git needs renames. If a tool has ever asked you for a path, it wants a block device underneath.
  • Cheap snapshots, because the service only stores the blocks that changed since the last one.

Where it hurts

  • One host at a time. Two machines that both need the data means running NFS or EFS or Lustre on top, which is a second system to operate.
  • The size is a decision you make up front. Growing a volume means expanding the volume and then running resize2fs or xfs_growfs to tell the filesystem about the new room. Shrinking usually isn’t possible at all.
  • You’re billed for what you allocated. A 1 TB volume holding 4 GB of data costs exactly what a full one costs.
  • It lives in one availability zone. If the zone goes, so does the volume, unless you’ve been taking snapshots.
  • Roughly $0.08 per GB-month for gp3 in us-east-1, against $0.023 for S3 Standard. Check current prices before you quote those, but the ratio holds.
  • Everything about it is yours to run: filesystem choice, backups, fsck after a bad shutdown, and the alert at 3 a.m. saying the disk is full.

Object Storage

Object storage drops the disk idea entirely. There’s no device to attach and no filesystem to create. You get a bucket, and into it you put objects: a blob of bytes, a key that identifies it, and whatever metadata you attach. All of it moves over HTTPS with PUT, GET, and DELETE.

The namespace is flat. aws s3 ls s3://my-bucket/raw/2026/ prints something that looks like a directory listing, but nothing is nested; the service is matching a string prefix against every key in the bucket, and the slashes are ordinary characters that the console draws as folders because people find that easier to read.

Objects are also immutable. There’s no seek-and-write, no appending. Change one byte in a 4 GB file and you upload 4 GB.

Many clients reaching one bucket of keyed objects over an HTTP API, replicated across three availability zones laptop web app Spark job CI pipeline GET / PUT over HTTPS Bucket (flat, no size limit) raw/2026/08/readings.parquet 142 MB · etag · owner · storage-class models/v3/weights.safetensors 4.1 GB · etag · owner · storage-class site/index.html 8 KB · etag · owner · storage-class billions more, same price per GB copy · AZ 1 copy · AZ 2 copy · AZ 3 replicated for you One byte changes, the whole object gets re-uploaded No host to attach to, no capacity to provision

Why everyone copied S3

S3 shipped in March 2006, before EC2, and the API turned out to be small enough that competitors implemented it rather than inventing their own. That is why MinIO, Cloudflare R2, Backblaze B2, Wasabi, Ceph, and Google Cloud Storage all answer S3 calls today, and why boto3, rclone, DuckDB, Spark, and pandas can read from any of them after you point them at a different endpoint. Time spent learning the S3 API is not time spent learning one vendor’s product. The lock-in is in the egress bill, not the interface.

What it’s good at

  • You never size it. There is nothing to provision and nothing to grow. A bucket holding 3 KB and a bucket holding 3 PB are configured identically. The only ceiling worth knowing is 5 TB per object, and anything over 5 GB goes up as a multipart upload, which the CLI and the SDKs handle for you.
  • It survives things. Every object is written to several separate facilities before the PUT returns. AWS quotes eleven nines of annual durability for S3 Standard, and you get that number without configuring replication, RAID, or backups.
  • Nothing is attached to it. The bucket has no host. A hundred machines across three regions can read the same object at the same time, and the data keeps existing after the cluster that produced it is torn down. That property is the reason data lakes are built this way.
  • Storing things is cheap enough to stop thinking about. Around $0.023 per GB-month for Standard, plus fractions of a cent per thousand requests.
  • Lifecycle rules move old data down the price ladder to Glacier and its colder tiers on a schedule you set once.
  • Every object has a URL, so the same bucket can serve a static site, hand out a dataset, or accept uploads through presigned links.

Where it hurts

  • Latency is an HTTP round trip, tens to hundreds of milliseconds. Reading one 200 MB Parquet file is fine. Reading 200,000 tiny files is miserable, and the fix is always the same: fewer, larger files.
  • Immutability shapes your whole pipeline. You don’t append a day of records to yesterday’s file, you write today’s file next to it and let the query engine read both.
  • It is not a filesystem and won’t pretend convincingly. No locks, no seek(), and a rename is a copy followed by a delete, which is why “moving” a large prefix takes real time and real money. s3fs and Mountpoint will mount a bucket for you, and the seams show under load.
  • S3 has been strongly read-after-write consistent since 2020, but plenty of S3-compatible services still aren’t. Verify before you build on it.
  • Requests and egress are where the surprise bills come from. A job issuing millions of small GETs can spend more on requests than on storage, and data leaving the cloud runs about $0.09 per GB.
  • Buckets are easy to expose. Public-by-accident buckets have leaked more records than most attacks have.

Picking one

Situation Use
A database, or anything with a hot write path Block
Software that expects an ordinary file path Block
Heavy random reads and writes inside large files Block
Raw data feeding analytics, a data lake, a warehouse staging area Object
Backups, archives, logs, model checkpoints Object
Anything many machines or many people read at once Object
Data that has to outlive the machine that made it Object

Most real systems use both, in a specific arrangement: the bucket holds the authoritative copy, and the volume holds a fast scratch copy. A training run pulls its dataset from S3 onto a local disk, hammers that disk with random reads for six hours, writes checkpoints back to S3, and then the instance and its volume are deleted. Nothing of value lived on the volume.

If you’re editing bytes where they sit, you want block. If you’re writing whole files once and reading them many times, you want object.

Going further