Skip to content
System design course
Ch.2 · The building blocks·concept ·5 min read

Checksums and data integrity

A small fingerprint that catches silent corruption in transit and at rest — because hardware and networks quietly flip bits.

The problem nobody likes to admit

Data gets corrupted silently. A bit flips on a network link, a disk sector rots, a buggy router mangles a packet. The bytes you read back aren’t always the bytes you wrote — and without a check, you’d never know until something downstream breaks mysteriously. A checksum is the cheap insurance against that.

How it works

A checksum is a small, fixed-size value computed from a block of data by a hash function (CRC32, MD5, SHA-256, …). The pattern:

  1. Compute the checksum when you store or send the data.
  2. Keep or transmit it alongside the data.
  3. Recompute it when you read or receive, and compare.

If the two differ, the data was corrupted — detected, not silently trusted. The strength of the function sets how reliably corruption is caught (and, for cryptographic hashes, whether tampering is caught too).

Where it’s used

  • Network transfers — TCP, TLS, and storage protocols checksum packets; systems like S3 verify an object’s checksum (e.g. ETag) on upload and download.
  • At rest — filesystems (ZFS), databases, and distributed stores checksum blocks to detect bit rot, often scrubbing in the background to find and repair corrupted replicas.
  • Replication & deduplication — compare checksums to tell whether two copies are identical or whether data actually changed, without comparing every byte.
  • Content addressing — Git, IPFS, and Merkle trees name data by its hash, so integrity is built into the identifier itself.

A note on Merkle trees

When you need to verify or compare large datasets efficiently, build a tree of checksums: leaves hash data blocks, parents hash their children. Comparing two replicas’ root hashes instantly tells you if anything differs, and you can walk the tree to find exactly which block diverged — without scanning everything. Dynamo-style stores use this for anti-entropy (repairing divergent replicas), and it’s the backbone of blockchains and Git.

The interview cue

It’s a small point, but mentioning integrity at the right moment is a maturity signal: “replicas are checksummed and scrubbed to catch silent corruption, and I’d use Merkle trees to compare replicas cheaply during anti-entropy repair.” You’re showing you remember that hardware lies, and that correctness includes the bytes actually surviving.