10.12 — Estimation: Latency Numbers, Capacity Math & Fermi Arithmetic

Every system-design interview asks for numbers, and every real design decision rests on them: how many servers, how much storage, will this fit in memory, is 200 ms achievable. This page gives the three tools — the latency numbers every engineer should know, capacity arithmetic (QPS, storage, bandwidth, connections), and Fermi estimation (getting within an order of magnitude from nothing) — plus the interview technique that turns them into a credible answer in five minutes.

1. Latency numbers every engineer should know

Jeff Dean's famous table, rounded to memorable magnitudes (these are approximate and hardware-dependent — the ratios are the point, and they've stayed stable for two decades):

Operation	Time	Useful comparison
L1 cache reference	~1 ns	the baseline
Branch mispredict	~3 ns	1.5
L2 cache reference	~4 ns	4× L1
Mutex lock/unlock	~17 ns
Main memory reference	~100 ns	100× L1 (1.6)
Compress 1 KB (snappy)	~2 μs
Send 1 KB over 1 Gbps network	~10 μs
SSD random read	~16 μs	150× RAM
Read 1 MB sequentially from memory	~50 μs
Round trip within a datacenter	~500 μs
Read 1 MB sequentially from SSD	~1 ms	20× memory
Disk seek (HDD)	~2 ms
Read 1 MB sequentially from HDD	~5 ms
Round trip CA → Netherlands → CA	~150 ms	speed of light, unimprovable

The five conclusions that actually drive design:

1. Memory is about 100 times faster than an SSD, and about 100,000 times faster than a round trip across a continent. That is why caching wins so clearly (10.2).
2. Reading in order beats jumping around by about ten times, on every kind of storage. That is why logs are fast (10.8.2) and why B-trees exist (7.3).
3. A round trip inside a datacenter takes about 0.5 ms. So tens of internal calls are affordable, but hundreds are not.
4. Latency between regions is physics. A round trip is about 150 ms, and no engineering removes it. Only not making the call does, which is what CDNs and regional copies buy you.
5. Network round trips outweigh everything else in a distributed system. So the design question is almost always how many round trips, not how fast is each one.

2. Capacity arithmetic

The numbers to have memorized because they turn requirement statements into infrastructure:

• Seconds per day ≈ 86,400 ≈ 10⁵; per month ≈ 2.6 × 10⁶. So 1 million requests/day ≈ 12 QPS average, and 1 QPS sustained ≈ 86k/day.
• Peak-to-average ratio is typically 2–5× for consumer products (higher for event-driven spikes) — always design to peak, and state the multiplier you assumed.
• Read/write ratio — usually 10:1 to 1000:1 for consumer systems. This single number decides most of the architecture (10.2).
• Storage: rows × bytes/row × replication factor × (1 + index overhead) × retention. Round bytes generously: a "small" record with a few IDs, timestamps, and short strings is ~200–500 bytes in practice, not 50.
• Bandwidth: QPS × payload size. 10k QPS × 10 KB = 100 MB/s ≈ 800 Mbps — which is where "we need a CDN" stops being a preference.
• Memory for caching: working set size × safety factor. The useful heuristic is that the hot 20% of data usually serves 80% of requests, so caching capacity is often a small fraction of total data.
• Connections: each app instance × pool size ≤ database max connections (10.2's scaling trap). 50 instances × 20 connections = 1000 — more than most default Postgres configurations allow, which is why poolers exist.

Worked example — a photo-sharing app: 10M daily active users, each uploading 2 photos/day and viewing 50. Writes: 20M uploads/day ÷ 86,400 ≈ 230 QPS average, ×3 peak ≈ 700 QPS. Reads: 500M views/day ≈ 5,800 QPS average, ×3 ≈ 17k QPS — a 25:1 read/write ratio, so reads dominate the design. Storage: 20M photos/day × 2 MB (original + thumbnails) = 40 TB/day, ≈ 1.2 PB/month — which immediately says object storage plus CDN, not a database, and makes retention/tiering a first-order product decision, not an afterthought. Bandwidth (reads): 17k QPS × 500 KB (typical delivered image) ≈ 8.5 GB/s — unambiguously a CDN problem; serving that from origin is not a scaling exercise but a different architecture. Notice how three multiplications converted a product statement into architectural constraints — that is the skill.

3. Fermi estimation

It is named after Enrico Fermi, who famously estimated the power of the first atomic bomb test by dropping scraps of paper and watching how far the blast pushed them. The technique has three steps:

1. Break the unknown number into smaller numbers you can put limits on.
2. Estimate each one to within a factor of ten.
3. Multiply them together.

Errors in separate estimates partly cancel out. So the answer is usually within 3 to 10 times the truth, which is close enough to make architecture decisions.

What makes it believable in an interview:

• Say every assumption out loud, and invite correction. "Assuming 50 million users, 20% active each day, and peak at 3 times average. Tell me if those are off."
• Round hard. Use 100,000 seconds in a day, not 86,400. Precision gives false comfort at this stage.
• Carry the units through every step. Units catch mistakes. If the answer does not come out in GB/s, the arithmetic is wrong.
• Check the result against something you know. "8.5 GB/s is about what a mid-sized CDN serves in one region. Plausible for a big photo app, absurd for a startup. Let me re-check the user count."

The most common estimation mistakes, worth guarding against on purpose:

• Forgetting copies and indexes in storage. Three copies plus indexes can double or triple a simple estimate.
• Using averages instead of peaks. Systems fail at the peak, not on average.
• Forgetting metadata and protocol overhead. A 1 KB payload is more than 1 KB on the wire.
• Mixing up bits and bytes in bandwidth. That factor of 8 has embarrassed many people at the whiteboard.

4. The expert lens

Estimation's purpose is to eliminate options, not to predict. You are not computing the answer; you are discovering that one design is impossible and another is comfortable. "40 TB/day" doesn't tell you which storage vendor to use — it tells you that a relational database is off the table and a lifecycle policy is mandatory. That's a decision made in ninety seconds that would otherwise take a design cycle to discover.

Know a handful of anchors and derive the rest. Nobody memorizes tables of numbers under pressure; you memorize ~10 anchors (memory ~100 ns, SSD read ~100 μs-ish, datacenter round trip ~0.5 ms, cross-continent ~150 ms, 86k seconds/day, a modern server handling low-thousands QPS for simple work, ~1 Gbps ≈ 125 MB/s) and derive everything else by multiplication and ratio. The anchors plus arithmetic beats the table plus recall.

Numbers change conversations from opinions to constraints. "That might not scale" is an opinion; "at 17k QPS and 500 KB per response, we'd need 8.5 GB/s from origin — a CDN isn't optional" is a constraint everyone can verify. In interviews it is the single strongest signal of seniority; in real design reviews it is how expensive mistakes get caught before they're built. Practice the arithmetic until it's fast, because the speed is what makes it usable in a conversation.

Next: 10.13 — Part 10's closing chapter: an honest engineering tour of blockchain, what its consensus buys, and where it does and doesn't belong.

Recall

• Latency anchors: L1 ~1 ns · RAM ~100 ns (100× L1) · SSD random read ~16 μs · datacenter round trip ~0.5 ms · SSD 1 MB sequential ~1 ms · HDD seek ~2 ms · cross-continent round trip ~150 ms (physics). Conclusions: caching wins by orders of magnitude; sequential beats random everywhere; count round trips, not per-call speed; cross-region latency can only be avoided, never optimized.
• Capacity maths:
  • 86,400 seconds in a day is about 10⁵, so 1 million a day is about 12 per second.
  • Design for the peak, 2 to 5 times the average, and say which.
  • The ratio of reads to writes decides the architecture.
  • Storage = rows × bytes per row × number of copies × (1 + index overhead) × how long you keep it.
  • Bandwidth = requests per second × size of each.
  • Connections = servers × pool size, which must stay under the database's maximum. Connection poolers exist for exactly this.
• Fermi method: decompose → bound each factor within an order of magnitude → multiply (errors partially cancel). Discipline: state assumptions aloud, round aggressively, carry units (the error detector), and sanity-check against a known quantity. Classic errors: ignoring replication/index overhead, using averages not peaks, forgetting protocol overhead, bits vs bytes.
• Lens: estimation eliminates options rather than predicting (40 TB/day rules out a relational store in 90 seconds); memorize ~10 anchors and derive the rest; numbers convert opinions into verifiable constraints — the strongest seniority signal in a design conversation.

Self-test: Give the ratio of RAM to SSD to cross-continent latency. Convert 1M requests/day to average and peak QPS. Compute storage for 20M records/day at 300 bytes with 3× replication over 90 days. Name four classic estimation errors. What is estimation actually for?

Quiz Bank

FoundationalWhich latency numbers should you know, and what design conclusions follow?

The numbers to remember:

• L1 cache: about 1 ns.
• Main memory: about 100 ns.
• A random read from an SSD: about 16 μs.
• A round trip inside a datacenter: about 0.5 ms.
• Reading 1 MB in order from an SSD: about 1 ms.
• A hard-disk seek: about 2 ms.
• A round trip across a continent: about 150 ms.

Ratios matter more than absolutes: memory is ~100× faster than SSD access latency and roughly a million times faster than a cross-continent round trip.

Design conclusions: (1) Caching is the highest-leverage optimization available — moving a read from cross-region to local memory is a five-order-of-magnitude change, which is why CDNs and cache tiers dominate scaling discussions (10.2). (2)

Sequential access beats random by ~10× on SSD and ~100× on HDD, which is why append-only logs are fast (10.8.2), why B-trees minimize random seeks ([7.3]), and why "read 1 MB sequentially" is comparable to a single random read on disk. (3)

Count round trips: at ~0.5 ms each, ten internal calls cost 5 ms (fine) and two hundred cost 100 ms (a redesign) — the distributed N+1 (10.11). (4) Cross-region latency is physics — ~150 ms is the speed of light in fiber plus routing, so it can only be avoided (replicas, CDNs, regional writes — 10.5), never tuned away. (5) Anything user-facing has a ~100–200 ms budget before it feels sluggish, which is your entire round-trip allowance — spend it deliberately.

FoundationalDo the capacity math for a system with 50M users, 30% daily active, each performing 20 actions/day with a 100:1 read/write ratio.

Daily active users: 50M × 30% = 15M DAU. Total actions/day: 15M × 20 = 300M/day. Average QPS: 300M ÷ 86,400 ≈ 3,500 QPS (use ~10⁵ s/day for mental math: 300M/10⁵ = 3,000 — close enough, and faster). Peak QPS: ×3 (typical consumer peak-to-average; state the assumption) ≈ 10,000 QPS.

Split by ratio: 100:1 reads to writes means ~99% reads → ~10,000 read QPS peak, ~100 write QPS peak. That split is the architectural headline: writes are trivially handled by a single primary database, while reads demand caching and/or replicas (10.2) — and it tells you where to spend design effort.

Storage: if each write stores ~500 bytes, 3M writes/day × 500 B = 1.5 GB/day raw; ×3 replication ×2 for indexes ≈ 9 GB/day ≈ 3.3 TB/year — comfortable for a single database with retention policy, which is a different conclusion than a naive raw-bytes estimate would suggest, and exactly why replication and index overhead must be in the formula.

Bandwidth: 10k read QPS × 20 KB response ≈ 200 MB/s ≈ 1.6 Gbps — significant but manageable, and a strong argument for caching responses at the edge. Connections: if 40 app instances each hold a pool of 20, that's 800 connections — above many default database limits, so a connection pooler is required (10.2). Five multiplications produced four architectural decisions.

AppliedExplain Fermi estimation and the discipline that makes it credible in an interview.

The method: decompose an unknown quantity into factors you can each bound within an order of magnitude, estimate each, and multiply. Independent errors partially cancel (some estimates high, some low), so the product typically lands within 3–10× of truth — which is more than sufficient to distinguish "one server" from "a fleet with a CDN."

The discipline that makes it credible: (1) Say every assumption out loud, and invite correction. For example: "50 million registered, 30% active each day, peak at 3 times average. Stop me if those are wrong." The interviewer really wants to see whether you reason openly. And if they privately disagree with a number, this turns it into something you can settle together. (2)

Round aggressively — 86,400 becomes 10⁵, 512 becomes 500; precision at this stage is false comfort and slows you down. (3) Carry units through every step — units are the built-in error detector: if you're computing bandwidth and the units don't reduce to bytes/second, the arithmetic is wrong, and the mistake is usually a missing or extra factor. (4)

Sanity-check against a known anchor — "8.5 GB/s is a large CDN's regional throughput, so for a global photo app that's plausible; for a startup it means I've overestimated users." (5) Say what the number means. An estimate is worthless until you turn it into a decision. For example: "so serving from our own servers is ruled out. This is a design built on a CDN and object storage." The classic mistakes to avoid are the four above: forgetting copies and indexes, using averages where peaks matter, ignoring protocol overhead, and mixing up bits and bytes.

InterviewIn a design interview, when and how should you produce numbers?

When: immediately after clarifying requirements and before drawing the architecture — because the numbers choose the architecture, and presenting a design first and estimating afterward inverts the logic (and risks defending a design the arithmetic disqualifies). Then again at the end, to size components and validate the design against the constraints you derived.

How, in about three minutes:

1. Start from the numbers about users that the interviewer gave you, or ask for them: daily active users, actions per user, data sizes and how long data is kept.
2. Work out the average requests per second. Multiply by a stated peak factor. Split it by the ratio of reads to writes.
3. Work out storage per day and per month, including copies and indexes.
4. Work out bandwidth: requests per second × size of each.
5. Then state the three constraints that follow. For example: "Reads outnumber writes 100 to 1, so caching and read copies. 40 TB a day rules out a relational database for the files. 8 GB/s going out means a CDN."

Presentation matters as much as arithmetic: round to memorable numbers, keep the running estimates visible on the board so you and the interviewer share state, flag which assumptions the design is sensitive to ("if peak is really 10× rather than 3×, we need X"), and never present a number without its implication.

The signal being tested is not arithmetic skill — it's whether you make architectural choices from evidence rather than from familiarity, and whether you can be corrected mid-flight without your design collapsing (which is why assumptions must be explicit and separable).

StaffA team proposes storing all application events in Postgres 'for now' — 500M events/day, 1 KB each, kept 2 years. Use estimation to evaluate.

Do the arithmetic first, then interpret. Raw ingest: 500M × 1 KB = 500 GB/day. Over two years: 500 GB × 730 ≈ 365 TB raw. Add realistic overheads: ×3 replication for durability (10.5) and roughly ×1.5–2 for indexes and row overhead (Postgres row headers, TOAST, index B-trees — [7.3]) ⇒ 1.6–2.2 PB of provisioned storage. Write rate: 500M ÷ 86,400 ≈ 5,800 writes/sec average, peaking perhaps 15–20k/sec — which is at or beyond the practical ceiling for a single Postgres primary with indexes and durability enabled, before any read load.

What it means. Three facts rule things out, and each came from one multiplication:

1. Petabytes in a relational database is wrong, both to run and to pay for. The storage bill alone dwarfs the cost of computing, and cleaning up and maintaining indexes at that size becomes the main job of the operations team.
2. 15,000 writes per second, all of them appends, is exactly what a log is built for, and what a B-tree database is not (10.8.2).
3. Keeping raw events for two years means people will ask questions about history. Those are analysis questions, not transactions. That is the job of a columnar warehouse (7.8), where compression usually shrinks data 5 to 10 times and queries get cheaper to match.

The counter-proposal, derived from the same numbers: stream events to a durable log for the operational window (days), land them in object storage in a columnar format (Parquet) partitioned by date for the long tail, and keep in Postgres only the small, queryable derived state the application actually serves (10.8.4's read models). Then ask the question the estimate exposes:

is two years of raw retention a real requirement, or an unexamined default? Storing aggregates beyond 90 days often satisfies the actual need at 1% of the cost — and that question, prompted by five minutes of arithmetic, is usually worth more than any of the technology choices.

Flashcards

FlashLatency anchors

L1 ~1 ns · RAM ~100 ns · SSD random ~16 μs · DC round trip ~0.5 ms · SSD 1 MB seq ~1 ms · HDD seek ~2 ms · cross-continent ~150 ms (physics).

FlashQPS conversion

86,400 s/day ≈ 10⁵ ⇒ 1M/day ≈ 12 QPS average. Always ×2–5 for peak and say the multiplier. 1 QPS ≈ 86k/day.

FlashStorage formula

rows × bytes/row × replication × (1 + index overhead) × retention. Naive estimates are 3–6× low.

FlashFermi discipline

Decompose → bound each factor → multiply. State assumptions aloud, round hard, carry units (error detector), sanity-check against an anchor, translate into a decision.

FlashClassic estimation errors

Ignoring replication/index overhead · averages instead of peaks · forgetting protocol overhead · bits vs bytes (factor of 8).

FlashWhat estimation is for

Eliminating options, not predicting. "40 TB/day" rules out a relational store in 90 seconds — a decision that otherwise costs a design cycle.

Scenario Drill

DrillEstimate, end to end, the infrastructure for a WhatsApp-like messaging service: 500M daily active users, 40 messages sent per user per day, average message 200 bytes, media in 10% of messages averaging 500 KB, messages retained 30 days on servers, plus online presence. Produce the numbers and the three architectural conclusions they force.

Message volume. 500M DAU × 40 = 20B messages/day ⇒ 20B ÷ 10⁵ ≈ 200k messages/sec average; peak ×3 ≈ 600k/sec. Each message is delivered to at least one recipient (group chats multiply this — assume ×1.5 for group fan-out) ⇒ ~900k deliveries/sec at peak. That number alone eliminates a single-cluster design and forces partitioning by conversation or user (10.6).

Text storage. 20B × 200 B = 4 TB/day raw; ×3 replication ≈ 12 TB/day; over 30 days retention ≈ 360 TB for text — large but tractable in a partitioned store, and the 30-day window is what keeps it so (worth confirming as a product decision, because "keep forever" would multiply it by 24× per two years).

Media storage. 10% of 20B = 2B media/day × 500 KB = 1 PB/day — and here the arithmetic screams: at 30 days that's 30 PB live, ×replication. Conclusion: media never touches the message store; it goes to object storage with lifecycle tiering, and messages carry references (10.8.4's claim-check pattern). This is the single most consequential number in the estimate.

Bandwidth. Text: 900k deliveries/sec × 200 B ≈ 180 MB/s — trivial. Media: if 10% of deliveries carry media, ~90k/sec × 500 KB ≈ 45 GB/s — unambiguously a CDN/edge-delivery problem, not an origin problem (10.2). Connections and presence. 500 million daily users, with say 20% connected at any moment, means 100 million connections open at the same time. A carefully tuned server can hold about 100,000 connections. That figure is real but optimistic (2.7). So you need at least about 1,000 servers just to hold the connections.

Then there is presence: online, offline and "typing…". Even one presence update per connection per minute is 1.6 million events per second. That is more than the message rate. So presence must be treated as short-lived data that can be lost and is heavily combined (10.7.1), not as durable messages.

The three architectural conclusions forced by these numbers: (1) Media and messages are different systems — 1 PB/day of media in object storage behind a CDN, with messages carrying pointers; conflating them makes the message store impossible. (2) Partition by conversation, with per-conversation ordering — 600k messages/sec cannot be serialized globally, and per-conversation is the smallest scope where ordering actually matters (10.3/10.6).

(3) Presence is a separate, ephemeral, lossy subsystem — its event rate exceeds the messaging rate, so treating it with messaging's durability guarantees would cost more than the product itself; it belongs in an in-memory, gossip-style or pub/sub tier with aggressive aggregation. Note what happened: five minutes of multiplication produced three decisions that would otherwise emerge painfully from a year of production, and each conclusion is defensible by pointing at one number.