10.12 — Estimation: Latency Numbers, Capacity Math & Fermi Arithmetic

Every system-design interview asks for numbers, and every real design decision rests on them: how many servers, how much storage, will this fit in memory, is 200 ms achievable. This page gives the three tools — the latency numbers every engineer should know, capacity arithmetic (QPS, storage, bandwidth, connections), and Fermi estimation (getting within an order of magnitude from nothing) — plus the interview technique that turns them into a credible answer in five minutes.

1. Latency numbers every engineer should know

Jeff Dean's famous table, rounded to memorable magnitudes (these are approximate and hardware-dependent — the ratios are the point, and they've stayed stable for two decades):

Operation	Time	Useful comparison
L1 cache reference	~1 ns	the baseline
Branch mispredict	~3 ns	1.5
L2 cache reference	~4 ns	4× L1
Mutex lock/unlock	~17 ns
Main memory reference	~100 ns	100× L1 (1.6)
Compress 1 KB (snappy)	~2 μs
Send 1 KB over 1 Gbps network	~10 μs
SSD random read	~16 μs	150× RAM
Read 1 MB sequentially from memory	~50 μs
Round trip within a datacenter	~500 μs
Read 1 MB sequentially from SSD	~1 ms	20× memory
Disk seek (HDD)	~2 ms
Read 1 MB sequentially from HDD	~5 ms
Round trip CA → Netherlands → CA	~150 ms	speed of light, unimprovable

The five conclusions that actually drive design: memory is ~100× faster than SSD and ~10⁵× faster than a cross-continent round trip — which is why caching wins so decisively (10.2); sequential beats random by an order of magnitude on every storage medium — the reason logs are fast (10.8.2) and B-trees exist ([7.3]); a datacenter round trip is ~0.5 ms, so tens of internal calls are affordable while hundreds are not; cross-region latency is physics (~150 ms round trip, and no engineering removes it — only not making the call does, which is what CDNs and regional replicas buy); and network round trips dominate everything else in a distributed system, so the design question is almost always how many round trips rather than how fast is each one.

2. Capacity arithmetic

The numbers to have memorized because they turn requirement statements into infrastructure:

• Seconds per day ≈ 86,400 ≈ 10⁵; per month ≈ 2.6 × 10⁶. So 1 million requests/day ≈ 12 QPS average, and 1 QPS sustained ≈ 86k/day.
• Peak-to-average ratio is typically 2–5× for consumer products (higher for event-driven spikes) — always design to peak, and state the multiplier you assumed.
• Read/write ratio — usually 10:1 to 1000:1 for consumer systems. This single number decides most of the architecture (10.2).
• Storage: rows × bytes/row × replication factor × (1 + index overhead) × retention. Round bytes generously: a "small" record with a few IDs, timestamps, and short strings is ~200–500 bytes in practice, not 50.
• Bandwidth: QPS × payload size. 10k QPS × 10 KB = 100 MB/s ≈ 800 Mbps — which is where "we need a CDN" stops being a preference.
• Memory for caching: working set size × safety factor. The useful heuristic is that the hot 20% of data usually serves 80% of requests, so caching capacity is often a small fraction of total data.
• Connections: each app instance × pool size ≤ database max connections (10.2's scaling trap). 50 instances × 20 connections = 1000 — more than most default Postgres configurations allow, which is why poolers exist.

Worked example — a photo-sharing app: 10M daily active users, each uploading 2 photos/day and viewing 50. Writes: 20M uploads/day ÷ 86,400 ≈ 230 QPS average, ×3 peak ≈ 700 QPS. Reads: 500M views/day ≈ 5,800 QPS average, ×3 ≈ 17k QPS — a 25:1 read/write ratio, so reads dominate the design. Storage: 20M photos/day × 2 MB (original + thumbnails) = 40 TB/day, ≈ 1.2 PB/month — which immediately says object storage plus CDN, not a database, and makes retention/tiering a first-order product decision, not an afterthought. Bandwidth (reads): 17k QPS × 500 KB (typical delivered image) ≈ 8.5 GB/s — unambiguously a CDN problem; serving that from origin is not a scaling exercise but a different architecture. Notice how three multiplications converted a product statement into architectural constraints — that is the skill.

3. Fermi estimation

Named for Enrico Fermi (who famously estimated the Trinity test's yield by dropping paper scraps), the technique is: decompose the unknown into quantities you can bound, estimate each within an order of magnitude, and multiply — errors in independent estimates tend to partially cancel, so the product is usually within 3–10× of truth, which is enough to make architectural decisions.

The discipline that makes it credible in an interview: state every assumption aloud and invite correction ("assuming 50M users, 20% daily active, 3× peak — tell me if those are off"), round aggressively (use 100k seconds/day, not 86,400 — precision is false comfort at this stage), carry units through every step (units are the error-detector: if the units don't come out as GB/s, the arithmetic is wrong), and sanity-check the result against something known ("8.5 GB/s is roughly a mid-sized CDN's regional throughput — plausible for a large photo app, absurd for a startup, so let me re-check the user count").

The most common estimation errors, worth guarding against explicitly: forgetting replication and index overhead in storage (3× replication plus indexes can double or triple naive estimates), using averages instead of peaks (systems fail at peak, not at average), forgetting metadata and protocol overhead (a 1 KB payload is not 1 KB on the wire), and confusing bits and bytes in bandwidth (a factor of 8 that has embarrassed many whiteboards).

4. The expert lens

Estimation's purpose is to eliminate options, not to predict. You are not computing the answer; you are discovering that one design is impossible and another is comfortable. "40 TB/day" doesn't tell you which storage vendor to use — it tells you that a relational database is off the table and a lifecycle policy is mandatory. That's a decision made in ninety seconds that would otherwise take a design cycle to discover.

Know a handful of anchors and derive the rest. Nobody memorizes tables of numbers under pressure; you memorize ~10 anchors (memory ~100 ns, SSD read ~100 μs-ish, datacenter round trip ~0.5 ms, cross-continent ~150 ms, 86k seconds/day, a modern server handling low-thousands QPS for simple work, ~1 Gbps ≈ 125 MB/s) and derive everything else by multiplication and ratio. The anchors plus arithmetic beats the table plus recall.

Numbers change conversations from opinions to constraints. "That might not scale" is an opinion; "at 17k QPS and 500 KB per response, we'd need 8.5 GB/s from origin — a CDN isn't optional" is a constraint everyone can verify. In interviews it is the single strongest signal of seniority; in real design reviews it is how expensive mistakes get caught before they're built. Practice the arithmetic until it's fast, because the speed is what makes it usable in a conversation.

Next: 10.13 — Part 10's closing chapter: an honest engineering tour of blockchain, what its consensus buys, and where it does and doesn't belong.

Recall

• Latency anchors: L1 ~1 ns · RAM ~100 ns (100× L1) · SSD random read ~16 μs · datacenter round trip ~0.5 ms · SSD 1 MB sequential ~1 ms · HDD seek ~2 ms · cross-continent round trip ~150 ms (physics). Conclusions: caching wins by orders of magnitude; sequential beats random everywhere; count round trips, not per-call speed; cross-region latency can only be avoided, never optimized.
• Capacity math: 86,400 s/day ≈ 10⁵ ⇒ 1M/day ≈ 12 QPS; design to peak (2–5× average, stated); read/write ratio decides the architecture; storage = rows × bytes × replication × (1 + index overhead) × retention; bandwidth = QPS × payload; connections = instances × pool ≤ DB max (poolers exist for this).
• Fermi method: decompose → bound each factor within an order of magnitude → multiply (errors partially cancel). Discipline: state assumptions aloud, round aggressively, carry units (the error detector), and sanity-check against a known quantity. Classic errors: ignoring replication/index overhead, using averages not peaks, forgetting protocol overhead, bits vs bytes.
• Lens: estimation eliminates options rather than predicting (40 TB/day rules out a relational store in 90 seconds); memorize ~10 anchors and derive the rest; numbers convert opinions into verifiable constraints — the strongest seniority signal in a design conversation.

Self-test: Give the ratio of RAM to SSD to cross-continent latency. Convert 1M requests/day to average and peak QPS. Compute storage for 20M records/day at 300 bytes with 3× replication over 90 days. Name four classic estimation errors. What is estimation actually for?

Quiz Bank

FoundationalWhich latency numbers should you know, and what design conclusions follow?

The anchors: L1 cache ~1 ns; main memory ~100 ns; SSD random read ~16 μs; datacenter round trip ~0.5 ms; 1 MB sequential from SSD ~1 ms; HDD seek ~2 ms; cross-continent round trip ~150 ms. Ratios matter more than absolutes: memory is ~100× faster than SSD access latency and roughly a million times faster than a cross-continent round trip.

Design conclusions: (1) Caching is the highest-leverage optimization available — moving a read from cross-region to local memory is a five-order-of-magnitude change, which is why CDNs and cache tiers dominate scaling discussions (10.2). (2)

Sequential access beats random by ~10× on SSD and ~100× on HDD, which is why append-only logs are fast (10.8.2), why B-trees minimize random seeks ([7.3]), and why "read 1 MB sequentially" is comparable to a single random read on disk. (3)

Count round trips: at ~0.5 ms each, ten internal calls cost 5 ms (fine) and two hundred cost 100 ms (a redesign) — the distributed N+1 (10.11). (4) Cross-region latency is physics — ~150 ms is the speed of light in fiber plus routing, so it can only be avoided (replicas, CDNs, regional writes — 10.5), never tuned away. (5) Anything user-facing has a ~100–200 ms budget before it feels sluggish, which is your entire round-trip allowance — spend it deliberately.

FoundationalDo the capacity math for a system with 50M users, 30% daily active, each performing 20 actions/day with a 100:1 read/write ratio.

Daily active users: 50M × 30% = 15M DAU. Total actions/day: 15M × 20 = 300M/day. Average QPS: 300M ÷ 86,400 ≈ 3,500 QPS (use ~10⁵ s/day for mental math: 300M/10⁵ = 3,000 — close enough, and faster). Peak QPS: ×3 (typical consumer peak-to-average; state the assumption) ≈ 10,000 QPS.

Split by ratio: 100:1 reads to writes means ~99% reads → ~10,000 read QPS peak, ~100 write QPS peak. That split is the architectural headline: writes are trivially handled by a single primary database, while reads demand caching and/or replicas (10.2) — and it tells you where to spend design effort.

Storage: if each write stores ~500 bytes, 3M writes/day × 500 B = 1.5 GB/day raw; ×3 replication ×2 for indexes ≈ 9 GB/day ≈ 3.3 TB/year — comfortable for a single database with retention policy, which is a different conclusion than a naive raw-bytes estimate would suggest, and exactly why replication and index overhead must be in the formula.

Bandwidth: 10k read QPS × 20 KB response ≈ 200 MB/s ≈ 1.6 Gbps — significant but manageable, and a strong argument for caching responses at the edge. Connections: if 40 app instances each hold a pool of 20, that's 800 connections — above many default database limits, so a connection pooler is required (10.2). Five multiplications produced four architectural decisions.

AppliedExplain Fermi estimation and the discipline that makes it credible in an interview.

The method: decompose an unknown quantity into factors you can each bound within an order of magnitude, estimate each, and multiply. Independent errors partially cancel (some estimates high, some low), so the product typically lands within 3–10× of truth — which is more than sufficient to distinguish "one server" from "a fleet with a CDN."

The discipline that makes it credible: (1) State every assumption aloud and invite correction — "50M registered, 30% daily active, 3× peak; stop me if those are wrong" — because an interviewer's real question is whether you reason transparently, and it also converts their private disagreement into a shared input. (2)

Round aggressively — 86,400 becomes 10⁵, 512 becomes 500; precision at this stage is false comfort and slows you down. (3) Carry units through every step — units are the built-in error detector: if you're computing bandwidth and the units don't reduce to bytes/second, the arithmetic is wrong, and the mistake is usually a missing or extra factor. (4)

Sanity-check against a known anchor — "8.5 GB/s is a large CDN's regional throughput, so for a global photo app that's plausible; for a startup it means I've overestimated users." (5) Say what the number means — the estimate is worthless until you translate it into a decision ("so origin serving is off the table; this is a CDN and object-storage design"). Classic errors to avoid: forgetting replication and index overhead, using averages where peaks matter, ignoring protocol/metadata overhead, and mixing bits with bytes.

InterviewIn a design interview, when and how should you produce numbers?

When: immediately after clarifying requirements and before drawing the architecture — because the numbers choose the architecture, and presenting a design first and estimating afterward inverts the logic (and risks defending a design the arithmetic disqualifies). Then again at the end, to size components and validate the design against the constraints you derived.

How, in about three minutes: start from the user-facing quantities the interviewer gave (or ask for them — DAU, actions per user, payload sizes, retention); compute average QPS, apply an explicit peak multiplier, split by read/write ratio; compute storage per day/month including replication and indexes; compute bandwidth as QPS × payload; then state the three constraints that follow ("reads dominate 100:1, so caching and replicas; 40 TB/day rules out a relational store for blobs; 8 GB/s of egress means CDN").

Presentation matters as much as arithmetic: round to memorable numbers, keep the running estimates visible on the board so you and the interviewer share state, flag which assumptions the design is sensitive to ("if peak is really 10× rather than 3×, we need X"), and never present a number without its implication.

The signal being tested is not arithmetic skill — it's whether you make architectural choices from evidence rather than from familiarity, and whether you can be corrected mid-flight without your design collapsing (which is why assumptions must be explicit and separable).

StaffA team proposes storing all application events in Postgres 'for now' — 500M events/day, 1 KB each, kept 2 years. Use estimation to evaluate.

Do the arithmetic first, then interpret. Raw ingest: 500M × 1 KB = 500 GB/day. Over two years: 500 GB × 730 ≈ 365 TB raw. Add realistic overheads: ×3 replication for durability (10.5) and roughly ×1.5–2 for indexes and row overhead (Postgres row headers, TOAST, index B-trees — [7.3]) ⇒ 1.6–2.2 PB of provisioned storage. Write rate: 500M ÷ 86,400 ≈ 5,800 writes/sec average, peaking perhaps 15–20k/sec — which is at or beyond the practical ceiling for a single Postgres primary with indexes and durability enabled, before any read load.

Interpretation — three disqualifying constraints, each from one multiplication: (1) petabyte-scale in a relational store is operationally and financially wrong (the storage bill alone dwarfs the compute, and vacuum/index maintenance at that volume becomes the dominant operational burden); (2) 15k sustained writes/sec of append-only data is precisely the workload a log is built for and a B-tree store is not (10.8.2); (3) two-year retention of raw events implies queries over historical data that are analytical, not transactional — a columnar warehouse's job ([7.8]), where compression typically achieves 5–10× and query cost drops accordingly.

The counter-proposal, derived from the same numbers: stream events to a durable log for the operational window (days), land them in object storage in a columnar format (Parquet) partitioned by date for the long tail, and keep in Postgres only the small, queryable derived state the application actually serves (10.8.4's read models). Then ask the question the estimate exposes:

is two years of raw retention a real requirement, or an unexamined default? Storing aggregates beyond 90 days often satisfies the actual need at 1% of the cost — and that question, prompted by five minutes of arithmetic, is usually worth more than any of the technology choices.

Flashcards

FlashLatency anchors

L1 ~1 ns · RAM ~100 ns · SSD random ~16 μs · DC round trip ~0.5 ms · SSD 1 MB seq ~1 ms · HDD seek ~2 ms · cross-continent ~150 ms (physics).

FlashQPS conversion

86,400 s/day ≈ 10⁵ ⇒ 1M/day ≈ 12 QPS average. Always ×2–5 for peak and say the multiplier. 1 QPS ≈ 86k/day.

FlashStorage formula

rows × bytes/row × replication × (1 + index overhead) × retention. Naive estimates are 3–6× low.

FlashFermi discipline

Decompose → bound each factor → multiply. State assumptions aloud, round hard, carry units (error detector), sanity-check against an anchor, translate into a decision.

FlashClassic estimation errors

Ignoring replication/index overhead · averages instead of peaks · forgetting protocol overhead · bits vs bytes (factor of 8).

FlashWhat estimation is for

Eliminating options, not predicting. "40 TB/day" rules out a relational store in 90 seconds — a decision that otherwise costs a design cycle.

Scenario Drill

DrillEstimate, end to end, the infrastructure for a WhatsApp-like messaging service: 500M daily active users, 40 messages sent per user per day, average message 200 bytes, media in 10% of messages averaging 500 KB, messages retained 30 days on servers, plus online presence. Produce the numbers and the three architectural conclusions they force.

Message volume. 500M DAU × 40 = 20B messages/day ⇒ 20B ÷ 10⁵ ≈ 200k messages/sec average; peak ×3 ≈ 600k/sec. Each message is delivered to at least one recipient (group chats multiply this — assume ×1.5 for group fan-out) ⇒ ~900k deliveries/sec at peak. That number alone eliminates a single-cluster design and forces partitioning by conversation or user (10.6).

Text storage. 20B × 200 B = 4 TB/day raw; ×3 replication ≈ 12 TB/day; over 30 days retention ≈ 360 TB for text — large but tractable in a partitioned store, and the 30-day window is what keeps it so (worth confirming as a product decision, because "keep forever" would multiply it by 24× per two years).

Media storage. 10% of 20B = 2B media/day × 500 KB = 1 PB/day — and here the arithmetic screams: at 30 days that's 30 PB live, ×replication. Conclusion: media never touches the message store; it goes to object storage with lifecycle tiering, and messages carry references (10.8.4's claim-check pattern). This is the single most consequential number in the estimate.

Bandwidth. Text: 900k deliveries/sec × 200 B ≈ 180 MB/s — trivial. Media: if 10% of deliveries carry media, ~90k/sec × 500 KB ≈ 45 GB/s — unambiguously a CDN/edge-delivery problem, not an origin problem (10.2). Connections and presence. 500M DAU with, say, 20% concurrently connected ⇒ 100M simultaneous long-lived connections. At ~100k connections per well-tuned server (a real, if optimistic, figure requiring careful tuning — 2.7), that's ~1,000 connection-handling servers minimum — and presence updates (online/offline/typing) at even one event per connection per minute is 1.6M events/sec, which is larger than the message rate and must be handled as ephemeral, lossy, heavily-aggregated data (10.7.1's eventual/lossy class) rather than as durable messages.

The three architectural conclusions forced by these numbers: (1) Media and messages are different systems — 1 PB/day of media in object storage behind a CDN, with messages carrying pointers; conflating them makes the message store impossible. (2) Partition by conversation, with per-conversation ordering — 600k messages/sec cannot be serialized globally, and per-conversation is the smallest scope where ordering actually matters (10.3/10.6).

(3) Presence is a separate, ephemeral, lossy subsystem — its event rate exceeds the messaging rate, so treating it with messaging's durability guarantees would cost more than the product itself; it belongs in an in-memory, gossip-style or pub/sub tier with aggressive aggregation. Note what happened: five minutes of multiplication produced three decisions that would otherwise emerge painfully from a year of production, and each conclusion is defensible by pointing at one number.