10.19 — The Trade-off Ledger

Interviewers do not ask "what is a message queue". They ask "would you use a queue here, and what does it cost you". The question is always a trade-off, and the answer is graded on whether you can argue both sides and then name the thing that decides it.

This page collects the recurring pairs from the whole Part into one place, each with the case for each side and the question that settles it.

0. How to argue a trade-off

Four moves, in order. They work for every pair below.

Name what is actually being traded. Not "A is faster" but "A gives up X to get Y". If you cannot say what is given up, you have not understood the trade.

Give the case for each side honestly. A candidate who can only argue one side has a preference, not a judgement. Interviewers specifically listen for whether you can defend the option you are not choosing.

Name the deciding question, and make it about the business rather than the technology. "How stale can this be before a customer complains?" decides more than any benchmark.

Answer it with the numbers you were given, and say what would change your answer. "At a thousand orders a day, I would keep it simple. At a million I would change to the other one, and the trigger would be when the queue depth stops draining overnight."

1. Strong versus eventual consistency

What is traded. Strong consistency means every reader sees the latest write, and pays for it in latency and availability, because the system must coordinate before answering. Eventual consistency means readers may see old data for a while, and gains speed and the ability to keep serving during a partition (10.7.1).

For strong: the application is simple, because you never write code handling "what if this is stale". Anything involving money, inventory, permissions or identity is far cheaper to get right this way.

For eventual: reads are served locally and fast, the system stays up when parts of it cannot talk to each other, and it scales without a coordination bottleneck.

The deciding question: what does a stale read cost? A follower count a few seconds out of date costs nothing. A permission check a few seconds out of date is a security incident. An available balance a few seconds out of date is an overdraft.

And the answer that scores: the choice is per operation, not per system. The same shop displays a product page from an eventually consistent cache and decrements stock with a strongly consistent conditional write. Applying one model to the whole system is the mistake.

2. Availability versus consistency, and what CAP actually says

What is traded. During a network partition, a system can keep answering with possibly-wrong data, or refuse to answer until it is sure. It cannot do both. That is CAP (10.7.1).

The two corrections worth making, because both are common misreadings.

CAP only applies during a partition. When the network is healthy you can have both, and the network is healthy nearly all the time. A system is not permanently "AP" or "CP"; it has a partition behaviour.

PACELC is the more useful statement. If Partitioned, choose Availability or Consistency; Else, choose Latency or Consistency. The second half is the one that governs daily life, because even with a perfect network, agreeing with other nodes costs a round trip and you must decide whether to pay it on every request.

The deciding question: during a partition, is a wrong answer worse than no answer? For a shopping cart, keep serving — a slightly wrong cart beats a dead site. For a payment authorisation, refuse — declining is recoverable, double-spending is not.

3. Latency, throughput and bandwidth

Three different words that get used interchangeably. Latency is how long one operation takes. Throughput is how many complete per second. Bandwidth is how much data can move per second.

They are not the same and they often conflict. Batching improves throughput and worsens latency, because an item waits for the batch to fill. Adding parallel workers improves throughput and does nothing for a single request's latency. And past the saturation point, pushing for more throughput makes latency explode as queues form.

The deciding question: who is waiting? A human staring at a screen cares only about latency. A nightly pipeline cares only about throughput. Saying which one the system is for tells you which to optimise, and the answer is often different per endpoint in the same service.

The number to carry: improving the average latency is usually the wrong goal. Users experience the tail — the p99 — because a single page makes many requests and the slowest one determines how the page feels (10.10).

4. Vertical versus horizontal scaling

What is traded. Vertical means a bigger machine: no code changes, no distribution, and a hard ceiling plus a single point of failure. Horizontal means more machines: no ceiling and genuine redundancy, at the cost of every problem in this Part.

For vertical: it is astonishingly under-used. Machines with hundreds of cores and terabytes of memory are available and cheap relative to an engineering team, and a vertical move buys years while requiring zero design work. A database that fits on one machine avoids sharding, distributed transactions and cross-node consistency entirely.

For horizontal: you eventually run out of machine, and one machine is one failure away from an outage regardless of how big it is.

The deciding question: are you near the ceiling, or do you need redundancy? Those are two different reasons and they call for different things. If it is redundancy, two moderate machines beat one enormous one. If it is capacity, the biggest available machine is usually cheaper than the engineering to distribute.

The mature answer: scale vertically until it hurts, horizontally for the tier that genuinely needs it, and keep the stateful parts vertical for as long as you can — because stateless services scale horizontally almost for free and databases do not (10.2).

5. Push versus pull

What is traded. Push means the producer sends to consumers as things happen. Pull means consumers ask when they are ready.

For push: low latency, and no wasted requests when nothing has changed.

For pull: the consumer controls its own rate, which is backpressure for free — a slow consumer simply asks less often, rather than being overwhelmed. Recovery is also simpler, because a consumer that was down catches up by asking for what it missed.

The deciding question: can the consumer keep up with the producer's worst burst? If yes, push. If no, pull, or push into a queue the consumer pulls from — which is what nearly every real system does, and which is why message brokers exist (10.8.1).

The classic instance is a social feed. Pushing a celebrity's post to fifty million followers at publish time is an enormous write burst; pulling means computing every reader's feed on every visit. Real systems do both — push for ordinary users, pull for accounts with huge followings — and being able to say why the hybrid exists is the point (11.8).

6. Batch versus stream

What is traded. Batch processes a large chunk on a schedule. Stream processes each event as it arrives.

For batch: simpler to write, simpler to restart, much more efficient per item, and easy to reason about because the input does not move while you work. Re-running yesterday is trivial.

For stream: results are current, and load is spread evenly instead of arriving as a nightly spike.

The deciding question: how fresh must the result be, and what is the cost of being a day behind? Fraud detection needs seconds. A monthly invoice does not. Most dashboards that people insist must be real-time turn out to be fine at five minutes, which is a much cheaper system.

The honest observation: streaming is meaningfully harder — late events, out-of-order events, exactly-once semantics, and state that must survive restarts. Choosing it when hourly batches would do is a common and expensive mistake.

7. Synchronous versus asynchronous

What is traded. Synchronous means the caller waits for the result and finds out immediately whether it worked. Asynchronous means the caller is told "accepted" and the work happens later.

For synchronous: simple to reason about, immediate errors, and a natural place to put a transaction.

For asynchronous: the caller's latency no longer depends on the slowest downstream system, a downstream outage becomes a delay rather than a failure, and bursts are absorbed by the queue.

The deciding question: does the user need the answer to continue? Charging a card at checkout — yes, synchronous. Sending the confirmation email — no, and doing it synchronously means the email provider's bad afternoon becomes your failed checkouts.

The cost that gets underestimated: asynchronous work needs status somewhere the user can see it, a retry policy, a dead-letter destination for work that never succeeds, and monitoring for backlog. "Fire and forget" without those is just "forget".

8. Normalised versus denormalised data

What is traded. Normalised stores each fact once and joins on read. Denormalised duplicates facts so reads need no join.

For normalised: one place to update, so inconsistency is impossible by construction. Storage is smaller. New query shapes are supported without a migration.

For denormalised: reads are a single lookup, which matters enormously when the join would cross a network or a partition boundary.

The deciding question: is this data read far more than it is written, and are the reads a known shape? Denormalise for known, high-volume read patterns. Keep the normalised copy as the source of truth and treat the denormalised one as derived, so it can always be rebuilt.

The rule that prevents the usual disaster: duplicated data must have exactly one writer and a rebuild path. Two systems independently maintaining copies of the same fact will diverge, and without a rebuild path you cannot fix it when they do.

9. REST, RPC and GraphQL

What is traded. REST models nouns and uses HTTP's own semantics. RPC models verbs and is usually more compact and faster. GraphQL lets the client specify exactly which fields it wants.

For REST: everything understands it. Caching works out of the box because HTTP caching is built around it. Debugging is a browser and a URL.

For RPC: lower overhead and a strongly typed contract, which suits service-to-service calls in a busy internal network.

For GraphQL: one request instead of five for a screen that needs data from several places, and clients evolve without server changes — genuinely valuable when many different clients need different subsets.

The deciding question: who is the client, and how variable are their needs? Public API with many unknown consumers — REST, because familiarity and caching win. Internal service-to-service — RPC. A rich application with many screens each needing a different slice — GraphQL.

The cost of GraphQL to name: HTTP caching largely stops working because every query is a different POST body, rate limiting becomes hard because one query can be arbitrarily expensive, and a badly shaped query can cause a storm of database calls behind the scenes. It moves complexity from the client to the server rather than removing it.

10. Stateful versus stateless services

What is traded. A stateless service keeps nothing between requests, so any instance can serve anything. A stateful one holds data in memory, which is fast and pins requests to instances.

For stateless: scaling is adding instances, deploys are invisible, failures cost nothing, and load balancing is trivial.

For stateful: no round trip to fetch state, which for a game session or a live document is the difference between usable and not.

The deciding question: can the state be reconstructed, and how expensive is fetching it? If it can be rebuilt from durable storage, hold it as a cache and stay logically stateless. If it genuinely cannot — a live collaborative document, an active game — accept statefulness, partition by entity so one owner exists per key, and design explicitly for what happens when that instance dies (9.5.4).

11. The availability arithmetic

This is not a trade-off but it settles several of them, and it is worth being able to produce.

A dependency in series multiplies. If your service needs a database at 99.9 percent, a cache at 99.9 percent and a payment provider at 99.9 percent, and any one failing means you fail, your ceiling is 0.999³ = 99.7 percent — about a day of downtime a year, from three components that each looked excellent.

Redundancy in parallel adds nines. Two independent components each at 99 percent, where either can serve, fail together only 0.01 × 0.01 = 0.01 percent of the time, giving 99.99 percent. The word doing the work is independent — two instances in the same rack sharing a power supply are not, and every real correlated failure lives in that word.

Three consequences.

Every synchronous dependency lowers your ceiling. This is the strongest technical argument for making non-essential calls asynchronous: an email provider you call synchronously is now part of your availability calculation, and an email provider you queue for is not.

A single point of failure caps everything. One load balancer, one primary database, one shared authentication service — the whole system inherits that component's number no matter what else you do (10.9).

Adding nines gets expensive very fast. 99.9 percent is about 8.8 hours of downtime a year and is achievable with good practice. 99.99 percent is 53 minutes and needs automated failover and real redundancy. 99.999 percent is five minutes a year, which is less time than a human takes to read an alert — so it can only be reached by systems that recover with nobody involved. Ask what the business actually needs before promising a number, because each additional nine roughly multiplies the cost.

12. What the interviewer will push on

"Strong or eventual consistency for this?" The wrong answer is one word. The right answer asks what a stale read costs, then splits the system: eventual for the product page, strong for the stock decrement. Applying one model everywhere is the mistake they are testing for.

"Explain CAP." Two corrections earn the points. CAP only binds during a partition, so a healthy system can have both. And PACELC is more useful, because the else branch — latency versus consistency when the network is fine — is what you actually pay for every day.

"Would you use a queue here?" They want the deciding question, which is whether the user needs the answer to continue. Then the cost that gets forgotten: asynchronous work needs visible status, a retry policy, a dead-letter destination and backlog monitoring, or it is not asynchronous, it is lost.

"Push or pull?" The deciding question is whether the consumer can survive the producer's worst burst. Then note that nearly every real system does both, via a queue — push into it, pull out of it — and that the celebrity-follower case is why hybrid feed designs exist.

"How available is your system?" They want the multiplication. Three synchronous dependencies at 99.9 percent gives 99.7 percent, and every synchronous call you add lowers the ceiling. This is the number that turns "should this be async?" from a style question into an arithmetic one.

"You said REST. Why not GraphQL?" Be able to argue the side you did not pick. GraphQL wins when many different clients need different slices; it costs you HTTP caching, straightforward rate limiting, and predictable database load. Naming those three costs unprompted is what shows you chose rather than defaulted.

The thing to volunteer that nobody asks for: say what would change your mind. "I would keep this synchronous now; I would move it behind a queue when the downstream p99 exceeds our latency budget, or when their availability starts capping ours." A trade-off with a stated trigger is a decision. Without one it is a preference.

Next: Part 11 puts every trade-off on this page to work, one complete system at a time, starting with 11.0.

Recall

• Argue a trade-off in four moves: name what is given up · make both cases honestly · state the deciding question in business terms · answer it with the given numbers and say what would change your mind.
• Strong vs eventual: deciding question is what does a stale read cost. Choose per operation, not per system.
• CAP binds only during a partition. PACELC is more useful: else, latency or consistency — and that is what you pay daily.
• Latency ≠ throughput ≠ bandwidth. Batching helps throughput and hurts latency. Deciding question: who is waiting? Optimise the tail, not the average.
• Vertical scaling is under-used — it buys years for zero design work. Deciding question: ceiling, or redundancy? Those need different answers.
• Push vs pull: deciding question is whether the consumer survives the producer's worst burst. Real systems do both, through a queue.
• Batch vs stream: streaming is genuinely harder — late events, ordering, state across restarts. Most "real-time" requirements are fine at five minutes.
• Sync vs async: does the user need the answer to continue? Async without visible status, retries, a dead-letter destination and backlog monitoring is not async, it is lost.
• Denormalise for known high-volume reads, with exactly one writer and a rebuild path.
• GraphQL costs HTTP caching, easy rate limiting, and predictable database load.
• Availability multiplies in series (three at 99.9% → 99.7%) and adds nines in parallel only if the failures are independent. Every synchronous dependency lowers your ceiling.

Self-test: What are the four moves for arguing a trade-off? What does CAP not say? Give the deciding question for push versus pull, and for sync versus async. Compute the availability of three serial dependencies at 99.9 percent. Name three costs of GraphQL. What must always accompany denormalised data?

Quiz Bank

FoundationalExplain CAP correctly, including the two things people usually get wrong, and then explain why PACELC is more useful.

What CAP says. During a network partition — when nodes cannot communicate — a distributed system must choose between remaining available (answering with data that may be stale or accepting writes that may conflict) and remaining consistent (refusing to answer until it can be sure). It cannot have both, because the two halves of a partition cannot agree.

The first thing people get wrong: treating it as a permanent property. Systems get labelled "AP" or "CP" as though that describes their normal operation. It does not. When the network is healthy — which is almost always — a system can be both consistent and available. CAP describes a partition behaviour, meaning what happens in an unusual situation, not a design category.

The second thing people get wrong: treating the choice as system-wide. It is per operation. The same database can serve a product listing from any replica, accepting staleness, and route a stock decrement through a strongly consistent path. Answering "we chose AP" for an entire system usually means the question was not thought about carefully.

Why PACELC is the more useful formulation. It reads: if Partitioned, choose Availability or Consistency; Else, choose Latency or Consistency.

The first half is CAP. The second half is what governs your system every single day. Even with a perfect network, making a read strongly consistent means coordinating with other nodes, and coordination costs a network round trip — often several. So the everyday question is not "what happens during a partition", which may occur a few times a year, but "am I willing to pay a round trip on this read to be certain it is current?", which you answer on every request.

That reframing is why PACELC is worth naming in an interview. It moves the discussion from a rare failure mode to the constant, unavoidable cost of consistency, and that cost is what actually shapes the design.

The answer that demonstrates you can apply it: for a shopping site, product pages take the latency win and accept staleness; the checkout's stock decrement pays the coordination cost; and during a partition the catalogue stays available while payments refuse rather than risk a double-spend. Three different answers in one system, each justified by what a wrong answer would cost.

AppliedAn interviewer asks whether to make order confirmation emails synchronous or asynchronous. Give the full argument.

Name the trade first. Synchronous means the checkout request waits for the email provider to accept the message, and the user learns immediately if it failed. Asynchronous means checkout completes, the email is queued, and something else sends it shortly afterwards.

The case for synchronous, argued honestly rather than dismissed. Errors surface immediately, so a wrong address or a provider outage is visible at the moment of purchase. There is no queue to operate, no worker to deploy, no backlog to monitor. And the code is one call in a function that already exists. For a small system sending a few hundred emails a day, this is genuinely the right answer, and saying so is better than reflexively reaching for a queue.

The case for asynchronous, and it becomes decisive quickly.

Your availability stops depending on theirs. This is the argument with a number attached. If checkout is 99.95 percent available and the email provider is 99.9 percent, then calling them synchronously makes checkout 0.9995 × 0.999 = 99.85 percent — you have taken on their downtime as your own. Queued, their outage is a delayed email and checkout stays at its own number.

Latency stops depending on theirs. An email API that normally takes 100 ms and occasionally takes 3 seconds adds that variance to every checkout, and it lands in your p99, which is what users feel.

Retries become possible. A synchronous send that fails has one chance — the user is waiting and you cannot retry for thirty seconds. Queued, it retries with backoff and eventually succeeds.

Bursts are absorbed. A flash sale produces a thousand orders a minute, and the provider rate-limits you. Queued, that is a slightly delayed inbox. Synchronous, it is failed checkouts.

The deciding question: does the user need the email to continue? They do not. They need the order to exist and the page to say so. The email is a notification about something that already happened, which is the textbook shape for asynchronous work.

The costs of async, stated so the answer is not one-sided. You need somewhere to put the work durably, a worker to consume it, a retry policy with backoff, a dead-letter destination for messages that will never succeed, monitoring on queue depth and age, and a way for support to answer "did the customer get their email?" — which means status recorded per message, not just an event fired. That is real work, and "fire and forget" without it is not asynchronous, it is lost.

And the detail that shows care: the email must be queued as part of the same transaction that creates the order, or you can create an order and lose the email, or send an email for an order that was rolled back. Writing the intent to a table in the order's transaction and having a worker pick it up is the outbox pattern (10.8.4), and it is the correct answer to the follow-up they are about to ask.

InterviewCompute your system's availability ceiling and explain what it implies for your design.

The arithmetic first. If a request needs several components and any of them failing means the request fails, the availabilities multiply.

A service depending synchronously on a database at 99.9 percent, a cache at 99.9 percent and a payment provider at 99.9 percent has a ceiling of 0.999 × 0.999 × 0.999 = 99.70 percent. That is roughly 26 hours of downtime a year, produced by three components that each individually look excellent.

Add a fourth dependency and it becomes 99.6 percent, about 35 hours. Each addition costs, and the cost compounds silently because nobody ever proposes "let us lower our availability" — they propose "let us call the recommendations service from checkout".

Redundancy works the other way. Two components in parallel, where either can serve the request, fail together only when both fail. Two at 99 percent give 1 - (0.01 × 0.01) = 99.99 percent.

The critical word is independent, and this is where the arithmetic gets abused. Two instances sharing a rack, a power supply, an availability zone, a deployment pipeline or a configuration store are not independent, and the correlated failure dominates the calculation completely. Real availability is set by the correlated failures, which is why the honest version of this analysis lists what the two copies share rather than just counting them.

What it implies for design, and this is the part that matters.

Every synchronous dependency lowers the ceiling, so removing one raises it. This turns "should this be asynchronous?" from a matter of taste into arithmetic. Queueing the email provider removes it from the calculation entirely. Making the recommendations call optional with a fallback removes it too — a dependency you can serve without is not on the multiplication.

A single point of failure caps everything. One primary database, one load balancer, one shared authentication service — whatever else you build, the system inherits that component's number.

Ask what the business needs before promising nines. 99.9 percent is about 8.8 hours a year and is reachable with good engineering practice. 99.99 percent is 53 minutes and requires automated failover, because a human cannot be paged, wake up and fix something inside that budget more than once. 99.999 percent is five minutes a year, which is shorter than most people's response time, so it can only be achieved by systems that recover with nobody involved at all. Each nine roughly multiplies the cost, and most products do not need the ones they claim to.

The sentence that makes this useful in a design review: for every synchronous call, ask whether the system can serve a degraded but useful response without it. If yes, it belongs behind a fallback or a queue and should leave the multiplication. If no, it is a genuine dependency and its availability is now part of your promise, so it needs to be as good as the promise you are making.

StaffA team wants to rebuild a nightly batch pipeline as a real-time streaming system because 'the dashboard should be live'. Work the trade-off.

Establish what is actually being asked for, because "live" is never the requirement. The requirement is a decision somebody makes from the dashboard, and the useful question is: what decision, and how much would it improve if the data were fresher? If a merchandiser adjusts a promotion once a morning, hourly data is indistinguishable from live. If an operations team responds to a fraud spike, minutes matter and a night is unusable. The answer is usually somewhere in between, and it is almost never "seconds".

The case for streaming, argued fairly. Results are current, so the dashboard reflects the last few minutes. Load is spread evenly instead of arriving as a nightly spike that requires capacity sitting idle all day. And a late-arriving day of data does not mean waiting until tomorrow.

The case for batch, which is stronger than its reputation. It is dramatically simpler: the input does not move while you process it, so there are no late events, no out-of-order events, and no partial state. It is far more efficient per item, because reading a million rows at once beats a million individual reads. It restarts cleanly — re-running yesterday is one command. And when the logic changes, reprocessing history is trivial, which is not a small thing, because analytics logic changes constantly.

The costs of streaming that teams underestimate, and naming these is the substance of the answer.

Late and out-of-order events. An event generated at 09:59 may arrive at 10:04. Do you amend the 09:00 hour, or count it in the 10:00 one? Every streaming system has to answer this — windows, watermarks, a lateness allowance — and the answer is business logic that batch never had to have, because batch simply waited.

State across restarts. A running count is state. When the process restarts, that state must be recovered exactly or numbers jump. This is a genuinely hard problem with real infrastructure behind it.

Duplicate delivery. At-least-once means an event may be processed twice, which for a counter is visible and wrong. Deduplication or idempotent aggregation is required (10.4).

Reprocessing is hard. When the logic changes — and it will — batch just re-runs. Streaming needs a replay strategy, and if the source has a retention window shorter than the history you need, it cannot be done at all.

Debugging is harder. A wrong number in a batch job is reproducible from fixed input. A wrong number in a stream may depend on an ordering that will never occur again.

The recommendation I would actually make. Ask what freshness the decision needs, and if the answer is anything above about five minutes — which it usually is — run the existing batch job more often rather than rebuilding it. Hourly, or every fifteen minutes, on the same code. This delivers most of the perceived benefit for a few days of work rather than a quarter, keeps every property that makes batch easy, and gives you real evidence about whether anybody's behaviour actually changes with fresher data.

Then, if a specific metric genuinely needs seconds — a fraud signal, a live operational count — stream that one metric alongside the batch pipeline, rather than converting everything. You end up with a small streaming path where the freshness pays for the complexity, and a batch path for the ninety percent where it does not.

The framing to leave them with: streaming is not a better version of batch, it is a different set of trade-offs with a genuinely higher operational cost. Buying it for the metrics that need it is good engineering; buying it for the whole pipeline because "live" sounds better is paying a permanent tax for a benefit nobody will use.

Flashcards

FlashArguing a trade-off

Name what is given up · make both cases · state the deciding question in business terms · answer with the given numbers and say what would change your mind.

FlashCAP and PACELC

CAP binds only during a partition, and the choice is per operation. PACELC: else, latency or consistency — and that is the cost you pay every day, not a few times a year.

FlashAvailability multiplies

Three synchronous dependencies at 99.9% → 99.7%, about 26 hours a year. Parallel redundancy adds nines only if failures are independent. Every sync dependency lowers your ceiling.

FlashDeciding questions

Stale read cost (consistency) · who is waiting (latency vs throughput) · ceiling or redundancy (scaling) · can the consumer survive the burst (push vs pull) · does the user need the answer to continue (sync vs async).

FlashAsync's hidden bill

Visible status, retry policy, dead-letter destination, backlog monitoring, and enqueue in the same transaction as the write (outbox). Without those it is not async, it is lost.

FlashGraphQL's three costs

HTTP caching largely stops working · rate limiting is hard because one query can be arbitrarily expensive · a bad query shape causes a storm of database calls. It moves complexity, it does not remove it.