9.5.1 — Threads, Races and the Critical Section

A customer support agent clicks Refund on order A-4471. The page hangs for a second, so they click it again. The refund is for $80. The customer receives $160.

Here is the handler that did it. Read it and try to spot the problem before the explanation:

async function refund(orderId: OrderId, amount: Money): Promise<void> {
  const order = await orders.load(orderId);          // (1)
  if (order.refundedAmount.gte(order.total)) {       // (2)
    throw new AlreadyFullyRefunded(orderId);
  }
  await payments.refund(order.paymentId, amount);    // (3)
  order.refundedAmount = order.refundedAmount.add(amount);
  await orders.save(order);                          // (4)
}

Line (1) reads the order from the database. Line (2) checks whether it has already been refunded in full, and refuses if it has. Line (3) actually sends the money back. Line (4) records what we did.

Nothing on any of those four lines is wrong. If you run this function a thousand times one after another, it behaves perfectly every single time. The bug only exists because of the word after. The agent's two clicks did not run one after another. They overlapped.

The first click reached line (1) and read refundedAmount: $0. Before it got to line (4) and wrote anything down, the second click also reached line (1), and it also read refundedAmount: $0. Both requests now believe nothing has been refunded. Both sail past the check on line (2). Both call the payment provider on line (3). Two refunds of $80 leave the building, and the customer keeps the extra.

This page is about that class of bug: what it is, why it happens, why it is so much harder to find than a normal bug, and what vocabulary you need in order to talk about it precisely. The pages after it are about the tools that fix it.

1. Concurrency and parallelism are not the same word

These two words get used interchangeably in conversation, and interviewers notice when you use them precisely.

Concurrency means your program is dealing with several things at once. Several pieces of work are in progress, and the program keeps track of all of them. It says nothing at all about how many of them are physically executing at any given instant.

Parallelism means several pieces of work are physically executing at the same instant, on different CPU cores. It is a hardware fact, not a program-structure fact.

The relationship between them is one-directional. Parallelism requires concurrency, because you cannot run two things at once unless you have two things in flight. But concurrency does not require parallelism at all. A single CPU core can be concurrent by switching rapidly between jobs, giving each one a slice of time.

Figure 1 — Concurrency is a structure, parallelism is a hardware fact. The top strip is one core switching between two jobs; the bottom is two cores running two jobs at once. Both are concurrent. Only the bottom is parallel.

The reason this distinction matters for design, rather than just for vocabulary, is that the two things solve different problems.

You reach for concurrency when your program spends most of its time waiting. A web server waiting on a database, a file download waiting on the network, a build tool waiting on disk. Here the CPU is idle most of the time, and the win comes from having something useful to do during the wait. One core is plenty. This is what people mean by I/O-bound work.

You reach for parallelism when your program spends most of its time computing. Resizing ten thousand images, hashing passwords, running a simulation. Here the CPU is the thing you have run out of, and the only cure is more CPUs actually working. This is CPU-bound work.

Getting this backwards is one of the most common and most expensive design mistakes in the field. Adding more threads to an I/O-bound service that is waiting on a database does not make it faster; it just means more threads waiting on the same database. Adding an async queue to a CPU-bound image resizer does not make it faster either; the CPU is still the bottleneck, and now you have a queue growing in front of it.

2. Processes and threads: who shares what

Part 2 built these two from the operating system's side (2.2 and 2.3). What matters for design is one single question: what memory is shared, and what is not.

A process is a running program with its own private memory. Two processes cannot see each other's variables at all. If they want to communicate, they have to send messages: over a socket, over a pipe, through a file, through a database.

A thread is a line of execution inside a process. All threads in one process share the same heap, which means they share every object your program has allocated. Each thread gets its own private call stack, so local variables are private, but anything reachable from a shared object is visible to all of them.

Figure 2 — The only difference that matters for design. Processes share nothing and must pass messages. Threads share the whole heap, which is what makes them fast and what makes them dangerous.

Threads are cheaper than processes on every axis. Creating one costs microseconds instead of milliseconds. Switching between two threads of the same process is cheaper than switching between processes, because the memory mapping does not have to change. And passing a million-item list between threads costs nothing, because you pass a reference rather than a copy.

That last advantage is also the entire problem. The million-item list that both threads can now reach is a million items that both threads can now modify at unpredictable moments.

The design consequence, stated once and used for the rest of the chapter: every concurrency bug in this chapter needs shared mutable state to exist. Remove either word — make the state unshared, or make it immutable — and the bug becomes impossible rather than unlikely. That is the deepest fix available, and it is why the last section of this page ranks it first.

3. The life of a thread

A thread is not simply running or stopped. It moves through a small set of states, and knowing them lets you read a thread dump or a profiler output and say something useful about it.

Figure 3 — Where a thread can be. Only RUNNING consumes a CPU. A thread that is BLOCKED or WAITING costs you memory for its stack and nothing else, which is why thousands of idle threads are wasteful but not fatal, and why thousands of runnable threads are a disaster.

NEW is a thread object that exists but has not been started.

RUNNABLE means the thread is ready to run and is sitting in the scheduler's queue waiting for a turn. It is not making progress, but nothing is stopping it except the shortage of cores.

RUNNING means it is on a core right now. The number of threads in this state can never exceed the number of cores you have.

BLOCKED means it tried to enter a critical section that another thread is holding, and it cannot proceed until that other thread lets go. We will spend the whole of 9.5.2 on what puts a thread here.

WAITING means it is voluntarily paused: waiting for a network reply, sleeping for a timer, or waiting to be woken by another thread that will announce a condition has become true.

TERMINATED means the function it was running has returned, or has thrown an exception that nobody caught.

The scheduler (2.3) decides which RUNNABLE thread becomes RUNNING, and it can take that decision away again at any moment. The moment when a running thread is paused and another takes its place is called a context switch, and its cost is worth carrying in your head. Saving and restoring the registers takes maybe a microsecond. The real cost is that the new thread arrives to find the CPU cache full of the old thread's data (1.6), so its first few thousand memory accesses are slow. The practical number to remember is a few microseconds of lost throughput per switch.

That number explains a rule you will meet everywhere in this chapter: more threads is not more speed. Once you have more runnable threads than cores, extra threads add switching cost without adding any capacity. This is why thread pools exist, and why their size is usually near the number of cores rather than near the number of tasks.

4. Why balance += amount is not one step

Now to the heart of it. Here is the smallest program that goes wrong. Two threads each add 1 to a counter, one thousand times each.

let balance = 0;

function addOneThousandTimes(): void {
  for (let i = 0; i < 1000; i++) {
    balance += 1;                    // looks like a single step. It is three.
  }
}

Run addOneThousandTimes on two threads that share balance, and the final value is not 2000. It is some number between 1000 and 2000, and it is a different number every time you run it.

The reason is that a CPU cannot add to memory directly. balance += 1 compiles into three separate machine steps (1.5):

1. Read the value of balance out of memory into a register.
2. Add one to the register.
3. Write the register back into balance.

The scheduler is allowed to pause a thread between any two of those steps. It does not know or care that the three of them were meant to belong together. So this ordering is perfectly legal:

Figure 4 — One increment vanishes. Nothing here is a bug in any single line. The bug is entirely in the ordering, and the ordering is chosen by the scheduler, not by you.

Some vocabulary, all of it worth using precisely because interviewers listen for it.

A race condition is when the correctness of your program depends on the order in which concurrent operations happen to run. The name is apt: two pieces of code are racing, and your program is only right if the right one wins.

A critical section is a region of code that touches shared state and must not be interleaved with another thread running the same region. Lines 1 to 3 above are the critical section. Notice that a critical section is defined by the data it touches, not by the lines themselves. Two different functions that both modify balance are in the same critical section even though they live in different files.

An operation is atomic if no other thread can ever observe it half-finished. Atomic literally means indivisible. The whole of concurrency control is about taking sequences that are not atomic and making them behave as if they were.

The specific failure in Figure 4 has its own name, lost update: two readers both read the same value, both compute a new value from it, and the second write silently erases the first. You will meet this exact term again in 10.4 when the two threads become two servers, because the shape does not change when the distance grows.

5. The second hazard: a write that nobody else can see

The interleaving problem is the famous one. There is a second problem that is less famous and even harder to debug, and it is the reason languages have keywords like volatile and libraries have things called memory barriers.

Modern CPUs do not write straight to main memory. Each core has its own cache (1.6). When a thread on core 0 writes to a variable, the new value may sit in core 0's cache for a while before it reaches memory. Meanwhile a thread on core 1 reading that variable may get an older copy out of core 1's cache. Compilers make this worse, in the sense of more surprising, because an optimising compiler is allowed to keep a variable in a register and reorder instructions, as long as the result is unchanged for a single thread running alone.

The consequence is a loop like this, which looks obviously terminating and is not:

let shutdownRequested = false;   // written by the signal handler thread

function workerLoop(): void {
  while (!shutdownRequested) {   // may never see the write from another thread
    processOneItem();
  }
}

In a language with real threads and no memory-model annotation on that flag, the compiler is entitled to notice that workerLoop never writes shutdownRequested, hoist the read out of the loop, and turn it into while (true). The other thread sets the flag, and the worker keeps going forever. The bug does not reproduce in a debug build, because the optimisation is off.

This is called a visibility problem, and it is separate from the interleaving problem in a way worth stating clearly:

Hazard	What goes wrong	Cured by
Interleaving	Steps of two operations mix	Mutual exclusion
Visibility	A write is never seen	A memory barrier

Every real locking primitive gives you both guarantees at once, which is why you rarely have to think about visibility separately. Acquiring a lock forces your core to see everyone else's committed writes; releasing one forces your writes out where others can see them. That two-for-one is the main reason "just use a mutex" is such durable advice.

In JavaScript and Node this hazard mostly cannot reach you, because plain JavaScript objects are never shared between threads. It returns the moment you use SharedArrayBuffer, which is exactly why the Atomics API exists, and that is the subject of 9.5.6.

6. Check-then-act, the shape behind the double refund

Go back to the refund bug at the top of the page. There is no += in it. There is no shared object in memory at all: the state lives in a database, and each request loaded its own copy. And yet it is the same bug.

The shape it shares with the counter is called check-then-act, sometimes written TOCTOU for time of check to time of use. You look at the world, you make a decision based on what you saw, and you act on that decision. The bug is the gap in the middle, during which the world can change and make your decision wrong.

Once you know the shape, you see it everywhere:

if (!fileExists(path)) createFile(path);           // two callers both see "no", both create
if (seatsLeft > 0) sellSeat();                      // two buyers both see 1, both sell
if (!cache.has(key)) cache.set(key, expensive());   // both miss, both compute, one wasted
if (user.role !== "banned") allowPost();            // ban lands between check and post
if (idempotencyKeyUnused(k)) processPayment();      // two retries both find it unused

Every one of those is two statements that need to be one. And that gives the general cure, which is worth memorising as a sentence because it applies from a single CPU register all the way out to a cluster of database replicas:

Do not check and then act. Make the check and the act into one indivisible operation, and let it fail when it loses.

In practice that sentence turns into a small number of concrete techniques, and it is genuinely useful to see them side by side now, because the rest of the chapter is mostly elaboration on them:

(a) One statement instead of a read and a write. The WHERE clause is the check, the UPDATE is the act, and the database performs both as one indivisible operation. Zero rows affected means you lost the race, and that is your "sold out" answer:

UPDATE seats SET remaining = remaining - 1
 WHERE event_id = ? AND remaining > 0;

(b) A constraint, which turns "check whether it exists" into "try it and catch the failure". The second refund attempt now fails at insert time and cannot get past the database at all:

CREATE UNIQUE INDEX ON refunds (order_id, idempotency_key);

(c) A lock, which makes your own check-then-act atomic by keeping everyone else out of it:

await locks.withKey(orderId, async () => {
  // load, check, refund, save — nobody else is inside this block for this order
});

Notice something about the ordering of that list. Option (a) and option (b) fix the problem where the state actually lives, and they keep working no matter how many servers you deploy. Option (c) fixes it in your process, which means it stops working the moment a second copy of your service starts up. That ranking — push the guarantee down to the state, and use a lock only when you cannot — is the single most valuable instinct in this whole chapter.

7. The four ways to make a race impossible

Before the tools, here is the map. When you find shared mutable state, you have four moves available, and they are listed in order of how much you should prefer them.

One: do not share it. Give each thread its own copy. A per-request object touched by exactly one request cannot race with anything. Thread-local storage, a fresh object per task, and Node's one-process-one-heap model are all this move. The bug becomes unrepresentable rather than prevented, which is always the strongest form of a fix.

Two: do not mutate it. If a value never changes after it is created, any number of threads can read it simultaneously with no coordination at all. Instead of editing, you build a new value and swap the reference. This is why functional programming and concurrency get talked about in the same breath (3.5), and why readonly and as const in TypeScript are quietly concurrency tools.

Three: let one owner touch it. Keep the state mutable but give it exactly one thread that is allowed to reach it, and have everybody else send that thread a message. Nobody races because nobody else can touch the data. This is the actor idea, built in 9.5.4, and it is the model your Node process already runs on.

Four: guard it. Leave the state shared and mutable, and put a lock in front of it so only one thread is inside the critical section at a time. This is the most flexible option and the most dangerous one, because now the correctness of your program depends on every single piece of code remembering to take the lock. This is what 9.5.2 is about, and 9.5.3 is about what happens when it goes wrong.

Most engineers reach for option four first because it is the one that gets taught first. Reaching for one, two and three first is what separates a design that stays correct as the team grows from one that needs a concurrency expert on call.

8. Why concurrency bugs are so much worse than normal bugs

It is worth being explicit about this, because it explains why the industry invests so heavily in avoiding these bugs rather than fixing them.

They are not reproducible. The bad interleaving needs a specific timing that might occur once in ten thousand runs. Your test suite runs it a hundred times and passes.

They hide from the debugger. Attaching a debugger, adding a log line, or running an instrumented build all change the timing, and usually change it in the direction that makes the bug stop happening. There is a name for a bug that disappears when you look at it: a Heisenbug.

They appear only under load. Which means production, on a Friday, during your biggest traffic day, and not on any developer's laptop.

The damage is silent. A crash tells you something is wrong. A lost update tells you nothing. The refund at the top of this page was discovered by the finance team during a monthly reconciliation, five weeks after it shipped.

They get worse with better hardware. More cores means more genuine parallelism means more chances for the bad interleaving. Code that has been fine for three years starts failing after a server upgrade.

That list is the argument for the ordering in section 7. You cannot test your way to concurrency correctness with any confidence. You have to design so that the bad states cannot be expressed.

9. What an interviewer is actually checking

Concurrency questions in a design interview are rarely about whether you can recite the definition of a semaphore. They are checking four specific things.

Do you spot the contested resource without being asked? In any design with a booking, a wallet, an inventory count, or a limited pool, there is exactly one thing that two users can grab at once. Naming it unprompted — "the last seat is the contested resource here, so the claim has to be atomic" — is the single highest-value sentence you can say in an LLD interview (9.7.1 grades this explicitly).

Do you know where the guarantee belongs? Candidates who say "I would put a mutex around it" get a follow-up: "you now have three instances of this service behind a load balancer." Candidates who say "I would make the decrement a conditional update in the database" do not get that follow-up, because they already answered it.

Can you distinguish concurrency from rate? Twenty requests in flight at once and twenty requests per second are different constraints with different tools. Confusing them is common.

Do you classify work before choosing a tool? Is this job waiting or computing? Must this action definitely happen, or is losing it acceptable? Does this need to be ordered per user, or globally? Every good concurrency design starts with those answers and ends with the mechanism, and the weak ones do it in the reverse order.

Next: 9.5.2 takes the fourth option from section 7 — guard it — and builds the entire toolbox: mutexes, semaphores, condition variables, read-write locks and the compare-and-swap instruction that all of them are secretly built from.

Recall

• Concurrency is dealing with several things at once (a program structure). Parallelism is executing several things at the same instant (a hardware fact). Concurrency suits waiting work; parallelism suits computing work.
• Threads share the heap and have private stacks; processes share nothing and must pass messages. The shared heap is the speed and the danger, both.
• A race condition is correctness that depends on timing. A critical section is code that touches shared state and must not interleave. Atomic means never observable half-done.
• balance += 1 is read, add, write — three steps, interruptible between any two. Two threads interleaving them produce a lost update.
• Two separate hazards: interleaving (cured by mutual exclusion) and visibility (cured by a memory barrier). Real locks give you both at once.
• Check-then-act (TOCTOU) is the shape behind almost every real-world race, including ones with no shared memory at all. The cure is to make the check and the act one indivisible operation that fails when it loses.
• Four moves, best first: don't share · don't mutate · one owner · guard with a lock. Push the guarantee down to where the state lives; a process-local lock stops working at two replicas.

Self-test: Give an example of concurrency without parallelism. Which three machine steps hide inside +=? Name the two hazards and the cure for each. Rewrite if (seats > 0) sell() so it cannot oversell. Why is a lock the last of the four moves rather than the first?

Quiz Bank

FoundationalExplain concurrency versus parallelism, and give a case where adding threads makes a system slower rather than faster.

Concurrency is a property of how a program is structured: several units of work are in progress and the program is managing all of them. Parallelism is a property of the machine at an instant: several units of work are executing simultaneously on different cores. Parallelism needs concurrency; concurrency does not need parallelism.

A single-core machine running a web server with a thousand open connections is concurrent and not parallel at all. It makes progress because almost every connection is waiting on a database or a network reply, so there is nearly always someone ready to run whenever the current job pauses.

A case where threads make things slower: a service that is already CPU-saturated on eight cores, given a pool of two hundred threads. All two hundred are runnable, so the scheduler now context-switches constantly. Each switch costs register save and restore, and, much more expensively, arrives at a CPU cache filled with the previous thread's data (1.6), so every thread runs slower than it would have. Throughput drops and latency variance explodes, because a job's completion time now depends on how many times it got descheduled.

The general rule this produces: a thread only helps when it will spend most of its life waiting. For work that is computing rather than waiting, the useful number of threads is close to the number of cores, and everything above that is pure overhead. This is exactly the reasoning behind sizing a worker pool (9.5.4) to the bounded resource rather than to the number of tasks.

FoundationalWhy is a single increment of a shared counter unsafe, and what exactly is lost?

counter += 1 is not one instruction. The CPU cannot add to memory in place, so it compiles to three: read the current value into a register, add one to the register, write the register back (1.5). The scheduler may pause the thread between any two of these, because it has no idea they were meant to belong together.

Now put two threads on it. Thread A reads 7. Before A writes, thread B also reads 7. A computes 8 and writes 8. B computes 8 and writes 8. Two increments happened; the counter advanced by one. The name for this is a lost update: B's read was based on a value that A was in the middle of replacing, so B's write silently erased A's work.

What is lost is not the value but the ordering guarantee. Each thread behaved correctly on the data it saw. The failure is that neither one's read-add-write was atomic, meaning indivisible, so the three steps of one operation got interleaved with the three steps of another.

The fix has to make the whole read-add-write into one step. There are three levels of it. The hardware level is a compare-and-swap instruction, which the CPU executes atomically and which is what every atomic counter in every language is built on (9.5.2). The language level is a mutex, which lets only one thread into the three-line region at a time. The database level is a single UPDATE ... SET n = n + 1 statement, which pushes the whole problem to a system that solved it decades ago.

AppliedA signup handler does: check whether the email is taken, and if not, insert the user. Two signups with the same email arrive together and both succeed. Fix it, and explain why the obvious fix is the wrong one.

The bug is check-then-act. Both requests run SELECT ... WHERE email = ?, both find nothing, both proceed to INSERT, and now there are two accounts with the same email. The gap between the check and the act is where the second request slipped in.

The obvious fix, and why it is wrong. The instinct is to wrap the check and the insert in an application-level lock so only one signup runs at a time. This is wrong for three separate reasons, and being able to give all three is what makes this a good interview answer.

First, it does not survive deployment. The moment you run two instances of the service behind a load balancer, each instance has its own lock object, and the two duplicate signups land on two different instances. The bug returns, and it returns at scale, which is the worst time to find it.

Second, it is far too coarse. Every signup in the system now waits behind every other signup, even though two people signing up with completely different emails were never in conflict.

Third, and most importantly, it protects only the paths that remember to take the lock. An admin tool, a data import, or a new endpoint written by someone who did not know about the lock will all walk straight past it. The guarantee is only as good as the discipline of everyone who ever writes code against that table.

The right fix: put the guarantee in the database, as a unique constraint.

CREATE UNIQUE INDEX users_email_unique ON users (lower(email));

Now the check and the act are one operation performed by the database, which does its own locking correctly and does it for every writer, forever. The handler becomes: attempt the insert, and catch the unique-violation error, translating it into a friendly "that email is already registered" response. Note that you now handle the duplicate as an expected outcome rather than as an exception you hope never happens, and that this handler is now correct at one instance and at fifty.

The general principle it demonstrates is the one that runs through this chapter: enforce the invariant where the state lives, not where the request happens to arrive. The application-level lock puts the guarantee in the wrong layer, and the wrong layer is one that gets replicated.

InterviewYour service is single-threaded Node. A colleague says race conditions are therefore impossible. Show them they are wrong, precisely.

They are half right, and the half they are right about is worth granting first, because it makes the correction land harder.

What is genuinely impossible in Node: two pieces of your JavaScript never execute at the same instant. There is one call stack, and a function runs from its first line to its last without any other JavaScript interleaving. So the classic memory-level race — two threads mangling the three machine steps of counter += 1 — cannot happen. That is why there is no mutex in the JavaScript standard library and why you have never needed one for a plain object.

What is entirely possible: every await is a place where your function pauses and other functions run to completion before yours resumes. The unit of atomicity in Node is not the function; it is the stretch of code between two awaits.

async function useCoupon(code: string): Promise<void> {
  const coupon = await db.getCoupon(code);         // ← pause. Other requests run here.
  if (coupon.usesLeft <= 0) throw new Expired();   // both requests see usesLeft = 1
  await db.setUses(code, coupon.usesLeft - 1);     // ← pause. Both write 0.
}

Two requests for the same coupon both read usesLeft: 1, both pass the check, and both write 0. The coupon was used twice. This is a lost update with no threads anywhere in sight, and it will happen in production the first time a customer double-clicks.

The precise statement to give the colleague: single-threaded guarantees that no two lines of your code run simultaneously. It guarantees nothing about the state of the world staying still across an await. Any invariant you check before an await must be re-established after it, or enforced somewhere that does not depend on your process at all.

And the fix, at the right layer: UPDATE coupons SET uses_left = uses_left - 1 WHERE code = ? AND uses_left > 0. If it affects zero rows, the coupon is spent. No lock, no in-process state, correct at any number of instances. If you additionally need per-key ordering inside one process — for fairness or to avoid hammering the database — an in-process queue per coupon code (9.5.6) sits on top of that, never instead of it.

StaffA team reports a bug that only happens in production, roughly twice a week, and never reproduces locally. Walk through how you would work out whether it is a concurrency bug, and how you would fix it without being able to reproduce it.

Step one: read the symptom for the fingerprint. Concurrency bugs have a recognisable signature. The data is internally inconsistent rather than merely wrong: a total that does not match the sum of its parts, a row whose status field contradicts its timestamps, two records that should have been mutually exclusive. Frequency scales with traffic rather than with a code path, so the bug clusters at peak hours. And it never reproduces under a debugger, because attaching one changes the timing.

Step two: find the check-then-act. Rather than hunting the bug, hunt the shape. Search the code path for any place where you read state, make a decision, and then write. Every one of those is a candidate, and in practice the offender is almost always the one touching the resource that two users can contend for: the balance, the seat, the quota, the idempotency key.

Step three: prove it from data, not from a repro. You do not need to reproduce a race to confirm it. You need evidence of overlap. Query the audit or request logs for two operations on the same entity whose intervals overlapped, and see whether the bad outcomes correlate with those overlaps. If every corrupted row has two concurrent writers in its history and no clean row does, you have your proof, and it is stronger evidence than a local repro would be.

Step four: fix by removing the possibility, not by narrowing the window. This is where teams go wrong under pressure. The tempting fixes are re-reading the value before writing, adding a small delay, or retrying on mismatch. All of them shrink the window and none of them close it, which means the bug returns at higher traffic and is now much rarer and much harder to find.

The fix must change what is expressible. In descending order of strength: make the operation a single conditional write at the database, so the invariant holds against every writer past and future; add a unique constraint so the duplicate cannot be stored at all; move the state behind a single owner so only one writer exists; and only if none of those fit, take a per-entity lock, scoped to the entity rather than global.

Step five: prove the fix without waiting two weeks. Write a load test that deliberately fires N concurrent requests at the same entity, which is the case normal load tests never generate because they spread traffic across many entities. If the fix is a conditional write, the assertion is that exactly one of the N requests succeeded and the other N−1 got a clean "you lost" response. That test fails reliably against the old code, which is what makes it worth keeping.

Step six: harvest the general lesson. Ask what other code touches the same entity, because a race in one handler is nearly always a race in three. And ask what would have caught this earlier: usually the answer is that the invariant was living in application code instead of in the schema, and moving it down is a change that protects every future handler as well as this one.

Scenario Drill

DrillYou are designing the seat-selection step of a cinema booking system. Users pick seats from a live map, hold them for ten minutes while paying, and the hold expires if they do not finish. Identify every race in that sentence and say where each guarantee belongs.

Read the requirement one clause at a time and name the contested resource in each. That habit — turning a sentence of English into a list of things two users can grab at once — is the whole skill.

Clause one: "users pick seats from a live map." The map is a read, so it does not race by itself, but it creates the illusion that drives every other race. Two users looking at the same screenshot of availability both believe seat H12 is free. The design consequence is that the map must never be treated as a reservation. It is a hint, it is allowed to be slightly stale, and the UI must be built to survive being told "actually, that one just went" at claim time. Designs that try to make the map perfectly accurate end up serialising every viewer behind every buyer, which is a large cost paid for an illusion.

Clause two: "hold them." This is the real contested resource, and the race is the classic check-then-act: two users both see H12 free, both write a hold, and the second overwrites the first. The guarantee belongs at the seat row in the database, expressed as a conditional write:

UPDATE seats
   SET held_by = ?, held_until = now() + interval '10 minutes'
 WHERE seat_id = ? AND (held_by IS NULL OR held_until < now());

The WHERE clause is the check, the SET is the act, and the database makes them one operation. Zero rows affected means you lost; that is a normal, expected response and the UI shows "someone just took that seat." Crucially this is correct with one server or with fifty, because the guarantee lives with the data rather than in a process.

For a group booking of four adjacent seats, one row at a time is not enough, because you can win two seats and lose two. Wrap the four conditional updates in one transaction and require all four to report one affected row; otherwise roll back and release. The invariant "all or nothing" now belongs to the transaction, which is the right home for it.

Clause three: "for ten minutes while paying." Two races hide here. The first is the expiry itself: at the moment the hold lapses, the original user's payment may be in flight while another user's claim arrives. Never let expiry be decided by a background job that writes held_by = NULL, because that job races with the payment. Instead make expiry implicit in the data — the held_until < now() test in the WHERE clause above already does this — so a hold expires by being ignored rather than by being deleted. There is then no moment where two writers disagree.

The second is the payment confirmation itself, which must be able to fail cleanly: UPDATE seats SET sold_to = ? WHERE seat_id = ? AND held_by = ? AND held_until > now(). If the user's hold lapsed while the card was processing, this affects zero rows, and the correct product behaviour is to refund and apologise rather than to sell a seat that someone else now holds. Deciding that before writing the code is what stops it becoming an incident.

Clause four: "the hold expires if they do not finish." Expired holds do eventually need cleaning up so the table does not grow forever, but note that the cleanup is now purely a housekeeping job with no correctness role at all, because the conditional writes already ignore stale holds. Being able to say "the sweeper is an optimisation, not a guarantee" is exactly the kind of separation an interviewer is listening for.

Where a lock would still earn its place. Notice that nothing above needed one. A per-showing lock would be justified for a different reason: fairness. If a blockbuster's tickets open at 9am and ten thousand people claim the same block simultaneously, conditional writes produce ten thousand simultaneous database round trips and nine thousand nine hundred losers. Putting a single ordered queue in front of each showing turns the stampede into a line, which lets you show a queue position and gives a much better experience. That is a user-experience decision layered on top of a correctness guarantee that already holds without it, and keeping those two motivations separate in your explanation is what makes the design sound deliberate. The full funnel for this shape is built in the flash-sale study, Chapter 11.16.

The summary sentence worth ending on: every race in this feature was a check-then-act on one seat row, and every fix was the same move — fold the check into the write and treat losing as a normal outcome. The locks that remain are there for fairness and load shaping, not for correctness.