Appearance
11.15 — Stock Exchange & Matching Engine
Two orders to buy the same share at the same price arrive 3 microseconds apart. Whichever one arrived first must be filled first, and both participants have paid for the right to be told, afterwards, exactly which one that was and why.
That sentence rules out most of this book. Sharding, caching, eventual consistency, "the read is a bit stale but nobody notices" — all of it is either irrelevant or actively wrong here. A cache cannot help when the answer must be computed from a state that changed a microsecond ago. Eventual consistency cannot help when the whole product is agreement about ordering. And a network hop cannot help when a network hop costs more than the entire latency budget.
So this study teaches the architecture that high-performance financial systems actually use — single-threaded, in memory, deterministic, journalled — and, more usefully, it teaches when scaling out is the inferior answer, which is a judgement most engineers never get to practise.
1. Requirements
Functional. Accept limit and market orders. Match by price and then by time. Cancel and modify. Publish a live feed of the order book and every trade. Enforce risk checks before an order can reach the book.
Non-functional, with numbers.
- Matching latency under 100 microseconds at p99. Microseconds, not milliseconds.
- Strict fairness. Orders are processed in the order they arrived, with no exception for anyone.
- Zero lost trades, complete auditability, and deterministic replay. A trade that happened must be reconstructible from the record, in order, forever.
- 1 million orders a second at peak.
Out of scope today: clearing and settlement, which is a separate system operating over days; regulatory report formats; and retail brokerage, which sits in front of an exchange rather than inside one.
The clarifying questions, and what each answer changes
"What is the total state, and does it fit in memory?" Ask this first, because the answer decides the entire architecture. If the order book fits in memory the design below is available; if it does not, none of it is.
"Is the latency target a p99 or an average?" In this domain the tail is the product. A system with a 30-microsecond average and a 4-millisecond p99.9 is not fast, it is occasionally catastrophic, and participants will measure it whether you do or not.
"Must the ordering be provable afterwards, or merely correct?" Provable is a much stronger requirement and it forces a journal. It is also the actual requirement in every regulated market.
"Can two orders for the same instrument be processed by different machines?" No, and asking makes the reason explicit: matching needs a single view of the book, so the unit of parallelism is the instrument, not the order.
"What happens when the price moves too far, too fast?" The answer is that trading halts — which is a state in the engine rather than an outage, and knowing that early stops someone from designing a system that cannot stop.
2. Estimation
Order flow. 1 million orders a second × ~100 bytes = 100 MB a second of inbound messages. What that forces: the input path needs care — parsing, validation and sequencing at that rate is real work — but it is not the hard part.
State size, which is the first decisive number. One instrument's order book might hold 100,000 resting orders at ~64 bytes each = about 6 MB. Every instrument together, across a whole venue, is a few gigabytes. What that forces: everything. The state fits in memory, which removes the primary reason to distribute state at all. No database on the hot path, no network call to fetch state, no cache to invalidate, no consistency protocol between shards of one book.
Latency budget, which is the second decisive number. The target is 100 microseconds end to end. A round trip through an ordinary operating-system network stack costs 100 to 500 microseconds — at or over the entire budget before any work is done. Kernel-bypass networking brings a hop down to roughly 10 microseconds, which is affordable once but not repeatedly. What that forces: matching cannot be a distributed computation. Any design in which the matcher consults another machine has already spent its budget. The work must happen in one process, on data already in cache.
Work per order. Walking a few price levels and updating linked lists is tens to hundreds of nanoseconds on cached data. What that forces: the inversion that makes this study unusual. Acquiring a lock costs more than the work the lock protects. Coordinating two cores costs more than doing it on one. That is why a single thread is not a limitation accepted reluctantly — it is faster.
Throughput of one core. At a few hundred nanoseconds per order, a single thread handles several million orders a second, comfortably above the requirement for any one instrument. What that forces: the per-instrument cap is real and it is far above real per-symbol volume, so it is accepted rather than engineered around.
Journal volume. 1 million messages a second × 100 bytes = 100 MB a second, ~8.6 TB a day, written sequentially. What that forces: sequential writes to fast storage, retained for years because the journal is the legal record. This is the largest storage consumer and the least negotiable.
3. The interface
Exchange protocols are binary and fixed-layout rather than JSON over HTTP, for a reason worth stating: parsing a text format costs microseconds you do not have, and variable-length fields cause allocation. The shape, expressed readably:
→ NewOrder { clientOrderId, instrument, side, price, qty, type, tif }
← Ack { clientOrderId, orderId, seq, ts }
← Fill { orderId, execId, qty, price, seq, ts, aggressor }
← Reject { clientOrderId, reason }
→ Cancel { orderId }
← Canceled { orderId, seq }Every outbound message carries the sequence number that produced it. That single field is what makes downstream consumers able to deduplicate, detect gaps, and resynchronise — and it is what makes the whole system auditable, because a fill can always be traced back to the exact input that caused it.
clientOrderId is the participant's own identifier, and it makes submission idempotent: a resend after a lost acknowledgement is recognised rather than creating a second order. The same reasoning as Idempotency-Key in 11.14, with a shorter deadline.
tif — time in force — is what happens to the unfilled part, and it is worth noticing that the order types are not different algorithms but different answers to that one question. A limit order rests. A market order takes whatever is available and cancels the rest. Immediate-or-cancel fills what it can right now and abandons the rest. Fill-or-kill executes completely or not at all.
The market data feed is a separate path with a different shape: a stream of book changes plus periodic snapshots, so that a consumer can join at any time by taking a snapshot and applying updates from its sequence number onward. That is the snapshot-then-stream handshake from 11.7, and it appears here for the same reason.
4. The order book
The structures are chosen for latency, not elegance.
Price levels live in an array indexed by tick when the price range is narrow — constant-time access and excellent cache behaviour — or in a sorted structure when it is not.
Each level is a linked queue of orders in arrival order, so the first order at a price is the first to be filled.
A map from order identifier to its node makes cancellation constant time. That last one is worth defending because it looks like an optimisation and is not: in real markets cancels vastly outnumber fills, often by an order of magnitude, so cancel is the common case and making it fast is making the system fast.
The trade happens at the resting order's price. An incoming buy willing to pay 100.53 that meets a resting sell at 100.52 trades at 100.52 — the person who was already there set the price. This is not a rounding convention, it is what makes posting an order worthwhile, and getting it backwards inverts the incentive to provide liquidity.
5. Architecture
Why one thread? Three reasons, and they compound.
Locks cost more than the work they protect. Coordinating two cores over a shared order book costs more in cache-line traffic than simply doing the work on one core.
A single core is fast enough — several million simple orders a second on cached data, which exceeds the requirement per instrument.
And a single thread processing a totally ordered input is a pure function, which gives determinism for free. That third reason turns out to be worth more than the first two combined.
The sequencer is the fairness authority. It stamps every inbound message with a monotonically increasing number and writes it to the journal before the engine sees it. Two consequences follow. The order in which things happened is a recorded fact rather than an emergent property. And the engine's entire history is replayable from the journal, which is event sourcing (10.8.4) at microsecond granularity.
Risk checks live in the gateway, not the engine. Credit limits, position limits, price bands that catch a mistyped order, and prevention of a participant trading with themselves — all of it happens before the engine, so the hot path stays pure and the checks scale horizontally across gateways. In most jurisdictions these checks are a legal requirement, and in engineering terms they are a validation boundary (9.6.1) with unusually high stakes.
6. Deep dives
6.1 Determinism, and what it buys
The engine is a pure function of its input sequence: the same inputs always produce the same outputs and the same final state. Maintaining that requires five prohibitions.
No wall-clock reads inside the engine. Time arrives as a field on the input message, stamped by the sequencer. now() returns a different value on replay, which breaks everything immediately — and any time-dependent behaviour, such as an order expiring, must read the injected timestamp instead.
No randomness. No random tie-breaking, no randomised data structures, nothing that varies between runs.
No input or output. The engine cannot query a database or call a service, because the answer might differ on replay.
No dependence on unspecified iteration order. Hash-map iteration order and pointer values must not affect the output.
A totally ordered, durably journalled input. The sequencer assigns the number and persists the message before processing, so the journal is a complete definition of the engine's history.
What that buys, and it is a great deal.
A hot standby with no replication protocol at all. A second engine consuming the same journal is byte-identically in sync. It is not catching up, it is already there. The hardest problem in stateful failover simply does not exist.
Recovery by replay. A crashed engine reconstructs its exact state from the journal in seconds, because the state is small and processing is fast.
Perfect reproduction of any incident. A production event is replayed exactly, into an instrumented build, and the behaviour is identical. For a system where a defect can cost millions in minutes, that is worth the coding discipline several times over.
Auditability. The journal is a defensible record of exactly what the venue received, in what order, and what it did — which is a regulatory requirement, not a convenience.
Safe upgrades. A new engine version can be run over historical journals and its outputs compared against the old version's, so a behavioural change is visible before it reaches production rather than after.
The generalisation worth carrying away: determinism converts state replication into input replication, which is dramatically simpler. It is the same insight behind replicated state machines in consensus protocols (10.7.2) and behind event sourcing.
6.2 Where the microseconds actually go
Reaching a 100-microsecond target is not achieved by writing tighter loops. It is achieved by removing whole categories of delay.
Bypass the operating system's network stack. The kernel's path from wire to application costs tens of microseconds on its own. Specialised networking hardware and libraries deliver packets straight to the application, bringing a hop to roughly ten microseconds.
Spin rather than sleep. A thread that blocks waiting for work must be woken by the scheduler, which costs microseconds and is unpredictable. A thread that busy-waits burns a core continuously and responds instantly. Burning a whole core to avoid a context switch is an absurd trade in most systems and a rational one here.
Allocate nothing on the hot path. Every object is pre-allocated in pools. In a garbage-collected language this is existential rather than merely helpful: a collection pause is measured in milliseconds, which is ten to a hundred times the entire budget, so these engines are written in languages without a collector or in one tuned so the collector never runs during a session.
Respect the cache. Data layout is chosen so that a match walks contiguous memory, and structures that different cores touch are separated so that writing one does not invalidate the other's cache line.
Pin threads to isolated cores. The engine's core runs nothing else — no interrupts, no other processes.
And the measurement discipline, which matters more than any single technique: latency is reported as a full distribution — p99, p99.9, p99.99 and maximum — because in this domain the tail is the product. A mean latency figure is close to meaningless: a system averaging 30 microseconds with a 4-millisecond worst case is not fast, it is occasionally disastrous, and participants measure it whether or not you publish it.
6.3 Fairness has a technical shape
Fairness is a market-design requirement that turns into specific engineering.
Equal network paths. Participants in the same facility are given cables of matched length, so that being physically closer to the engine confers no advantage. This is real, and it is measured in metres.
Deterministic sequencing at a single point. There is one place where order is decided, and it decides for everyone.
No queue jumping. No priority tier, no fast lane, no exceptions.
And sometimes, deliberate delay. Some venues insert a small randomised delay before orders reach the book, specifically to blunt the advantage of being a few microseconds faster. That is a policy decision implemented in the sequencer, and it is a good example of a requirement that comes from market design rather than from engineering — and one that an engineer optimising purely for latency would never propose.
6.4 Market data must never slow the engine
The engine emits every book change and every trade. A publisher fans those out to subscribers, who are numerous, geographically spread, and of wildly varying speed.
The rule is that the market data path must never apply back-pressure to the engine. A slow consumer is dropped, or served from a separate buffer, and is never permitted to slow matching — because matching is the venue and market data is a view of it.
A dropped consumer recovers by snapshot plus stream: take a snapshot of the book, then apply updates from the snapshot's sequence number onward. The sequence numbers on every outbound message are what makes that join possible without a gap or a duplicate.
6.5 Sharding, and its hard limit
Different instruments are independent order books, so they parallelise perfectly: put a hundred symbols on one machine, split them across machines as volume demands, and there is no coordination between them at all.
One book cannot be split across machines, and it is worth being precise about why rather than treating it as a limitation. Matching requires a single consistent view of the book, so splitting it would require distributed agreement about ordering on every order — which reintroduces exactly the network latency the design exists to avoid, and which makes the fairness guarantee much harder to prove.
So one instrument's throughput is capped by one core. That cap is far above real per-symbol volume, and it is accepted deliberately rather than engineered around — which is an unusual sentence in a systems design book and the right one here.
6.6 Halts are a feature, not an outage
Venues stop trading when prices move beyond defined thresholds. That is not the system failing; it is a state in the engine's own state machine, entered deliberately, with defined rules for entering and leaving.
Naming it matters for the design, because a system built on the assumption that it must always keep matching cannot stop cleanly. Halting must be a first-class transition that pauses matching, keeps the book, continues to accept cancels, and publishes its own state on the market data feed — which is a set of behaviours you cannot retrofit into an engine that only knows how to run.
7. Decision Ledger
| Decision | Alternatives | Why this | What it costs |
|---|---|---|---|
| Single-threaded, in-memory engine | multi-threaded; distributed matching | locks cost more than the work; no network in the path; determinism comes free | one instrument's throughput is capped by one core |
| A sequencer that journals input before processing | write state to a database | replay-based recovery, and a standby needing no replication protocol | the sequencer is a critical path and a failure domain of its own |
| Determinism enforced by prohibition | pragmatic impurity where convenient | replay, standby and incident reproduction all depend on it | strict discipline; one stray clock read silently breaks it |
| Shard by instrument only | split a hot book across machines | correctness and fairness need one view of the book | a hot instrument cannot be scaled further |
| Risk checks in the gateway | inside the engine | keeps the hot path pure and scales horizontally | limits become a distributed decision needing care |
| Trade at the resting order's price | trade at the incoming price | posting an order is only worthwhile if it sets the price | none — the alternative inverts the incentive |
| Kernel bypass, busy-wait, zero allocation | ordinary server engineering | the only way to reach microsecond targets | burns cores; specialised code and hardware; a small hiring pool |
| Market data may be dropped, never buffered into the engine | back-pressure from slow consumers | matching is the venue; the feed is a view of it | slow consumers must implement snapshot-and-resync |
8. Scale and failure
Failover is the interesting case, and it has one catastrophic mode. Because standbys are byte-identical, promotion is not a state-transfer problem — it is purely the question of who is allowed to speak. Get that wrong and two engines both believe they are primary, both publish fills, and the market has two divergent books with trades that cannot be unwound. In a system whose output is legally binding, that is unrecoverable in the strict sense: you cannot un-execute a trade a counterparty has already acted on.
| What breaks | Blast radius | How you find out | What keeps it running | Recovery |
|---|---|---|---|---|
| Engine crashes | that instrument, briefly | heartbeat; sequence stops advancing | the standby is already in sync | promote, or replay the journal into a fresh instance in seconds |
| Two primaries | catastrophic — divergent books | epoch mismatch rejected downstream | fencing on the output path, plus a single promotion authority | halt rather than guess; never self-promote on a timeout |
| Sequencer fails | everything for that venue | sequence gap | the sequencer is itself replicated with consensus | the most scrutinised component in the system |
| Standby diverges | invisible until failover | continuous state-hash comparison at checkpoints | nothing else would catch it | rebuild the standby by replay; find the determinism violation |
| Journal write slow | matching stalls behind it | journal write latency | fast sequential storage, pre-allocated | this is the one I/O the design cannot remove |
| Market data consumer falls behind | that consumer only | per-consumer lag | drop it rather than slow the engine | it rejoins by snapshot and sequence number |
| A participant's algorithm goes wrong | the book, and the market | per-participant order rate | gateway rate limits and a kill switch per participant | pull the switch; the requirement is regulatory as well as sensible |
| Price moves beyond thresholds | trading, deliberately | the engine's own state | a halt is a state, not a failure | resume by rule, publishing the transition |
Three defences make split brain structurally impossible rather than unlikely.
Put the authority in the sequencer, and make the sequencer the thing with consensus. A replicated sequencer means there is exactly one legitimate input sequence, and an engine that cannot obtain sequence numbers cannot produce valid output. The engines then need no leader election of their own.
Fence the output path. Every published fill carries an epoch number, and downstream consumers reject anything from a stale epoch. This is the guarantee that survives a failure of the detection logic, and it must exist regardless of how good that logic is.
Never let a standby promote itself on a timeout. Silence is indistinguishable from a network partition, so a standby that promotes itself because it stopped hearing from the primary is precisely the split-brain generator. Promotion goes through a single arbiter.
And the standby-divergence row deserves its own note, because it is silent by construction. A stray clock read or a hash-iteration dependency makes the standby drift, and nothing detects it until the day you fail over and discover two different books. The invariant that catches it is a continuous state-hash comparison between primary and standby at sequence checkpoints, alarming on any mismatch. That belongs in production, permanently (10.10).
What the interviewer will push on
"Everything else in this book scales out. Why is this single-threaded?" They want the reasoning, not the fact. Four points: the state is small so distributing it solves nothing; a network round trip exceeds the entire latency budget so distribution is unaffordable; the work per order is so small that a lock costs more than the work it protects; and a single thread over a totally ordered input is a pure function, which yields determinism for free. Then state the cost you accepted — one instrument's throughput is capped by one core — rather than waiting to be asked.
"What exactly does determinism require, and what does it give you?" Five prohibitions: no clock, no randomness, no I/O, no unspecified iteration order, and a durably journalled totally ordered input. Five payoffs: a standby with no replication protocol, recovery by replay, exact reproduction of any incident, an auditable legal record, and the ability to run a new version over historical journals and compare outputs. The tell is naming the clock explicitly, because that is the prohibition that gets broken accidentally.
"How do you prevent two engines from both thinking they are primary?" The strong answer does not rely on detection. Move the authority to a replicated sequencer so that only one legitimate input sequence exists; fence the output path with epoch numbers so a stale primary's messages are rejected even if everything else fails; and never allow a standby to self-promote on a timeout, because silence and partition are the same observation. Close with the priority: halting for thirty seconds is an incident, running two books is an existential problem.
"Your standby has been silently diverging for a week. How would you know?" You would not, unless you built the check — which is the point of the question. A continuous state-hash comparison at sequence checkpoints is the only thing that catches an accidental clock read before it matters, and it has to run in production rather than anywhere else.
"Which operation do you optimise first?" Cancel, and saying so demonstrates familiarity with real order flow rather than with the textbook picture. Cancels outnumber fills by roughly an order of magnitude, which is why the map from order identifier to list node exists and why cancellation is constant time.
"A slow market-data subscriber is causing back-pressure. What do you do?" Drop them. Matching is the venue; the feed is a view of it, and a view must never be able to slow the thing it is viewing. Then describe how they rejoin — a snapshot plus updates from a sequence number — which is why every outbound message carries one.
Volunteer this, because nobody asks: the deep lesson of this study is a decision rule rather than an architecture. Ask whether the system's hard part is throughput of independent work, or agreement about one shared thing. Most systems are the first, which is why the distributed toolkit dominates the rest of this book. When it is the second, distributing is not merely unnecessary — it makes correctness harder to prove and latency worse at the same time. Recognising which kind of problem you have, before choosing tools, is the skill this page is really teaching.
Next: 11.16 — the same contention problem without the microseconds. Ten thousand people want the last hundred units, and the question becomes how to say no to 9,900 of them quickly, correctly, and without melting the database.
Recall
- Two numbers decide the architecture: the whole book fits in memory (a few gigabytes), and one network round trip through an ordinary stack (100–500 µs) exceeds the entire 100 µs budget. So: no database, no cache, no distributed computation on the hot path.
- Single-threaded, in memory, fed by a sequencer that stamps a monotonic order and journals every input before the engine sees it. Locks cost more than the work; one core does millions of orders a second; and one thread over an ordered input is a pure function.
- Determinism requires five prohibitions: no wall clock (time arrives on the message), no randomness, no I/O, no unspecified iteration order, and a durable totally ordered journal.
- It buys five things: a standby that is byte-identical with no replication protocol, recovery by replay, exact reproduction of any incident, a legally defensible audit record, and safe upgrades by replaying historical journals.
- Price then time. Better price first; at equal price, earlier order first. The trade happens at the resting order's price. Structures: price levels as an array or sorted map, a queue per level, and an order-id-to-node map so cancel is O(1) — cancels outnumber fills by roughly ten to one.
- Shard by instrument only. One book cannot be split, because matching needs one view; the per-core cap is accepted deliberately.
- Risk checks in the gateway, market data that may be dropped but never back-pressures, and latency reported as a full distribution because the tail is the product.
- Split brain is the catastrophic mode. Consensus on the sequencer, epoch fencing on outputs, a single promotion authority, and halting rather than guessing. Detect a diverging standby with a continuous state-hash comparison.
Self-test: Which two numbers rule out the usual distributed toolkit? Give the five prohibitions and the five payoffs of determinism. Which operation must be constant time, and why that one? What can and cannot be sharded, and why? Name the three defences against two primaries.
Quiz Bank
FoundationalWhy is a matching engine single-threaded and in memory when everything else in this book scales out?
Because the constraints invert the usual reasoning, in four steps.
The state is small. One instrument's order book is a few megabytes; every instrument together is a few gigabytes. The primary reason to distribute state — it does not fit — simply does not apply, so distributing buys nothing and costs coordination.
The latency budget forbids the network. The target is 100 microseconds end to end, while a round trip through an ordinary operating-system network stack costs 100 to 500 (10.12). Any design in which matching consults another machine has spent its entire budget before doing any work.
The work per order is tiny. Walking a few price levels and updating linked lists is tens to hundreds of nanoseconds on cached data. At that scale, acquiring a lock costs more than the work the lock protects, and the cache-line traffic between two cores contending over one book costs more than doing it on one core. A single thread in a hot loop over in-cache data handles several million orders a second, which is above the requirement.
And the fourth reason is worth more than the first three: determinism. A single thread processing a totally ordered input sequence is a pure function, which yields three enormous properties at no additional cost. A standby consuming the same journal is byte-identically in sync with no replication protocol and nothing to reconcile. Recovery is journal replay rather than state transfer. And any production incident can be reproduced exactly by replaying the sequence into an instrumented build.
What is given up, stated honestly: one instrument's throughput is capped by a single core and cannot be raised by adding machines. That is accepted for two reasons. The cap sits far above real per-symbol volume. And fairness requires a single view of the book anyway — splitting one book across machines would require distributed agreement about ordering on every order, which reintroduces exactly the latency the design exists to remove and makes the fairness guarantee much harder to demonstrate. Scaling happens by instrument, and that is sufficient.
InterviewWhat does deterministic replay require, and what does it give you?
What it requires — five prohibitions, and the first is the one that gets broken by accident.
No wall-clock reads inside the engine. Time is injected as a field on each input message, stamped by the sequencer. A call to now() returns a different value on replay and destroys determinism instantly, and any time-dependent behaviour — an order expiring, a session boundary — must read the injected value instead.
No randomness. No random tie-breaking, no randomised structures, nothing that varies run to run.
No input or output. Everything arrives through the sequenced stream. The engine cannot query a database or call a service, because the answer might differ when the sequence is replayed later.
No dependence on unspecified iteration order. Hash-map iteration order, pointer values, and anything else that varies between runs must not affect output.
A totally ordered, durably journalled input stream. The sequencer assigns a monotonic number and persists the message before the engine processes it, so the journal is the complete definition of the engine's history.
What it gives you — and the payoff is disproportionate to the discipline.
A hot standby with no replication protocol. A second engine consuming the same journal holds byte-identical state. It is not catching up; it is already there. Failover needs no state transfer, no catch-up window and no reconciliation — the hardest problem in stateful failover ceases to exist.
Recovery by replay. A crashed engine rebuilds its exact state from the journal in seconds, because the state is small and processing is fast.
Exact reproduction of any incident. A production event is replayed into an instrumented build and behaves identically. In a system where a defect costs millions within minutes, that is worth the discipline several times over.
An auditable record. The journal is a defensible statement of exactly what the venue received, in what order, and what it did about it — a regulatory requirement rather than a convenience.
Safe upgrades. A new engine version can be run over historical journals and its outputs compared with the previous version's, so any behavioural change is visible before it goes near live trading.
The generalisation to carry away: determinism converts state replication into input replication, which is dramatically simpler. It is the same insight behind replicated state machines in consensus protocols (10.7.2) and behind event sourcing (10.8.4).
StaffDesign the failover story. What are the failure modes, and how do you make the catastrophic one impossible?
The setup. A primary and one or more standbys consume the identical journalled input sequence. Because processing is deterministic, the standbys hold byte-identical state with no state-transfer protocol at all — a standby is not catching up, it is already there. Failover therefore reduces to one question: which instance is allowed to speak?
The catastrophic mode is two primaries. Both publish fills, the market has two divergent books, and trades exist that cannot be unwound. In a system whose output is legally binding, that is unrecoverable in the strict sense — you cannot un-execute a trade a counterparty has already acted upon. Everything in the failover design exists to make this impossible rather than unlikely.
Defence one: move the authority to the sequencer, and put consensus there. A replicated sequencer means exactly one legitimate input sequence exists, and an engine unable to obtain sequence numbers cannot produce valid output. The engines then need no leader election of their own, because their outputs are only meaningful in relation to sequenced inputs.
Defence two: fence the output path. Every published fill carries an epoch number, and downstream consumers and the market-data publisher reject messages from a stale epoch. This is the guarantee that survives a total failure of the detection logic, so it must exist regardless of how good the detection is. A partitioned old primary that resumes and starts publishing is simply ignored.
Defence three: never allow self-promotion on a timeout. A standby that promotes itself because it stopped hearing from the primary is precisely the split-brain generator, because silence is indistinguishable from a network partition. Promotion goes through a single arbiter.
Defence four: prefer halting to guessing. A venue that stops trading for thirty seconds while promotion is resolved has an incident. A venue running two books has an existential legal problem. That priority should be written down before it is needed.
The other failure modes, briefly. Sequencer failure is the true single point of failure, which is why it carries its own replication and is the most scrutinised component in the system. A diverging standby — some determinism violation slipped in — is invisible until failover, so it needs a continuous state-hash comparison between primary and standby at sequence checkpoints, alarming on any mismatch. That invariant is what catches an accidental clock read before it matters, and it belongs in production permanently. Journal corruption is handled by replication, checksums, and periodic replay into a shadow instance. Partial output at the moment of failure is handled by downstream deduplication on sequence number, which makes a re-published message harmless.
The staff framing: deterministic replay makes the standby trivially correct, which moves the entire difficulty into deciding who may speak. So the design effort belongs in sequencing, fencing and promotion authority — and any effort spent on state replication instead is effort spent on a problem that determinism already solved.
Flashcards
FlashWhy not distributed
The book fits in memory (a few GB) and one ordinary network round trip (100–500 µs) exceeds the entire 100 µs budget. No database, no cache, no network on the hot path. The work per order is smaller than a lock.
FlashWhat one thread buys
Determinism, for free. The engine becomes a pure function of its input sequence, so the standby needs no replication protocol, recovery is replay, incidents reproduce exactly, and upgrades can be diffed against historical journals.
FlashThe five prohibitions
No wall clock (time arrives on the message) · no randomness · no I/O · no unspecified iteration order · plus a durably journalled, totally ordered input. The clock is the one broken by accident.
FlashPrice then time
Better price first; at equal price, the earlier order. The trade happens at the resting order's price, which is what makes posting worthwhile. Cancel must be O(1) via an order-id-to-node map — cancels outnumber fills roughly ten to one.
FlashSharding limit
Instruments are independent and parallelise perfectly. One book cannot be split, because matching needs a single view and splitting reintroduces network agreement. The per-core cap is accepted deliberately.
FlashSplit-brain defences
Consensus on the sequencer · epoch fencing on every output so a stale primary is rejected · a single promotion authority, never timeout self-promotion · halt rather than guess. Plus a continuous state-hash check to catch a silently diverging standby.
Scenario Drill
DrillApply this architecture outside finance. Where else does a single-threaded deterministic in-memory engine with a journal beat the usual distributed design, and where would it be a mistake?
The signature to look for is always the same three conditions: the state fits in memory, the operation needs a single consistent view, and either latency or auditability is extreme.
Where it wins.
Multiplayer game servers. A match's world state is small, every player's action must be ordered consistently for everyone, and rollback and replay for lag compensation and cheat investigation is exactly deterministic replay. This is why authoritative game servers are built this way rather than as a set of services.
Real-time auction and advertising exchanges. Bounded state per auction, strict fairness requirements, and a sub-millisecond budget.
Inventory allocation for a flash sale (11.16). One authoritative counter per item, contention is the entire problem, and a single-threaded owner with a journal outperforms any distributed locking scheme while making overselling structurally impossible rather than merely unlikely.
Workflow engines where correctness and audit matter more than throughput (11.21). Journalling inputs and replaying gives free history and the ability to reconstruct exactly why a workflow took the path it took.
Anything whose job is to impose an order — sequencers, rate limiters, allocators.
Where it is a mistake.
When the state does not fit in memory, or grows without bound. The whole design rests on a small working set. A system whose state is terabytes gets none of the benefits and every one of the constraints.
When the workload is embarrassingly parallel and latency-tolerant. A web service doing independent per-user reads gains nothing from a single thread and loses the ability to scale by adding machines, which is the cheapest scaling available.
When the work per operation is large — image processing, model inference, complex analytical queries. One thread becomes the bottleneck immediately, and the premise that a lock costs more than the work inverts completely.
When the operation genuinely requires calling something else. Input and output break determinism and block the single thread. There is a correct workaround — issue the call outside the engine and feed the result back in as a sequenced input message — but it turns straightforward code into an asynchronous state machine, and that cost should be weighed rather than discovered.
When the team cannot sustain the discipline. One accidental clock read or random value in the engine silently destroys both the standby synchronisation and the replay guarantee, and the failure is invisible until the day you fail over. Without the continuous state-hash comparison running in production, this design is a trap rather than an asset — which is why that invariant is not optional for anyone choosing this shape.
The decision rule to carry away: ask whether the system's hard part is throughput of independent work or agreement about one shared thing. Most systems are the first, which is why the distributed toolkit dominates the rest of this book. When it is the second, distributing is not merely unnecessary — it makes correctness harder to prove and latency worse at the same time, and choosing it anyway is the most expensive kind of default.