9.7.26 — The Live Data Family

"Design a music streaming app." · "Design a live cricket score service."

A song is playing on your phone. You pick up your laptop and it is playing there too, at the right second. Somewhere else, four million people are watching a cricket match, and a wicket falls: every one of them should see it within a second or two, including the person who opened the app half a second after it happened.

These look like two unrelated products. They are the same problem twice, and the shared shape is this: something changes over time, people join at arbitrary moments, and everyone must end up with the same picture. The design skeleton is identical. What separates them is one question, and it decides almost everything else.

1. The one question that separates them

Is the changing thing private to one person, or shared by everyone?

Your playback position is yours. Three devices care about it and nobody else does, so the "fan-out" is three, and the hard part is that all three can write — you can press pause on any of them.

A cricket match is shared. Four million people care about it and none of them can change it, so the hard part is fan-out and nothing else. There is exactly one writer, sitting in the ground.

	Music playback	Live match
Who writes	Any of your devices	One scorer
Who reads	Your devices	Millions
Hard part	Conflicting writes	Fan-out
Corrections	Rare	Routine

One writer with millions of readers is a caching problem. Many writers with a handful of readers is a conflict problem. Recognising which one you have been handed, in the first two minutes, is what stops you designing the wrong system carefully.

2. The skeleton both problems share

Three pieces, and they recur in every live system in this book.

An append-only log of what happened. A ball is bowled; a track is skipped. Each is an event with a sequence number, and the events are never edited.

A current state that is a fold over the log. The scorecard is computed from the balls; the playback state is computed from the playback events. Nothing stores a total that the log cannot rebuild — the same rule the wallet applies to a balance in 9.7.10.

Snapshot then stream, for anyone joining late. A new watcher cannot replay four hours of balls, and cannot start from "the next event" either — they would have no score at all. They get the current state as of a sequence number, then every event after that number. The sequence is what stitches the two halves together with no gap and no duplicate.

Figure 1 — Snapshot plus stream, joined by a sequence number. The snapshot states which event it already contains, and the stream starts at the next one. That number is the entire mechanism.

interface Snapshot<S> {
  state: S;
  throughSequence: bigint;          // (1)
  takenAt: Instant;
}

interface Update<E> {
  sequence: bigint;                 // (2)
  event: E;
}

function onJoin<S, E>(snap: Snapshot<S>, stream: Stream<Update<E>>): void {
  let have = snap.throughSequence;
  stream.on("update", u => {
    if (u.sequence <= have) return;              // (3) already applied
    if (u.sequence > have + 1n) return resync(); // (4) a gap — start over
    apply(u.event); have = u.sequence;
  });
}

(1) The snapshot names the last event it includes. This is the whole design in one field.

(2) Every update carries its number, so the client can check rather than trust.

(3) An update the snapshot already covered is dropped. This happens constantly and is not an error — the snapshot was built while the stream was already flowing.

(4) A gap means something was lost, and the only safe response is to fetch a fresh snapshot. The alternative — carrying on with a hole — leaves a client quietly showing a wrong score with no way to notice. Detecting the gap is the point of numbering every update.

3. Music: the queue is computed, not stored

The obvious model is a list of upcoming tracks. It breaks the moment anything interesting happens.

interface PlaybackState {
  sourceId: SourceId;               // (1) the album, playlist or radio seed
  sourceVersion: number;            // (2)
  order: "inOrder" | { shuffled: { seed: number } };   // (3)
  index: number;                    // (4) position within the resolved order
  trackId: TrackId;                 // (5)
  positionMs: number;               // (6)
  status: "playing" | "paused";
  repeat: "off" | "all" | "one";
  updatedAt: Instant;
  updatedByDevice: DeviceId;        // (7)
}

(1) What is being played comes from a source, not a copied list. A playlist of 800 songs would otherwise be duplicated into every device's playback state.

(2) The playlist can change while you are listening — someone adds a track to a shared playlist, or you remove one. The version lets a device notice its resolved order is stale and rebuild it, instead of silently playing from a list that no longer exists.

(3) Shuffle is a seed, not a shuffled array, and section 5 shows why that one choice solves three problems at once.

(4) Where you are in the resolved order. With the source, version and seed, the whole queue is reproducible on any device from four small fields.

(5) The track identifier is stored as well as derivable, so a device can start playing audio immediately without resolving the order first. It is a cached value, and if it disagrees with index, the index wins.

(6) Position in milliseconds. Not seconds: a two-hour podcast resumed to the nearest second is fine, and a track that skips half a second on every device handover is noticeable.

(7) Which device wrote this, which section 4 needs.

The size of that structure is the point. It fits in a few hundred bytes, so it can be written on every pause, every skip and every few seconds of playback without cost, and it can be pushed to your other devices instantly. A stored queue of 800 track identifiers could do none of that.

The user's own edits are a separate list. "Play this next" and a manually reordered queue are not part of the source, so they live in a short override list that is consumed before the computed order resumes. Keeping the two apart is what stops a manual insert from being destroyed the next time the source order is recomputed.

4. Music: three devices, all of them writers

You pause on your phone, and your laptop must stop. Two seconds later you press play on the laptop. Both devices are writing the same small state, and the naive rule — last write wins — produces a real, common bug.

The bug, concretely. Your laptop is playing and sends a position update every five seconds. You pause on your phone. The laptop's next scheduled update arrives a moment later, still saying playing, and playback resumes on its own. Users report this as "it randomly starts playing again", and it is one of the more annoying bugs a music app can have.

The fix is to separate two kinds of write, which are genuinely different things:

A command — play, pause, skip, seek. This is a user intention, it happens once, and it must not be undone by anything that was already in flight.

A heartbeat — "I am still playing and I am now at 01:42". This is a report, it happens constantly, and it is meaningless if it is stale.

function applyWrite(current: PlaybackState, w: Write): PlaybackState {
  if (w.kind === "command")                                       // (1)
    return { ...apply(current, w), version: current.version + 1 };

  if (w.deviceId !== current.updatedByDevice) return current;     // (2)
  if (w.basedOnVersion !== current.version) return current;       // (3)
  return { ...current, positionMs: w.positionMs, updatedAt: w.at };
}

(1) A command always applies and bumps the version. Intentions win over reports, always, and this single line removes the phantom-resume bug.

(2) A heartbeat from a device that is not the one currently playing is dropped. Only the active device gets to report progress, so the laptop's stale report cannot speak for a session the phone now owns.

(3) A heartbeat that was computed before the last command is dropped as well, because it describes a world that no longer exists. Comparing versions rather than timestamps means this works without any assumption that the two devices' clocks agree — the same reason the exchange in 9.7.25 sequences its inputs instead of trusting clocks.

Only one device plays at a time, and that must be enforced rather than assumed. The state names the active device; a device that starts playing takes ownership, and every other device is told and stops. That transfer is a command, so it obeys rule (1) and cannot be reversed by a heartbeat still in flight.

Offline is a real state, not a failure. A phone on a plane keeps playing downloaded tracks and accumulates events with local sequence numbers. When it reconnects, those events are ordered after whatever the server already had — with one exception worth naming: a position from an offline session is not merged, it replaces, because it is the only record of what actually happened. Play counts merge; positions do not. Being explicit about which fields merge and which replace is what makes offline sync tractable instead of a permanent source of odd behaviour.

5. Music: shuffle, done properly

Shuffle is where a small design decision has a visible effect on users, and where an interviewer can tell in one question whether you have thought about it.

The wrong version: pick a random track each time the current one ends.

// this is not shuffle, and users will notice
const next = tracks[Math.floor(Math.random() * tracks.length)];

Three things break. Tracks repeat before others have played at all, which on a 20-track album is very obvious. There is no "previous", because nothing recorded where you have been. And two devices resuming the same session produce different orders, so a handover changes what plays next.

The right version: shuffle the whole order once, from a stored seed.

function resolveOrder(source: TrackId[], order: Order): TrackId[] {
  if (order === "inOrder") return source;
  return seededShuffle(source, order.shuffled.seed);              // (1)
}

function seededShuffle(items: TrackId[], seed: number): TrackId[] {
  const out = [...items];
  const rand = mulberry32(seed);                                  // (2)
  for (let i = out.length - 1; i > 0; i--) {                      // (3)
    const j = Math.floor(rand() * (i + 1));                       // (4)
    [out[i], out[j]] = [out[j], out[i]];
  }
  return out;
}

(1) The order is a function of the source and the seed. Nothing is stored beyond the seed itself.

(2) A seeded generator: the same seed always produces the same sequence of numbers. This is what makes the shuffle reproducible on every device.

(3) and (4) This is the Fisher–Yates shuffle, walking backwards and swapping each item with a randomly chosen one at or before it. It produces every possible ordering with equal probability, which the tempting alternative — sorting by a random key — does not. It runs in one pass with no extra memory.

What the seed buys, all at once. Every device computes the identical order, so a handover is seamless. "Previous" works, because the order exists as a list and the index simply moves backwards. Nothing repeats until everything has played, because it is a permutation. And storing a shuffle of 800 tracks costs one number.

Then the honest part, which is a good thing to volunteer. True random shuffle feels wrong to people. A genuinely random permutation will sometimes put three songs by the same artist together, and users report that as a bug — they believe the shuffle is broken because their intuition about randomness is wrong. Real music services therefore apply a spreading pass afterwards, nudging tracks by the same artist or album apart. It is deliberately less random, and it is what users mean when they say shuffle. Saying that out loud shows you distinguish the correct answer from the right answer.

Repeat interacts with shuffle and needs a stated rule. Repeat-one holds the index still. Repeat-all wraps the index to zero — and the question is whether the shuffle is re-seeded on the wrap. Re-seeding gives a fresh order on each pass, which is almost always what people want; keeping the seed replays the same order forever, which they notice within two passes.

6. Live scores: the ball is the event, everything else is a fold

Now the shared-state half of the family. One person in the ground records what happened; millions read it.

interface BallEvent {
  sequence: bigint;                 // (1)
  matchId: MatchId;
  innings: number;
  over: number;
  ballInOver: number;               // (2)
  strikerId: PlayerId;
  bowlerId: PlayerId;
  runsOffBat: number;
  extras: { kind: ExtraKind; runs: number } | null;   // (3)
  wicket: WicketInfo | null;
  recordedAt: Instant;
}

(1) The sequence is the ordering, and it is what section 2's snapshot-plus-stream is joined on.

(2) Over and ball number are recorded rather than counted, because extras mean the ball count and the delivery count differ. A wide is a delivery that is not a ball, so a system that derives position by counting rows will drift.

(3) Extras carry their kind, because the kind changes several derived numbers at once: a wide adds a run and does not count as a ball, a bye adds runs to the team but not to the batter, and a no-ball adds a run and brings a free hit. Storing "1 extra run" without the kind makes the scorecard uncomputable.

Nothing else is stored. The team total, the wickets, every batter's runs, every bowler's economy rate, the run rate, the required rate — all of it is a fold over these events.

function scorecard(balls: BallEvent[]): Scorecard {
  return balls.reduce(applyBall, emptyScorecard());               // (1)
}

(1) One fold, one function to reason about, and it is the same fold that runs live and that runs during a replay. There is no separate "live" path that can disagree with the recomputed one.

In production the fold is incremental — a maintained running scorecard updated per ball rather than recomputed from scratch — but the full fold is kept and is authoritative. Any doubt is settled by recomputing, which is the same three-condition rule this book puts on every cached aggregate: updated in step, rebuildable, alerted on divergence.

The derived numbers are where a design earns its keep. A batter's strike rate, a bowler's economy, the partnership, the required run rate, the fall-of-wicket list — none are stored, all are computed, and all stay correct automatically when section 7 happens.

7. Live scores: corrections, which are routine

A scorer makes a mistake. The four runs were actually a boundary off the pads, so they were byes, not runs off the bat. This is not rare — it happens several times in a match — and how the design handles it separates a real answer from a diagram.

The wrong answer is to edit the event. Everyone who already received the original still has it, the sequence they have no longer matches what the server holds, and there is no way for a client to discover that something behind it changed.

The right answer is a correction event that points at the ball it corrects:

interface CorrectionEvent {
  sequence: bigint;                 // (1) a NEW number, at the end
  correctsSequence: bigint;         // (2) the ball being corrected
  replacement: BallEvent;           // (3)
  reason: string;
  correctedBy: ScorerId;
}

(1) The correction is appended like everything else, so a client that has kept up simply receives it in order. No history is rewritten, and no client is left holding a version of the log the server no longer has.

(2) It names its target, so the fold knows which ball to replace.

(3) The whole replacement ball, not a patch of changed fields. A patch requires anyone applying it to know the original, which a client that joined later does not.

The fold handles it in one place. Before folding, apply the corrections to the ball list; then fold as normal. Every derived number — the total, the batter's runs, the bowler's economy, the run rate — updates automatically and consistently, because none of them were stored. This is the payoff for section 6's discipline, and it is worth naming as such: the reason nothing is stored is so that a correction never has to be applied in fifteen places.

The client's job is small but must be defined. A correction means the current state has changed in a way that is not a simple append, so the client re-folds from its own snapshot, or fetches a new one. Since corrections are rare compared to balls, refetching is perfectly acceptable and much simpler than incremental un-application.

Two product rules that belong in the design: a correction that changes a significant event — a wicket that was not out — should be visible rather than silent, because millions of people saw the original. And corrections are attributed to a scorer, because a pattern of them is a training issue that somebody has to be able to see.

8. Live scores: getting one event to four million people

The write side is one person. The read side is the entire problem.

Nobody polls the database. Four million clients asking "what is the score" every two seconds is two million reads a second against a row that changes once a minute, and almost every one of those reads returns the same bytes. The scorecard is served from a cache updated by the fold, and updates are pushed to connected clients rather than asked for (10.16).

The fan-out is a tree, not a loop. One engine cannot write four million times. The engine publishes each event once; a layer of gateway servers each hold a share of the connections and each receive that one event and write it to their own connected clients. Adding gateways adds capacity, and the engine's work does not change with audience size — which is the property that matters, since the audience is the thing that varies by a factor of a thousand between a domestic game and a final.

Clients that fall behind must be dropped rather than buffered. A phone on a poor connection cannot keep up, and buffering for it consumes memory on a shared gateway that everyone else is using. The rule is to bound the buffer, disconnect the client when it overflows, and let it reconnect with a fresh snapshot. Slow clients must never be able to degrade fast ones, and this is the standard backpressure decision from 9.5.4.

The snapshot itself is cacheable, and this is the biggest single win. Every one of the four million joins asks for the same snapshot, and it is identical for all of them. Serve it as a static object with a very short lifetime, and the join burst — which is what actually breaks these systems, at the start of play and after a wicket — is absorbed by a cache rather than by the engine.

One honest caveat to state before being asked. Different viewers will be a second or two apart, because the network is not uniform. That is acceptable here in a way it is not on the exchange in 9.7.25: nobody trades on a score. But the ordering per client must be strict — a viewer must never see the wicket before the ball that took it — and the sequence number is what guarantees that, no matter how uneven the delivery is.

9. What the interviewer will push on

"Someone opens the app mid-match. What do they get?" A snapshot stamped throughSequence, then every event after that number. Not "the score and then updates", which loses anything arriving between the two requests. The client drops updates at or below the snapshot's number and treats a gap as a signal to resync — which is what numbering every update is for.

"Do you store the playback queue?" No. Store the source, its version, a shuffle seed and an index. The whole queue is reproducible from a few hundred bytes, which is what makes it cheap to write constantly and push to every device. A stored list of 800 identifiers can do neither.

"Pause on the phone, and it starts playing again by itself." Separate commands from heartbeats. A command is an intention and always wins; a heartbeat is a report and is dropped if it comes from a device that no longer owns playback or was computed before the last command. Compare versions, not timestamps, so nothing depends on two devices' clocks agreeing.

"How does shuffle work?" A seeded Fisher–Yates permutation of the whole order, stored as one number. That gives an identical order on every device, a working "previous", and no repeats until everything has played. Then volunteer the part nobody asks: a truly random shuffle feels broken to users, so real services spread the same artist apart afterwards, deliberately making it less random.

"The scorer made a mistake." A correction event appended at the end, naming the ball it replaces and carrying the whole replacement. Never an edit. Every derived number fixes itself because nothing was stored — which is the actual reason for the fold, and saying that connection is stronger than describing the mechanism.

"Four million viewers." The engine publishes once; gateway servers hold shares of the connections and fan out. Nobody polls. The join snapshot is cached and identical for everyone, which absorbs the burst that actually breaks these systems. Slow clients are disconnected rather than buffered.

The thing to volunteer that nobody asks for: which fields merge and which replace when an offline device reconnects. Play counts merge, because both sides are true. Playback position replaces, because the offline device is the only witness to what really happened. Candidates say "sync when reconnected" and stop; naming the per-field rule is what shows you have shipped one of these rather than drawn one.

Recall

• One axis separates the family: private state with several writers (playback) versus shared state with one writer and millions of readers (a live match).
• The skeleton is the same: an append-only event log, a state that is a fold over it, and snapshot plus stream for late joiners.
• The snapshot carries throughSequence. Without it, an event arriving between the two requests is lost silently.
• Clients drop updates at or below the snapshot number and resync on a gap. That is why every update is numbered.
• The playback queue is computed, from source, source version, shuffle seed and index — a few hundred bytes, so it can be written constantly and pushed everywhere.
• Commands beat heartbeats. An intention always applies; a report is dropped if it is from a non-owning device or was computed before the last command. Compare versions, not clocks.
• Shuffle is a stored seed driving a Fisher–Yates permutation: identical on every device, "previous" works, nothing repeats early. Real services then spread the same artist apart, because true randomness feels broken.
• On reconnect, say which fields merge and which replace: play counts merge, position replaces.
• A ball stores its over and ball number, because extras make deliveries and balls differ.
• Corrections are appended events naming the ball they replace, carrying the whole replacement, never a patch and never an edit.
• Nothing derived is stored, so one correction fixes every number at once.
• Fan-out is a tree: the engine publishes once, gateways hold shares of the connections. The engine's work does not grow with the audience.
• Cache the join snapshot — it is identical for everyone and the join burst is what actually breaks these systems.
• Slow clients are disconnected, not buffered, so they cannot degrade everyone sharing the gateway.

Self-test: What does the snapshot have to state? Why is the queue not a list? What stops a stale heartbeat resuming playback? What does one seed buy you? Why does a correction fix the strike rate for free? Why is the snapshot the thing to cache?

Quiz Bank

FoundationalSomeone opens the app during a match. Show exactly how they get a correct picture, and why the obvious approach loses events.

The obvious approach has a hole in it. Ask for the current score, then subscribe to updates:

const score = await fetchScore();   // ← a ball is bowled at this moment
subscribe(onBall);                  // ← subscription starts here

The ball bowled between those two lines is in neither. It was not in the score, because the score was built before it. It was not delivered by the subscription, because the subscription did not exist yet. The client now shows a score that is four runs light, and — this is the part that matters — nothing will ever tell it so. The next ball arrives and is applied on top of a wrong base. The client is quietly wrong for the rest of the match.

Doing it the other way round does not help either. Subscribing first and then fetching means the client receives updates for events the snapshot may or may not already include, and with no way to tell which, it either double-counts or discards blindly.

The fix is one field.

interface Snapshot<S> { state: S; throughSequence: bigint; takenAt: Instant; }
interface Update<E>  { sequence: bigint; event: E; }

The snapshot states which event it already contains. The client subscribes first, buffers what arrives, then fetches the snapshot, then applies only the buffered updates whose sequence is greater than throughSequence. There is no gap and no double count, and the two halves are stitched by a number rather than by timing.

let have = snap.throughSequence;
stream.on("update", u => {
  if (u.sequence <= have) return;              // covered by the snapshot
  if (u.sequence > have + 1n) return resync(); // a gap — refetch
  apply(u.event);
  have = u.sequence;
});

The first check happens constantly and is not an error. The snapshot was built while the stream was already flowing, so overlap is the normal case. Dropping the duplicates is exactly what the number is for.

The second check is the one worth dwelling on. A gap means an event was lost — a dropped connection, an overflowing buffer, a gateway restart. The client cannot repair a hole in a fold, because every later number depends on it. The only safe response is to throw away the local state and fetch a fresh snapshot. That sounds expensive and is not: gaps are rare, snapshots are small and cached, and the alternative is a client showing a wrong score with total confidence.

Why this shape appears everywhere in the book. A trading client detects a gap in its private feed the same way in 9.7.25, a message consumer tracks an offset the same way in 9.7.19, and a wallet rebuilds a balance from entries in 9.7.10. The recurring idea is that a numbered, append-only log plus a fold gives you correctness you can verify, and being able to say "this is the same mechanism as X" is worth more in an interview than the mechanism itself.

One practical detail to add. The snapshot is identical for every joining client, so it is served from a cache with a short lifetime rather than computed per request. At the start of play, or in the thirty seconds after a wicket, joins arrive in an enormous burst — and that burst, not the steady state, is what takes these systems down.

AppliedDesign the playback state so that three devices stay in sync, and fix the bug where pausing on the phone lets the laptop resume playback.

The state is deliberately tiny, because it is written constantly and pushed to every device.

interface PlaybackState {
  sourceId: SourceId;
  sourceVersion: number;
  order: "inOrder" | { shuffled: { seed: number } };
  index: number;
  trackId: TrackId;
  positionMs: number;
  status: "playing" | "paused";
  repeat: "off" | "all" | "one";
  version: number;
  updatedByDevice: DeviceId;
  updatedAt: Instant;
}

No queue is stored. The source identifier says what is playing, the version says which revision of that playlist, the seed says how it was shuffled, and the index says where you are. Any device can rebuild the exact same order from those four fields. A stored list of 800 track identifiers would be too big to write on every skip and too big to push to three devices continuously, and it would go stale the moment the playlist changed.

Now the bug. The laptop is playing and reports its position every five seconds. You pause on your phone. A moment later the laptop's already-in-flight report arrives saying status: "playing", positionMs: 102000, last-write-wins applies it, and the music starts again. Users describe this as "it randomly resumes", and it is entirely caused by treating two very different writes as one kind.

The two kinds are:

Commands. Play, pause, skip, seek. A user intention, happening once, which must never be undone by something already in flight.

Heartbeats. "Still playing, now at 01:42." A report, happening constantly, worthless if stale.

function applyWrite(current: PlaybackState, w: Write): PlaybackState {
  if (w.kind === "command")
    return { ...apply(current, w), version: current.version + 1 };

  if (w.deviceId !== current.updatedByDevice) return current;
  if (w.basedOnVersion !== current.version)   return current;
  return { ...current, positionMs: w.positionMs, updatedAt: w.at };
}

Commands always apply and bump the version. That single line kills the phantom resume: the phone's pause raised the version, so the laptop's report — computed against the previous version — no longer applies.

Heartbeats are dropped on two independent grounds. From a device that does not currently own playback, because only the active device may report progress. And from a stale version, because the report describes a world that no longer exists.

Versions rather than timestamps, deliberately. Two devices' clocks disagree by seconds routinely and can disagree by minutes. A rule of "the newest timestamp wins" therefore lets a device with a fast clock overwrite a command that genuinely happened later. A version is a single counter that only the server increments, so it cannot be wrong. This is the same reason the exchange sequences its inputs instead of trusting clocks in 9.7.25, and the same reason the auction uses a server-assigned sequence in 9.7.20.

Ownership must be explicit, not implied. The state names the active device. Starting playback on the laptop is a command that transfers ownership; every other device is told and stops. Because it is a command, it obeys the first rule, so an in-flight heartbeat from the previous owner cannot take it back.

Then offline, which is where the per-field rules matter. A phone with no network keeps playing downloads and queues its events locally. On reconnect they are ordered after what the server already had, with one distinction stated explicitly:

Play counts merge. Both sides are true — you played four tracks on the plane and two on your laptop, and the answer is six.

Position replaces. The offline device is the only witness to what actually happened, so its position is not reconciled with anything; it wins.

Naming which fields merge and which replace is the whole of offline sync. "We sync on reconnect" is not a design, and this per-field rule is the thing that makes the behaviour predictable to a user rather than mysterious.

InterviewImplement shuffle. Then explain what changes when the user presses previous, when they hand over to another device, and when the playlist changes underneath them.

The version people write first is not shuffle:

const next = tracks[Math.floor(Math.random() * tracks.length)];

It picks a random track each time. On a twenty-track album that repeats a song before half the album has played, which users notice immediately and report as broken. It has no memory, so "previous" is impossible. And two devices produce different sequences, so a handover changes what plays next.

Shuffle the whole order once, from a stored seed:

function seededShuffle(items: TrackId[], seed: number): TrackId[] {
  const out = [...items];
  const rand = mulberry32(seed);
  for (let i = out.length - 1; i > 0; i--) {
    const j = Math.floor(rand() * (i + 1));
    [out[i], out[j]] = [out[j], out[i]];
  }
  return out;
}

This is Fisher–Yates: walk backwards, and swap each item with one chosen at random from the positions at or before it. Two properties make it the right choice. It produces every possible ordering with equal probability, which the tempting alternative of sorting by a random key does not — that one is biased in ways that depend on the sort implementation. And it runs in a single pass with no extra allocation.

mulberry32 is a seeded generator: given the same seed it emits the same sequence of numbers every time. That is the entire trick. Math.random() cannot be seeded, so it cannot be reproduced, so it cannot give two devices the same order.

Now the three follow-ups.

Previous. The order is a real list and the index simply moves back by one. With the random-pick version there is nothing to move back to, so an implementation would have to store a history of played tracks — extra state that the seed makes unnecessary. This is a good example of a small representation choice removing a feature's cost entirely.

Handover to another device. The other device receives sourceId, sourceVersion, seed and index, resolves the identical order, and continues at the same point. Nothing about the queue crosses the network, which is why the handover is instant even for an 800-track playlist. If the shuffled array were stored instead, the handover would mean transferring the array, and the two devices could still disagree if either had rebuilt it.

The playlist changes underneath. Someone adds a track to a shared playlist while you are listening. sourceVersion changes, so the device notices its resolved order is stale. The rule has to be stated, and the humane one is: rebuild the order from the new source and the same seed, then re-find the current track by identifier and set the index to its new position. That keeps what is playing exactly where it is while the rest of the order absorbs the change. The alternative — keeping the index — would jump the listener to a different song, which is a very visible bug for a very small saving.

A related rule that must be decided rather than left to emerge: repeat-all wraps the index to zero, and the question is whether the seed is regenerated on the wrap. Re-seeding gives a fresh order on the second pass, which is what people expect. Keeping the seed replays the identical order forever, and listeners notice by the third pass.

And the honest ending, which is worth volunteering. A mathematically correct shuffle feels wrong. A uniform permutation will regularly place three tracks by the same artist together, and users report that as a broken shuffle, because human intuition about randomness expects things to be more evenly spread than random actually is. So real music services run a spreading pass afterwards that pushes same-artist and same-album tracks apart. It is deliberately less random, and it is what people mean by the word. Distinguishing the correct answer from the right answer is the point of saying it.

StaffA wicket falls with four million people watching, and thirty seconds later the third umpire reverses it. Walk the whole path, from the scorer's tablet to every viewer's screen.

Start with the write, because it is one person. The scorer records a ball event; it is appended to the log with the next sequence number; the incremental fold updates the running scorecard. Total work: one append and one small update. Audience size does not appear in this path at all, and that is the property the whole design is protecting.

Then the fan-out, which is a tree rather than a loop. The engine publishes the event once. A layer of gateway servers each hold a share of the four million connections and each receive that single publish, then write it to their own connected clients. Adding gateways adds capacity; the engine's cost does not move. A design where the engine writes to four million connections has an engine whose work grows with popularity, which is the one thing you cannot afford in a system whose audience varies by a factor of a thousand between an ordinary match and a final.

Nobody polls. Four million clients asking every two seconds is two million reads a second, almost all returning identical bytes, against state that changes once a minute. Updates are pushed over open connections, and the reads that remain are served from a cache.

Now the burst, which is what actually breaks these systems. A wicket is the moment everyone opens the app. Tens of thousands of joins arrive in a few seconds, and each one wants a snapshot. The snapshot is identical for all of them, so it is served from a cache with a very short lifetime, and the burst is absorbed by cache reads rather than by folds. This is the single highest-value optimisation in the design, and it is aimed precisely at the event that creates the load.

Slow clients are disconnected, not buffered. A phone in a tunnel cannot keep up. Buffering for it consumes memory on a gateway shared by tens of thousands of other people, so the rule is a bounded buffer, disconnection on overflow, and reconnection with a fresh snapshot. One client must never be able to degrade the rest, which is the standard backpressure decision from 9.5.4.

Then the reversal, thirty seconds later. Four million people have already seen "OUT". The wrong response is to edit the ball event: every client already holds the original, their sequence no longer matches the server's, and nothing tells them the past changed.

interface CorrectionEvent {
  sequence: bigint;          // a NEW number, appended at the end
  correctsSequence: bigint;  // the ball being replaced
  replacement: BallEvent;    // the whole ball, not a patch
  reason: string;
  correctedBy: ScorerId;
}

It is appended, so it travels the same path as everything else — one publish, gateway fan-out, delivered in order. A client that has kept up simply receives it. Nothing is rewritten, and no client ends up holding a version of history the server has discarded.

It carries the whole replacement rather than a patch, because a client that joined twenty seconds ago has no idea what the original said, and a patch would be unapplicable to it.

Every derived number fixes itself. The team total, the batter's score, the bowler's wicket count, the strike rate, the required run rate, the fall-of-wicket list — none of them were stored, all of them are folds. Applying corrections to the ball list and re-folding produces a fully consistent scorecard with no per-field repair logic anywhere. This is the actual reason nothing derived is stored, and it is the strongest single point to make in this answer: the discipline in section 6 was paid for by exactly this moment.

The client's response is defined and slightly blunt. A correction is not a simple append to a fold, so the client re-folds from its snapshot or fetches a new one. Refetching is fine because corrections are rare and snapshots are cached, and it is far simpler than incremental un-application — which is a genuine trade worth naming rather than engineering around.

Two product decisions that belong in the design, not the code. A reversal of something as significant as a wicket must be shown as a reversal rather than silently swapped, because four million people saw the original and a value that changes with no explanation reads as a bug. And the correction records who made it, because a pattern of corrections from one scorer is something a human needs to be able to see.

The honest caveat, stated before being asked. Viewers will be one or two seconds apart, because networks are not uniform, and that is acceptable here in a way it would not be on an exchange — nobody trades on a cricket score. But per client the order is strict: nobody sees the wicket before the ball that took it, and nobody sees the reversal before the wicket. The sequence number guarantees that regardless of how uneven delivery is, which is why the ordering promise is per client rather than global.

What I would monitor. The age of the newest event each gateway has delivered, which is the real measure of how far behind viewers are; the rate of client resyncs, since a rise means gaps are happening and something upstream is dropping events; the snapshot cache hit rate during a burst, because a fall in it is the leading indicator of an outage rather than a symptom; and the count of connections disconnected for slowness, which is normal at a low rate and a network problem at a high one.

Flashcards

FlashThe axis that splits the family

Private state with many writers (your playback across three devices) versus shared state with one writer and millions of readers (a live match). The first is a conflict problem; the second is a fan-out problem.

FlashSnapshot plus stream

The snapshot states throughSequence — the last event it contains — and the stream starts after it. "Give me the score, then updates" loses anything arriving between the two calls, and nothing ever tells the client.

FlashWhy the queue is not a list

Store source, source version, shuffle seed and index. A few hundred bytes reproduce an 800-track order on any device, so it is cheap to write on every skip and to push everywhere. A stored list is neither.

FlashCommands beat heartbeats

A command is an intention and always applies, bumping the version. A heartbeat is a report and is dropped if it comes from a non-owning device or a stale version. Compare versions, never clocks — devices disagree by seconds.

FlashOne seed, three features

A seeded Fisher–Yates permutation gives an identical order on every device, a working "previous", and no early repeats — for the storage cost of one number. Then real services spread same-artist tracks apart, because true randomness feels broken to people.

FlashCorrections are appends

A correction event names the ball it replaces and carries the whole replacement, never a patch and never an edit. Every derived number fixes itself, which is the actual reason nothing derived is stored.

Next: 9.7.27 — the calendar, where one stored row can mean nine hundred meetings and one of them was moved.