3.8.6 — Scaling the Process: cluster, Workers & Children

One event loop on one thread uses one core. A 16-core production box running a single Node process wastes ~94% of its compute — and a CPU-heavy task on that one loop blocks every request (3.8.2). Node's answer is three distinct multiplication mechanisms, each escaping a different limit: cluster multiplies processes to use every core for I/O-bound serving; worker_threads adds threads for CPU-bound JavaScript; child_process runs other programs. Interviews love "which and why"; production loves not confusing them. This page builds each mechanism, the communication machinery they share (message passing, and the SharedArrayBuffer+Atomics escape hatch), and the decision framework.

1. cluster: one port, many processes

The problem: an 8-core server, an I/O-bound API, one process ≈ one busy core. The fix — run 8 Node processes and share the port:

import cluster from "cluster";
import { availableParallelism } from "os";

if (cluster.isPrimary) {
  for (let i = 0; i < availableParallelism(); i++) cluster.fork();  // one per core
  cluster.on("exit", (worker) => cluster.fork());   // a crashed worker is REPLACED
} else {
  http.createServer(handler).listen(3000);   // every worker "listens" on 3000 — how?!
}

The port-sharing trick: workers don't really race for the port. On most platforms the primary owns the listening socket and distributes accepted connections round-robin to workers over IPC (on Linux it can instead let workers share the socket and have the kernel wake one per connection — historically prone to uneven distribution, hence round-robin as default). Each worker is a complete process (2.2), with its own memory, its own V8, its own event loop and its own thread pool. So nothing is shared. You must either send each user to the same worker every time, or better, keep shared state outside, in Redis (Part 7.6). On the plus side, a crashing worker loses only its own requests in progress, which is exactly the separation you want.

Honest modern framing: cluster is the mechanism; in production its job is usually held by PM2 (pm2 start app.js -i max — cluster + restarts + reload) or by containers (one process per container, replicas × orchestrator — 2.9/Part 13) — same principle, supervision moved up a layer. Know cluster to explain what those tools do.

2. worker_threads: JavaScript on real threads

The problem cluster can't solve: a single 2-second CPU task (image resize, big parse, crypto) still freezes whichever worker runs it. For CPU-bound JavaScript, Node offers real OS threads (2.3) — with a twist that dodges two decades of threading pain:

// main.js
import { Worker } from "worker_threads";

function heavyTask(data) {
  return new Promise((resolve, reject) => {
    const w = new Worker("./resize-worker.js", { workerData: data });
    w.on("message", resolve);           // results arrive as MESSAGES
    w.on("error", reject);
    w.on("exit", (code) => code !== 0 && reject(new Error(`exit ${code}`)));
  });
}

// resize-worker.js — its own V8 instance, heap, and event loop
import { parentPort, workerData } from "worker_threads";
parentPort.postMessage(resize(workerData));   // CPU burn happens HERE, off the main loop

The twist: workers share almost nothing by default. Each has its own V8 isolate and heap; postMessage transfers data by structured clone (a deep copy — cyclic objects fine, functions not). No shared variables ⇒ no data races by construction — the 2.4 horror show (locks, atomicity bugs) simply can't occur. The costs: each worker uses a lot of memory, because it is a whole V8. Workers are slow to start, so keep them in a pool and reuse them. The piscina library is the standard choice. And big payloads are slow to copy. For that there are two ways out. The first is transferable objects: postMessage(buf, [buf.buffer]) moves an ArrayBuffer instead of copying it, and the sender can no longer use it. The second is true shared memory:

const sab = new SharedArrayBuffer(4);        // ONE memory region, visible to both
const counter = new Int32Array(sab);         // typed view (3.8.3)
// worker:  Atomics.add(counter, 0, 1);      // atomic — no torn reads/lost updates
// main:    Atomics.load(counter, 0);        // and Atomics.wait/notify = futex-style

SharedArrayBuffer + Atomics reintroduce genuine shared memory — and with it, every 2.4 hazard, tamed only by atomic operations and careful protocol design. Use for high-frequency counters, ring buffers, wait/notify signaling; default to message passing everywhere else. (Also note: the libuv pool of 3.8.2 runs C/C++ tasks — it never runs your JavaScript; worker_threads is how JS gets onto threads.)

3. child_process: running other programs

Third problem: the best tool isn't JavaScript — ffmpeg, ImageMagick, git, a Python model. child_process spawns real OS processes (2.2's fork/exec) with four APIs whose differences are exam-grade:

API	Runs	Output	Use when
spawn	program directly	streams (stdout/stderr)	default — long output, large data, streaming (3.8.4)
exec	command via a shell	buffered string (capped)	short trusted commands needing shell features (pipes, globs)
execFile	program directly	buffered string	short output, no shell — safer with arguments
fork	another Node script	+ an IPC message channel	Node↔Node coordination (it's what cluster uses)

There are two traps.

Putting user input into exec is command injection. exec("convert " + filename), with filename = "x; rm -rf /", runs the attacker's command (Chapter 8.5). The fix is execFile("convert", [filename]), which passes arguments straight to the program without a shell reading them.

The buffered APIs have a maxBuffer, about 1 MB by default. A command that prints a lot dies in the middle for no obvious reason. Use spawn and stream the output instead. Children communicate over pipes (2.8) — child.stdout is a Readable you can pipeline — plus fork's message channel for structured Node↔Node talk.

4. Choosing: the decision framework

Figure 1 — The three multipliers. Ask what's scarce: cores for serving → cluster/replicas; loop freedom under CPU work → worker threads; a capability JavaScript lacks → child processes. Past one machine, the same question becomes a message queue.

Ask, in order: (1) Is the loop blocked by JavaScript computation? → worker_threads (pooled). (2) Is one process too little for I/O-bound traffic? → cluster/PM2/replicas. (3) Is the capability outside Node? → child_process (spawn; never exec with user input). They compose: clustered workers each owning a small thread pool; a worker spawning ffmpeg. And the ceiling: when the work outgrows the machine or must survive restarts, the pattern graduates to a queue + worker service (Part 10.8) — same separation, network-sized.

5. The expert lens

Share nothing by default; share memory by explicit exception. Node's whole concurrency story — loop + processes + message-passing threads — is a bet against shared mutable state, and it's the same conclusion the wider industry reached (Erlang processes, Go's "share memory by communicating", actor systems — 3.5/Part 9.5). SharedArrayBuffer exists precisely so the exception is visible and deliberate: when you see Atomics in review, the full 2.4 discipline applies — and its absence around shared memory is a bug report waiting.

Isolation is a feature you're buying, not overhead. Copy-cost complaints about structured clone miss what the copy purchases: a worker crash that can't corrupt the main heap, a cluster worker whose death loses only its own requests, a child process the OS fully reclaims. Threads-with-locks systems pay the reverse price — one bad write anywhere, undefined behavior everywhere. When latency budgets genuinely can't afford copies, escalate deliberately: transferables → SAB — narrowing the shared surface at each step rather than abandoning isolation wholesale.

The mechanisms rank by respawn cost, and that's the real sizing guide. Thread < process < container, in startup and memory — but all three are expensive enough that pooling and reuse is the universal pattern (worker pools, PM2's fixed worker count, replica sets). Creating concurrency units per-request is the anti-pattern at every scale; the design question is always "how many long-lived units, fed by what queue?" — which is 2.3's thread-pool wisdom, and Part 10's, at one machine's scale.

Next: the standard library's long tail — 3.8.7: fs internals, process & signals, the debugging toolchain, and the module zoo.

Recall

• cluster: N processes (own memory/V8/loop each) sharing one port — primary accepts and round-robins connections via IPC; crashes are isolated and workers replaced; state must externalize (Redis). Production spelling: PM2 or container replicas. For I/O-bound, all cores.
• worker_threads: real threads for CPU-bound JS — each an isolated V8; communicate by postMessage structured clone (no shared vars ⇒ no data races); pool them (piscina). Escapes: transferables (zero-copy move) and SharedArrayBuffer + Atomics (true shared memory — full 2.4 discipline returns). libuv's pool never runs JS — workers are how JS reaches threads.
• child_process: spawn (streams — default), exec (shell! injection risk with input; maxBuffer cap), execFile (no shell — safe args), fork (Node child + IPC channel — cluster's substrate). Children talk over pipes; pipeline their stdio.
• Decision: loop blocked by JS compute → workers; I/O-bound beyond one core → cluster/replicas; capability outside Node → children; beyond one machine → queue + worker service (Part 10). Compose freely; pool everything.

Self-test: How do cluster workers share one port, mechanically? Why can't a data race occur between worker threads by default, and what reintroduces the risk? spawn vs exec vs execFile vs fork — one line each, plus the injection trap. Which mechanism for: a 2 s image resize; an 8-core API server; calling ffmpeg? Why pool workers?

Quiz Bank

FoundationalHow does cluster let multiple processes serve one port, and what constraints follow from workers being processes?

cluster.fork() creates full child processes running your script. The port trick: workers' listen(3000) doesn't bind separately — the primary owns the listening socket, accepts connections, and distributes them round-robin to workers over IPC (the default policy; the alternative shared-socket mode lets the kernel wake a worker directly but historically balanced poorly). Because the workers are separate processes (2.2), three things follow:

• They share no memory. Caches, sessions and rate-limit counters kept in a module quietly become one per worker. That causes the classic "login only works every third request" bug. So state moves out to Redis or a database, or each user is always sent to the same worker.
• Memory use is multiplied by the number of workers.
• A crash stays contained. When a worker dies, only its requests in progress are lost, and the primary starts a new one.

In production the same architecture wears PM2 (-i max) or container-replica clothes — cluster is the mechanism those tools operationalize.

FoundationalWhy are there no data races between worker threads by default, and what do transferables and SharedArrayBuffer change?

Because nothing is shared: each worker owns a separate V8 isolate and heap, and postMessage communicates by structured clone — a deep copy. A data race requires two threads touching the same memory (2.4); with copies, there is no same memory — the hazard is eliminated by construction, not by locking discipline. Transferables (postMessage(data, [arrayBuffer])) avoid copy cost by moving ownership — zero-copy, and the sender's reference is neutered, so exclusivity is preserved (still no race). SharedArrayBuffer changes the game: one real memory region mapped into both isolates, viewed through TypedArrays — races, torn updates, and visibility problems all return, which is why Atomics exists (atomic read/modify/write, plus wait/notify futex-style blocking — 2.4's primitives, JavaScript edition). Escalation ladder: messages → transferables → SAB+Atomics, widening the shared surface only as measured need demands.

AppliedCompare spawn, exec, execFile, and fork — including the two classic traps.

• spawn(cmd, args) runs the program directly, and gives you its input and output as streams. It is the default, and the right choice when output is long or large, because you can pipe it (3.8.4).
• exec(cmdString) runs the command through a shell and collects all the output into one string. You get shell features such as pipes and wildcards, but also two traps. If any user input reaches the string, you have command injection: "convert " + filename with "x; rm -rf /" (Chapter 8.5). Never do this. And maxBuffer, about 1 MB by default, kills commands that print a lot.
• execFile(file, args) runs the program directly, with no shell, and collects the output. It is the safe choice for short commands with arguments, because the arguments are passed as an array and no shell ever reads them.
• fork(script) is spawn made for Node child processes. It adds a message channel between parent and child (child.send and process.on("message")). Cluster is built on it.

Rules of thumb: use spawn by default. Use execFile for short, trusted commands. Use exec only for fixed command strings with no outside input at all. Use fork when two Node processes need to talk.

InterviewFor each workload, pick the mechanism and defend it: (a) API server on 16 cores; (b) per-request PDF rendering taking ~1.5 s of CPU; (c) transcoding uploads with ffmpeg; (d) a shared live counter read by all of them.

(a) cluster/replicas — I/O-bound serving scales by processes across cores; 16 workers (PM2 -i max or container replicas), state externalized. Worker threads would add nothing: the loop isn't compute-blocked, and threads share nothing that helps serving. (b) worker_threads, pooled — 1.5 s of JS/native compute on the loop caps that process at ~0.7 req/s (3.8.2); a pool of workers sized ≈ cores handles rendering while loops keep serving; pool (piscina), don't spawn per request (startup + memory cost). Cluster alone contains the damage per worker but still freezes whichever worker renders. (c) child_process.spawn — ffmpeg is an external binary; spawn it, stream stdin/stdout through pipeline, cap concurrency, and never exec with user-influenced filenames (injection). (d) SharedArrayBuffer + Atomics within one process's workers (Atomics.add/load — no lost updates); but across cluster processes no memory is shared — the honest answer is external state (Redis INCR — Part 7.6) or per-worker counts aggregated by the primary. Distinguishing (b) from (a) and knowing (d)'s process boundary is precisely what the question probes.

StaffA team moved image processing to worker_threads but p99 got worse: workers are spawned per request, large images are postMessage'd both ways, and under load the box swaps. Fix the architecture and state the principles.

Three self-inflicted costs. Per-request spawning: each Worker boots a V8 isolate — tens of ms and real memory; under load, spawn/teardown churn dominates and concurrent workers are unbounded, which is worse than the original problem (unbounded threads vs one blocked loop — 2.3). Fix: a fixed pool sized ≈ availableParallelism() (piscina), fed by a bounded queue with backpressure to callers (429/queue-position beyond a limit — visible policy, 3.8.4's lesson at request scale).

Large images copied twice. postMessage(buffer) makes a full copy of megabytes on the way in, and the result is copied again on the way back. That doubles the CPU work and the peak memory. The fix is transferables: postMessage(buf, [buf.buffer]) moves the buffer instead of copying it. That fits here, because the sender is finished with the original. Even better, do not send pixels at all. Send file paths. Let the workers stream from disk (3.8.4), write the results to disk or S3, and send back only the details about the result.

Swapping: unbounded workers × per-worker V8 heaps × in-flight image copies exceeded RAM (2.5); the pool bound + path-passing collapses the footprint to pool-size × working set. Verify with loop lag (should stay flat), pool queue depth, and RSS. Principles for the write-up: pool long-lived concurrency units, never per-request; move references (paths, transferables), not payloads; bound every queue and make overflow a visible policy. And flag the graduation path: if demand keeps growing, this pool becomes a queue-fed worker service (Part 10.8) — same shape, machine-independent.

Flashcards

Flashworker_threads vs cluster vs child_process

workers: CPU-bound JS in-process (pooled). cluster: N processes, all cores, I/O-bound serving. children: external programs (spawn/stream).

FlashCluster port sharing

Primary owns the socket, accepts, round-robins connections to workers over IPC. Workers = full processes: no shared state, isolated crashes.

FlashWorker communication ladder

postMessage structured clone (copy, race-free) → transferables (zero-copy move) → SharedArrayBuffer + Atomics (true sharing — 2.4 discipline returns).

Flashspawn/exec/execFile/fork

spawn: direct, streams (default). exec: shell, buffered, injection + maxBuffer traps. execFile: direct, buffered, safe args. fork: Node child + IPC.

FlashThe universal sizing rule

Pool long-lived units ≈ cores, feed from a bounded queue, backpressure overflow. Never spawn per request — thread, process, or container.

Scenario Drill

DrillDesign the execution architecture for a document service: REST API (I/O-bound), on-upload virus scan via clamscan (external binary), OCR in a native Node library (~3 s CPU per doc), and a live 'docs processed today' badge on the dashboard. One 8-core VM. Specify every mechanism, sizing, and communication path.

Map each workload to its multiplier. API tier: the REST surface is I/O-bound → cluster via PM2 -i 6 (reserve ~2 cores' headroom for the OCR pool and scans; exact split tuned by measurement) — six processes, connections round-robined, session/state in Redis since worker memory is per-process.

Virus scan: clamscan is an external binary → spawn (never exec — filenames are user-influenced; injection), streaming the file in and parsing verdict output; bound concurrent scans (say 4) with a queue — scans are I/O+CPU mixed and unbounded children would stampede the box.

OCR: 3 s of in-process compute → worker_threads pool, and here's the subtlety on a shared VM: the pool must be sized against the whole box's budget, not per cluster worker — 6 API processes each spawning 4 OCR threads = 24 CPU threads on 8 cores (contention meltdown). Correct shape:

one separate OCR process, which PM2 runs next to the API. It owns a pool of about 4 to 6 workers, and it is fed by a queue, either a Redis list or, at the simplest, messages from the API workers over fork. The API processes add {docId, path} to the queue and wait for a "done" event. Pixels never cross between processes. Only paths do, and the workers stream the images from disk (3.8.4). This also pre-builds the graduation path: the OCR process is already a queue-fed worker service; moving it to another machine is a connection-string change (Part 10.8).

The badge count. It has to add up counts from 6 API processes plus the OCR process. Separate processes share no memory, so SharedArrayBuffer is the wrong tool. The honest tool is Redis INCR. Or the OCR service can be the only writer, and publish the count through pub/sub. Keep SharedArrayBuffer with Atomics for workers inside one process, in case the OCR workers ever need a fast shared progress counter.

Failure/ops wiring: every child and worker gets error listeners (3.8.5), scans/OCR carry AbortSignals tied to job cancellation, PM2 restarts crashed members, and dashboards watch loop lag per API worker, OCR queue depth, and scan-child count. The architecture in one sentence: processes for cores, one pooled worker service for CPU, spawned children for foreign binaries, Redis for anything two processes must both see.