9.7.31 — Logging Framework

"Design a logging library: levels, multiple destinations, per-module context, and it must not slow the application down."

A service starts returning slow responses. Nothing changed in the business logic, the database is fine, and CPU is at 90% with no obvious culprit. Somebody eventually notices that a new release added detailed logging to the request path, about fifty lines per request, each one written straight to a file.

Fifty writes at roughly 0.2 milliseconds each is ten milliseconds of blocking work added to every request, and in a single-threaded runtime that is ten milliseconds nothing else can use. At five hundred requests a second, the service would need five seconds of blocking write time per second of wall-clock time, which it does not have, so requests queue and latency climbs until something times out.

The logging was added to explain the service. It became the thing that needed explaining.

The rule this design has to hold: the diagnostic system must be more reliable than the system it diagnoses. If logging can slow the service, fill its memory, or fail with it, then the one tool you have during an incident is part of the incident.

1. The hot path does three things and none of them is I/O

Figure 1 — Two paths that never touch. The caller's path ends at a bounded buffer and returns. Everything expensive — formatting for each destination, batching, actually writing — happens on the flusher's path, where nobody is waiting.

type Level = "debug" | "info" | "warn" | "error" | "fatal";

const SEVERITY: Record<Level, number> = {                      // (1)
  debug: 10, info: 20, warn: 30, error: 40, fatal: 50,
};

class Logger {
  constructor(
    private readonly minLevel: Level,
    private readonly pipeline: LogPipeline,                    // (2)
    private readonly context: Record<string, unknown> = {},    // (3)
  ) {}

  info(msg: string, fields?: Fields)  { this.#log("info", msg, fields); }
  warn(msg: string, fields?: Fields)  { this.#log("warn", msg, fields); }
  error(msg: string, fields?: Fields) { this.#log("error", msg, fields); }

  #log(level: Level, msg: string, fields?: Fields): void {
    if (SEVERITY[level] < SEVERITY[this.minLevel]) return;     // (4)

    this.pipeline.submit({                                     // (5)
      ts: Date.now(),
      level,
      msg,
      ...this.context,
      ...fields,
    });
  }
}

(1) Levels as numbers, so comparing them is an integer comparison rather than a lookup in a list. This matters because of (4).

(2) The pipeline is passed in rather than reached for. Section 5 is about why that one decision changes what the whole library can do.

(3) Context is data this logger always attaches: which module it belongs to, which request it is serving.

(4) The level check is the first line and it costs one integer comparison. That is what makes it acceptable to leave debug logging in production code permanently. A log.debug(...) call in a hot loop with the level set to info costs a comparison and a return, which is nothing.

But notice what it does not save. By the time #log runs, the caller has already built the fields object and already turned whatever they were logging into a string. If somebody writes log.debug("state", { snapshot: JSON.stringify(bigObject) }), that stringify runs on every iteration regardless of level, because arguments are evaluated before the call. The gate protects the pipeline; it cannot protect the caller from work they did before calling.

Two ways out, and both should be mentioned. Expose isEnabled(level) so an expensive block can be skipped entirely, or accept a function for the fields so construction happens only if the record survives the gate. Neither is needed for ordinary logging, and both matter in a loop that runs a hundred thousand times.

(5) The context is spread first and the call's own fields second, so a call can override its logger's context. That ordering is a decision: the alternative protects context from being shadowed, which sounds safer and makes it impossible to correct a wrong value at a call site. Overriding is the more useful default, and the fix for the risk is section 7's reserved names.

What is not in this method is the point of it. No formatting for a particular destination, no serialising to JSON, no file handle, no network. The caller does a comparison, builds a small object, pushes it onto an array, and returns.

2. Per-module loggers that cost almost nothing

A log line saying connection failed is nearly useless. The same line saying which module, which request, and which customer is the difference between a two-minute investigation and an afternoon.

child(extra: Record<string, unknown>): Logger {                // (1)
  return new Logger(this.minLevel, this.pipeline, { ...this.context, ...extra });
}

(1) A child logger shares the pipeline and copies the context. That split is the whole design of this method. There is one buffer, one flusher and one set of destinations in the whole process, no matter how many loggers exist. What multiplies is a small object holding a few key-value pairs.

const log = rootLogger.child({ module: "checkout" });          // (1)

app.use((req, res, next) => {
  req.log = log.child({ requestId: req.id, userId: req.user?.id });  // (2)
  next();
});

req.log.info("payment authorised", { amountPaise: 249900 });   // (3)

(1) One logger per module, created once at startup.

(2) One logger per request, created per request. This is a cheap object allocation, and it is the reason every line from this request carries the request identifier without anyone remembering to add it.

(3) The call site says only what is new. Module, request and user are already attached.

The alternative that people reach for and should not, is passing the request identifier as a parameter to every function that might log. It works for two levels and collapses at four, because the identifier has to be threaded through functions that do not care about it, and one function that forgets breaks the chain for everything below it. A child logger carries the context down the call path as a value rather than as a discipline.

In Node there is a mechanism for carrying that context without passing the logger either, which is asynchronous local storage, covered in 9.9.4. It solves a real problem and it is worth being clear that the child logger is the structure and the storage is only a way to reach it without a parameter.

3. The buffer must be bounded, and what to drop is a decision

class LogPipeline {
  #buffer: LogRecord[] = [];
  #dropped = 0;                                                // (1)

  constructor(
    private sinks: Sink[],
    private capacity: number,
    private flushMs: number,
  ) {
    setInterval(() => void this.#flush(), flushMs).unref();     // (2)
  }

  submit(r: LogRecord): void {
    if (this.#buffer.length >= this.capacity) {                 // (3)
      this.#dropped++;
      const victim = this.#buffer.findIndex(x => x.level === "debug"); // (4)
      if (victim >= 0) this.#buffer.splice(victim, 1);
      else this.#buffer.shift();                                // (5)
    }
    this.#buffer.push(r);
    if (this.#buffer.length >= this.capacity * 0.8) void this.#flush();  // (6)
  }

  async #flush(): Promise<void> {
    if (this.#buffer.length === 0) return;
    const batch = this.#buffer.splice(0);                       // (7)
    await Promise.allSettled(this.sinks.map(s => s.write(batch)));  // (8)
  }
}

(1) A counter of everything thrown away. Lost logs you know about are a completely different situation from lost logs you do not, and this single number is what makes the difference. Exposed as a metric, it tells you during an incident that the picture you are looking at is incomplete.

(2) A periodic flush so that a quiet service still gets its logs out rather than holding them until the buffer fills. unref so this timer alone does not keep the process alive.

(3) The buffer has a hard limit, and this is the requirement that separates a real logger from a toy. Consider what happens without one. A destination becomes slow, which usually means an incident, which is also when the service is logging most. Records arrive faster than they leave, the array grows, and the process runs out of memory. The logging killed the service during the incident it was meant to explain.

(4) and (5) When it is full, something has to go, and choosing what is the interesting part.

Block the caller until there is room. This makes logging synchronous again through the back door, and it does so exactly when the system is struggling. It is the worst of the options for a service.

Drop the newest. Simple, and it discards the records describing the situation right now, which are the ones you need.

Drop the oldest. Better, and it still throws away the beginning of the incident, which is usually the most valuable part.

Drop by severity first. Discard a debug record before an error one, and only fall back to dropping the oldest when there is nothing low-priority left. This is what (4) does, and it is the right policy because during an incident the buffer fills with routine chatter while the few lines that matter are the errors.

The scan in (4) is linear over the buffer, which is fine because it only runs when the buffer is already full, and a full buffer is a bad situation where a slightly expensive scan is the least of the problems. Where that is not acceptable, one buffer per severity band gives the same policy with no scanning.

(6) Filling up triggers a flush early rather than waiting for the timer, so a burst is drained instead of dropped.

(7) splice(0) takes everything and leaves the buffer empty in one step, so records arriving during the flush go into a fresh buffer rather than into the batch being written.

(8) allSettled and not all. With all, one failing destination rejects the whole thing and the other destinations may never be attempted. With allSettled, a dead network destination and a working file destination are independent, so the file keeps receiving everything. One broken destination must not silence the others, and that is a one-word difference.

4. Destinations, and what belongs to whom

interface Sink {
  write(records: LogRecord[]): Promise<void>;                  // (1)
}

class StdoutSink implements Sink {
  async write(records: LogRecord[]): Promise<void> {
    const text = records.map(r => JSON.stringify(r)).join("\n") + "\n";  // (2)
    process.stdout.write(text);
  }
}

(1) The interface takes an array, not one record. Every destination has a per-operation cost — a write call, a network request, a lock — and paying it once for two hundred records rather than two hundred times is most of the benefit of having a pipeline at all.

(2) Formatting belongs to the destination, not to the logger. A log shipper wants one JSON object per line; a developer's terminal wants colours and aligned columns; an error tracker wants its own payload shape. All three receive the same records and each turns them into what it needs. If the logger formatted, every destination would receive a string already shaped for somebody else and would have to parse it back.

Structured records rather than formatted strings, and it is worth being concrete about why. Compare:

[2026-07-31 14:02:11] ERROR checkout: payment failed for user 8812 after 3 retries
{"ts":1785412931000,"level":"error","module":"checkout","msg":"payment failed",
 "userId":8812,"retries":3}

The first can only be searched with text patterns, and the pattern breaks the moment somebody rewords the message. The second can be filtered by user, counted by retry count, and grouped by module without anybody having written a parser. The message becomes a label and the data becomes fields, and that is what makes a log searchable at any scale beyond one server.

Which destinations to configure, and one to argue against. Standard output is the right default in a container, because the platform already collects it and the application never touches the network to produce a log line. A file is right where there is no platform collector. An in-memory ring of the last few thousand records, exposed on a protected endpoint, is worth far more than it costs during an incident, because it survives when the shipping path is backed up.

The one to push back on is a destination that makes an HTTP request from inside the application. It puts a network dependency into the diagnostic path, so when the log service is slow the buffer fills and records are dropped, and when it is down there is a connection error to handle inside the thing that handles errors. Writing to standard output and letting a separate collector process ship it keeps the failure outside the process. If a network destination is genuinely required, it belongs on a separate thread with its own queue, which is what Pino's transports do.

5. One pipeline, injected, and the trap that makes that hard

The tempting shape is a global:

Logger.getInstance().info("order placed");                     

It is pleasant to use, because logging is needed everywhere and threading a logger through every constructor is tedious. And it costs four specific things.

Configuration is frozen at first use. The first call creates the logger with whatever configuration exists at that moment. If something logs during module loading, before configuration is read, the whole process runs at the default level with the default destinations, and changing the setting afterwards does nothing. This failure is invisible: everything works, and the levels are simply wrong.

Nothing can be swapped. Anything that wants to observe what a component logged, or route one module's output somewhere else, has to reach into a global and put it back.

There is one per process, which is not one per service. Eight worker processes have eight independent pipelines, eight buffers, and eight drop counters. Nothing about the global spelling makes that visible, so people reason about "the logger" as though there were one.

Nobody can tell what depends on it. A class that calls a global logger has a dependency that appears in no signature, so it can be moved, reused or run in a different context and silently keep writing to something that no longer makes sense.

The alternative is ordinary and costs one line at startup:

const pipeline = new LogPipeline([new StdoutSink()], 10_000, 200);   // (1)
const rootLogger = new Logger("info", pipeline);                     // (2)

const checkout = new CheckoutService(
  paymentGateway,
  rootLogger.child({ module: "checkout" }),                          // (3)
);

(1) One pipeline, built where everything else is built, after configuration is available.

(2) One root logger.

(3) Every component receives a child. It feels global to use, because everything has one, and it is a value that was handed over rather than a global that was reached for.

And now the honest part, because "just inject it" is not a complete answer. Some logging genuinely has nowhere to receive a logger: a top-level error handler, a module that runs at import time, a small utility called from everywhere. For those, a module-level logger that is configured once at startup is fine, and the difference from a static singleton is that it is a variable somebody set rather than an instance created on first touch by whoever happened to log first. The rule that survives is: one pipeline for the process, created deliberately at a known moment, and how a particular call site reaches it is a much smaller question than when it was built.

6. fatal, and the one place synchronous writing is right

Everything above exists to keep writing off the caller's path. There is exactly one exception.

When the process is about to die — an unrecoverable error, an uncaught exception, a shutdown signal already being acted on — the buffer is about to die with it. Records sitting in memory waiting for the next flush will never be written, and those are precisely the records explaining why the process is ending.

fatal(msg: string, fields?: Fields): void {
  this.#log("fatal", msg, fields);
  this.pipeline.flushSync();                                   // (1)
}

(1) A blocking write, deliberately. There is nothing to protect any more: no future request will be slowed, because there are no future requests.

Two conditions on this, or it becomes the problem it was meant to avoid. It must be used only where the process really is ending, because a fatal sprinkled into ordinary error handling puts synchronous I/O back on the request path with a respectable name. And it needs a timeout, because a synchronous flush to a destination that is not responding turns a crashing process into a hanging one, and a hanging process is worse than a crashed one — it fails health checks slowly, holds its port, and blocks the restart.

7. Redaction, reserved names, and sampling

Redaction, and the direction that matters. Passwords, tokens, card numbers and personal data end up in logs by accident, almost always because somebody logged a whole object: log.info("request", { body: req.body }).

The instinct is to strip a list of known-bad field names. That list is a deny-list, and it is wrong for the usual reason: it protects you from the fields somebody thought of. The first time a client sends passcode instead of password, or a new endpoint adds ssn, the secret is logged and nobody finds out until an audit.

The right direction is an allow-list: only fields explicitly permitted are logged, and everything else is dropped or replaced. It is more work and it fails safely, which is the correct trade for something whose failure is a security incident. Where a full allow-list is impractical, the compromise is allow-listing at the boundaries where untrusted data enters — request bodies, external responses — and permitting free-form fields elsewhere.

Reserved field names. Once records are structured, the field names are an interface. A module that logs { level: "high" } as a business field collides with the record's own level and breaks every dashboard filtering on it. So a small set of names — timestamp, level, message, module, request identifier, service — is reserved and rejected as user fields. It is a check in one place that prevents a class of confusing breakage.

Sampling, which is the part people only learn during an incident. When something fails for 20% of requests at a thousand requests a second, that is two hundred nearly identical error records a second. They fill the buffer, cause drops, and add nothing after the first few: the tenth copy of the same stack trace teaches nothing the first did not.

The fix is to log an example and count the rest. Keep a small map from an error signature to a count, emit the first occurrence in each window in full, and at the end of the window emit one record saying this error happened 11,431 times. The information is preserved and the volume is not, and it is the difference between a diagnostic channel that works during an incident and one that saturates exactly when needed.

8. What the interviewer will push on

"Why is logging asynchronous?" They want the arithmetic, not the principle. Fifty lines per request at 0.2 milliseconds each is ten milliseconds of blocking work added to every request, and in a single-threaded runtime nothing else runs during it. At five hundred requests a second the service needs five seconds of write time per second and cannot get it. The hot path does a comparison, builds an object, and pushes it onto an array.

"The buffer fills up. What do you do?" This is the real question on this topic. Blocking is worst, because it makes logging synchronous again exactly when the system is struggling. Dropping the newest discards the current situation; dropping the oldest discards the start of the incident. Drop by severity first, so routine chatter goes before errors, and count the drops and expose the count, because logs you know are missing are a different situation from logs you do not. Candidates who say "unbounded, it is only logs" have described the mechanism that turns a slow log destination into an out-of-memory crash.

"Why is Logger.getInstance() a problem?" They are checking whether you can name concrete costs rather than repeat that globals are bad. Configuration frozen at first use, so anything logging during startup locks in the defaults invisibly. Nothing swappable. One per process, which is not one per service, and eight workers have eight buffers. And a dependency that appears in no signature. The complete answer also concedes that a module-level logger configured once at startup is fine, because the real rule is one pipeline created deliberately at a known moment.

"What does child() share and what does it copy?" It shares the pipeline and copies the context. One buffer, one flusher and one set of destinations for the process; a small object per module and per request. The follow-up worth pre-empting is why not pass a request identifier as a parameter: it works for two levels, collapses at four, and one function that forgets breaks the chain for everything below it.

"One of your destinations is down. What happens to the others?" Nothing, because the flush uses allSettled rather than all. With all, one rejection can stop the others being attempted, so a dead network destination silences the file. One word, and it is the difference between losing one destination and losing all of them.

"When is it right to write a log line synchronously?" Exactly once: fatal, when the process is ending and the buffer is about to die with it. There are no future requests to protect. Two conditions come with it, and volunteering them matters: only where the process genuinely ends, and with a timeout, or a crashing process becomes a hanging one that fails health checks slowly and blocks its own restart.

"How do you keep secrets out of logs?" An allow-list, not a deny-list. A deny-list protects against the field names somebody thought of, and fails the first time a client sends passcode instead of password. The failure mode of getting this wrong is a security incident, which is exactly when you want the more conservative direction.

The thing to volunteer that nobody asks for: sampling repetitive errors. An error affecting 20% of a thousand requests a second produces two hundred nearly identical records a second, which fills the buffer, causes drops, and adds nothing after the first few. Emit one exemplar per window plus a count. Almost nobody raises it, and it is the single change that decides whether the diagnostic channel survives the incident it exists for.

Recall

• The rule: the diagnostic system must be more reliable than the system it diagnoses.
• The hot path is a level check, a record, and a push. Fifty synchronous writes per request is ten milliseconds of blocking work per request, which a single-threaded runtime cannot afford.
• The level gate is first and costs one integer comparison, which is what makes leaving debug calls in production fine. It does not save the caller's own work building arguments — use isEnabled or lazy fields in a hot loop.
• child() shares the pipeline and copies the context. One buffer per process, one small object per module and per request.
• Threading a request identifier through every function works for two levels and collapses at four. A child logger carries context as a value, not as a discipline.
• The buffer must be bounded. A slow destination during an incident is exactly when logging spikes, and an unbounded buffer turns that into an out-of-memory crash.
• Overflow policy, ranked: never block the caller; drop by severity first, so chatter goes before errors; fall back to oldest. Always count the drops and expose the count.
• Flush with allSettled, not all, so one dead destination cannot silence the working ones.
• Destinations take a batch, and formatting belongs to the destination, so one set of records serves a shipper, a terminal and an error tracker.
• Structured records, not formatted strings. The message becomes a label and the data becomes fields, which is what makes logs filterable without a parser.
• Do not ship logs over the network from inside the application. Write to standard output and let a separate collector move them, so the log service's outage stays outside the process.
• One pipeline per process, created deliberately after configuration is read. A static singleton freezes configuration at first use, hides its dependents, and is one per process rather than per service.
• fatal flushes synchronously, because the process is ending and there are no future requests to protect. Only there, and with a timeout, or a crash becomes a hang.
• Redact with an allow-list, not a deny-list, because a deny-list only covers the field names somebody thought of.
• Reserve the envelope's field names, or a module logging level: "high" breaks every dashboard.
• Sample repetitive errors: one exemplar per window plus a count. Two hundred identical stack traces a second fill the buffer and teach nothing after the first.

Self-test: Do the arithmetic on synchronous logging. Rank the four overflow policies and say why blocking is worst. What does child() share versus copy? Why allSettled? Name four concrete costs of a static singleton. Where is synchronous writing correct, and what two conditions come with it?

Quiz Bank

FoundationalWalk the path of a single log call and defend every decision on it.

The call:

req.log.info("payment authorised", { amountPaise: 249900 });

Step one — the level gate, first, and one integer comparison.

if (SEVERITY[level] < SEVERITY[this.minLevel]) return;

Levels are numbers rather than strings so this is a comparison rather than a lookup. It is first so that a debug call in a hot loop with the level set to info costs essentially nothing, which is what makes it acceptable to leave debug logging in shipped code permanently.

And the limit of the gate, which should be stated rather than left for the follow-up. By the time this line runs, the caller has already built the fields object and already evaluated anything inside it. log.debug("state", { snapshot: JSON.stringify(big) }) runs that stringify on every iteration whatever the level, because arguments are evaluated before the call. Two ways out: expose isEnabled(level) so a costly block can be skipped, or accept a function for the fields so it is only called if the record survives. Neither matters for ordinary logging and both matter in a loop running a hundred thousand times.

Step two — build the record.

{ ts: Date.now(), level, msg, ...this.context, ...fields }

The logger's context is spread before the call's own fields, so a call site can override an inherited value. That ordering is a choice: protecting context from being shadowed sounds safer and makes it impossible to correct a wrong value where you notice it. The risk it creates is handled separately by reserving the envelope's own field names.

Step three — push and return.

this.pipeline.submit(record);

That is the end of the caller's involvement. No formatting, no JSON, no file handle, no network. The whole point of the design is what does not happen here. Fifty writes per request at 0.2 milliseconds each is ten milliseconds of blocking work added to every request, and in a single-threaded runtime that is ten milliseconds during which nothing else can run. At five hundred requests a second the service would need five seconds of write time per second, so requests queue and latency climbs until something times out. That is a service made slow by the logging installed to explain it.

Step four — the buffer, bounded, with a policy.

if (this.#buffer.length >= this.capacity) {
  this.#dropped++;
  const victim = this.#buffer.findIndex(x => x.level === "debug");
  if (victim >= 0) this.#buffer.splice(victim, 1);
  else this.#buffer.shift();
}

The bound is what stops a slow destination from becoming an out-of-memory crash, and it matters most during an incident, when the destination is slowest and the logging is heaviest. Dropping a debug record before an error one is the right policy because the buffer fills with routine chatter while the few valuable lines are the errors. And every drop is counted, because knowing the picture is incomplete is a completely different situation from not knowing.

Step five — the flusher, on a timer or when the buffer is filling.

const batch = this.#buffer.splice(0);
await Promise.allSettled(this.sinks.map(s => s.write(batch)));

splice(0) empties the buffer in one step, so records arriving during the flush go into a fresh buffer rather than into the batch being written. Destinations take the whole batch, so a per-write cost is paid once for two hundred records rather than two hundred times. And allSettled rather than all means a dead network destination cannot stop the file destination being attempted, which is a one-word difference between losing one destination and losing all of them.

Step six — the destination formats. The shipper writes one JSON object per line, the developer terminal writes colours and columns, the error tracker builds its own payload. All from the same records, because formatting is the destination's concern. A logger that formatted would hand every destination a string shaped for somebody else.

AppliedYour log destination slows down during an incident. Walk through exactly what happens and what your design does about it.

The situation is the worst possible combination and it is also the normal one. A destination slows down — a collector under pressure, a disk that is nearly full, a network path that is congested — and the reason it is under pressure is usually the same incident that is causing the service to log far more than usual. Production of records goes up at the same moment consumption goes down.

Without a bound, here is the sequence. Records accumulate in the array. The array grows to tens of thousands of entries, then hundreds of thousands. Each record holds strings and objects, so memory climbs. Garbage collection runs more often and takes longer, which makes the service slower, which makes requests pile up, which produces more logging. Eventually the process is killed for memory. The logging killed the service during the incident it was installed to explain, and the records describing the failure were in the buffer that died with the process.

So the buffer is bounded, and something has to be thrown away. Four options, worst first.

Block the caller until there is room. This makes logging synchronous again through the back door, and it does so precisely when the system is already struggling. Every request now waits on the log destination. This is the option that turns a logging problem into an outage, and it is what a naive implementation of backpressure produces.

Drop the newest. Easy, and it throws away the records describing what is happening right now, which are the ones being looked for.

Drop the oldest. Better, and it throws away the beginning of the incident, which is usually the most valuable part, because that is where the cause is.

Drop by severity first. Discard a debug record before an info one, and an info before an error, falling back to oldest only when everything left is equally important. This is right because during an incident the buffer fills with routine traffic while the handful of records that matter are the errors.

const victim = this.#buffer.findIndex(x => x.level === "debug");
if (victim >= 0) this.#buffer.splice(victim, 1);
else this.#buffer.shift();

The scan is linear over the buffer and only runs when the buffer is already full, which is a bad situation where a scan is the least of the problems. Where that is unacceptable, one buffer per severity band gives the same policy with no scanning at all.

Whatever is dropped, the count is kept and exposed as a metric. This is small and it is the difference between two very different situations. Looking at logs with a drop count of zero means you are seeing everything. Looking at logs with a drop count of 40,000 means the story has holes and you should not conclude anything from an absence. Silent loss is far worse than known loss, because it produces confident wrong conclusions.

Three things that reduce how often this happens at all.

Sample repetitive errors. An error hitting 20% of a thousand requests a second produces two hundred nearly identical records a second. The tenth copy of a stack trace teaches nothing. Emit the first in each window and a count for the rest: the information survives and the volume does not. This is the single most effective change, because saturation during an incident is almost always one error repeated, not many different ones.

Do not put a network call in the application's destination list. Writing to standard output and letting a separate collector ship it means the slow thing is outside the process, so it fills the operating system's pipe buffer rather than the application's heap.

Keep an in-memory ring of the last few thousand records, on a protected endpoint. It is bounded by construction, it never touches the shipping path, and during an incident it is the fastest way to see what a struggling process was doing.

And one thing to raise deliberately: raise the buffer and lower the level when an incident starts. A single switch that increases capacity and stops accepting debug records fleet-wide costs very little and directly attacks the cause, which is volume. It is worth building before you need it, because the moment you need it is the moment nobody wants to deploy a change.

InterviewWhy not Logger.getInstance()? Give the concrete costs, and say where a module-level logger is acceptable.

Start by conceding why it is tempting, because an answer that just says globals are bad is not an answer. Logging is needed in every layer, and threading a logger through every constructor is genuinely tedious. The pull towards a global is real.

Four concrete costs.

Configuration freezes at first use. The instance is created by whoever logs first. If any module logs while being imported, before configuration has been read, the process runs at the default level with the default destinations for its entire life. Nothing errors. Setting the level to debug in configuration simply has no effect, and the failure is invisible, because everything appears to work and only the levels are wrong.

Nothing can be substituted. Anything that wants to route one module's output elsewhere, or observe what a component recorded, has to reach into a global and put it back afterwards. That is not a testing point; it is a limit on what the library can do at all.

One per process is not one per service. Eight worker processes have eight pipelines, eight buffers and eight drop counters, and eight is where every count actually lives. The global spelling hides this completely, so people reason about "the logger" as if there were one, and are then surprised when a drop count means something different from what they assumed.

The dependency is invisible. A class that calls a global logger has a dependency that appears in no signature. It can be moved, reused, or run in a different context, and it silently keeps writing to something that may no longer make sense there.

What to do instead, and it is one line at startup:

const pipeline = new LogPipeline([new StdoutSink()], 10_000, 200);
const rootLogger = new Logger("info", pipeline);

const checkout = new CheckoutService(gateway, rootLogger.child({ module: "checkout" }));

One pipeline, built where everything else is built, after configuration is available. Every component receives a child logger, which is a small object sharing that one pipeline. It feels global to use because everything has one, and it is a value that was handed over rather than a global reached for.

Now the honest part, because "inject everything" is not a complete answer either. Some places genuinely have nowhere to receive a logger: a top-level uncaught-exception handler, code that runs at import time, a small utility called from a hundred places where threading a logger through would be worse than the problem.

For those, a module-level logger set once at startup is fine:

let logger: Logger = new Logger("info", new LogPipeline([new StdoutSink()], 10_000, 200));
export function setLogger(l: Logger) { logger = l; }
export function getLogger() { return logger; }

The difference from the static singleton is not cosmetic. This is a variable that somebody set at a known moment, after configuration was read. The singleton is an instance created on first touch by whichever code happened to log first, which is a moment nobody chose and nobody can see.

So the rule that actually survives is not "never use a global". It is one pipeline for the process, created deliberately at a known point, after configuration. How a particular call site reaches it is a much smaller question, and getting the second right while getting the first wrong is the failure that matters.

StaffDesign logging for a platform of forty services with a hard requirement that logging can never take a service down. Specify what runs in each process, what all forty agree on, and one thing you refuse to build.

In each process, the design from this page with three things hardened.

The pipeline is bounded, drops by severity, counts what it drops, and exposes that count as a metric. Destinations are batched and independent. And the destination list is deliberately short: standard output only, plus an in-memory ring of the last few thousand records on a protected endpoint.

That second point is most of the "can never take a service down" requirement, and it is worth being explicit about why. If the application never opens a network connection to produce a log line, then a log-collection outage cannot become a request-path failure. Records go to a pipe, the platform's collector reads the pipe, and when the collector is slow the operating system's pipe buffer fills and writes start to block, which the bounded buffer converts into counted drops rather than growing memory. The failure has been moved outside the process, which is the only reliable way to keep it out of the request path.

The ring buffer is cheap and repeatedly valuable. It is bounded by construction, it is on nobody's critical path, and during an incident it shows what a struggling process was doing right now without waiting for anything to be shipped and indexed.

What all forty services agree on: the envelope.

A fixed set of fields on every record — time, level, service, module, request identifier, trace identifier, message — plus free-form fields underneath. Those names are reserved, and a service logging level: "high" as a business field is rejected, because the envelope is an interface and a collision breaks every dashboard filtering on it.

The trace identifier is the one that makes forty services usable together. It is created at the edge if absent, passed along in a header on every internal call, and attached to every child logger. Then a single request across nine services is one filter rather than nine investigations. This is a propagation discipline rather than a product you buy. A tracing system formalises it and Part 10.10 covers that, and the discipline is what makes any of it work.

Live level control. Levels per module, in configuration, reloaded without a deploy, so an on-call engineer can turn payments.* to debug for ten minutes and turn it back. Every change is recorded with who made it, because a level change during an incident is part of the incident's timeline.

An incident switch that does three things at once: raise buffer capacity, stop accepting debug records platform-wide, and snapshot every ring buffer. Log volume is the second wave of most incidents, and having one lever rather than a deployment is what makes it usable at three in the morning.

Sampling as a platform default, not a per-service choice. An error affecting 20% of traffic at a thousand requests a second is two hundred near-identical records a second per service. Across forty services during a shared dependency failure, that is what saturates the collector. One exemplar per window plus a count preserves everything anybody needs and removes almost all the volume. Making it a default rather than an option means it is on in the services that did not think about it, which are the ones that will produce the flood.

Redaction as an allow-list at the boundaries. Request bodies, external responses and anything else arriving from outside are logged field by field from a permitted list. Anywhere else, free-form fields are allowed. A deny-list of known secret names covers only what somebody thought of, and the first passcode or ssn goes straight to the index. The failure here is a security incident, so the conservative direction is the correct trade.

What I refuse to build: a central logging service that applications call.

It is the thing people ask for, and it inverts the requirement completely. Forty services making HTTP requests to a log API means that API's outage breaks every caller's hot path, its slowness becomes everybody's latency, and its capacity planning becomes a shared failure that arrives during exactly the incidents where logs matter most. It also rebuilds, worse, what standard output and a collector already do reliably.

Logs leave a process through a local, non-blocking, droppable channel, and aggregation is the collector's job. That sentence is the architecture, and everything else on this page is an implementation of it.

What I would measure across the platform. Drop counts per service, which should be near zero and each spike of which means somebody is logging too much or a collector is behind. The gap between a record's timestamp and when it appears in the index, because a growing gap is the early warning for the second wave. Services that have stopped producing logs entirely, which is usually a crashed process rather than a quiet one. And the ratio of error records to requests per service, because a change in that ratio is often the first sign of an incident and it is free to compute from data already flowing.

Flashcards

FlashThe arithmetic for asynchronous logging

Fifty lines per request at 0.2 ms each is 10 ms of blocking work added to every request. At 500 requests a second that is five seconds of write time needed per second. The hot path does a comparison, builds an object, and pushes it.

FlashThe overflow policy

Never block the caller. Drop by severity first, so chatter goes before errors, falling back to oldest. Always count the drops and expose the count — logs you know are missing are a different situation from logs you do not.

Flashchild()

Shares the one pipeline, copies the context. One buffer and one flusher per process; a small object per module and per request. It carries the request identifier as a value rather than as a discipline every function must remember.

FlashallSettled, not all

With all, one failing destination can stop the others being attempted, so a dead network destination silences the file. One word, and it is the difference between losing one destination and losing every one.

FlashThe singleton's four costs

Configuration frozen at first use; nothing swappable; one per process, so eight workers have eight buffers; and a dependency that appears in no signature. The rule that survives: one pipeline created deliberately, after configuration.

FlashSampling repetitive errors

An error affecting 20% of a thousand requests a second is two hundred identical records a second, which fills the buffer and teaches nothing after the first. Emit one exemplar per window plus a count.

Next: 9.7.7 — chess, where there is no concurrency and no I/O at all, and the entire difficulty is discovering that the obvious model asks the wrong question.