5.8 — Request/Response, Server Push & Webhooks

A client asks, a server answers. That covers most of what software does, and HTTP is built for it.

Then someone asks for a chat message to appear without a refresh, or a stock price to update live, or a payment provider to tell you when a card actually cleared — and the request/response shape stops fitting, because in all three cases the party with the news is the server, and HTTP gives the server no way to speak first.

This page is the family of answers, chosen by the shape of the problem.

1. The two questions that pick the protocol

Before any technology, answer these:

Who initiates? If the client always knows when it wants something, request/response is correct and everything else is overhead. If the server has news the client cannot predict, you need push.

Is it one-off or continuous? A single result when a job finishes is a different problem from a hundred updates a second.

Everything below follows from those two answers.

2. Request/response: REST, RPC, GraphQL

REST is HTTP used as intended: resources at URLs, methods with the semantics from Chapter 5.6.1, status codes carrying meaning, cacheable by default because GET is safe. Chapter 9.6.1 covers the design rules.

Its strength is that it needs nothing beyond HTTP — every proxy, cache, CDN and browser already understands it. Its weaknesses are over-fetching (the endpoint returns 40 fields, you need 3) and under-fetching (you need three endpoints to render one screen, so three round trips).

gRPC is remote procedure calls over HTTP/2, with messages defined in a schema:

service Orders {
  rpc GetOrder (GetOrderRequest) returns (Order);          // (1)
  rpc WatchOrders (WatchRequest) returns (stream Order);   // (2)
}
message Order { string id = 1; int32 quantity = 2; }       // (3)

1. A method with typed request and response, from which client and server code is generated in a dozen languages.
2. stream makes the response a sequence — server streaming, and gRPC also supports client streaming and bidirectional streaming, all over HTTP/2's multiplexed streams (Chapter 5.6.1).
3. The numbers are field tags, and they are the schema's contract. Protobuf encodes the tag, not the name, so you may rename a field freely and must never reuse a number. Adding a new field with a new number is backwards compatible; changing 2 from int32 to string silently corrupts old clients.

gRPC's binary encoding is far more compact than JSON, and the generated code removes a whole class of client-server mismatch. Its cost is that browsers cannot speak it natively — they cannot control HTTP/2 frames from JavaScript — so browser clients need gRPC-Web plus a proxy. gRPC is the right default for service-to-service traffic and usually the wrong one for public browser-facing APIs.

GraphQL lets the client specify exactly the fields it wants in one request, which solves over- and under-fetching directly. The costs are real and worth naming: caching becomes your problem, because everything is a POST to one URL so HTTP caching and CDNs no longer apply; an arbitrarily nested query can be arbitrarily expensive, so you need query depth and complexity limits or you have shipped a denial-of-service endpoint; and the N+1 problem appears immediately, needing per-request batching (the DataLoader pattern) to avoid one database query per returned item.

The honest decision rule. REST for public APIs and anything a CDN should cache. gRPC between your own services, especially where the schema and code generation pay off. GraphQL when many different clients need different shapes of the same data — which is a real problem for a company with a web app, two mobile apps and a partner integration, and is not a real problem for one service with one consumer.

3. The server-push ladder

Four techniques, in increasing order of capability. The right answer is usually further down this list than people reach for.

Short polling

Ask every N seconds.

setInterval(() => fetch('/api/messages').then(render), 5000);

Simple, works everywhere, no special infrastructure. And wasteful: with 10,000 clients polling every 5 seconds, the server handles 2,000 requests a second, and almost all of them return "nothing new" — full HTTP request, headers, cookies, TLS records, for an empty answer. Latency is also up to N seconds by construction.

Correct when updates are rare, latency of tens of seconds is acceptable, and simplicity matters more than efficiency.

Long polling

Ask, and the server holds the request open until it has something to say, or until a timeout. The client immediately asks again.

Latency drops to near zero and the empty responses disappear. The costs: each waiting client holds a connection and server-side state, and every message still costs a full HTTP round trip to re-establish. It is a genuine improvement over short polling and it was how chat worked for a decade.

Server-Sent Events

SSE is a one-way stream from server to client, over ordinary HTTP:

// Client — a browser built-in, no library
const stream = new EventSource('/api/events');      // (1)
stream.onmessage = (e) => render(JSON.parse(e.data));
stream.addEventListener('price', (e) => updatePrice(e.data));   // (2)

// Server — Express
app.get('/api/events', (req, res) => {
  res.set({
    'Content-Type': 'text/event-stream',            // (3)
    'Cache-Control': 'no-cache',
    'Connection': 'keep-alive',
  });
  const send = (event: string, data: unknown, id: string) =>
    res.write(`id: ${id}\nevent: ${event}\ndata: ${JSON.stringify(data)}\n\n`);  // (4)

  const timer = setInterval(() => res.write(': keepalive\n\n'), 20000);          // (5)
  req.on('close', () => clearInterval(timer));                                   // (6)
});

1. EventSource is native in every browser. It reconnects automatically on disconnect, with backoff, and that is the feature that matters most.
2. Named events let one stream carry several kinds of message.
3. The content type is what makes it a stream rather than a slow response.
4. The format is text: field: value lines, and a blank line terminates one event. That double newline is the framing (Chapter 5.4.1). The id field is the important one — on reconnect the browser automatically sends Last-Event-ID, so the server can resume from where the client left off and no messages are lost across a reconnection.
5. A comment line every 20 seconds, and this is not optional. Proxies, load balancers and mobile carriers close idle connections after 30–120 seconds. Without periodic traffic your stream dies silently and the symptom is "it works locally and stops after a minute in production".
6. Clean up when the client disconnects, or you leak a timer per connection.

SSE is the under-used option and should usually be the default for server push. It is plain HTTP, so it passes through every proxy, works with HTTP/2's multiplexing, carries authentication cookies and headers normally, and is a handful of lines on both sides. It is one-directional and text-only — but most "real-time" requirements are one-directional notifications, and the client can simply use ordinary requests for anything it needs to send.

Its one real limitation is on HTTP/1.1, where the six-connections-per-origin limit (Chapter 5.6.1) means six open tabs exhaust the budget. Over HTTP/2 that disappears.

WebSocket

A full-duplex, persistent, binary-capable connection. It starts as HTTP and then stops being HTTP:

GET /socket HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==

HTTP/1.1 101 Switching Protocols            ← (1)
Upgrade: websocket
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

1. 101 is the only status code that means "this connection is no longer HTTP". After it, both sides speak the WebSocket frame protocol on the same TCP connection.

Use WebSocket when you genuinely need bidirectional, low-latency, high-frequency messaging: collaborative editing, multiplayer games, trading interfaces, chat with typing indicators and presence.

And know what it costs, because these are the things that surprise teams in production:

• No automatic reconnection. Unlike EventSource, you write reconnect-with-backoff yourself, plus the resume logic to fetch what was missed.
• Authentication is awkward. The browser's WebSocket API cannot set custom headers, so you cannot send an Authorization header. Options are a cookie (which works, and brings CSRF-style concerns — there is no same-origin policy on WebSocket, so you must check the Origin header server-side), a token in the query string (which lands in logs, Chapter 5.7), or authenticating in the first message after connecting.
• The connection is stateful, so a stateless load balancer no longer works. You need sticky routing or a shared pub/sub layer so any server can reach any client (Chapter 10.16).
• Backpressure is your problem. A slow client's messages queue in server memory. Unbounded, that is an out-of-memory crash; the fix is a bounded buffer and a policy for what to drop.
• Some corporate proxies block it. Fallback matters for enterprise users.

The decision table:

	Short poll	Long poll	SSE	WebSocket
Direction	client pulls	client pulls	server → client	both
Latency	up to N sec	near zero	near zero	near zero
Reconnect	n/a	manual	automatic + resume	manual
Plain HTTP	yes	yes	yes	no, after upgrade
Binary	no	no	no	yes
Complexity	trivial	low	low	high

Choose SSE unless you need client-to-server messages at the same rate. That single rule covers notifications, live dashboards, progress bars, feeds and price tickers.

4. Webhooks: the server calls you

All of the above assume a browser or an app that connects outward. What about two servers, where the news arrives minutes or hours later — a payment settles, a video finishes encoding, a shipment moves?

Polling is wasteful and slow. So the roles invert: you register a URL, and the other system makes an HTTP request to you when something happens. That is a webhook — sometimes called a reverse API.

Everything about webhooks is about the failure cases, and this is the checklist that separates a working integration from a support ticket.

Verify the signature. Your endpoint is a public URL, so anyone can post to it. The sender signs the raw body with a shared secret and sends the signature in a header:

app.post('/webhooks/payments',
  express.raw({ type: 'application/json' }),                        // (1)
  (req, res) => {
    const signature = req.get('X-Signature') ?? '';
    const expected = crypto.createHmac('sha256', SECRET)
                           .update(req.body)                        // (2)
                           .digest('hex');
    if (!crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))) {  // (3)
      return res.status(401).end();
    }
    const event = JSON.parse(req.body.toString());
    if (await alreadyProcessed(event.id)) return res.status(200).end();  // (4)
    await enqueue(event);                                           // (5)
    res.status(200).end();                                          // (6)
  });

1. The raw body, not the parsed one. express.json() consumes the stream and re-serialising the object produces different bytes — different key order, different whitespace — so the signature will never match. This is the single most common webhook bug (Chapter 9.9.1).
2. HMAC over exactly the bytes that were sent.
3. A constant-time comparison. === returns as soon as it finds a differing byte, so the time it takes leaks how many leading bytes were correct, which lets an attacker forge a signature byte by byte. This is a timing attack and timingSafeEqual exists to stop it.
4. Idempotency is mandatory, not optional. Senders retry on any non-2xx response and on timeouts, so you will receive duplicates — including duplicates of events you processed successfully but acknowledged too slowly. Deduplicate on the event id (Chapter 9.6.3).
5. Enqueue, do not process inline. Senders time out in a few seconds. Doing real work in the handler risks a timeout, which triggers a retry, which produces a duplicate.
6. Return 2xx quickly. The response means "received", not "processed".

Also handle: out-of-order delivery (an updated event may arrive before created, so use timestamps or sequence numbers rather than trusting arrival order), replay protection (reject events with an old timestamp so a captured request cannot be re-sent later), and secret rotation (accept two secrets during a changeover).

And know the constraint: a webhook requires you to have a publicly reachable URL. Behind NAT (Chapter 5.3.3) or on a laptop, you cannot receive one — which is why tunnelling tools exist for local development, and why some providers offer a polling API as an alternative.

5. Long-running work: the correct request/response pattern

A request that takes two minutes should not hold an HTTP connection for two minutes. Proxies time out, retries duplicate the work, and the client cannot tell a slow success from a failure.

The pattern:

POST /api/reports                    →  202 Accepted
                                        Location: /api/reports/8891
                                        Retry-After: 5

GET /api/reports/8891                →  200 OK
                                        {"status": "running", "progress": 0.4}

GET /api/reports/8891                →  200 OK
                                        {"status": "done", "result": "/files/r8891.csv"}

202 Accepted means "I have taken this, it is not finished". The client polls the status URL, or supplies a webhook, or subscribes to an SSE stream for completion. This is the correct shape for any operation longer than a few seconds, and it is what Chapter 11.18's job scheduler formalises.

What the interviewer will push on

"WebSocket or SSE for live notifications?" SSE, unless the client sends messages at a similar rate. Give the concrete reasons: automatic reconnection with Last-Event-ID resume, plain HTTP so proxies and auth work normally, and far less code. Reaching for WebSocket by default is the answer they are screening against.

"Your SSE stream dies after 60 seconds in production but works locally. Why?" An idle-timeout in a proxy, load balancer or carrier. Send a comment line every 15–30 seconds. This is asked because it is the actual bug everyone hits once.

"How do you authenticate a WebSocket?" The browser API cannot set headers, so: a cookie (and then you must check Origin server-side, since there is no same-origin policy), a token in the query string (which leaks into logs), or an auth message immediately after connecting. Naming the header limitation is what shows you have built one.

"Design a webhook receiver." Verify the signature over the raw body with a constant-time comparison, deduplicate by event id, enqueue rather than process inline, and return 2xx fast. Then handle out-of-order delivery and replay. The raw-body detail is the tell.

"Why is GraphQL harder to cache?" Everything is a POST to one URL, so HTTP caching and CDNs do not apply and caching moves into your application. Then add the two other real costs: query complexity limits to avoid a denial-of-service endpoint, and batching to avoid N+1.

"When is gRPC wrong?" Public browser-facing APIs — browsers cannot speak it without a proxy — and anywhere you want HTTP caching or human-readable debugging. It is right between your own services.

"How do you handle a two-minute operation over HTTP?" 202 Accepted with a status URL, then poll or notify. Never hold the connection: proxies time out and retries duplicate the work.

One thing to volunteer: point out that a webhook sender retries on timeouts as well as errors, so you will receive duplicates of events you actually processed successfully — which means idempotency is not a defensive extra, it is a correctness requirement of the pattern. Most candidates treat deduplication as optional hardening.

Recall

• Two questions pick the protocol: who initiates, and one-off or continuous.
• REST is HTTP as intended and cacheable; gRPC is schema-first RPC over HTTP/2 (field numbers are the contract, never reuse them) and browsers cannot speak it natively; GraphQL fixes over- and under-fetching at the cost of HTTP caching, query-complexity limits and N+1.
• The push ladder: short poll (wasteful, high latency) → long poll → SSE → WebSocket. Choose SSE unless the client sends messages at the same rate.
• SSE is plain HTTP with data: lines terminated by a blank line; EventSource reconnects automatically and resumes via Last-Event-ID, and you must send a keepalive every 15–30 seconds or proxies kill the idle stream.
• WebSocket starts with 101 Switching Protocols and then is not HTTP: no auto-reconnect, no custom headers (so Origin must be checked server-side), stateful so sticky routing or a shared pub/sub layer is needed, and backpressure is yours to bound.
• Webhooks invert the direction. Verify the signature over the raw body with a constant-time comparison, deduplicate by event id because senders retry on timeouts too, enqueue rather than process inline, and return 2xx fast. Handle out-of-order delivery and replay.
• Anything longer than a few seconds returns 202 Accepted with a status URL — never hold the connection open.

Self-test: Give three reasons to prefer SSE over WebSocket · What does Last-Event-ID buy you? · Why does an SSE stream die after 60 seconds in production? · Why must webhook signature verification use the raw body? · Why is === unsafe for comparing signatures? · What does 202 Accepted mean and when do you return it?

Next: 5.9 drops from protocols to code — the socket API that every one of these is built on, what a file descriptor holding a connection actually is, and how a program binds an address, listens, and reads bytes.