10.21 — API & Service Interfaces

APIs are the connective tissue of distributed systems. Designing them requires choosing the right communication paradigm, ensuring reliability under failure, and securing access.

Communication Paradigms

REST (Representational State Transfer)

REST models the system as a collection of resources (e.g., /users/123, /orders/456). Operations are mapped to HTTP verbs (GET, POST, PUT, PATCH, DELETE).

• Pros: Ubiquitous, leverages standard HTTP caching, easy to understand.
• Cons: Over-fetching (getting too much data) or under-fetching (requiring multiple requests to get related data).

GraphQL

GraphQL exposes a single endpoint. The client sends a query specifying exactly the shape of the data it needs, and the server resolves it.

• Pros: Solves over/under-fetching, excellent for complex client-side applications.
• Cons: Harder to cache at the network level, because every query is a POST to one URL. A client can also send a deeply nested query that is very expensive to answer, so servers enforce depth and cost limits. And a naive server makes one database call per item in a list, the N+1 problem covered in the interview questions below.

gRPC

Developed by Google, gRPC uses Protocol Buffers (protobuf) as the Interface Definition Language (IDL) and HTTP/2 for transport.

• Pros: Binary payload (smaller, faster parsing than JSON), strongly typed contracts, supports bidirectional streaming.
• Cons: Not natively supported by browsers (requires gRPC-Web proxy), harder to debug with standard network tools.

Real-Time and Asynchronous Patterns

When standard request-response isn't enough, we need patterns for pushing data or handling long-running tasks.

Long Polling vs Server-Sent Events vs WebSockets

• Long Polling: The client requests data. The server holds the request open until data is available, responds, and the client immediately reconnects. Use when real-time is needed but WebSockets are unavailable.
• Server-Sent Events (SSE): A unidirectional channel over standard HTTP. The client subscribes, and the server pushes events. Perfect for live feeds (e.g., stock tickers, notification streams).
• WebSockets: A persistent, bidirectional, full-duplex TCP connection. Ideal for chat applications or multiplayer games where both sides send high-frequency data.
• Webhooks: Server-to-server communication. Server A registers a URL with Server B. When an event happens on B, it makes an HTTP POST to A's URL.

Core API Design Principles

Idempotency

In distributed systems, networks drop packets. A client sends POST /checkout, the server charges the card, and the response is lost on the way back. The client cannot tell "my request never arrived" from "it worked and I missed the answer", so it retries. Without protection, the customer is charged twice.

An operation is idempotent when doing it twice has the same effect as doing it once. A charge is not naturally idempotent, so the API makes it idempotent: the client generates one random key per logical payment and sends it in an Idempotency-Key header on the first attempt and every retry. The server remembers each key and what happened to it.

The obvious implementation is wrong, and it is worth seeing why. "Look up the key; if it is not there, insert it and charge" is two steps. Two retries arriving in the same millisecond both look, both find nothing, and both charge. This is the check-then-act race from 2.4. The fix is to make the claim a single operation the database decides: an insert against a unique key, where exactly one caller can win.

import { createHash } from "node:crypto";
import { db, paymentGateway } from "./services";

interface CheckoutRequest { userId: string; orderId: string; amountPaise: number; }
interface ApiResponse { status: number; body: unknown; }

const hashOf = (req: CheckoutRequest) =>
  createHash("sha256").update(JSON.stringify(req)).digest("hex");

export async function processCheckout(req: CheckoutRequest, key: string): Promise<ApiResponse> {
  const claimed = await db.query(                                          // (1)
    `INSERT INTO idempotency_keys (user_id, idem_key, request_hash, state, lease_until)
     VALUES ($1, $2, $3, 'in_flight', now() + interval '30 seconds')
     ON CONFLICT (user_id, idem_key) DO NOTHING`,
    [req.userId, key, hashOf(req)]);

  if (claimed.rowCount === 0) {                                            // (2)
    const { rows: [prior] } = await db.query(
      `SELECT request_hash, state, response FROM idempotency_keys
       WHERE user_id = $1 AND idem_key = $2`, [req.userId, key]);
    if (prior.request_hash !== hashOf(req))                                // (3)
      return { status: 422, body: { code: "IDEMPOTENCY_KEY_REUSED" } };
    if (prior.state === "completed") return prior.response;               // (4)

    const takeover = await db.query(                                       // (5)
      `UPDATE idempotency_keys SET lease_until = now() + interval '30 seconds'
       WHERE user_id = $1 AND idem_key = $2 AND state = 'in_flight' AND lease_until < now()`,
      [req.userId, key]);
    if (takeover.rowCount === 0)
      return { status: 409, body: { code: "REQUEST_IN_FLIGHT", retryAfter: 2 } };
  }

  const charge = await paymentGateway.charge({                             // (6)
    amountPaise: req.amountPaise,
    idempotencyKey: `${req.userId}:${key}`,
  });

  const response: ApiResponse = { status: 201, body: { orderId: req.orderId, chargeId: charge.id } };
  await db.query(                                                          // (7)
    `UPDATE idempotency_keys SET state = 'completed', response = $3
     WHERE user_id = $1 AND idem_key = $2`, [req.userId, key, response]);
  return response;
}

1. The atomic claim. ON CONFLICT DO NOTHING means the insert either creates the row or does nothing because the row already exists, and the database guarantees only one concurrent caller creates it. The key is scoped to the user, so two customers who happen to generate the same key never see each other's responses. The row starts in_flight with a 30-second lease.
2. rowCount === 0 means somebody claimed this key before us. Read what they left behind.
3. Same key, different body. That is never a genuine retry; it means the client is reusing keys. Refusing with 422 is far safer than replaying the stored response, which would tell the client a payment succeeded that was never made.
4. The first attempt finished. Send back its stored response exactly, so every retry sees the same answer as the first success.
5. The first attempt is still in_flight. If its lease is still running, it is probably working right now, so answer 409 and let the client come back. If the lease has run out, the first attempt most likely crashed half-way. This conditional UPDATE takes over the lease, and because it is one statement, only one retry can win the takeover.
6. The key travels downstream. The charge may already have happened in the crashed attempt. Passing a key derived from ours to the payment provider means the provider's own deduplication returns the same charge instead of creating a second one. Retry safety only works if every hop that has a side effect takes part.
7. Record the result. If the process dies right before this line, the row stays in_flight, the lease expires, and the next retry takes over at step (5) and is protected by step (6).

What each situation returns, as run against a database with real unique-key behaviour:

Situation	Response	Cards charged
Two identical requests at the same moment	201 and 409	1
A retry after success	the stored 201, same charge id	1
Same key, different amount	422	1
Crash after charging, retry within 30 s	409	1
Crash after charging, retry after 30 s	201, same charge id	1

Two details that are easy to miss. Do not delete the key when the charge throws. A timeout from the provider is ambiguous in exactly the same way the original lost response was, so deleting the key would reopen the double charge. Leave it in_flight and let the lease decide. Give keys an expiry of a day or two, so the table does not grow forever. The full treatment, including how this meets ETags and versioning, is 9.6.3, and the consumer-side version for queues is in 10.4.

Pagination: Offset vs. Cursor

• Offset Pagination (?limit=10&offset=100): Easy to implement, but fails at scale. The database must scan and skip the first 100 rows. If rows are added/deleted during pagination, items will be skipped or duplicated.
• Cursor Pagination (?limit=10&after=cursor_xyz): The cursor is usually an encoded string representing the last seen item's ID and sort column. The database queries WHERE id > last_id. It is highly performant and stable against data mutations.

Versioning

APIs evolve. Breaking changes require versioning.

• URI Routing: /v1/users (Explicit, easily visible).
• Header Versioning: Accept: application/vnd.company.v1+json (Cleaner URIs, but harder to test in a browser).

Security: Identity & Access

AuthN, AuthZ, and SSO

• Authentication (AuthN): Validating who the user is (e.g., checking password, verifying a JWT signature).
• Authorization (AuthZ): Validating what the user is allowed to do (e.g., checking if the user has the admin role to delete a file).
• Single Sign-On (SSO): Allows a user to log in once to a central Identity Provider (IdP) and access multiple independent applications. Modern SSO typically uses SAML (enterprise) or OIDC/OAuth2 (consumer/modern enterprise).

Infrastructure: Gateway & Rate Limiting

API Gateway

An API Gateway sits between clients and your microservices. It acts as a reverse proxy, insulating clients from the internal architecture.

API Gateway interacting with downstream services

Gateway Responsibilities:

1. Routing: Mapping /orders to the Order Service.
2. Protocol Translation: Converting a client's REST call into an internal gRPC call.
3. Cross-Cutting Concerns: Offloading authentication, SSL termination, and rate limiting so individual services don't have to implement them.

Rate Limiting

Prevents abuse and cascaded failures. The most common algorithm is the Token Bucket:

• A bucket holds up to N tokens.
• Tokens are added at a constant rate R.
• Each request removes 1 token.
• If the bucket is empty, the request is rejected (HTTP 429 Too Many Requests).

In distributed systems, the token bucket state is typically stored in an in-memory cache like Redis to ensure limits apply globally across all Gateway instances.

---

What the interviewer will push on

1. "How do you implement rate limiting in a distributed system?"Answer: You cannot use local memory. You must use a centralized datastore like Redis. A naïve Redis implementation requires a read then a write, which causes race conditions. You must use Redis INCR or Lua scripts to ensure atomicity.
2. "How does gRPC achieve better performance than REST?"Answer: gRPC uses HTTP/2 (multiplexing multiple requests over a single TCP connection) and Protobufs (binary encoding). Binary encoding requires much less CPU to serialize/deserialize compared to JSON, and the payloads are significantly smaller.
3. "Explain the N+1 problem in GraphQL."Answer: If you query a list of 10 authors, and for each author you request their books, a naïve GraphQL resolver will execute 1 query for authors, and then 10 separate queries for the books (1 + 10 = 11 queries). The solution is to use a tool like DataLoader to batch and deduplicate the nested requests into a single IN (...) database query.
4. "Why use Cursor pagination over Offset?"Answer: Offset pagination performance degrades linearly as the offset grows because the DB must compute and skip all previous rows. Cursor pagination uses an index (e.g., WHERE id > X), so the database seeks straight to the starting row in O(\log N) and then reads one page, no matter how deep the page is. Page 10,000 costs the same as page 1.

---

Recall

• REST: HTTP verbs, Resource-oriented.
• GraphQL: Client specifies data shape, solves over/under-fetching.
• gRPC: Protobufs, binary, HTTP/2, fast service-to-service.
• WebSockets / SSE: Real-time bidirectional / real-time unidirectional push.
• Idempotency: A property where repeated identical requests yield the same result without duplicate side-effects. Crucial for handling retries.
• AuthN vs AuthZ: Who you are vs. What you can do.
• API Gateway: Central entry point handling routing, auth, and rate limiting.

Quiz:

1. Why is offset pagination dangerous for large datasets?
2. If a client needs a live feed of stock prices but never sends data back to the server, should you use WebSockets, SSE, or Long Polling?
3. How does an idempotency key prevent duplicate charges during a network timeout?

Next: 10.22 looks behind the API at where the data lands, and how the storage engine decides what is fast.