Appearance
10.14.3 — CDN and the Edge
Your servers are in London. A user in Sydney requests a 200 KB product image.
The round trip is about 250 ms, and a fresh HTTPS connection needs several of them — one for the TCP handshake, one or two for the TLS handshake, one for the request and first byte. Before a single byte of image arrives, roughly three quarters of a second has gone. The image itself then takes a few more round trips to transfer, because a new connection starts cautiously and speeds up over time.
The same image served from a machine in Sydney takes about 15 ms.
Nothing about the file changed. The only difference is distance, and distance is the one part of a system you cannot optimise in software. This is what a CDN — a content delivery network — exists to fix: keep copies of your content in hundreds of locations around the world, and serve each user from the one nearest them.
1. Why distance beats bandwidth
The instinct is that a slow page needs more bandwidth. Usually it needs fewer round trips.
Light travels about 200,000 km per second through fibre, and real routes are not straight lines. London to Sydney is roughly 17,000 km, so a one-way trip is about 85 ms at best and around 125 ms in practice, making a round trip 250 ms. No amount of bandwidth changes that number. You can upgrade to a 100 Gbit link and the first byte still takes 250 ms to come back.
This is why the round-trip count dominates for small responses. A request needing four round trips before the content starts flowing costs a full second from Sydney and 40 ms from a nearby city — a twenty-five-fold difference produced entirely by geography.
Two consequences follow, and both are worth stating in an interview because they reframe the problem.
Putting content near users is the single largest performance win available for anything served to a geographically spread audience, and it is usually far cheaper than any equivalent gain from optimising code.
A CDN helps even for content it cannot cache, because the user's connection is established with a nearby machine, and that machine holds a long-lived, already-warmed connection back to your origin. The handshakes happen over 15 ms instead of 250 ms, and only the final fetch crosses the ocean.
2. How a request reaches the nearest location
A CDN has hundreds of points of presence, which are just data centres holding caches. Two mechanisms route a user to a near one.
DNS-based routing. The CDN runs the DNS for your hostname and answers each lookup with the address of a nearby location, based on where the query came from. Simple and widely used. Its weakness is that it sees the address of the user's DNS resolver, not the user, so somebody using a resolver in another country can be sent a long way from home.
Anycast. The same address is announced from many locations at once, and the internet's own routing delivers each packet to whichever announcement is nearest. This routes on the actual network path rather than a guess, and it fails over automatically — if a location goes offline, its announcement disappears and traffic moves to the next nearest without any DNS change or waiting for a record to expire.
3. The cache key, and how it gets destroyed
A CDN decides whether it already has a response by computing a cache key. By default that is the method plus the full URL, and that default is what makes CDNs work.
It is also what makes them silently useless, because two things routinely poison it.
Cookies. If the origin sends a Set-Cookie header, or the CDN is configured to include cookies in the key, then every user has a different key and the hit rate goes to zero — while every dashboard says the CDN is working. This is the most common reason a CDN delivers nothing: a session cookie attached to static asset responses. Serve assets from a path or hostname that never sees cookies.
A careless Vary header. Vary tells the cache which request headers change the response, so each distinct value gets its own entry. Vary: Accept-Encoding is correct and cheap, because there are two or three encodings. Vary: User-Agent is a disaster, because there are millions of user-agent strings and you have just given every browser build its own cache entry.
And the security failure of the same mechanism, which is worth naming because it is a real attack. If the origin varies its response on a header that is not in the cache key — an unkeyed header — an attacker can send a request with a malicious value, get a poisoned response stored, and have it served to everybody else. This is cache poisoning, and the defence is that anything affecting the response must either be in the key or must not affect the response.
4. Telling the CDN what to do
The Cache-Control header is the contract, and the directives split into groups.
How long, and for whom.
max-age=300 — any cache may keep this for 300 seconds. s-maxage=3600 — shared caches (the CDN) may keep it for an hour, overriding max-age for them only. That pair is genuinely useful: a short browser lifetime so users see updates soon, a long CDN lifetime so your origin is protected.
public means shared caches may store it. private means only the user's own browser may — use it for anything personalised.
What must never be stored.
no-store means do not write this anywhere, at all. This is what you want for personal or sensitive responses.
no-cache does not mean that, despite the name, and the confusion is a standard interview probe. It means store it, but revalidate with the origin before reusing it. So no-cache is a caching instruction and no-store is the refusal.
Serving old content deliberately.
stale-while-revalidate=60 — after expiry, keep serving the old response for up to another 60 seconds while fetching a fresh one in the background. This is 10.14.2's serve-stale pattern, standardised as a header. Nobody waits for a rebuild.
stale-if-error=86400 — if the origin is failing, keep serving the stale response for up to a day rather than showing an error. This one line turns an origin outage into slightly old content for anyone whose request is cacheable, and it is the cheapest availability improvement in this entire chapter.
Revalidation without re-transfer. The origin sends an ETag — a short identifier for this exact version of the response. Later, the cache asks again with If-None-Match: "<that etag>". If nothing changed, the origin replies 304 Not Modified with no body, and a 200 KB transfer becomes a couple of hundred bytes. Last-Modified with If-Modified-Since is the older equivalent with one-second resolution. This is the same conditional-request machinery your own API should use (9.6.3).
5. Protecting the origin: tiered caching
A CDN with three hundred locations has a problem you would not guess: on a cache miss, three hundred locations can each fetch the same object from your origin. A popular new video launching worldwide produces three hundred simultaneous requests for the same file, which is 10.14.2's stampede with geography instead of concurrency as the cause.
Every serious CDN offers this under some name — shield, tiered cache, mid-tier. Turn it on. It costs one configuration setting and it is the difference between a launch that works and an origin that falls over at the moment of maximum attention.
6. Getting rid of stale content
Two approaches, and one of them is nearly always right.
Purging tells the CDN to drop an object. It works, and it has three weaknesses: it takes seconds to tens of seconds to reach every location, browsers that already downloaded the file keep it regardless, and a purge of everything triggers exactly the stampede section 5 was protecting you from.
Versioned URLs make the problem disappear. Put a hash of the content in the filename — app.9f2c1e.js — so a change produces a new URL. The old URL is never requested again, nothing needs purging, and every copy anywhere in the world is correct by construction. This also lets you set an effectively infinite lifetime on the asset, because a file at that URL can never change.
The standard arrangement, and the answer to give: the HTML document has a short lifetime and is not versioned, and everything it references is versioned with a one-year lifetime. Deploying means uploading new versioned assets and then publishing an HTML file that points at them. The only thing that ever needs invalidating is the HTML, which is small and cheap to revalidate.
Keep purging for the cases versioning cannot cover: content that must be removed for legal reasons, a wrong price on a cached page, an accidental leak.
7. What belongs on a CDN
Obviously. Images, video, fonts, scripts, stylesheets, downloads. Anything static, large, or requested by many people.
Yes, with attention. Public API responses that are the same for everyone — a product listing, a store locator, exchange rates — cached for tens of seconds. Small windows still remove enormous load, because a listing cached for thirty seconds absorbs everything except two requests per minute per location.
Carefully. HTML for pages that are the same for everyone, with a short lifetime plus stale-while-revalidate. The trap is personalisation: a page rendering "Hello, Priya" must never be shared, and the standard fix is to serve one cacheable page and fetch the personal parts separately from the browser.
Never. Anything personal or authenticated, unless the response is explicitly private and the CDN is configured to respect it. The failure mode here is severe and has happened to large companies: one user's account page served to another because a Cache-Control header was missing.
8. Running code at the edge, honestly
CDNs now let you run small pieces of code at their locations. It is genuinely useful for a narrow set of jobs.
What it is good at: routing decisions (send this country to that origin), rewriting requests and responses (add security headers, normalise the cache key), authorisation checks that reject bad requests before they cross an ocean, redirects, and A/B assignment. All of these are small, fast, and about deciding rather than computing.
What it is not good at: anything needing your database. The edge is close to the user and far from your data, so a piece of edge code that makes three database calls to London has made the request slower than serving it from London would have been. Edge compute moves computation to the user; it does not move your data, and the data is usually what the request actually needs.
The honest summary: use the edge for decisions that need only the request itself. Everything that needs your data belongs where your data is, or needs its own replicated store at the edge, which is a much larger commitment than it first appears.
9. Keeping other people from using your bandwidth
Signed URLs. The URL carries an expiry and a signature, so a link works for a limited window and cannot be forged. This is how paid downloads and private video segments are protected without the CDN needing to talk to your authentication system on every request.
Referrer and origin restrictions stop another site embedding your images and billing you for the traffic. Weak on their own, since headers can be forged, but they stop casual hotlinking.
Rate limiting at the edge is the one worth emphasising, because it is the only place that can absorb a large attack. Traffic blocked at three hundred locations never reaches your origin at all, which means the edge is both your performance layer and your first line of defence.
10. What the interviewer will push on
"What is the difference between no-cache and no-store?" A direct probe, and a surprising number of people get it backwards. no-store means never write this down anywhere. no-cache means store it but check with the origin before reusing it. If you want a response to leave no trace, no-store is the only correct answer.
"Why is your CDN hit rate two percent?" They are testing whether you know what destroys a cache key. The two usual culprits are cookies attached to asset responses, giving every user a unique key, and an over-broad Vary header such as Vary: User-Agent. Both look fine on a dashboard while delivering nothing.
"You deploy a new version. How do users get it?" The strong answer is versioned URLs with a content hash, an infinite lifetime on assets, and a short-lived HTML document that points at them — so nothing ever needs purging and every copy worldwide is correct by construction. Purging is the fallback for legal removals and mistakes, not the deployment mechanism.
"A video launches globally. What happens to your origin?" Three hundred locations each miss and each fetch from you. The answer is an origin shield so the fan-in happens at a mid-tier, plus pre-warming the object into the shields before the launch. This is the stampede with geography as the cause.
"How would you cache an API response that differs per user?" Usually you would not, at the shared layer. The better answer is to split the response: one cacheable public part served from the edge, and a small personalised part fetched separately. If it truly must be per-user, mark it private so only the browser stores it, and make sure the CDN is configured to honour that — because getting this wrong means serving one user's data to another.
"Would you run your API at the edge?" They want to see whether you know that edge compute moves code and not data. It is right for decisions that need only the request — routing, rewriting, auth rejection, redirects. It is wrong for anything that has to call your database across an ocean, which is slower than just serving it from where the database is.
The thing to volunteer that nobody asks for: stale-if-error. One header means an origin outage degrades into slightly old content instead of an error page for everyone whose request is cacheable. It is the cheapest availability improvement in this whole chapter and almost nobody sets it.
Next: 10.15 — once traffic has crossed the network and arrived, something has to decide which of your servers handles it.
Recall
- Distance is the cost you cannot optimise away. London to Sydney is a 250 ms round trip; several handshakes make that most of a second before the first byte. Bandwidth does not change it, so fewer round trips is the goal.
- A CDN helps even for uncacheable content, because handshakes happen against a nearby machine over an already-warm connection to your origin.
- Anycast routes on the real network path and fails over automatically; DNS routing sees the resolver rather than the user.
- The cache key is method plus URL by default, and it is destroyed by cookies on asset responses and by an over-broad
Vary. A header that changes the response but is not in the key is a cache poisoning hole. no-store= never write it down.no-cache= store it, revalidate before reuse.s-maxagelets the CDN keep it longer than the browser does.stale-while-revalidatemeans nobody waits for a rebuild.stale-if-errorturns an origin outage into slightly old content.ETag+If-None-Match→304revalidates without re-transferring the body.- Origin shield / tiered caching stops 300 locations each fetching the same miss from you. Turn it on.
- Versioned URLs beat purging: a content hash in the filename means a change creates a new URL, so nothing needs invalidating and assets can live forever. Short-lived HTML points at long-lived assets.
- Edge compute moves code, not data. Good for routing, rewriting, auth rejection, redirects. Bad for anything that must call your database.
Self-test: Why does a CDN help even for content it cannot cache? What are the two usual reasons for a near-zero hit rate? State the difference between no-cache and no-store. What does an origin shield prevent? Why are versioned URLs better than purging? What is edge compute bad at, and why?
Quiz Bank
FoundationalExplain what a CDN actually does and why it helps even for content that cannot be cached.
What it does. A CDN keeps copies of your content in hundreds of data centres around the world and serves each user from a nearby one. A user in Sydney gets your image from a machine in Sydney rather than from your servers in London.
Why that matters so much comes down to physics rather than engineering. Light travels about 200,000 km per second in fibre, and real routes wander, so a London-to-Sydney round trip is around 250 ms and no equipment upgrade changes that. Establishing a connection needs several round trips — TCP, then TLS, then the request — so a user can wait three quarters of a second before the first byte of content appears. Served locally, the same exchange costs 15 ms. Bandwidth is irrelevant to this; round-trip count and distance are the whole story for small responses.
Now the part people miss: it helps even when nothing is cached. Suppose the response is a personalised page that genuinely cannot be shared. The user still connects to the nearby CDN machine, so all the handshake round trips happen over 15 ms instead of 250 ms. The CDN then forwards the request to your origin over a connection it already holds open and has already warmed up, so there is no handshake and no slow start on that leg either. What was four slow round trips plus a transfer becomes four fast round trips plus one slow one.
The same applies to uploads and to APIs. This is why CDNs are sold for "dynamic acceleration" as well as for caching, and it is a genuinely different benefit rather than marketing.
Two more benefits worth naming. Attack traffic is absorbed across hundreds of locations rather than arriving at your origin, so the edge is your first line of defence as well as your performance layer. And your origin's bandwidth bill drops sharply, because the expensive repeated transfers happen from the CDN's network rather than yours.
AppliedYour CDN reports a two percent hit rate on a mostly static site. Diagnose it.
A two percent hit rate on static content means the cache key is different for nearly every request. There are four realistic causes and they are quick to check in order.
Cookies on asset responses. The most common cause by a wide margin. If your application sets a session cookie on every response, and your assets are served from the same hostname and path as the application, the CDN sees Set-Cookie and — quite correctly — refuses to share the response between users. Every user gets their own entry, so nothing is ever a hit. The fix is to serve assets from a hostname or path that never touches the session, and to strip Set-Cookie from asset responses at the edge.
An over-broad Vary header. Vary: User-Agent gives every browser build its own cache entry, and there are millions of user-agent strings in the wild. Vary: Accept-Encoding is fine because there are two or three values. Check what your framework is sending by default; several add Vary headers you did not ask for.
Query strings in the key. If analytics or advertising parameters ride along on asset URLs, then logo.png?utm_source=email and logo.png?utm_source=twitter are different objects. Configure the CDN to ignore parameters that do not affect the response, or strip them.
Missing or hostile Cache-Control. If the origin sends no caching headers, the CDN falls back to conservative defaults or refuses to store at all. If it sends no-store, or private, or max-age=0, the CDN is doing exactly what you told it. This is worth checking first because it is the fastest thing to look at.
How to find it in practice: request the same asset twice with curl -I and look at the response headers. The CDN's own headers will tell you whether it was a hit, a miss, or explicitly uncacheable, and most will name the reason. Two identical requests both reporting a miss with a "cannot cache" reason gives you the answer in thirty seconds, which beats reasoning about it.
And the point worth making about the metric itself: a low hit rate is invisible in every other measurement. Latency looks acceptable, errors are zero, the CDN dashboard shows plenty of traffic. Nothing is broken; you are simply paying for a CDN and receiving a slightly faster proxy. That is why hit rate belongs on a dashboard somebody actually looks at.
InterviewHow do you ship a new version of your JavaScript so every user gets it immediately, without purging?
Versioned URLs, and the mechanism is worth spelling out because the elegance is the argument.
Build the file, hash its contents, and put the hash in the filename: app.9f2c1e.js. Change one character of the source and the hash changes, so the new build is at app.4b71d0.js — a different URL entirely.
Now set an effectively infinite lifetime on it: Cache-Control: public, max-age=31536000, immutable. That is safe in a way it never normally is, because the content at that URL can never change. If the content changed, the URL would have changed.
The result is that invalidation disappears as a problem. Nothing needs purging, because the old URL is simply never requested again. Every CDN location, every corporate proxy, and every browser anywhere in the world is correct by construction, including ones you have never heard of and cannot send a purge to.
The one thing that must not be versioned is the HTML, because it is the entry point and the browser has to be able to find it at a stable address. So the HTML gets a short lifetime — thirty seconds, or a few minutes with stale-while-revalidate — and it is the document that points at the current asset filenames. Deploying means uploading the new versioned assets first, then publishing the HTML that references them.
Three details that make it robust.
Upload assets before publishing the HTML. If the order is reversed, a user who fetches the new HTML in that window requests an asset that does not exist yet, and gets a broken page.
Keep the old assets around. A user with the previous HTML cached will still be requesting the previous filenames for as long as their copy lives. Deleting old assets on deploy breaks exactly those users, and it is a self-inflicted incident that looks mysterious because it only affects people who visited recently.
immutable is worth adding. Without it, a browser revalidates on reload even when the entry is fresh; with it, a reload skips the request entirely.
And when you still need purging: legal takedowns, a wrong price on a cached page, an accidental leak of private data. Purging is the emergency tool, not the deployment mechanism — and note that a purge-everything triggers a stampede on your origin, so the emergency tool has its own cost.
StaffDesign the caching and delivery layer for a video streaming service, from upload to a viewer in another country.
Video is where CDN thinking becomes structural rather than an optimisation, because the numbers make everything else impossible.
Start with the shape of the content, because it determines every later decision. A video is not one file. It is transcoded into several qualities, and each quality is cut into segments of a few seconds, with a small manifest listing them. That structure exists for adaptive playback — the player measures its own bandwidth and picks the next segment's quality — and it happens to be perfect for caching, because a segment is small, immutable, and requested by everyone watching that title at that point.
Immutability is the gift here, so take it. A segment's content never changes once produced, so give segments a content-addressed URL and a one-year lifetime. No purging, no revalidation, no staleness question at all. The manifest is the only mutable thing, and it gets a short lifetime — seconds for a live stream, minutes for on-demand.
The economics force the design. One hour of 1080p is roughly 3 GB. A hundred thousand concurrent viewers is 300 TB per hour if served from your origin, which is not a bandwidth bill anyone wants and probably not a link anyone has. The CDN is therefore not a performance layer here; it is the only way the service can exist. Origin egress must approach zero, and the target is that your origin serves each segment once per shield, not once per viewer.
Tiered caching is mandatory, not optional. With hundreds of edge locations, a newly released episode would otherwise produce hundreds of simultaneous fetches per segment. Shields fan that in, so the origin sees a handful. For a scheduled release you go further and pre-warm: push the first few minutes of segments into the shields before the release time, so the launch spike finds a warm cache rather than creating one.
Popularity is extremely skewed and the caches should reflect it. A small number of titles account for most viewing, so the hot set fits comfortably in edge storage while the long tail does not. Let the edges hold the popular titles and let the tail be served from the shields, accepting a slightly higher first-byte time for something few people watch. Sizing the edge for the hot set rather than the catalogue is what makes the storage bill sane.
Live is a different problem and should be separated explicitly. Segments are produced seconds before they are watched, so there is no pre-warming and the manifest changes constantly. Manifests get lifetimes of a second or two, segments remain immutable and cacheable, and the whole point becomes the fan-in: a hundred thousand viewers requesting the newest segment within the same two seconds is the stampede in its purest form, and only a tiered structure with request collapsing at each layer survives it.
Access control without breaking caching. Video must be protected, and per-request calls to your authentication system would defeat the whole design. Signed URLs solve it: the player receives URLs carrying an expiry and a signature, the edge verifies the signature locally with no call home, and the URLs expire quickly enough to limit sharing. The signature is not part of the cache key, or every viewer would get a unique key and the hit rate would collapse — which is a subtle configuration point and a good thing to mention, because it is exactly the kind of detail that silently ruins a CDN deployment.
Failure behaviour, decided in advance. If the origin is unavailable, stale-if-error keeps existing content playing. If a shield is unavailable, edges fall back to another shield or to the origin, and the important property is that they do not all fall back simultaneously to the same place. And the player itself has a fallback that matters more than any of this: if segments arrive slowly, it drops to a lower quality rather than stopping, so a degraded network becomes a softer picture instead of a spinning circle. That is the most user-visible resilience decision in the whole design, and it lives in the client rather than the infrastructure — which is worth saying, because it is the part interviewers rarely hear.
Flashcards
FlashWhy a CDN helps uncacheable content
Handshakes happen against a nearby machine over an already-warm connection to your origin. Four slow round trips plus a transfer becomes four fast ones plus one slow one.
Flashno-cache versus no-store
no-store: never write it down anywhere. no-cache: store it, but revalidate with the origin before reuse. Only no-store means "leave no trace".
FlashWhat kills a CDN hit rate
Cookies on asset responses (every user gets a unique key) · over-broad Vary such as User-Agent · tracking query strings in the key · missing or hostile Cache-Control.
FlashVersioned URLs
Content hash in the filename. A change creates a new URL, so nothing needs purging and assets get a one-year immutable lifetime. Short-lived HTML points at them. Upload assets before publishing HTML; keep the old ones.
Flashstale-if-error
Keep serving the stale response while the origin is failing. One header turns an outage into slightly old content. Cheapest availability win available.
FlashOrigin shield
Without it, 300 edge locations each fetch the same miss from you. Shields fan that in to a handful. Pre-warm them before a scheduled launch.