13.6.8 — Why Docker Alone Is Not Enough

Forty containers across six servers. At 03:14 a server loses power.

Seven containers are gone. Docker on the surviving five machines does not know that server existed, so nothing is restarted anywhere. The load balancer keeps sending a share of traffic to a dead address for another ninety seconds. Two of the seven were the only copies of their service.

Somebody's phone rings. They open a laptop and start typing docker run commands on whichever machines have room, guessing at the flags, reading a wiki page from eight months ago for the environment variables.

Nothing about Docker is broken here. Docker did exactly what it promises: it ran containers on the machine it was installed on. The problem is that "the machine it was installed on" is the wrong unit. Everything on this page follows from that one mismatch.

1. Docker's honest scope

One Docker daemon manages one machine. It knows about its own containers, its own images, its own bridge network, its own volumes. It has no idea that other machines exist.

This is not an oversight. It is a clean boundary, and it is why Docker is simple enough to learn in an afternoon. But it means every question that spans more than one machine has no answer inside Docker:

• Which of my six servers should run this container?
• What happens to its work when that server dies?
• How do callers find it when its address changes?
• How do I replace all twelve copies of a service without dropping requests?
• How do I run more copies when traffic doubles, and fewer at 3 a.m.?

A single-machine tool cannot answer fleet-level questions, and no amount of Docker expertise changes that.

2. Building the answer yourself

The most useful way to understand orchestration is to try to solve the 03:14 outage with the tools you already have. Each fix works, and each one creates the next problem.

Fix 1: restart policies. --restart=unless-stopped brings a container back if it crashes or the host reboots.

Which fails when the host does not come back. A restart policy is a local instruction to a local daemon on a machine that is now a paperweight.

Fix 2: a script on a monitoring box. Poll each container's health endpoint; when one fails, SSH somewhere and start a replacement.

Which immediately raises: start it where? Now you need to know each machine's free CPU and memory, which containers are already there, whether this workload needs the machine with the GPU, and whether you have accidentally put all three copies of the same service on one server. You have started writing a scheduler.

Fix 3: keep a placement table. A file recording what should run where, and how much of each machine is spoken for.

Which must be updated by every deploy, must not be edited by two people at once, must survive its own machine dying, and must be reconciled against reality — because a machine that comes back after a network partition may still be running the container you already replaced. You have started writing a consistent, replicated datastore.

Fix 4: service discovery. Containers get a new IP whenever they move, so a config file full of addresses is stale within a day. You add a DNS server the scripts update.

Which needs its records removed the moment a container is unhealthy — not when its TTL expires — or traffic keeps flowing to a dead instance for a minute. You have started writing a health-driven service registry.

Fix 5: rolling deploys. A script that stops old containers and starts new ones a few at a time.

Which must check that each new container is actually serving before proceeding, must never drop below a minimum number of healthy copies, must stop and reverse if the new version fails, and must not fight with the health-monitor script from Fix 2 that is trying to restart the containers you are deliberately stopping. You have started writing a deployment controller.

Fix 6: scaling. More copies when the queue backs up.

Which needs metrics collection, a decision policy, hysteresis so it does not oscillate, and — when no machine has room — a way to add machines. You have started writing an autoscaler.

Figure 1 — The six fixes and what each one becomes. Every organisation that ran containers at scale walked down the left column and arrived at the right one. The interesting fact is not that orchestrators exist — it is that everyone independently derived the same six components.

Six patches later you are maintaining a distributed system with a scheduler, a consistent datastore, a service registry, a deployment controller and an autoscaler — as a side project, with no tests, understood by two people, on top of your actual job.

This is not hypothetical. Between 2012 and 2015 essentially every company running containers at scale built exactly this, independently, and every one of them concluded the same thing: it is a real distributed systems problem, and it should be solved once by people who do it full time.

3. Where Kubernetes came from

Google had already solved it, a decade early and out of sight.

Borg (from about 2003). Google's internal cluster manager. Every service — search, Gmail, Maps — ran as a job on Borg, sharing machines with batch work to keep utilisation high. Borg ran hundreds of thousands of jobs across tens of thousands of machines per cell.

Google described it publicly only in 2015, in a paper called Large-scale cluster management at Google with Borg, which is worth reading because most of Kubernetes' design decisions appear there first with the reasoning attached. Borg introduced the ideas that matter: declared desired state, a central scheduler, resource requests and limits, job priority and preemption, and the practice of packing services and batch work onto the same machines.

Omega (about 2013) was the research successor, exploring a shared-state scheduler where several schedulers work against one consistent store rather than queuing behind a single one. That idea shaped the way Kubernetes' components all read and write through one API.

Kubernetes (2014). Joe Beda, Brendan Burns and Craig McLuckie proposed rebuilding Borg's ideas in the open, for containers, on any infrastructure. Announced in June 2014, version 1.0 in July 2015, and donated on that day to the newly formed Cloud Native Computing Foundation — a deliberate choice to make it neutral rather than a Google product.

The name is Greek for helmsman, the person steering a ship. "K8s" is the eight letters between K and s. Same joke as i18n.

The competition, and why it mattered:

Apache Mesos came out of Berkeley in 2009 and ran Twitter and Airbnb. It was a general resource manager with frameworks on top, of which Marathon ran long-lived services. More flexible, and one layer more abstract than most teams wanted.

Docker Swarm was Docker's answer, built in and easy — the same Compose file, across machines. Simpler, less capable, and it arrived after Kubernetes had momentum.

HashiCorp Nomad is a single binary that schedules containers and also plain executables and virtual machines. Genuinely simpler to operate, and still used where that simplicity is the point.

Kubernetes won between 2015 and 2017, for four reasons worth naming:

The API is the product. Everything is a resource in a uniform, versioned, extensible API. You can add your own resource types and your own controllers and they behave exactly like the built-in ones. Competitors had features; Kubernetes had a platform other people could build on.

Declarative, not imperative. You state what should be true. Controllers make it true and keep it true. That one choice is why self-healing, rolling updates and GitOps are all the same mechanism rather than three features.

Vendor neutrality. Under the CNCF, no single company owned it, so every cloud provider could adopt it without handing a competitor an advantage. Amazon, the last holdout, announced managed Kubernetes in late 2017.

And then everyone else conceded. Docker added Kubernetes support to its own enterprise product in 2017. Mesosphere added it. The argument was over in about two years.

4. The one idea

You declare desired state. Controllers continuously make reality match it.

Not "start three copies" but "three copies should exist". The difference is everything.

loop forever:
    desired = read from the API server
    actual  = observe the world
    if desired != actual:
        act to close the gap

This is a control loop, and the same shape appears in a thermostat. You do not tell a thermostat to switch on the heating. You tell it the room should be 20 degrees. It compares, acts, and keeps comparing forever. When somebody opens a window, nothing needs to notify it — the gap reappears and it acts again.

Everything Kubernetes does is this loop, applied to different objects:

• Kill a pod and one appears. The gap reappeared.
• A machine dies and its pods are recreated elsewhere. The gap reappeared.
• A rolling update is a controller moving from one desired state to another in steps, watching health as it goes.
• This is why kubectl delete pod feels like it does nothing — you deleted an instance, not the declaration that created it.
• And it is why GitOps works: put the declaration in Git, point a controller at Git, and deployment becomes a merge (Chapter 13.6.14).

The imperative contrast makes the value obvious. An imperative system executes your command once. If it half-succeeds, you are left with a half-state that nobody is responsible for fixing, and the next command assumes a starting point that may not exist. A declarative system has no half-states to own — there is only the gap, and something is always closing it.

5. The gap list, answered

Every problem from section 2, and what Kubernetes provides for it.

The problem	What Kubernetes gives you
Which machine?	Scheduler — filters nodes that fit, scores the rest, places the pod
Host dies	Controllers notice missing pods and recreate them elsewhere
Where is the truth?	etcd, a replicated consistent store, behind one API server
Changing IPs	Services — a stable name and address in front of a moving set of pods
Deploy without a gap	Deployment controller — rolling update, health-gated, with rollback
Traffic doubled	HPA for more pods, Cluster Autoscaler for more machines
Is it healthy?	Probes — readiness controls traffic, liveness restarts
Configuration	ConfigMaps and Secrets, mounted or injected, versioned as objects
Storage that follows a pod	PersistentVolumeClaims and CSI drivers
Node maintenance	Drain plus PodDisruptionBudgets, so upgrades do not take a service down
Who may do what	RBAC, namespaces, quotas, network policies
Don't put all copies on one rack	Topology spread and anti-affinity

And one more, which is the deeper reason large organisations adopt it: a uniform API across every environment. The same manifests describe the same system on a laptop, in Azure, in AWS, and in a data centre. Teams stop learning one deployment system per environment.

6. What Kubernetes does not give you

Being honest here is what separates useful advice from enthusiasm, and every item is a real cost somebody has to carry.

It is not a platform-as-a-service. There is no git push to deploy. Kubernetes gives you the primitives; the developer experience on top is something you build or buy.

It does not build or test your code. CI is a separate system (Chapter 13.7).

Its secret handling is weak by default. Secrets are base64-encoded, not encrypted, until you enable encryption at rest and restrict access. Most teams end up with an external secret store (Chapter 13.6.14).

It gives you no observability. No metrics, no dashboards, no log aggregation, no tracing. All of that is added.

It does not manage state well on its own. Databases in Kubernetes work, with a good operator, and a managed database is still the right default for most teams.

It does not control cost. It makes it easy to run a lot of things, which makes it easy to spend a lot. Requests that are set too high waste capacity invisibly across every node.

And it is genuinely hard. Networking, storage, upgrades, security policy, resource tuning, and a vocabulary of about sixty object types. Realistic time to competence is months, not weeks, and running the control plane yourself is a specialised job that almost nobody should take on — which is precisely why managed services exist, and why Chapter 13.6.15 is about one.

7. So when should you not use it?

You probably do not need it if you run a handful of services, have one or two teams, have no platform team, and no unusual scheduling requirements. Managed container services — Azure Container Apps, AWS App Runner, Google Cloud Run — give you rolling deploys, autoscaling, health management, HTTPS and scale-to-zero with no cluster to own (Chapter 13.3). For a team of eight running six services, that is the correct answer and it is not a compromise.

You probably do need it if you run many services across several teams, need advanced scheduling (GPUs, specific hardware, batch alongside serving), want one operational model across clouds and on-premises, need per-team isolation with quotas and policy, or depend on ecosystem tooling that assumes it.

The failure mode to avoid is adopting it because it is where you expect to end up eventually. Teams that start there spend their first year operating a cluster instead of shipping product.

And the reason that mistake is cheap to avoid: migrating to Kubernetes later is genuinely straightforward, because the container is the same artefact. The image you run on a managed container service is the image a pod runs. What you write later is a manifest, not a rewrite. Deferring the decision costs you almost nothing; making it early costs you a year.

What the interviewer will push on

"Why isn't Docker enough?" Because a Docker daemon manages one machine, so every fleet-level question has no answer inside it: placement, rescheduling after a host failure, service discovery across changing IPs, health-gated rolling updates, and scaling. The answer that lands walks the derivation — restart policies fail when the host is gone, a health script needs a scheduler, a placement table needs a consistent store — because it shows you understand orchestration as a set of problems rather than a product.

"What is the core idea of Kubernetes?" Declarative desired state plus control loops that reconcile forever. Then the consequences: self-healing and rolling updates are the same mechanism, kubectl delete pod does nothing lasting, and GitOps is just pointing the loop at a repository. The weak answer lists features; the strong one derives the features from the loop.

"Where did it come from?" Google's Borg, built from around 2003 and described publicly in 2015, via the Omega research system. Kubernetes was announced in 2014 and given to the CNCF at 1.0 in 2015. Naming what Borg contributed — declared state, central scheduling, requests and limits, packing batch alongside serving — is more valuable than the dates.

"Why did it beat Swarm and Mesos?" An extensible API where your own resource types behave like built-in ones, the declarative model, and vendor neutrality under the CNCF that let every cloud adopt it without helping a competitor. Swarm was simpler but less capable and late; Mesos was more general and one abstraction too high.

"When would you not use Kubernetes?" A handful of services, one or two teams, no platform team, nothing unusual to schedule. Managed container services cover it. The point that matters most: migrating later is easy because the container is the same artefact, so the cost of waiting is near zero and the cost of adopting early is a year of operating instead of building.

"What does Kubernetes not solve?" No CI, no developer experience, weak secrets by default, no observability, no cost control, and databases still want a managed service. Naming these is what makes an advocate credible rather than enthusiastic.

One thing to volunteer: point out that Kubernetes makes it easy to run many things, and therefore easy to spend a great deal of money without noticing — over-set resource requests reserve capacity nobody uses, on every node, forever. Teams discover this in month six of a migration. Mentioning it signals that you have operated a cluster rather than deployed to one.

Recall

• One Docker daemon manages one machine. Every fleet question — placement, rescheduling, discovery, rolling updates, scaling — is outside its scope by design.
• Patch the gap yourself and you rebuild six components: reconcile loop, scheduler, consistent store, service registry, deployment controller, autoscaler. Everyone who tried arrived at the same six.
• Lineage: Google Borg (~2003, paper 2015) gave declared state, central scheduling, requests and limits, and packing batch with serving. Omega (2013) gave the shared-state design. Kubernetes announced 2014, 1.0 and CNCF in 2015.
• It beat Swarm (simpler, weaker, late) and Mesos (more general, too abstract) on an extensible API, the declarative model, and vendor neutrality that let every cloud adopt it.
• The one idea: declare desired state; controllers reconcile forever. A thermostat, not a switch. Self-healing, rolling updates and GitOps are all the same loop — and it is why kubectl delete pod does nothing lasting.
• The gap list answered: scheduler · controllers · etcd behind one API server · Services for stable addressing · Deployments for health-gated rollout · HPA and Cluster Autoscaler · probes · ConfigMaps and Secrets · PVCs · drain plus PodDisruptionBudgets · RBAC and quotas · topology spread.
• It does not give you: a PaaS, CI, strong secrets by default, observability, good stateful defaults, or cost control. Competence takes months.
• Do not adopt it early "because we will need it". Managed container services cover a handful of services well, and migration later is easy because the container is the same artefact.

Self-test: Why does a restart policy not survive a host failure? · Which six components does anyone patching around Docker eventually build? · What did Borg contribute that you still use every day? · State the control loop in one sentence and derive self-healing from it. · Name three things Kubernetes does not provide. · What makes deferring the Kubernetes decision cheap?

Next: 13.6.9 opens the control plane. Five components, one datastore, and the exact path a kubectl apply takes from your terminal to a running container — including the four checkpoints it must pass before it is even written down.