13.6.3 — Docker's Components, Piece by Piece

Restart the Docker daemon on a busy server and something surprising happens: the running containers keep running. Logs keep flowing, requests keep being served. The daemon that started them is gone and they do not care.

Stop containerd, though, and everything dies.

That one experiment tells you Docker is not a program. It is a stack of four programs handing work down to each other, each with a different job and a different lifetime, and the reason a daemon restart is survivable is a small process called a shim that sits between them. This page walks the whole chain, because when a container is stuck in Created forever or an image pull hangs, the first useful question is always which of these four is stuck.

1. The chain, end to end

Figure 1 — What docker run actually touches. Four processes, three protocols. The CLI never runs a container; dockerd never runs a container either. Only runc does, and it exits immediately afterwards, leaving the shim as the container's parent.

2. docker — the client

The docker command is an HTTP client and nothing else. It builds a request, sends it to the daemon, prints the response.

You can prove it with curl, since the daemon speaks plain HTTP over a Unix domain socket:

curl --unix-socket /var/run/docker.sock http://localhost/v1.44/containers/json   # (1)
curl --unix-socket /var/run/docker.sock -X POST \
     "http://localhost/v1.44/containers/abc123/start"                            # (2)

(1) The exact request docker ps makes. The response is a JSON array of running containers.

(2) The exact request docker start abc123 makes.

Two things follow from the client being a network client.

DOCKER_HOST points it somewhere else. Set DOCKER_HOST=ssh://user@buildserver and every docker command in that shell operates on a remote machine, with the images and containers living there. This is how remote build machines and CI runners are wired up, and it is the whole feature.

And the socket is root. Anyone who can write to /var/run/docker.sock can ask the daemon to start a container that mounts the host's / and runs as root. Access to the Docker socket is equivalent to root on the machine, with no exceptions and no clever mitigations. This is why membership of the docker group is a privilege decision, not a convenience one, and why mounting the socket into a container "so it can build images" is one of the most common serious misconfigurations in CI systems (Chapter 8.6.2).

3. dockerd — the daemon

The long-running process that does the management work, as root. Its responsibilities are broader than people expect:

Serving the API. Everything the CLI, Docker Compose, an IDE plugin, or Testcontainers asks for arrives here.

Images and registry authentication. It holds your registry credentials, decides what needs pulling, and talks to registries over HTTPS.

Building. A docker build is handled here and handed to BuildKit.

Networking. Creating bridges, allocating container IPs, writing the iptables rules that publish a port, and running the embedded DNS server that makes container names resolve (Chapter 13.6.6).

Volumes. Creating, mounting and tracking named volumes and their drivers.

Logging. Collecting each container's standard output and standard error and passing it to the configured log driver.

What it does not do is run containers. It asks containerd to.

Why the daemon can restart without killing containers

Because it is not the parent of any container process. The shim is. When dockerd stops, the shims keep holding the containers' input and output pipes and keep waiting for their exit codes. When dockerd starts again, it reconnects to containerd, containerd reconnects to the shims, and management resumes.

This is configurable — live-restore in /etc/docker/daemon.json — and it is the reason a Docker upgrade on a production host is not automatically an outage.

The daemon has been criticised for years for doing too much in one root process, and that criticism is the reason for two things you will meet: containerd being split out into its own project, and Podman existing at all (section 9).

4. containerd — the container supervisor

Extracted from Docker in 2016 and donated to the CNCF in 2017, containerd is the piece that actually manages container lifecycle. It is deliberately narrow: it does containers, not developer experience.

Its jobs:

Image management. Pulling from registries, verifying digests, storing content by hash in a content-addressable store.

Snapshots. Turning image layers into a mounted root filesystem via OverlayFS. The "snapshotter" is the component that does this and it is pluggable.

Container lifecycle. Create, start, stop, delete, pause, exec.

Handing off to a low-level runtime to do the actual creation.

Two facts make containerd matter far beyond Docker:

Kubernetes talks to containerd directly. Since Kubernetes 1.24 (2022), the kubelet speaks the Container Runtime Interface (CRI) and containerd implements it natively. Docker is not installed on a modern Kubernetes node. This is what the "Kubernetes is deprecating Docker" headlines of 2020 actually meant, and it caused a great deal of unnecessary alarm: what was removed was dockershim, the adapter Kubernetes maintained to speak to dockerd. Images built by Docker continued to work perfectly, because the image format is an OCI standard, not a Docker product.

You can drive containerd directly with ctr (its low-level debugging tool) or nerdctl (a Docker-compatible CLI). On a Kubernetes node, crictl is the tool that speaks CRI and is what you use when a pod will not start and you need to see what the runtime thinks (Chapter 13.6.10).

5. containerd-shim — the small process that matters

One shim process per container, and it exists to solve a specific problem: somebody has to be the container's parent, hold its pipes, and collect its exit code, and that somebody must not be a process you ever want to restart.

Look at the process tree on a host running one container:

systemd
├─ dockerd
├─ containerd
└─ containerd-shim-runc-v2 -namespace moby -id 3f2a...
   └─ node server.js          ← the container

Note what is not there: the container is not a child of dockerd, and not a child of containerd either. The shim was started by containerd and then re-parented to systemd, deliberately, so that it survives everything above it.

What the shim does:

• Keeps the standard input, output and error pipes open, so log collection survives a daemon restart.
• Waits for the container's exit code and reports it upward whenever there is somebody to report to.
• Keeps the container's terminal alive for docker attach.
• Reaps the process so it does not become a zombie.

It is a few megabytes of memory per container, and it is the reason live-restore is possible at all.

6. runc — the thing that actually makes the container

runc is a small program that reads a JSON file and creates a container. It runs for a few milliseconds and exits.

It is the reference implementation of the OCI Runtime Specification, donated by Docker in 2015. The JSON file it reads — config.json — is that specification made concrete: the root filesystem path, the process to run, the namespaces to create, the cgroup limits, the capabilities, the seccomp profile, the mounts.

You can use it by hand, which is the clearest way to see that Docker's magic is a config file:

mkdir -p /tmp/demo/rootfs && cd /tmp/demo
docker export $(docker create alpine) | tar -x -C rootfs    # (1)
runc spec                                                    # (2)
runc run demo                                                # (3)

(1) A root filesystem, exactly as in Chapter 13.6.2.

(2) Generates a default config.json in the current directory. Open it: it lists the namespaces to unshare, the capability set, the seccomp rules, the mounts for /proc and /sys, and the command to run. This file is the complete definition of a container.

(3) Creates the namespaces, applies the cgroups, pivots the root, drops privileges and executes the process. You get a shell in Alpine. No Docker daemon was involved.

Because the runtime is a specification, it is swappable, and the alternatives are real production choices:

Runtime	What it is	Use it when
runc	The default, native speed	Almost always
crun	Same job, written in C	Faster start, lower memory; the default on Podman
gVisor (runsc)	A user-space kernel intercepting system calls	Untrusted code, accepting a performance cost
Kata Containers	Each container in a tiny VM	Hard multi-tenant isolation
Wasm runtimes	WebAssembly instead of Linux processes	Tiny, fast-starting, sandboxed workloads

gVisor and Kata are the answer to "containers share a kernel". gVisor puts a re-implementation of the Linux system call surface, written in Go, between the container and the real kernel. Kata puts an actual lightweight VM around each container. Both are configured as a runtime class and applied per workload, which is how a cloud provider runs your untrusted code next to somebody else's.

7. BuildKit — the build engine

docker build has been BuildKit since Docker 23, and it is a genuinely different engine from the sequential builder that came before.

The old builder ran instructions one at a time, top to bottom, producing a container per step. BuildKit parses the Dockerfile into a graph of operations, works out what depends on what, and then:

Runs independent stages in parallel. Two build stages that do not depend on each other build simultaneously.

Skips anything unreachable. If the final stage does not use stage 3, stage 3 never runs.

Caches at the operation level, and the cache is exportable. --cache-to and --cache-from let a CI runner push its build cache to a registry and the next runner pull it, so a fresh machine gets warm-cache build times (Chapter 13.7).

Adds mounts that exist only during the build, which is where two important features come from:

RUN --mount=type=cache,target=/root/.npm npm ci        # (1)
RUN --mount=type=secret,id=token cat /run/secrets/token  # (2)

(1) A cache mount. The package manager's download cache persists between builds but is not part of any layer, so builds get faster and the image does not grow.

(2) A secret mount. The secret is available at that path while the instruction runs and is in no layer afterwards. This is the only correct way to use a credential during a build, and Chapter 13.6.5 explains why the obvious alternatives leak.

docker buildx is the CLI front end that exposes BuildKit's extra features: multi-platform builds, remote builders, and alternative outputs. The multi-platform case is now routine, because ARM laptops and ARM servers are both common:

docker buildx build --platform linux/amd64,linux/arm64 -t registry/api:1.4 --push .

This produces one tag that resolves to two images, via a manifest list (Chapter 13.6.4). The client pulling it gets the one matching its architecture, automatically.

8. The pieces around the edges

The registry. A server that stores images and serves them over an HTTPS API. Docker Hub, Azure Container Registry, Amazon ECR, GitHub Container Registry, Harbor, or the plain registry:2 image you can run yourself in one command. Chapter 13.6.4 covers the protocol.

Storage drivers. How the daemon stacks layers on disk. overlay2 is the answer on every modern Linux system and you should not change it. The older devicemapper, aufs and btrfs drivers exist for historical reasons and each brought its own performance surprises. Check with docker info | grep Storage.

Network drivers. bridge (the default), host, none, overlay (multi-host, for Swarm), macvlan (a real MAC address on the physical network), plus third-party plugins. Chapter 13.6.6 works through each.

Volume drivers. local by default; plugins add network storage, cloud disks and NFS.

Logging drivers. json-file by default, which writes each container's output to a JSON file on the host. The default has no size limit, which is a genuine way to fill a disk, so setting max-size and max-file is a standard production step. Alternatives send logs straight to journald, syslog, fluentd, or a cloud service.

Docker Compose. A separate tool that reads a YAML file and makes a sequence of API calls to the daemon. It has no special powers — everything it does you could do with docker commands (Chapter 13.6.7).

Docker Desktop. On macOS and Windows, a Linux virtual machine plus a management interface. Containers need a Linux kernel, and a Mac does not have one, so Desktop runs a small Linux VM and puts your containers inside it. This explains the surprises: bind-mounted source directories are slow because file operations cross a VM boundary; the VM has a fixed memory allocation you may need to raise; and localhost port publishing involves an extra forwarding hop. On Windows you can also run Windows containers, which share the Windows kernel and can only run Windows images.

9. The standards, and the alternatives they made possible

The Open Container Initiative publishes three specifications, and they are why none of this is locked to one vendor:

The Runtime Specification — what a runtime must do with a root filesystem and a config.json. Implemented by runc, crun, gVisor, Kata.

The Image Specification — what an image is: layers, a configuration object, a manifest, all content-addressed by digest. Implemented by everyone.

The Distribution Specification — the HTTPS API a registry serves.

Kubernetes adds one of its own, the Container Runtime Interface (CRI) — the gRPC interface the kubelet uses to ask a runtime for pods and containers.

Because of these, the ecosystem is genuinely plural:

Tool	What it replaces	The interesting difference
Podman	docker and dockerd	No daemon. Runs containers as child processes, works fully rootless, drop-in CLI
Buildah	docker build	Builds images without a daemon and without a Dockerfile if you prefer scripts
Skopeo	docker pull/push	Moves and inspects images between registries without a local daemon
nerdctl	docker	Docker-compatible CLI straight onto containerd
CRI-O	containerd, for Kubernetes	A runtime that does only what Kubernetes needs, nothing more

Podman's no-daemon design is the substantive one. There is no root process holding everything; each container is a child of the command that started it, and rootless mode means the whole stack runs as your user with a user namespace mapping you to root inside. The trade is that features which need a long-running manager — such as restart policies surviving a logout — are handled by generating systemd units instead.

All of them produce and consume the same images. An image built by Buildah runs under Docker, and an image built by Docker runs under CRI-O in Kubernetes. That interoperability is the point of the standards, and it is why "Kubernetes removed Docker" changed nothing about how you build.

10. Which component is broken?

This is the payoff for knowing the chain, and it is the fastest diagnostic table in this Part.

Symptom	Component	What to run
Cannot connect to the Docker daemon	dockerd is down, or you lack socket permission	systemctl status docker
Pull hangs or is slow	dockerd network, registry, or rate limit	docker pull with --debug, check credentials
no space left on device	Host disk full of images and layers	docker system df, then docker system prune
Container stuck in Created	containerd or runc failed to start it	journalctl -u containerd
exec format error	Image built for another CPU architecture	docker inspect the image's architecture
Container dies on daemon restart	live-restore is off	Set it in daemon.json
Build is slow, cache never hits	BuildKit cache invalidated by instruction order	docker build --progress=plain
Disk fills over weeks	json-file logs with no rotation	Set max-size in daemon.json

docker system df deserves a habit. It shows how much disk is held by images, containers, volumes and the build cache separately, and it is almost always the build cache or dangling images that has eaten the disk on a build machine.

What the interviewer will push on

"Walk me through what happens when you type docker run nginx." CLI sends an HTTP request over the Unix socket; dockerd resolves the image and pulls if needed; containerd unpacks layers and prepares an OverlayFS snapshot; a config.json is written; a shim is started; runc reads the config, creates namespaces and cgroups, pivots the root, drops capabilities and execves the process; runc exits and the shim remains as the parent. The detail that shows real understanding is runc exiting — people usually assume something supervises the process continuously.

"Why does restarting the Docker daemon not kill running containers?" Because containers are children of their shims, not of the daemon, and shims are re-parented to init. The daemon reconnects through containerd when it comes back. Naming live-restore as the setting is the credible extra.

"What did 'Kubernetes deprecates Docker' actually mean?" Kubernetes removed dockershim, the adapter that let the kubelet talk to dockerd, and now talks to containerd or CRI-O directly through the CRI. Images were never affected, because the image format is an OCI standard. This question is really testing whether you panicked at a headline.

"Why is access to the Docker socket equivalent to root?" Because the daemon runs as root and will do whatever the API asks, including starting a container that bind-mounts the host's root filesystem with full privileges. There is no permission model inside the API to prevent it. The follow-up is usually about CI systems mounting the socket into build containers, and the answer is rootless BuildKit, Buildah, or a dedicated build service.

"What is containerd and why was it split out?" Because a single root daemon doing image management, builds, networking, volumes, logging and container lifecycle is too much in one place, and because Kubernetes needed a runtime without Docker's developer-experience layer. containerd does images, snapshots and lifecycle; that narrowness is what let it become the industry default underneath both Docker and Kubernetes.

"Podman versus Docker?" Podman has no daemon and runs rootless properly, so each container is a child of your command and a compromise does not hand over a root daemon. The trade is that anything needing a long-running supervisor is handed to systemd instead. Both produce identical OCI images.

One thing to volunteer: mention that the default json-file log driver has no rotation limit, so a chatty container will fill a host disk over weeks and take every other container on that host down with it. It is a two-line change in daemon.json and it is the most boring outage in this Part.

Recall

• Four programs, three protocols. docker (HTTP client over a Unix socket) → dockerd (API, builds, networking, volumes, logs, registry auth) → containerd (images, snapshots, lifecycle, CRI) → containerd-shim (one per container, holds pipes and exit code) → runc (creates the container, then exits).
• The daemon restarts without killing containers because the shim, not the daemon, is the parent — enabled by live-restore. Killing containerd kills everything.
• Write access to /var/run/docker.sock is root on the host. No mitigation. Mounting it into CI containers is a serious misconfiguration.
• runc reads config.json — the OCI runtime spec made concrete. runc spec generates one. Swappable: crun (faster), gVisor (user-space kernel), Kata (VM per container) for untrusted code.
• BuildKit builds a graph, runs stages in parallel, skips unreachable ones, exports cache to a registry, and provides --mount=type=cache and --mount=type=secret. buildx adds multi-platform builds via manifest lists.
• Storage driver overlay2 on any modern host. Default log driver json-file has no size limit — set max-size. Docker Desktop on Mac/Windows is a Linux VM, which is why bind mounts are slow.
• Three OCI specs — runtime, image, distribution — plus Kubernetes' CRI. This is why removing dockershim changed nothing about images, and why Podman, Buildah, Skopeo, nerdctl and CRI-O all interoperate.
• Diagnose by component: cannot connect = dockerd · stuck in Created = containerd/runc · exec format error = wrong architecture · disk full = docker system df.

Self-test: Which process is a running container's parent, and why does that matter? · What exactly was deprecated when Kubernetes "dropped Docker"? · Why is the Docker socket a root credential? · What does runc do, and how long does it live? · Which two BuildKit mounts exist only during the build, and what is each for? · Why are bind mounts slow on a Mac?

Next: 13.6.4 takes apart the artefact all of this moves around — what an image is on disk and in a registry, why a digest is not a tag, and how a pull decides which bytes it already has.