13.6.2 — What a Container Really Is

Run a container and then look at the host's process list:

docker run -d --name api node:22-slim node -e "setInterval(()=>{},1000)"
ps -ef | grep node

root      48211  48190  0 14:02 ?  00:00:00 node -e setInterval(()=>{},1000)

There it is. A perfectly ordinary process on the host, with an ordinary process ID. No virtual machine, no emulator, nothing between it and the CPU. If you kill 48211 from the host, the container dies.

There is no such thing as a container in the Linux kernel. Search the kernel source and you will not find a container object, a container system call, or a container data structure. What exists is a set of independent features that, used together, make a normal process believe it is alone on a machine. Docker's job is to switch those features on in the right order.

This page is the ingredient list, and by the end of it you will have built a container by hand with no container software at all.

1. The three ingredients

Ingredient	Kernel feature	What it gives you
What the process can see	Namespaces	Isolation
What the process can use	cgroups	Limits and accounting
What the process runs on	Union filesystem	The image

Add a fourth, security hardening — capabilities, seccomp, AppArmor or SELinux — and you have a container. Everything Docker does is the assembly of these four.

2. Namespaces: giving a process its own reality

A namespace is a kernel-level trick that changes what a set of processes can see. Normally every process on a machine shares one process list, one network stack, one set of mount points, one hostname. A namespace makes a private copy of one of those, and processes inside it see only the copy.

The key insight, and the one worth holding on to: the process is not moved anywhere and it is not restricted from doing anything. It is simply shown a different answer to a question. "What processes exist?" gets a different answer inside a PID namespace. "What is at /usr/lib?" gets a different answer inside a mount namespace. The process is not lied to by an emulator — the kernel genuinely keeps separate bookkeeping for it.

There are eight namespaces. Here is each one, what it hides, and where you feel it.

PID namespace — its own process numbering

Inside, your process is PID 1. It sees itself and its children and nothing else. The host still sees it as PID 48211 — one process, two numbers, depending on who is looking.

docker run --rm alpine ps aux

PID   USER     COMMAND
    1 root     ps aux

One process on the whole machine, according to the container. The host is running four hundred.

This matters enormously and is the source of real production bugs, because PID 1 in Linux is special. It is meant to be the init system, so the kernel treats it differently:

• PID 1 does not get the default signal handlers. For any other process, a SIGTERM with no handler installed means "terminate". For PID 1, a SIGTERM with no handler means nothing happens at all. This is why so many containers take exactly the full grace period to stop and then get killed — the orchestrator sends SIGTERM, the application never installed a handler, and being PID 1 means the default kill does not apply.
• PID 1 must reap orphans. When any process's parent dies, the orphan is re-parented to PID 1, which is expected to call wait() on it and clear its entry from the process table. An application that is not written to be init does not do this, and dead children accumulate as zombies until the process table fills.

Chapter 13.6.6 shows both fixes: handle SIGTERM in your code, and use --init when you need a real init process.

Mount namespace — its own filesystem tree

The oldest namespace (Linux 2.4.19, 2002) and the one that carries the image. The container gets its own set of mount points, so /, /usr, /etc and everything else can be completely different from the host's.

This is the answer to the dependency problem from Chapter 13.6.1. The container's /usr/lib/libssl.so is the one from its image. The host's is the host's. Neither can see the other, so neither can break the other.

Network namespace — its own network stack

Not just its own IP address. Its own everything: interfaces, routing table, firewall rules, socket table, /proc/net. A fresh network namespace contains exactly one interface, lo, and it is down.

docker run --rm alpine ip addr

1: lo: <LOOPBACK,UP> mtu 65536
    inet 127.0.0.1/8 scope host lo
2: eth0@if42: <BROADCAST,MULTICAST,UP> mtu 1500
    inet 172.17.0.3/16 scope global eth0

eth0 here is one end of a veth pair — a virtual Ethernet cable with two ends, where whatever goes in one end comes out the other. One end lives in the container's namespace and is called eth0; the other end lives on the host and is plugged into a bridge. That is the whole of Docker's default networking, and Chapter 13.6.6 draws it.

Because the namespace has its own port space, two containers can both listen on port 8080 with no conflict. Neither is "really" on the host's port 8080 unless you publish it.

UTS namespace — its own hostname

The smallest one. It isolates the hostname and the domain name, which is why hostname inside a container returns the container ID rather than the machine's name. ("UTS" is UNIX Time-sharing System, the name of the kernel structure that happens to hold these two fields — the name means nothing useful.)

IPC namespace — its own shared memory

Isolates System V inter-process communication and POSIX message queues: shared memory segments, semaphores, queues. Two processes can only use shared memory to talk if they are in the same IPC namespace. This is the namespace you deliberately share when you want two containers to use shared memory, which some databases and scientific tools need.

User namespace — its own idea of who root is

The most security-relevant and the most confusing. It maps user and group IDs between inside and outside. UID 0 inside the namespace can be mapped to UID 100000 on the host.

The consequence is worth stating plainly: the process believes it is root, has all the powers of root over its own container, and is an unprivileged nobody as far as the host is concerned. If it escapes, it escapes as user 100000, who owns nothing.

This is not on by default in Docker, for compatibility reasons that have to do with file ownership on mounted volumes. It is worth turning on, and it is the foundation of "rootless" containers where the daemon itself does not run as root.

Cgroup namespace — its own view of its limits

Hides the host's control-group hierarchy so a container sees its own cgroup as the root. Without it, a process could read the host's entire resource-control tree. It also matters for a subtler reason covered in section 4: it affects what a runtime library sees when it asks how much memory it has.

Time namespace — its own clock offsets

The newest (Linux 5.6, 2020), rarely used. It lets a container have a different boot time and monotonic clock, mainly to make checkpoint and restore of a running container work correctly.

Figure 1 — One process, eight private views. The process in the middle is a normal host process. Each namespace changes the answer to one question the process can ask the kernel. Nothing is emulated and nothing is copied; the kernel just keeps separate bookkeeping per namespace.

3. Build a container by hand

Namespaces are created with two system calls: clone() when starting a new process, and unshare() to move the current process into fresh ones. The unshare command-line tool exposes this directly, so you can build a container with tools that ship with any Linux distribution.

Run this on a Linux machine as root:

mkdir -p /tmp/mycontainer                                        # (1)
docker export $(docker create alpine) | tar -x -C /tmp/mycontainer   # (2)

unshare --pid --mount --uts --ipc --net --fork --mount-proc \    # (3)
  chroot /tmp/mycontainer /bin/sh                                # (4)

(1) A directory that will become the container's root filesystem.

(2) We need a root filesystem from somewhere, and the fastest honest way to get one is to export Alpine Linux's from an image. docker create makes a container without starting it; docker export streams its filesystem as a tar archive; tar -x unpacks it. You now have a complete miniature Linux userland — /bin, /etc, /usr — in a directory, about 8 MB of it. Nothing is running.

(3) This is the container. unshare creates new namespaces and runs a command inside them. --pid gives fresh process numbering, --mount a fresh mount table, --uts a fresh hostname, --ipc fresh shared memory, --net a fresh network stack. --fork is required with --pid because the current process cannot change its own PID — a new child has to be born inside the new namespace to become PID 1. --mount-proc remounts /proc inside, so that tools reading process information see the new namespace's processes rather than the host's.

(4) chroot switches the root directory to our unpacked filesystem, then runs the shell inside it.

Inside that shell:

/ # ps aux
PID   USER     COMMAND
    1 root     /bin/sh
    2 root     ps aux

/ # hostname new-name && hostname
new-name

/ # ip addr
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN

/ # ls /usr/lib
libcrypto.so.3  libssl.so.3  ...

Read those four results carefully, because they are the entire lesson.

ps shows two processes on a machine running hundreds — the PID namespace. The hostname changed with no effect on the host — the UTS namespace. The network has one loopback interface and it is down, so this container has no connectivity at all until somebody builds it some — the network namespace. And /usr/lib holds Alpine's libraries even if the host is Ubuntu — the mount namespace plus the root filesystem.

You have built a container. What is missing compared to what Docker gives you is a real list, and it is what the rest of this Part covers: no resource limits (section 4), no image layering or sharing (13.6.4), no network wiring (13.6.6), no security profile (section 6), no lifecycle management, no logging, no way to ship it to another machine.

chroot is also not the real mechanism. Docker uses pivot_root, which detaches the old root entirely rather than merely changing where / points. chroot leaves the old root reachable in ways a determined root process can exploit, which is why "chroot is not a security boundary" has been true since 1979.

4. cgroups: what the process is allowed to use

Namespaces control what a process can see. They do nothing about what it can consume. A process in a perfect set of namespaces can still allocate every byte of memory on the machine and take every CPU cycle. Isolation without limits is not isolation.

Control groups (cgroups) are the kernel's mechanism for measuring and capping resource use per group of processes. They are exposed as a filesystem, which makes them unusually easy to inspect. Modern systems use cgroup v2, mounted at /sys/fs/cgroup, which unified what v1 spread across a dozen separate hierarchies.

Set a memory limit by hand:

mkdir /sys/fs/cgroup/demo                       # (1)
echo "100M" > /sys/fs/cgroup/demo/memory.max    # (2)
echo $$ > /sys/fs/cgroup/demo/cgroup.procs      # (3)

(1) Creating a directory creates a control group. The kernel populates it with control files immediately.

(2) Writing to a control file sets a limit. This one caps the group at 100 MB of memory.

(3) $$ is the current shell's PID, so this moves the shell — and everything it starts — into the group. From this moment, the shell and all its children together may use 100 MB.

Allocate 200 MB in that shell and the kernel's out-of-memory killer terminates the process. Not the machine's OOM killer choosing a victim across the whole system — the cgroup's, acting only within the group that broke its limit.

The controllers that matter for containers:

memory — memory.max is the hard ceiling; exceeding it means the process is killed, because you cannot politely slow down a memory allocation. memory.high is a softer throttle that puts the process under heavy reclaim pressure instead. memory.current reports live usage.

cpu — works on a quota over a period. cpu.max of 50000 100000 means "50,000 microseconds of CPU per 100,000 microsecond period", which is half a core. Exceeding the CPU limit does not kill anything, it throttles: the kernel stops scheduling the group until the next period begins. This distinction between memory and CPU produces two completely different production symptoms and Chapter 13.6.13 comes back to it hard.

io — read and write bandwidth and operation limits per block device.

pids — the maximum number of processes. This is the defence against a fork bomb, and it is one line: pids.max.

The two cgroup traps that bite real applications

Trap one: /proc/meminfo and nproc do not respect your cgroup. Those interfaces read the host's numbers, because /proc was designed decades before cgroups. So a Java process in a container limited to 512 MB, on a host with 64 GB, historically saw 64 GB, sized its heap as a percentage of that, and was killed the moment it tried to grow. The Java runtime has been container-aware since JDK 10 and now reads the cgroup limits, but the general problem remains for any runtime or library that asks the OS how big the machine is. Thread pools sized by "CPU count" are the other common victim: a container limited to half a core, on a 64-core host, happily creating 64 worker threads.

Trap two: the page cache counts against your memory limit. File data the kernel caches on your behalf is charged to your cgroup. An application that reads a large file can appear to be using far more memory than its heap suggests. The kernel will reclaim that cache under pressure rather than kill you, but it makes memory graphs confusing until you know.

5. The image: a filesystem built from stacked layers

The third ingredient. Our hand-built container used a plain directory, which works but shares nothing — a hundred containers would need a hundred copies.

A union filesystem presents several directories stacked on top of each other as if they were one directory. Linux's implementation is OverlayFS, and it has exactly three inputs:

• lowerdir — one or more read-only layers, stacked. These are the image.
• upperdir — one writable layer on top. This is the container.
• merged — the combined view, which is what the process sees as /.

The rules of the merged view:

Reads find the topmost copy. A file present in both a lower layer and the upper layer is served from the upper one.

Writes go to the upper layer only. Modifying a file that lives in a read-only layer triggers copy-up: the kernel copies the whole file into the upper layer first, then applies the change. The lower layer is untouched. This is why writing one byte into a 2 GB file inside a container copies 2 GB.

Deletes are recorded, not performed. You cannot delete from a read-only layer, so OverlayFS writes a whiteout marker into the upper layer — a special entry meaning "if you find this name below, pretend it is not there". This is the mechanism behind the single most important fact about image size: a later Dockerfile instruction that deletes a file adds a whiteout marker and shrinks nothing, because the data is still in the lower layer and still in the registry. Chapter 13.6.5 shows the consequences.

Figure 2 — OverlayFS, and why deleting does not shrink. Read-only image layers below, one writable layer per container above, one merged view for the process. The whiteout entry hides big.tar from the process without removing a single byte from the layer that contains it.

Two consequences worth carrying with you:

Layers are shared on disk. Fifty containers from the same image share one copy of every read-only layer. Each gets only its own small writable layer. This is why container density is so much better than VM density, and why pulling the fifth image from a registry is usually fast — most layers are already there.

The writable layer disappears when the container is removed. Anything written inside a container that is not on a mounted volume is gone. This is not a bug and it is not a limitation to work around — it is the property that makes containers replaceable, and it is why data lives in volumes (Chapter 13.6.6) or in a database somewhere else.

6. The security layer

Namespaces, cgroups and a filesystem give you isolation and limits. They do not give you a security boundary on their own, because a process running as root inside a container is talking to the same kernel as everything else on the machine. Three more mechanisms narrow what it can ask that kernel to do.

Capabilities split root's historically all-or-nothing power into about forty separate privileges: bind to a port below 1024 (CAP_NET_BIND_SERVICE), change file ownership (CAP_CHOWN), load a kernel module (CAP_SYS_MODULE), and so on. Docker drops most of them by default and keeps roughly fourteen. A container is "root" but cannot load kernel modules, change the system clock, or access raw devices.

--privileged switches every one of them back on, plus device access and a permissive seccomp profile. It is the flag people reach for when something does not work, and it very nearly removes the boundary. Almost every use of --privileged in the wild should be one or two specific --cap-add flags instead.

seccomp filters system calls. The kernel exposes over 300 of them; the great majority of applications use a few dozen. Docker's default profile blocks around 44, including whole families that only exist for kernel manipulation. This directly shrinks the attack surface: a kernel vulnerability in a system call your container is not allowed to make cannot be reached from your container.

AppArmor or SELinux add mandatory access control — rules about which paths and operations a labelled process may touch, enforced regardless of file permissions.

No privilege escalation. Setting no-new-privileges stops a process gaining privileges through a setuid binary, closing a classic escalation path.

The layered picture: namespaces decide what exists, cgroups decide how much, capabilities decide which privileged operations are possible, seccomp decides which system calls are reachable, and mandatory access control decides which paths are allowed. Chapter 8.6.2 works through hardening this properly.

7. So what happens when you type docker run?

Everything above, in order. This is the whole sequence, and Chapter 13.6.3 names which component performs each step.

1. The CLI sends a request over a socket. docker run nginx becomes an HTTP request to the Docker daemon over a Unix socket. The CLI is not running anything itself.
2. The daemon checks for the image locally. If it is missing, it contacts the registry, downloads the manifest, and pulls the layers it does not already have.
3. Layers are unpacked and stacked. Each is extracted into its own directory, and OverlayFS is asked to merge them with a fresh empty writable layer on top.
4. A configuration bundle is written: the merged root filesystem path, the command to run, environment variables, mounts, the namespace list, the cgroup limits, the capability set and the seccomp profile. This is an OCI runtime specification, a JSON file.
5. The low-level runtime creates the namespaces, with clone() carrying the flags for each one requested.
6. The cgroup is created and the process is placed in it, with the memory, CPU, and process-count limits written to its control files.
7. The root filesystem is switched with pivot_root to the merged directory.
8. Capabilities are dropped, the seccomp filter is installed, the user is set.
9. execve() runs your program. From here it is a normal Linux process, running at native speed, with no supervisor in the path.

Step 9 is the one to remember: there is no ongoing overhead. A containerised process does not run through a translation layer. The isolation was configured at startup and costs nothing per instruction afterwards. The only measurable runtime cost is in the storage driver for heavy filesystem work and in the network path if traffic crosses a bridge and gets translated.

What the interviewer will push on

"What is a container, at the kernel level?" A normal process with three things attached: namespaces for what it can see, cgroups for what it can use, and a union filesystem for what it runs on. The tell of a real answer is the sentence "the kernel has no container object" — it shows the person has looked rather than repeated a diagram. The weak answer describes Docker's user interface.

"Name the namespaces and say what each isolates." PID, mount, network, UTS, IPC, user, cgroup, time. If you can name five with their effect, that is credible. Then add the one that pays: the user namespace, because it is what makes root inside not root outside, and note that it is off by default in Docker.

"Why does my container ignore SIGTERM?" Because the process is PID 1 and PID 1 does not get default signal handlers, so an unhandled SIGTERM does nothing at all until the grace period expires and SIGKILL arrives. The follow-up is usually about the shell form of CMD making sh PID 1 — Chapter 13.6.5.

"Why did my Java application get OOM-killed when the heap limit was well under the container limit?" Because /proc/meminfo reports the host's memory, not the cgroup's, so a runtime sizing itself from "available memory" sizes for the wrong machine. Modern JVMs read cgroup limits, but any library asking the OS how big the machine is has the same bug. Mentioning that thread pools sized by CPU count fail the same way is what makes this answer land.

"Why does deleting a file in a later Dockerfile step not make the image smaller?" Because a union filesystem cannot delete from a read-only layer, so it writes a whiteout marker that hides the name. The bytes remain in the earlier layer and are still pulled on every deploy. Fix by deleting in the same instruction that created the file, or with a multi-stage build.

"What does --privileged actually do, and why is it dangerous?" It restores all capabilities, allows access to all host devices, and relaxes the seccomp profile — which is most of the boundary. Almost every real need is one specific capability, so the good answer names --cap-add NET_ADMIN as the alternative.

One thing to volunteer: point out that the isolation is configured once at process creation and costs nothing afterwards, so a containerised process runs at native speed. People frequently assume a container is a small virtual machine with ongoing overhead, and knowing that execve is the last step of container creation is what makes the performance conversation correct.

Recall

• The kernel has no container object. A container is a process plus namespaces (what it sees) plus cgroups (what it uses) plus a union filesystem (what it runs on), plus capabilities, seccomp and MAC for hardening.
• Eight namespaces: PID · mount · network · UTS · IPC · user · cgroup · time. Mount carries the image and ends dependency conflicts; network gives a private port space; user maps root inside to an unprivileged UID outside and is off by default in Docker.
• PID 1 has no default signal handlers and must reap orphans — the source of "it always takes 30 seconds to stop" and of zombie build-up.
• cgroup v2 is a filesystem: mkdir makes a group, writing memory.max sets a cap, writing a PID into cgroup.procs moves a process in. Memory over limit kills; CPU over limit throttles.
• /proc/meminfo and CPU count report the host, so runtimes and thread pools size themselves for the wrong machine. The page cache also counts against the memory limit.
• OverlayFS: read-only lowerdir layers, one writable upperdir per container, one merged view. Writes trigger copy-up of the whole file; deletes write a whiteout marker and free nothing. Layers are shared across containers on disk; the writable layer dies with the container.
• Hardening layers: capabilities split root into ~40 privileges (Docker keeps ~14) · seccomp blocks ~44 system calls by default · AppArmor/SELinux add path rules · no-new-privileges blocks setuid escalation. --privileged undoes nearly all of it.
• docker run in order: pull · unpack and stack layers · write the OCI config · clone() the namespaces · create the cgroup · pivot_root · drop capabilities and install seccomp · execve. After that it is a normal process at native speed.

Self-test: Which namespace ends the libssl conflict, and how? · Why does unshare --pid need --fork? · What is the observable difference between breaking a memory limit and a CPU limit? · What exactly does a whiteout entry do to image size? · Why is chroot not a security boundary and what does Docker use instead? · Where does the runtime overhead of a container come from?

Next: 13.6.3 opens the toolbox. Typing docker run sets off a chain through four separate programs — the CLI, the daemon, containerd, and runc — and knowing which one does what is the difference between guessing and diagnosing when something is stuck.