13.6.7 — Docker Compose, Key by Key

A new engineer joins on Monday. The onboarding document is eleven pages: install PostgreSQL 14 (not 15, the migrations break), install Redis, install RabbitMQ, create these three databases, run these seeds, set these nine environment variables, start the four services in this order.

By Wednesday afternoon they have it working, except their PostgreSQL is 16 because that is what the package manager offered, and a date-handling test fails in a way nobody else can reproduce.

With Compose the onboarding document is one line:

docker compose up

Same PostgreSQL version as production, same Redis, same broker, same seeds, started in the right order, thrown away with docker compose down. This page is every key in that file and what each one actually does — plus the one behaviour that catches everybody, which is that depends_on does not mean what it looks like it means.

1. What Compose is

A program that reads a YAML file and makes Docker API calls. It has no special powers. Every docker compose command maps onto commands you could type yourself; the file just records them so nobody has to remember.

What it gives you concretely:

• One command starts a whole set of services with their networks and volumes.
• A private network created automatically, so service names resolve as hostnames (Chapter 13.6.6).
• A project name that namespaces everything, so two projects on one machine do not collide.
• Lifecycle — start, stop, rebuild, tail logs across all services at once.

And to say the limit up front so nothing on this page is misleading: Compose runs on one machine. No scheduling across servers, no self-healing when a server dies, no rolling updates. Chapter 13.6.8 is about exactly that gap.

2. A complete file

This is a realistic local environment for a checkout service: an API, a background worker, a database, a cache and a message broker.

name: checkout                                    # (1)

services:
  api:
    build:                                        # (2)
      context: .
      dockerfile: Dockerfile
      target: dev                                 # (3)
      args:
        NODE_VERSION: "22"
    image: checkout-api:dev                       # (4)
    ports:
      - "3000:3000"
      - "9229:9229"                               # (5)
    environment:                                  # (6)
      NODE_ENV: development
      DATABASE_URL: postgres://app:app@db:5432/checkout
      REDIS_URL: redis://cache:6379
      AMQP_URL: amqp://guest:guest@broker:5672
    env_file:
      - .env.local                                # (7)
    volumes:
      - ./src:/app/src                            # (8)
      - /app/node_modules                         # (9)
    depends_on:
      db:     { condition: service_healthy }      # (10)
      cache:  { condition: service_started }
      broker: { condition: service_healthy }
    healthcheck:
      test: ["CMD", "node", "-e", "fetch('http://localhost:3000/health').then(r=>process.exit(r.ok?0:1))"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 20s                           # (11)
    restart: unless-stopped

  worker:
    build: { context: ., target: dev }
    command: ["node", "dist/worker.js"]           # (12)
    environment:
      AMQP_URL: amqp://guest:guest@broker:5672
      DATABASE_URL: postgres://app:app@db:5432/checkout
    depends_on:
      broker: { condition: service_healthy }
    deploy:
      replicas: 2                                 # (13)

  db:
    image: postgres:17                            # (14)
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: checkout
    volumes:
      - pgdata:/var/lib/postgresql/data           # (15)
      - ./db/init:/docker-entrypoint-initdb.d:ro  # (16)
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d checkout"]
      interval: 5s
      timeout: 3s
      retries: 10
    ports:
      - "127.0.0.1:5432:5432"                     # (17)

  cache:
    image: redis:7-alpine
    command: ["redis-server", "--save", "60", "1", "--loglevel", "warning"]
    volumes:
      - redisdata:/data

  broker:
    image: rabbitmq:3-management
    ports:
      - "127.0.0.1:15672:15672"
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]
      interval: 10s
      retries: 5

volumes:                                          # (18)
  pgdata:
  redisdata:

(1) The project name. Everything created is prefixed with it — containers become checkout-api-1, the network checkout_default. Without it, Compose uses the directory name, so two checkouts of the same repository in differently named folders quietly become two separate projects.

(2) build instead of image means Compose builds it, from the given context and Dockerfile.

(3) Builds only up to a named stage of the multi-stage Dockerfile from Chapter 13.6.5. A dev stage keeps development dependencies and a file watcher; production builds target runtime.

(4) The tag to give the built image. Useful so you can docker run it independently.

(5) The Node debugger port, so you can attach a debugger from your editor to code running inside the container.

(6) Service names as hostnames. db, cache, broker resolve on the Compose network. This is the payoff of the automatic network, and it is why these URLs are readable rather than a list of IPs.

(7) A file of variables, not committed. Values here are overridden by anything in environment.

(8) A bind mount of the source, so an edit on your machine is instantly visible inside the container.

(9) An anonymous volume over node_modules, and this line is a trick worth understanding. The bind mount at (8) would otherwise hide the container's installed modules with whatever is in your host directory — which may be empty, or built for the wrong platform if your laptop is a Mac and the image is Linux. Mounting an anonymous volume at that exact path masks the bind mount there, so the container keeps the modules it installed at build time.

(10) The condition is the important part and section 3 is entirely about it.

(11) A grace window while the service starts, during which failures do not count toward retries. Without it, a service that takes twenty seconds to boot is marked unhealthy before it has a chance.

(12) Overrides the image's CMD. Same image, different process — this is how a worker and an API share one build.

(13) Two copies of the worker. Compose supports replicas on one machine; they share the queue, which is the normal pattern for background workers.

(14) Pin the version. postgres:latest will change under you and is the source of the Wednesday afternoon in the opening.

(15) A named volume for the data, so docker compose down and up keeps your database. Without it, every restart is an empty database.

(16) A convention of the PostgreSQL image: any .sql or .sh file in that directory runs on first initialisation. This is how you get seeded data for free. It only runs when the data directory is empty, which is why editing a seed file appears to do nothing until you remove the volume.

(17) Bound to loopback only, so your laptop can connect with a database client but nothing on the coffee-shop network can.

(18) Named volumes must be declared here, or Compose will not create them.

3. depends_on — the trap

depends_on with no condition waits only for the container to start. Not for the process inside to be ready. PostgreSQL's container is "started" the instant the process launches; it is not accepting connections for another two to five seconds while it initialises.

So the naive version fails:

depends_on:
  - db        # waits for the container to start, nothing more

The API starts, tries to connect, gets ECONNREFUSED, and exits. This is the single most common Compose confusion, and people usually work around it with a sleep 10 in the entrypoint, which is both slow and unreliable.

The fix is a health check plus a condition:

db:
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U app -d checkout"]
    interval: 5s
    retries: 10

api:
  depends_on:
    db: { condition: service_healthy }

Now Compose actually waits until pg_isready succeeds.

Three conditions exist:

Condition	Waits for
service_started	The container to start (the old default)
service_healthy	Its health check to pass
service_completed_successfully	The container to exit with code 0 — for migration jobs

service_completed_successfully is how you run migrations before the API starts:

migrate:
  build: .
  command: ["npm", "run", "migrate"]
  depends_on:
    db: { condition: service_healthy }

api:
  depends_on:
    migrate: { condition: service_completed_successfully }

One honest caveat. Even with health conditions, your application should retry its connections at startup. Compose does this for you locally, but a real cluster restarts pods in any order at any time, and an application that cannot survive its database being briefly unavailable will fail there regardless. Ordering at startup is a convenience; retry logic is the actual requirement.

4. Networking and names

Compose creates one network per project and attaches every service to it. Service names resolve to container IPs, and replicas: 2 means the name resolves to both, round-robin.

Split networks when you want isolation:

services:
  api:
    networks: [frontend, backend]
  db:
    networks: [backend]        # (1)
  proxy:
    networks: [frontend]

networks:
  frontend:
  backend:
    internal: true             # (2)

(1) The database is not on frontend, so the proxy cannot reach it at all — not by policy, but because there is no path.

(2) internal: true means no route to the outside world. Containers on it can talk to each other and nothing else. This is the closest Compose comes to a network policy, and it is a genuinely useful default for a data tier.

Ports versus expose:

ports:  ["3000:3000"]     # reachable from your machine
expose: ["3000"]          # documentation only — other services reach it anyway

Services on the same network reach each other on any port with no ports entry at all. ports is only about reaching in from the host. Publishing a database port is for your local database client, nothing else, and it should be bound to 127.0.0.1.

5. Overrides, profiles and variables

The automatic override file

Compose reads compose.yaml and then compose.override.yaml if it exists, merging the second over the first. This is the intended way to keep a shared base and personal or environment-specific changes:

# compose.yaml — committed, shared
services:
  api:
    image: registry.example.com/checkout-api:1.4

# compose.override.yaml — local, not committed
services:
  api:
    build: .                       # build locally instead of pulling
    volumes: ["./src:/app/src"]    # live source
    environment:
      LOG_LEVEL: debug

Or name files explicitly, which is what CI does:

docker compose -f compose.yaml -f compose.ci.yaml up --abort-on-container-exit

The merge rules are worth knowing because one of them surprises people: maps merge key by key, and lists are replaced entirely, not appended. An override that sets command or ports replaces the whole thing.

Profiles

Services with a profile only start when that profile is requested.

services:
  api: {}                        # always starts
  mailhog:
    image: mailhog/mailhog
    profiles: ["dev-tools"]      # only with the profile
  seed:
    build: .
    command: ["npm", "run", "seed"]
    profiles: ["seed"]

docker compose up                              # api only
docker compose --profile dev-tools up          # api + mailhog
docker compose run --rm seed                   # one-shot task

This is how you keep optional tools — a mail catcher, an admin console, a load generator — in the same file without starting them every day.

Variable interpolation

services:
  api:
    image: checkout-api:${TAG:-dev}          # (1)
    environment:
      API_KEY: ${API_KEY:?API_KEY is required}   # (2)

(1) Use TAG if set, otherwise dev. (2) Fail immediately with that message if it is unset. Far better than starting and failing obscurely twenty seconds later.

Values come from the shell environment and from a .env file next to the Compose file. Note the distinction that trips people: the .env file feeds interpolation into the YAML, while env_file: feeds variables into the container. They are different mechanisms with similar names.

6. The commands

docker compose up -d                      # (1)
docker compose up --build                 # (2)
docker compose up --watch                 # (3)
docker compose ps                         # (4)
docker compose logs -f api worker         # (5)
docker compose exec api sh                # (6)
docker compose run --rm api npm test      # (7)
docker compose down                       # (8)
docker compose down -v                    # (9)
docker compose config                     # (10)

(1) Start everything in the background. (2) Rebuild images first. Compose does not rebuild automatically when a Dockerfile changes, and forgetting this leads to "my change did nothing" more often than any other Compose mistake. (3) File watching: rebuild or sync into the container on change, configured per service with a develop.watch block. This replaced most hand-rolled live-reload setups. (4) Status of the project's services, including health. (5) Follow logs from selected services, colour-coded per service. (6) A shell in a running service. (7) A one-off container, separate from the running ones. --rm cleans up. This is how you run tests, migrations or a shell against the project's environment. (8) Stop and remove containers and networks. Volumes survive.(9) Also removes volumes — this deletes your local database. Occasionally what you want, never by accident. (10) Prints the fully merged, interpolated configuration. Run this when overrides, profiles and variables have produced something you did not expect; it shows exactly what Compose will act on.

docker compose up in the foreground stops everything on Ctrl-C. In CI you want --abort-on-container-exit so the whole project stops when the test container finishes, and --exit-code-from api so the pipeline gets the test's exit code.

7. Compose for integration testing

The best use of Compose after local development. Your service against real dependencies — a real PostgreSQL, not a mock; a real broker, not an in-memory stub — started fresh and destroyed afterwards.

docker compose -f compose.yaml -f compose.test.yaml up \
  --abort-on-container-exit --exit-code-from tests
docker compose down -v

The pattern that makes this reliable: every dependency has a health check, the test service depends on all of them with service_healthy, and the whole project is torn down with -v afterwards so the next run starts from nothing.

Testcontainers is the library version of this idea — it starts containers from inside your test code and cleans them up, which suits tests that need a fresh database per test rather than per run. Same concept, finer granularity.

8. Why this is not production

Everything above is genuinely excellent for one machine. Here is what it does not do, and each line is a reason Chapter 13.6.8 exists.

One machine. No scheduling across a fleet. If the machine is full, you are done. If the machine dies, everything on it is down and nothing moves it elsewhere.

No rolling updates. docker compose up with a new image stops the old container and starts the new one. There is a gap with no service, and no health gate before traffic returns, and no automatic rollback if the new version is broken.

No self-healing beyond restart. A restart policy restarts a container on the same host. It cannot notice that the host is unreachable, or that the process is alive but wedged, or move anything anywhere.

No horizontal scaling that means anything. replicas: 2 gives two containers on one machine sharing its CPU.

No secret management. Values in a file or in environment variables. No encryption, no rotation, no per-environment access control.

No declarative reconciliation. Compose applies changes when you run it. Nothing is watching afterwards; drift is not detected or corrected.

And Docker Swarm, briefly, because people ask. Swarm is Docker's own clustering mode: docker swarm init, then docker stack deploy takes a Compose file with a deploy section and runs it across several machines with rolling updates and an overlay network. It is genuinely simpler than Kubernetes and it works. It also lost the orchestration competition decisively, so the ecosystem, the hiring pool, the managed services and the tooling all went elsewhere. Choosing it today means choosing a technology with a small and shrinking community, which is a real operational cost even when the software itself is fine.

What the interviewer will push on

"What does depends_on guarantee?" Only that the container started, unless you add a condition. service_healthy waits for a health check to pass; service_completed_successfully waits for a job to exit 0, which is how migrations run before an API. The answer that lands adds that your application should retry anyway, because a real cluster gives no ordering guarantees at all.

"How do services find each other?" Compose creates a project network and the service name resolves via Docker's embedded DNS. The detail worth adding is that ports is only about reaching in from the host — services on the same network reach each other on any port with no ports entry, so publishing a database port is purely for your local client.

"Is Compose suitable for production?" No, and the reasons should be specific: one machine, no rolling updates so there is a gap with no service, no rescheduling when a host dies, no real secret handling, and nothing reconciling drift afterwards. It is excellent for local development and integration testing.

"How do you keep a shared file and local changes?" compose.override.yaml merges automatically over compose.yaml, or name files explicitly with -f for CI. Then the merge rule that bites: maps merge, lists are replaced entirely.

"Why did my code change not take effect?" Either the source is not bind-mounted so the container is running what was baked into the image, or the image was not rebuilt — up does not rebuild unless you pass --build. docker compose config settles arguments about what is actually configured by printing the merged result.

One thing to volunteer: mention the anonymous volume over node_modules when bind-mounting source. Without it, the host directory hides the modules installed during the build — often empty, or built for the wrong platform on a Mac — and the container fails with missing modules in a way that looks like a broken image. It is one line and it explains a failure that otherwise costs an afternoon.

Recall

• Compose is a YAML file turned into Docker API calls. It gives one command, a project network with DNS by service name, a project namespace, and lifecycle across services.
• depends_on alone waits only for container start. Use condition: service_healthy, or service_completed_successfully for migration jobs. Retry in the application regardless — a cluster gives no ordering.
• healthcheck needs start_period or a slow-starting service is marked unhealthy before it boots.
• Named volumes for data (declared in the top-level volumes:), bind mounts for source, and an anonymous volume over node_modules so the bind mount does not hide installed dependencies.
• ports is only about reaching in from the host — services on one network reach each other without it. Bind local database ports to 127.0.0.1. internal: true on a network cuts it off from the outside.
• compose.override.yaml merges automatically; maps merge key by key, lists are replaced entirely. .env feeds interpolation into YAML; env_file: feeds the container.
• Profiles keep optional tools in the file without starting them. up does not rebuild — pass --build. docker compose config prints the merged truth.
• down keeps volumes, down -v deletes your database. For CI: --abort-on-container-exit --exit-code-from tests.
• Not production: one machine, no rolling updates (a gap with no service), no rescheduling on host failure, no real secrets, nothing reconciling drift. Swarm solved this and lost the ecosystem.

Self-test: What exactly does depends_on wait for, and how do you make it wait for readiness? · Why does a bind-mounted source directory break node_modules, and what fixes it? · Which two mechanisms does .env versus env_file: control? · What happens to a list key in an override file? · Name three things Compose cannot do that production needs.

Next: 13.6.8 takes those five limits seriously. What does it actually take to run containers across a fleet of machines, why did every large company build the same system independently, and what did Google's decade of doing it turn into?