Hardening Production Multi-Container Topologies with Docker Compose v2

restart: always is not a resilience strategy. It is a retry loop that hides an ordering bug until the night it doesn’t. A Compose stack that survives production treats startup order, network reachability, credentials, and resource ceilings as explicit, testable contracts. This piece takes a typical four-service stack (Nginx, a FastAPI app, PostgreSQL 16, Redis 7) and hardens each of those contracts in turn.

1. The Fragility of Naive Stacks

Most Compose files start life like this:

services:
  api:
    build: ./api
    ports: ["8000:8000"]
    depends_on: [postgres, redis]
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_PASSWORD: devpassword
  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]

It works on a laptop and fails in four predictable ways elsewhere:

• Bootstrap latency. On an empty volume, the Postgres entrypoint runs initdb, starts a temporary server, applies init scripts, stops it, and starts the real server. That takes seconds. Short-form depends_on only waits for the container to start, so the API’s first connection attempt lands mid-initialization and fails.
• Migration races. If the API runs migrations at boot and you scale to two replicas, two processes race to apply the same schema change. If you run migrations by hand, the app can serve traffic against an old schema.
• A flat network. Every service shares the default network, and ports: publishes Redis on every host interface. Anything that reaches the host reaches the cache, and anything that compromises the proxy can reach the database directly.
• Plain-text credentials. POSTGRES_PASSWORD sits in the Compose file, in docker inspect output, and in the environment of every process in the container.

2. Deterministic Startup via Health Checks

Compose v2 implements the Compose Specification’s long-form depends_on, where each dependency carries a condition:

Condition	Satisfied when	Use for
service_started	The dependency’s container has started (the default)	Dependencies with their own connection retry logic
service_healthy	The dependency’s healthcheck reports healthy	Databases, caches, upstream APIs
service_completed_successfully	The dependency ran to completion and exited 0	Migrations, seed jobs, one-shot init tasks

The distinction that matters is process versus readiness. service_started tells you PID 1 exists. It says nothing about whether port 5432 accepts connections. service_healthy gates on a probe you define:

healthcheck:
  test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U app -d app"]
  interval: 5s       # time between probes
  timeout: 3s        # a probe slower than this counts as a failure
  retries: 10        # consecutive failures before "unhealthy"
  start_period: 30s  # failures here don't count toward retries

Note the -h 127.0.0.1. During first-boot initialization, the official Postgres entrypoint runs its temporary server with listen_addresses='': Unix socket only. A socket-based pg_isready can report ready during init, just before that server is stopped. Probing over TCP only succeeds once the real server is listening.

Migrations become their own service. It runs once, after the database is healthy, and the application gates on its exit code:

migrate:
  command: ["alembic", "upgrade", "head"]
  depends_on:
    postgres:
      condition: service_healthy
  restart: "no"

app:
  depends_on:
    migrate:
      condition: service_completed_successfully

If a migration fails, the app never starts, which is exactly what you want. And because only one container applies the schema, scaling app no longer causes races.

3. Network Segmentation & Secrets Hardening

            Internet
               │
     Host :80  (:443 when TLS terminates here)
               │
┌──────────────┼───────────────────── dmz ─────────────┐
│              ▼                                       │
│     ┌─────────────────┐  proxy_pass  ┌────────────┐  │
│     │ Nginx (UID 101) │ ───────────▶ │  FastAPI   │  │
│     └─────────────────┘   :8000      │ (UID 10001)│  │
│                                      └─────┬──────┘  │
└────────────────────────────────────────────┼─────────┘
┌────────────────────────── backend_net ─────┼─────────┐
│              (internal: true, no egress)   │         │
│                         ┌──────────────────┴──────┐  │
│                         ▼                         ▼  │
│              ┌──────────────────┐    ┌──────────────┐│
│              │ PostgreSQL 16    │    │ Redis 7      ││
│              │ (UID 70)  :5432  │    │ (UID 999)    ││
│              └──────────────────┘    └──────────────┘│
│        no host ports; unreachable from nginx         │
└──────────────────────────────────────────────────────┘

Split the topology into two user-defined bridge networks:

• dmz: Nginx and the app. Only Nginx publishes a host port.
• backend_net: the app, Postgres, and Redis, marked internal: true. Docker creates no route from an internal network to the outside, so the data stores cannot reach the internet and publish nothing to the host.

The app is the only service on both networks. Nginx cannot resolve the name postgres, because it shares no network with it.

For credentials, move passwords out of environment: into top-level secrets:. Compose mounts each secret at /run/secrets/<name>, and the value never appears in docker inspect or the process environment. The official Postgres image reads *_FILE variants natively:

environment:
  POSTGRES_PASSWORD_FILE: /run/secrets/db_password

Your own code reads the file the same way:

import os
from pathlib import Path

def read_secret(env_var: str) -> str:
    return Path(os.environ[env_var]).read_text().strip()

db_password = read_secret("DB_PASSWORD_FILE")

One caveat: outside Swarm, Compose implements file-sourced secrets as bind mounts, and silently ignores the long-syntax uid, gid, and mode options for them. They only apply to secrets sourced from environment. With file secrets, the host file’s permissions decide who inside the container can read it. Lock down the directory on the host (chmod 700 secrets/) and make the files readable by the service UIDs (chmod 444). A container reads a bind-mounted file without traversing the host directory.

4. Resource Limits & Non-Root Enforcement

Without limits, one leaking worker can trigger the kernel OOM killer for the whole host, taking down the database with it. Compose v2 applies deploy.resources outside Swarm:

deploy:
  resources:
    limits:
      cpus: "1.0"     # hard CFS quota: at most one core of CPU time
      memory: 512M    # cgroup ceiling; exceeding it kills this container, not its neighbors
      pids: 200       # caps fork bombs and thread leaks
    reservations:
      memory: 256M

cpus is a ceiling. The separate cpu_shares key is only a relative weight that applies when CPUs are contended. Use limits for isolation, and shares only to prioritize.

Then strip privilege:

• user: "10001:10001" runs as a fixed non-root UID that matches no account on the host, so a container escape lands on an identity that owns nothing.
• read_only: true makes the root filesystem immutable; give each service writable tmpfs paths only where it needs scratch space.
• cap_drop: [ALL] and security_opt: ["no-new-privileges:true"] remove capabilities and block setuid escalation.

Use image-native non-root UIDs where they exist: 70 for postgres:16-alpine, 999:1000 for redis:7-alpine, and 101 for nginxinc/nginx-unprivileged. Running the images as those users from the start means the entrypoints never need root to drop privileges.

Volume ownership is the trap here. A named volume is safe: Docker seeds it from the image’s /var/lib/postgresql/data, which the Postgres image creates with permissions that accept a fixed user:. A bind mount of a host directory is not. The directory is typically owned by root, initdb running as UID 70 cannot write to it, and the container exits on first boot. If you must bind-mount, chown -R 70:70 the host path first. Never reuse a data directory initialized by the Debian-based image, whose postgres user is UID 999.

5. The Production Blueprint

name: fastapi-platform

# Shared hardening, merged into every service
x-hardening: &hardening
  read_only: true
  cap_drop: [ALL]
  security_opt:
    - no-new-privileges:true
  restart: unless-stopped

x-api-image: &api-image
  build:
    context: ./api
  image: fastapi-platform/api:1.0.0

services:
  nginx:
    <<: *hardening
    image: nginxinc/nginx-unprivileged:1.27-alpine
    user: "101:101"
    ports:
      - "80:8080"                    # the only published port in the stack
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
    tmpfs:
      - /tmp:size=16m                # pid file and proxy buffers
    networks: [dmz]
    depends_on:
      app:
        condition: service_healthy
    deploy:
      resources:
        limits: { cpus: "0.50", memory: 128M, pids: 100 }

  app:
    <<: [*hardening, *api-image]
    user: "10001:10001"
    environment:
      DB_HOST: postgres
      DB_NAME: app
      DB_USER: app
      DB_PASSWORD_FILE: /run/secrets/db_password
      REDIS_HOST: redis
      REDIS_PASSWORD_FILE: /run/secrets/redis_password
    secrets: [db_password, redis_password]
    tmpfs:
      - /tmp:size=64m
    networks: [dmz, backend_net]     # no ports: reachable only through nginx
    depends_on:
      migrate:
        condition: service_completed_successfully
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=2)"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 20s
    deploy:
      resources:
        limits: { cpus: "1.0", memory: 512M, pids: 200 }
        reservations: { memory: 256M }

  migrate:
    <<: [*hardening, *api-image]
    user: "10001:10001"
    command: ["alembic", "upgrade", "head"]
    restart: "no"                    # one-shot: overrides the shared policy
    environment:
      DB_HOST: postgres
      DB_NAME: app
      DB_USER: app
      DB_PASSWORD_FILE: /run/secrets/db_password
    secrets: [db_password]
    tmpfs:
      - /tmp:size=16m
    networks: [backend_net]
    depends_on:
      postgres:
        condition: service_healthy

  postgres:
    <<: *hardening
    image: postgres:16-alpine
    user: "70:70"                    # image-native postgres UID; no root entrypoint step
    environment:
      POSTGRES_DB: app
      POSTGRES_USER: app
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets: [db_password]
    volumes:
      - pgdata:/var/lib/postgresql/data
    tmpfs:
      - /var/run/postgresql:size=1m  # Unix socket and lock file
      - /tmp:size=16m
    shm_size: 128mb
    networks: [backend_net]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U app -d app"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 30s
    stop_grace_period: 60s           # time to checkpoint before SIGKILL
    deploy:
      resources:
        limits: { cpus: "2.0", memory: 1G, pids: 300 }

  redis:
    <<: *hardening
    image: redis:7-alpine
    user: "999:1000"
    # The password arrives on stdin ("-"), so it never appears in the process list.
    # $$ escapes Compose interpolation so the shell evaluates $(cat ...).
    command:
      - sh
      - -c
      - |
        exec redis-server - 
          --maxmemory 192mb 
          --maxmemory-policy allkeys-lru 
          --save "" 
          --appendonly no <<EOF
        requirepass $$(cat /run/secrets/redis_password)
        EOF
    secrets: [redis_password]
    networks: [backend_net]
    healthcheck:
      test: ["CMD-SHELL", "REDISCLI_AUTH="$$(cat /run/secrets/redis_password)" redis-cli ping | grep -q PONG"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 5s
    deploy:
      resources:
        limits: { cpus: "0.50", memory: 256M, pids: 100 }

networks:
  dmz:
    driver: bridge
  backend_net:
    driver: bridge
    internal: true                   # no route to the outside world

volumes:
  pgdata:

secrets:
  db_password:
    file: ./secrets/db_password.txt
  redis_password:
    file: ./secrets/redis_password.txt

Redis runs as a pure cache: persistence is off, and maxmemory sits at 75% of the container’s memory limit. That headroom covers connection buffers and fragmentation, so Redis evicts keys before the cgroup kills it.

Nginx rate-limits per client IP before traffic reaches the app:

# nginx/default.conf (included in the http context)
limit_req_zone $binary_remote_addr zone=api:10m rate=20r/s;
limit_req_status 429;

server {
    listen 8080;
    server_tokens off;
    client_max_body_size 1m;

    location / {
        limit_req zone=api burst=40 nodelay;
        proxy_pass http://app:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 2s;
        proxy_read_timeout 30s;
    }
}

If Nginx sits behind a cloud load balancer, configure set_real_ip_from with the load balancer’s range. Otherwise $binary_remote_addr is the load balancer’s IP, and every client shares a single rate-limit bucket. TLS terminates either at that load balancer or in an additional listen 8443 ssl server block. On rootless Docker, publishing host port 80 or 443 fails unless net.ipv4.ip_unprivileged_port_start is lowered, so map to unprivileged host ports there and front them with the load balancer.

Control	nginx	app	migrate	postgres	redis
Networks	dmz	dmz, backend_net	backend_net	backend_net	backend_net
Host port	80	none	none	none	none
Runs as	101	10001	10001	70	999
Read-only root	yes	yes	yes	yes	yes
Gate before start	app healthy	migrate exited 0, redis healthy	postgres healthy	none	none
Memory ceiling	128M	512M	none	1G	256M

6. Operational Verification & Open Source Reference

Hardening that isn’t verified is a guess. After docker compose up -d --wait, which blocks until services are running and healthy:

# Validate the merged model: anchors, interpolation, secrets paths
docker compose config --quiet

# Health per service: expect "healthy" for app, postgres and redis
# (nginx has no healthcheck; it starts only after app is healthy)
docker compose ps

# The migration must have exited 0
docker compose ps -a migrate --format '{{.Service}} {{.State}} exit={{.ExitCode}}'

# Full probe history when something stays "starting"
docker inspect --format '{{json .State.Health}}' "$(docker compose ps -q postgres)"

# Non-root and immutable: expect uid=10001 and "Read-only file system"
docker compose exec app sh -c 'id; touch /probe'

# Segmentation: nginx must not resolve the database
docker compose exec nginx getent hosts postgres || echo "isolated"

# Limits applied: the LIMIT column should match deploy.resources
docker stats --no-stream

Wire the first three checks into CI, so a regression in startup ordering fails the pipeline instead of a deploy.

Run the blueprint yourself

The complete, runnable topology lives in the production-compose/ directory of devops-infra-toolkit. It contains this exact docker-compose.yml, the Nginx configuration, and a minimal FastAPI service with an Alembic migration. Generate the two secret files, run docker compose up --build --wait, and every check above should pass. The repository’s top-level docker-compose.yml is the naive shape from section 1, so diffing the two files shows every control in this piece side by side.

Compose hardening contains what runs inside the stack. It does not govern who may drive the Docker daemon itself. That boundary matters most when the client is an autonomous AI agent. The security architecture specification of docker-mcp-guardian lays out that container isolation model. It covers why socket access is root-equivalent, how a read-only-by-default MCP layer narrows what an agent can ask of the daemon, and where a filtering socket proxy fits. Issues and pull requests are welcome on both repositories.