Docker Compose for ML Development: Reproducible Environments That Actually Work

"Works on my machine" is a running joke. For ML, it's a daily reality. Different CUDA versions, Python versions, and system libraries make reproducing a teammate's setup a day-long project.

Docker Compose fixes this. Here's a setup that works across development, staging, and production.

The Full docker-compose.yml

version: '3.8'

services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      - MODEL_PATH=/app/models/sentiment
      - LOG_LEVEL=debug
    volumes:
      - ./models:/app/models:ro
      - ./logs:/app/logs
    command: uvicorn main:app --host 0.0.0.0 --port 8000 --reload
    healthcheck:
      test: curl -f http://localhost:8000/health || exit 1
      interval: 15s
      timeout: 5s
      retries: 3
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Key Decisions

GPU Access

The deploy.resources.reservations.devices block gives the container GPU access. Without it, torch.cuda.is_available() returns False inside the container even if the host has NVIDIA drivers.

Verify GPU access:

docker compose up -d
docker compose exec api python -c "import torch; print(torch.cuda.is_available())"

Should print True. If it doesn't, install nvidia-container-toolkit:

sudo apt install nvidia-container-toolkit
sudo systemctl restart docker

Volume Mounts

./models:/app/models:ro mounts your local models directory as read-only. The container can load weights but can't modify them. This means:

• You can update model files on the host without rebuilding the image
• Multiple containers can share the same model weights
• No accidental overwrites from inside the container

./logs:/app/logs lets you tail log files from the host:

tail -f logs/uvicorn.log

Health Check

Without a health check, Docker only knows if the process is running. With one, it knows if the app is actually working. The health check here pings /health every 15 seconds. After 3 failures, Docker marks the container unhealthy.

Combined with a reverse proxy (nginx, Traefik), unhealthy containers get automatically removed from the load balancer.

Environment Variables

All config through env vars — never hardcoded. The MODEL_PATH tells the app where to find weights. LOG_LEVEL switches between debug and info without code changes.

Development vs Production

Development Compose:

command: uvicorn main:app --reload
volumes:
  - .:/app  # mount entire codebase for hot reload
environment:
  - LOG_LEVEL=debug

Production Compose:

command: uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
volumes:
  - ./models:/app/models:ro  # only models, not code
environment:
  - LOG_LEVEL=warning

Key differences: no code volume mount (immutable image), multiple workers, minimal logging, no hot reload.

Multi-Service Setup

For a full ML stack:

services:
  api:
    build: .
    # ... GPU inference server
  
  nginx:
    image: nginx:alpine
    ports:
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
    depends_on:
      api:
        condition: service_healthy
  
  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    ports:
      - "9090:9090"

Nginx handles TLS termination and throttling. Prometheus scrapes /metrics from the API container. depends_on with condition: service_healthy means nginx doesn't start until the API is actually ready — no more "connection refused" during startup races.

Common Pitfalls

Pitfall 1: Installing CUDA inside the container. Don't. CUDA drivers come from the host. The container only needs the CUDA runtime libraries, which are included in the PyTorch base image.

Pitfall 2: Running as root. Add USER 1000 to your Dockerfile. Compose doesn't override this.

Pitfall 3: Forgetting to clean up. Stopped containers, unused volumes, and old images eat disk space:

docker compose down -v
docker system prune -af  # careful — removes everything unused

Why Compose Over Kubernetes

For teams with fewer than 10 services, Docker Compose is simpler than K8s:

• Single YAML file, no cluster management
• Built-in health checks and dependencies
• Same file works on developer laptops and GPU servers
• CI/CD can run docker compose up --abort-on-container-exit for integration tests

When you outgrow Compose, migrating the service definitions to Kubernetes manifests is straightforward. The Docker images don't change.

We bundle a production docker-compose.yml with every deployment at Roptal. It includes health checks, resource limits, volume mounts, and GPU reservations — ready to deploy.