CI/CD for ML Models: Automating Deployment Without Breaking Production

Most CI/CD guides are written for web apps. Deploy a new commit, run tests, ship it. ML models are different — you're deploying not just code, but also model weights, dependencies, and sometimes entirely new runtime requirements.

Here's a CI/CD pipeline that works for ML inference.

The Pipeline

Push to GitHub
    ↓
[1] Lint + Type Check (2 min)
    ↓
[2] Unit Tests (5 min)
    ↓
[3] Build Docker Image (10 min)
    ↓
[4] Push to Container Registry (2 min)
    ↓
[5] Deploy Canary (10% traffic, 24h observation)
    ↓
[6] Health Check Loop (continuous)
    ↓
[7] Promote to Production (100% traffic)

Step 1: Lint + Type Check

# .github/workflows/deploy.yml
name: Deploy
on:
  push:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install ruff
      - run: ruff check .
      - run: ruff format --check .

Catches syntax errors and formatting issues before they hit production. 2 minutes. Free.

Step 2: Unit Tests

  test:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest -x --timeout=60

Test the API endpoints, not the model accuracy. Check that /health returns 200, /predict accepts valid input, /predict rejects invalid input, and graceful shutdown works.

Don't test model accuracy in CI. Training-time metrics are validation, not CI tests. Testing inference output requires loading model weights (slow) and deterministic outputs (not guaranteed across hardware).

Step 3: Build Docker Image

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t my-registry/sentiment:${{ github.sha }} .
      - run: docker push my-registry/sentiment:${{ github.sha }}

Tag with git SHA, not latest. If a deploy goes wrong, you know exactly which commit to roll back to.

Step 4: Deploy Canary

Don't deploy 100% immediately. Deploy to 10% of traffic and monitor for 24 hours.

  deploy-canary:
    needs: build
    steps:
      - run: |
          aws sagemaker create-endpoint-config \
            --endpoint-config-name sentiment-canary-${{ github.sha }} \
            --production-variants VariantName=canary,ModelName=sentiment-${{ github.sha }},InstanceType=ml.g4dn.xlarge,InitialVariantWeight=0.1

10% of traffic goes to the new version. 90% stays on the old version. If error rates spike on the canary, the old version is unaffected.

Step 5: Health Check Loop

After deploying the canary, run continuous health checks:

for i in $(seq 1 100); do
  response=$(curl -s -o /dev/null -w "%{http_code}" https://endpoint/predict \
    -H "Content-Type: application/json" \
    -d '{"text": "test"}')
  
  if [ "$response" != "200" ]; then
    echo "Health check failed: $response"
    exit 1
  fi
  sleep 30
done

If any check fails, the pipeline stops and notifies the team. The canary never gets promoted.

Step 6: Check Metrics

Before promoting, verify that the canary's metrics match or improve on the baseline:

Metric	Baseline	Canary	Action if degraded
Error rate	< 0.1%	Compare	Rollback if > baseline
p95 latency	200ms	Compare	Rollback if > 1.5x
Prediction distribution	Baseline PSI	Current PSI	Investigate if PSI > 0.2

Step 7: Promote

aws sagemaker update-endpoint \
  --endpoint-name sentiment-prod \
  --endpoint-config-name sentiment-${{ github.sha }}

100% of traffic now goes to the new version. The old version stays warm for 24 hours (instant rollback if needed).

Rollback

aws sagemaker update-endpoint \
  --endpoint-name sentiment-prod \
  --endpoint-config-name sentiment-$(git rev-parse HEAD~1)

One command. Switches back to the previous commit's deployment. Takes ~30 seconds.

Secrets Management

Never put API keys in GitHub Actions YAML. Use repository secrets:

env:
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

GitHub encrypts these at rest. They're masked in logs. Rotate them quarterly.

Model Registry vs Git

Model weights shouldn't go in Git. Large files (>100MB) break Git's performance model. Use a model registry:

• Hugging Face Hub: huggingface_hub Python library, versioned, public/private
• AWS S3 + versioning: Simple, cheap, works with any framework
• DVC: Git-compatible, handles large files, tracks experiments

In your CI/CD, the deployment step downloads the latest model version from the registry, not from Git.

Full Pipeline Time

Step	Duration
Lint + Type Check	2 min
Unit Tests	5 min
Build + Push Image	12 min
Deploy Canary	5 min
Health Check Loop	60 min
Promote	2 min
Total	~86 min

An hour and a half from push to production. Most of it is the health check loop (giving the canary time to prove itself). The actual deployment work is 25 minutes.

Roptal automates steps 4-7 — canary deployment, health checks, promotion, and rollback — without configuring CI/CD per cloud provider. Push to GitHub, the rest is handled.