Deploying ML Models to Azure Container Apps: A Step-by-Step Guide

Azure Container Apps just got GPU support, and the pricing is competitive — $0.39/hr for a T4 instance with pay-per-use billing. Here's how to deploy a FastAPI + PyTorch model to Azure in under 15 minutes.

Why Azure Container Apps?

Azure Container Apps is Microsoft's serverless container platform. It sits between Azure Container Instances (simple but limited) and AKS (powerful but complex). For ML inference, it hits the sweet spot:

• GPU support (T4, A10, A100 in preview)
• Serverless scaling — zero to N based on HTTP traffic, KEDA under the hood
• Integrated with Azure Monitor — logs, metrics, alerts out of the box
• VNet integration — deploy to your private network
• Managed TLS — automatic HTTPS certificates

Prerequisites

• A FastAPI + PyTorch model in a GitHub repo
• An Azure subscription
• Azure CLI installed and logged in (az login)
• Docker installed locally

Step 1: Containerize Your Model

Your Dockerfile needs to be GPU-aware for Azure:

FROM pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=5s --retries=3 CMD curl -f http://localhost:8000/health || exit 1
USER 1000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Key details:

• Use the runtime image, not devel — saves 2GB
• Non-root user (UID 1000)
• Health check on /health endpoint
• Port 8000 (Azure default for Container Apps is 80, but you can configure this)

Step 2: Create a Container Registry

az acr create \
  --name mymlmodelsacr \
  --resource-group ml-inference \
  --sku Basic \
  --admin-enabled true

Push your image:

az acr login --name mymlmodelsacr
docker build -t mymlmodelsacr.azurecr.io/sentiment:v1 .
docker push mymlmodelsacr.azurecr.io/sentiment:v1

Step 3: Create the Container App with GPU

az containerapp create \
  --name sentiment-api \
  --resource-group ml-inference \
  --environment ml-inference-env \
  --image mymlmodelsacr.azurecr.io/sentiment:v1 \
  --target-port 8000 \
  --ingress external \
  --registry-server mymlmodelsacr.azurecr.io \
  --registry-username mymlmodelsacr \
  --cpu 2.0 \
  --memory 8.0Gi \
  --gpu 1 \
  --gpu-type T4 \
  --min-replicas 1 \
  --max-replicas 5 \
  --scale-rule-name http-rule \
  --scale-rule-type http \
  --scale-rule-http-concurrency 10

This creates:

• An HTTP endpoint with external ingress
• 1-5 replicas based on concurrency (scale up at 10 concurrent requests)
• 1 NVIDIA T4 GPU per replica
• 2 vCPU + 8GB RAM per replica

Step 4: Set Environment Variables

az containerapp update \
  --name sentiment-api \
  --resource-group ml-inference \
  --set-env-vars \
    MODEL_PATH=/app/models/sentiment \
    BATCH_SIZE=32 \
    LOG_LEVEL=info

Step 5: Test

curl -X POST https://sentiment-api.yellowground-12345.eastus.azurecontainerapps.io/predict \
  -H "Content-Type: application/json" \
  -d '{"text": "Azure GPU inference works great!"}'

Cost Analysis

Item	Monthly Cost
Container App (T4, 1 replica, 730h)	~$285
ACR (Basic tier)	$5
Data transfer (outbound)	~$20-50
Azure Monitor logs	~$15
Total	~$345

This is slightly more expensive than AWS SageMaker ($275) but cheaper than GCP Cloud Run GPU ($390). The serverless scaling means you only pay for what you use — if your traffic is bursty, actual costs are lower.

Azure vs AWS vs GCP for GPU Inference

Feature	Azure Container Apps	AWS SageMaker	GCP Cloud Run
GPU types	T4, A10, A100	T4, A10G, A100	T4, L4, A100
Min replicas	0-1	1	0-1
Cold start	30-60s	45-90s	15-30s
VNet integration	Yes	Yes	Yes
Autoscaling	KEDA (HTTP/metrics)	CloudWatch	CPU/Memory
Monthly (T4, 24/7)	~$345	~$275	~$390

The Roptal Advantage

Deploying to Azure via CLI works, but it's one cloud. With Roptal, you deploy the same Docker image to Azure, AWS, GCP, or RunPod — switch providers based on cost, GPU availability, or geographic requirements. No CLI per cloud. No different configs per platform.

Connect your repo, review the generated Dockerfile, click deploy. Roptal handles the rest.

Join the waitlist →