How to Deploy a Private FastAPI Model Endpoint in an AWS VPC

A public inference endpoint is convenient, but it is not always acceptable. Internal copilots, fraud models, document classifiers, and healthcare workloads often need to stay inside a private network.

This guide outlines a private FastAPI deployment on AWS using an Application Load Balancer, private subnets, and security groups.

Target Architecture

Internal client / VPN / application
        |
Internal Application Load Balancer
        |
Private subnet: ECS or EC2 container
        |
FastAPI model service on port 8000

The container has no public IP. The load balancer is internal. Only approved network paths can reach the model.

Network Layout

Create at least two private subnets in separate availability zones. Place the model containers there. Create an internal ALB across the same subnets.

Security groups:

1. ALB security group: allows inbound 443 only from your application subnets, VPN CIDR, or corporate network.
2. Model service security group: allows inbound 8000 only from the ALB security group.
3. No public inbound rule on the model service.

This is more important than adding API keys to an exposed endpoint. Network isolation is the first layer.

FastAPI Health Endpoint

from fastapi import FastAPI
import torch

app = FastAPI()
model_ready = False

@app.on_event("startup")
async def startup():
    global model_ready
    # Load the model here.
    model_ready = True

@app.get("/health")
def health():
    if not model_ready:
        return {"status": "starting"}
    return {
        "status": "healthy",
        "gpu": torch.cuda.is_available(),
    }

Configure the ALB target group health check to call /health. Set a sufficient healthy threshold; loading a model can take longer than a typical web service boot.

Authentication Still Matters

Private does not mean trusted. An internal service can be compromised. Use service-to-service authentication:

• JWT signed by your identity provider
• AWS IAM request signing for AWS-native clients
• mTLS for high-assurance internal services
• Short-lived API tokens stored in Secrets Manager

Do not hardcode tokens in a Docker image or repository.

Accessing S3 Without Public Internet

If weights live in S3, add a Gateway VPC endpoint for S3. The model container can then download artifacts without a NAT gateway or public egress.

Benefits:

• Lower NAT gateway charges
• No public path to S3
• Bucket policy can require the specific VPC endpoint

Example bucket condition:

{
  "Condition": {
    "StringEquals": {
      "aws:sourceVpce": "vpce-0123456789abcdef"
    }
  }
}

Logging Without Leaking Inputs

Log operational metadata, not raw sensitive prompts or documents.

logger.info(
    "prediction_complete",
    extra={
        "latency_ms": latency_ms,
        "input_bytes": len(request.text.encode("utf-8")),
        "model_version": model_version,
    },
)

Avoid request bodies in CloudWatch logs unless you have a documented retention and redaction policy.

Deployment Checklist

• Containers in private subnets
• Internal ALB only
• Model service accepts traffic only from ALB security group
• IAM role has minimum S3/ECR permissions
• Health checks validate model readiness
• Secrets from Secrets Manager or Parameter Store
• Logs exclude sensitive request content
• At least two replicas across availability zones

Roptal can deploy a repository into your existing cloud account while preserving your VPC, credential, and ownership boundaries. The control plane orchestrates the deployment; your infrastructure remains yours.