Building an MCP Server for ML Deployment: Let AI Agents Ship Models

The Model Context Protocol (MCP) lets AI agents interact with external tools through a standardized interface. We built an MCP server for Roptal that lets Claude, Cursor, and other MCP-compatible agents deploy and manage ML models.

Here's how it works and how to build your own.

What MCP Does

MCP defines a client-server protocol where AI agents can call tools and access resources. A tool is a function the agent can invoke. A resource is data the agent can read.

For ML deployment, the tools are:

1. scan_repository(repo_url) → deployment profile
2. generate_dockerfile(repo_url) → Dockerfile content
3. deploy_model(repo, cloud, region) → endpoint URL
4. check_deployment_status(deployment_id) → status
5. get_logs(deployment_id) → log stream
6. rollback_deployment(deployment_id) → previous endpoint

The Server

An MCP server exposes tools over SSE (Server-Sent Events) or stdio. We chose SSE for web-based clients and stdio for local tools.

# mcp_server.py — simplified example
import asyncio
from mcp.server import Server, NotificationOptions
from mcp.server.models import InitializationCapabilities

server = Server("roptal-deploy")

@server.list_tools()
async def handle_list_tools():
    return [
        Tool(
            name="scan_repository",
            description="Scan a GitHub repository and return a deployment profile",
            inputSchema={
                "type": "object",
                "properties": {
                    "repo_url": {
                        "type": "string",
                        "description": "GitHub repository URL (e.g., https://github.com/user/repo)"
                    }
                },
                "required": ["repo_url"]
            }
        ),
        Tool(
            name="deploy_model",
            description="Deploy an ML model from a GitHub repository to a cloud provider",
            inputSchema={
                "type": "object",
                "properties": {
                    "repo_url": {"type": "string"},
                    "cloud": {"type": "string", "enum": ["aws", "gcp", "azure", "runpod"]},
                    "region": {"type": "string"},
                    "gpu_type": {"type": "string", "enum": ["t4", "a10g", "a100"]},
                },
                "required": ["repo_url", "cloud", "region"]
            }
        ),
    ]

@server.call_tool()
async def handle_call_tool(name: str, arguments: dict):
    if name == "scan_repository":
        repo = arguments["repo_url"]
        # Call the actual scan logic
        result = await scan_repository(repo)
        return [TextContent(type="text", text=json.dumps(result, indent=2))]
    
    if name == "deploy_model":
        # Call the actual deployment logic
        result = await deploy_model(**arguments)
        return [TextContent(type="text", text=f"Deployment started: {result['endpoint']}")]

async def main():
    async with stdio_server() as (read_stream, write_stream):
        await server.run(
            read_stream,
            write_stream,
            InitializationCapabilities(
                sampling={},
                experimental={},
                roots={}
            ),
        )

asyncio.run(main())

What the Agent Can Do

Once the MCP server is connected to Claude Desktop or Cursor, the agent can:

User prompt: "I have a FastAPI sentiment model at github.com/acme/sentiment-api. Deploy it to AWS on a T4 GPU."

The agent:

1. Calls scan_repository("github.com/acme/sentiment-api") — gets the deployment profile
2. Reads the Dockerfile from the scan result — verifies it's production-ready
3. Calls deploy_model(repo="github.com/acme/sentiment-api", cloud="aws", region="us-east-1", gpu_type="t4") — starts deployment
4. Periodically calls check_deployment_status() — waits for live
5. Returns the endpoint URL and monitoring dashboard link

No clicking through a UI. No CLI commands. The agent handles the entire flow.

Security Considerations

MCP tools have access to your deployment infrastructure. Lock this down:

1. Tool-level auth: Each tool call verifies the client has permission. Use API keys scoped to specific operations.
2. Input validation: Never trust tool arguments from an AI agent. Validate all inputs before executing.
3. Rate limiting: An agent could call deploy_model 100 times in 10 seconds. Add cooldowns.
4. Audit logging: Log every tool call with the agent's identity, arguments, and result.

@server.call_tool()
async def handle_call_tool(name: str, arguments: dict, context):
    # Validate auth
    api_key = context.request_context.lifespan_context.get("api_key")
    if not validate_api_key(api_key):
        raise McpError(ErrorCode.Unauthorized, "Invalid API key")
    
    # Rate limit
    await rate_limiter.check(name, api_key)
    
    # Log
    logger.info(f"Tool called: {name} by {api_key[:8]}... args: {arguments}")
    
    # Execute
    return await execute_tool(name, arguments)

Where MCP for Deployment Shines

Agent-driven operations: "Deploy this model, set up monitoring, and alert me if error rate exceeds 1%." The agent orchestrates multiple tools.

Automated rollbacks: The monitoring tool detects drift. The agent calls rollback_deployment without human intervention.

Multi-cloud coordination: "Deploy this model to AWS and GCP, set up cross-cloud failover." One prompt, multiple tool calls.

Roptal's MCP server implements these tools and more. Connect Claude Desktop or Cursor to deploy models directly from your editor. Full docs at docs.roptal.com/mcp.