Archestra.AI MCP Server: Gateway, Registry, Orchestrator, Credentials

Overview

Archestra.AI MCP Server is a lightweight server that implements a Model Context Protocol (MCP) to centralize gateway, registry, orchestration, credentials, and cost-management for large language model (LLM) usage in the enterprise. It acts as a single control plane for routing inference requests, managing model metadata and credentials, applying orchestration policies, and tracking usage and cost across multiple LLM backends.

By consolidating these responsibilities, the MCP Server simplifies multi-model deployments, enforces access and routing policies, enables centralized cost accounting, and provides a chat-ready platform for teams to integrate LLMs securely and consistently. It is designed to be deployed in typical enterprise environments (Docker or Kubernetes) and integrates with backing services like Postgres, Redis, and Prometheus.

Features

• Centralized model registry: store model metadata, versions, tags, and preferred backends
• Gateway / proxy: route API calls to configured LLM backends with request shaping
• Orchestrator: define routing and orchestration logic (fan-out, fallback, cost-aware routing)
• Credential management: store, rotate, and scope API keys per team or environment
• Cost and usage tracking: record token/response usage and compute cost per-backend
• Chat platform primitives: session management, message history, and context handling
• Observability: metrics, logging, and audit trails for governance and billing
• Extensible: REST APIs and web UI, pluggable adapters for new backends

Installation / Configuration

Minimum requirements: Docker or Kubernetes, Postgres, Redis.

Docker Compose example (local quickstart):

version: "3.8"
services:
  postgres:
    image: postgres:15
    environment:
      POSTGRES_USER: mcp
      POSTGRES_PASSWORD: mcp
      POSTGRES_DB: mcp
    volumes:
      - pgdata:/var/lib/postgresql/data

  redis:
    image: redis:7
    command: ["redis-server","--save","900","1"]

  mcp-server:
    image: archestraai/mcp-server:latest
    ports:
      - "8080:8080"
    environment:
      DATABASE_URL: postgres://mcp:mcp@postgres:5432/mcp
      REDIS_URL: redis://redis:6379
      SECRET_KEY: change-me
      ENABLE_COSTING: "true"
    depends_on:
      - postgres
      - redis

volumes:
  pgdata:

Environment variables (most common):

• DATABASE_URL — Postgres connection string
• REDIS_URL — Redis connection string for caching and locks
• SECRET_KEY — application secret for signing tokens
• ENABLE_COSTING — enable cost tracking (true/false)
• METRICS_PORT — Prometheus metrics endpoint port
• LOG_LEVEL — debug/info/warn/error

Kubernetes (simplified Deployment snippet):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mcp-server
spec:
  replicas: 2
  template:
    spec:
      containers:
        - name: mcp-server
          image: archestraai/mcp-server:latest
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: mcp-secrets
                  key: database_url
            - name: REDIS_URL
              value: redis://redis:6379
            - name: SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: mcp-secrets
                  key: secret_key
          ports:
            - containerPort: 8080

For production, secure secrets via your cluster secrets manager, configure TLS, and integrate with your identity provider.

Available Resources

Core HTTP endpoints (typical paths — confirm from your deployment):

• /api/v1/registry — manage models and backends (CRUD)
• /api/v1/gateway — entry point for inference requests and routing
• /api/v1/orchestrator — define orchestration plans and policies
• /api/v1/credentials — store and rotate backend credentials
• /api/v1/costs — usage and cost reports
• /api/v1/chat — session and message APIs for chat applications
• /metrics — Prometheus metrics
• /docs or /swagger — API documentation (if enabled)

Tooling and integrations:

• Web UI for registry, credentials, and basic dashboards
• SDKs / client libraries (check GitHub repo)
• Prometheus metrics and Grafana dashboards for monitoring
• Audit logs for compliance and billing

GitHub: https://github.com/archestra-ai/archestra

Use Cases

1. Multi-backend cost-aware routing

   • Register multiple backends (OpenAI, Anthropic, internal LLM) in the registry with per-backend cost rates.
   • The orchestrator routes requests to the lowest-cost backend that meets capability constraints (e.g., model size or streaming support).
   • Result: automated cost savings while preserving SLA.

2. Centralized credential management and rotation

   • Store API keys in the MCP credentials store scoped by team or environment.
   • Rotate a compromised key centrally; the gateway immediately uses the new key without changing client code.

3. Model governance and version control

   • Teams register model metadata and create canonical names (model aliases).
   • Deployments reference aliases; operators can point an alias to a new model version for safe rollouts.

4. Chat platform with conversation state

   • Use /api/v1/chat to maintain session history and context for multi-turn conversations.
   • Leverage orchestrator policies to call different models for system messages, user messages, or summarization tasks.

5. Billing and chargeback

   • Export /api/v1/costs reports per-team, per-project, or per-backend.
   • Use cost data to implement internal chargeback or budgeting for LLM usage.

Getting started tips

• Begin by deploying a single-node Docker Compose stack for evaluation.
• Register one backend and a dummy model, then send a simple gateway request to validate end-to-end flow.
• Enable metrics and inspect Prometheus to validate usage records and cost calculation.
• Review and tighten credential scopes before moving traffic from development to production.

For full API reference, adapters, and deployment examples, see the project repository: https://github.com/archestra-ai/archestra.