Prefect MCP Server for ETL/ELT Orchestration

Overview

The Prefect MCP Server provides a lightweight MCP (Model Context Protocol) adapter that uses Prefect (Server or Cloud) and the prefect Python client to run and manage ETL/ELT workflows. It exposes a small HTTP API that lets other systems — including model-driven agents and orchestration layers — discover and invoke Prefect flows as tools in an MCP-compatible way.

This server is useful when you want to treat data pipelines as callable tools from higher-level automation or model-based decision systems. It bridges model-driven workflows and production-grade orchestration offered by Prefect: flows remain managed, scheduled, and observable in Prefect, while external callers get a predictable, tool-like interface for triggering and querying runs.

Features

• Expose Prefect flows as MCP-style tools callable over HTTP
• Support for Prefect Server (self-hosted) and Prefect Cloud (SaaS)
• Use the official prefect Python client to create flow runs and inspect status
• Lightweight HTTP server implemented to integrate easily into existing stacks
• Basic endpoints to list available tools/flows and to start/query runs
• Configurable via environment variables for tokens, API URLs, and defaults

Installation / Configuration

Install directly from the repository (pip from GitHub):

pip install git+https://github.com/allen-munsch/mcp-prefect.git

Common environment variables used to configure the server:

Example .env:

PREFECT_API_URL=https://api.prefect.cloud
PREFECT_API_KEY=sk.your_prefect_key_here
MCP_PREFECT_HOST=0.0.0.0
MCP_PREFECT_PORT=8080

Run the server locally with Uvicorn (example module path — adjust if you install differently):

uvicorn mcp_prefect.server:app --host ${MCP_PREFECT_HOST:-0.0.0.0} --port ${MCP_PREFECT_PORT:-8080}

If you use Docker, create a simple Dockerfile that installs the package and runs Uvicorn, or run the container in your orchestration environment and pass environment variables through your container runtime.

Available Resources

Typical HTTP endpoints provided by the server (paths and exact names may vary; check the running server’s OpenAPI docs):

• GET /tools — List available MCP tools (mapped Prefect flows)
• POST /run — Start a flow run (tool invocation). Body includes flow identifier and parameters.
• GET /runs/{run_id} — Query run status and logs

Example: list tools

curl http://localhost:8080/tools
# returns JSON: [{ "id": "etl_daily", "description": "...", "parameters": {...} }, ...]

Example: start a flow

curl -X POST http://localhost:8080/run \
  -H "Content-Type: application/json" \
  -d '{"flow_id":"etl_daily","parameters":{"date":"2026-04-01"}}'

Python client example showing a simple invocation wrapper that calls the MCP server:

import requests

MCP_URL = "http://localhost:8080"

def run_flow(flow_id: str, parameters: dict = None):
    payload = {"flow_id": flow_id, "parameters": parameters or {}}
    resp = requests.post(f"{MCP_URL}/run", json=payload)
    resp.raise_for_status()
    return resp.json()  # likely contains run_id and initial status

result = run_flow("etl_daily", {"date": "2026-04-01"})
print(result)

Use Cases

• On-demand ETL invoked from a model: A model evaluates current KPIs and requests fresh ETL runs through the MCP server when data looks stale. The MCP server starts the Prefect flow and returns run metadata that the model or orchestrator can poll.
• Human-in-the-loop backfills: Data engineers or analysts trigger parameterized backfills (e.g., start_date/end_date) through a simple HTTP form or command-line tool that uses the MCP server API, while Prefect maintains logs, retries, and notifications.
• Integrating third-party systems: External SaaS services or automation platforms that can call HTTP endpoints can trigger Prefect-managed workflows without embedding Prefect client logic; the MCP server handles Prefect authentication and flow mapping.
• Hybrid mode (Cloud + local): Use Prefect Cloud for observability and scheduling while running the MCP server in your VPC — calls to start flows remain local and secure, while Prefect Cloud manages state and UI.

Links and Resources

• GitHub repository: https://github.com/allen-munsch/mcp-prefect
• Prefect docs: https://docs.prefect.io
• Prefect Python client: https://pypi.org/project/prefect

If you need to extend the server, explore the repository code to add custom flow-to-tool mappings, authentication layers, or richer result handling (webhooks, callbacks, or streaming logs).