DataHub MCP Server: Metadata Search, Lineage, SQL

Overview

The DataHub MCP Server implements the Model Context Protocol (MCP) on top of DataHub, exposing metadata and catalog operations to models, agents, and developer tools. It provides a small HTTP API that lets external systems search metadata, traverse dataset lineage, and issue SQL queries against configured query endpoints, so models can make metadata-aware decisions without direct access to the underlying data platforms.

This server acts as a bridge between DataHub’s metadata graph (GMS) and tools that need structured metadata: discovery UIs, LLM agents, governance tooling, and automated pipelines. By centralizing metadata access through a protocol-compatible API, you get consistent search, lineage exploration, and query execution workflows that simplify integration and accelerate analytics workflows.

Features

• Search data assets, schemas, and metadata using flexible query parameters
• Traverse lineage upstream/downstream for datasets and columns
• Run SQL queries via configured query engines and return structured results
• Standard HTTP API compatible with MCP clients and agent frameworks
• Pluggable configuration for DataHub GMS endpoint and query backends
• Lightweight deployment: containerized or run as a Python service

Installation / Configuration

Clone and run the server from source, or run the official container. Below are common ways to get started.

Clone and run locally (development)

git clone https://github.com/acryldata/mcp-server-datahub.git
cd mcp-server-datahub
pip install -r requirements.txt
uvicorn mcp_server.app:app --host 0.0.0.0 --port 8080

Run with Docker

docker build -t mcp-server-datahub .
docker run -e DATAHUB_GMS_URL="http://datahub-gms:8080" \
           -e DATAHUB_GMS_TOKEN="your-token" \
           -p 8080:8080 \
           mcp-server-datahub

Example docker-compose snippet

version: "3.7"
services:
  mcp-server:
    image: acrylidata/mcp-server-datahub:latest
    environment:
      DATAHUB_GMS_URL: "http://datahub-gms:8080"
      DATAHUB_GMS_TOKEN: "your-token"
      QUERY_BACKEND_URL: "http://query-service:8000"
    ports:
      - "8080:8080"

Common environment variables

Adjust these to match your DataHub deployment and any external query engines you want the server to use for SQL execution.

Available Resources

The server exposes MCP-style HTTP endpoints (paths may vary based on version). Typical endpoints include:

• GET /health
  • Returns basic server status.
• POST /search
  • Search for datasets, tables, views, or other DataHub entities.
  • Request body example:

    { "query": "customer orders", "type": "dataset", "limit": 10 }
    

• POST /lineage
  • Fetch upstream or downstream lineage for an entity (dataset, table, or column).
  • Request body example:

    { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,db.table,PROD)", "direction": "upstream", "depth": 3 }
    

• POST /sql
  • Execute SQL via the configured query backend and return rows, schema, and execution metadata.
  • Request body example:

    { "sql": "SELECT id, COUNT(*) FROM analytics.events GROUP BY id LIMIT 10", "catalog": "redshift-prod" }
    

Responses are JSON and include metadata such as entity URNs, display names, schemas, lineage edges, and query result rows. The exact response shape follows the MCP server conventions and aims to be machine-friendly for LLMs and agents.

Use Cases

• Metadata-aware question answering
  • An agent uses /search to find the “orders” dataset, inspects its schema, and then runs targeted SQL via /sql to produce an answer that references column names and counts.
• Impact analysis before schema changes
  • Query /lineage upstream/downstream to identify jobs, dashboards, and downstream datasets impacted by a column rename or type change.
• Automated data discovery and catalog enrichment
  • Periodically search and summarize new datasets, fetch schema details, and push model-generated descriptions back into DataHub (using DataHub’s APIs).
• Debugging data quality issues
  • Combine search results with lineage to trace where anomalous values originate and run SQL checks against candidate datasets.

Getting Help & Next Steps

• Start by connecting the server to your DataHub GMS (set DATAHUB_GMS_URL and DATAHUB_GMS_TOKEN).
• If you plan to use SQL execution, point QUERY_BACKEND_URL at a compatible query service or proxy.
• Review your DataHub permissions and tokens to ensure the MCP server can read entities and lineage.
• Explore the repository for example clients and integration tests to see request/response examples in practice.

This server is intended for developers building LLM agents, BI tooling, and automation around DataHub metadata. Use it to centralize metadata access and enable smarter, metadata-driven analytics.