MCP Server for Unstructured Platform Workflows

Overview

The MCP (Model Context Protocol) server is a lightweight orchestration layer for running unstructured-data processing workflows on the Unstructured Platform. It provides a standard API to register and execute workflows composed of document ingestion, chunking, model calls, tool integrations, and enrichment steps. By centralizing workflow logic, the MCP server makes it easier to deploy reproducible processing pipelines and to integrate model-in-the-loop steps with external tools and storage.

This server is useful when you want a consistent runtime for document-centric pipelines—regardless of whether the components are local functions, remote API calls, or third-party tools. It exposes endpoints to manage workflow definitions, start executions, and observe progress or results. Developers can run the server locally for development or deploy it as a container in production.

Features

• REST API for registering, listing, and executing workflows
• Pluggable tool integrations (e.g., OCR, embedding, summarization, vector stores)
• Execution lifecycle management: start, status, cancel, result retrieval
• Configurable storage backends for intermediate artifacts
• Authentication support via API keys or tokens
• Docker-friendly: run locally or inside orchestration platforms
• Webhooks or polling options to receive completion events
• Lightweight and framework-agnostic so you can extend it with custom tools

Installation / Configuration

You can run the MCP server either from source (Python) or as a Docker container.

From source (recommended for development):

git clone https://github.com/Unstructured-IO/UNS-MCP.git
cd UNS-MCP
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
export MCP_HOST=0.0.0.0
export MCP_PORT=8080
export MCP_API_KEY=your_api_key_here
python -m mcp.server

Using Docker:

# Build image (if building locally)
docker build -t uns-mcp:latest .

# Run container
docker run -p 8080:8080 \
  -e MCP_API_KEY=your_api_key_here \
  -e STORAGE_PATH=/data \
  -v $(pwd)/data:/data \
  uns-mcp:latest

Common environment variables

Configuration can also be provided via a YAML or JSON file if the server supports it—check the repo for examples/config templates.

Available Resources

Typical REST endpoints exposed by the server:

CLI tooling (if included) may provide commands such as mcp register, mcp run, and mcp status.

Available tool types (examples)

• extractor (text, metadata)
• segmenter (chunk/split)
• model (LLM or classifier)
• embedding (vectorizer)
• storage (S3/local)
• retriever (vector similarity)

Use Cases

1. Document ingestion + summarization

• Register a workflow that ingests PDFs, extracts text, chunks into passages, calls a summarization model, and stores summaries.
• Start executions via the API with a document URL and retrieve the final summary.

Example POST to start execution:

curl -X POST http://localhost:8080/execute \
  -H "Authorization: Bearer your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "workflow_id": "pdf-summarize-v1",
    "inputs": {
      "document_url": "https://example.com/report.pdf"
    }
  }'

2. Vector indexing pipeline

• A workflow that extracts text, creates embeddings, and pushes vectors to a vector store for semantic search. Useful for building retrieval-augmented systems.

3. Human-in-the-loop validation

• Execute a workflow that produces candidate annotations and emits webhooks for manual review. After human approval, the server continues pipeline steps (e.g., publishing metadata).

4. Multi-tool orchestration

• Chain OCR -> language model for extraction -> post-processing tool (regex normalizer) -> storage. The MCP server coordinates the tool sequence and preserves intermediate artifacts for debugging.

Example: Simple Python client

import requests

MCP_URL = "http://localhost:8080"
API_KEY = "your_api_key_here"

resp = requests.post(
    f"{MCP_URL}/execute",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "workflow_id": "pdf-summarize-v1",
        "inputs": {"document_url": "https://example.com/annual_report.pdf"}
    }
)
data = resp.json()
execution_id = data["execution_id"]

# poll status
while True:
    r = requests.get(f"{MCP_URL}/executions/{execution_id}",
                     headers={"Authorization": f"Bearer {API_KEY}"})
    status = r.json()
    if status["state"] in ("completed", "failed"):
        print(status)
        break

Tips for Developers

• Start by running the server locally and exploring /tools and /workflows to understand built-in integrations.
• Use versioned workflow IDs so you can update pipelines without breaking existing executions.
• Store intermediate artifacts (for debugging) but configure retention policies for production.
• Implement retries and idempotency in custom tools to handle transient failures.

The MCP server is intended to be extended—add custom tool modules, hook in your storage backends, and integrate monitoring to fit your unstructured-data processing needs.