VI
OfficialSearchMedia

VideoDB Director MCP Server for AI Video Workflows

Create AI video workflows with MCP server for automatic editing, moderation, voice cloning, highlights and searchable video moments via APIs and chat.

View on GitHub

Overview

The VideoDB Director MCP Server implements a Model Context Protocol (MCP) compatible backend for AI-driven video workflows. It centralizes video metadata, transcripts, timeline segments, and action intents so models and agents can discover, query, and operate on video content via a consistent HTTP/WebSocket API. The server is designed to be used as the context layer between video storage, ML models (speech-to-text, vision, moderation, TTS/voice-clone), and orchestration agents that drive editing, moderation, and highlight extraction.

By exposing structured contexts (videos, segments, annotations, searchable moments) and action endpoints (trim, label, moderate, synthesize voice), the MCP server makes it straightforward to build workflows such as automatic editing, content moderation, voice cloning, and interactive chat-driven video search. Developers can plug models and tools into the MCP server to power both automated pipelines and conversational experiences that reference precise video moments.

Features

  • MCP-compatible HTTP and WebSocket API for context management and eventing
  • Store and query video-level metadata and time-aligned segments (shots, scenes, moments)
  • Attach transcripts, labels, embeddings and moderation signals to segments
  • Search moments by text, tags, and embeddings for semantic video navigation
  • Action endpoints to request edits, moderation, voice synthesis, and exports
  • Webhook / event stream support for async processing and agent orchestration
  • Lightweight storage backend with pluggable persistence (local files, S3, DB)
  • Designed for integration with agents, LLMs and ML model services

Installation / Configuration

Two common deployment approaches: Docker (recommended for quick start) and running from source. Replace image/name or commands with the specific artifact published by your organization if different.

Quickstart — Docker

# pull and run (example image name)
docker run -d \
  -p 8080:8080 \
  -e MCP_PORT=8080 \
  -e STORAGE_PATH=/data/mcp \
  -v "$(pwd)/mcp-data":/data/mcp \
  --name director-mcp \
  video-db/director-mcp:latest

Run from source (example)

# clone the repository
git clone https://github.com/video-db/agent-toolkit.git
cd agent-toolkit/modelcontextprotocol

# install dependencies (example; follow language/runtime instructions in repo)
# Python example:
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python -m modelcontextprotocol.server --port 8080 --storage ./data

# Node example:
npm install
npm run dev

Configuration (environment variables)

MCP_PORT=8080          # HTTP port to bind
STORAGE_PATH=/data     # local storage path for contexts and indexes
LOG_LEVEL=info         # debug|info|warn|error
WEBHOOK_URL=...        # external webhook for async events
EMBEDDING_PROVIDER=... # URL/credentials for embedding service

Refer to the repository README for exact runtime flags and supported persistence backends.

Available Resources

Common MCP resources and endpoints (examples; server implements similar REST/WebSocket APIs):

EndpointMethodPurpose
/contextsPOST/GETCreate or list video contexts (metadata, duration, source URL)
/contexts/{id}/segmentsPOST/GETAdd or query time-aligned segments (shots, scenes, clips)
/segments/{id}/annotationsPOST/GETAttach transcripts, labels, embeddings, moderation flags
/searchPOSTSemantic or keyword search over transcripts, tags, embeddings
/actionsPOST/GETSubmit and track edit/moderation/voice synthesis actions
/events (WS)WebSocketSubscribe to async notifications (completed jobs, new segments)

Tooling and integrations

  • WebSocket event stream for real-time agent reactions
  • Webhooks for external processing pipelines
  • Pluggable embedding and model providers for semantic search and labeling

Use Cases

  1. Automatic highlight extraction

    • Ingest video and transcript into MCP as a context and segments.
    • Run a model to score segment “interestingness” and write scores to annotations.
    • Query /search or segments sorted by score and request an /actions export to produce a highlights reel.

    Example: mark segments with score > 0.8 and call action to stitch clips:

    curl -X POST http://localhost:8080/actions \
  -H "Content-Type: application/json" \
  -d '{"type":"stitch_clips","context_id":"vid123","filters":{"score_gt":0.8}}'

  2. Moderation pipeline

    • Transcribe audio and attach text annotations to segments.
    • Run moderation models that add labels (violence, hate, adult) to segment annotations.
    • Use /search to find flagged moments and create review tasks or remove clips automatically.

  3. Voice cloning & dubbing

    • Store speaker metadata and reference audio for cloning.
    • Create an /actions request to synthesize dialogue for a given segment using a voice clone model.
    • Return generated audio asset linked to the segment for downstream editing.

  4. Conversational video search via chat

    • An agent queries the MCP server to find “funniest moment around minute 2” by issuing a semantic search.
    • The server returns matching segments with timestamps and transcript snippets.
    • The chat interface displays a clip preview and can trigger an edit/download action.

Getting Started Tips

  • Model outputs should annotate segments rather than overwrite contexts — keep
Tags:mediasearch