Supadata MCP Server: YouTube TikTok X Data API

Overview

Supadata MCP Server is an open-source Model Context Protocol (MCP) server that converts media URLs — specifically YouTube, TikTok, and X (formerly Twitter) videos and pages — into structured, machine-friendly data. The server extracts metadata, text (titles, descriptions, captions/transcripts, article text), thumbnails, and optionally comments or other context so applications and AI models can consume consistent JSON rather than raw HTML or platform-specific APIs.

By standardizing web and social media content into a compact schema, the MCP server accelerates workflows such as search indexing, summarization, dataset generation, content moderation, and prompt augmentation for LLMs. It is designed to be self-hosted or deployed as a service in front of downstream model pipelines, keeping data processing close to your application and under your control.

Features

• Convert YouTube, TikTok, and X pages/videos to structured JSON
• Extract metadata: title, author, published date, duration, thumbnails
• Extract text assets: description, captions/subtitles, transcripts, article body
• Optional extraction of comments, likes, and engagement metrics (where available)
• Consistent, schema-driven output suitable for LLM context windows and ingestion
• HTTP API suitable for server-to-server calls or local development
• Simple deployment options (Docker or build from source)

Installation / Configuration

Clone the repository and run via Docker, or build and run from source.

Clone repo:

git clone https://github.com/supadata-ai/mcp.git
cd mcp

Run with Docker (recommended for quick start):

# Pull image (replace with official image name if available)
docker run -p 3000:3000 \
  -e MCP_PORT=3000 \
  -e LOG_LEVEL=info \
  -e ALLOWED_ORIGINS='*' \
  supadata/mcp:latest

Run via docker-compose:

version: "3.8"
services:
  mcp:
    image: supadata/mcp:latest
    ports:
      - "3000:3000"
    environment:
      MCP_PORT: 3000
      LOG_LEVEL: info
      ALLOWED_ORIGINS: "https://your.app"

Build and run from source (example for Go-based repo):

# build (if the project is Go)
go build -o mcp ./cmd/mcp
./mcp --port 3000 --log-level info

Common environment variables:

• MCP_PORT: HTTP listen port (default 3000)
• LOG_LEVEL: debug | info | warn | error
• ALLOWED_ORIGINS: CORS allowed origins
• API_KEY: (optional) token for simple auth on server endpoints

Adjust configuration via environment variables or configuration file as provided by the repo.

Available Resources

The server exposes HTTP endpoints that accept URLs or identifiers and return structured JSON. Typical available resources include:

Standard JSON fields returned (may vary by resource and availability):

• id, platform
• title, description
• author (name, id, handle)
• published_at, duration_seconds
• thumbnails: array of URLs + dimensions
• captions / transcripts: array or text block with timestamps
• comments: optionally included, paginated
• raw_html / canonical_url (optional)

Note: Availability of fields depends on platform policies and the content being scraped.

Use Cases

1. Summarization and Prompt Augmentation

• Extract a video transcript and feed it into an LLM to generate summaries, key points, or Q&A. This allows long-form video content to be reduced into concise prompts for AI agents.

Example curl:

curl "http://localhost:3000/v1/extract?url=https://www.youtube.com/watch?v=VIDEO_ID" \
  -H "Authorization: Bearer $API_KEY"

2. Search Indexing and Discovery

• Index title, description, and transcript text in a search engine (Elasticsearch, Meilisearch) to enable full-text search across multimedia sources.

3. Dataset Generation for ML

• Produce structured records (metadata + transcript + engagement metrics) to build training datasets for recommendation models, summarization tasks, or moderation classifiers.

4. Content Moderation and Safety

• Extract text and comments for automated policy checks or human review workflows. The MCP output can be passed to classifiers or rule engines.

5. Feed Aggregation & Embedding Generation

• Normalize disparate platforms into a single schema to compute embeddings for semantic feeds, content similarity, or recommendation signals.

Example Output (excerpt)

{
  "id": "VIDEO_ID",
  "platform": "youtube",
  "title": "Example Video",
  "author": { "name": "Creator", "id": "creator_id", "handle": "creator_handle" },
  "published_at": "2024-03-01T12:00:00Z",
  "duration_seconds": 360,
  "thumbnails": [
    { "url": "https://i.ytimg.com/...", "width": 1280, "height": 720 }
  ],
  "transcript": [
    { "start": 0.0, "text": "Intro line..." },
    { "start": 5.2, "text": "Next line..." }
  ],
  "description": "Full description text..."
}

Next Steps

• Review the repo README for platform-specific usage notes and rate limits.
• Integrate the MCP server behind an API layer to enforce auth and caching.
• Add monitoring and logging for scraping failures or content format changes.

This server is a practical component to normalize web and social media content into structured data for applications and AI pipelines. Use it as a preprocessing step to keep downstream models lean and focused on higher-level tasks.