MCP Server: Fast Token-Efficient Website-to-Markdown

Overview

MCP Server: Fast Token-Efficient Website-to-Markdown is a small web service that converts arbitrary web pages into clean, token-efficient Markdown suitable for consumption by language models and other automated pipelines. It uses Mozilla Readability under the hood to extract the main article content and metadata, then converts the cleaned HTML to Markdown so you get compact, semantically meaningful text instead of noisy HTML, ads, or navigation chrome.

The server is built to be used as an MCP (Model Context Protocol) tool: you can call it from an LLM orchestration layer or directly via HTTP to retrieve only the parts of a page that matter. That makes it especially useful when you want to reduce token use, improve retrieval-augmented generation (RAG), or pre-process large numbers of pages before feeding them to an LLM.

Features

• Fast, focused extraction using Mozilla Readability to isolate article content
• Output in clean Markdown (and raw HTML/JSON where useful)
• Token-efficient: strips boilerplate, ads, and unrelated page chrome
• Simple HTTP API designed for integration with MCP-enabled workflows
• Optional Docker image for quick deployment
• Configurable timeout and content sanitization settings
• Lightweight and suitable for local or containerized deployment

Installation / Configuration

Clone the repository and run the server locally (Node/npm example):

git clone https://github.com/just-every/mcp-read-website-fast.git
cd mcp-read-website-fast
npm install
npm run start

Run via Docker (recommended for quick, reproducible deployment):

# Pull an image (if published) or build locally
docker build -t mcp-read-website-fast .
docker run --rm -p 3000:3000 \
  -e PORT=3000 \
  -e TIMEOUT_SECONDS=10 \
  -e LOG_LEVEL=info \
  mcp-read-website-fast

Common environment variables

• PORT — HTTP server port (default 3000)
• TIMEOUT_SECONDS — max time to fetch & parse a page (recommended 10)
• LOG_LEVEL — logging verbosity (e.g., debug, info, warn, error)
• ALLOWED_HOSTS — optional comma-separated host allowlist

If you run in production behind an ingress, ensure proper request size and timeout settings are applied at the proxy.

Available Resources

Typical HTTP endpoints (examples — check the repo for exact routes):

• POST /extract

  • Request JSON: { “url”: “https://example.com/article”, “format”: “markdown” }
  • Response JSON: { “title”: “…”, “url”: “…”, “content_html”: “…”, “content_markdown”: “…”, “excerpt”: “…”, “length”: 1234 }

• POST /mcp

  • Accepts MCP-style tool calls so an LLM orchestration layer can request a URL and receive a compact context payload suitable for model input.

Example curl (extract Markdown):

curl -X POST http://localhost:3000/extract \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com/article","format":"markdown"}'

Example response schema

Use Cases

• Retrieval-Augmented Generation (RAG): fetch targeted article content as compact Markdown to reduce token usage and provide focused context for an LLM.
• Data ingestion for knowledge bases: convert large numbers of articles into normalized Markdown before indexing.
• Research pipelines: programmatically scrape and store readable article content without noise.
• Summarization and QA: provide a clean article body to a summarization model to improve accuracy and reduce hallucination risk.
• Automated monitoring: extract and diff main content across versions of a page to detect meaningful changes.

Concrete example — integrate with an LLM workflow

1. LLM identifies a URL to consult.
2. LLM orchestrator calls MCP Server /mcp (or /extract) to retrieve content_markdown.
3. The orchestrator sends the returned Markdown as the model context, keeping the prompt small and relevant.

Tips and Troubleshooting

• If pages are partially extracted, increase TIMEOUT_SECONDS or enable a headless-browser fetch option (if provided).
• Test with a variety of news/blog formats—Readability works best for article-like pages; highly dynamic single-page apps may require rendering first.
• Use ALLOWED_HOSTS to prevent abuse in publicly exposed deployments.
• If you need additional sanitization (remove images or scripts), post-process content_markdown before sending to models.

Links and Further Reading

• Mozilla Readability: https://github.com/mozilla/readability
• MCP (Model Context Protocol) patterns: use the server as a “tool” to provide external context to language models
• Repository: https://github.com/just-every/mcp-read-website-fast

This server is a practical building block when you need clean, token-efficient website content for model-driven systems and automated pipelines.