Html2md MCP Server: Playwright HTML to Markdown

Overview

Html2md MCP Server is a small Model Context Protocol (MCP) service that converts web pages and HTML fragments into compact Markdown. It combines a headless browser (Playwright) for JavaScript-rendered sites, trafilatura for content extraction, and BeautifulSoup4 for targeted cleaning and conversion. The service implements an MCP-compatible tool so language models and other MCP-aware clients can request HTML-to-Markdown conversions programmatically.

This is useful when you want readable, minimal Markdown summaries of web content for ingestion into LLM contexts, note-taking systems, or downstream text processing. The server supports authenticated access, rendering pages that require JavaScript, and configuration options to control extraction and output length.

Features

• Converts URLs or inline HTML to compact, structured Markdown
• Uses Playwright to render JavaScript-heavy pages before extraction
• Uses trafilatura for robust content extraction (boilerplate removal, language handling)
• Uses BeautifulSoup4 for post-processing and HTML cleanup
• Authentication support (Bearer token) to protect the service
• MCP-compatible tool endpoint so models and MCP clients can call it like any other tool
• Configurable options: render_js toggle, max output length, and request-specific credentials for sites requiring basic auth

Installation / Configuration

Prerequisites:

• Python 3.9+
• git, pip
• (Optional) Docker

Clone and install with pip:

git clone https://github.com/sunshad0w/html2md-mcp.git
cd html2md-mcp
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Install Playwright browsers
playwright install
# or install only Chromium
playwright install chromium

Environment variables (example):

export MCP_PORT=8080
export AUTH_TOKEN="changeme"         # token required for requests
export PLAYWRIGHT_HEADLESS=true
export MAX_MARKDOWN_CHARS=20000

Start the server (example):

python server.py --port ${MCP_PORT:-8080}

Docker (example Dockerfile + docker-compose):

Dockerfile (simple example):

FROM python:3.11-slim
WORKDIR /app
COPY . /app
RUN pip install -r requirements.txt
RUN playwright install --with-deps
ENV PLAYWRIGHT_HEADLESS=true
CMD ["python", "server.py", "--port", "8080"]

docker-compose.yml snippet:

version: "3.8"
services:
  html2md:
    build: .
    ports:
      - "8080:8080"
    environment:
      - AUTH_TOKEN=changeme
      - PLAYWRIGHT_HEADLESS=true

Available Resources

The server leverages these libraries and tools:

• Playwright — headless browser automation for rendering JS pages
• trafilatura — main text extraction engine that strips boilerplate and extracts main content
• BeautifulSoup4 — HTML parsing and cleanup
• MCP (Model Context Protocol) — tool registration and invocation model; the server exposes an MCP-compatible tool named “html2md”

Repository: https://github.com/sunshad0w/html2md-mcp

API / Tool Interface

The server exposes an MCP-compatible tool named “html2md”. It accepts a JSON input that should contain either a url or html field. Common optional fields control rendering and output size.

Typical request (HTTP POST to the tool endpoint):

• Endpoint: POST /tools/html2md
• Headers:
  • Authorization: Bearer
  • Content-Type: application/json

  Request body schema (JSON):

  {
    "url": "https://example.com/article",
    "html": null,
    "render_js": true,
    "max_chars": 10000,
    "basic_auth": { "username": "user", "password": "pass" }
  }
  

  Response (JSON):

  {
    "markdown": "# Article title\n\nThe converted content ...",
    "meta": {
      "source": "https://example.com/article",
      "language": "en",
      "extraction_method": "trafilatura+playwright"
    }
  }
  

  Authentication: include Authorization: Bearer <AUTH_TOKEN> with each request unless the server is configured to be open.

  Use Cases

  • Ingesting web articles into a knowledge base: fetch a URL, convert to compact Markdown, store as an article record for semantic search or LLM prompt context.

    • Example: automated pipeline fetches blog post -> html2md -> store Markdown + metadata.

  • Summarization and prompt prep: retrieve JS-heavy news pages, convert to Markdown, then pass into an LLM summarizer or question-answering tool with tight token budgets.

  • Data collection for training: extract readable textual content without boilerplate and save it as Markdown for annotation or training corpora.

  • Authenticated scraping for private dashboards: use embedded basic_auth in the request to log in to a protected resource (or use Playwright to navigate login flows in more advanced configs) and extract content.

  Concrete curl example — convert a URL:

  curl -sS -X POST "http://localhost:8080/tools/html2md" \
    -H "Authorization: Bearer changeme" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://example.com/some-article",
      "render_js": true,
      "max_chars": 8000
    }'
  

  Convert inline HTML:

  curl -sS -X POST "http://localhost:8080/tools/html2md" \
    -H "Authorization: Bearer changeme" \
    -H "Content-Type: application/json" \
    -d '{
      "html": "<html><body><

  Tags:web-scraping