PI
Media

Pixelle MCP Server Omnimodal ComfyUI Converter

Convert ComfyUI workflows into MCP server tools with zero code—enable Text, Image, Sound & Video generation via an omnimodal Chainlit web interface.

View on GitHub
Quick Install
npx -y @AIDC-AI/Pixelle-MCP

Overview

Pixelle MCP Server Omnimodal ComfyUI Converter lets developers expose ComfyUI workflows as MCP (Model Context Protocol) tools without writing glue code. It detects ComfyUI workflows, wraps them as standardized MCP tools, and serves them through a Chainlit-based web interface that supports text, image, audio, and video modalities. The result is a single omnimodal endpoint you can call from UI clients, assistants, or downstream systems to run complex generative pipelines.

This approach is useful when you want to reuse visual node graphs (ComfyUI) as programmatic building blocks in conversational or multimodal applications. Instead of translating workflows into custom APIs, Pixelle automatically maps workflow inputs/outputs to MCP tool signatures and hosts a developer-friendly UI and HTTP API for invoking those tools.

Features

  • Zero-code conversion of ComfyUI workflows into MCP tools
  • Multi-modality: Text, Image, Sound, and Video tool generation
  • Chainlit web UI for interactive exploration and invocation
  • Automatic discovery of workflows and node I/O mapping
  • HTTP MCP-compatible API for programmatic integration
  • Configurable ComfyUI endpoint and tool sandboxing
  • Optional API key / token-based access control
  • Local-first: runs against a local or remote ComfyUI instance

Installation / Configuration

Prerequisites:

  • Python 3.10+
  • A running ComfyUI instance (REST API)
  • Optional: Chainlit (for the web UI) — pip will install it if missing

Quick setup (example):

git clone https://github.com/AIDC-AI/Pixelle-MCP.git
cd Pixelle-MCP
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Environment variables (example):

# Point to your ComfyUI server (default: http://localhost:8188)
export COMFYUI_URL="http://localhost:8188"

# Port to expose MCP HTTP API
export MCP_PORT=8000

# Port for Chainlit web UI
export CHAINLIT_PORT=9000

# Optional API key to protect endpoints
export PIXELLE_API_KEY="changeme"

Start the MCP server (example using uvicorn):

# Using the packaged ASGI app
uvicorn pixelle_mcp.app:app --host 0.0.0.0 --port ${MCP_PORT:-8000}

Start the Chainlit UI (if included):

# Launch Chainlit UI that talks to the MCP server
chainlit run chainlit_app.py --port ${CHAINLIT_PORT:-9000}

Configuration file (optional yaml config/config.yaml):

comfyui_url: "http://localhost:8188"
mcp:
  host: "0.0.0.0"
  port: 8000
security:
  api_key: "changeme"
workflows:
  auto_discover: true
  include: []   # names or ids to include
  exclude: []   # names or ids to exclude

Available Tools / Resources

Pixelle exposes several resource endpoints and generated tools:

  • GET /health — server status
  • GET /workflows — discovered ComfyUI workflows (metadata)
  • POST /mcp/invoke — MCP-style invocation endpoint
  • POST /mcp/discover — trigger a (re)scan of ComfyUI workflows
  • Chainlit web UI — interactive tool discovery and execution

Generated tool types (examples):

  • sd_generate — image generation via a Stable Diffusion workflow
  • audio_synth — short audio clip generation workflow
  • video_render — frame sequence rendering + composition workflow
  • text_refiner — text-prompt paraphrase / refinement flow

Example cURL to call a generated tool:

curl -X POST "http://localhost:8000/mcp/invoke" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${PIXELLE_API_KEY}" \
  -d '{
    "tool": "sd_generate",
    "inputs": {
      "prompt": "A serene mountain lake at sunset",
      "width": 1024,
      "height": 576,
      "seed": 42
    }
  }'

Response will follow MCP conventions: status, tool outputs (base64 image/audio or URL to artifact), and optional logs.

Use Cases

  • Rapid prototyping: Turn your existing ComfyUI node graphs into callable tools for chatbots and agents without rewriting workflows.

    • Example: Expose a ComfyUI SDXL pipeline as “sd_generate” and call it from a conversational agent to produce images on demand.

  • Multimodal assistants: Combine text generation, image generation, and audio synthesis into a single assistant flow.

    • Example: Generate an image, produce a spoken description of the image, and return both artifacts in one conversation turn.

  • Batch media production: Programmatically render sequences for video output.

    • Example: Use a “video_render” workflow to render per-frame images from prompts and then compose them into an MP4, returned as a downloadable artifact.

  • Experimentation and tuning: Use the Chainlit UI to

Tags:media