Vibe Check MCP Server for Self-Improving Agent Alignment

Overview

Vibe Check MCP Server is a lightweight Model Context Protocol (MCP) server designed to help LLM-driven agents self-assess and self-improve their behavior. It exposes a simple API that “vibe-checks” agent outputs against alignment objectives such as staying on-scope, avoiding unnecessary bloat, surfacing tunnel-vision risks, and catching misreads or hallucinations. The server can run alongside agents and provide rapid feedback, scoring, and corrective suggestion chains so agents can adapt their next actions.

By embedding a dedicated alignment-checking layer between a model and downstream actions, teams can reduce regressions from prompt drift, scope creep, and overconfident hallucinations. The MCP server supports configurable checks, thresholding, and retraining hooks, making it easy to integrate into development, CI, and monitoring pipelines for safety-focused agent deployments.

Features

• Lightweight MCP-compatible HTTP server for alignment checks
• Pluggable checks for: scope creep, bloat, tunnel vision, and misreads
• Scoring, confidence calibration, and corrective suggestions
• Configurable thresholds and per-agent policies
• Webhook and metrics hooks for integrations and observability
• Retraining / feedback export for continual improvement
• Docker and local development modes

Installation / Configuration

Run with Docker (recommended for quick start)

# Build image locally
docker build -t vibe-check-mcp-server .

# Run with a local config mounted
docker run -p 8080:8080 -v $(pwd)/config:/app/config vibe-check-mcp-server

Run locally (Node.js example)

# Clone repo and install
git clone https://github.com/PV-Bhat/vibe-check-mcp-server.git
cd vibe-check-mcp-server
npm install

# Start server (reads ./config/default.json)
npm start

Minimal configuration (config/default.json)

{
  "server": { "port": 8080 },
  "checks": {
    "scope_creep": { "enabled": true, "threshold": 0.7 },
    "bloat": { "enabled": true, "threshold": 0.6 },
    "tunnel_vision": { "enabled": true, "threshold": 0.5 },
    "misread": { "enabled": true, "threshold": 0.8 }
  },
  "webhooks": { "on_alert": "https://hooks.example.com/agent-alert" }
}

API usage example (HTTP)

POST /vibe-check HTTP/1.1
Host: localhost:8080
Content-Type: application/json

{
  "agent_id": "agent-123",
  "context": "User asked to summarize the document about climate policy.",
  "candidate": "Agent output text here...",
  "metadata": { "model": "gpt-4o", "timestamp": "2026-04-10T12:00:00Z" }
}

Example JSON response

{
  "scores": {
    "scope_creep": 0.2,
    "bloat": 0.45,
    "tunnel_vision": 0.1,
    "misread": 0.8
  },
  "verdict": "needs_attention",
  "suggestions": [
    "Verify facts X and Y from source A.",
    "Trim irrelevant background details from paragraphs 2-4."
  ]
}

Available Tools / Resources

• /vibe-check — Main endpoint: accepts candidate texts and returns scores, verdicts, and suggestions.
• /feedback — Submit human or automated feedback to improve check models and thresholds.
• /agents — Register or update agent-specific policies (scopes, allowed tooling).
• /metrics — Prometheus-compatible metrics endpoint for monitoring check rates and alert counts.
• Webhook integration — Trigger external workflows when certain checks exceed configured thresholds.
• Export hooks — Periodic export of flagged examples for retraining or human review.

Table: Key endpoints

Use Cases

1. Prevent scope creep in task-oriented agents

   • An agent answering customer support questions can be checked to ensure it does not start offering unrelated product recommendations. If the scope_creep score exceeds threshold, the server returns a concise suggestion to refocus on the user’s original problem.

2. Reduce bloat in generated responses

   • For agents that must produce concise outputs (e.g., summaries, alerts), the bloat checker flags outputs that include excessive background or verbosity. The agent can then compress the response and re-run the check.

3. Detect tunnel vision in iterative chains

   • Agents that iteratively refine solutions may become fixated on a suboptimal hypothesis. The tunnel_vision check identifies lack of alternative considerations and suggests reevaluation prompts or external data fetches.

4. Catch misreads and hallucinations before actions

   • Before an agent executes a high-impact action (e.g., sending an email or publishing code), the misread check validates whether core facts referenced by the agent are grounded in the provided context; if not, it triggers a verification step or human review.

5. Continuous improvement loop

   • Use /feedback to collect human-corrected examples for retraining. Export flagged cases and feed them into model fine-tuning or prompt engineering workflows, shrinking false positives and improving check precision over time.

Getting Started Tips

• Start with conservative thresholds and iterate based on false positive rates.
• Register agent scopes via /agents to reduce noisy alerts.
• Wire /metrics into your monitoring stack to track alignment drift over time.
• Use feedback exports to build a dataset of real failure modes for targeted fixes.

The Vibe Check MCP Server is intended as a modular, observability-first alignment layer you can run alongside agents to make behavior safer, more focused, and continuously improving.