Reexpress MCP Server: Statistical Verification for LLMs

Overview

Reexpress MCP Server provides a lightweight, local service that adds statistically principled verification to tool-calling LLM workflows. It implements the Similarity–Distance–Magnitude (SDM) verification approach to estimate predictive uncertainty for LLM outputs using a calibrated estimator that ensembles cloud models (gpt-5.2-2025-12-11, gemini-3-pro-preview) with a local generative model (granite-3.3-8b-instruct). The server is intended for developers and data scientists who want an automated, auditable second opinion on search results, QA responses, and data-science assertions produced by LLMs.

The server runs on macOS (Sequoia 15+ on Apple silicon) or Linux. Data sent to cloud LLMs uses standard API calls (Azure/OpenAI and Google) while the SDM estimator inference and verification logic run locally. The design emphasizes conservative file access and easy adaptation: you can append the Reexpress prompt to the end of your chat text or invoke the Reexpress tool from an LLM agent to get a calibrated verification score, and you can dynamically update the estimator via add-true / add-false operations to adapt to task-specific behavior.

Features

• SDM-based statistical verification that outputs calibrated verification probabilities and introspective diagnostics
• Ensemble estimator combining cloud models and a local granite-3.3-8b-instruct model
• Dynamic, incremental updates via ReexpressAddTrue and ReexpressAddFalse tools
• Conservative file-access controls: ReexpressDirectorySet and ReexpressFileSet
• Training and calibration scripts included for full retraining or domain adaptation
• Static HTML output generation for human-readable verification reports
• Cross-platform: macOS (Apple silicon) and Linux support

System requirements

Installation / Configuration

Below are typical steps to get the server running. See the repository for full details and platform-specific instructions.

Clone the repo and create a Python environment:

git clone https://github.com/ReexpressAI/reexpress_mcp_server.git
cd reexpress_mcp_server
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Set required environment variables (example):

export OPENAI_API_KEY="sk-..."
export GOOGLE_API_KEY="..."
# Optional: point to any local model cache or custom config
export GRANITE_MODEL_CACHE="/path/to/granite/cache"

Start the MCP server (example command — check repo for exact launch script):

# Example (replace with the project's provided run script if present)
python -m reexpress_mcp_server.server --config config.yaml

Configuration options include API keys, which models to include in the ensemble, file-access whitelists, and ports. The repository contains a CONFIG.md with full configuration fields.

Available Tools / Resources

• Reexpress (main verification tool): returns a verification probability, explanation, and diagnostic metadata for a model response.
• ReexpressAddTrue / ReexpressAddFalse: mark a verified example as true or false to update the estimator incrementally.
• ReexpressDirectorySet / ReexpressFileSet: explicit file-access tools to control which additional files the server may forward to cloud APIs.
• Training and calibration scripts: retrain the SDM estimator or recalibrate on custom datasets.
• Static HTML output generator: create human-readable reports of reexpression runs.
• Documentation pages: HOW_TO_USE.md, OUTPUT_HTML.md, DATA.md, EVAL.md, and a system demonstration paper included under documentation/.

Example usage patterns

1. Verify a direct QA response

• Append the Reexpress call at the end of your tool-calling LLM’s response step. The LLM calls the Reexpress tool and receives:
  • verification_probability: 0.92
  • explanation: short rationale
  • nearest_examples: calibration examples used

2. Use verification to drive agent behavior

• An LLM agent receives low verification probability (< 0.5). It can then:
  • call an external tool or web search
  • ask the user for clarification
  • attempt a safer, constrained answer

3. Incremental domain adaption

• If you see a false positive or false negative, call:

curl -X POST http://localhost:PORT/tools/ReexpressAddTrue \
  -H "Content-Type: application/json" \
  -d '{"example_id":"example-123", "response":"...", "label": true}'

• Future verifications take this new evidence into account dynamically.

Use cases

• Search & QA: provide calibrated confidence for passages or answers derived from retrieved documents.
• Data science validation: verify model claims about datasets, statistics, or code outputs before committing results to analysis or dashboards.
• Agent orchestration: let agents decide whether to consult external tools, retry, or escalate to a human based on verification probability.
• Model evaluation: use included evaluation scripts and the OpenVerification1 dataset to benchmark verification performance.

Links & support

Repository: https://github.com/ReexpressAI/reexpress_mcp_server

For platform-specific installation notes, training recipes, and reproducibility scripts, see the documentation directory in the repo. The project includes a changelog and an FAQ to