Overview

The Operative WebEvalAgent MCP Server implements a Model Context Protocol (MCP)–compatible backend tailored for automated testing, debugging, and investigation of web applications. It lets language models and autonomous agents interact with web pages and testing tools in a structured way: running browser actions, collecting screenshots and network traces, executing small JS snippets, and returning rich context that an LLM can use to plan next steps.

This server is useful when you want an agent to autonomously reproduce reported bugs, triage UI regressions, or iteratively explore application behavior without manually writing brittle end-to-end scripts. By exposing a consistent MCP interface, the WebEvalAgent integrates with LLM-driven workflows, CI pipelines, and human-in-the-loop debugging processes.

Features

Implements an MCP-compatible API for agent-driven sessions and tool invocation
Headless browser automation for navigating and interacting with web apps
JavaScript evaluation in page context for quick probing and DOM inspection
Screenshot and DOM dump capture for visual and structural debugging
Network capture/tracing to inspect requests/responses
Pluggable tool architecture so you can add custom analyzers or reporters
Session-level logging and artifact collection for reproducible runs
Designed to be run locally, in CI, or as a containerized service

Installation / Configuration

Prerequisites: Docker (recommended) or Node.js (if running locally). Clone the repository and run the server using Docker or directly.

Clone the repo:

git clone https://github.com/Operative-Sh/web-eval-agent.git
cd web-eval-agent

Run with Docker:

# Build
docker build -t web-eval-agent .

# Start container (example binds server port 3000)
docker run -p 3000:3000 \
  -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  -e MCP_PORT=3000 \
  web-eval-agent

Run locally (Node.js example):

# Install dependencies
npm install

# Set env vars and start
export OPENAI_API_KEY="your_api_key_here"
export MCP_PORT=3000
npm start

Configuration (example env variables)

OPENAI_API_KEY=your-openai-key        # for LLM integration (optional)
MCP_PORT=3000                         # port the MCP server listens on
BROWSER_HEADLESS=true                 # run browser in headless mode
ARTIFACT_DIR=./artifacts              # where to save screenshots/logs
LOG_LEVEL=info

Refer to the repository’s config file for more options and advanced tuning.

Available Tools / Resources

The server exposes built-in tools that agents can call via MCP sessions. Typical default tools include:

Browser: navigate, click, type, waitForSelector, form submission
JS Runner: evaluate arbitrary JavaScript inside the page context and return values
Screenshot: capture a PNG of the current viewport or full page
DOM Dump: return a structured snapshot of the DOM (HTML or serialized tree)
Network Trace: capture requests/responses, timing, and headers
Console Logs: capture browser console output and errors
Artifact Store: collect artifacts (screenshots, traces) for later inspection

Tool outputs are returned as structured JSON with metadata and links to artifact files when applicable.

API / Session Examples

Start a session (example):

POST /sessions
Content-Type: application/json

{
  "goal": "Reproduce the 500 error when submitting the login form",
  "model": "gpt-4o-mini",
  "max_steps": 10
}

Invoke a tool inside a session (example):

POST /sessions/{session_id}/invoke
Content-Type: application/json

{
  "tool": "browser.navigate",
  "args": {
    "url": "https://staging.example.com/login"
  }
}

Fetch artifacts (example):

GET /sessions/{session_id}/artifacts

(Endpoint paths and payloads may vary—consult the repo README or OpenAPI spec included in the project for exact contract details.)

Use Cases

Automated triage of user-reported bugs: start a session with a user description, let the agent reproduce the issue, collect screenshots, network traces, and a minimal reproduction that a developer can use.
Regression testing in CI: run the MCP server in a container step that executes agent-driven checks to validate critical user flows before deployment.
Exploratory testing during development: ask an agent to probe edge cases (form inputs, session timeouts, localization) and produce a checklist of failing or suspicious behaviors.
Root-cause diagnosis: have the agent collect console logs, network requests, and evaluate DOM state to suggest likely sources of a UI bug and propose a tentative fix or test reproduction steps.

Tips for Developers

Start with small, deterministic goals for the agent and increase complexity iteratively.
Persist artifacts to a shared location (S3, artifact store) for CI visibility.
Add custom tools for domain-specific checks (accessibility validators, performance markers).
Use retryable steps and timeouts to reduce flakiness when running headless browsers in CI.

For detailed API documentation, examples, and advanced configuration, see the project’s README and OpenAPI/Swagger spec in the repository.

Operative WebEvalAgent MCP Server for Web App Testing