Sourcerer MCP Server: Token-Efficient Code Search & Navigation

Overview

Sourcerer MCP Server implements the Model Context Protocol (MCP) for codebases, exposing compact, addressable views of source files and symbols to make semantic code search and navigation token-efficient. Instead of shipping whole files or repositories to a language model, the server lets you request precise slices, symbol metadata, and cross-reference information so you only include the minimal context an LLM needs to answer a query.

This approach reduces token waste, speeds up downstream model calls, and enables accurate jump-to-definition, reference listing, and contextual code search workflows inside IDEs, bots, or backend services. Sourcerer is intended for teams who want to combine local repo intelligence with LLM-based analysis without repeatedly transmitting large amounts of code.

GitHub: https://github.com/st3v3nmw/sourcerer-mcp

Features

• Exposes fine-grained file slices (byte or token ranges) for minimal context delivery
• Symbol indexing and metadata (definitions, locations, type hints)
• Cross-reference queries (find-references, jump-to-definition)
• Fast semantic search endpoints to find relevant files or snippets
• HTTP API that follows MCP patterns for easy integration with LLM pipelines and editors
• Optional repository indexing for low-latency lookups
• Lightweight, deployable as a single binary or Docker container

Installation / Configuration

Clone the repository and run locally, or use the provided Docker image.

Clone:

git clone https://github.com/st3v3nmw/sourcerer-mcp.git
cd sourcerer-mcp

Run locally (example using Go toolchain if the project is Go-based):

# from repo root
go build ./cmd/sourcerer-mcp
./sourcerer-mcp --repo-root /path/to/repo --port 8080

Docker:

# build
docker build -t sourcerer-mcp .

# run
docker run -d \
  -p 8080:8080 \
  -v /path/to/source:/data/repo:ro \
  -e REPO_ROOT=/data/repo \
  --name sourcerer-mcp \
  sourcerer-mcp

Configuration (example environment variables / flags)

# .env example
PORT=8080
REPO_ROOT=/data/repo
INDEX_PATH=/data/index
MAX_SLICE_TOKENS=512
CACHE_TTL=300

Adjust index paths, maximum slice size (to control token lengths sent to models), and cache settings to match your usage pattern.

Available Resources

The server exposes HTTP endpoints to fetch minimal context and metadata. Below is a concise overview; adapt to actual endpoint names in your deployment.

Example: retrieve a file slice (token- or byte-range)

curl "http://localhost:8080/mcp/file?path=src/main.py&start=120&end=420"

Example: look up a symbol and its definition

curl "http://localhost:8080/mcp/symbol?name=MyClass&path=src"

Example: semantic search (POST with query and optional filters)

curl -X POST http://localhost:8080/mcp/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "parse HTTP headers",
    "max_results": 5,
    "filters": { "lang": "python", "path_prefix": "src/"}
  }'

Responses are JSON and include file URIs, ranges, and optionally minimal context snippets ready to be concatenated into model prompts.

Use Cases

• Token-efficient code completion: fetch only the function body and signature rather than the whole file, and provide that slice to the LLM for completion or explanation.
  • Example: for autocompletion in an editor, request GET /mcp/file with start/end byte offsets of the current function, then send that to the model.
• Jump-to-definition without full repo transmission: query /mcp/symbol to get exact file and range for a definition, then request just that slice for inspection.
  • Example: a chat assistant sees a symbol name in user input, calls /mcp/symbol to find its location and returns the small defining snippet with an explanation.
• Semantic search for relevant code snippets: use /mcp/search to identify top-matching functions or classes, then retrieve minimal context for summarization or refactoring suggestions.
  • Example: find “database connection” implementations across services, present each function signature and 50–200 token context to the model for clustering.
• Large-repo question answering: break long answers into focused retrieval steps—use manifest listing to constrain to a submodule, search for candidates, retrieve slices, and pass only those slices into the LLM prompt chain.

Tips for Integration

• Set MAX_SLICE_TOKENS conservatively (e.g., 256–1024) to ensure model prompts remain within your token budget.
• Combine semantic search scores with static heuristics (file path, recent commits) to prioritize slices you feed to the model.
• Cache frequently requested slices and symbol lookups at the application layer to minimize repeated IO and indexing overhead.

Resources

• Repository: https://github.com/st3v3nmw/sourcerer-mcp
• Recommended next steps: run the server against a small test repository, explore the manifest and search endpoints, and adapt slice sizes based on your target model’s token limits.

This server is intended as a practical building block to reduce token waste and accelerate accurate code discovery when integrating LLMs with real-world codebases.