Overview

MCP Server is a lightweight server that converts source code repositories into a searchable knowledge base for semantic code search. It extracts, normalizes, and chunks code and associated text (comments, README, tests), embeds those chunks using OpenAI embeddings, and stores them in a vector index. A Model Context Protocol (MCP) compatible server then provides a standard API for retrieval and for supplying contextual snippets to an LLM such as GPT-5 to create precise, context-aware answers about the codebase.

This tool is useful when you want fast, semantic search across large projects (including monorepos), to power code search UIs, developer onboarding bots, automated audit assistants, or contextual LLM prompts that need relevant code snippets. It emphasizes intelligent chunking (token-aware, overlap support, language-aware heuristics) and configurable storage options for vectors so you can adapt it to your infra and scale.

Features

Convert source repositories into an indexed, semantic knowledge base
Token-aware, language-sensitive chunking with configurable chunk size and overlap
Embeddings via OpenAI (configurable to other providers)
Pluggable vector store backends (local FAISS/SQLite, remote vector DBs like Pinecone/Weaviate)
MCP (Model Context Protocol) compatible API for retrieval and context delivery to LLMs
CLI and HTTP server interfaces for indexing, status and search
Support for metadata (file path, language, repo, line ranges) to improve relevance and traceability
Health and monitoring endpoints for production readiness

Installation / Configuration

Prerequisites:

Node.js (>=18) or relevant runtime as documented in the repo
An OpenAI API key (or other embeddings provider key)
Optional: vector DB credentials if using a managed store

Clone and install:

git clone https://github.com/vezlo/src-to-kb.git
cd src-to-kb
# If Node.js project
npm install

Example .env (basic):

OPENAI_API_KEY=sk-...
EMBEDDINGS_MODEL=text-embedding-3-large
MCP_PORT=8080
VECTOR_STORE=sqlite   # options: sqlite, faiss, pinecone, weaviate
VECTOR_STORE_PATH=./data/vectors.db
CHUNK_TOKENS=800
CHUNK_OVERLAP=50

Run the server:

# development
npm run dev

# production
npm run build
npm start

Index a repository (CLI):

# index a local folder
node cli/index.js --repo ./my-repo --namespace my-project

# or via HTTP
curl -X POST http://localhost:8080/index \
  -H "Content-Type: application/json" \
  -d '{"path":"./my-repo","namespace":"my-project"}'

Configuration file (JSON) example:

{
  "chunkTokens": 800,
  "chunkOverlap": 50,
  "ignore": ["node_modules", ".git"],
  "vectorStore": {
    "type": "sqlite",
    "path": "./data/vectors.db"
  },
  "embeddings": {
    "provider": "openai",
    "model": "text-embedding-3-large"
  }
}

Available Resources

CLI: index, reindex, purge, status
REST API endpoints:
- GET /health — basic health check
- POST /index — start indexing a repo or folder
- POST /query — semantic search returning relevant chunks
- GET /status — indexing status and metrics
Vector store adapters: sqlite, faiss, pinecone, weaviate (configurable)
Logging and metrics (stdout, structured logs, optional Prometheus hooks)
Example client snippets for curl, Python and Node

Example API: semantic query request

curl -X POST http://localhost:8080/query \
  -H "Content-Type: application/json" \
  -d '{"namespace":"my-project","query":"How does authentication work?","topK":8}'

Response contains ranked chunks with metadata (path, start/end line, language) and relevance score.

Use Cases

Developer onboarding: search for “how do I run tests” and return code snippets, test commands and related docs from the repo to accelerate ramp-up.
Semantic code search: find examples of a pattern (e.g., “database transaction retry”) across services even if naming differs.
LLM-powered assistants: supply the most relevant code chunks as context to GPT-5 via MCP for precise code explanations, refactor suggestions, or automated PR descriptions.
Security review and auditing: query for usage of sensitive APIs (e.g., “eval”, “exec”, crypto misuse) and surface exact file ranges and call sites.
Documentation generation: collect code + doc snippets to automatically create or enrich API docs and READMEs.

Tips & Best Practices

Tune chunk size and overlap to balance context completeness and embedding costs; larger chunks reduce cross-chunk context but increase token usage.
Include repository metadata (service, language) to improve filtering and reduce false positives in search.
Use a persistent, scalable vector store (e.g., Pinecone or Weaviate) for large codebases or multi-repo installations.
Periodically reindex or use a change-watcher to keep embeddings in sync with the repo.

For more details and contributor guidelines, see the project on GitHub: https://github.com/vezlo/src-to-kb.

MCP Server: Semantic Code Search with GPT-5