Chroma MCP Server: Embeddings, Vector and Full-Text Search

Overview

Chroma MCP Server is a Model Context Protocol (MCP) compatible server that exposes Chroma’s embedding-powered document store for use by LLMs and other tools. It provides vector search, full-text search, metadata filtering, and persistent document storage so applications can build memory, knowledge bases, or semantic search over arbitrary content.

The server is intended for self-hosting: it can run ephemeral in-memory clients for fast testing, persistent file-backed clients for production storage, HTTP clients that proxy to a Chroma instance, or cloud clients that connect to Chroma Cloud. It standardizes access with MCP so LLMs and client apps can request context using a stable set of tools and commands.

Features

• Vector search (semantic search) using stored embeddings
• Full-text search and advanced metadata filtering
• Collection management: create, rename, configure, delete
• Document operations: add, update, fetch, delete, and batch operations
• HNSW index tuning for fast nearest-neighbor queries
• Multiple embedding backends (see Embedding functions)
• Multiple client types: ephemeral, persistent, HTTP and cloud
• Compatible with MCP tooling and integrations like Claude Desktop

Installation / Configuration

You can run the MCP server from a prebuilt binary (if available), or build and run it from the repository. Typical invocation uses command-line flags to select client type and storage.

Run an ephemeral (in-memory) server for development:

# Start ephemeral MCP server
chroma-mcp --client-type ephemeral

Run a persistent (file-backed) server and set the storage directory:

# Start persistent MCP server
chroma-mcp --client-type persistent --data-dir /path/to/chroma-data

Connect to Chroma Cloud with tenant/database credentials:

# Start cloud client (connects to api.trychroma.com)
chroma-mcp --client-type cloud \
  --tenant your-tenant-id \
  --database your-database-name \
  --api-key your-api-key

If you prefer environment configuration, use a dotenv file and pass its path:

chroma-mcp --dotenv-path /custom/path/.env --client-type persistent --data-dir /path/to/data

Configuration flags referenced above:

• –client-type: ephemeral | persistent | http | cloud
• –data-dir: filesystem path (for persistent)
• –tenant, –database, –api-key: cloud client parameters
• –dotenv-path: custom .env location for API keys and secrets

Security note: avoid placing long-lived API keys on shared CLI histories. Prefer .env files restricted to appropriate file-system permissions.

Embedding Functions

Chroma MCP supports multiple embedding backends that generate vectors used for semantic search. Supported backends include:

• default (Chroma’s built-in)
• openai
• cohere
• jina
• voyageai
• roboflow

When you create a collection you can select an embedding function; that choice is persisted in the collection configuration (feature introduced in v1.0.0). Collections created with older versions (<= 0.6.3) may not have an embedding function persisted.

Common environment variables for external embedding providers (set in your shell or .env):

OPENAI_API_KEY=
COHERE_API_KEY=
JINA_API_KEY=
VOYAGEAI_API_KEY=
ROBOFLOW_API_KEY=

(Exact names may vary by provider; consult your hosting or provider docs if an integration requires a different key name.)

Available Tools (MCP tools)

The server exposes a set of MCP tools that map to common collection and document operations:

• chroma_list_collections — list collections with pagination
• chroma_create_collection — create collection and HNSW config
• chroma_peek_collection — preview documents in a collection
• chroma_get_collection_info — collection metadata and stats
• chroma_get_collection_count — document count
• chroma_modify_collection — rename or update collection metadata
• chroma_delete_collection — delete collection and contents
• chroma_add_documents — insert documents with optional metadata/IDs
• chroma_query_documents — semantic query with filters
• chroma_get_documents — fetch documents by ID or filter
• chroma_update_documents — update content, metadata, or embeddings
• chroma_delete_documents — remove documents by ID or filter

These tools are exposed via MCP so an LLM runtime or orchestrator can call them programmatically.

Client Types Comparison

Use Cases

• LLM app memory: store user conversation chunks and retrieve relevant context with semantic search to provide consistent, stateful responses.
• Knowledge base and support: index product docs, release notes, and tickets to answer support queries via vector+full-text search.
• Code search: embed code snippets and perform similarity search for code reuse or vulnerability detection.
• Desktop integrations: wire the MCP server into tools like Claude Desktop to provide local or cloud-backed document context for local LLMs.
• Augmented retrieval: combine metadata filtering (date, author, tags) with vector ranking to narrow search scope before nearest-neighbor scoring.

Example: adding documents and querying

# pseudocode illustrating typical flow (client SDK or MCP call)
create_collection(name="kb", embedding="openai")
add_documents(collection="kb", docs=[{"id":"d1","text":"Install steps...","meta":{"version":"1.2"}}])
query_documents(collection="kb", query="how to install", top_k=5, filter={"version":"1.2"})

Getting Help and Next Steps

• Read the MCP specification at modelcontextprotocol.io for integration patterns.
• Check provider docs for embedding API keys and rate limits.
• For fleet or production deployment, use the persistent client with backups and monitor HNSW and index sizing.

This server bridges Chroma’s fast embedding store with MCP-compatible LLM tooling—use it to add reliable, scalable semantic retrieval and full-text capabilities to your AI applications.