Summary¶

ChromaDB is a genuine embedded in-process library that requires no separate server. Its default embedding function uses ONNX Runtime (not PyTorch), keeping the total install around ~150 MB on disk — far less than the "2.2 GB with PyTorch" figure from prior research. However, ChromaDB ships a monolithic dependency set (grpcio, kubernetes, uvicorn, opentelemetry) regardless of deployment mode, which is bloated for pure embedded use. Collections cannot be queried across simultaneously — for picobot's KB topics, a single collection with a topic metadata field is the correct architecture.

(a) ChromaDB Client Modes¶

ChromaDB has three client modes, two of which are fully embedded (no server process):

Ephemeral — data lives only in process memory, lost on exit. Good for tests and one-shot scripts.

Persistent — data written to a directory on disk. This is the mode relevant to picobot. It is fully in-process: ChromaDB runs inside your Python process with no sockets, no background threads serving HTTP. The persist directory contains SQLite databases and binary HNSW index files. You can ship it bundled with your application.

Client-Server — connects over HTTP/gRPC to a separately running Chroma server. There is also a chromadb-client thin-client pip package (~22 MB) for this mode that strips out onnxruntime and embedding functions.

For picobot, PersistentClient is the right choice — embedded, no server, data at ~/.picobot/data/chroma/.

(b) Dependency Tree — Does It Pull In PyTorch?¶

The definitive answer: No, ChromaDB does NOT require PyTorch.¶

ChromaDB's pyproject.toml (from chroma-core/chroma main branch) lists these required dependencies:

onnxruntime >= 1.14.1       # ML inference runtime (ONNX, not PyTorch)
tokenizers >= 0.13.2         # Rust-based tokenizer (HuggingFace)
numpy >= 1.22.5
grpcio >= 1.58.0             # gRPC (for internal architecture)
kubernetes >= 28.1.0         # K8s client (!)
uvicorn[standard] >= 0.18.3  # ASGI server
opentelemetry-api, -sdk, -exporter-otlp-proto-grpc  # Telemetry
pydantic, httpx, orjson, rich, typer, tqdm, bcrypt, pypika, mmh3,
posthog, pybase64, PyYAML, jsonschema, tenacity, overrides,
importlib-resources, build

PyTorch is NOT listed. Neither is sentence-transformers nor torch.

The prior research claiming "ChromaDB + PyTorch = 2.2 GB" was based on using the SentenceTransformer embedding function (chromadb.utils.embedding_functions.SentenceTransformerEmbeddingFunction), which is an optional integration that requires you to pip install sentence-transformers, which in turn pulls in PyTorch. But this is user-chosen, not a ChromaDB requirement.

Default Embedding Function: ONNX-based, No PyTorch¶

ChromaDB's default embedding function is ONNXMiniLM_L6_V2. It: - Uses the all-MiniLM-L6-v2 model converted to ONNX format - Runs on onnxruntime (CPU, ~17 MB wheel) - Downloads the ONNX model (~79 MB) on first use to ~/.cache/chroma/onnx_models/all-MiniLM-L6-v2/ - Produces 384-dimensional dense vectors - Requires zero configuration — just collection.add(documents=[...]) and it embeds automatically

No API keys, no external services, no PyTorch.

Estimated Total Install Size¶

Individual package wheel sizes (macOS ARM64):

This is roughly 15x smaller than the 2.2 GB PyTorch-based setup. The estimate could be ~200 MB with all transitive dependencies fully unpacked, but the order of magnitude is correct: hundreds of MB, not gigabytes.

Bloat Concern¶

ChromaDB ships a single monolithic package for all deployment modes. Even in embedded mode you get: - kubernetes (K8s Python client) — completely unnecessary for embedded - grpcio — used for internal component communication and telemetry export - uvicorn — needed only for server mode - opentelemetry-exporter-otlp-proto-grpc — telemetry exporting

These add ~20 MB of wheels that serve no purpose in embedded mode. There is no chromadb[lite] or chromadb[embedded] extras option. This is a known community complaint but the Chroma team has not split the package.

Can You Avoid onnxruntime Entirely?¶

Yes, but you must provide your own embedding function. If you pass pre-computed embeddings directly:

collection.add(
    ids=["id1"],
    embeddings=[[0.1, 0.2, ...]],  # pre-computed vectors
    documents=["text"],
    metadatas=[{"topic": "self"}]
)

...then onnxruntime is imported but never used. However, it is still a required install dependency — you cannot pip install chromadb without it being installed. You would need to either: 1. Accept the ~17 MB onnxruntime overhead (it just sits unused), or 2. Fork/patch chromadb to make onnxruntime optional (not practical)

For picobot, accepting the 17 MB is fine — it buys us a working default embedder with zero config.

(c) Collections: Can You Have One Per KB Topic?¶

Yes, But You Shouldn't¶

ChromaDB supports multiple collections within a single persistent database:

client = chromadb.PersistentClient(path="./chroma")
self_col = client.get_or_create_collection("kb-self")
user_col = client.get_or_create_collection("kb-user")
projects_col = client.get_or_create_collection("kb-projects")
domains_col = client.get_or_create_collection("kb-domains")
journal_col = client.get_or_create_collection("kb-journal")

Each collection has its own HNSW index, its own metadata namespace, and its own embedding function (or shared default).

Cross-Collection Search: NOT Supported¶

ChromaDB has no native cross-collection query. You cannot do:

# THIS DOES NOT EXIST
client.query_all_collections(query_texts=["something"])

If you use separate collections per topic, searching across the entire KB requires:

# Manual fan-out — ugly and slow
results = []
for col_name in ["kb-self", "kb-user", "kb-projects", "kb-domains", "kb-journal"]:
    col = client.get_collection(col_name)
    r = col.query(query_texts=["search term"], n_results=5)
    results.extend(zip(r["distances"][0], r["documents"][0], r["metadatas"][0]))
# Now manually sort by distance and take top-k
results.sort(key=lambda x: x[0])
top_results = results[:10]

This has multiple problems: 1. Five separate HNSW searches instead of one 2. Manual result merging and re-ranking 3. Distance scores from different HNSW indices are not directly comparable (different index structures) 4. More complex code for every query path

Recommended: Single Collection with Metadata¶

The ChromaDB community consensus and official recommendation is to use one collection with a metadata field for categorization:

collection = client.get_or_create_collection("kb")

# Add with topic metadata
collection.add(
    ids=["self-identity-001"],
    documents=["I am a research agent focused on..."],
    metadatas=[{"topic": "self", "title": "Identity", "tags": "identity,core"}]
)

# Search across all topics
collection.query(query_texts=["research methodology"], n_results=10)

# Search within a specific topic
collection.query(
    query_texts=["research methodology"],
    n_results=10,
    where={"topic": "projects"}
)

This gives you: - Single HNSW index = one search, globally comparable distances - Topic-scoped search via where filter = equivalent to collection-per-topic - Cross-topic search for free (omit the where clause) - Simpler code, fewer moving parts

(d) Metadata Filtering Alongside Vector Search¶

ChromaDB supports rich metadata filtering combined with vector similarity in a single query. This is crucial for picobot's KB architecture.

Supported Operators¶

Combined Vector + Metadata Query¶

# Semantic search for "deployment strategy" but only in projects and domains topics
results = collection.query(
    query_texts=["deployment strategy"],
    n_results=5,
    where={"$or": [
        {"topic": "projects"},
        {"topic": "domains"}
    ]}
)

This performs vector similarity search after applying the metadata filter, returning only results from the specified topics ranked by semantic relevance.

Document Content Filtering¶

ChromaDB also supports where_document for filtering on the document text itself:

results = collection.query(
    query_texts=["python packaging"],
    where_document={"$contains": "pyproject.toml"},
    n_results=5
)

This is a substring/keyword filter on the raw document text, applied alongside the vector search.

(e) Total Install Size with Minimal Embedding Function¶

Scenario: ChromaDB with default ONNX embeddings (no PyTorch)¶

Comparison with alternatives¶

ChromaDB's ~160 MB is reasonable for a CLI tool. The prior "2.2 GB" figure was a misattribution of PyTorch's weight to ChromaDB itself.

(f) Architecture Recommendation for Picobot¶

Proposed Design: Single Collection, Metadata-Based Topics¶

import chromadb

# Initialize once, store at known location
client = chromadb.PersistentClient(path=os.path.expanduser("~/.picobot/data/chroma"))
collection = client.get_or_create_collection(
    name="knowledge-base",
    metadata={"hnsw:space": "cosine"}  # cosine similarity
)

Mapping KB Topics to Metadata¶

Each KB entry gets stored with structured metadata:

collection.add(
    ids=[entry_id],            # e.g., "self/identity" or a UUID
    documents=[entry_content], # the markdown content of the KB entry
    metadatas=[{
        "topic": "self",       # maps to KB topic dirs: self/, user/, projects/, etc.
        "title": "Identity and Goals",
        "tags": "identity,core,agent",
        "created": "2025-06-01",
        "updated": "2025-12-15",
        "path": "self/identity.md"  # original file path for reference
    }]
)

Query Patterns¶

# Global semantic search (across all topics)
collection.query(query_texts=["how to handle API rate limits"], n_results=10)

# Topic-scoped search
collection.query(
    query_texts=["deployment patterns"],
    where={"topic": "projects"},
    n_results=5
)

# Multi-topic search
collection.query(
    query_texts=["user preferences"],
    where={"topic": {"$in": ["user", "self"]}},
    n_results=5
)

# Tag-based filtering
collection.query(
    query_texts=["python"],
    where={"tags": {"$contains": "python"}},
    n_results=5
)

# Recent entries only
collection.query(
    query_texts=["anything new"],
    where={"updated": {"$gte": "2025-11-01"}},
    n_results=10
)

Why NOT Collections-Per-Topic¶

The single-collection approach is strictly better for picobot's use case.

Sync with Existing KB¶

ChromaDB would sit alongside the existing file-based KB, not replace it. The file system remains the source of truth:

1. On picobot kb sync or at session start, scan KB files for changes
2. Upsert changed entries into ChromaDB (by file path as ID)
3. Delete ChromaDB entries for removed KB files
4. ChromaDB provides fast semantic search; file system provides human-readable browsing and git sync

Key Consideration: onnxruntime Is a Hard Dependency¶

Even if picobot uses a different embedding strategy (e.g., API-based embeddings from the LLM provider), installing chromadb will pull in onnxruntime. This is unavoidable without forking. The ~17 MB overhead is acceptable.

If picobot wanted to avoid onnxruntime entirely, it would need to use sqlite-vec or another vector store that does not bundle an inference runtime. But then picobot would need to manage embeddings generation separately.

Key Takeaways¶

1. ChromaDB IS a true embedded library — PersistentClient runs entirely in-process with no server, sockets, or background processes.

2. PyTorch is NOT required — the default embedding function uses ONNX Runtime (~17 MB), not PyTorch (~2 GB). The "2.2 GB" figure from prior research was wrong.

3. Total install: ~160 MB — chromadb wheel + onnxruntime + grpcio + other deps + cached ONNX model. Reasonable for a CLI tool.

4. No cross-collection queries — use a single collection with a topic metadata field, not one collection per KB topic.

5. Rich metadata filtering — $eq, $in, $and, $or, $contains, etc., all combinable with vector similarity in a single query. This maps perfectly to KB topic/tag/date filtering.

6. Bloat exists but is tolerable — grpcio, kubernetes, uvicorn are dragged in unnecessarily for embedded mode (~20 MB wasted). No [lite] install option exists.

7. For picobot: single PersistentClient with one knowledge-base collection, topic/title/tags/dates as metadata, cosine similarity, default ONNX embeddings. File-based KB remains source of truth; ChromaDB is a search index synced from it.