Monitoring ChromaDB with Vigilmon

ChromaDB has become the default vector database for AI applications — it's where your RAG pipeline stores and retrieves embeddings, where your semantic search index lives, and where your recommendation system looks up nearest neighbors. When you run ChromaDB as a standalone server in production, its health directly determines whether your AI application can answer questions, find relevant documents, or serve recommendations. A ChromaDB crash means every downstream AI feature goes blind. Vigilmon gives you continuous visibility into ChromaDB's server health, query performance, memory footprint, and data persistence so your AI stack stays reliable.

What You'll Set Up

• ChromaDB API server health monitor
• Query latency tracking via a synthetic probe endpoint
• HNSW index memory usage alert
• Persistence directory disk usage monitor
• Embedding ingestion rate heartbeat
• API error rate monitoring
• Collection integrity spot-check
• Alert channels with appropriate thresholds

Prerequisites

• ChromaDB running as a server (chroma run or Docker container)
• ChromaDB API accessible over HTTP (default port 8000)
• A free Vigilmon account

Step 1: Monitor ChromaDB Server Health

ChromaDB exposes a heartbeat endpoint that confirms the server process is alive and the API is responding.

1. Log in to vigilmon.online and click Add Monitor.
2. Set Type to HTTP / HTTPS.
3. URL: http://your-server-ip:8000/api/v1/heartbeat (or https://chromadb.yourdomain.com/api/v1/heartbeat if reverse-proxied).
4. Set Check interval to 1 minute.
5. Set Expected HTTP status to 200.
6. Under Keyword check, enter nanosecond_heartbeat — ChromaDB's heartbeat response includes this key confirming the server is processing requests.
7. Click Save.

If you're on ChromaDB 0.4+ which changed the API path, use:

http://your-server-ip:8000/api/v1

The root API endpoint returns version information and confirms the server is functional.

Step 2: Monitor the Version/Info Endpoint

ChromaDB's version endpoint provides a secondary health signal and confirms the expected version is running (useful for detecting unexpected downgrades after container restarts):

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:8000/api/v1/version
3. Check interval: 5 minutes
4. Expected HTTP status: 200
5. Under Keyword check, enter the ChromaDB version you're running (e.g., 0.5) to alert if a container restart pulled a different image.
6. Click Save.

Step 3: Monitor Query Latency with a Synthetic Probe

Vector similarity search latency degrades when the HNSW index grows large or when memory pressure forces index segments to be paged to disk. Set up a synthetic probe that performs a real query against a known collection and alerts when latency exceeds your threshold.

Create a small health-check service:

#!/usr/bin/env python3
# chromadb_probe.py — deploy alongside ChromaDB as a health sidecar
from flask import Flask, jsonify
import chromadb
import time

app = Flask(__name__)
chroma_client = chromadb.HttpClient(host="localhost", port=8000)

# Pre-populate a probe collection with known test vectors
PROBE_COLLECTION = "_vigilmon_probe"

def ensure_probe_collection():
    try:
        col = chroma_client.get_or_create_collection(PROBE_COLLECTION)
        # Add probe documents if empty
        if col.count() == 0:
            col.add(
                documents=["probe document for latency testing"],
                ids=["probe-1"]
            )
    except Exception:
        pass

@app.route('/health')
def health():
    try:
        ensure_probe_collection()
        col = chroma_client.get_collection(PROBE_COLLECTION)
        start = time.time()
        results = col.query(query_texts=["probe query"], n_results=1)
        latency_ms = (time.time() - start) * 1000
        
        if latency_ms > 500:  # p99 threshold
            return jsonify({"status": "slow", "latency_ms": latency_ms}), 503
        if not results["ids"][0]:
            return jsonify({"status": "no_results"}), 503
        return jsonify({"status": "ok", "latency_ms": round(latency_ms, 2)}), 200
    except Exception as e:
        return jsonify({"status": "error", "error": str(e)}), 503

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8001)

Deploy with:

pip install flask chromadb
python chromadb_probe.py

Then add a Vigilmon monitor:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:8001/health
3. Check interval: 2 minutes
4. Expected HTTP status: 200
5. Under Keyword check, enter "ok" to confirm latency is within threshold.
6. Click Save.

Step 4: Monitor HNSW Index Memory Usage

ChromaDB's HNSW index is held in RAM. When memory usage exceeds available RAM, the OS begins swapping the index to disk, causing severe query latency degradation. Monitor process memory to catch this before it impacts users.

Extend the probe service with a memory check:

import psutil
import os

MEMORY_WARN_PERCENT = 80

@app.route('/memory')
def memory():
    total_ram = psutil.virtual_memory().total
    used_ram = psutil.virtual_memory().used
    percent = (used_ram / total_ram) * 100
    
    # Also check ChromaDB process specifically
    chroma_rss_mb = 0
    for proc in psutil.process_iter(['name', 'memory_info']):
        if 'chroma' in proc.info['name'].lower():
            chroma_rss_mb = proc.info['memory_info'].rss / (1024 * 1024)
            break
    
    status = "ok"
    if percent > MEMORY_WARN_PERCENT:
        status = "high"
    
    if status != "ok":
        return jsonify({"status": status, "system_percent": round(percent, 1), 
                        "chroma_rss_mb": round(chroma_rss_mb, 1)}), 503
    return jsonify({"status": status, "system_percent": round(percent, 1),
                    "chroma_rss_mb": round(chroma_rss_mb, 1)}), 200

Add a monitor:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:8001/memory
3. Check interval: 3 minutes
4. Expected HTTP status: 200

Step 5: Monitor Persistence Directory Disk Usage

ChromaDB persists its SQLite database and HNSW index files to a local directory. As you ingest embeddings, this directory grows. Running out of disk space causes ChromaDB to fail writes, which can corrupt in-progress index updates.

Add a disk check to the probe service:

DISK_WARN_PERCENT = 80
CHROMA_DATA_DIR = "/chroma/chroma"  # adjust to your --path argument

@app.route('/disk')
def disk():
    usage = psutil.disk_usage(CHROMA_DATA_DIR)
    percent = usage.percent
    free_gb = usage.free / (1024 ** 3)
    
    if percent > DISK_WARN_PERCENT:
        return jsonify({"status": "high", "percent": percent, "free_gb": round(free_gb, 2)}), 503
    return jsonify({"status": "ok", "percent": percent, "free_gb": round(free_gb, 2)}), 200

Add a Vigilmon monitor:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:8001/disk
3. Check interval: 10 minutes
4. Expected HTTP status: 200

Step 6: Track Embedding Ingestion Rate via Heartbeat

If your application is continuously ingesting embeddings (indexing new documents, processing a queue), a heartbeat confirms the ingestion pipeline is active. A gap in heartbeats tells you ingestion has stalled before your users notice stale search results.

1. Click Add Monitor → Cron Heartbeat.
2. Set Expected interval to match your ingestion frequency (e.g., 10 minutes for a near-real-time pipeline).
3. Copy the heartbeat URL: https://vigilmon.online/heartbeat/abc123.
4. Ping the heartbeat from your ingestion code after each successful batch:

import requests
import chromadb

HEARTBEAT_URL = "https://vigilmon.online/heartbeat/abc123"

def ingest_batch(documents, embeddings, ids):
    client = chromadb.HttpClient(host="localhost", port=8000)
    collection = client.get_collection("my_collection")
    
    collection.add(
        documents=documents,
        embeddings=embeddings,
        ids=ids
    )
    
    # Ping heartbeat after successful ingestion
    try:
        requests.get(HEARTBEAT_URL, timeout=5)
    except Exception:
        pass  # Don't fail ingestion if heartbeat ping fails

If the heartbeat doesn't arrive within the expected window, Vigilmon alerts you that embedding ingestion has stopped.

Step 7: Monitor Collection Count and Integrity

Unexpected collection deletion (a misconfigured cleanup job, a bug in your application) can silently break features that depend on those collections. Monitor collection count to detect unexpected deletions.

Add a collection check to the probe service:

MIN_EXPECTED_COLLECTIONS = 1  # set to your expected minimum

@app.route('/collections')
def collections():
    try:
        cols = chroma_client.list_collections()
        count = len(cols)
        if count < MIN_EXPECTED_COLLECTIONS:
            return jsonify({"status": "low", "count": count}), 503
        return jsonify({"status": "ok", "count": count}), 200
    except Exception as e:
        return jsonify({"status": "error", "error": str(e)}), 503

Add a monitor:

1. Click Add Monitor → HTTP / HTTPS.
2. URL: http://your-server-ip:8001/collections
3. Check interval: 5 minutes
4. Expected HTTP status: 200

Step 8: Configure Alerting

Go to Alert Channels in Vigilmon and configure your notification channels:

• Email: your AI/ML team or on-call address
• Slack: #ai-infrastructure or #chromadb-alerts via webhook
• PagerDuty: for production RAG applications where ChromaDB outage means the product is broken

Recommended thresholds:

| Monitor | Recommended threshold | |---|---| | ChromaDB server health | Alert immediately on first failure | | Query latency probe | Alert immediately on slow/error response | | Memory usage | Alert at >80% system memory | | Disk usage | Alert at >80% persistence directory | | Ingestion heartbeat | Alert after 2× expected interval with no ping | | Collection count | Alert immediately on count drop below minimum |

Step 9: Monitor ChromaDB TCP Port for Low-Level Connectivity

Sometimes the HTTP API may be blocked by a firewall rule or misconfigured proxy even when the ChromaDB process is running. Add a TCP-level monitor as an independent check:

1. Click Add Monitor → TCP Port.
2. Host: your-server-ip
3. Port: 8000
4. Check interval: 1 minute
5. Click Save.

If the HTTP health monitor fails but TCP is up, the issue is in the application layer. If both fail, the process has crashed or the port is blocked.

Conclusion

A complete Vigilmon setup for ChromaDB covers every failure mode your AI application depends on:

• Server health — catches process crashes and API unavailability
• Query latency — catches HNSW index performance degradation before users notice slow responses
• Memory monitoring — catches index swapping before it causes severe latency
• Disk usage — prevents write failures from full persistence directories
• Ingestion heartbeat — detects pipeline stalls before search results go stale
• Collection integrity — catches accidental deletions that silently break AI features

With these monitors running, your RAG pipeline and semantic search features have the reliability foundation they need — and you catch infrastructure problems before they become AI application bugs.

Get started with a free Vigilmon account.