Vector Databases
A vector database stores high-dimensional embeddings alongside metadata, and provides fast approximate nearest-neighbor (ANN) search across millions of vectors. The core operation: given a query vector, return the k vectors most similar to it.
What Makes Them Different
A traditional database indexes discrete values and returns exact matches. A vector database indexes points in continuous high-dimensional space and returns approximate nearest neighbors by geometric distance. Approximation is a deliberate tradeoff — exact nearest-neighbor search at scale is too slow, and the small accuracy loss is acceptable for most retrieval tasks.
Index Types
HNSW (Hierarchical Navigable Small World) — a graph-based index. Each vector is a node; edges connect nearby nodes across multiple layers. Fast queries (single-digit milliseconds at 10M scale), good recall, but high memory usage. Default choice for most production systems.
IVF (Inverted File Index) — partitions the vector space into clusters (Voronoi cells), then searches only the nearest clusters. Much lower memory footprint than HNSW. Recall drops if the cluster count is poorly tuned. Good fit for cost-sensitive deployments with moderate recall requirements.
Flat index — brute-force exhaustive search. Exact recall, but O(n) per query. Only viable under ~100k vectors. Use it for prototyping or when the dataset fits in memory and query latency isn't critical.
Most production systems use HNSW or a hybrid (IVF with HNSW on top of each cluster).
Key Players
| System | Notes |
|---|---|
| Pinecone | Managed, serverless option. Fast setup, no infra. Costs add up at scale. |
| Weaviate | Open-source, self-hostable. Multi-modal, built-in BM25 hybrid search. |
| pgvector | Postgres extension. Good enough for millions of vectors; avoids a separate service. |
| Chroma | Embedded or client-server. Lightweight, common in Python prototyping. |
| Qdrant | Open-source, Rust-based. High performance, good filtering, payload-rich metadata. |
When to Use vs. When Not To
Use a vector database when:
- You have more than ~100k vectors and need sub-second query latency
- You need filtered ANN search (similarity + metadata predicates)
- You're running a production rag system with real traffic
Skip the dedicated vector database when:
- You have fewer than 100k vectors: numpy + cosine similarity in memory is fast enough and removes an infrastructure dependency
- You already run Postgres: pgvector gets you most of the way there without a new service
# Simple cosine search without a vector DB
import numpy as np
def cosine_search(query_vec, corpus_vecs, k=5):
query_norm = query_vec / np.linalg.norm(query_vec)
corpus_norm = corpus_vecs / np.linalg.norm(corpus_vecs, axis=1, keepdims=True)
scores = corpus_norm @ query_norm
top_k = np.argsort(scores)[::-1][:k]
return top_k, scores[top_k]
Hybrid Search
Most production retrieval systems combine vector search with keyword search (BM25). Pure vector search misses exact-match queries; pure keyword search misses semantic variations. Hybrid search with reciprocal rank fusion outperforms either alone for general-purpose RAG.