Embeddings
Embeddings are fixed-length numerical vectors that encode semantic meaning. Two pieces of text that mean similar things produce vectors that are close together in high-dimensional space. This geometric property is what makes embeddings useful — you can measure meaning with arithmetic.
How They're Generated
An embedding model takes text as input and returns a vector (an array of floats). Dimension count varies by model: OpenAI's text-embedding-3-large outputs 3072 dimensions; many open-source models use 384, 768, or 1536. Larger dimensions generally capture more nuance but cost more to store and search.
import anthropic
client = anthropic.Anthropic()
# Anthropic does not currently offer a native embedding endpoint.
# Common choices: OpenAI, Cohere, or open-source via sentence-transformers.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
vector = model.encode("The transformer architecture changed everything.")
# vector.shape → (384,)
The same model must be used for all embeddings in a system. Mixing models produces nonsensical similarity scores.
Similarity Metrics
Cosine similarity — the angle between two vectors, ignoring magnitude. Most common for text search. Returns -1 to 1; values above 0.85 typically indicate high semantic overlap.
Dot product — cosine similarity times magnitude. Faster to compute, used in systems where vectors are normalized at index time (which makes dot product equal to cosine similarity).
Euclidean distance (L2) — actual geometric distance. Less common for text; sensitive to magnitude differences.
For RAG and semantic search, cosine similarity or normalized dot product is the default choice.
Core Use Cases
Semantic search — retrieve documents by meaning, not keyword match. A query for "engine failure" finds documents about "motor malfunction" that a BM25 index would miss.
Clustering — group documents by topic without predefined labels. Run k-means or HDBSCAN on embeddings to discover structure in a corpus.
RAG retrieval — the retrieval step in rag produces a query embedding, searches a vector database for nearest neighbors, and returns the top-k chunks as context for the generation step.
Deduplication — flag near-duplicate content by thresholding cosine similarity. Useful for cleaning training datasets or de-duping user submissions.
Anomaly detection — vectors far from the cluster centroid are outliers. Works for identifying off-topic content or unusual user inputs.
Chunking Strategy Matters
Embedding quality depends heavily on what you embed. Long documents compress poorly into a single vector; chunk them into 200–500 token segments with overlap. The chunk boundary strategy (sentence, paragraph, semantic section) affects retrieval quality more than model choice for most tasks.
Limitations
Embeddings encode compressed representations, not facts. They're good at capturing topical and stylistic similarity but fail on negation ("not X" often embeds close to "X"), precise numbers, and rare proper nouns. Don't use embedding similarity as a factual correctness signal.