Embeddings & Similarity
Embeddings convert text into fixed-size vectors where semantic similarity corresponds to geometric closeness. They power semantic search, clustering, and RAG retrieval.
Recipe
Quick-reference recipe card - copy-paste ready.
from openai import OpenAI
import numpy as np
client = OpenAI()
emb = client.embeddings.create(model="text-embedding-3-small", input=["query text"])
vec = np.array(emb.data[0].embedding)
# cosine similarity
def cosine(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))When to reach for this:
- Semantic search over documents, code, or support articles.
- Clustering or deduplicating text content.
- RAG retrieval step before LLM generation.
- Recommendation systems based on text similarity.
Working Example
"""embeddings_similarity.py - embed, index, and search documents."""
from __future__ import annotations
import numpy as np
from openai import OpenAI
client = OpenAI()
MODEL = "text-embedding-3-small"
documents = [
"FastAPI is a modern Python web framework for building APIs.",
"PyTorch is a deep learning framework with dynamic computation graphs.",
"pandas 2.2 provides DataFrame operations for tabular data analysis.",
"pytest is the standard testing framework for Python projects.",
]
def embed_texts(texts: list[str]) -> np.ndarray:
response = client.embeddings.create(model=MODEL, input=texts)
vectors = [item.embedding for item in response.data]
arr = np.array(vectors, dtype=np.float32)
norms = np.linalg.norm(arr, axis=1, keepdims=True)
return arr / norms # L2 normalize for cosine via dot product
doc_embeddings = embed_texts(documents)
query = "How do I test my Python code?"
query_vec = embed_texts([query])[0]
scores = doc_embeddings @ query_vec # cosine similarity (normalized)
ranked = sorted(zip(scores, documents), reverse=True)
for score, doc in ranked:
print(f"{score:.3f} {doc}")What this demonstrates:
- Batch embedding with OpenAI
text-embedding-3-small. - L2 normalization so dot product equals cosine similarity.
- Ranking documents by relevance to a natural language query.
- Semantic match: "test Python code" ranks pytest highest.
Deep Dive
How It Works
- Embedding model encodes text into a fixed-dimension vector (e.g., 1536).
- Similar texts produce vectors with high cosine similarity.
- Normalization (L2) makes dot product equivalent to cosine similarity.
- Vector index (FAISS, pgvector) enables fast nearest-neighbor search at scale.
- Same model must be used for indexing and querying.
Embedding Models
| Model | Dims | Provider | Use |
|---|---|---|---|
| text-embedding-3-small | 1536 | OpenAI | General purpose, cost-effective |
| text-embedding-3-large | 3072 | OpenAI | Higher quality |
| all-MiniLM-L6-v2 | 384 | Local (sentence-transformers) | Offline, fast |
| voyage-3 | 1024 | Voyage AI | Retrieval-optimized |
Python Notes
# Local embeddings with sentence-transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
vectors = model.encode(documents, normalize_embeddings=True)Gotchas
- Mixing embedding models - different models produce incompatible vector spaces. Fix: one model per index; re-embed all documents on model change.
- Not normalizing vectors - dot product without normalization is not cosine similarity. Fix: L2-normalize before indexing and search.
- Embedding very long texts - models have token limits; long docs get truncated. Fix: chunk documents before embedding.
- Comparing across different dimensions - 384-dim vs 1536-dim vectors cannot be compared. Fix: consistent model and dimension.
- Ignoring embedding cost at scale - embedding millions of docs adds up. Fix: cache embeddings; batch requests; use smaller models.
- Keyword search expectation - embeddings miss exact keyword matches. Fix: hybrid search (embeddings + BM25).
Alternatives
| Alternative | Use When | Don't Use When |
|---|---|---|
| API embeddings (OpenAI) | No GPU, high quality | Data cannot leave your network |
| sentence-transformers | Local/offline, free | Need highest retrieval quality |
| BM25 keyword search | Exact term matching | Semantic/paraphrase matching needed |
| Hybrid (BM25 + dense) | Production RAG | Simple prototype with few docs |
FAQs
What is cosine similarity?
- Measures angle between two vectors: range [-1, 1], typically [0, 1] for text.
- 1.0 = identical direction; 0 = orthogonal (unrelated).
How many dimensions do I need?
- 384-1536 for most applications.
- Higher dims capture nuance but cost more storage and compute.
- Match your vector database configuration.
Should I embed queries and documents the same way?
- Yes - same model, same preprocessing.
- Some models have different prefixes for query vs document (e.g., "query: " prefix).
How do I store embeddings?
- PostgreSQL with pgvector extension.
- Chroma, Qdrant, Pinecone for dedicated vector stores.
- See Vector Databases.
What is a good similarity threshold?
- Depends on model and domain; typically 0.7-0.85 for relevant matches.
- Calibrate on labeled query-document pairs.
How do I reduce embedding dimensions?
client.embeddings.create(model="text-embedding-3-small", input=texts, dimensions=512)- OpenAI supports Matryoshka dimension reduction.
Can I embed code?
- General text embeddings work for code search.
- Specialized models (CodeBERT, voyage-code) may improve results.
How do I batch embedding requests?
- OpenAI accepts up to 2048 inputs per request.
- Batch for cost efficiency on large corpora.
What is hybrid search?
- Combine dense (embedding) and sparse (BM25) retrieval scores.
- Reciprocal Rank Fusion merges ranked lists.
How do embeddings relate to RAG?
- Embed documents at ingestion; embed query at retrieval.
- Top-k similar chunks feed the LLM as context.
- See RAG Basics.
How do I update embeddings when docs change?
- Re-embed changed documents and upsert in the vector index.
- Track document version/hash to avoid stale vectors.
Are embeddings deterministic?
- Same model and input produce the same vector (minor float variance).
- Cache embeddings by content hash for idempotency.
Related
- RAG Basics - retrieval pipeline
- Vector Databases - storage and search
- Chunking & Ingestion - document splitting
- OpenAI & Other SDKs - embedding API
- Retrieval & Re-ranking - hybrid search
Stack versions: This page was written for Python 3.14.0 (stable 3.14, maintenance 3.13), FastAPI 0.115+, Django 5.2, Flask 3.1, Pydantic 2, PyTorch 2.6+, pandas 2.2+, Polars 1.x, ruff 0.9+, and uv 0.6+.