Vector Databases for AI Cheat Sheet
Explains embeddings, approximate nearest neighbor search, and indexing strategies like HNSW, with code for storing and querying vectors using common libraries.
2 PagesIntermediateMar 20, 2026
Core Concepts
Foundations of vector search.
- Embedding- A dense numeric vector representing the semantic meaning of text, images, or other data, produced by a model
- Similarity metric- Cosine similarity, dot product, or Euclidean (L2) distance used to compare vectors; must match how the embedding model was trained
- Approximate Nearest Neighbor (ANN)- Trades a small amount of recall for large speedups over exact nearest-neighbor search at scale
- HNSW (Hierarchical Navigable Small World)- Graph-based ANN index offering strong recall/speed trade-offs; the most common index type in production vector DBs
- IVF (Inverted File Index)- Clusters vectors into partitions (via k-means) and searches only the nearest partitions, common in FAISS
- Metadata filtering- Combining vector similarity search with structured filters (e.g., date, category) in the same query
Local ANN Search with FAISS
Build and query an in-memory HNSW index.
python
import faissimport numpy as npdim = 384index = faiss.IndexHNSWFlat(dim, 32) # 32 = M, neighbors per nodeindex.hnsw.efConstruction = 200vectors = np.random.rand(10000, dim).astype('float32')index.add(vectors)query = np.random.rand(1, dim).astype('float32')distances, indices = index.search(query, k=5) # top-5 nearest neighbors
Storing and Querying with Chroma
A lightweight vector DB for embedding-based retrieval (e.g., RAG).
python
import chromadbclient = chromadb.Client()collection = client.create_collection("docs")collection.add( ids=["doc1", "doc2"], documents=["Paris is the capital of France.", "The Eiffel Tower is in Paris."], metadatas=[{"source": "wiki"}, {"source": "wiki"}],)results = collection.query(query_texts=["What city is the Eiffel Tower in?"], n_results=2)print(results["documents"])
Choosing a Vector Database
Key differentiators across common options.
- FAISS- Library, not a server; fastest for local/in-memory search but no built-in persistence, filtering, or multi-tenancy
- Chroma- Lightweight, easy local setup, popular for prototyping RAG applications
- Pinecone- Fully managed cloud service with metadata filtering, namespaces, and horizontal scaling built in
- pgvector- Postgres extension adding vector columns/indexes, useful when you want vectors alongside existing relational data
- Weaviate / Milvus / Qdrant- Self-hostable or managed vector DBs with hybrid (vector + keyword) search and filtering support
Tuning HNSW Recall vs. Speed
efConstruction, efSearch, and M trade index build cost, query latency, and recall against each other.
python
import faissdim = 384# M: neighbors per node (higher = better recall, more memory, slower build)index = faiss.IndexHNSWFlat(dim, 48)index.hnsw.efConstruction = 300 # higher = better graph quality, slower buildindex.add(vectors)# efSearch controls the query-time recall/latency trade-off;# raise it for higher recall at the cost of query latency, no rebuild neededindex.hnsw.efSearch = 128distances, indices = index.search(query, k=10)# Rule of thumb: efSearch >= k, and tune it per-workload rather than# baking one value in -- batch jobs can afford a much higher efSearch# than a user-facing p99-sensitive endpoint.
pgvector: HNSW Index & Filtered Query
Creating a vector column with an HNSW index and combining it with a relational WHERE clause in one query.
sql
CREATE EXTENSION IF NOT EXISTS vector;CREATE TABLE documents ( id bigserial PRIMARY KEY, content text, category text, embedding vector(384));-- HNSW index (Postgres 16+ / pgvector 0.5+); tune m and ef_constructionCREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 64);-- Set search-time recall knob per sessionSET hnsw.ef_search = 100;SELECT id, content, embedding <=> '[0.12, 0.04, ...]' AS distanceFROM documentsWHERE category = 'engineering' -- pre-filter narrows the candidate setORDER BY embedding <=> '[0.12, 0.04, ...]'LIMIT 10;
Qdrant: Filtered Vector Search with Payload
Combining a vector query with structured payload filters and score threshold in a single request.
python
from qdrant_client import QdrantClientfrom qdrant_client.models import Filter, FieldCondition, MatchValue, Rangeclient = QdrantClient(url="http://localhost:6333")results = client.query_points( collection_name="docs", query=query_vector, query_filter=Filter( must=[ FieldCondition(key="category", match=MatchValue(value="tutorial")), FieldCondition(key="published_year", range=Range(gte=2023)), ] ), score_threshold=0.75, limit=10, with_payload=True,)for point in results.points: print(point.score, point.payload["title"])
Vector Compression & Quantization
Techniques for shrinking index memory footprint at scale, roughly ordered from lossiest to least lossy.
- Binary quantization- Collapses each dimension to 1 bit; ~32x memory reduction, needs a rerank pass over full-precision vectors to recover accuracy
- Product Quantization (PQ)- Splits vectors into sub-vectors, clusters each independently, and stores cluster IDs; large memory savings with moderate recall loss, used by FAISS IVF-PQ
- Scalar Quantization (SQ)- Reduces float32 components to int8, roughly a 4x memory reduction with minimal recall impact -- often the best default trade-off
- Matryoshka Representation Learning (MRL)- Embedding models trained so the first N dimensions of the vector are independently useful, letting you truncate to a shorter vector for a cheap first-pass filter
- Two-stage retrieve-and-rerank- Use a compressed/truncated index to fetch a wide candidate set cheaply, then rerank the top candidates with full-precision vectors or a cross-encoder
- Disk-based ANN (DiskANN/Vamana)- Keeps the graph on SSD instead of RAM, trading some latency for the ability to index billions of vectors on commodity hardware
Evaluating Retrieval Quality: Recall@K
Measuring how often the ANN index's approximate results overlap with brute-force ground truth.
python
import numpy as npdef recall_at_k(approx_ids: np.ndarray, exact_ids: np.ndarray, k: int) -> float: """approx_ids/exact_ids: (n_queries, k) arrays of neighbor indices.""" hits = 0 for approx_row, exact_row in zip(approx_ids[:, :k], exact_ids[:, :k]): hits += len(set(approx_row) & set(exact_row)) return hits / (len(approx_ids) * k)# ground truth via brute-force flat indexflat_index = faiss.IndexFlatIP(dim)flat_index.add(vectors)_, exact_ids = flat_index.search(queries, k=10)_, approx_ids = index.search(queries, k=10) # the HNSW/IVF index under testprint(f"recall@10 = {recall_at_k(approx_ids, exact_ids, 10):.3f}")# track this metric per index-config change (efSearch, nprobe, PQ bits)# so quantization/speed tuning doesn't silently regress quality
Pro Tip
Always use the same distance metric the embedding model was trained/normalized for (usually cosine similarity) -- mixing, say, an L2 index with cosine-normalized embeddings silently degrades retrieval quality without throwing an error.
Was this cheat sheet helpful?
Explore Topics
#VectorDatabasesForAI#VectorDatabasesForAICheatSheet#DataScience#Intermediate#CoreConcepts#Local#ANN#Search#Databases#MachineLearning#CheatSheet#SkillVeris
