Vector Databases (Pinecone/Weaviate) Cheat Sheet
Explains vector database fundamentals such as embeddings, ANN search, and metadata filtering, with practical Pinecone and Weaviate setup examples.
2 PagesIntermediateMar 15, 2026
Pinecone Setup & Query
Create a serverless index, upsert vectors, and run a filtered similarity search.
python
from pinecone import Pinecone, ServerlessSpecpc = Pinecone(api_key="YOUR_API_KEY")# Create a serverless index (dimension must match your embedding model)pc.create_index( name="products", dimension=1536, metric="cosine", # cosine | euclidean | dotproduct spec=ServerlessSpec(cloud="aws", region="us-east-1"))index = pc.Index("products")# Upsert vectors with metadataindex.upsert(vectors=[ {"id": "vec1", "values": [0.1, 0.2, 0.3], "metadata": {"category": "shoes"}}, {"id": "vec2", "values": [0.4, 0.1, 0.9], "metadata": {"category": "bags"}},])# Query for nearest neighbors, filtered by metadataresults = index.query( vector=[0.1, 0.2, 0.3], top_k=5, include_metadata=True, filter={"category": {"$eq": "shoes"}})
Weaviate Setup & Query
Connect to Weaviate Cloud, define a collection with an auto-vectorizer, and run semantic search.
python
import weaviatefrom weaviate.classes.init import Authfrom weaviate.classes.config import Configure, Property, DataTypeclient = weaviate.connect_to_weaviate_cloud( cluster_url="https://your-cluster.weaviate.network", auth_credentials=Auth.api_key("YOUR_API_KEY"),)# Create a collection with a built-in vectorizer modulearticles = client.collections.create( name="Article", vectorizer_config=Configure.Vectorizer.text2vec_openai(), properties=[ Property(name="title", data_type=DataType.TEXT), Property(name="body", data_type=DataType.TEXT), ],)# Insert an object (Weaviate auto-generates the embedding)articles.data.insert({"title": "Hello", "body": "World"})# Semantic searchresponse = articles.query.near_text(query="machine learning", limit=5)for obj in response.objects: print(obj.properties)client.close()
Core Concepts
Vocabulary shared across most vector database products.
- Embedding- A dense numeric vector (commonly 384-1536 dims) produced by an ML model that captures the semantic meaning of text, images, or audio.
- HNSW- Hierarchical Navigable Small World, the graph-based approximate nearest neighbor (ANN) algorithm both Pinecone and Weaviate use by default for fast similarity search.
- Cosine similarity- The default distance metric for most text embeddings; measures the angle between two vectors while ignoring magnitude.
- Namespace (Pinecone)- A logical partition within a Pinecone index used to isolate tenants or data subsets without creating separate indexes.
- Collection (Weaviate)- Weaviate's equivalent of a table/schema; defines properties, vectorizer, and index configuration for a set of objects.
- Hybrid search- Combines dense vector similarity with sparse keyword (BM25) search; supported natively by both Pinecone (via sparse-dense vectors) and Weaviate's hybrid() query.
- Metadata filtering- Restricts ANN search to vectors matching structured filters (category, date, tenant) applied alongside the vector search.
- top_k / limit- The number of nearest neighbors to return; larger values trade off latency and recall.
Pinecone vs. Weaviate
Key differences to consider when choosing between the two.
- Deployment- Pinecone: fully managed, serverless-only SaaS. Weaviate: open-source (self-host via Docker/Kubernetes) or Weaviate Cloud managed offering.
- Vectorization- Pinecone stores and searches vectors you supply (bring your own embeddings, or use its integrated inference API). Weaviate can auto-vectorize objects at insert time via built-in modules.
- Query interface- Pinecone uses a simple query() call with a filter dict. Weaviate uses GraphQL under the hood plus a fluent Python/TS client (near_text, near_vector, bm25, hybrid).
- Multi-tenancy- Pinecone isolates tenants with namespaces per index. Weaviate has explicit multi-tenancy support per collection with isolated tenant shards.
- Data model- Pinecone stores id + vector + flat metadata. Weaviate collections have typed schemas (properties), closer to a document database.
- Scaling- Pinecone serverless auto-scales storage/compute per index. Weaviate scaling depends on the cluster/shard configuration you provision when self-hosted.
Pinecone Sparse-Dense Hybrid Search
Combine a dense embedding with a sparse BM25-style vector in a single query for hybrid relevance ranking.
python
from pinecone_text.sparse import BM25Encoderbm25 = BM25Encoder().default()bm25.fit(corpus) # list of raw document strings# Upsert with both dense and sparse valuesindex.upsert(vectors=[{ "id": "doc1", "values": dense_embedding, # e.g. 1536-dim OpenAI embedding "sparse_values": bm25.encode_documents("wireless noise cancelling headphones"), "metadata": {"category": "electronics"}}])# Query with alpha-weighted hybrid scoring (alpha=1 -> pure dense, 0 -> pure sparse)def hybrid_scale(dense, sparse, alpha=0.8): hs = {"indices": sparse["indices"], "values": [v * (1 - alpha) for v in sparse["values"]]} hd = [v * alpha for v in dense] return hd, hsh_dense, h_sparse = hybrid_scale(query_dense, bm25.encode_queries("noise cancelling"))results = index.query(vector=h_dense, sparse_vector=h_sparse, top_k=10, include_metadata=True)
Weaviate Hybrid Search + Reranking
Blend BM25 keyword search with vector search using alpha, then apply a cross-encoder reranker module.
python
from weaviate.classes.query import HybridFusionresponse = articles.query.hybrid( query="transformer attention mechanism", alpha=0.5, # 0 = pure BM25, 1 = pure vector fusion_type=HybridFusion.RELATIVE_SCORE, limit=20, query_properties=["title^2", "body"], # boost title matches 2x)# Rerank the top candidates with a cross-encoder module (e.g. reranker-cohere)reranked = articles.query.hybrid( query="transformer attention mechanism", alpha=0.5, limit=20,).with_rerank( property="body", query="transformer attention mechanism",)for obj in reranked.objects: print(obj.metadata.rerank_score, obj.properties["title"])
Weaviate Multi-Tenancy
Enable per-tenant isolated shards on a collection and target queries to a specific tenant.
python
from weaviate.classes.config import Configuretenants_collection = client.collections.create( name="TenantDocs", multi_tenancy_config=Configure.multi_tenancy(enabled=True, auto_tenant_creation=True),)# Add tenants explicitly (or rely on auto_tenant_creation on insert)from weaviate.classes.tenants import Tenanttenants_collection.tenants.create([Tenant(name="acme-corp"), Tenant(name="globex")])# All reads/writes must be scoped with .with_tenant()tenant_scope = tenants_collection.with_tenant("acme-corp")tenant_scope.data.insert({"title": "Acme onboarding doc"})results = tenant_scope.query.near_text(query="onboarding", limit=5)
ANN Index Tuning Parameters
Knobs that trade off recall, latency, and memory for HNSW-based indexes.
- M (max connections)- Number of bidirectional links per HNSW graph node. Higher M improves recall but increases memory footprint and index build time; typical range 16-64.
- efConstruction- Search breadth used while building the graph. Higher values produce a higher-quality graph at the cost of slower indexing; commonly 100-500.
- efSearch / ef- Search breadth used at query time. Increasing it raises recall and latency simultaneously; tune per-query when a request needs higher precision.
- Product quantization (PQ)- Compresses vectors into short codes to shrink memory usage at the cost of some recall; Pinecone's pod-based (p1/p2) indexes support this, serverless handles compression internally.
- Pre-filtering vs. post-filtering- Pre-filtering (Weaviate's default) restricts the ANN graph traversal to matching objects before searching, avoiding the 'filtered-out results' problem post-filtering has with small top_k.
- Recall vs. latency curve- Benchmark with your real embedding distribution and filter selectivity — HNSW recall claims from vendor docs rarely transfer directly to filtered, high-cardinality metadata workloads.
Pinecone Batched Upsert with Backoff
Chunk large upserts and retry transient failures, which is required for production ingestion pipelines.
python
import timefrom itertools import islicedef chunks(iterable, size=100): it = iter(iterable) while batch := list(islice(it, size)): yield batchdef upsert_with_retry(index, vectors, max_retries=5): for batch in chunks(vectors, size=100): for attempt in range(max_retries): try: index.upsert(vectors=batch, namespace="prod") break except Exception as e: wait = min(2 ** attempt, 30) print(f"upsert failed ({e}); retrying in {wait}s") time.sleep(wait) else: raise RuntimeError("upsert batch failed after max retries")
Pro Tip
When you switch embedding models, re-embed and re-upsert every vector — cosine distance between vectors produced by different models is meaningless, and silently mixing them corrupts search quality without throwing any error.
Was this cheat sheet helpful?
Explore Topics
#VectorDatabasesPineconeWeaviate#VectorDatabasesPineconeWeaviateCheatSheet#Database#Intermediate#PineconeSetupQuery#WeaviateSetupQuery#CoreConcepts#PineconeVsWeaviate#Databases#CheatSheet#SkillVeris
