RAG (Retrieval-Augmented Generation) Cheat Sheet
Design retrieval-augmented pipelines covering chunking strategies, hybrid search, reranking, and evaluation of grounded LLM answers.
3 PagesIntermediateFeb 25, 2026
Chunk Documents for Retrieval
Split long text into overlapping chunks sized for the embedding model's context window.
python
from langchain_text_splitters import RecursiveCharacterTextSplittersplitter = RecursiveCharacterTextSplitter( chunk_size=800, chunk_overlap=120, separators=["\n\n", "\n", ". ", " ", ""],)chunks = splitter.split_text(long_document)print(f"{len(chunks)} chunks, avg len {sum(len(c) for c in chunks)//len(chunks)}")
Hybrid Search (Dense + Sparse)
Combine keyword (BM25) and vector search scores for more robust retrieval.
python
from rank_bm25 import BM25Okapibm25 = BM25Okapi([c.split() for c in chunks])bm25_scores = bm25.get_scores(query.split())vector_scores = vector_index.similarity_scores(query, top_k=len(chunks))# reciprocal rank fusion of the two rankingsdef rrf_score(rank, k=60): return 1.0 / (k + rank)combined = {}for rank, idx in enumerate(np.argsort(-bm25_scores)): combined[idx] = combined.get(idx, 0) + rrf_score(rank)for rank, idx in enumerate(np.argsort(-vector_scores)): combined[idx] = combined.get(idx, 0) + rrf_score(rank)
Rerank Retrieved Chunks
Use a cross-encoder to rescore the top candidates before sending them to the LLM.
python
from sentence_transformers import CrossEncoderreranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")pairs = [(query, chunk) for chunk in top_20_chunks]scores = reranker.predict(pairs)ranked = [c for _, c in sorted(zip(scores, top_20_chunks), reverse=True)]top_5 = ranked[:5]
Build a Grounded Prompt
Assemble retrieved context into a prompt that instructs the model to only answer from context.
python
SYSTEM = """Answer only using the CONTEXT below. If the answer isn'tcontained in the context, say you don't know. Cite the [source] for each claim."""context = "\n\n".join(f"[{c.metadata['source']}] {c.text}" for c in top_5)messages = [ {"role": "system", "content": SYSTEM}, {"role": "user", "content": f"CONTEXT:\n{context}\n\nQUESTION: {query}"},]
Common RAG Failure Modes
The most frequent ways a RAG pipeline breaks in production.
- Chunk too large- dilutes relevance signal, buries the useful sentence in noise
- Chunk too small- loses surrounding context needed to answer correctly
- No reranking- top-k vector search alone often misses the best passage
- Missing metadata filters- retrieval mixes documents across tenants/versions/dates
- Stale index- source docs changed but the vector store was never re-embedded
- No answer refusal- model hallucinates instead of saying context is insufficient
Hypothetical Document Embeddings (HyDE)
Improve recall on short or ambiguous queries by embedding a hypothetical answer instead of the raw query.
python
def hyde_retrieve(query, llm, embed_model, vector_index, k=5): hypothetical = llm.complete( f"Write a short passage that would answer this question:\n{query}" ).text hyde_vector = embed_model.encode(hypothetical) return vector_index.search(hyde_vector, top_k=k)# HyDE closes the gap between a terse query's embedding and a document's# embedding, since the hypothetical passage lives in the same 'style space'
Multi-Query Retrieval
Generate several paraphrased queries and union their retrieved chunks to reduce sensitivity to phrasing.
python
def multi_query_retrieve(query, llm, retriever, n_variants=4, k=5): prompt = ( f"Generate {n_variants} different ways to ask this question, " f"one per line:\n{query}" ) variants = [query] + llm.complete(prompt).text.strip().split("\n") seen, merged = set(), [] for v in variants: for chunk in retriever.search(v, top_k=k): if chunk.id not in seen: seen.add(chunk.id) merged.append(chunk) return merged
Small-to-Big (Parent Document) Retrieval
Search over small child chunks for precision, then return the larger parent chunk for generation context.
python
from langchain.retrievers import ParentDocumentRetrieverfrom langchain.storage import InMemoryStorefrom langchain_text_splitters import RecursiveCharacterTextSplitterchild_splitter = RecursiveCharacterTextSplitter(chunk_size=200)parent_splitter = RecursiveCharacterTextSplitter(chunk_size=1500)retriever = ParentDocumentRetriever( vectorstore=vector_store, # indexes only the small child chunks docstore=InMemoryStore(), # holds the full parent chunks child_splitter=child_splitter, parent_splitter=parent_splitter,)retriever.add_documents(raw_docs)# search matches on precise child text but returns the richer parent contextresults = retriever.invoke("what triggers a circuit breaker retry?")
Evaluate a RAG Pipeline with RAGAS
Score retrieval and generation quality separately using automated, LLM-graded RAG metrics.
python
from datasets import Datasetfrom ragas import evaluatefrom ragas.metrics import faithfulness, answer_relevancy, context_precision, context_recalldataset = Dataset.from_dict({ "question": questions, "answer": generated_answers, "contexts": retrieved_contexts, # list[list[str]] per question "ground_truth": reference_answers,})result = evaluate( dataset, metrics=[faithfulness, answer_relevancy, context_precision, context_recall],)print(result) # faithfulness catches hallucination, context_recall catches bad retrieval
Advanced RAG Architectures
Patterns that go beyond a single retrieve-then-generate pass.
- Self-RAG- model emits reflection tokens to decide whether to retrieve and to critique its own draft answer
- Corrective RAG (CRAG)- a lightweight grader scores retrieved chunks and triggers a web search fallback on low confidence
- GraphRAG- builds a knowledge graph from the corpus and retrieves via graph traversal plus community summaries
- Agentic RAG- an LLM agent iteratively plans multiple retrieval calls across tools/indexes before answering
- Contextual compression- an LLM or extractor trims each retrieved chunk to only the sentences relevant to the query before prompting
- Query routing- classifies the query first to send it to the right index (e.g. FAQ vs. code vs. tabular store)
Pro Tip
Evaluate retrieval and generation separately — measure context recall (did we fetch the right chunk?) before measuring answer quality, since a perfect generator can't fix bad retrieval.
Was this cheat sheet helpful?
Explore Topics
#RAGRetrievalAugmentedGeneration#RAGRetrievalAugmentedGenerationCheatSheet#DataScience#Intermediate#ChunkDocumentsForRetrieval#Hybrid#Search#Dense#MachineLearning#DevOps#CheatSheet#SkillVeris
