What You'll Build
In this exercise, you will construct a Retrieval-Augmented Generation (RAG) system designed for cricket match analysis. The system retrieves relevant historical match statistics, player performance records, and tactical insights from a structured knowledge base, then uses a language model to generate contextual commentary and predictive analysis grounded in those retrieved documents.
The pipeline combines vector similarity search over cricket match embeddings with prompt engineering to synthesize real-time match context with historical patterns. This enables the model to produce accurate, grounded predictions about player performance, team strategy, and match outcomes without requiring any retraining of the underlying model.
This end-to-end RAG pipeline illustrates a key practical benefit of retrieval augmentation: by anchoring generative responses in factual cricket data, the system significantly reduces the risk of hallucination that would otherwise arise from relying solely on the model's parametric knowledge.
Analogy🏏Cricket
🏏 Think of it like cricket: Imagine you're a cricket analyst preparing for a match commentary. Before the game, you've collected match scorecards, player performance history, pitch reports, and weather data from the stadium—your 'knowledge base.' When a question comes up during live commentary, like 'Has Rohit Sharma ever scored a century here before?', you don't rely on memory alone (which might be fuzzy or incomplete). Instead, you quickly flip through your indexed filing system (the vector database) to retrieve the most relevant match records from that venue, check Rohit's actual performances there, and synthesize your answer with hard facts. The retrieval step is like scanning your organized files to find the right scorecards; the ranking is like prioritizing the most recent or relevant matches; the augmentation is the language model synthesizing a coherent, authoritative answer from those retrieved facts. Without RAG, you'd guess and risk broadcasting incorrect statistics—with RAG, you ground every answer in verified match data, just as professional analysts do with their research notes during live commentary.
Prerequisites
- Familiarity with vector embeddings, cosine similarity, and semantic search fundamentals for document retrieval
- Understanding of prompt engineering, chain-of-thought techniques, and how context windows affect model responses
- Basic knowledge of Python, including data structures, file I/O, and working with dictionaries and lists
- Experience with language model APIs (OpenAI, Anthropic, or local models) and token counting for context management
- Conceptual grasp of RAG architecture: retriever component, ranking strategies, and integration with generative models
Setup & Project Structure
You will organize your RAG cricket analysis project using a modular architecture in which document retrieval, embedding generation, and language model prompting are maintained as distinct, independently testable components. The project directory should contain a knowledge base of cricket match records stored as JSON documents, an embeddings cache for vector similarity search, and a main orchestration script that coordinates retrieval and generation.
Analogy🏏Cricket
🏏 Think of it like cricket: a professional team is not one crowd doing everything — it has a groundstaff preparing the pitch, an analytics unit crunching data, a selection panel picking the eleven, and a captain making the final call on the field. Your RAG project's four layers mirror this division of labour. Just as the groundstaff ingest and prepare the surface, your data layer loads and embeds the cricket match JSON. Just as the analytics unit searches records for patterns, your retrieval layer runs vector similarity to find relevant matches. Just as the selection panel narrows a longlist to the best eleven, your ranking layer reranks candidates for quality. And just as the captain synthesises everything into a match plan, your generation layer prompts the LLM with the retrieved context. The payoff of keeping these roles separate is the same reason clubs don't ask the groundsman to bat at three: each concern can be tested and improved independently without breaking the others.
To support this architecture, install the essential dependencies: a vector database client such as LanceDB or FAISS, an embedding model library such as sentence-transformers, and the appropriate language model client libraries. This separation of concerns enables you to develop and validate each RAG component in isolation before integration, closely simulating production patterns in which retrieval indices are updated independently of generation pipelines.
bash
#!/bin/bash
# Cricket RAG Analysis System - Project Setup
# Create project directory structure
mkdir -p cricket_rag_analysis
cd cricket_rag_analysis
# Create subdirectories for different components
mkdir -p knowledge_base # Cricket match records and statistics
mkdir -p embeddings # Vector embeddings cache
mkdir -p retriever # Retrieval component code
mkdir -p generator # Generation component code
mkdir -p utils # Utility functions
mkdir -p tests # Test cases
# Create Python virtual environment
python3 -m venv venv
source venv/bin/activate
# Install required dependencies
pip install --upgrade pip
pip install sentence-transformers==2.2.2 # For embedding models
pip install lancedb==0.3.0 # Vector database
pip install numpy==1.24.3 # Numerical computing
pip install openai==1.3.5 # OpenAI API
pip install python-dotenv==1.0.0 # Environment variables
pip install pandas==2.0.3 # Data manipulation
pip install requests==2.31.0 # HTTP requests
# Create main configuration file
cat > config.py << 'EOF'
# Cricket RAG Configuration
EMBEDDING_MODEL = "all-MiniLM-L6-v2"
VECTOR_DB_PATH = "./embeddings/cricket_db"
KNOWLEDGE_BASE_PATH = "./knowledge_base"
EMBEDDING_DIMENSION = 384
TOP_K_RETRIEVAL = 5
MODEL_NAME = "gpt-3.5-turbo"
TEMPERATURE = 0.7
MAX_TOKENS = 500
EOF
# Create .gitignore for sensitive data
cat > .gitignore << 'EOF'
venv/
.env
__pycache__/
*.pyc
embeddings/
.DS_Store
EOF
echo "✅ Cricket RAG project structure created successfully!"
echo "📂 Project layout:"
tree -L 2 . 2>/dev/null || find . -maxdepth 2 -type d | sed 's|./||' | sort
Step 1 — Foundation
Step 1 establishes the knowledge base and embedding infrastructure. This involves loading cricket match records, computing dense vector embeddings for each document using a pre-trained language model, and storing the resulting embeddings in a queryable vector database.
This phase transforms unstructured cricket match data — including player statistics, team lineups, commentary, and pitch conditions — into numerical vectors. Semantically similar matches are positioned in nearby regions of the embedding space, allowing the retriever to index documents efficiently and perform rapid similarity-based lookups when queries arrive at inference time.
Without this foundational embedding layer, retrieval cannot function. The system would have no mathematical basis for determining which historical matches are most relevant to a given query, making the entire RAG pipeline inoperable.
Analogy🏏Cricket
🏏 Think of it like cricket: Before a cricket tournament, the organizers must catalog every match—they assign each match a unique identity card that captures essential information: teams, date, venue, winning margin, key player performances, and significant moments. This is like your MatchDocument structure. Then they assign each match a numeric 'fingerprint' (embedding) based on its characteristics—two similar matches (like India vs Pakistan in bilateral ODI at the same venue) get similar fingerprints, while different matches (India vs USA in T20) get distinct fingerprints. Just as scorecards use numerical encoding to store all match data compactly, embeddings compress match semantics into dense vectors. The batch embedding process is like creating a centralized digital archive of all match fingerprints rather than computing them one-by-one, which saves time during tournament season. Without this foundational fingerprinting step, you couldn't quickly find 'all matches where Rohit Sharma scored a century under similar conditions'—you'd have to read every scorecard manually.
python
import json
import numpy as np
from sentence_transformers import SentenceTransformer
import lancedb
from pathlib import Path
from typing import List, Dict, Any
class CricketKnowledgeBaseBuilder:
"""Builds and indexes the cricket knowledge base with embeddings."""
def __init__(self, embedding_model_name: str = "all-MiniLM-L6-v2", db_path: str = "./embeddings/cricket_db"):
self.embedding_model = SentenceTransformer(embedding_model_name)
self.db_path = db_path
self.db = None
Path(self.db_path).parent.mkdir(parents=True, exist_ok=True)
def load_cricket_knowledge_base(self) -> List[Dict[str, Any]]:
"""Load cricket match records from JSON knowledge base."""
cricket_documents = [
{
"match_id": "IND_AUS_TEST_2023_01",
"teams": "India vs Australia",
"venue": "MCG, Melbourne",
"pitch_type": "fast_bouncy",
"match_type": "Test",
"content": "India toured Australia for Test series. Jasprit Bumrah bowled with exceptional accuracy. Rohit Sharma scored 150 runs. Australia won by 8 wickets. Pitch favored pace bowling with high bounce. Weather conditions cool and overcast.",
"player_stats": {"Rohit Sharma": {"runs": 150, "dismissal": "lbw"}, "Jasprit Bumrah": {"wickets": 3, "economy": 2.1}}
},
{
"match_id": "IND_AUS_TEST_2023_02",
"teams": "India vs Australia",
"venue": "SCG, Sydney",
"pitch_type": "slow_turning",
"match_type": "Test",
"content": "Second Test in Sydney. Axar Patel exploited turning pitch. Virat Kohli scored 89 runs. India won by 6 wickets. Pitch turned significantly on Day 3. Spinners dominated second innings.",
"player_stats": {"Virat Kohli": {"runs": 89, "dismissal": "caught"}, "Axar Patel": {"wickets": 5, "economy": 2.8}}
},
{
"match_id": "IND_AUS_TEST_2023_03",
"teams": "India vs Australia",
"venue": "WACA, Perth",
"pitch_type": "fast_bouncy",
"match_type": "Test",
"content": "Third Test at Perth. Aggressive Australian batting on flat pitch. David Warner scored 164 runs. Australia dominated. Fast bowlers ineffective on flat surface. Weather hot and sunny.",
"player_stats": {"David Warner": {"runs": 164, "dismissal": "not_out"}, "Mitchell Starc": {"wickets": 2, "economy": 3.5}}
},
{
"match_id": "IND_AUS_ODI_2023_01",
"teams": "India vs Australia",
"venue": "MCG, Melbourne",
"pitch_type": "flat_batting",
"match_type": "ODI",
"content": "ODI series begins. India bats first. Shubman Gill scored 98 runs. Australia chased successfully. Powerplay aggression from Australian openers. India bowlers leaked 25 runs in final over.",
"player_stats": {"Shubman Gill": {"runs": 98, "dismissal": "caught"}, "Steve Smith": {"runs": 112, "dismissal": "not_out"}}
},
{
"match_id": "IND_SA_TEST_2023_01",
"teams": "India vs South Africa",
"venue": "Wanderers, Johannesburg",
"pitch_type": "fast_bouncy",
"match_type": "Test",
"content": "India's tour of South Africa. Kagiso Rabada took 6 wickets. India collapsed in first innings for 179. South Africa declared at 402. Pace bowling was destructive. High bounce challenged Indian batsmen.",
"player_stats": {"Virat Kohli": {"runs": 45, "dismissal": "caught"}, "Kagiso Rabada": {"wickets": 6, "economy": 2.2}}
}
]
return cricket_documents
def create_embeddings(self, documents: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""Generate embeddings for each document."""
embeddings_with_docs = []
for doc in documents:
# Combine relevant fields for embedding
text_to_embed = f"{doc['teams']} {doc['venue']} {doc['pitch_type']} {doc['match_type']} {doc['content']}"
# Generate embedding vector
embedding = self.embedding_model.encode(text_to_embed, convert_to_numpy=True)
# Store embedding with document
doc_with_embedding = doc.copy()
doc_with_embedding['embedding'] = embedding.tolist()
embeddings_with_docs.append(doc_with_embedding)
print(f"✓ Embedded: {doc['match_id']} - {doc['teams']}")
return embeddings_with_docs
def build_vector_database(self, embedded_documents: List[Dict[str, Any]]) -> None:
"""Create LanceDB vector database with embeddings."""
# Convert embeddings back to numpy for LanceDB
data = []
for doc in embedded_documents:
data.append({
"match_id": doc["match_id"],
"teams": doc["teams"],
"venue": doc["venue"],
"pitch_type": doc["pitch_type"],
"match_type": doc["match_type"],
"content": doc["content"],
"player_stats": json.dumps(doc["player_stats"]),
"vector": doc["embedding"]
})
# Create LanceDB table
self.db = lancedb.connect(self.db_path)
self.db.create_table("cricket_matches", data=data, mode="overwrite")
print(f"\n✅ Vector database created with {len(data)} matches")
def build(self) -> None:
"""Orchestrate the complete knowledge base building process."""
print("🏏 Starting Cricket Knowledge Base Builder...\n")
# Load documents
documents = self.load_cricket_knowledge_base()
print(f"📚 Loaded {len(documents)} cricket match documents\n")
# Generate embeddings
print("🔢 Generating embeddings...")
embedded_docs = self.create_embeddings(documents)
# Build vector database
print("\n💾 Building vector database...")
self.build_vector_database(embedded_docs)
print("\n🎯 Knowledge base foundation ready!")
if __name__ == "__main__":
builder = CricketKnowledgeBaseBuilder()
builder.build()
Step 2 — Core Logic
Step 2 implements the core retrieval logic, which takes a user query about a cricket match situation and surfaces the most relevant historical matches from the vector database using semantic similarity. The retriever embeds the incoming query using the same model established in Step 1, then performs a similarity search to identify the top-k most contextually relevant cricket matches.
The quality of retrieved documents directly determines the quality of final generation. If retrieval fails to surface relevant historical context, the language model cannot augment its response with grounded facts, making this step critical to overall pipeline performance.
The retriever ranks matches by cosine similarity in embedding space, effectively solving a nearest-neighbor search problem. While this operation would be prohibitively slow with traditional keyword matching, vector indices enable it to execute in milliseconds, making semantic retrieval practical at inference time.
Analogy🏏Cricket
🏏 Think of it like cricket: Consider a cricket statistician preparing for commentary before India plays New Zealand at the Wankhede Stadium. A viewer asks, "How does Virat Kohli perform at home against New Zealand bowlers?" The statistician doesn't just do a simple grep search for 'Virat Kohli' and 'New Zealand'—that would pull up irrelevant matches like Kohli's T20 performance against NZ in Dubai. Instead, they understand the semantic intent: we need home Test matches or ODIs where Kohli faced NZ pace bowling in comparable conditions. They mentally rank candidates: matches at Wankhede itself rank highest, recent performances rank higher than 10-year-old data, and actual head-to-head ODIs rank higher than practice matches. This is exactly what the core logic does: semantic search understands your query's intent (not just keywords), and intelligent ranking orders results by relevance factors (venue, recency, format, player presence). The reranker acts like the statistician's domain knowledge—it refines rough search results into precisely-ranked candidates. Without this step, you might retrieve matches with excellent semantic similarity but low practical relevance.
python
import numpy as np
from sentence_transformers import SentenceTransformer
import lancedb
from typing import List, Dict, Any, Tuple
import json
class CricketRetriever:
"""Retrieves relevant cricket match documents using semantic similarity."""
def __init__(self, embedding_model_name: str = "all-MiniLM-L6-v2", db_path: str = "./embeddings/cricket_db"):
self.embedding_model = SentenceTransformer(embedding_model_name)
self.db = lancedb.connect(db_path)
self.table = self.db.open_table("cricket_matches")
def encode_query(self, query: str) -> np.ndarray:
"""Convert query text to embedding vector."""
return self.embedding_model.encode(query, convert_to_numpy=True)
def retrieve_similar_matches(self, query: str, top_k: int = 5) -> List[Dict[str, Any]]:
"""Retrieve top-k most similar cricket matches for the query."""
# Encode the query
query_embedding = self.encode_query(query)
# Search the vector database
results = self.table.search(query_embedding).limit(top_k).to_list()
# Process results
retrieved_matches = []
for i, result in enumerate(results, 1):
# Parse player stats if stored as JSON string
player_stats = result.get("player_stats", "{}")
if isinstance(player_stats, str):
player_stats = json.loads(player_stats)
match_context = {
"rank": i,
"match_id": result.get("match_id", "unknown"),
"teams": result.get("teams", ""),
"venue": result.get("venue", ""),
"pitch_type": result.get("pitch_type", ""),
"match_type": result.get("match_type", ""),
"content": result.get("content", ""),
"player_stats": player_stats,
"similarity_score": float(result.get("_distance", 0)) # LanceDB uses _distance
}
retrieved_matches.append(match_context)
return retrieved_matches
def format_retrieved_context(self, matches: List[Dict[str, Any]]) -> str:
"""Format retrieved matches into a contextual prompt for the generator."""
formatted_context = "🏏 RETRIEVED HISTORICAL CONTEXT:\n\n"
for match in matches:
formatted_context += f"Match {match['rank']}: {match['match_id']}\n"
formatted_context += f" Teams: {match['teams']}\n"
formatted_context += f" Venue: {match['venue']}\n"
formatted_context += f" Pitch Type: {match['pitch_type']}\n"
formatted_context += f" Match Type: {match['match_type']}\n"
formatted_context += f" Key Details: {match['content']}\n"
if match['player_stats']:
formatted_context += f" Player Performance:\n"
for player, stats in match['player_stats'].items():
formatted_context += f" - {player}: {stats}\n"
formatted_context += f" Similarity Score: {match['similarity_score']:.4f}\n\n"
return formatted_context
if __name__ == "__main__":
# Initialize retriever
retriever = CricketRetriever()
# Test retrieval with various cricket scenarios
test_queries = [
"India batting first on a fast bouncy pitch against Australia at MCG. What worked before?",
"South Africa pace bowling strategy on bouncy Johannesburg wicket",
"Rohit Sharma against Mitchell Starc on turning pitch conditions",
"ODI chase strategy against Australia with flat wicket conditions"
]
print("🏏 CRICKET RAG RETRIEVER - TEST RUN\n" + "="*60 + "\n")
for query_idx, query in enumerate(test_queries, 1):
print(f"Query {query_idx}: {query}\n")
# Retrieve matches
matches = retriever.retrieve_similar_matches(query, top_k=3)
# Format and display context
context = retriever.format_retrieved_context(matches)
print(context)
print("-" * 60 + "\n")
Step 3 — Integration & Enhancement
Step 3 integrates the retriever with a language model to create the complete RAG pipeline. The generator accepts the user query, invokes the retriever to fetch relevant cricket match context, augments a carefully engineered prompt with the retrieved documents, and passes this enriched context to the language model for generation.
Prompt engineering is the critical enhancement at this stage. The system constructs prompts that explicitly instruct the model to ground its analysis in the retrieved historical matches, cite specific examples, and avoid speculation beyond the scope of the provided context. This directly embodies the core RAG principle: the model generates responses conditioned on factual retrieved content rather than relying solely on its training data.
To reinforce factual consistency, the system implements guardrails ensuring that generated cricket analysis references at least one retrieved match and remains consistent with the historical data surfaces by the retriever.
Analogy🏏Cricket
🏏 Think of it like cricket: The retrieval system (Step 2) found relevant match records, but now the commentator must synthesize these raw statistics into a coherent narrative for the broadcast audience. The commentator receives indexed research cards with match data, player statistics, and key moments, then crafts a compelling narrative: "When Virat Kohli faced Mitchell Starc at the MCG in 2023 [retrieved fact], he employed an aggressive approach [synthesis from context], scoring 85 off 95 balls [retrieved statistic], which set up India's successful chase [retrieved outcome]." The commentator carefully attributes each claim to its source—'according to the 2023 MCG match' or 'in that particular series'—so the audience knows where information comes from. If the research suggests Kohli scored 85 but the commentator accidentally says 95, that's a hallucination that must be caught. This synthesis process is what Step 3 does: the language model reads retrieved context and generates fluent, attributable answers rather than relying on generic pre-training knowledge. Understanding this integration reveals why RAG quality depends on both retrieval precision AND language model grounding—a brilliant response that contradicts retrieved facts is worse than a mediocre response that stays faithful to context.
python
from typing import List, Dict, Any
import os
from dotenv import load_dotenv
# Load environment variables
load_dotenv()
class CricketRAGPipeline:
"""Complete RAG pipeline integrating retrieval with language model generation."""
def __init__(self, retriever: 'CricketRetriever', model_name: str = "gpt-3.5-turbo"):
self.retriever = retriever
self.model_name = model_name
# Note: In production, use actual OpenAI client
# For this example, we demonstrate the orchestration
try:
import openai
self.client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
except:
self.client = None
print("⚠️ OpenAI client not available. Using mock responses for demo.")
def build_augmented_prompt(self, user_query: str, retrieved_context: str) -> str:
"""Construct a prompt that combines user query with retrieved context."""
prompt = f"""You are an expert cricket analyst with deep knowledge of match strategy and player psychology.
You have been provided with HISTORICAL CRICKET CONTEXT retrieved from similar past matches:
{retrieved_context}
Based ONLY on the historical context provided above, answer this question about the current match:
CURRENT MATCH QUERY: {user_query}
IMPORTANT INSTRUCTIONS:
1. Ground your analysis in the retrieved historical matches. Cite specific player names and match details.
2. Identify patterns and strategies that worked (or failed) in similar situations.
3. Make predictions about the current match based on these historical precedents.
4. If the retrieved context doesn't directly address the query, say so explicitly.
5. Do NOT invent cricket facts or statistics not present in the retrieved context.
6. Structure your response with: ANALYSIS, KEY PATTERNS, PREDICTION, CONFIDENCE LEVEL.
RESPONSE:"""
return prompt
def generate_analysis(self, user_query: str, top_k: int = 5) -> Dict[str, Any]:
"""Execute the complete RAG pipeline."""
print(f"\n🏏 CRICKET RAG ANALYSIS PIPELINE\n" + "="*70)
print(f"Query: {user_query}\n")
# Step 1: Retrieve relevant matches
print("🔍 Step 1: Retrieving relevant historical matches...")
retrieved_matches = self.retriever.retrieve_similar_matches(user_query, top_k=top_k)
print(f"✓ Retrieved {len(retrieved_matches)} relevant matches")
# Step 2: Format context
print("\n📝 Step 2: Formatting retrieved context...")
formatted_context = self.retriever.format_retrieved_context(retrieved_matches)
print(f"✓ Context formatted ({len(formatted_context)} characters)")
# Step 3: Build augmented prompt
print("\n🤖 Step 3: Building augmented prompt...")
augmented_prompt = self.build_augmented_prompt(user_query, formatted_context)
print(f"✓ Augmented prompt created ({len(augmented_prompt)} characters)")
# Step 4: Generate response (mock if client unavailable)
print("\n💭 Step 4: Generating analysis...")
if self.client:
try:
response = self.client.chat.completions.create(
model=self.model_name,
messages=[
{"role": "system", "content": "You are an expert cricket analyst."},
{"role": "user", "content": augmented_prompt}
],
temperature=0.7,
max_tokens=500
)
generated_text = response.choices[0].message.content
except Exception as e:
generated_text = self._mock_response(user_query, retrieved_matches)
print(f"⚠️ Using mock response (API error: {str(e)[:50]}...)")
else:
generated_text = self._mock_response(user_query, retrieved_matches)
print("⚠️ Using mock response (client not configured)")
print("✓ Analysis generated")
# Step 5: Compile results
result = {
"query": user_query,
"retrieved_matches": [
{
"match_id": m["match_id"],
"teams": m["teams"],
"venue": m["venue"],
"pitch_type": m["pitch_type"],
"similarity_score": m["similarity_score"]
}
for m in retrieved_matches
],
"analysis": generated_text,
"context_length": len(formatted_context),
"augmented_prompt_length": len(augmented_prompt)
}
return result
def _mock_response(self, query: str, matches: List[Dict[str, Any]]) -> str:
"""Generate mock cricket analysis response for demo purposes."""
match_summaries = [m['teams'] + " at " + m['venue'] for m in matches[:2]]
return f"""ANALYSIS:
Based on the retrieved historical matches, particularly {', '.join(match_summaries)}, the current scenario shows clear patterns from past performance.
KEY PATTERNS:
1. Pitch conditions significantly impact batting strategy - similar pitch types show 65-75% pattern consistency
2. Player performance varies by venue and opposition - venue-specific averages shift by 15-25 runs
3. Opening partnerships are more stable on bouncy pitches, with success rate improving 20% on Australian grounds
PREDICTION:
The batsman should expect a challenging period in the first 3 overs given the pace bowling conditions. Historical data from similar scenarios suggests:
- Defending off-stump deliveries should be prioritized (67% success rate)
- Attacking short balls yields positive results (72% boundary conversion)
- Expected run rate: 3.2-4.1 runs per over in powerplay conditions
CONFIDENCE LEVEL: 78%
(Based on 5 highly relevant historical matches with similar pitch, venue, and opposition conditions)"""
def run_analysis_suite(self, queries: List[str]) -> List[Dict[str, Any]]:
"""Run complete RAG analysis for multiple cricket scenarios."""
results = []
for i, query in enumerate(queries, 1):
print(f"\n\n{'='*70}")
print(f"ANALYSIS {i} of {len(queries)}")
print(f"{'='*70}")
result = self.generate_analysis(query, top_k=3)
results.append(result)
# Display result
print(f"\n📊 RETRIEVED MATCHES:")
for match in result['retrieved_matches']:
print(f" ✓ {match['match_id']}: {match['teams']} ({match['pitch_type']})")
print(f"\n📋 GENERATED ANALYSIS:")
print(result['analysis'])
return results
if __name__ == "__main__":
# Import retriever from Step 2
from cricket_retriever import CricketRetriever
# Initialize components
retriever = CricketRetriever()
pipeline = CricketRAGPipeline(retriever)
# Define test scenarios
test_queries = [
"India is batting first against Australia on a fast bouncy pitch at MCG. What should be the batting strategy for the opening partnership?",
"South Africa is preparing to defend on a pitch that's expected to take turn. Based on similar conditions, what bowling attack should they deploy?",
"Jasprit Bumrah is about to bowl the death overs against aggressive Australian batsmen. What has worked in similar high-pressure situations?"
]
# Execute RAG pipeline
results = pipeline.run_analysis_suite(test_queries)
print(f"\n\n{'='*70}")
print("✅ RAG PIPELINE COMPLETE")
print(f"{'='*70}")
print(f"Total queries processed: {len(results)}")
Step 4 — Testing & Verification
Step 4 validates that the complete RAG system functions correctly through end-to-end testing with cricket-specific scenarios. This involves verifying that retrieval returns contextually appropriate matches and confirming that generated analysis accurately cites retrieved facts.
Analogy🏏Cricket
🏏 Think of it like cricket: before a plan is trusted, a team plays a full warm-up fixture that runs every phase together — openers see the new ball, bowlers find rhythm, fielders hold catches — to expose faults while they are cheap to fix. Your Step 4 is that fixture. Just as a practice game exercises the whole team in sequence rather than in isolated nets, the test harness runs ingestion, retrieval, ranking, and generation end to end on sample cricket queries. Just as coaches log how each phase performed to spot the weak link, the harness logs intermediate results at every stage so you can see exactly where behaviour drifts. Passing individual nets never guarantees a winning eleven — only the full fixture reveals how the parts combine. The payoff is match-ready confidence: a pipeline proven to move a query cleanly from raw match data all the way to a grounded answer before it faces real users.
In practice, you should run the integrated system against diverse queries ranging from batting strategy questions to bowling tactics. During these tests, monitor that similarity scores make intuitive sense and that generated responses reference specific player names and match details drawn from the retrieved context.
This validation step serves three purposes: it ensures the retriever is not surfacing irrelevant matches, confirms that the embeddings capture domain semantics correctly, and verifies that the language model respects the constraint of grounding its output in the augmented context.
bash
#!/bin/bash
# Cricket RAG System - Testing & Verification
echo "🏏 CRICKET RAG SYSTEM - TESTING & VERIFICATION"
echo "============================================================="
# Activate virtual environment
source venv/bin/activate
# Step 1: Build knowledge base and embeddings
echo ""
echo "[Test 1] Building cricket knowledge base and embeddings..."
python3 -c "
from cricket_knowledge_base import CricketKnowledgeBaseBuilder
builder = CricketKnowledgeBaseBuilder()
builder.build()
print('✓ Knowledge base built successfully')
" 2>&1 | head -20
echo ""
echo "[Test 2] Verifying vector database creation..."
python3 -c "
import lancedb
db = lancedb.connect('./embeddings/cricket_db')
table = db.open_table('cricket_matches')
print(f'✓ Vector database verified: {len(table.search([0]*384).limit(100).to_list())} matches indexed')
" 2>&1
# Step 2: Test retriever component
echo ""
echo "[Test 3] Testing cricket match retrieval..."
python3 << 'RETRIEVER_TEST'
from cricket_retriever import CricketRetriever
retriever = CricketRetriever()
test_queries = [
"India opening partnership against fast bowling on bouncy pitch",
"Pace bowling strategy on turning pitch",
"Batting collapse recovery after early wickets"
]
print("\nTesting Retrieval Quality:")
print("-" * 60)
for query in test_queries:
matches = retriever.retrieve_similar_matches(query, top_k=3)
print(f"\nQuery: {query[:50]}...")
print(f"Retrieved {len(matches)} matches:")
for match in matches:
print(f" • {match['teams']} - Similarity: {match['similarity_score']:.4f}")
print("\n✓ Retriever tests passed")
RETRIEVER_TEST
# Step 3: Test RAG pipeline
echo ""
echo "[Test 4] Testing complete RAG pipeline..."
python3 << 'RAG_TEST'
from cricket_retriever import CricketRetriever
from cricket_rag_pipeline import CricketRAGPipeline
retriever = CricketRetriever()
pipeline = CricketRAGPipeline(retriever)
test_query = "Rohit Sharma batting first on MCG against Mitchell Starc. Historical context?"
print(f"\nTest Query: {test_query}")
print("="*60)
result = pipeline.generate_analysis(test_query, top_k=3)
print(f"\nRAG Pipeline Results:")
print(f" • Query processed: ✓")
print(f" • Matches retrieved: {len(result['retrieved_matches'])}")
print(f" • Context size: {result['context_length']} characters")
print(f" • Prompt size: {result['augmented_prompt_length']} characters")
print(f" • Analysis generated: ✓")
print(f"\nFirst 200 chars of analysis:")
print(f" {result['analysis'][:200]}...")
print(f"\n✓ RAG pipeline tests passed")
RAG_TEST
# Step 4: Performance metrics
echo ""
echo "[Test 5] Verifying performance metrics..."
python3 << 'METRICS_TEST'
import time
from cricket_retriever import CricketRetriever
retriever = CricketRetriever()
# Measure retrieval latency
queries = [
"Fast bowling on bouncy pitch",
"Batting strategy on turning surface",
"Powerplay aggression tactics"
]
latencies = []
for query in queries:
start = time.time()
retriever.retrieve_similar_matches(query, top_k=5)
latencies.append((time.time() - start) * 1000) # Convert to ms
avg_latency = sum(latencies) / len(latencies)
print(f"\nPerformance Metrics:")
print(f" • Average retrieval latency: {avg_latency:.2f} ms")
print(f" • Min latency: {min(latencies):.2f} ms")
print(f" • Max latency: {max(latencies):.2f} ms")
print(f" • Status: ✓ Performance acceptable (< 500ms)" if avg_latency < 500 else " • Status: ⚠️ Performance needs optimization")
METRICS_TEST
# Step 5: Factuality verification
echo ""
echo "[Test 6] Verifying factuality grounding..."
python3 << 'FACTUALITY_TEST'
from cricket_retriever import CricketRetriever
from cricket_rag_pipeline import CricketRAGPipeline
retriever = CricketRetriever()
pipeline = CricketRAGPipeline(retriever)
# Test factuality
queries_with_expected_context = [
("India vs South Africa pace bowling", ["South Africa", "Johannesburg", "pace"]),
("Rohit Sharma Melbourne Test", ["Rohit Sharma", "MCG", "Australia"]),
("Spin bowling strategy", ["Axar Patel", "Sydney", "turning"])
]
print("\nFactuality Grounding Tests:")
print("-" * 60)
for query, expected_terms in queries_with_expected_context:
result = pipeline.generate_analysis(query, top_k=2)
analysis = result['analysis'].lower()
matched_terms = [term for term in expected_terms if term.lower() in analysis]
match_rate = len(matched_terms) / len(expected_terms) * 100
status = "✓" if match_rate >= 50 else "⚠️"
print(f"\n{status} Query: {query[:40]}...")
print(f" Expected terms found: {len(matched_terms)}/{len(expected_terms)} ({match_rate:.0f}%)")
print(f"\n✓ Factuality verification complete")
FACTUALITY_TEST
# Final summary
echo ""
echo "============================================================="
echo "✅ ALL TESTS COMPLETED SUCCESSFULLY"
echo "============================================================="
echo ""
echo "Test Summary:"
echo " [✓] Knowledge base construction"
echo " [✓] Vector database creation"
echo " [✓] Semantic retrieval accuracy"
echo " [✓] RAG pipeline integration"
echo " [✓] Performance metrics"
echo " [✓] Factuality grounding"
echo ""
echo "System ready for production use!"
echo ""
Warning: One of the most common failures in RAG systems is query-context mismatch where the embedding model represents the user query and documents in different semantic spaces, causing relevant matches to be ranked low. This happens when your embedding model was pretrained on general text but your cricket domain uses specialized terminology. Fix this by: (1) Fine-tuning the embedding model on cricket-specific document pairs using contrastive learning, (2) Using domain-specific prompting in queries (e.g., "cricket match context: India vs Australia Test") to guide the embedding, (3) Implementing a hybrid retrieval system that combines semantic search with keyword matching on cricket-specific fields like player names and venues, or (4) Testing retrieval quality on a benchmark set of queries where you know the correct matches should be returned. Always validate that your top-5 retrieved matches make intuitive sense before deploying RAG to production.
Extension Challenge: Implement a feedback loop where the system learns which retrieved matches were actually useful for correct predictions. Add a scoring mechanism that tracks whether generated analyses that cited specific retrieved matches led to accurate predictions. Use this feedback to fine-tune the embedding model or adjust retrieval ranking weights. For example, if matches retrieved for "MCG bouncy pitch" regularly appear in high-accuracy analyses, the system should learn to weight MCG-specific vectors higher. This creates a self-improving RAG system where retrieval quality continuously improves as you gather more match data. Consider implementing A/B testing where some queries use top-5 retrieved matches and others use top-10, measuring which configuration produces more accurate tactical predictions.
- RAG pipelines eliminate hallucination by conditioning generation on retrieved factual context, reducing false cricket statistics or invented player records.
- Vector embeddings enable semantic understanding of cricket queries—models recognize that 'bouncy pitch' and 'pace-friendly surface' are equivalent concepts despite different wording.
- Retrieval latency is critical for real-time applications; vector databases achieve millisecond-level search on large match datasets, compared to seconds with keyword search.
- Prompt engineering in RAG must explicitly instruct models to cite retrieved matches and avoid inventing facts beyond provided context, creating trustworthy domain analysis.
- Embedding model choice shapes retrieval quality; domain-specific models (fine-tuned on cricket matches) significantly outperform general models on specialized terminology.
- RAG systems require continuous validation that retrieved matches are contextually appropriate; automated benchmarking prevents silent degradation of retrieval quality.