What You'll Build
In this project, you will construct a specialized retrieval-augmented generation (RAG) system designed to answer detailed questions about cricket match statistics, player performance, and historical tournament data.
The system implements a multi-stage RAG pipeline that retrieves relevant match records from a vector database, ranks them by relevance, and synthesizes answers using a language model augmented with contextual cricket statistics. It incorporates semantic search over match metadata, relevance re-ranking, and the ability to handle complex queries requiring cross-match analysis.
You will work with embedding models to vectorize cricket match descriptions and implement a retriever capable of distinguishing between different match formats — Test, ODI, and T20 — as well as player-specific performance contexts. The final system will demonstrate how RAG can handle domain-specific knowledge retrieval where the accuracy of retrieved context directly impacts answer quality.
Analogy🏏Cricket
🏏 Think of it like cricket: Imagine you're a cricket analyst preparing for a match commentary. Before the game, you've collected match scorecards, player performance history, pitch reports, and weather data from the stadium—your 'knowledge base.' When a question comes up during live commentary, like 'Has Rohit Sharma ever scored a century here before?', you don't rely on memory alone (which might be fuzzy or incomplete). Instead, you quickly flip through your indexed filing system (the vector database) to retrieve the most relevant match records from that venue, check Rohit's actual performances there, and synthesize your answer with hard facts. The retrieval step is like scanning your organized files to find the right scorecards; the ranking is like prioritizing the most recent or relevant matches; the augmentation is the language model synthesizing a coherent, authoritative answer from those retrieved facts. Without RAG, you'd guess and risk broadcasting incorrect statistics—with RAG, you ground every answer in verified match data, just as professional analysts do with their research notes during live commentary.
Prerequisites
- Proficiency with Python 3.9+ and ability to work with virtual environments and dependency management via pip.
- Understanding of vector embeddings, semantic similarity, and how embeddings capture meaning in high-dimensional space.
- Familiarity with basic LLM concepts including prompting, context windows, and how language models generate responses.
- Knowledge of document retrieval concepts: indexing, similarity search, and ranking metrics like cosine similarity.
- Experience with JSON/dict data structures and ability to parse and manipulate structured metadata from match records.
Setup & Project Structure
The project requires setting up a Python environment with specialized libraries for embeddings, vector storage, and language models. Dependencies include sentence-transformers for embedding generation, chromadb for vector storage and retrieval, and a compatible LLM API wrapper.
Analogy🏏Cricket
🏏 Think of it like cricket: a professional team is not one crowd doing everything — it has a groundstaff preparing the pitch, an analytics unit crunching data, a selection panel picking the eleven, and a captain making the final call on the field. Your RAG project's four layers mirror this division of labour. Just as the groundstaff ingest and prepare the surface, your data layer loads and embeds the cricket match JSON. Just as the analytics unit searches records for patterns, your retrieval layer runs vector similarity to find relevant matches. Just as the selection panel narrows a longlist to the best eleven, your ranking layer reranks candidates for quality. And just as the captain synthesises everything into a match plan, your generation layer prompts the LLM with the retrieved context. The payoff of keeping these roles separate is the same reason clubs don't ask the groundsman to bat at three: each concern can be tested and improved independently without breaking the others.
You will structure the project with separate modules for data loading, retrieval, ranking, and synthesis. Supporting this structure, you will create a data directory to store cricket match records in JSON format, an embeddings cache for pre-computed vectors, and configuration files for model parameters. This modular organization allows you to independently develop and test each RAG component before integration.
bash
#!/bin/bash
# Cricket RAG System Setup
# Create project directory structure
mkdir -p cricket_rag_system
cd cricket_rag_system
mkdir -p data/match_records
mkdir -p embeddings_cache
mkdir -p models
mkdir -p logs
# Create Python virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install required dependencies
pip install --upgrade pip
pip install sentence-transformers==2.2.2
pip install chromadb==0.4.21
pip install pydantic==2.4.2
pip install python-dotenv==1.0.0
pip install requests==2.31.0
# Create initial project structure files
touch config.yaml
touch requirements.txt
touch main.py
echo "Project structure created successfully"
# Verify installation
python -c "import sentence_transformers; import chromadb; print('Dependencies installed successfully')"Step 1 — Foundation
The foundation of the system establishes the core data structures and loads cricket match records into memory. You will create classes to represent cricket match entities with comprehensive metadata, including teams, players, format, venue, date, and full statistics.
This foundational step also implements the data loader, which reads match records from JSON files, validates their structure, and prepares them for embedding generation. Ensuring consistent structure at this stage means all retrieved context will contain the metadata needed for effective ranking and synthesis downstream.
Additionally, you will initialize the embedding model that converts match descriptions and queries into vector representations. This establishes the semantic space in which retrieval will occur, forming the basis for all subsequent similarity-based operations.
Analogy🏏Cricket
🏏 Think of it like cricket: Before a cricket tournament, the organizers must catalog every match—they assign each match a unique identity card that captures essential information: teams, date, venue, winning margin, key player performances, and significant moments. This is like your MatchDocument structure. Then they assign each match a numeric 'fingerprint' (embedding) based on its characteristics—two similar matches (like India vs Pakistan in bilateral ODI at the same venue) get similar fingerprints, while different matches (India vs USA in T20) get distinct fingerprints. Just as scorecards use numerical encoding to store all match data compactly, embeddings compress match semantics into dense vectors. The batch embedding process is like creating a centralized digital archive of all match fingerprints rather than computing them one-by-one, which saves time during tournament season. Without this foundational fingerprinting step, you couldn't quickly find 'all matches where Rohit Sharma scored a century under similar conditions'—you'd have to read every scorecard manually.
python
#!/usr/bin/env python3
# Step 1: Foundation - Data Structures and Loading
from dataclasses import dataclass, field
from typing import List, Dict, Optional
from datetime import datetime
import json
import os
from pathlib import Path
from sentence_transformers import SentenceTransformer
# Define cricket match entity structure
@dataclass
class CricketPlayer:
"""Represents a cricket player with performance metrics."""
name: str
role: str # batsman, bowler, all-rounder, wicketkeeper
team: str
runs_scored: int = 0
wickets_taken: int = 0
economy_rate: float = 0.0
strike_rate: float = 0.0
@dataclass
class CricketMatch:
"""Represents a complete cricket match with comprehensive metadata."""
match_id: str
teams: tuple # (team1, team2)
format: str # Test, ODI, T20
venue: str
date: str
toss_winner: str
toss_decision: str
match_winner: Optional[str]
winning_margin: str
team1_score: str
team2_score: str
player_of_match: str
description: str # Free text summary for embedding
key_moments: List[str] = field(default_factory=list)
notable_performances: Dict[str, str] = field(default_factory=dict)
class CricketDataLoader:
"""Loads and validates cricket match records from JSON files."""
def __init__(self, data_dir: str = "data/match_records"):
self.data_dir = Path(data_dir)
self.matches: List[CricketMatch] = []
self.embedding_model = None
def load_embedding_model(self, model_name: str = "all-MiniLM-L6-v2"):
"""Initialize sentence transformer model for embeddings."""
print(f"Loading embedding model: {model_name}")
self.embedding_model = SentenceTransformer(model_name)
return self.embedding_model
def load_matches_from_json(self, file_path: str) -> List[CricketMatch]:
"""Load match records from JSON file and create CricketMatch objects."""
with open(file_path, 'r', encoding='utf-8') as f:
data = json.load(f)
matches = []
for match_data in data if isinstance(data, list) else [data]:
match = CricketMatch(
match_id=match_data.get('match_id', ''),
teams=(match_data.get('team1', ''), match_data.get('team2', '')),
format=match_data.get('format', 'ODI'),
venue=match_data.get('venue', ''),
date=match_data.get('date', ''),
toss_winner=match_data.get('toss_winner', ''),
toss_decision=match_data.get('toss_decision', ''),
match_winner=match_data.get('match_winner'),
winning_margin=match_data.get('winning_margin', ''),
team1_score=match_data.get('team1_score', ''),
team2_score=match_data.get('team2_score', ''),
player_of_match=match_data.get('player_of_match', ''),
description=match_data.get('description', ''),
key_moments=match_data.get('key_moments', []),
notable_performances=match_data.get('notable_performances', {})
)
matches.append(match)
self.matches.extend(matches)
return matches
def generate_embeddings(self) -> Dict[str, List[float]]:
"""Generate vector embeddings for all match descriptions."""
if self.embedding_model is None:
self.load_embedding_model()
embeddings = {}
descriptions = [match.description for match in self.matches]
print(f"Generating embeddings for {len(descriptions)} matches...")
embedded_vectors = self.embedding_model.encode(descriptions, convert_to_tensor=False)
for match, embedding in zip(self.matches, embedded_vectors):
embeddings[match.match_id] = embedding.tolist()
return embeddings
def validate_matches(self) -> bool:
"""Validate that all matches have required fields."""
for match in self.matches:
if not match.match_id or not match.description:
print(f"Invalid match: {match.match_id} - missing required fields")
return False
print(f"Validated {len(self.matches)} matches successfully")
return True
# Example usage and demonstration
if __name__ == "__main__":
# Initialize data loader
loader = CricketDataLoader(data_dir="data/match_records")
# Load embedding model
loader.load_embedding_model()
# Create sample match records (in production, load from JSON)
sample_matches = [
CricketMatch(
match_id="IND_AUS_2023_001",
teams=("India", "Australia"),
format="ODI",
venue="Melbourne Cricket Ground",
date="2023-11-19",
toss_winner="India",
toss_decision="bat",
match_winner="India",
winning_margin="6 wickets",
team1_score="286/8",
team2_score="282 all out",
player_of_match="Virat Kohli",
description="India defeated Australia in a thrilling ODI at MCG. Rohit Sharma's aggressive batting set the tone, followed by Virat Kohli's composed 85. Australia's chase fell short despite Travis Head's 72. Jasprit Bumrah took 3 crucial wickets in the death overs.",
key_moments=[
"Rohit Sharma hits three consecutive sixes off Mitchell Starc",
"Virat Kohli steadies innings with 85-run partnership",
"Jasprit Bumrah bowls yorkers in death overs"
],
notable_performances={
"Rohit Sharma": "57 off 42 balls",
"Virat Kohli": "85 off 95 balls",
"Jasprit Bumrah": "3/48 in 8 overs"
}
),
CricketMatch(
match_id="IND_ENG_2023_002",
teams=("India", "England"),
format="Test",
venue="Lord's Cricket Ground",
date="2023-09-08",
toss_winner="England",
toss_decision="bat",
match_winner="India",
winning_margin="93 runs",
team1_score="416 and 245",
team2_score="283 and 285",
player_of_match="Ravichandran Ashwin",
description="India wins Test match at Lord's. England's strong start undone by India's disciplined bowling. Ravichandran Ashwin's 12-wicket haul (5/88 and 7/57) proves decisive. Rohit Sharma's 84 in first innings and 50 in second provides crucial runs.",
key_moments=[
"Joe Root and Ben Stokes partnership of 95 in first innings",
"Ravichandran Ashwin takes 7 wickets in second innings",
"Siraj takes crucial late wickets"
],
notable_performances={
"Rohit Sharma": "84 and 50",
"Ravichandran Ashwin": "12 wickets in match",
"Jasprit Bumrah": "5/87 in first innings"
}
)
]
# Add sample matches to loader
loader.matches = sample_matches
# Validate matches
print("\n=== STEP 1: Foundation ===\n")
loader.validate_matches()
# Generate embeddings
embeddings = loader.generate_embeddings()
print(f"\nGenerated {len(embeddings)} embeddings")
# Display sample match structure
print(f"\nSample Match Structure:")
first_match = sample_matches[0]
print(f"Match ID: {first_match.match_id}")
print(f"Teams: {first_match.teams}")
print(f"Format: {first_match.format}")
print(f"Venue: {first_match.venue}")
print(f"Winner: {first_match.match_winner}")
print(f"Player of Match: {first_match.player_of_match}")
print(f"Embedding dimension: {len(embeddings[first_match.match_id])}")Step 2 — Core Logic
Step 2 implements the core retrieval logic by building a vector database and introducing intelligent ranking mechanisms. You will create a retriever that embeds incoming queries into the same vector space as your match records and then performs similarity search to identify the most relevant matches.
This step also includes implementing a reranker that applies more sophisticated metrics — such as BM25 hybrid search or cross-encoder models — to refine the initial retrieval results. The ranking process considers multiple factors: semantic similarity, recency of matches, match format relevance, and whether specific players are mentioned in the query.
To further improve retrieval accuracy, you will implement query preprocessing to handle cricket-specific terminology, including entity recognition for player names and team abbreviations. This ensures that domain-specific language is correctly interpreted before queries are matched against the vector database.
Analogy🏏Cricket
🏏 Think of it like cricket: Consider a cricket statistician preparing for commentary before India plays New Zealand at the Wankhede Stadium. A viewer asks, "How does Virat Kohli perform at home against New Zealand bowlers?" The statistician doesn't just do a simple grep search for 'Virat Kohli' and 'New Zealand'—that would pull up irrelevant matches like Kohli's T20 performance against NZ in Dubai. Instead, they understand the semantic intent: we need home Test matches or ODIs where Kohli faced NZ pace bowling in comparable conditions. They mentally rank candidates: matches at Wankhede itself rank highest, recent performances rank higher than 10-year-old data, and actual head-to-head ODIs rank higher than practice matches. This is exactly what the core logic does: semantic search understands your query's intent (not just keywords), and intelligent ranking orders results by relevance factors (venue, recency, format, player presence). The reranker acts like the statistician's domain knowledge—it refines rough search results into precisely-ranked candidates. Without this step, you might retrieve matches with excellent semantic similarity but low practical relevance.
python
#!/usr/bin/env python3
# Step 2: Core Logic - Retrieval and Ranking
import chromadb
from chromadb.config import Settings
from sentence_transformers import SentenceTransformer
from typing import List, Tuple, Dict
import json
from datetime import datetime
from dataclasses import asdict
class CricketRAGRetriever:
"""Implements semantic search and ranking for cricket matches."""
def __init__(self, embedding_model_name: str = "all-MiniLM-L6-v2"):
self.embedding_model = SentenceTransformer(embedding_model_name)
self.db = chromadb.Client(Settings(
chroma_db_impl="duckdb",
persist_directory="./embeddings_cache",
anonymized_telemetry=False
))
self.collection = None
self.match_metadata = {} # Store metadata separately
def initialize_collection(self, collection_name: str = "cricket_matches"):
"""Initialize ChromaDB collection for match storage."""
# Delete existing collection if it exists
try:
self.db.delete_collection(name=collection_name)
except:
pass
# Create new collection
self.collection = self.db.get_or_create_collection(
name=collection_name,
metadata={"hnsw:space": "cosine"}
)
print(f"Initialized collection: {collection_name}")
def add_matches_to_collection(self, matches: List, embeddings: Dict[str, List[float]]):
"""Add cricket matches with embeddings to the vector database."""
ids = []
documents = []
metadatas = []
vectors = []
for match in matches:
match_id = match.match_id
ids.append(match_id)
documents.append(match.description)
# Create metadata dict
metadata = {
"teams": ",".join(match.teams),
"format": match.format,
"venue": match.venue,
"date": match.date,
"match_winner": match.match_winner or "draw",
"player_of_match": match.player_of_match,
"team1_score": match.team1_score,
"team2_score": match.team2_score
}
metadatas.append(metadata)
vectors.append(embeddings[match_id])
self.match_metadata[match_id] = match
# Add to ChromaDB
self.collection.add(
ids=ids,
documents=documents,
metadatas=metadatas,
embeddings=vectors
)
print(f"Added {len(ids)} matches to collection")
def retrieve_matches(self, query: str, k: int = 5) -> List[Tuple]:
"""Perform semantic search for relevant cricket matches."""
# Embed the query
query_embedding = self.embedding_model.encode(query, convert_to_tensor=False).tolist()
# Search in ChromaDB
results = self.collection.query(
query_embeddings=[query_embedding],
n_results=k,
include=["distances", "documents", "metadatas"]
)
# Extract and format results
retrieved = []
for i in range(len(results['ids'][0])):
match_id = results['ids'][0][i]
distance = results['distances'][0][i]
similarity = 1 - distance # Convert distance to similarity
metadata = results['metadatas'][0][i]
document = results['documents'][0][i]
retrieved.append({
'match_id': match_id,
'similarity': similarity,
'teams': metadata['teams'],
'format': metadata['format'],
'venue': metadata['venue'],
'date': metadata['date'],
'document': document
})
return retrieved
def rank_matches(self, query: str, retrieved: List[Dict]) -> List[Dict]:
"""Apply intelligence ranking to refine retrieval results."""
# Extract query entities
query_lower = query.lower()
for result in retrieved:
score = result['similarity']
# Boost for format relevance
if 'test' in query_lower and result['format'].lower() == 'test':
score += 0.1
elif 'odi' in query_lower and result['format'].lower() == 'odi':
score += 0.1
elif 't20' in query_lower and result['format'].lower() == 't20':
score += 0.1
# Boost for venue relevance
if any(venue in query_lower for venue in result['venue'].lower().split()):
score += 0.08
# Boost for recent matches (more recent = higher score)
try:
match_date = datetime.strptime(result['date'], "%Y-%m-%d")
days_old = (datetime.now() - match_date).days
recency_boost = 0.05 * (1 - min(days_old / 365, 0.5)) # Max 0.025 boost
score += recency_boost
except:
pass
# Boost for player mentions in query
player_boost = 0
match_obj = self.match_metadata.get(result['match_id'])
if match_obj:
for player_name in match_obj.notable_performances.keys():
if player_name.lower() in query_lower:
player_boost += 0.06
score += player_boost
result['final_score'] = min(score, 1.0) # Cap at 1.0
# Sort by final score
retrieved.sort(key=lambda x: x['final_score'], reverse=True)
return retrieved
def search(self, query: str, k: int = 5, use_ranking: bool = True) -> List[Dict]:
"""Complete search pipeline: retrieve and optionally rank."""
print(f"\nSearching for: {query}")
retrieved = self.retrieve_matches(query, k=k*2) # Retrieve more for reranking
if use_ranking:
ranked = self.rank_matches(query, retrieved)
return ranked[:k] # Return top-k after ranking
return retrieved[:k]
# Example usage and demonstration
if __name__ == "__main__":
print("\n=== STEP 2: Core Logic (Retrieval & Ranking) ===\n")
# Initialize retriever
retriever = CricketRAGRetriever()
retriever.initialize_collection()
# Sample embeddings (in production, use real embeddings)
embeddings = {
"IND_AUS_2023_001": [0.1, 0.2, -0.15, 0.08, 0.3, -0.2], # Simplified for demo
"IND_ENG_2023_002": [0.15, 0.25, -0.1, 0.12, 0.35, -0.18]
}
# Create sample matches (same as Step 1)
from step1_foundation import CricketMatch # In practice, import from Step 1
sample_matches = [
type('CricketMatch', (), {
'match_id': 'IND_AUS_2023_001',
'teams': ('India', 'Australia'),
'format': 'ODI',
'venue': 'Melbourne Cricket Ground',
'date': '2023-11-19',
'toss_winner': 'India',
'toss_decision': 'bat',
'match_winner': 'India',
'winning_margin': '6 wickets',
'team1_score': '286/8',
'team2_score': '282 all out',
'player_of_match': 'Virat Kohli',
'description': 'India defeated Australia in a thrilling ODI at MCG. Rohit Sharma aggressive batting set the tone with 57 runs off 42 balls. Virat Kohli composed 85 runs. Australia chase fell short despite Travis Head 72. Jasprit Bumrah took 3 crucial wickets in death overs bowling excellent yorkers.',
'key_moments': ['Rohit hits sixes', 'Kohli steadies', 'Bumrah bowls yorkers'],
'notable_performances': {'Rohit Sharma': '57 off 42', 'Virat Kohli': '85 off 95', 'Jasprit Bumrah': '3/48'}
})(),
type('CricketMatch', (), {
'match_id': 'IND_ENG_2023_002',
'teams': ('India', 'England'),
'format': 'Test',
'venue': "Lord's Cricket Ground",
'date': '2023-09-08',
'toss_winner': 'England',
'toss_decision': 'bat',
'match_winner': 'India',
'winning_margin': '93 runs',
'team1_score': '416 and 245',
'team2_score': '283 and 285',
'player_of_match': 'Ravichandran Ashwin',
'description': 'India wins Test at Lord\'s. England strong start undone by Indian bowling discipline. Ravichandran Ashwin magical 12-wicket haul with 5/88 and 7/57 proves decisive. Rohit Sharma 84 and 50 provides crucial batting anchors.',
'key_moments': ['Root-Stokes partnership', 'Ashwin takes 7 wickets', 'Siraj crucial wickets'],
'notable_performances': {'Rohit Sharma': '84 and 50', 'Ravichandran Ashwin': '12 wickets', 'Jasprit Bumrah': '5/87'}
})()
]
# Add matches to collection
retriever.add_matches_to_collection(sample_matches, embeddings)
# Demonstrate search queries
test_queries = [
"How did Virat Kohli perform against Australia in recent ODI?",
"Ravichandran Ashwin Test match performance at Lord's",
"Jasprit Bumrah wickets in recent international matches"
]
for query in test_queries:
results = retriever.search(query, k=2, use_ranking=True)
print(f"\nTop results for: {query}")
for i, result in enumerate(results, 1):
print(f"{i}. {result['match_id']} (Score: {result['final_score']:.3f})")
print(f" Teams: {result['teams']} | Format: {result['format']} | Venue: {result['venue']}")Step 3 — Integration & Enhancement
Step 3 integrates the retriever with a language model to synthesize answers grounded in the retrieved context. You will implement a context-aware prompt builder that formats retrieved match records into coherent input for the language model, along with citation tracking to attribute answers to specific matches.
This step also introduces conversation history management to support multi-turn queries, where subsequent questions build on previously established context. Filtering mechanisms are added to exclude irrelevant results before synthesis, and answer validation is implemented to detect hallucinations — instances where the model's output contradicts the retrieved context.
Finally, you will add streaming support for real-time response generation and error handling for edge cases, such as scenarios where no relevant matches are found. Together, these features produce a robust, production-aware synthesis layer.
Analogy🏏Cricket
🏏 Think of it like cricket: The retrieval system (Step 2) found relevant match records, but now the commentator must synthesize these raw statistics into a coherent narrative for the broadcast audience. The commentator receives indexed research cards with match data, player statistics, and key moments, then crafts a compelling narrative: "When Virat Kohli faced Mitchell Starc at the MCG in 2023 [retrieved fact], he employed an aggressive approach [synthesis from context], scoring 85 off 95 balls [retrieved statistic], which set up India's successful chase [retrieved outcome]." The commentator carefully attributes each claim to its source—'according to the 2023 MCG match' or 'in that particular series'—so the audience knows where information comes from. If the research suggests Kohli scored 85 but the commentator accidentally says 95, that's a hallucination that must be caught. This synthesis process is what Step 3 does: the language model reads retrieved context and generates fluent, attributable answers rather than relying on generic pre-training knowledge. Understanding this integration reveals why RAG quality depends on both retrieval precision AND language model grounding—a brilliant response that contradicts retrieved facts is worse than a mediocre response that stays faithful to context.
python
#!/usr/bin/env python3
# Step 3: Integration & Enhancement - RAG Synthesis
from typing import List, Dict, Optional
from datetime import datetime
import json
import re
class ContextFormatter:
"""Formats retrieved matches into coherent context for language models."""
@staticmethod
def format_match_context(match_dict: Dict) -> str:
"""Convert a single match result into formatted context."""
context = f"""
Match: {match_dict['match_id']}
Teams: {match_dict['teams']}
Format: {match_dict['format']}
Venue: {match_dict['venue']}
Date: {match_dict['date']}
Result: {match_dict.get('match_winner', 'Unknown')}
Match Summary:
{match_dict['document']}
"""
return context
@staticmethod
def build_system_prompt() -> str:
"""Create the system prompt for grounded answer generation."""
return """You are a knowledgeable cricket expert assistant. Answer questions about cricket matches, player performance, and statistics using the provided match context.
IMPORTANT RULES:
1. ONLY use information from the provided match context
2. If the context doesn't contain relevant information, say 'I don't have this information in the retrieved matches'
3. Always cite which match/date your information comes from
4. If context contradicts your answer, follow the context
5. Do NOT make up statistics or match details
6. Be specific with numbers, dates, and player names
Provided Match Context:
"""
@staticmethod
def build_user_query(question: str, retrieved_context: List[Dict]) -> str:
"""Build complete prompt with question and context."""
prompt = ContextFormatter.build_system_prompt()
# Add retrieved match context
for i, match in enumerate(retrieved_context, 1):
prompt += f"\n--- Retrieved Match {i} ---\n"
prompt += ContextFormatter.format_match_context(match)
# Add user question
prompt += f"\n\nQuestion: {question}\n\nAnswer: "
return prompt
class RAGSynthesizer:
"""Synthesizes answers using retrieved context and language model."""
def __init__(self):
self.conversation_history = []
self.retrieved_matches = {}
def extract_citations(self, answer: str) -> List[str]:
"""Extract match citations from the answer text."""
# Look for patterns like [Match: IND_AUS_2023_001] or match IDs
pattern = r'(?:Match:|match)\s*(?:\[)?([A-Z]+_[A-Z]+_\d{4}_\d{3})'
matches = re.findall(pattern, answer, re.IGNORECASE)
return matches
def validate_answer(self, answer: str, context: List[Dict]) -> Dict:
"""Validate answer against retrieved context for hallucinations."""
validation = {
'is_valid': True,
'warnings': [],
'citations': self.extract_citations(answer)
}
# Check if answer is too generic
generic_phrases = ['it is common knowledge', 'generally speaking', 'most people know']
for phrase in generic_phrases:
if phrase.lower() in answer.lower():
validation['warnings'].append(
"Answer uses generic knowledge instead of retrieved context"
)
validation['is_valid'] = False
# Check if answer cites retrieved matches
if not validation['citations'] and len(answer) > 50:
validation['warnings'].append(
"Answer doesn't cite specific match sources"
)
return validation
def synthesize_answer(
self,
question: str,
retrieved_matches: List[Dict],
simulate_llm: bool = True
) -> Dict:
"""Synthesize answer from question and retrieved context."""
if not retrieved_matches:
return {
'answer': "I don't have relevant match information to answer this question.",
'sources': [],
'confidence': 0.0,
'validation': {'is_valid': True, 'warnings': []}
}
# Build prompt
prompt = ContextFormatter.build_user_query(question, retrieved_matches)
# In production, call real LLM (OpenAI, Anthropic, etc.)
# For demonstration, create a structured answer from context
if simulate_llm:
answer = self._simulate_llm_response(question, retrieved_matches)
else:
# TODO: Implement actual LLM call
# answer = call_llm_api(prompt)
answer = "[LLM API call would go here]"
# Validate answer
validation = self.validate_answer(answer, retrieved_matches)
# Calculate confidence based on context quality
confidence = sum(m.get('final_score', 0) for m in retrieved_matches) / len(retrieved_matches)
return {
'question': question,
'answer': answer,
'sources': [m['match_id'] for m in retrieved_matches],
'confidence': confidence,
'validation': validation,
'retrieved_context': retrieved_matches
}
def _simulate_llm_response(self, question: str, context: List[Dict]) -> str:
"""Simulate LLM response by intelligently combining context."""
if not context:
return "No relevant matches found."
top_match = context[0]
# Extract player name from question
question_lower = question.lower()
player_name = None
# Simple player detection (in production, use NER)
for word in question_lower.split():
if word.capitalize() in ['Rohit', 'Virat', 'Jasprit', 'Ravichandran', 'Travis', 'Joe', 'Ben']:
player_name = word.capitalize() + " " + (question.split(word.capitalize() + " ")[1].split()[0] if "Sharma" in question else "")
break
# Construct answer
answer = f"Based on {top_match['match_id']}"
if 'performance' in question_lower or 'score' in question_lower or 'runs' in question_lower:
answer += f", which was played on {top_match['date']} between {top_match['teams']} at {top_match['venue']}, "
answer += f"the match involved key performances in a {top_match['format']} format. "
answer += f"The retrieved context shows {top_match['document'][:150]}..."
elif 'bowling' in question_lower or 'wickets' in question_lower:
answer += f", the match showed important bowling moments at {top_match['venue']} between {top_match['teams']}. "
answer += f"Key performance details: {top_match['document'][:200]}..."
else:
answer += f" (played on {top_match['date']} at {top_match['venue']}) provides relevant context. "
answer += f"Details: {top_match['document'][:250]}..."
answer += f" [Match: {top_match['match_id']}]"
return answer
def add_conversation_turn(
self,
question: str,
answer: str,
sources: List[str]
):
"""Add a turn to conversation history for context in follow-up questions."""
self.conversation_history.append({
'timestamp': datetime.now().isoformat(),
'question': question,
'answer': answer,
'sources': sources
})
def get_conversation_context(self, max_turns: int = 3) -> str:
"""Get recent conversation history as context for follow-up questions."""
recent_turns = self.conversation_history[-max_turns:]
context = ""
for turn in recent_turns:
context += f"Q: {turn['question']}\nA: {turn['answer']}\n\n"
return context
class CricketRAGPipeline:
"""Complete RAG pipeline integrating retrieval and synthesis."""
def __init__(self, retriever, synthesizer):
self.retriever = retriever
self.synthesizer = synthesizer
def query(
self,
question: str,
k: int = 3,
use_conversation_context: bool = True
) -> Dict:
"""Execute complete RAG pipeline: retrieve, validate, synthesize."""
# Add conversation context to question if available
augmented_question = question
if use_conversation_context:
conv_context = self.synthesizer.get_conversation_context()
if conv_context:
augmented_question = f"{conv_context}\nFollow-up: {question}"
# Step 1: Retrieve relevant matches
retrieved = self.retriever.search(augmented_question, k=k)
# Step 2: Synthesize answer from retrieved context
result = self.synthesizer.synthesize_answer(
question,
retrieved,
simulate_llm=True
)
# Step 3: Add to conversation history
self.synthesizer.add_conversation_turn(
question,
result['answer'],
result['sources']
)
return result
# Example usage and demonstration
if __name__ == "__main__":
print("\n=== STEP 3: Integration & Enhancement (RAG Synthesis) ===\n")
# Initialize synthesizer
synthesizer = RAGSynthesizer()
# Create sample retrieved context
sample_context = [
{
'match_id': 'IND_AUS_2023_001',
'teams': 'India vs Australia',
'format': 'ODI',
'venue': 'Melbourne Cricket Ground',
'date': '2023-11-19',
'match_winner': 'India',
'document': 'India defeated Australia by 6 wickets. Rohit Sharma scored 57 off 42 balls with aggressive batting. Virat Kohli scored 85 off 95 balls providing stability. Jasprit Bumrah took 3 wickets (3/48) including crucial death overs.',
'final_score': 0.92
},
{
'match_id': 'IND_AUS_2022_002',
'teams': 'India vs Australia',
'format': 'ODI',
'venue': 'Sydney Cricket Ground',
'date': '2022-12-02',
'match_winner': 'Australia',
'document': 'Australia defeated India by 4 runs in a tight contest. Virat Kohli scored 63 off 75 balls. Jasprit Bumrah took 2 wickets. David Warner scored crucial 67 for Australia.',
'final_score': 0.78
}
]
# Test synthesis
test_questions = [
"How did Virat Kohli perform against Australia in ODIs?",
"What was Jasprit Bumrah's role in India vs Australia matches?"
]
for question in test_questions:
print(f"\nQuestion: {question}")
print("-" * 70)
result = synthesizer.synthesize_answer(
question,
sample_context,
simulate_llm=True
)
print(f"Answer: {result['answer']}")
print(f"\nConfidence: {result['confidence']:.2%}")
print(f"Sources: {', '.join(result['sources'])}")
print(f"Validation: {result['validation']['is_valid']}")
if result['validation']['warnings']:
print(f"Warnings: {'; '.join(result['validation']['warnings'])}")
# Demonstrate conversation history
print("\n\nConversation History:")
print("-" * 70)
for i, turn in enumerate(synthesizer.conversation_history, 1):
print(f"Turn {i}: Q: {turn['question'][:60]}...")
print(f" Sources: {turn['sources']}")Step 4 — Testing & Verification
Testing and verification ensure that the RAG system retrieves relevant matches, generates accurate answers grounded in context, and handles edge cases gracefully. You will run integration tests that validate the complete pipeline end-to-end and execute query sets against your data to verify retrieval quality and factual accuracy.
Analogy🏏Cricket
🏏 Think of it like cricket: before a plan is trusted, a team plays a full warm-up fixture that runs every phase together — openers see the new ball, bowlers find rhythm, fielders hold catches — to expose faults while they are cheap to fix. Your Step 4 is that fixture. Just as a practice game exercises the whole team in sequence rather than in isolated nets, the test harness runs ingestion, retrieval, ranking, and generation end to end on sample cricket queries. Just as coaches log how each phase performed to spot the weak link, the harness logs intermediate results at every stage so you can see exactly where behaviour drifts. Passing individual nets never guarantees a winning eleven — only the full fixture reveals how the parts combine. The payoff is match-ready confidence: a pipeline proven to move a query cleanly from raw match data all the way to a grounded answer before it faces real users.
Testing should cover diverse query types, including player-specific questions, match format comparisons, venue-specific questions, and multi-entity queries. In addition, you should verify that the system correctly rejects queries without relevant context, that citations accurately map to retrieved sources, and that conversation history properly supports follow-up questions.
bash
#!/bin/bash
# Step 4: Testing and Verification
echo "=== STEP 4: Testing & Verification ==="
echo ""
# Activate virtual environment
source venv/bin/activate
echo "[TEST 1] Running unit tests for data loading..."
python3 -c "
import json
from pathlib import Path
# Test data loader
test_matches = [
{
'match_id': 'TEST_IND_PAK_2023',
'team1': 'India',
'team2': 'Pakistan',
'format': 'ODI',
'venue': 'Arun Jaitley Stadium',
'date': '2023-10-14',
'toss_winner': 'Pakistan',
'toss_decision': 'bat',
'team1_score': '356/9',
'team2_score': '345 all out',
'match_winner': 'India',
'winning_margin': '8 runs',
'player_of_match': 'Hardik Pandya',
'description': 'India defeated Pakistan in a thrilling ODI. Rohit Sharma scored 62. Hardik Pandya took 3/45 in the final overs.',
'key_moments': ['Hardik bowling spell', 'Siraj defending boundaries'],
'notable_performances': {'Rohit Sharma': '62 off 68', 'Hardik Pandya': '3/45 in 8 overs'}
},
{
'match_id': 'TEST_IND_NZ_2023',
'team1': 'India',
'team2': 'New Zealand',
'format': 'Test',
'venue': 'Wankhede Stadium',
'date': '2023-11-25',
'toss_winner': 'India',
'toss_decision': 'bat',
'team1_score': '263 and 170/4',
'team2_score': '280 all out',
'match_winner': 'India',
'winning_margin': 'Innings and 13 runs',
'player_of_match': 'Mohammed Siraj',
'description': 'India beats NZ in Test cricket at home. Siraj took 7 wickets across two innings. Excellent bowling performance in overcast conditions.',
'key_moments': ['Siraj bowling spell', 'India chase runs'],
'notable_performances': {'Mohammed Siraj': '7 wickets', 'Ravichandran Ashwin': '4 wickets'}
}
]
# Validate structure
for match in test_matches:
assert 'match_id' in match, 'Missing match_id'
assert 'description' in match, 'Missing description'
assert 'format' in match, f'Missing format in {match[\"match_id\"]}'
print(f'✓ Validated {match[\"match_id\"]}')
print(f'✓ [PASS] All {len(test_matches)} test matches validated')
echo ''
"
echo "[TEST 2] Testing embedding generation..."
python3 -c "
from sentence_transformers import SentenceTransformer
import numpy as np
print('Loading embedding model...')
model = SentenceTransformer('all-MiniLM-L6-v2')
# Test sentences
sentences = [
'India defeated Australia by 6 wickets in ODI cricket.',
'Virat Kohli scored 85 runs off 95 balls at MCG.',
'Jasprit Bumrah took 3 wickets for 48 runs.',
]
print(f'Encoding {len(sentences)} test sentences...')
embeddings = model.encode(sentences, convert_to_tensor=False)
# Verify embeddings
for i, (sent, emb) in enumerate(zip(sentences, embeddings)):
print(f' Sentence {i+1}: {len(emb)} dimensions')
assert len(emb) == 384, f'Expected 384 dims, got {len(emb)}'
print(f'✓ [PASS] Embeddings generated successfully')
echo ''
"
echo "[TEST 3] Testing semantic similarity..."
python3 -c "
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
import numpy as np
model = SentenceTransformer('all-MiniLM-L6-v2')
queries = [
'How did Virat Kohli perform against Australia?',
'What was Kohli score at MCG?',
]
contexts = [
'Virat Kohli scored 85 off 95 balls at MCG against Australia in ODI cricket.',
'Jasprit Bumrah took 3 crucial wickets in death overs bowling excellent yorkers.',
'Rohit Sharma aggressive batting scored 57 off 42 balls setting up Indias chase.',
]
print('Computing semantic similarities...')
for query in queries:
query_emb = model.encode([query])
context_embs = model.encode(contexts)
similarities = cosine_similarity(query_emb, context_embs)[0]
best_idx = np.argmax(similarities)
print(f'Query: {query[:40]}...')
print(f'Best match (idx {best_idx}): {contexts[best_idx][:60]}... (score: {similarities[best_idx]:.3f})')
print(f'✓ [PASS] Semantic similarity working correctly')
echo ''
"
echo "[TEST 4] Testing complete RAG pipeline..."
python3 << 'PYTEST'
import json
from typing import List, Dict
# Mock classes for testing
class MockRetriever:
def search(self, query: str, k: int = 3):
return [
{'match_id': 'IND_AUS_2023_001', 'teams': 'India vs Australia',
'format': 'ODI', 'venue': 'MCG', 'date': '2023-11-19',
'document': 'India defeated Australia by 6 wickets. Kohli 85, Bumrah 3/48.',
'final_score': 0.92},
{'match_id': 'IND_AUS_2022_002', 'teams': 'India vs Australia',
'format': 'ODI', 'venue': 'SCG', 'date': '2022-12-02',
'document': 'Australia defeated India by 4 runs. Kohli 63.',
'final_score': 0.78}
]
class MockSynthesizer:
def synthesize_answer(self, question: str, context: List[Dict], simulate_llm: bool = True):
return {
'question': question,
'answer': f'Based on retrieved matches, {context[0]["match_id"]}: {context[0]["document"]}',
'sources': [m['match_id'] for m in context],
'confidence': 0.85,
'validation': {'is_valid': True, 'warnings': []}
}
# Test pipeline
retriever = MockRetriever()
synthesizer = MockSynthesizer()
test_queries = [
'How did Virat Kohli perform against Australia?',
'What was Jasprit Bumrah bowling performance?',
'India vs Australia recent matches',
]
print('Testing RAG pipeline with sample queries...')
for query in test_queries:
retrieved = retriever.search(query, k=2)
result = synthesizer.synthesize_answer(query, retrieved)
print(f'\nQuery: {query}')
print(f' Retrieved: {len(result["sources"])} matches')
print(f' Confidence: {result["confidence"]:.1%}')
print(f' Answer length: {len(result["answer"])} characters')
assert len(result['sources']) > 0, 'No sources returned'
assert result['confidence'] > 0, 'Invalid confidence score'
assert len(result['answer']) > 20, 'Answer too short'
print(f'\n✓ [PASS] RAG pipeline test completed successfully')
PYTEST
echo ""
echo "[TEST 5] Testing with cricket-specific queries..."
python3 << 'CRICKET_TEST'
test_cases = [
{
'query': 'Rohit Sharma ODI performance at home',
'expected_keywords': ['Rohit', 'Sharma', 'ODI'],
},
{
'query': 'Test match bowling analysis Jasprit Bumrah',
'expected_keywords': ['Test', 'bowling', 'Bumrah'],
},
{
'query': 'T20 cricket Virat Kohli strike rate',
'expected_keywords': ['T20', 'Kohli', 'strike'],
},
]
print('Validating cricket-specific query handling...')
for test_case in test_cases:
query = test_case['query']
keywords = test_case['expected_keywords']
# Verify keywords are in query
missing_keywords = [kw for kw in keywords if kw.lower() not in query.lower()]
if not missing_keywords:
print(f'✓ Query structure valid: {query[:50]}...')
else:
print(f'✗ Missing keywords: {missing_keywords}')
print(f'✓ [PASS] Cricket-specific query validation complete')
CRICKET_TEST
echo ""
echo "=== EXPECTED OUTPUT SUMMARY ==="
echo "✓ Unit tests for data loading: PASS"
echo "✓ Embedding generation test: PASS"
echo "✓ Semantic similarity test: PASS"
echo "✓ RAG pipeline test: PASS"
echo "✓ Cricket-specific queries: PASS"
echo ""
echo "All tests completed successfully!"
echo "The RAG system is ready for deployment."Warning: The most common error is embedding dimension mismatch between your query encoder and stored embeddings. If you change embedding models between indexing and query time, your similarity scores will be meaningless because the vector spaces don't align. ALWAYS ensure you use the same embedding model throughout the pipeline. Additionally, watch for incomplete match metadata—if your JSON records don't contain critical fields like 'date' or 'venue', the reranking logic will silently fail or crash. Validate your input data structure immediately after loading. Another frequent mistake is overshooting the retrieval window: if you retrieve k=10 matches but your context window is only 2000 tokens, you may only be using the first 2-3 matches effectively in synthesis, making the additional retrievals wasted computation.
Extension Challenge: Enhance your RAG system with the following advanced features: (1) Implement cross-encoder reranking by loading a specialized model like 'cross-encoder/ms-marco-MiniLM-L-12-v2' to re-score your top-k results before synthesis—this typically improves answer quality by 15-25%. (2) Add multi-hop retrieval where follow-up questions automatically retrieve matches related to previous answers (e.g., 'Tell me more about that player's other performances'). (3) Implement a fact verification module that checks if synthesized claims appear in the retrieved context, and returns a confidence flag if the model makes statements unsupported by context. (4) Create a query expansion module using an LLM to generate alternative phrasings of the user's question, retrieve results for all variants, and ensemble the rankings—this helps catch matches that use different terminology. (5) Add temporal analysis: track how player performance trends over time by automatically grouping retrieved matches chronologically and identifying patterns.
- RAG combines semantic retrieval with language model synthesis: the retriever finds relevant context from a knowledge base, then the generator produces answers grounded in that context rather than relying solely on pre-training.
- Embedding quality directly impacts retrieval effectiveness—use domain-appropriate embedding models and ensure consistency between query and document encoding for meaningful similarity scores.
- Intelligent ranking beyond similarity search is critical: boost results by recency, format relevance, player presence, and venue to order candidates by practical importance not just semantic similarity.
- Context grounding prevents hallucination: validating that synthesized answers stay faithful to retrieved facts and citing specific sources ensures trustworthy, attributable responses.
- Conversation history enables multi-turn understanding: storing previous Q&A turns allows follow-up questions to build on prior context, making the system conversational rather than stateless.
- Edge case handling is essential for production: implement fallbacks for zero-result queries, validate metadata consistency before storage, and test with adversarial queries designed to break your assumptions.