How does relevance scoring work in Elasticsearch with BM25?
Learn how Elasticsearch ranks results with BM25: term frequency saturation, inverse document frequency, length normalization, and tuning k1 and b.
Expected Interview Answer
Elasticsearch ranks matching documents with BM25, a scoring function that rewards documents containing more of the search terms (term frequency), weights rarer terms higher (inverse document frequency), and normalizes for document length so long documents do not win by sheer size.
BM25 combines three signals. Term frequency (TF) increases the score as a term appears more often, but with saturation controlled by the k1 parameter so the tenth occurrence adds far less than the first. Inverse document frequency (IDF) gives rarer terms across the index more weight, since a match on an uncommon word is more meaningful than a match on a common one. Field-length normalization, tuned by the b parameter, discounts matches in unusually long fields so a term buried in a huge document scores lower than the same term in a short, focused one. Elasticsearch computes this per matching term and sums the contributions into the final _score.
- Ranks the most relevant documents first
- Rare, discriminating terms carry more weight via IDF
- TF saturation prevents keyword stuffing from dominating
- Length normalization keeps long documents from unfairly winning
- Tunable via k1 and b for domain-specific relevance
AI Mentor Explanation
BM25 is like ranking batters not just by total runs but by context. Scoring more against a bowler counts (term frequency), yet the first big innings matters more than the twentieth thanks to saturation. Runs against a rare, feared bowler weigh heavier than runs against a part-timer (inverse document frequency), and a knock across ninety overs is discounted versus the same runs in a tight chase (length normalization).
Step-by-Step Explanation
Step 1
Match documents
Find all documents containing at least one of the query terms via the inverted index.
Step 2
Compute term frequency
Count each term's occurrences per document, applying k1 saturation so repeats add less.
Step 3
Apply inverse document frequency
Weight each term by how rare it is across the index — rarer terms score higher.
Step 4
Normalize for length
Use the b parameter to discount matches in fields longer than the average length.
Step 5
Sum per-term contributions
Add each term's weighted score into the document's final _score and sort descending.
What Interviewer Expects
- Naming term frequency, inverse document frequency, and length normalization
- Understanding TF saturation and the k1 parameter
- Understanding the b parameter for length normalization
- Knowing BM25 is the default replacing classic TF-IDF
- Awareness of how to inspect scores with the explain API
Common Mistakes
- Confusing BM25 with plain TF-IDF and ignoring saturation
- Thinking more term repetitions always linearly increase score
- Ignoring field-length normalization when debugging rankings
- Assuming scores are comparable across different queries or indices
- Forgetting IDF makes rare terms more influential
Best Answer (HR Friendly)
“Elasticsearch decides which results to show first using a formula called BM25. It rewards documents that contain the search words more often, gives extra weight to unusual words, and avoids letting very long documents win just because they are big, so the most genuinely relevant results appear at the top.”
Code Example
GET /articles/_search
{
"explain": true,
"query": {
"match": { "body": "elasticsearch relevance" }
}
}PUT /articles
{
"settings": {
"index": {
"similarity": {
"custom_bm25": {
"type": "BM25",
"k1": 1.3,
"b": 0.75
}
}
}
},
"mappings": {
"properties": {
"body": { "type": "text", "similarity": "custom_bm25" }
}
}
}Follow-up Questions
- What do the k1 and b parameters control in BM25?
- How does BM25 differ from classic TF-IDF?
- How do you debug why a document scored the way it did?
- Why are _score values not comparable across different queries?
- How does IDF change when a term becomes more common in the index?
MCQ Practice
1. Which BM25 component gives rarer terms more weight?
Inverse document frequency (IDF) raises the weight of terms that appear in fewer documents across the index.
2. What does the k1 parameter primarily control in BM25?
k1 controls term-frequency saturation — how much additional occurrences of a term keep contributing to the score.
3. Why does length normalization exist in BM25?
The b parameter discounts matches in longer-than-average fields so length alone does not inflate relevance.
Flash Cards
What are BM25's three core signals? — Term frequency (with saturation), inverse document frequency, and field-length normalization.
What does k1 do? — Controls how quickly term-frequency contribution saturates as a term repeats.
What does b do? — Controls the strength of field-length normalization (0 = none, 1 = full).
How do you inspect a score? — Add "explain": true to the search or use the _explain API to see per-term contributions.
Continue Learning
Related Interview Questions
What is the difference between filter context and query context in Elasticsearch?
medium
Elasticsearch vs Built-In Database Full-Text Search: When to Use Which?
hard
What is a document, index, and shard in Elasticsearch?
easy
What is the difference between keyword and text field types in Elasticsearch?
medium