How do you improve indexing and search performance in Elasticsearch?
Practical Elasticsearch performance tuning: bulk indexing, refresh and replica strategy, filter-context caching, shard sizing, and efficient pagination.
Expected Interview Answer
You improve Elasticsearch performance by tuning both write and read paths: batch writes with the bulk API, right-size shards and mappings, use the filter context and caches for reads, and scale replicas and hardware to match the workload.
On the indexing side, use bulk requests, increase the refresh interval for heavy ingestion, disable replicas during initial loads, and avoid unnecessary fields or dynamic mapping. On the search side, prefer filters over queries so results are cacheable, limit returned fields with source filtering, avoid deep pagination in favor of search_after, and keep shard counts sensible. Monitoring with the profile API, slow logs, and hot threads guides where to tune.
- Higher indexing throughput via bulk requests and tuned refresh
- Faster, cacheable searches by using the filter context
- Balanced shard sizing that avoids oversized or excessive shards
- Lower resource usage through lean mappings and source filtering
- Predictable scaling by matching replicas and hardware to load
AI Mentor Explanation
Improving Elasticsearch is like coaching a cricket side for both innings. In the field (indexing) you drill efficient batches — collecting several balls' data at once rather than logging each ball separately — and rest bowlers during heavy sessions. When batting (searching) you play cacheable, repeatable shots by reusing scouting notes. Bulk writes, tuned refresh, filters, and caches are those combined batting-and-bowling optimizations.
Step-by-Step Explanation
Step 1
Batch your writes
Use the bulk API to send many documents per request instead of one document per call.
Step 2
Tune refresh and replicas for ingest
Raise the refresh interval and drop replicas to zero during heavy initial loads, then restore them.
Step 3
Design lean mappings
Disable dynamic mapping where unneeded, index only fields you search, and use keyword vs text deliberately.
Step 4
Use the filter context for reads
Put non-scoring conditions in filters so results are cached and reused across queries.
Step 5
Right-size shards and paginate wisely
Keep shards in a healthy size range and use search_after instead of deep from/size pagination.
Step 6
Measure and iterate
Use the profile API, slow logs, and hot threads to find bottlenecks before changing settings.
What Interviewer Expects
- Separating write-path from read-path optimizations
- Knowledge of bulk API, refresh interval, and replica strategy
- Understanding filter context versus query context and caching
- Awareness of shard sizing and pagination pitfalls
- Use of profiling and monitoring tools to guide tuning
Common Mistakes
- Indexing documents one at a time instead of using bulk
- Creating far too many shards, exhausting cluster overhead
- Using query context for filters, losing cache benefits
- Relying on deep from/size pagination that scans huge result sets
- Tuning settings blindly without profiling or slow-log evidence
Best Answer (HR Friendly)
“You speed up Elasticsearch by writing data in batches, keeping the data structure lean, and making searches reusable so results can be cached. You also size the storage units sensibly and monitor performance so you fix the real bottleneck rather than guessing.”
Code Example
PUT logs/_settings
{
"index": { "refresh_interval": "30s", "number_of_replicas": 0 }
}
POST _bulk
{ "index": { "_index": "logs" } }
{ "level": "info", "msg": "started" }
{ "index": { "_index": "logs" } }
{ "level": "warn", "msg": "retrying" }GET logs/_search
{
"query": {
"bool": {
"filter": [
{ "term": { "level": "error" } },
{ "range": { "@timestamp": { "gte": "now-1d" } } }
]
}
}
}Follow-up Questions
- Why are filters cacheable while scoring queries are not?
- How do you choose the number of primary shards for an index?
- What is search_after and why is it better than deep pagination?
- How does the refresh interval trade off latency against throughput?
- How would you use the profile API to diagnose a slow query?
MCQ Practice
1. Which technique most improves bulk indexing throughput during a large initial load?
A longer refresh interval and zero replicas reduce write overhead during heavy ingestion; replicas are restored afterward.
2. Why prefer the filter context over the query context for exact-match conditions?
Filters do not score documents, so Elasticsearch can cache and reuse their results, making repeated searches faster.
3. What is the recommended alternative to deep from/size pagination?
search_after paginates efficiently using the sort values of the last hit, avoiding the cost of scanning large offsets.
Flash Cards
Fastest way to index many docs? — The bulk API — send many documents per request instead of one call per document.
Filter vs query context? — Filters are non-scoring and cacheable; queries compute relevance scores and are not cached.
Ingest-time replica tip? — Set replicas to zero during a large initial load, then restore them once ingestion completes.
Better than deep pagination? — search_after uses the last hit's sort values to page efficiently without scanning huge offsets.
Continue Learning
Related Interview Questions
What is the difference between a primary shard and a replica shard in Elasticsearch?
medium
How does Elasticsearch achieve near real-time search?
hard
What is the query then fetch process in Elasticsearch search?
medium
How do you handle pagination in Elasticsearch and why is deep pagination a problem?
medium