What is index lifecycle management (ILM) in Elasticsearch?
Learn how Elasticsearch ILM automates hot, warm, cold, frozen, and delete phases with rollover to manage time-series data, cut cost, and enforce retention.
Expected Interview Answer
Index lifecycle management (ILM) is an Elasticsearch feature that automatically moves an index through defined phases — hot, warm, cold, frozen, and delete — based on age, size, or document count, so time-series data is managed without manual intervention.
You attach an ILM policy to an index template so new indices inherit it. Each phase can trigger actions such as rollover, shrink, force merge, migrate to cheaper hardware tiers, set replicas, or delete. ILM works hand-in-hand with rollover, which starts a fresh index once the current one crosses a threshold, keeping shard sizes healthy and letting old data age out cheaply.
- Automates rollover, retention, and deletion of time-series indices
- Moves aging data to cheaper hardware tiers to cut cost
- Keeps shard sizes optimal, avoiding oversized or tiny shards
- Reduces manual operational toil and human error
- Enforces consistent retention policies across many indices
AI Mentor Explanation
Think of managing a cricket club's match footage. This week's game sits on the pavilion's fast SSD for instant replay, last month's moves to a slower shared drive, last season's is boxed in cold storage, and footage older than five years is discarded. ILM automates exactly that aging: fresh indices stay on hot nodes, older ones migrate to warm and cold tiers, and stale data is deleted on schedule.
Step-by-Step Explanation
Step 1
Define an ILM policy
Create a policy specifying the hot, warm, cold, frozen, and delete phases and the actions in each.
Step 2
Set phase triggers
Choose transition conditions such as index age, primary shard size, or document count.
Step 3
Attach via index template
Reference the policy and a rollover alias in an index template so new indices adopt it automatically.
Step 4
Enable rollover
Let the hot phase roll over to a fresh index when it crosses a size or age threshold to keep shards healthy.
Step 5
Add tier and cleanup actions
Configure migrate, shrink, force merge, and finally delete so aging data moves to cheaper hardware and old data is removed.
What Interviewer Expects
- Knowledge of the hot, warm, cold, frozen, and delete phases
- Understanding of rollover and why it keeps shard sizes healthy
- How policies attach through index templates and aliases
- Awareness of data tiers and cost optimization
- Realistic use cases like logs and metrics time-series data
Common Mistakes
- Confusing ILM with snapshots or backups
- Forgetting to set a rollover alias in the index template
- Applying ILM to non-time-series indices where it adds little value
- Setting phase thresholds that create oversized or tiny shards
- Assuming delete phase runs instantly rather than on the poll interval
Best Answer (HR Friendly)
“Index lifecycle management lets Elasticsearch automatically age data through stages — from fast storage when it is new to cheaper storage as it gets older, and finally deleting it. It saves money and manual work, which is why it is used heavily for logs and other time-based data.”
Code Example
PUT _ilm/policy/logs-policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover": { "max_primary_shard_size": "50gb", "max_age": "7d" }
}
},
"delete": {
"min_age": "30d",
"actions": { "delete": {} }
}
}
}
}Follow-up Questions
- How does the rollover action decide when to create a new index?
- What is the difference between the cold and frozen phases?
- How do data tiers map to node roles in a cluster?
- What happens if an ILM action fails midway through a phase?
- How would you migrate an existing index onto an ILM policy?
MCQ Practice
1. Which action typically runs in the hot phase to keep shard sizes healthy?
Rollover starts a fresh index once thresholds like size or age are crossed, preventing indices from growing indefinitely.
2. What is the primary goal of moving indices to the cold or frozen tier?
Cold and frozen tiers use cheaper hardware for infrequently queried data, reducing cost while keeping it searchable.
3. How is an ILM policy usually applied to new indices?
An index template ties the policy and rollover alias to new indices so they inherit lifecycle management automatically.
Flash Cards
What are the ILM phases? — Hot, warm, cold, frozen, and delete — indices progress through them based on age, size, or doc count.
What does rollover do? — Creates a fresh write index once the current one crosses a size or age threshold, keeping shards healthy.
How is ILM attached to indices? — Via an index template that references the policy and a rollover alias, so new indices inherit it.
Why move data to cold/frozen tiers? — To store rarely accessed data on cheaper hardware while keeping it searchable, cutting cost.