What Index Lifecycle Management Solves
Time-series data — logs, metrics, traces, events — is typically written once into a fresh index and then queried heavily while recent, then queried less and less as it ages, until it's only kept for compliance or rare historical lookups before eventual deletion. Index Lifecycle Management (ILM) automates the operational work this pattern demands: rolling over to a new index once the current one hits a size, document count, or age threshold; migrating older indices onto cheaper, less powerful hardware tiers; shrinking or force-merging indices to reduce their resource footprint once they're no longer being written to; and finally deleting indices once they pass a retention deadline — all without a human running manual curator scripts on a cron job.
Cricket analogy: A cricket board automatically rotating this season's match footage from primary broadcast servers to archive storage, and eventually purging decades-old low-demand footage per a retention policy, mirrors ILM's phase automation.
The Four ILM Phases
An ILM policy defines up to four phases — hot, warm, cold, and delete (frozen is also available for very old, rarely-queried searchable-snapshot data) — each with configurable actions and a min_age that controls when the index transitions into it. The hot phase almost always includes a rollover action (new-index-generation, e.g., rolling over at 50GB or 30 days, whichever comes first); warm typically includes shrink (reduce shard count now that the index is read-only), force merge (collapse Lucene segments to reduce overhead), and set_priority (deprioritize recovery); cold often includes searchable snapshots or further downsampling; and delete simply removes the index once it's outlived its usefulness. Each phase's min_age is measured from the index's rollover date, not its creation date, so a policy like hot for 7 days, warm for 30 days, delete after 90 days is straightforward to reason about.
Cricket analogy: A stadium's scoreboard-graphics package moving through prep, live broadcast, highlights-reel, and finally archive stages, each with its own timing rule, mirrors ILM's hot, warm, cold, and delete phases.
PUT _ilm/policy/logs-30d-retention
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": { "max_primary_shard_size": "50gb", "max_age": "7d" },
"set_priority": { "priority": 100 }
}
},
"warm": {
"min_age": "7d",
"actions": {
"shrink": { "number_of_shards": 1 },
"forcemerge": { "max_num_segments": 1 },
"set_priority": { "priority": 50 }
}
},
"cold": {
"min_age": "14d",
"actions": {
"set_priority": { "priority": 0 }
}
},
"delete": {
"min_age": "30d",
"actions": { "delete": {} }
}
}
}
}Index Templates and Rollover Aliases
ILM works hand-in-hand with index templates and a write alias. You create an index template that applies the ILM policy and an initial index (e.g., logs-app-000001) to any matching new index, and you point a write alias (e.g., logs-app) at that first backing index; applications always index into the alias, never a literal index name. When the rollover action fires, ILM creates the next generation index (logs-app-000002), swaps the write alias to point there, and leaves the old index searchable but no longer receiving writes — the application code never has to change, and searches against the alias transparently span all the generations that still exist.
Cricket analogy: A ground always labeled 'Home Ground' regardless of which physical stadium currently hosts matches after a stadium swap, so fans always know where to go, mirrors an alias always pointing at the current write index.
Rollover triggers on whichever condition is met first among max_primary_shard_size, max_age, max_docs, or max_size — set these based on your actual ingest rate so shards land in the recommended size range rather than rolling over too eagerly or too rarely.
Force-merging an index that is still receiving writes is expensive and can degrade cluster performance; ILM only applies forcemerge in the warm phase after rollover has already made the index read-only. Never manually force-merge a live, actively-indexed index.
- ILM automates rollover, tier migration, shrink/force-merge, and deletion for time-series indices.
- Policies define hot, warm, cold, delete (and optionally frozen) phases, each with a min_age.
- min_age is measured from the rollover date of the index, not its creation date.
- ILM pairs with index templates and a write alias so applications never reference literal index names.
- Rollover fires on the first-met condition among size, age, or document count thresholds.
- Warm-phase actions like shrink and forcemerge should only run after rollover makes the index read-only.
- GET _ilm/explain shows exactly which phase, action, and step an index currently sits in.
Practice what you learned
1. What is the primary purpose of the rollover action in the hot phase?
2. From when is a phase's min_age typically measured for an index using rollover?
3. Why should forcemerge only be applied in the warm phase, not on a live index?
4. What role does a write alias play in an ILM-managed index pattern?
5. Which command shows the current ILM phase, action, and step an index is in?
Was this page helpful?
You May Also Like
Shards and Replicas
How Elasticsearch splits indices into shards for scalability and duplicates them as replicas for resilience and read throughput.
Elasticsearch Cluster Architecture
How Elasticsearch nodes join together into a cluster, elect a master, and coordinate to store and serve data reliably.
Elasticsearch Performance Tuning
Practical techniques for tuning indexing throughput, query latency, and resource usage in Elasticsearch.
Monitoring Elasticsearch
Which metrics, APIs, and alerting practices keep an Elasticsearch cluster healthy and catch problems before they cause outages.