Why Nested Documents Need Special Aggregations
When a field is mapped as type nested, Elasticsearch indexes each element of the array as a separate, hidden Lucene document internally linked to its parent, precisely so that the relationships between fields inside each object, like a single review's rating and author staying together, aren't lost the way they would be if the array were flattened into parallel arrays of values. Because of this internal split, ordinary aggregations on a nested-object subfield don't see those hidden documents at all; you need the nested aggregation to explicitly step into that hidden index space before you can aggregate on it.
Cricket analogy: It's like keeping each ball of an over as its own record, so a wide off a specific bowler stays linked to that bowler, rather than flattening the over into separate 'bowlers' and 'outcomes' lists that lose the pairing.
The nested Aggregation
The nested aggregation re-scopes the aggregation context into the hidden nested documents identified by its path parameter, and any sub-aggregations placed inside it, terms, avg, stats, or further nested aggregations for doubly-nested arrays, operate against those nested documents rather than the parent. This is how you'd compute, say, the average rating across every comment stored in a nested comments field, or find the most common tag used inside nested review objects, correctly treating each nested object as its own aggregatable unit.
Cricket analogy: It's like stepping into ball-by-ball data to compute the average runs conceded per delivery a specific bowler bowled, rather than averaging across whole overs, the way granular Hawk-Eye analytics work.
PUT /articles
{
"mappings": {
"properties": {
"title": { "type": "text" },
"comments": {
"type": "nested",
"properties": {
"author": { "type": "keyword" },
"rating": { "type": "integer" }
}
}
}
}
}
GET /articles/_search
{
"size": 0,
"aggs": {
"comment_stats": {
"nested": { "path": "comments" },
"aggs": {
"avg_rating": { "avg": { "field": "comments.rating" } }
}
}
}
}Returning to Parent Context: reverse_nested
Once you're inside a nested aggregation context, you sometimes need to reference a field that lives on the parent root document, for example grouping by the article's author (a root-level field) while still aggregating over nested comment ratings; the reverse_nested aggregation joins back out from the nested scope to its parent, letting a sub-aggregation tree cross between nested and root fields within one query. A common pattern is nested comments to bucket by comments.tag, then reverse_nested back to the root to bucket by the article's category, effectively answering 'for each comment tag, which root article categories does it appear in most.'
Cricket analogy: It's like drilling into ball-by-ball data to find which bowlers concede boundaries, then joining back out to the match level to see which venues those matches were played at, connecting delivery detail to match context.
A useful combined pattern is: nested aggregation into comments, a terms aggregation on comments.tag, then a reverse_nested aggregation with a further terms on the root category field. This produces, for each comment tag, a breakdown of which article categories it appears under most often, joining nested-child statistics back to root-level groupings in a single request.
Performance and Mapping Considerations
Every element in a nested array becomes its own Lucene document internally, so a document with 50 nested comments actually contributes 51 Lucene documents to the index, which multiplies storage and can strain the index.mapping.nested_objects.limit setting (10,000 by default) on documents with very large nested arrays. For high-cardinality or frequently-updated nested arrays, it's often worth evaluating whether a flattened, denormalized structure or a separate child index using the join field type is a better fit than deeply nested objects, since nested aggregations do more internal join work than flat bucket aggregations.
Cricket analogy: It's like recording each ball as a separate scoring event, so a 300-run T20 innings can generate hundreds of hidden records, the way full ball-by-ball datasets balloon compared to a single scorecard row.
Exceeding index.mapping.nested_objects.limit throws a mapping exception at index time, and deeply nested arrays with thousands of elements per document can noticeably slow both indexing and nested aggregations, since Elasticsearch must join hidden child documents back to their parent on every query. Monitor nested array sizes in production and consider capping them or restructuring to a join field or a separate index if they grow unbounded.
- Fields mapped as type nested are indexed as separate hidden Lucene documents so relationships between sibling fields inside each array element are preserved.
- Ordinary aggregations can't see into nested objects; the nested aggregation with a path parameter is required to step into that hidden document scope.
- reverse_nested lets a sub-aggregation join back out from nested scope to root-level fields, enabling mixed nested/root grouping in one query.
- A common pattern nests into a child array, buckets by a child field, then reverse_nests to bucket by a parent field for cross-referencing.
- Each nested array element is a real internal Lucene document, so large nested arrays multiply document count and can hit index.mapping.nested_objects.limit.
- Deeply nested, high-cardinality arrays hurt indexing and aggregation performance; consider denormalization or the join field type as alternatives.
- Nested aggregations do genuine internal join work at query time, making them more expensive than flat bucket aggregations on the same data volume.
Practice what you learned
1. Why does the nested field type store each array element as a separate Lucene document?
2. What is required to aggregate on a subfield of a nested-type field?
3. What does the reverse_nested aggregation do?
4. What happens if a document's nested array exceeds index.mapping.nested_objects.limit?
5. What is a viable alternative to deeply nested, high-cardinality arrays for performance reasons?
Was this page helpful?
You May Also Like
Bucket Aggregations
Understand how Elasticsearch's bucket aggregations group documents by terms, ranges, and dates, and how they nest with metric aggregations for multi-level breakdowns.
Metric Aggregations
Learn how Elasticsearch's metric aggregations compute single- and multi-value numeric summaries like avg, sum, cardinality, and percentiles over matched documents.
Aggregation Performance
Learn what drives aggregation performance in Elasticsearch, doc_values, shard sizing, caching, and approximate algorithms, and how to keep large aggregations fast.