What is a document, index, and shard in Elasticsearch?
Learn what a document, index, and shard are in Elasticsearch, how they relate, how sharding enables scaling, with examples and interview answers.
Expected Interview Answer
In Elasticsearch a document is a single JSON record (the basic unit of data), an index is a named collection of related documents, and a shard is a self-contained Lucene subdivision of an index that stores part of its data and enables horizontal scaling.
Documents are stored as JSON and identified by an _id within an index. An index is logically like a table but is physically split into one or more shards so it can grow beyond a single machine. Each shard is a complete, independent Apache Lucene index; Elasticsearch distributes shards across nodes and routes each document to a specific shard using a hash of its routing value, so queries can run in parallel across shards.
- Documents keep data in flexible JSON form
- Indices group related data logically
- Shards allow an index to scale horizontally across nodes
- Parallel search across shards improves throughput
- Sharding enables datasets larger than one machine
AI Mentor Explanation
A document is like a single ball's scorecard entry recording batter, bowler and runs. An index is the whole match's scorebook grouping every such entry together. A shard is like splitting that thick scorebook across several scorers, each keeping a full section independently so the whole innings can be recorded and read in parallel without one scorer becoming a bottleneck.
Step-by-Step Explanation
Step 1
Model your data as documents
Represent each real-world record as a JSON document with fields; Elasticsearch stores and indexes it under an auto or supplied _id.
Step 2
Group documents into an index
Create an index (e.g. products) to hold all documents of the same kind, defining mappings for how fields are indexed.
Step 3
Choose a shard count
Set number_of_shards at index creation to decide how many pieces the index is split into; this is fixed for the life of the index.
Step 4
Route documents to shards
Elasticsearch hashes the routing value (default _id) modulo shard count to deterministically place each document in one primary shard.
Step 5
Query across shards
A search scatters to every shard, each returns partial results, and the coordinating node gathers and merges them into a final response.
What Interviewer Expects
- Clear definition of document, index, and shard
- Understanding that a shard is a Lucene index
- Awareness that shard count is fixed at creation
- Knowledge of document routing to shards
- Explanation of how sharding enables horizontal scale
Common Mistakes
- Confusing an index with a shard
- Saying shard count can be changed freely after creation
- Thinking a document must follow a rigid fixed schema
- Not knowing a shard is a complete independent Lucene index
- Ignoring that documents are routed to a specific shard
Best Answer (HR Friendly)
“A document is a single record of data stored as JSON, an index is a collection that groups many related documents together, and a shard is a slice of that index kept on a server. Splitting an index into shards lets Elasticsearch spread data across machines so it can store and search huge amounts of information quickly.”
Code Example
PUT /products
{
"settings": {
"number_of_shards": 3,
"number_of_replicas": 1
}
}
PUT /products/_doc/1
{
"name": "Wireless Mouse",
"price": 24.99,
"inStock": true
}Follow-up Questions
- How does Elasticsearch decide which shard a document goes to?
- Why can't you change the number of primary shards after index creation?
- What is the relationship between a shard and a Lucene index?
- What happens if you have too many small shards?
- How does an index differ from a database table?
MCQ Practice
1. What is the basic unit of data stored in Elasticsearch?
The document, a single JSON record identified by an _id, is the smallest unit of data you store and retrieve in Elasticsearch.
2. What is an Elasticsearch shard technically?
Each shard is a fully self-contained Apache Lucene index that stores a portion of the data and can be searched on its own.
3. When can the number of primary shards for an index be set?
The primary shard count is fixed at index creation because document routing depends on it; changing it requires reindexing into a new index.
Flash Cards
What is a document in Elasticsearch? — A single JSON record, the basic unit of data, identified by an _id within an index.
What is an index? — A named collection of related documents, roughly analogous to a table, split physically into shards.
What is a shard? — A self-contained Apache Lucene index that stores part of an index's data and enables horizontal scaling.
How are documents assigned to shards? — Elasticsearch hashes the routing value (default _id) modulo the primary shard count to pick one shard.
Continue Learning
Related Interview Questions
What is the difference between keyword and text field types in Elasticsearch?
medium
How does Elasticsearch handle scaling and cluster health (green, yellow, red)?
medium
What is the difference between a primary shard and a replica shard in Elasticsearch?
medium
What is the difference between filter context and query context in Elasticsearch?
medium