Why Bulk Indexing Exists
Indexing documents one at a time via individual PUT or POST requests incurs the full network round trip and per-request overhead for every single document, which becomes a severe bottleneck once you need to ingest thousands or millions of records. The _bulk API solves this by accepting many operations in a single HTTP request body, using a compact newline-delimited JSON format where each action line is immediately followed by its associated document source line.
Cricket analogy: A team bus ferrying all eleven players to the ground in one trip instead of sending eleven separate taxis mirrors how the _bulk API batches many document operations into one HTTP request.
The Bulk Request Format
Each operation in a bulk request is a pair of JSON lines, an action-and-metadata line specifying index, create, update, or delete along with the target _index and _id, followed by the source document line for index and create actions, or a partial doc object for update; delete actions have no source line at all. Every line, including the final one, must end with a newline character, and a single malformed line does not fail the whole request, instead the response items array reports per-operation success or failure so you must always check it rather than assuming a 200 status means everything succeeded.
Cricket analogy: A scorer logging each ball as two entries, the delivery type and then the outcome, mirrors how each bulk operation is two JSON lines, an action line followed by a source line.
curl -X POST "localhost:9200/_bulk" -H "Content-Type: application/x-ndjson" --data-binary '
{ "index": { "_index": "products", "_id": "101" } }
{ "name": "Trail Running Shoe", "price": 89.99 }
{ "update": { "_index": "products", "_id": "102" } }
{ "doc": { "price": 74.50 } }
{ "delete": { "_index": "products", "_id": "103" } }
'
Always inspect the errors field at the top of the bulk response and iterate the items array for any item.*.status of 400 or higher; a bulk call can return HTTP 200 overall while individual operations inside it fail.
Sizing Batches and Handling Backpressure
There is no universal ideal batch size; a common starting point is somewhere between 5 and 15 megabytes of request payload or a few thousand documents per bulk call, whichever comes first, and you should benchmark with your own document sizes and cluster hardware rather than trusting a fixed number blindly. If the cluster's bulk thread pool queue fills up under sustained load, Elasticsearch rejects further bulk requests with a 429 TOO_MANY_REQUESTS error, so production ingestion pipelines need retry logic with exponential backoff to handle this gracefully rather than dropping data.
Cricket analogy: A bowler pacing their overs across a full spell instead of bowling every ball at maximum effort and breaking down by over three mirrors right-sizing bulk batches instead of overloading the cluster.
Do not retry an entire failed bulk request blindly, retry only the specific items that reported a 429 or 5xx status in the response, since resubmitting successfully-indexed items can cause redundant work or, for non-idempotent update scripts, incorrect results.
- The _bulk API batches many index, create, update, and delete operations into one HTTP request.
- Each operation is two newline-terminated JSON lines: an action-and-metadata line and a source line (delete has none).
- A 200 response does not guarantee every operation succeeded; always check the items array for per-operation errors.
- A common starting batch size is 5-15MB or a few thousand documents, but benchmark against your own data.
- 429 TOO_MANY_REQUESTS from a full bulk thread pool queue requires retry logic with exponential backoff.
- Only retry the specific failed items from a bulk response, not the entire batch.
- Bulk indexing dramatically reduces per-document network and request overhead compared to individual requests.
Practice what you learned
1. What format does the _bulk API request body use?
2. If a bulk request returns HTTP 200, what can you conclude?
3. What does a bulk API call return when the cluster's bulk thread pool queue is full?
4. Which bulk action type has no accompanying source document line?
5. What is the recommended retry strategy after a bulk request encounters partial failures?
Was this page helpful?
You May Also Like
Creating an Index
Learn how Elasticsearch indices are created, configured with shards and replicas, and named for reliable, scalable search.
Mapping Types Explained
Understand the core Elasticsearch field data types, text vs keyword, numbers, dates, and objects, and how each shapes search behavior.
Dynamic vs Explicit Mapping
Compare Elasticsearch's automatic field-type inference against explicitly defined mappings, and know when to use each.