ELK Stack (Elasticsearch/Logstash/Kibana) Cheat Sheet
Reference for Elasticsearch queries, Logstash pipeline configuration, and Kibana usage for centralized log aggregation and search.
3 PagesAdvancedFeb 12, 2026
Elasticsearch Query DSL
A basic search query filtering by term and range.
json
GET /logs-*/_search{ "query": { "bool": { "must": [ { "match": { "message": "error" } } ], "filter": [ { "term": { "level": "ERROR" } }, { "range": { "@timestamp": { "gte": "now-1h" } } } ] } }, "sort": [{ "@timestamp": "desc" }], "size": 50}
Logstash Pipeline
A pipeline parsing Apache logs with grok and shipping to Elasticsearch.
ruby
input { file { path => "/var/log/apache2/access.log" start_position => "beginning" }}filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}" } } date { match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ] }}output { elasticsearch { hosts => ["http://localhost:9200"] index => "apache-logs-%{+YYYY.MM.dd}" }}
Elasticsearch _cat & Index APIs
Useful admin/inspection commands via curl.
bash
curl -X GET "localhost:9200/_cat/indices?v" # List indicescurl -X GET "localhost:9200/_cluster/health?pretty" # Cluster healthcurl -X DELETE "localhost:9200/logs-2023.01.01" # Delete an indexcurl -X PUT "localhost:9200/logs-2024.01.01" -H 'Content-Type: application/json' -d '{"settings":{"number_of_shards":1}}'
Key Concepts
Core ELK stack terminology.
- index- A collection of documents in Elasticsearch, roughly analogous to a database table
- shard- A horizontal partition of an index distributed across nodes for scale
- grok filter- Logstash pattern matching that parses unstructured text into structured fields
- Kibana Discover- UI for interactively searching and filtering indexed log documents
- index lifecycle management (ILM)- Automates rollover, shrink, and deletion of indices based on age/size
- Beats- Lightweight shippers (Filebeat, Metricbeat) that forward data to Logstash/Elasticsearch
Bucket + Metric Aggregations
Terms aggregation bucketing by service with nested error-count and p95 latency metrics.
json
GET /logs-*/_search{ "size": 0, "query": { "range": { "@timestamp": { "gte": "now-24h" } } }, "aggs": { "by_service": { "terms": { "field": "service.keyword", "size": 10 }, "aggs": { "error_count": { "filter": { "term": { "level": "ERROR" } } }, "p95_latency": { "percentiles": { "field": "duration_ms", "percents": [95] } } } } }}
ILM Policy + Index Template
A hot-warm-cold-delete lifecycle policy attached to an index template with an explicit mapping.
json
PUT _ilm/policy/logs-policy{ "policy": { "phases": { "hot": { "actions": { "rollover": { "max_size": "50gb", "max_age": "1d" } } }, "warm": { "min_age": "3d", "actions": { "shrink": { "number_of_shards": 1 }, "forcemerge": { "max_num_segments": 1 } } }, "cold": { "min_age": "14d", "actions": { "searchable_snapshot": { "snapshot_repository": "backups" } } }, "delete": { "min_age": "30d", "actions": { "delete": {} } } } }}PUT _index_template/logs-template{ "index_patterns": ["logs-*"], "template": { "settings": { "index.lifecycle.name": "logs-policy", "number_of_shards": 1 }, "mappings": { "dynamic": "strict", "properties": { "@timestamp": { "type": "date" }, "level": { "type": "keyword" } } } }}
Mutate/Ruby Filters & Dead Letter Queue
Field renaming, a Ruby filter computing a derived field, and enabling the DLQ for unprocessable events.
ruby
filter { mutate { rename => { "[host][name]" => "hostname" } remove_field => ["agent", "ecs"] } ruby { code => "event.set('duration_bucket', (event.get('duration_ms').to_f / 100).floor * 100)" } if [level] == "ERROR" { mutate { add_tag => ["needs_triage"] } }}# logstash.ymldead_letter_queue.enable: truedead_letter_queue.max_bytes: 1gbpath.dead_letter_queue: /var/lib/logstash/dead_letter_queue
Advanced Concepts
Cluster-scale ELK terminology beyond basic indexing and search.
- circuit breaker- Elasticsearch memory guard that aborts a request before it triggers an OutOfMemoryError
- mapping explosion- Uncontrolled field-count growth from dynamic mapping on high-cardinality/unstructured documents, degrading cluster health
- hot-warm-cold architecture- Tiered node roles matched to ILM phases, moving older indices to cheaper storage/hardware
- snapshot/restore- Incremental backup of indices to a registered repository (S3, GCS, shared FS) for DR and cluster migration
- cross-cluster search (CCS)- Querying indices across multiple linked Elasticsearch clusters from a single request
- reindex API- Server-side copy of documents from one index to another, used for mapping changes without downtime
- Painless- Elasticsearch's sandboxed scripting language for scripted fields, update-by-query, and ingest pipelines
Painless: Update by Query & Script Fields
Bulk-computing a derived field with update_by_query, and a runtime script field evaluated at search time.
json
POST /logs-2024.01.01/_update_by_query{ "script": { "source": "ctx._source.severity_score = params.map.get(ctx._source.level)", "lang": "painless", "params": { "map": { "ERROR": 3, "WARN": 2, "INFO": 1 } } }, "query": { "exists": { "field": "level" } }}GET /logs-*/_search{ "script_fields": { "age_days": { "script": { "source": "(System.currentTimeMillis() - doc['@timestamp'].value.millis) / 86400000" } } }}
Pro Tip
Prefer Filebeat shipping directly to Elasticsearch (or via an ingest pipeline) over a heavyweight Logstash agent on every host — reserve Logstash for centralized, complex transformations to reduce per-node resource overhead.
Was this cheat sheet helpful?
Explore Topics
#ELKStackElasticsearchLogstashKibana#ELKStackElasticsearchLogstashKibanaCheatSheet#DevOps#Advanced#ElasticsearchQueryDSL#LogstashPipeline#Elasticsearch#Cat#DataStructures#Databases#CheatSheet#SkillVeris
