Logstash
By Elastic
Logstash is an open-source, server-side data processing pipeline that ingests data from multiple sources, transforms it, and forwards it to a destination such as Elasticsearch for search and analysis.
Definition
Logstash is an open-source, server-side data processing pipeline that ingests data from multiple sources, transforms it, and forwards it to a destination such as Elasticsearch for search and analysis.
Overview
Logstash is one of the original components of the Elastic Stack (commonly known as the ELK Stack, alongside Elasticsearch and Kibana), built to solve the practical problem of collecting logs from many different systems in many different formats and normalizing them into a consistent structure. A Logstash pipeline is defined in three stages: inputs (reading from files, syslog, Kafka, databases, or message queues), filters (parsing and enriching events—most notably with the grok filter for extracting structured fields from unstructured text), and outputs (commonly sending data to Elasticsearch, though it can write to many other destinations). It runs on the JVM, which gives it broad plugin support but also a heavier resource footprint than lighter-weight shippers. Because Logstash can be resource-intensive at scale, many pipelines now pair it with lightweight shippers like Filebeat for collection and use Logstash only for heavier transformation, or replace it in simpler cases with alternatives like Fluentd. It still remains a common choice when complex parsing or enrichment is needed before data reaches Elasticsearch or dashboards in tools like Grafana.
Key Features
- Input plugins for files, syslog, message queues, databases, and APIs
- Grok and other filters for parsing unstructured log data into structured fields
- Output plugins supporting Elasticsearch and many other destinations
- Pipeline-based configuration with inputs, filters, and outputs
- Built-in buffering and backpressure handling for bursty data
- Extensible plugin ecosystem for custom inputs, filters, and outputs
- Runs as part of the broader Elastic Stack alongside Elasticsearch and Kibana