Elasticsearch Cluster Architecture

An Elasticsearch cluster is a collection of one or more nodes (server processes) that share the same cluster.name, hold your data collectively, and provide unified search and indexing capability across all the data. Each node runs a copy of the Elasticsearch software, joins the cluster by discovering other nodes, and takes on one or more roles that determine what work it does. Even a single-node deployment is technically a cluster of size one, but production systems typically run three or more nodes so that data and workload can be distributed and the loss of one machine does not take the whole system down.

Elasticsearch nodes are configured with roles in elasticsearch.yml or via node.roles: master-eligible nodes (master) participate in cluster-level decisions like creating indices and tracking which nodes are alive; data nodes (data, or the finer-grained data_hot, data_warm, data_cold, data_frozen) actually store shards and execute CRUD and search operations against them; ingest nodes run ingest pipelines to pre-process documents before indexing; and coordinating-only nodes (no special role) route requests, merge results from data nodes, and can act as a lightweight smart load balancer. A single node can hold multiple roles simultaneously, which is common in small clusters, while large production clusters deliberately separate roles onto dedicated hardware profiles.

Master-eligible nodes participate in a quorum-based election (built on a Raft-like consensus protocol since Elasticsearch 7.x) to choose exactly one active master, which is responsible for maintaining cluster state, creating or deleting indices, and tracking shard allocation. The cluster requires a majority of master-eligible nodes to be reachable to elect and retain a master; this is why production clusters use an odd number of master-eligible nodes (commonly 3) and rely on the cluster.initial_master_nodes setting during bootstrap. If fewer than a quorum of master-eligible nodes can communicate, the cluster refuses to elect a master rather than risk two masters disagreeing about state, a scenario known as split-brain that older versions were more vulnerable to.

The cluster state is a data structure, held in memory and replicated to every node, that describes the cluster's metadata: which indices exist, their mappings and settings, which nodes are in the cluster, and where every shard is currently allocated. Only the elected master publishes updates to cluster state, and every node applies those updates locally so all nodes have a consistent view. Nodes find each other through discovery, seeded by discovery.seed_hosts (a static list) or a hosts provider like EC2 or Azure discovery plugins; once nodes can talk to each other, the cluster's overall health is reported as green (all primary and replica shards allocated), yellow (all primaries allocated but some replicas are not), or red (some primary shards are unallocated, meaning data is unavailable).

Elasticsearch scales horizontally: to handle more data or query load, you add data nodes, and the cluster rebalances shards across them automatically. For resilience, production topologies typically dedicate three master-eligible nodes (small, stable, not serving heavy search traffic) separately from data nodes (which can be scaled up and down more freely), and often adopt a hot-warm-cold-frozen architecture where recently written, frequently queried data lives on fast hot nodes (NVMe SSD, more CPU) while older data ages onto cheaper warm and cold nodes with less powerful hardware, controlled by Index Lifecycle Management policies. This separation of concerns means a spike in search traffic or a noisy re-indexing job doesn't jeopardize the cluster's ability to make basic coordination decisions.