What is the role of etcd in a Kubernetes cluster?
Learn what etcd does in Kubernetes: the consistent key-value store of cluster state, how Raft keeps it reliable, and why snapshots matter for recovery.
Expected Interview Answer
etcd is the distributed, consistent key-value store that acts as the single source of truth for a Kubernetes cluster, holding all cluster state and configuration such as nodes, pods, secrets, and ConfigMaps.
The API server is the only component that talks to etcd directly; every other component reads and writes state through the API server, which persists it to etcd. etcd uses the Raft consensus algorithm to replicate data across an odd number of members (typically 3 or 5) so the cluster stays consistent and can tolerate node failures. If etcd is lost without a backup, the desired state of the entire cluster is gone, which is why regular etcd snapshots are a critical operational practice.
- Single source of truth for all cluster state
- Strong consistency via the Raft consensus algorithm
- High availability through an odd-numbered member quorum
- Watch API lets components react to state changes in real time
- Snapshot-based backup and restore for disaster recovery
AI Mentor Explanation
etcd is the official match scorebook that records every wicket, run, and player position as the one authoritative record. Umpires and scorers never keep private versions; they all reconcile to this single book, and if it is destroyed mid-match with no carbon copy, the true state of the game is lost and cannot be reconstructed reliably.
Step-by-Step Explanation
Step 1
State is written
A user runs kubectl apply; the API server validates the request and persists the resulting desired state into etcd.
Step 2
etcd reaches consensus
The write is replicated to a majority (quorum) of etcd members using Raft before being acknowledged as committed.
Step 3
Controllers watch
Controllers and the scheduler use the API server's watch API, which streams etcd changes, to detect the new desired state.
Step 4
Reconciliation
Controllers act to make actual cluster state match what etcd records, then write status back through the API server into etcd.
Step 5
Protect the store
Operators take periodic etcd snapshots so the entire cluster state can be restored after catastrophic failure.
What Interviewer Expects
- etcd is a distributed key-value store, not a general database for app data
- Only the API server communicates with etcd directly
- Awareness of Raft consensus and odd-numbered quorum
- Understanding that losing etcd means losing cluster state
- Knowledge that etcd should be backed up with snapshots
Common Mistakes
- Claiming worker nodes or kubelets talk to etcd directly
- Confusing etcd with a place to store application data
- Forgetting that etcd needs an odd number of members for quorum
- Not mentioning backup/restore via snapshots
- Thinking the scheduler writes to etcd rather than through the API server
Best Answer (HR Friendly)
“etcd is the memory of a Kubernetes cluster: it reliably stores everything the cluster needs to know about its own setup. Because it is so important, it is copied across several machines for safety and backed up regularly so nothing is lost.”
Code Example
# Save a snapshot of the current cluster state
ETCDCTL_API=3 etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
snapshot save /var/lib/etcd/snapshot.db
# Check the snapshot is valid
ETCDCTL_API=3 etcdctl snapshot status /var/lib/etcd/snapshot.db -w table
# Restore into a new data directory during disaster recovery
ETCDCTL_API=3 etcdctl snapshot restore /var/lib/etcd/snapshot.db \
--data-dir=/var/lib/etcd-restoredFollow-up Questions
- Why does etcd require an odd number of members?
- How would you back up and restore etcd in production?
- What is the Raft consensus algorithm and why does etcd use it?
- What happens to the cluster if etcd loses quorum?
- Why is only the API server allowed to talk to etcd?
MCQ Practice
1. Which Kubernetes component communicates directly with etcd?
The API server is the only component that reads from and writes to etcd; all others go through the API server.
2. What consensus algorithm does etcd use to stay consistent?
etcd uses the Raft consensus algorithm to replicate writes and maintain a consistent log across members.
3. Why should an etcd cluster have an odd number of members?
An odd number lets etcd form a clear majority quorum, maximizing fault tolerance while avoiding split-brain ties.
Flash Cards
What is etcd? — A distributed, consistent key-value store that holds all Kubernetes cluster state and configuration.
Which component talks to etcd? — Only the kube-apiserver; every other component reads and writes state through it.
How does etcd stay consistent? — It replicates writes across members using the Raft consensus algorithm and requires a quorum.
Why back up etcd? — Losing etcd without a snapshot means losing the entire desired state of the cluster.