What are hinted handoffs in Cassandra?
Learn what hinted handoffs are in Cassandra: how coordinators store hints for down replicas, the hint window, and why repair is still needed for consistency.
Expected Interview Answer
A hinted handoff is a mechanism that preserves writes destined for a replica that is temporarily down or unreachable. The coordinator node stores a 'hint' — a record of the missed mutation plus its target — locally, and once the offline replica comes back online within the hint window, the coordinator replays the stored mutations to it, improving write availability and helping replicas catch up.
When a write is issued, the coordinator sends it to all responsible replicas. If one is unavailable, the coordinator writes a hint to its local hints store instead of losing the mutation. Hints are retained for up to max_hint_window_in_ms (default 3 hours); if the replica returns within that window, the coordinator streams the hints to it. Hinted handoff is a best-effort optimization, not a consistency guarantee: hints can be dropped if the window expires or the coordinator itself fails, and a write can still succeed at the requested consistency level without any hint being stored. That is why hinted handoff must be paired with anti-entropy repair to guarantee eventual consistency, and it does not count toward satisfying the write consistency level unless configured to.
- Improves write availability during brief replica outages
- Helps a recovered replica catch up quickly without full repair
- Reduces the amount of data anti-entropy repair must reconcile
- Avoids losing writes aimed at a momentarily down node
- Works transparently as part of the normal write path
AI Mentor Explanation
A hinted handoff is like a teammate jotting down messages for a player who has briefly left the dressing room. Rather than lose the instructions, the teammate keeps a note pad of everything the absent player missed, and the moment they walk back in, reads it all out so they're caught up. But if the player is gone for hours, the notes are eventually binned, and the player must rely on the full match review to fill the gaps — just as hints expire and repair takes over.
Step-by-Step Explanation
Step 1
Write reaches coordinator
A client write goes to a coordinator, which forwards the mutation to all responsible replicas.
Step 2
Detect a down replica
If a target replica is unreachable, the coordinator cannot deliver the mutation to it.
Step 3
Store a hint
The coordinator writes a hint (the mutation plus its intended target) to its local hints store instead of losing it.
Step 4
Wait within the hint window
Hints are retained up to max_hint_window_in_ms (default 3 hours); beyond that they are dropped.
Step 5
Replay on recovery
When the replica comes back within the window, the coordinator streams the stored hints to bring it up to date.
What Interviewer Expects
- How hints preserve writes for a down replica
- The role of the coordinator and local hints store
- max_hint_window_in_ms and hint expiry behavior
- That hinted handoff is best-effort, not a consistency guarantee
- Why repair is still required alongside hinted handoff
Common Mistakes
- Believing hinted handoff guarantees consistency on its own
- Assuming a hint always counts toward the write consistency level
- Ignoring hint window expiry, expecting hints to persist indefinitely
- Not running repair, relying on hints to reconcile everything
- Confusing hinted handoff with read repair or anti-entropy repair
Best Answer (HR Friendly)
“A hinted handoff is Cassandra's way of not losing a write when one of the servers that should store it is temporarily down. The server handling the request keeps a note of the missed update and, once the offline server is back within a few hours, sends it over so it catches up. It is a helpful safety net, but the cluster still runs a separate repair job to guarantee everything stays in sync.”
Code Example
# Enable/disable hint storage for down replicas
hinted_handoff_enabled: true
# How long a coordinator keeps hints for a down replica (default 3h)
max_hint_window_in_ms: 10800000
# Throttle for replaying hints so recovery doesn't overwhelm the node
hinted_handoff_throttle_in_kb: 1024
# Where hints are stored on the coordinator
hints_directory: /var/lib/cassandra/hints
# Note: hints are best-effort. Always run 'nodetool repair' for eventual consistency.Follow-up Questions
- How does hinted handoff differ from read repair and anti-entropy repair?
- What is max_hint_window_in_ms and what happens when it is exceeded?
- Does a hint count toward satisfying the write consistency level?
- What happens to hints if the coordinator itself fails?
- Why is hinted handoff not sufficient for guaranteeing consistency?
MCQ Practice
1. What does the coordinator do when a target replica is down during a write?
The coordinator records a hint (mutation plus target) in its hints store and replays it once the replica returns within the hint window.
2. What happens to hints after max_hint_window_in_ms elapses?
Hints expire after the window and are discarded; anti-entropy repair is then responsible for reconciling any divergence.
3. Is hinted handoff a consistency guarantee?
Hints can be dropped or lost, so hinted handoff is a best-effort availability aid that must be complemented by nodetool repair.
Flash Cards
What is a hinted handoff? — The coordinator stores a hint for a write aimed at a down replica and replays it when the replica recovers within the hint window.
How long are hints kept? — Up to max_hint_window_in_ms (default 3 hours); after that they are dropped.
Is hinted handoff a consistency guarantee? — No — it is best-effort; anti-entropy repair is still needed for eventual consistency.
Where are hints stored? — In the coordinator node's local hints store (hints_directory).
Hinted handoff vs read repair? — Hints preserve writes for down replicas; read repair fixes stale replicas during reads.