Distributed Transactions

A distributed transaction is a set of operations spanning multiple independent nodes, databases, or services that must succeed or fail together as a single atomic unit, even though no single node has full visibility or control over the others. This is straightforward within one database thanks to ACID transaction support, but becomes genuinely hard once you split state across services — for example, debiting a wallet service and crediting an order service in a microservices architecture — because a partial failure (one side commits, the other doesn't) leaves the system in an inconsistent state that's hard to detect and even harder to repair automatically. Distributed transaction techniques exist specifically to prevent or gracefully recover from these partial-failure states, at some cost in latency, availability, or implementation complexity.

Two-phase commit is the classical protocol for atomic distributed transactions. In the 'prepare' (voting) phase, a coordinator asks every participant to lock the relevant resources and confirm they are able to commit; each participant replies 'yes' (ready) or 'no' (abort). Only if every participant votes 'yes' does the coordinator move to the 'commit' phase, telling everyone to make the change permanent and release locks; if any participant votes 'no,' the coordinator tells everyone to abort and roll back. 2PC guarantees atomicity across nodes, but it has a serious weakness: if the coordinator crashes after participants have voted 'yes' but before sending the final commit/abort decision, participants are left blocked, holding locks indefinitely, unable to safely decide on their own whether to commit or abort — this is why 2PC is called a blocking protocol, and it's a major reason it fell out of favor for large-scale, loosely-coupled systems.

The saga pattern sidesteps 2PC's blocking problem by abandoning true atomicity in favor of a sequence of local transactions, each with a corresponding compensating action that can undo its effect if a later step fails. Instead of holding distributed locks across services for the duration of a transaction, each service commits its own local transaction immediately and publishes an event or is invoked by an orchestrator to trigger the next step; if step N fails, the saga executes compensating transactions for steps 1 through N-1 in reverse order (e.g., 'refund payment' compensates 'charge card'). Sagas can be coordinated via choreography (each service reacts to events from the previous one, with no central coordinator) or orchestration (a central saga orchestrator explicitly calls each step and handles failures) — orchestration is generally easier to reason about and debug at scale, while choreography reduces coupling but can make the overall flow harder to trace. Because each local transaction commits immediately rather than waiting on a global lock, sagas sacrifice isolation compared to ACID: other parts of the system can observe intermediate, partially-completed states before the saga finishes (e.g., inventory temporarily shows stock reserved before payment is confirmed), which must be handled explicitly through semantic locks, versioning, or read paths designed to tolerate it.

Because distributed transactions (especially sagas, which rely on retries when a step fails or a message is delayed) can result in a step being invoked more than once, every step and every compensating action must be idempotent — applying it multiple times must produce the same result as applying it once. This is typically achieved with idempotency keys (a unique ID per logical operation that the service checks before re-applying an effect) so that network retries, at-least-once message delivery, or orchestrator restarts never cause a double-charge or a double-refund. Idempotency is not optional in distributed transaction design — without it, the retry mechanisms that make sagas resilient to partial failure become a source of new bugs.