The Saga Pattern

In a monolith, a multi-step business operation like 'place order, reserve inventory, charge payment' can be wrapped in a single ACID database transaction: if any step fails, the whole thing rolls back atomically. Once those steps live in separate services with separate databases (per the database-per-service pattern), you can't use one transaction anymore. The classic distributed alternative, two-phase commit (2PC), requires a coordinator to lock resources across all participants until every service votes to commit, which does not scale well, doesn't work across heterogeneous systems like message queues, and creates availability risk — if the coordinator or any participant is down, everything blocks. The Saga pattern solves this by breaking the operation into a series of local transactions, each committed independently, with a defined compensating transaction to semantically undo each step if a later one fails.

There are two ways to coordinate a saga's steps. In choreography, there is no central coordinator: each service publishes an event when it completes its local transaction, and the next service in the flow subscribes to that event and reacts, publishing its own event in turn. This keeps services fully decoupled but makes the overall business process hard to see — you have to trace events across many services to understand the flow, and adding a new step means modifying multiple existing services' subscriptions. In orchestration, a dedicated saga orchestrator (sometimes a simple state machine, sometimes a workflow engine like Temporal, AWS Step Functions, or Camunda) explicitly calls each service in sequence and decides what to do on failure. Orchestration centralizes the business logic, making it easier to reason about and monitor, at the cost of introducing a new component that itself needs to be built reliably.

A compensating transaction is not a database rollback — it's a new, separate business operation that semantically reverses an already-committed step. If the Payment service already charged a customer's card and the Shipping service then discovers the item is out of stock, the compensation isn't 'undoing' the charge as if it never happened; it's issuing a refund, a distinct, auditable transaction. This matters because the original transaction may have had external side effects (an email confirmation sent, a loyalty point awarded) that a raw rollback can't undo. Compensations must also be idempotent and retriable, because in a distributed system the same compensation message might be delivered more than once, and must also handle the awkward case where the compensation itself fails, often requiring a dead-letter queue and manual intervention or a retry-with-backoff strategy.

Teams new to sagas often make a few recurring mistakes. First, forgetting that steps can fail out of order relative to compensations arriving — network retries mean a compensation message could theoretically arrive before its corresponding action is fully recorded, so idempotency keys and careful ordering (or using a per-saga correlation ID with a state machine) are essential. Second, treating semantic locks as optional: because there's no database-level lock across services, a saga in progress can leave inventory 'reserved' but visible to other processes unless the reservation itself is explicit business state, not just an in-flight assumption. Third, letting sagas grow too large — a saga spanning eight services with complex branching compensation logic becomes nearly impossible to reason about; it's often better to break it into smaller sagas with clear business boundaries, sometimes composing a saga of sagas.