What is the Outbox Pattern?
Learn how the outbox pattern solves the dual-write problem, guarantees reliable event publishing, and pairs with CDC tools.
Expected Interview Answer
The outbox pattern solves the dual-write problem by writing both a business state change and the event describing it into the same local database transaction, then relying on a separate background process to reliably publish that event to a message broker afterward.
The dual-write problem occurs when a service updates its own database and then separately publishes a message to a broker: if the process crashes between the two operations, the database and the message stream diverge, either losing an event or publishing one for a change that never actually committed. The outbox pattern fixes this by adding an outbox table in the same database, and in the same local transaction that updates business rows, also inserting a row describing the event to be published — since both writes share one ACID transaction, they are guaranteed to either both happen or neither happen. A separate relay process (often a change data capture tool like Debezium tailing the database log, or a simple polling worker) then reads unpublished outbox rows and publishes them to the broker, marking them published once the broker acknowledges receipt. This gives at-least-once delivery of events that are guaranteed consistent with the committed database state, at the cost of extra infrastructure and the requirement that downstream consumers handle duplicate or out-of-order messages idempotently.
- Eliminates the dual-write problem between a database update and a message publish
- Guarantees published events are always consistent with what was actually committed
- Works with any broker since publishing is decoupled into a separate relay process
- Combines naturally with change data capture tools for low-latency, reliable relaying
AI Mentor Explanation
The outbox pattern is like a scorer who, in the same stroke of the pen, updates the official scorebook and writes a duplicate entry on a runner slip to be carried to the broadcast booth, rather than updating the scorebook and separately shouting the update across the ground. If the scorer is interrupted right after writing, both the scorebook and the runner slip already reflect the same entry together, so nothing is half-done. A dedicated runner then carries slips to the booth one at a time and only discards a slip once the booth confirms receipt. That guaranteed together-or-not-at-all write, relayed afterward, is exactly what the outbox pattern achieves for database updates and published events.
Step-by-Step Explanation
Step 1
Write business data and outbox row together
In one local database transaction, update the business table and insert an event row into the outbox table.
Step 2
Commit atomically
Because both writes are in the same transaction, they either both persist or neither does, eliminating the dual-write gap.
Step 3
Relay reads unpublished events
A poller or a change-data-capture process (e.g., Debezium tailing the log) picks up new outbox rows.
Step 4
Publish and mark done
The relay publishes each event to the broker and marks it published, retrying safely since consumers must handle duplicates idempotently.
What Interviewer Expects
- Clearly explains the dual-write problem the pattern solves
- Describes writing business data and the event in the same local transaction
- Mentions a relay mechanism: polling or CDC (e.g., Debezium)
- Notes the at-least-once delivery guarantee and the need for idempotent consumers
Common Mistakes
- Not identifying the dual-write problem as the reason the pattern exists
- Assuming the outbox pattern guarantees exactly-once delivery (it guarantees at-least-once)
- Forgetting that consumers must be idempotent to handle duplicate or replayed events
- Confusing the outbox pattern with simply using a distributed transaction across the database and the broker
Best Answer (HR Friendly)
“The outbox pattern solves the problem of a service crashing between saving data and telling other services about the change. Instead of doing those two things separately, it saves the data update and a record of the event in the very same database transaction, so they are guaranteed to happen together. A separate background process then reliably picks up that event record and delivers it to other services, so nothing is ever silently lost.”
Code Example
def create_order(db, order):
with db.transaction():
db.execute(
"INSERT INTO orders (id, user_id, total) VALUES (%s, %s, %s)",
(order.id, order.user_id, order.total),
)
db.execute(
"INSERT INTO outbox (id, event_type, payload, published) "
"VALUES (%s, %s, %s, false)",
(uuid4(), "OrderCreated", order.to_json()),
)
# both rows commit together or not at all
def relay_outbox_events(db, broker):
rows = db.query(
"SELECT id, event_type, payload FROM outbox WHERE published = false ORDER BY id LIMIT 100"
)
for row in rows:
broker.publish(topic=row.event_type, message=row.payload)
db.execute("UPDATE outbox SET published = true WHERE id = %s", (row.id,))Follow-up Questions
- How does change data capture (CDC) improve on a simple polling relay for the outbox table?
- Why does the outbox pattern only guarantee at-least-once delivery, and how should consumers handle that?
- How would you clean up old published rows in the outbox table without losing unpublished ones?
- How does the outbox pattern compare to using a distributed transaction across the database and the broker directly?
MCQ Practice
1. What problem does the outbox pattern solve?
Writing to a database and a broker as two separate operations risks one succeeding without the other; the outbox pattern makes them atomic locally.
2. What delivery guarantee does the outbox pattern typically provide?
Relay retries can cause the same event to be published more than once, so consumers must be able to safely handle duplicates.
3. Which tool is commonly used to relay outbox rows via change data capture?
Debezium tails the database transaction log and streams new outbox rows to a broker without requiring inefficient polling.
Flash Cards
What is the outbox pattern? — Writing a business update and its event record in the same local transaction, then relaying the event separately.
What problem does it solve? — The dual-write problem where a database update and a message publish could otherwise diverge on crash.
What delivery guarantee does it give? — At-least-once delivery of events consistent with committed database state.
Common relay mechanism? — A polling worker or a change data capture tool like Debezium tailing the database log.