What is Change Data Capture (CDC)?
Learn how change data capture streams database changes in order, log-based vs polling CDC, and real-world CDC use cases.
Expected Interview Answer
Change data capture is a technique for detecting and streaming every row-level insert, update, and delete made to a database as an ordered sequence of events, typically by tailing the database transaction log rather than polling tables or relying on application code to emit events manually.
Log-based CDC tools like Debezium connect to a database write-ahead log (Postgres logical replication slots, MySQL binlog, MongoDB oplog) and read the same stream of changes the database itself uses for replication, turning each committed row change into a structured event without adding any load to the application or requiring schema changes for triggers. This differs from polling-based CDC, which periodically queries for rows with a changed timestamp column and can miss deletes or fast successive updates between polls, and from trigger-based CDC, which adds write overhead and coupling directly inside the database. Because log-based CDC reads committed changes in the exact order they were written, it naturally provides an ordered, low-latency stream that other services can subscribe to for cache invalidation, search index updates, data warehouse replication, or as the relay mechanism for the outbox pattern. The main operational costs are needing replication-level database access, handling schema evolution in the change stream, and ensuring downstream consumers handle the eventual-consistency lag between the source write and the CDC event arriving.
- Captures every insert/update/delete without polling overhead or missed changes
- Log-based CDC adds negligible load on the source database and requires no application code changes
- Provides a naturally ordered, low-latency event stream for downstream consumers
- Enables use cases like search index sync, cache invalidation, and outbox-pattern relaying
AI Mentor Explanation
Change data capture is like a broadcast production team tapping directly into the stadium official scoring feed instead of having a runner periodically walk over and copy down the scoreboard by hand. Every single change to the score, however small, streams to the broadcast booth the instant it is recorded, in the exact order it happened, without anyone needing to poll the scoreboard or miss a quick double-change between visits. If the runner method were used instead, a fast sequence of two wickets between visits could be missed or merged into one update. That direct, ordered tap into the source-of-truth feed is exactly what log-based CDC does for a database.
Step-by-Step Explanation
Step 1
Connect to the transaction log
The CDC tool attaches to the database write-ahead log or binlog (e.g., via a Postgres replication slot or MySQL binlog reader).
Step 2
Read committed changes in order
Each committed insert, update, or delete is read from the log in the exact order it was written, with no polling gap.
Step 3
Transform into structured events
Row changes are converted into structured event payloads (before/after values, operation type, table name).
Step 4
Stream to downstream consumers
Events are published to a broker (e.g., Kafka) for consumers like search indexers, caches, and data warehouses to subscribe to.
What Interviewer Expects
- Explains log-based CDC and contrasts it with polling-based and trigger-based approaches
- Names a concrete tool (Debezium) and log mechanism (binlog, WAL, oplog)
- Mentions ordering guarantees and low added load on the source database
- Connects CDC to real use cases: search sync, cache invalidation, outbox relaying, data warehousing
Common Mistakes
- Assuming CDC requires polling the database on a schedule
- Not knowing a concrete CDC tool or the underlying log mechanism it reads
- Ignoring that CDC introduces eventual-consistency lag between write and event arrival
- Confusing CDC with database replication itself rather than a stream of structured change events
Best Answer (HR Friendly)
“Change data capture is a way of streaming every change made to a database — every insert, update, and delete — out to other systems in real time, in the exact order it happened. Instead of periodically checking the database for what changed, it taps directly into the database internal change log, which is more efficient and never misses fast, back-to-back changes. Other systems, like a search index or a cache, can then subscribe to that stream to stay up to date automatically.”
Code Example
name: orders-cdc-connector
config:
connector.class: io.debezium.connector.postgresql.PostgresConnector
database.hostname: db.internal
database.port: "5432"
database.user: cdc_reader
database.dbname: orders_db
plugin.name: pgoutput
slot.name: orders_cdc_slot
table.include.list: public.orders,public.order_items
topic.prefix: orders
snapshot.mode: initial
publication.autocreate.mode: filteredFollow-up Questions
- How does log-based CDC differ from polling-based and trigger-based change capture?
- How would you handle a schema change in the source table without breaking downstream CDC consumers?
- How does CDC relate to and support the outbox pattern for reliable event publishing?
- What happens to downstream consumers if the CDC connector falls behind or restarts from an old position in the log?
MCQ Practice
1. What does log-based CDC read to detect database changes?
Log-based CDC tails the same transaction log the database uses internally for replication, capturing changes with no added query load.
2. What is a key weakness of polling-based change detection compared to log-based CDC?
Polling only sees the state at each check interval, so rapid or deleted changes between polls can be missed entirely.
3. Which is a common real-world use case for change data capture?
CDC streams are commonly consumed to keep search indexes, caches, and data warehouses synchronized with the source database.
Flash Cards
What is change data capture? — A technique for streaming row-level database changes as an ordered sequence of events.
Log-based vs polling-based CDC? — Log-based reads the transaction log directly with no gaps; polling can miss fast changes between checks.
Name a popular CDC tool. — Debezium, which tails database logs like Postgres WAL, MySQL binlog, or MongoDB oplog.
A common CDC use case? — Relaying outbox events, syncing search indexes, invalidating caches, or replicating into a data warehouse.