Data Lineage
Data lineage is the tracked record of where a piece of data originated, how it has moved and transformed across systems, and where it is ultimately consumed — providing an end-to-end map of a dataset's life.
Definition
Data lineage is the tracked record of where a piece of data originated, how it has moved and transformed across systems, and where it is ultimately consumed — providing an end-to-end map of a dataset's life.
Overview
Modern data pipelines pass information through many stages: raw ingestion, ETL or ELT transformations, joins across multiple sources, aggregations, and finally consumption in dashboards or machine-learning models. When a number in a report looks wrong, data lineage is what lets a team trace backward through every transformation to find the root cause, rather than guessing. Lineage can be captured at different levels of granularity: table-level lineage shows which tables feed into which, while column-level lineage tracks exactly which source columns contribute to a specific derived field, useful for precise impact analysis. Lineage information is typically gathered automatically by parsing SQL queries, pipeline code (e.g. dbt models or Airflow DAGs), or by instrumenting the data pipeline itself to emit lineage events as data moves. Lineage is a critical companion to a data catalog and a cornerstone of practical data governance: it supports impact analysis before making schema changes, speeds up root-cause analysis when data quality issues appear, and helps demonstrate regulatory compliance by showing exactly how sensitive data flows through an organization. Because lineage depends on parsing transformation logic, it works especially well with tools that already model dependencies explicitly, such as dbt's DAG of models — a pattern covered in dbt & Analytics Engineering — or orchestration tools like Apache Airflow that define pipeline steps as a directed graph.
Key Concepts
- Maps the end-to-end path of data from origin through every transformation to final use
- Can be tracked at table-level or fine-grained column-level granularity
- Often captured automatically by parsing SQL, pipeline code, or DAG definitions
- Enables impact analysis before making schema or pipeline changes
- Speeds up root-cause analysis when downstream data looks wrong
- Supports compliance by showing how sensitive data flows through systems