Airbyte
By Airbyte, Inc.
Airbyte is an open-source data integration platform that moves data from source systems — databases, SaaS apps, and files — into destinations like data warehouses and lakes, using an ELT (extract, load, transform) approach.
Definition
Airbyte is an open-source data integration platform that moves data from source systems — databases, SaaS apps, and files — into destinations like data warehouses and lakes, using an ELT (extract, load, transform) approach.
Overview
Airbyte's core idea is a large, standardized catalog of pre-built connectors: each source (a database, API, or SaaS tool) and destination (a warehouse or lake) implements a common specification, so new integrations can be added or swapped without rewriting pipeline code. Connectors run as isolated Docker containers, which keeps each one's dependencies self-contained and makes the catalog easier for the community to extend. Airbyte can be self-hosted or used as a managed cloud service, and pipelines can be orchestrated on a schedule or triggered from external tools. Airbyte deliberately stops at loading raw data into the destination, leaving transformation to purpose-built tools — most commonly dbt — which follows the modern ELT pattern of load-then-transform rather than the older ETL pattern of transforming data before it lands. This division of labor lets Airbyte focus purely on reliable extraction and loading while transformation logic stays version-controlled and testable in SQL. In production data stacks, Airbyte pipelines are frequently scheduled and monitored with orchestrators like Apache Airflow or Dagster, and the resulting tables commonly land in warehouses such as Snowflake or BigQuery for analytics. Because the barrier to adding a new connector is relatively low, Airbyte's catalog has grown to cover hundreds of sources, making it a common choice for data teams that need to consolidate scattered data — CRMs, ad platforms, product databases — into one place without hand-writing custom extraction scripts for each one. Concepts like ELT pipelines and analytics engineering are covered in more depth in the dbt & Analytics Engineering course.
Key Features
- Large, standardized catalog of source and destination connectors
- Connectors packaged as isolated Docker containers for consistent execution
- Self-hosted (open-source) and managed cloud deployment options
- ELT-first design that loads raw data before transformation
- Incremental sync support to avoid re-extracting unchanged data
- Connector Development Kit (CDK) for building custom connectors
- Scheduling and basic orchestration, with hooks for external orchestrators
- Change data capture support for select database sources