Soda
By Soda (Soda Data N.V.)
Soda is a data quality monitoring and testing platform that lets data teams define checks against their datasets and run them automatically to catch missing values, schema changes, freshness delays, and other data issues before they reach…
Definition
Soda is a data quality monitoring and testing platform that lets data teams define checks against their datasets and run them automatically to catch missing values, schema changes, freshness delays, and other data issues before they reach downstream reports or models. It is built around an open-source checks language called SodaCL, paired with a cloud product for scheduling, alerting, and visualizing data quality results across an organization's data pipelines.
Overview
Soda emerged from the recognition that data pipelines can run successfully from an engineering standpoint while still delivering wrong or incomplete data, a failure mode traditional pipeline monitoring does not catch because it only checks whether a job finished, not whether the data it produced is correct. Soda's approach is to let teams write explicit, human-readable checks about what the data should look like, and to run those checks as a routine part of the data pipeline rather than as an afterthought. Mechanically, checks are written in SodaCL, a YAML-based checks language that lets a user specify assertions such as row counts staying within a range, no duplicate values in a key column, or column freshness not exceeding a threshold. The open-source Soda Core library executes these checks against a connected data source such as a warehouse table, and results can be sent to Soda's cloud platform, which tracks check history over time, visualizes failures, and routes alerts to tools like Slack when a check fails, giving data teams a feedback loop similar to software test suites but applied to data itself. Within the data observability and quality space, Soda sits alongside tools like Great Expectations and Monte Carlo, but distinguishes itself with a lighter-weight, YAML-first checks language intended to be approachable for analysts as well as engineers, rather than requiring a full Python testing framework. It focuses specifically on explicit, author-defined checks rather than purely automated anomaly detection, complementing rather than replacing metadata catalogs like DataHub or Amundsen, which describe what data exists rather than whether it is currently correct. In practice, data engineering and analytics teams embed Soda checks into their pipeline orchestration, commonly running them as a step in an Airflow DAG or a dbt run, so that a failing check can halt a pipeline or flag a dataset before it reaches a dashboard or a machine learning model. It is also used to formalize data contracts between teams, where a downstream consumer's expectations about a dataset are encoded as Soda checks that the producing team's pipeline must satisfy. The trade-off with Soda's approach is that checks must be authored and maintained deliberately; it does not automatically infer every possible data quality rule without configuration, unlike some anomaly-detection-first tools that flag statistical deviations with no setup. Teams adopting Soda need to invest time identifying which checks matter for their most critical datasets, and the value scales with how thoroughly checks are written and kept up to date as pipelines evolve.
Key Features
- YAML-based SodaCL language for defining explicit data quality checks
- Open-source Soda Core library for running checks locally or in CI
- Cloud platform for scheduling, history tracking, and alerting
- Integration with orchestrators like Airflow and transformation tools like dbt
- Checks for freshness, completeness, duplication, and schema validity
- Slack and other alerting integrations for failed check notifications
- Support for formalizing data contracts between producing and consuming teams