Great Expectations (Data Quality) Cheat Sheet
Define, validate, and document data quality expectations for pipelines using Great Expectations suites, checkpoints, and data docs.
2 PagesIntermediateJan 20, 2026
Create an Expectation Suite
Define reusable data quality rules against a pandas or SQL data source.
python
import great_expectations as gxcontext = gx.get_context()source = context.sources.add_pandas("orders_source")asset = source.add_dataframe_asset(name="orders", dataframe=df)validator = context.get_validator( batch_request=asset.build_batch_request(), expectation_suite_name="orders_suite",)validator.expect_column_values_to_not_be_null("order_id")validator.expect_column_values_to_be_between("amount", min_value=0, max_value=100000)validator.expect_column_values_to_be_in_set("status", ["pending", "completed", "cancelled"])validator.save_expectation_suite(discard_failed_expectations=False)
Run a Checkpoint
Bundle a batch and a suite into a checkpoint and execute validation as a pipeline step.
python
checkpoint = context.add_or_update_checkpoint( name="orders_checkpoint", validations=[{ "batch_request": asset.build_batch_request(), "expectation_suite_name": "orders_suite", }],)result = checkpoint.run()print(result.success) # False if any expectation failedif not result.success: raise ValueError("Data quality check failed — halting pipeline")
Validate a SQL Table
Point Great Expectations at a warehouse table via a SQLAlchemy connection string.
python
source = context.sources.add_sql( name="warehouse", connection_string="postgresql://user:pass@host/db")asset = source.add_table_asset(name="orders_tbl", table_name="orders")validator = context.get_validator( batch_request=asset.build_batch_request(), expectation_suite_name="orders_suite",)validator.expect_table_row_count_to_be_between(min_value=1)
Frequently Used Expectations
A starting set of expectation methods covering the majority of data quality checks.
- expect_column_values_to_not_be_null- flags unexpected nulls in a required column
- expect_column_values_to_be_unique- enforces primary-key-like uniqueness
- expect_column_values_to_be_between- range check for numeric columns
- expect_column_values_to_match_regex- format validation, e.g. emails or phone numbers
- expect_table_row_count_to_be_between- guards against empty or unexpectedly small loads
- expect_column_pair_values_A_to_be_greater_than_B- cross-column consistency checks
Author a Custom Expectation
Extend Great Expectations with a domain-specific rule when the built-in expectations don't cover a business constraint.
python
from great_expectations.execution_engine import PandasExecutionEnginefrom great_expectations.expectations.expectation import ColumnMapExpectationfrom great_expectations.expectations.metrics.map_metric_provider import ( ColumnMapMetricProvider, column_condition_partial,)class ColumnValuesToBeValidSku(ColumnMapMetricProvider): condition_metric_name = "column_values.valid_sku" @column_condition_partial(engine=PandasExecutionEngine) def _pandas(cls, column, **kwargs): return column.str.match(r"^[A-Z]{3}-\d{5}$")class ExpectColumnValuesToBeValidSku(ColumnMapExpectation): map_metric = "column_values.valid_sku" success_keys = ("mostly",)# now usable like any built-in expectationvalidator.expect_column_values_to_be_valid_sku("sku", mostly=0.99)
Detect Distribution Drift
Flag statistical drift between a reference batch and the current batch instead of just range/null checks.
python
reference_partition = validator.get_column_partition( column="transaction_amount", n_bins=10)validator.expect_column_kl_divergence_to_be_less_than( column="transaction_amount", partition_object=reference_partition, threshold=0.1,)validator.expect_column_quantile_values_to_be_between( column="latency_ms", quantile_ranges={ "quantiles": [0.5, 0.95, 0.99], "value_ranges": [[10, 50], [100, 300], [300, 800]], },)
Wire Checkpoint Actions for Alerts
Extend a checkpoint's action_list so failures page a team channel and every run refreshes Data Docs automatically.
python
checkpoint = context.add_or_update_checkpoint( name="orders_checkpoint", validations=[{ "batch_request": asset.build_batch_request(), "expectation_suite_name": "orders_suite", }], action_list=[ {"name": "store_validation_result", "action": {"class_name": "StoreValidationResultAction"}}, {"name": "update_data_docs", "action": {"class_name": "UpdateDataDocsAction"}}, { "name": "send_slack_notification", "action": { "class_name": "SlackNotificationAction", "slack_webhook": "${SLACK_WEBHOOK}", "notify_on": "failure", "notify_with": ["local_site"], }, }, ],)
Publish Data Docs to S3
Move the generated HTML validation reports from local disk to a shared, team-accessible site.
python
context.add_data_docs_site( site_name="s3_site", site_config={ "class_name": "SiteBuilder", "store_backend": { "class_name": "TupleS3StoreBackend", "bucket": "my-gx-docs", "prefix": "data_docs", }, "site_index_builder": {"class_name": "DefaultSiteIndexBuilder"}, },)context.build_data_docs(site_names=["s3_site"])
Core GX Architecture Concepts
The building blocks that fit together underneath the validator/checkpoint API.
- Data Context- the entry point holding all project configuration, datasources, and stores
- Datasource / Asset- a connection plus a named, queryable slice of data (a table, a dataframe, a file)
- Batch Request- a query describing which slice of an asset to pull for validation
- Expectation Suite- a named, versioned collection of expectations attached to a data asset
- Validation Result- the pass/fail output of running a suite against a batch, persisted to a Store
- Store- pluggable backend (filesystem, S3, Postgres) for suites, results, and metrics
- Checkpoint- bundles a batch request + suite + action_list into one runnable, orchestrator-friendly unit
Pro Tip
Wire checkpoint.run() into your orchestrator (Airflow, Dagster) as a hard gate before the load step — catching a schema drift or null spike before it reaches the warehouse is far cheaper than debugging a downstream dashboard.
Was this cheat sheet helpful?
Explore Topics
#GreatExpectationsDataQuality#GreatExpectationsDataQualityCheatSheet#DataScience#Intermediate#CreateAnExpectationSuite#RunACheckpoint#ValidateASQLTable#FrequentlyUsedExpectations#MachineLearning#DevOps#CheatSheet#SkillVeris
