Data Warehousing Concepts Cheat Sheet
Covers dimensional modeling, star vs. snowflake schemas, ETL/ELT pipelines, OLAP operations, and slowly changing dimensions for building analytical data warehouses.
Core Concepts
Fundamental building blocks of a data warehouse.
- Fact table- Central table storing quantitative business metrics (sales amount, quantity) at a specific grain, with foreign keys to dimensions
- Dimension table- Descriptive table (customer, product, date) that provides context for facts via lookups
- Star schema- Fact table directly linked to denormalized dimension tables; fast for reads, simple joins
- Snowflake schema- Dimensions normalized into sub-dimensions; saves space but requires more joins
- Grain- The level of detail a single fact row represents (e.g., one row per order line item)
- Surrogate key- System-generated integer key used instead of natural keys to join facts and dimensions
Star Schema Query
Typical analytical query joining a fact table to its dimensions.
SELECT d.year, p.category, SUM(f.sales_amount) AS total_salesFROM fact_sales fJOIN dim_date d ON f.date_key = d.date_keyJOIN dim_product p ON f.product_key = p.product_keyWHERE d.year IN (2023, 2024)GROUP BY d.year, p.categoryORDER BY d.year, total_sales DESC;
Slowly Changing Dimension (Type 2)
Track historical changes by inserting a new row and expiring the old one.
-- Expire the current rowUPDATE dim_customerSET end_date = CURRENT_DATE, is_current = FALSEWHERE customer_id = 42 AND is_current = TRUE;-- Insert the new versionINSERT INTO dim_customer (customer_id, name, city, start_date, end_date, is_current)VALUES (42, 'Jane Doe', 'Austin', CURRENT_DATE, NULL, TRUE);
OLAP Operations
Common ways analysts navigate a multidimensional cube.
- Roll-up- Aggregate data by climbing up a hierarchy (e.g., day -> month -> year)
- Drill-down- Move from summarized to more detailed data (year -> quarter -> day)
- Slice- Filter the cube to a single value on one dimension (e.g., Year = 2024)
- Dice- Filter using multiple dimensions to produce a smaller sub-cube
- Pivot (rotate)- Reorient the cube's axes to view data from a different perspective
ETL vs. ELT
Two approaches to loading data into a warehouse.
- ETL- Extract, Transform, Load: transform data in a staging area before loading into the warehouse
- ELT- Extract, Load, Transform: load raw data first, transform inside the warehouse using its compute (common with Snowflake, BigQuery)
- Idempotency- Pipelines should be safely re-runnable without duplicating data
- CDC (Change Data Capture)- Technique for incrementally capturing only changed source rows instead of full reloads
Modeling Methodologies
Competing philosophies for structuring an enterprise warehouse beyond a single star schema.
- Kimball (dimensional)- Bottom-up, business-process-oriented conformed star schemas; optimized for query performance and analyst usability
- Inmon (CIF)- Top-down, normalized (3NF) enterprise data warehouse feeding subject-specific dimensional data marts
- Data Vault- Hubs (business keys), links (relationships), and satellites (descriptive/history attributes); highly auditable and resilient to source changes
- One Big Table (OBT)- Fully denormalized wide table trading storage/duplication for simplicity in modern columnar warehouses
- Conformed dimension- A dimension (e.g. dim_date) shared identically across multiple fact tables so metrics can be compared across business processes
- Bus matrix- Kimball planning artifact mapping business processes (rows) to shared conformed dimensions (columns)
SCD Type 2 via MERGE
Atomic upsert pattern for expiring and inserting dimension rows in one statement, avoiding race conditions between separate UPDATE/INSERT calls.
MERGE INTO dim_customer AS targetUSING staging_customer AS srcON target.customer_id = src.customer_id AND target.is_current = TRUEWHEN MATCHED AND (target.city <> src.city OR target.email <> src.email) THEN UPDATE SET target.end_date = CURRENT_DATE, target.is_current = FALSEWHEN NOT MATCHED THEN INSERT (customer_id, name, city, email, start_date, end_date, is_current) VALUES (src.customer_id, src.name, src.city, src.email, CURRENT_DATE, NULL, TRUE);-- Follow with a second INSERT for rows that changed, since MERGE can't-- both close the old row and insert a new one for the same key in one passINSERT INTO dim_customer (customer_id, name, city, email, start_date, end_date, is_current)SELECT src.customer_id, src.name, src.city, src.email, CURRENT_DATE, NULL, TRUEFROM staging_customer srcJOIN dim_customer d ON d.customer_id = src.customer_id AND d.is_current = FALSE AND d.end_date = CURRENT_DATE;
Fact Table Design Patterns
Beyond the basic transaction fact table: handling snapshots, accumulating processes, and factless events.
-- Periodic snapshot fact: one row per account per dayCREATE TABLE fact_account_snapshot ( snapshot_date_key INT, account_key INT, balance_amount NUMERIC(18,2), PRIMARY KEY (snapshot_date_key, account_key));-- Accumulating snapshot: one row per order, updated as it moves through stagesCREATE TABLE fact_order_pipeline ( order_key INT PRIMARY KEY, ordered_date_key INT, shipped_date_key INT, delivered_date_key INT, days_order_to_ship INT);-- Factless fact table: records an event/relationship with no measureCREATE TABLE fact_student_attendance ( date_key INT, student_key INT, class_key INT);
Warehouse Performance Concepts
How modern columnar/MPP warehouses (Snowflake, BigQuery, Redshift) achieve query speed at scale.
- Columnar storage- Stores each column contiguously so aggregate queries scan only the columns referenced, not full rows
- Partitioning- Physically splits a table by a key (often date) so queries prune irrelevant partitions instead of full scans
- Clustering / sort keys- Co-locates rows with similar values on disk to improve predicate pushdown and join performance
- Materialized view- Precomputed, incrementally refreshed query result stored as a table for fast repeat access
- Query result caching- Warehouse returns a cached result set instantly when an identical query runs again on unchanged data
- Micro-partition pruning- Metadata about min/max values per micro-partition lets the optimizer skip partitions that can't match a filter
Window Functions for Warehouse Analytics
Common analytical patterns run directly against fact tables without self-joins or subqueries.
-- Running total of sales per region ordered by dateSELECT d.date, f.region, SUM(f.sales_amount) OVER (PARTITION BY f.region ORDER BY d.date) AS running_total, RANK() OVER (PARTITION BY d.year ORDER BY SUM(f.sales_amount) DESC) AS region_rankFROM fact_sales fJOIN dim_date d ON f.date_key = d.date_keyGROUP BY d.date, f.region, d.year;-- Month-over-month change using LAGSELECT month, revenue, revenue - LAG(revenue) OVER (ORDER BY month) AS mom_changeFROM monthly_revenue;
Design fact tables around a single, well-documented grain before adding dimensions -- changing the grain later forces a rebuild of every downstream report.