dbt Packages

Analogy categoryAnalogies

What a Package Is and How to Install One

A dbt package is just another dbt project — with its own models, macros, tests, and seeds — that you install into your own project so you can call its macros or build on top of its models without copy-pasting code. You declare dependencies in a packages.yml file at your project root, listing each package's source (dbt Hub, a Git URL, or a local path) and version, then run dbt deps to download them into the dbt_packages/ directory.

🏏

Cricket analogy: This is like a franchise signing an overseas fast bowler on loan for a season rather than spending years developing an equivalent bowler from its own academy — you get proven capability by 'installing' outside talent.

yaml

# packages.yml
packages:
  - package: dbt-labs/dbt_utils
    version: [">=1.1.0", "<2.0.0"]
  - package: calogica/dbt_expectations
    version: [">=0.10.0", "<0.11.0"]
  - git: "https://github.com/my-org/dbt-internal-macros.git"
    revision: v2.3.0

Using Package Macros and Tests

The most widely used package, dbt_utils, provides macros like generate_surrogate_key (hashing multiple columns into one deterministic key), date_spine (generating a gap-free calendar table), and pivot, all called with a package prefix like dbt_utils.generate_surrogate_key. dbt_utils and dbt_expectations also ship pre-built generic tests — dbt_utils.equal_rowcount, dbt_utils.not_accepted_values, dbt_expectations.expect_column_values_to_be_between — so you get statistically rigorous data quality checks referenced from schema.yml exactly like a built-in test.

🏏

Cricket analogy: This is like a franchise's analytics team using a licensed, industry-standard win-probability model instead of building their own from scratch, called by a simple function whenever needed during a broadcast.

sql

-- models/marts/fct_orders.sql
select
    {{ dbt_utils.generate_surrogate_key(['order_source', 'source_order_id']) }} as order_id,
    order_date,
    customer_id,
    amount
from {{ ref('stg_orders') }}

yaml

# schema.yml — using a package-provided generic test
columns:
  - name: amount
    tests:
      - dbt_utils.accepted_range:
          min_value: 0
          max_value: 100000

Version Pinning and Package Management

Version ranges in packages.yml, like [">=1.1.0", "<2.0.0"], protect you from a breaking major-version upgrade silently changing macro behavior underneath your project, while still letting patch and minor updates flow in automatically on the next dbt deps run. package-lock.yml, generated automatically alongside dbt deps, pins the exact resolved version actually installed, giving your team and CI environment a reproducible, identical set of package versions the same way a lockfile does in npm or pip.

🏏

Cricket analogy: This is like a franchise locking in a specific season's playing conditions regulations rather than automatically adopting every mid-season ICC rule change, while still applying agreed minor clarifications as they're issued.

Always commit package-lock.yml to version control alongside packages.yml — without it, two developers (or your local machine versus CI) could resolve slightly different package versions on dbt deps, leading to subtle, hard-to-reproduce test or macro behavior differences.

Private packages referenced via a git: URL with a plain HTTPS path will fail to clone in CI environments without stored credentials — use an SSH URL or a token-embedded HTTPS URL, and never commit that token directly into packages.yml; use environment variable substitution instead.

A dbt package is a full dbt project (models, macros, tests) installed into yours via packages.yml and dbt deps.
Packages can come from the dbt Hub, a public/private Git URL, or a local filesystem path.
dbt_utils and dbt_expectations are the most widely used packages, providing macros and pre-built generic tests.
Package macros are called with a namespace prefix, e.g. dbt_utils.generate_surrogate_key or dbt_utils.date_spine.
Package-provided tests are referenced from schema.yml with the same prefix, e.g. dbt_utils.accepted_range.
Version ranges in packages.yml prevent breaking major upgrades while allowing safe patch/minor updates.
package-lock.yml pins exact resolved versions and should be committed for reproducible installs across environments.