Installing dbt
dbt Core is distributed as a set of Python packages: a base dbt-core package plus a database-specific adapter package such as dbt-snowflake, dbt-bigquery, dbt-redshift, or dbt-postgres. You install both together with pip, ideally inside an isolated Python virtual environment so dbt's dependencies don't collide with other projects on the same machine, and the adapter you install determines which warehouse dbt can connect to and which warehouse-specific SQL features (like Snowflake's QUALIFY clause) are available in your models.
Cricket analogy: Choosing the right dbt adapter is like a batter selecting the correct bat willow — English willow for Test cricket conditions versus Kashmir willow for local games — the base skill is the same but the equipment must match the pitch.
# Create and activate an isolated environment, then install dbt with the Snowflake adapter
python3 -m venv dbt-env
source dbt-env/bin/activate
pip install dbt-core dbt-snowflake
# Confirm installation and see which adapters are available
dbt --versionCreating a New Project
Once dbt is installed, running dbt init <project_name> scaffolds a new project directory with the standard folder structure — models/, seeds/, snapshots/, macros/, tests/, and analyses/ — plus a dbt_project.yml file that declares the project name, dbt version constraints, and default configuration for each folder. The init command also walks you interactively through creating a connection profile, prompting for warehouse credentials and writing them into profiles.yml so that the newly scaffolded project has something to connect to right away.
Cricket analogy: Running dbt init is like a franchise setting up a new IPL team — it lays out the standard structure (squad, coaching staff, kit) that every team needs before a single match is played.
dbt init jaffle_shop
# Prompts interactively for:
# 1) which database adapter to use (e.g. snowflake)
# 2) account, user, password/role, warehouse, database, schema
# Result: a jaffle_shop/ directory plus an entry in ~/.dbt/profiles.ymlprofiles.yml
profiles.yml stores the actual warehouse connection details — account identifiers, usernames, passwords or keys, and the target schema — and by default it lives outside the project directory in ~/.dbt/profiles.yml rather than inside the git-tracked project folder. This separation exists deliberately so that credentials never get committed to version control: the dbt_project.yml file inside your repo simply references a profile name, and each developer or CI environment supplies its own profiles.yml with its own credentials pointing at that same profile name.
Cricket analogy: profiles.yml living outside the repo is like a player's personal locker combination being kept separately from the team's shared playbook — everyone follows the same playbook but authenticates with their own credentials.
Never commit profiles.yml or any file containing warehouse passwords, private keys, or API tokens to git. Keep it in ~/.dbt/profiles.yml (outside the repo) or inject credentials via environment variables using Jinja's env_var() function, and add any local override files to .gitignore.
- Install dbt with pip, pairing dbt-core with a warehouse-specific adapter like dbt-snowflake or dbt-bigquery.
- Use a Python virtual environment to isolate dbt's dependencies from other projects.
- dbt init <project_name> scaffolds the standard folder structure and dbt_project.yml.
- dbt init also interactively creates an entry in profiles.yml for your warehouse connection.
- profiles.yml stores actual credentials and lives outside the version-controlled project by default (~/.dbt/profiles.yml).
- Never commit credentials — use env_var() in Jinja to inject secrets from environment variables instead.
- dbt --version confirms the installed dbt-core version and which adapters are available.
Practice what you learned
1. What must you install alongside dbt-core to connect to Snowflake?
2. What does the dbt init command create?
3. Where does profiles.yml live by default, and why?
4. What is the recommended way to supply secrets to profiles.yml without hardcoding them?
Was this page helpful?
You May Also Like
What Is dbt?
An introduction to dbt (data build tool), the SQL-first framework that lets analysts and engineers transform data already loaded in a warehouse using version-controlled, testable code.
The dbt Project Structure
A tour of a dbt project's folders and configuration files, and the staging/intermediate/marts convention used to organize models.
Connecting to a Warehouse
How dbt uses adapters and profiles.yml to authenticate against a data warehouse, and how to verify the connection with dbt debug.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
ProgrammingPowerShell Study Notes
Programming · 30 topics