Practical dbt: From Basics to Advanced
In this lesson, you’ll learn how to:
-
Install dbt locally using a Python virtual environment
-
Connect dbt to PostgreSQL using a profile configuration
-
Run a few basic dbt commands to validate the setup
By the end of this lesson, you’ll have a complete local environment ready to explore dbt and start building your first transformations.
Prerequisites
If you plan to follow along and practice, ensure you have the following are installed on your system:
Provision a local PostgreSQL (Docker)
In this step, we will provision a PostgreSQL database that runs locally. The following command will start a PostgreSQL instance using Docker, making it accessible on a port 5432 which is essential for the setup of dbt local postgresSQL.
docker run --name db-postgres \
-e POSTGRES_USER=dbt_user \
-e POSTGRES_PASSWOR=dbt_pass \
-e POSTGRES_DB=dbt_db \
-p 5432:5432 \
-d postgres
Create and activate a Python virtual environment
Below, you’ll create a Python virtual environment named dbt-env (or you can choose any name).
python -m venv dbt-env
.\dbt-env\Scripts\activate
Once activated, your terminal prompt should show the environment name (e.g., dbt-env).
python -m venv dbt-env
source dbt-env/bin/activate
Upgrade pip and essential tools (Optional)
After activating your virtual environment, it’s a good practice to upgrade pip and a few core packaging tools. This helps avoid installation issues and ensures smoother dependency resolution.
python -m pip install --upgrade pip setuptools wheel
Install dbt-core and postgres adpater
With your environment ready, install dbt-core along with the postgres adapter. The adapter is what enables dbt to connect to and run models on Postgres.
pip install dbt-core==1.10.13 dbt-postgres==1.9.1
Verify Installation
Once the installation is complete, run the following command to verify that dbt is installed correctly.
dbt --version
The command output will look like this:
Core:
- installed: 1.10.13
- latest: 1.10.13 - Up to date!
Plugins:
- postgres: 1.9.1 - Up to date!
Setting up a dbt project
The first step when working with dbt is to initialize a project by providing its name to the init command. Run the following command to create a project named ecommerce_project.
dbt init ecommerce_project -s
The -s option is used to skip providing PostgreSQL database information interactively. This will be covered in the next section, where we’ll explore how to connect dbt with PostgreSQL.
When the command finishes you should get a project with the sturcture bellow:
ecommerce_project/
│── analyses/
│── macros/
│── models/
│── seeds/
│── snapshots/
│── tests/
│── dbt_project.yml
│── README.md
Connect dbt to PostgreSQL database
To connect dbt to a PostgreSQL database, you’ll create a file named profiles.yml. This file contains the connection details for your data platform. When you run dbt Core, it checks the profile name defined in dbt_project.yml, then looks up the matching entry in profiles.yml to establish the connection.
Inside the profiles.yml file, under the dev target, provide the PostgreSQL database details provisioned in the previous section, or use your own database credentials if you’re not using the one provisioned above.
ecommerce_project:
target: dev
outputs:
dev:
type: postgres
host: localhost
user: dbt_user
password: dbt_pass
port: 5432
dbname: postgres
schema: dbttest
threads: 1
Test the Connection
To test the connection between dbt and postgres database, run the command below:
dbt debug
If the connection is established successfully, you should see the following messages: [OK connection ok] and [All checks passed!].
Conclusion
In this tutorial, we learned how to set up dbt locally using a Python virtual environment. We explored the essential folders in the dbt project structure and concluded by connecting dbt to a PostgreSQL database. We hope you find this guide helpful for your setup dbt local postgresSQL journey. Now, it’s time to get hands-on with dbt’s capabilities.
Let’s move on to the next tutorial: Explore dbt models.
Finish Course Early?
You have not completed all required lessons and assessments.