Setting up dbt™ Projects in Git

Last updated on Mar 14, 2024

At this time, the dbt Core™ on Hevo is in BETA. Please reach out to Hevo Support or your account executive to enable it for your team.

A dbt project contains the models that dbt runs to transform the data in your data platforms. Each project contains a configuration file called dbt_project.yml, which is created by dbt for your project. This configuration file helps dbt identify the dbt project directory and also how dbt should run the project. If you do not have a dbt project, follow the steps in this page to create one.


Perform the following steps to create a dbt project and link it to your Git repository. These steps use GitHub as the Git provider.

Prerequisites


1. Create a Git repository

  1. Log in to your GitHub account, and create a new GitHub repository. For example, dbt-sample-repository.

  2. Change the repository visibility to Private if you do not want your repository to be publicly available. Default value: Public.

  3. Retain all the default values for the other settings, and click Create repository.

  4. Copy the displayed commands and the HTTPS URL of your new repository. You need this information to link your new repository to your dbt project.

2. Set up your dbt project

  1. In a Terminal window, enter the following command to create your dbt project. This creates sample models for you by default. For example, the following command creates for you:

    • The dbt project, first_dbt_project.

    • The models, my_first_dbt_model.sql and my_second_dbt_model.sql.

      $ dbt init first_dbt_project
      
  2. Navigate to your project directory. For example, /home/harry.potter/first_dbt_project.

  3. Edit the dbt_project.yml file in your project directory, and if required, update the values for name, profile, and models to your project name.

3. Connect dbt Core to your data platform

dbt connects to your data platform using a profile, which is a YAML file and contains the connection details for your data platform.

  1. Open a Terminal window and navigate to the profile directory called .dbt, which is a hidden folder in your home directory. The profiles.yml file is automatically created in this location when you initiate your dbt project. For example, /home/harry.potter/.dbt/profiles.yml.

    Note: Hevo does not use this profiles.yml file. Instead, it generates the file from the Destination configured in your dbt Project in Hevo.

  2. Edit the profiles.yml file to add the connection details for your data platform. The following is a sample profiles.yml created for a PostgreSQL database.

    Sample Profile File

    Note:

    • Replace the values enclosed in square brackets ([…]) with your own. For example, [port] with 5432.
    • Ensure that the project name in your profiles.yml file matches the project name in your dbt_project.yml file.
  3. Navigate to your dbt project directory, and enter the following command to verify that dbt can connect to your data platform:

    $ dbt debug
    

    You should see the following output:

    Connection test: [OK connection ok]
    
    All checks passed!
    
  4. Enter the following command to run the models defined in the dbt project:

    dbt run
    

    You should see the following output:

    Project Run

  1. Open a Terminal window and navigate to your dbt project directory. For example, /home/harry.potter/first_dbt_project.

  2. Enter the following commands to link the GitHub repository you created in the Create a Git repository step:

    git init
    git branch -M main
    git add .
    git commit -m "Create a dbt project"
    git remote add origin https://github.com/<username>/dbt-sample-repository.git
    git push -u origin main
    

    Note: Replace the placeholder values in the command above with your GitHub username.

  3. Connect to your Git repository. You should see the dbt project you created.



Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Mar-15-2024 NA Added clarification about Hevo generating the profiles.yml file.
Nov-25-2022 NA New document.

Tell us what went wrong