dbt™ Models

Last updated on Jan 20, 2023
On This Page

At this time, the dbt Core™ on Hevo is in BETA. Please reach out to Hevo Support or your account executive to enable it for your team.

dbt™ (Data Building Tool) is a transformation tool that enables you to view and consume the data loaded to your Destination tables into a form conducive to performing analytics and reporting by BI tools.

Hevo integrates with dbt Core™, the open-source CLI offering from dbt. In dbt, you create models containing SQL SELECT statements to query the data from the tables in your Destination. dbt compiles and runs these models against the tables specified by the query. It creates the target tables and views in your Destination, using Jinja, its template engine, and loads the results to them.

dbt in Hevo

With Hevo’s dbt functionality, you can:

  • Configure dbt projects in Hevo for the following Destination types:

    • BigQuery

    • Snowflake

    • Amazon Redshift

    • Databricks

    • PostgreSQL

Read Configuring a dbt Project for the steps to do this.

  • Connect to your existing dbt projects created in any Git provider such as Bitbucket, GitHub, or GitLab.

    Note: You should have at least read-only access to the Git repository.

  • Run the dbt models on your Destination data.

  • Configure a dbt project to run models on a specific branch in your Git repository. For example, to test your created models.

  • Schedule your dbt project to:

    • Run all of the models in your project together,

    • Run each model individually, or

    • Trigger your dbt project whenever Events are loaded to a specific Destination table.

  • View the model execution history and the complete activity log for the dbt project.

  • Integrate your dbt projects with Hevo Workflows.

All the dbt projects that you and your team create are hosted in a Git repository, thus allowing you to control and record the changes made to any dbt project.

dbt Projects and Core

Read Setting up dbt Projects in Git for the steps to create a sample dbt project.

A dbt project typically includes these resources:

  • Models: Models are the .sql files containing the SQL SELECT statements to transform your data into a form ready for analytics or as input to another model. One SQL file can contain only one model.

  • Seeds: Seeds are CSV files containing data that augments the existing data in your data warehouse tables. For example, a CSV file that maps a country’s code to its name.

    You should not use seeds to load data containing sensitive personal information or large amounts of data.

  • Snapshots: Snapshots are means by which you can capture data change in your tables. For example, you can create a snapshot of the orders table to capture the date and time when the status of the order changes.

  • Tests: Tests are SQL queries that you can write to test the models in your project.

  • Macros: Macros are pieces of Jinja code that can be reused multiple times in your models. Macros are defined in .sql files and are similar to functions in other programming languages.

  • Sources: Sources describe the data warehouse tables that contain the data to be transformed. You can list the schema name and table names in the sources.yml file and reference them in your models.

  • Docs: Docs contain the documentation files for your dbt project, which you can render as a website.

  • Analyses: Analyses contain SQL files that you want dbt to compile but not execute. For example, suppose you want to test an ad-hoc query and see the results of that query in your BI tool. For this, you can create an analysis file and compile the query in dbt. You can then run the compiled SQL file in your BI tool to see the results.

    The compiled files enable you to select from models independent of your project environment.

The following is the directory structure of a sample dbt project called jaffle_shop, within the Git repository.

dbtProjectStructure



Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Nov-25-2022 NA New document.

Tell us what went wrong