Creating a dbt project

Introduction to dbt

Mike Metzger

Data Engineer

What is a dbt project

  • Encompass components for working with data in dbt

    • Project configuration
    • Data sources & destinations
    • SQL queries
    • Templates
    • Documentation

  • Implemented as a folder structure

dbt folder structure

Introduction to dbt

How to create a new project

  • Use the dbt init command
    • Asks the name of the project
    • Asks which database / data warehouse type
  • Can consolidate with dbt init <projectname>
  • Creates the top level project folder and all needed structure
repl:~/workspace$ dbt init
16:38:37  Running with dbt=1.8.4
Enter a name for your project (letters, digits, underscore): test_project
Which database would you like to use? [1] duckdb
Enter a number: 1
...
Introduction to dbt

Defining configuration with project profiles

  • A profile represents a given deployment scenario
    • Development
    • Staging / Test
    • Production
  • A dbt project can have multiple profiles
  • Defined in the profiles.yml file
marketing_campaign_results:
  outputs:
    dev:
      type: duckdb
      path: dbt.duckdb
    prod:
      type: snowflake
      ...
  target: dev
Introduction to dbt

YAML

  • Yet Another Markup Language
  • Text file, but spacing matters (like Python)
  • Used in many development scenarios for configuration
  • Rules can be tricky, mainly keep entries lined up as in examples
marketing_campaign_results:
  outputs:
    dev:
      type: duckdb
      path: dbt.duckdb
    prod:
      type: snowflake
      ...
  target: dev
Introduction to dbt

DuckDB

  • Open-source serverless database
    • Similar to sqlite
  • Designed for analytics
  • Vectorized (meaning FAST)
  • Easy to use

duckdb logo

Introduction to dbt

Let's Practice!

Introduction to dbt

Preparing Video For Download...