Automating with dbt build

Intermediate dbt

Mike Metzger

Data Engineer

Review

  • sources and seeds feed initial data to dbt
  • models handle the transformation of data (usually from sources / seeds) for downstream users
  • tests can validate sources, seeds, models
    • Built-in (unique, not_null, relation, accepted_values)
    • Singular
    • Generic / Reusable
  • dbt build performs all these tasks, usually in production
Intermediate dbt

dbt build

dbt build:

  • Combination of multiple tasks
  • Runs models (dbt run)
  • Run validations via tests (dbt test)
  • Process any seeds (dbt seed)
  • Other options available (dbt snapshot)
  • DOES NOT perform dbt docs operations

Commands can be run individually

Building

1 Photo by Randy Fath on Unsplash
Intermediate dbt

dbt build - why?

  • Individual subcommands work well, but don't handle all potential issues
    • dbt run doesn't validate first (ie, no tests are run)
    • dbt seed may not be complete for certain queries
  • dbt build will determine dependencies and run all tests prior to production changes
  • dbt build may be overkill if only testing or small incremental changes are made
  • Steps can be run manually instead if required
Intermediate dbt

dbt build options

  • dbt build --select <object>
  • dbt build -d
  • dbt build --exclude <object>
Intermediate dbt

Let's practice!

Intermediate dbt

Preparing Video For Download...