Creating and generating dbt documentation

Introduction to dbt

Mike Metzger

Data Engineer

Why document?

  • Sharing data details with other consumers

 

  • Centralize sources of documentation

 

  • Providing details for updates / changes / etc

 

  • Creating examples, suggestions for use, SLA details

Documents

Introduction to dbt

Creating documentation in dbt

  • Can provide documentation with model definitions
  • Can add documentation about columns within models
  • Automatically show data lineage / DAG
  • Document any test / validations
  • View generated warehouse information
    • Column data types
    • Data sizes
version: 2

models:
  - name: taxi_rides_raw
    description: Yellow Taxi raw data
    access: public
  - name: avg_fare_per_day
    description: Average ride per day
    access: public
Introduction to dbt

Generating documentation in dbt

  • dbt docs
    • dbt docs -h
    • dbt docs generate
  • Creates the documentation website based on project
  • Should be run after dbt run
Introduction to dbt

Accessing documentation

  • Web browser
  • dbt docs serve
    • Should only be used locally / for development
  • Copy content to other hosting service
    • dbt cloud
    • Amazon S3
    • Nginx / Apache / etc

dbt documentation view

Introduction to dbt

Documentation example

  • View
    • Models
    • Description information
    • Column details
    • Lineage graphs

dbt documentation view

Introduction to dbt

Let's practice!

Introduction to dbt

Preparing Video For Download...