Templating with Jinja

Introduction to Apache Airflow in Python

Mike Metzger

Data Engineer

What is a template?

  • A defined format that allows Airflow to substitute information automatically
  • Often plain text / text files
  • Simplifies access and re-use

 

Illustration of a template with placeholders for substituted values

Introduction to Apache Airflow in Python

What is Jinja?

  • Simple text-based templating language
  • Used in many tools beyond Airflow
  • {{ ... }} represents a template substitution
    • Similar to a Python f-string
  • More dynamic usage, supports for loops

Illustration of a Jinja template using double-brace substitutions

Introduction to Apache Airflow in Python

Airflow Jinja functions

  • logical_date: When a Dag run occurs
  • ds: The logical_date in YYYY-MM-DD format
  • logical_date.year, logical_date.month, logical_date.day: Access the date components
  • params: The runtime parameters of the Dag run
  • Calculations, loops, and more
    • macros.ds_add: Allows date calculations
Introduction to Apache Airflow in Python

Example using SmtpNotifier

  • Substitutions in subject field
@dag(dag_id=`sales_etl`,
     on_failure_callback=SmtpNotifier(
         to='[email protected]',
         from_email='[email protected]',

         subject='sales_etl on {{ logical_date }} has failed!'
     )
)
  • Subject: sales_etl on 2026-04-30 failed!
Introduction to Apache Airflow in Python

Example using FileSensor

  • Configure the filepath dynamically
  • Look for files on the day before the ds (logical_date in YYYY-MM-DD )
  • Use macros.ds_add
    filewatcher = FileSensor(
task_id="wait_for_files",
filepath="/data/{{ macros.ds_add(ds, -1) }}/input.csv",
...
)
  • If ds is 2026-04-30, the filepath would be /data/2026-04-29/input.csv
Introduction to Apache Airflow in Python

Jinja notes

  • Can't be used everywhere
  • Look at the documentation for templated attributes
  • Check the code view

Airflow Dag code view showing the read-only Python source that defines the Dag

Introduction to Apache Airflow in Python

Let's practice!

Introduction to Apache Airflow in Python

Preparing Video For Download...