Branching

Introduction to Apache Airflow in Python

Mike Metzger

Data Engineer

Branching

Branching in Airflow:

Provides conditional logic
Using BranchPythonOperator
from airflow.operators.python import BranchPythonOperator
Takes a python_callable to return the next task id (or list of ids) to follow

Branching example

def branch_test(**kwargs):
  if int(kwargs['ds_nodash']) % 2 == 0:
    return 'even_day_task'
  else:
    return 'odd_day_task'

Branching example

def branch_test(**kwargs):
  if int(kwargs['ds_nodash']) % 2 == 0:
    return 'even_day_task'
  else:
    return 'odd_day_task'


branch_task = BranchPythonOperator(task_id='branch_task',dag=dag,
       provide_context=True,
       python_callable=branch_test)


start_task >> branch_task >> even_day_task >> even_day_task2

branch_task >> odd_day_task >> odd_day_task2

Branching graph view

Branching DAG graph view

Branching even days

Branching graph view - Even days

Branching odd days

Branching graph view - odd days

Let's practice!

Introduction to Apache Airflow in Python