Introduction to Apache Airflow in Python
Mike Metzger
Data Engineer
Branching in Airflow:
BranchPythonOperator
from airflow.operators.python import BranchPythonOperator
python_callable
to return the next task id (or list of ids) to followdef branch_test(**kwargs):
if int(kwargs['ds_nodash']) % 2 == 0:
return 'even_day_task'
else:
return 'odd_day_task'
def branch_test(**kwargs): if int(kwargs['ds_nodash']) % 2 == 0: return 'even_day_task' else: return 'odd_day_task'
branch_task = BranchPythonOperator(task_id='branch_task',dag=dag, provide_context=True, python_callable=branch_test)
start_task >> branch_task >> even_day_task >> even_day_task2
branch_task >> odd_day_task >> odd_day_task2
Introduction to Apache Airflow in Python