Introduction to Apache Airflow in Python
Mike Metzger
Data Engineer
# New way, Airflow 2.x+
EmptyOperator(task_id='example')
# Old way, Airflow <2.0
EmptyOperator(task_id='example', dag=dag_name)
BashOperator(
task_id='bash_example',
bash_command='echo "Example!"',
# Next line only for Airflow before version 2
dag=dag
)
BashOperator(
task_id='bash_script_example',
bash_command='runcleanup.sh',
)
from airflow.operators.bash import BashOperator
example_task = BashOperator(task_id='bash_ex', bash_command='echo 1', )
bash_task = BashOperator(task_id='clean_addresses',
bash_command='cat addresses.txt | awk "NF==10" > cleaned.txt',
)
Introduction to Apache Airflow in Python