Introduction to Apache Airflow in Python
Mike Metzger
Data Engineer
Templates:
Jinja
templating languageCreate a task to echo a list of files:
t1 = BashOperator( task_id='first_task', bash_command='echo "Reading file1.txt"', dag=dag)
t2 = BashOperator( task_id='second_task', bash_command='echo "Reading file2.txt"', dag=dag)
templated_command=""" echo "Reading {{ params.filename }}" """
t1 = BashOperator(task_id='template_task', bash_command=templated_command, params={'filename': 'file1.txt'}, dag=example_dag)
Output:
Reading file1.txt
templated_command=""" echo "Reading {{ params.filename }}" """
t1 = BashOperator(task_id='template_task', bash_command=templated_command, params={'filename': 'file1.txt'}, dag=example_dag)
t2 = BashOperator(task_id='template_task', bash_command=templated_command, params={'filename': 'file2.txt'}, dag=example_dag)
Introduction to Apache Airflow in Python