diff --git a/kedro-airflow/README.md b/kedro-airflow/README.md index 1a698620a..ff4e4610d 100644 --- a/kedro-airflow/README.md +++ b/kedro-airflow/README.md @@ -56,30 +56,43 @@ You can use the additional command line argument `--jinja-file` (alias `-j`) to kedro airflow create --jinja-file=./custom/template.j2 ``` +Similarly, the included DAG `kwargs` template can be custom provided: + +```bash +kedro airflow create --kwargs-template=./custom/kwargs_template.j2 +``` + #### How can I pass arguments to the Airflow DAGs dynamically? `kedro-airflow` picks up configuration from `airflow.yml` files. Arguments can be specified globally, or per pipeline: ```yaml -# Example of global arguments for Airflow DAGs -kwargs: | - start_date=datetime(2023, 1, 1), - max_active_runs=3, +# Global parameters +default: + start_date: [2023, 1, 1] + max_active_runs: 3 # https://airflow.apache.org/docs/stable/scheduler.html#dag-runs - schedule_interval="@once", - catchup=False, + schedule_interval: "@once" + catchup: false # Default settings applied to all tasks - default_args=dict( - owner="airflow", - depends_on_past=False, - email_on_failure=False, - email_on_retry=False, - retries=1, - retry_delay=timedelta(minutes=5) - ), - -# Example configuration per pipeline (overrides `kwargs` from above) + default_args: + owner: "airflow" + depends_on_past: false + email_on_failure: false + email_on_retry: false + retries: 1 + retry_delay: 5 + +# Arguments specific to the pipeline (overrides the parameters above) data_science: - kwargs: | - schedule_interval="@once", + default_args: + owner: "airflow-ds" ``` + +Arguments can also be passed via `--params` in the command line: + +```bash +kedro airflow create --params "schedule_interval='@weekly'" +``` + +These variables are passed to the Jinja2 template.