In this lesson, you’ll learn what a DAG (Directed Acyclic Graph) is in Apache Airflow, why it is the foundation of every workflow, and how tasks are organized inside a DAG.

What is a DAG?

  • A DAG (Directed Acyclic Graph) is a collection of tasks with dependencies between them.

  • It tells Airflow what tasks to run and in what order.

  • DAGs are written in Python code.

Breaking Down the Term

  • Directed : Each edge (arrow) points in one direction (e.g. Task A → Task B).
  • Acyclic : No cycles or loops are allowed (Task A → Task B → Task A ❌).
  • Graph: A set of nodes (tasks) connected by edges (dependencies).

👉 This ensures workflows run in a logical order and don’t get stuck in infinite loops.

DAG Anatomy

  • DAG object: Holds metadata (dag_id, schedule_interval, start_date, catchup, default_args).
  • Tasks: Instances of Operators (BashOperator, PythonOperator, etc.).
  • Dependencies: Set using >>,  <<, or .set_upstream()/ .set_downstream()
  • TaskInstance: A single execution of a task for a specific DAG run.
  • DAG Run: An instance of the DAG for a particular schedule or manual trigger.

DAG Parameters

Every DAG has a few essential parameters:

  • dag_id : Unique identifier for the DAG.

  • start_date : The date from which the DAG starts running.

  • schedule : How often it should run (daily, hourly, weekly, etc.).

  • catchup : Controls whether to backfill past runs.

  • default_args: Common settings shared across tasks.

Creating a DAG in Airflow

There are three main ways to declare a DAG:

1. Context Manager (Recommended)

from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime, timedelta

default_args = {
    "owner": "data-team",
    "retries": 1,
    "retry_delay": timedelta(minutes=5),
}

with DAG(
    dag_id="example_ctx_dag",
    default_args=default_args,
    schedule_interval="@daily",
    start_date=datetime(2024, 1, 1),
    catchup=False,
    tags=["example"],
) as dag:
    t1 = BashOperator(task_id="print_date", bash_command="date")
    t2 = BashOperator(task_id="sleep", bash_command="sleep 5")
    t1 >> t2

2. Explicit DAG Assignment

from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime

dag = DAG("explicit_dag", start_date=datetime(2024,1,1), schedule_interval="@daily")

t1 = BashOperator(task_id="t1", bash_command="echo 1", dag=dag)
t2 = BashOperator(task_id="t2", bash_command="echo 2", dag=dag)

3. @dag Decorator (Airflow 2.x+)

from airflow.decorators import dag
from airflow.operators.empty import EmptyOperator
from datetime import datetime


# Define Dag using taskflow
@dag(start_date=datetime(2025, 3, 1), schedule="@daily")
def generate_dag():
    EmptyOperator(task_id="task")


generate_dag()

INFO

When using a decorator method to create a DAG, if dag_id is not provided, it defaults to the name of the function defining the DAG. In the example above, since we did not specify a dag_id, the DAG name will be generate_dag_decorator.

Lesson Summary

  • A DAG is the backbone of Airflow workflows.

  • It is Directed (tasks flow forward), Acyclic (no loops), and a Graph (tasks + dependencies).

  • DAGs are defined in Python, and tasks are placed inside them.

  • Dependencies determine the order in which tasks run.

PreviousNext