Apache Airflow: From Basics to Mastery
In this lesson, you’ll learn what tasks and operators are in Apache Airflow, how they work together, and how to use them to build workflows.
What is a Task?
A task is the smallest unit of work in Airflow.
Each task represents one step in a workflow (DAG).
It has upstream and downstream dependencies set between them in order to express the order they should run in.
There are three basic kinds of Task:
Operator
- Sensor
- A TaskFlow-decorated
What is an Operator?
An operator is a template that tells Airflow what kind of task to run.
When you use an operator in a DAG, it becomes a task instance.
Tip
Think of it like this:
Operator = Blueprint (what the task does).
Task = Real execution of that operator inside the DAG.
Types of Operators
Airflow provides many built-in operators, such as:
PythonOperator – Runs a Python function.
BashOperator – Executes a Bash command/script.
EmailOperator – Sends an email.
SqlOperator – Runs SQL queries against a database.
HttpOperator – Makes API requests.
DummyOperator – Placeholder for workflow structure.
Tip
You can also create Custom Operators when built-ins don’t fit your needs.
Tasks and Operators in Action
from airflow import DAG
from airflow.operators.python import PythonOperator
from datetime import datetime
# Define a function for the Python task
def greet():
print("Hello, Airflow learner!")
# Define the DAG
with DAG(
dag_id="example_dag",
start_date=datetime(2025, 1, 1),
schedule_interval="@daily",
catchup=False
) as dag:
# Create a task using PythonOperator
task1 = PythonOperator(
task_id="greet_task",
python_callable=greet
)
Here:
PythonOperator= Operator (the blueprint).task1= Task instance created from that operator.
Task Dependencies
Tasks can be connected using
>>or<<operators.
task1 >> task2
Lesson Summary
Tasks = Units of work in a DAG.
Operators = Blueprints that define what the task does.
Airflow provides many operators (Python, Bash, SQL, API, Email).
Tasks created from operators can be linked with dependencies to build workflows.
Finish Course Early?
You have not completed all required lessons and assessments.