Core Concepts Questions
Interviewers frequently start with fundamentals: what is a DAG, what is the difference between an operator and a task, and what is XCom used for. A DAG (Directed Acyclic Graph) is the definition of a workflow's structure, a set of tasks and their dependencies with no cycles; an operator is a template describing a single unit of work (like PythonOperator or BashOperator), while a task is a specific instantiation of an operator within a DAG, and a task instance is one run of that task for a particular execution date. Being able to clearly articulate this operator-vs-task-vs-task-instance hierarchy, and explain why cycles aren't allowed (the scheduler couldn't determine a valid execution order), signals genuine hands-on Airflow experience rather than memorized definitions.
Cricket analogy: A DAG is like a full match's fixture and batting order plan; an operator is like the general role of 'opening batsman'; a task is a specific named player assigned that role for this match; and a task instance is that player's actual innings played on a specific date.
Scheduling and Execution Questions
A very common trick question is: 'If a DAG has start_date=2026-01-01 and schedule="@daily", when does the first run actually execute?' The answer trips up many candidates: Airflow schedules a run only after the interval has fully elapsed, so the run for the Jan 1 interval (covering Jan 1 00:00 to Jan 2 00:00) actually triggers at the end of that period, on Jan 2, and its execution_date/logical_date is stamped Jan 1, representing the start of the interval it covers, not when it ran. Interviewers also probe understanding of depends_on_past, which when True blocks a task instance from running until the same task succeeded in the previous DagRun, useful for strictly sequential aggregations but dangerous if a single historical failure can permanently stall the pipeline.
Cricket analogy: The 'execution_date represents interval start, not trigger time' concept is like a day-night match officially dated for the day it started even though the presentation ceremony happens well after midnight.
Architecture Questions
Expect questions on Airflow's core components: the scheduler (parses DAGs and decides what should run), the executor (determines how and where tasks physically execute, e.g., LocalExecutor, CeleryExecutor, KubernetesExecutor), the metadata database (stores DAG state, task instance state, connections, variables), and the webserver (serves the UI). A strong follow-up is explaining the difference between a Sensor and an Operator: a Sensor is a special operator subclass that polls for a condition to become true (a file arriving, a partition existing, an external DagRun finishing) before allowing downstream tasks to proceed, and modern Airflow encourages using mode="reschedule" on sensors instead of the default mode="poke" so the sensor releases its worker slot between poll attempts rather than blocking it for the entire wait.
Cricket analogy: A Sensor is like a twelfth man watching the boundary rope for a ball to cross before signaling 'four runs, play on,' repeatedly checking rather than doing the batting itself, unlike a normal operator that does the actual hitting.
Scenario-Based Questions
A common scenario question: 'A DAG has been failing for the past 3 days due to an upstream schema change, and now that it's fixed you need to backfill the missed runs without waiting for new schedules.' The expected answer covers airflow dags backfill -s 2026-07-07 -e 2026-07-10 dag_id, understanding that backfill creates DagRuns for each missed interval and respects max_active_runs so it doesn't overwhelm the system, and confirming the tasks are idempotent so replaying those dates is safe. Another common scenario tests understanding of max_active_runs and max_active_tasks (pool/concurrency limits): if a downstream system can only handle 2 concurrent connections, a candidate should suggest a custom Pool with 2 slots assigned to the relevant tasks, rather than throttling the entire DAG's parallelism globally.
Cricket analogy: Backfilling missed DagRuns is like rescheduling three washed-out league matches on later dates, using the same fixed rules and squad as if they'd been played on time, rather than skipping them entirely.
# Sensor with reschedule mode releases the worker slot between polls
from airflow.sensors.filesystem import FileSensor
wait_for_file = FileSensor(
task_id="wait_for_upstream_file",
filepath="/data/incoming/orders.csv",
poke_interval=60,
timeout=60 * 60 * 4,
mode="reschedule",
)
# CLI: backfill three missed daily runs after fixing an upstream bug
# airflow dags backfill -s 2026-07-07 -e 2026-07-10 orders_etlWhen explaining execution_date/logical_date in an interview, always tie it to the interval it represents, not the wall-clock time the task actually ran. This single concept trips up more candidates than any other Airflow scheduling detail.
Be careful recommending depends_on_past=True in interview scenarios without caveats. It can permanently stall a pipeline if one historical run fails and is never manually cleared, since every subsequent run will refuse to start.
- Know the operator vs. task vs. task instance hierarchy precisely; interviewers use it to gauge hands-on experience.
- Understand that execution_date/logical_date represents the start of the covered interval, not the actual trigger time.
- Be able to explain depends_on_past and its risk of permanently stalling a pipeline after one historical failure.
- Know the role of each core component: scheduler, executor, metadata database, and webserver.
- Explain Sensors vs. Operators, and why mode='reschedule' is preferred over mode='poke' for long waits.
- Be ready to design solutions for backfills (airflow dags backfill) and concurrency control (custom Pools).
- Ground every answer in a concrete example DAG or CLI command rather than abstract definitions alone.
Practice what you learned
1. What is the correct relationship between an operator, a task, and a task instance?
2. For a DAG with start_date=2026-01-01 and schedule='@daily', what does the execution_date of the first run represent?
3. What risk does depends_on_past=True introduce?
4. Why is mode='reschedule' generally preferred over mode='poke' for a long-running Sensor?
5. If a downstream API can only handle 2 concurrent connections, what is the best way to enforce that limit in Airflow?
Was this page helpful?
You May Also Like
Airflow Best Practices
Practical guidelines for writing reliable, maintainable Airflow DAGs, covering idempotency, DAG design, dependency management, and testing.
Building an ETL DAG
A hands-on walkthrough of designing and implementing a complete Extract-Transform-Load pipeline as an Airflow DAG, including failure handling and local testing.
Airflow Quick Reference
A condensed cheat sheet of essential Airflow CLI commands, common operators/sensors, scheduling syntax, and key configuration settings.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingApache Flink Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics
ProgrammingPowerShell Study Notes
Programming · 30 topics