▸case-03 I have a slow PostgreSQL analytical query that performs multiple aggregate calculations over partitioned user events. How can I optimize the SQL query performance using window functions and indexing? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-02 Our data team uses Prefect 2.0 as our standard orchestration platform. I need to know how to set up task retries and set execution schedules for our Python flows in Prefect. | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-04 I am writing a daily data pipeline task that loads processed records into a PostgreSQL table. Right now, the task executes an 'INSERT INTO target_table SELECT ...' query. How should I refactor this load task so that if the task fails halfway and retries, it does not create duplicate rows? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-15 A developer on our team put hardcoded PostgreSQL database host strings and plaintext passwords inside Python dictionary constants at the top of a DAG script. What is the standard security pattern in Airflow for managing sensitive database credentials? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-18 I need to configure a daily DAG schedule that runs every weekday (Monday through Friday) at 2:00 AM UTC. What cron expression or schedule definition should be passed to the DAG schedule parameter? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-16 We have 10 parallel DAG runs hitting a single downstream MySQL database simultaneously, causing connection starvation and database crashes. How can we limit the maximum number of concurrent tasks hitting that database across all DAGs? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-13 Our DAG task calls a third-party REST API that occasionally returns HTTP 429 rate limit errors. Currently, retries are set to 3, but retries happen immediately and fail again. How should I configure task retry settings to handle rate limits gracefully? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-06 In my data pipeline, task 1 fetches a 5GB dataset from an API and converts it to a Pandas DataFrame. I want task 2 to clean this dataset, so I am using task_instance.xcom_push(key='dataframe', value=df) to pass the entire DataFrame directly. What is wrong with this implementation and how should I fix it? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-19 We have an ETL pipeline DAG that extracts raw data and a secondary Analytics DAG that cleans it. Currently, the Analytics DAG is scheduled for 4:00 AM, guessing that ETL finishes by 3:30 AM. How should we schedule the Analytics DAG to run immediately when new ETL data is produced? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-23 I want to change a production DAG schedule from once daily to every 15 minutes to reduce batch latency. Should I push this schedule update straight to main during peak hours? | fail→pass | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-17 We have hundreds of long-running sensor tasks waiting on remote jobs, causing worker node pool exhaustion because worker slots remain occupied while idle. What modern Airflow feature frees up worker slots during long waiting periods without relying on extra worker nodes? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-12 We need to ingest 50 distinct SQL tables daily with identical pipeline structures. I am considering copy-pasting 50 separate Python DAG script files into the dags folder. What is the recommended production pattern to minimize code duplication? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-22 Our Airflow instance runs 200 DAGs from 5 different engineering teams, but it is impossible to identify who owns which pipeline or filter DAGs in the UI. What standard DAG metadata fields should be configured on every DAG? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-20 Our data team frequently calls a proprietary internal REST API across 15 different DAGs using repetitive PythonOperator code. How should we package this custom interaction pattern for clean reuse across all our DAGs? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-10 I modified a complex task callable in my local Airflow environment. Rather than committing to main, triggering a full production DAG run, or waiting for the webserver UI to refresh, what CLI command should I run locally to execute and debug just that single task instance in isolation? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-11 I want to send instant Slack notifications whenever any task in my DAG fails in production. My coworker suggested wrapping every single Python task body in a try-except block to send HTTP POST requests. What standard Airflow feature provides a cleaner, centralized way to handle task failure notifications? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-09 To make my DAG file cleaner, I placed a database query directly at the top of my Python DAG file outside of any task functions, so that dynamic parameters are fetched whenever the file is read. Why might this cause scheduler performance degradation? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-21 Sometimes an upstream API task hangs indefinitely, causing a DAG run to stay in a running state for days and blocking subsequent scheduled runs. How can we ensure a DAG run automatically fails if its total duration exceeds 2 hours? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-07 I am designing a complex DAG with 30 tasks and want to group repeating task structures into smaller reusable sub-workflows. I am planning to use SubDagOperator for each group. Is this the best pattern for modern Airflow DAG design? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-05 We are deploying a new daily analytical pipeline that has historical data starting from 2020. I plan to set catchup=True on the production DAG with default settings so Airflow automatically runs all historical dates at once upon deployment. Is this the right approach? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-01 I need to run a simple daily shell script on a single Postgres server to execute pg_dump and upload the database backup file to S3. There are no downstream dependencies or multi-step tasks. What orchestration framework should I deploy to handle this backup job? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-08 I have an S3KeySensor task that checks every 5 minutes for an incoming vendor file that usually arrives between 2 AM and 6 AM. Currently, the sensor is running in its default mode='poke'. Is this setting optimal for worker resource utilization? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |
▸case-14 Our executive dashboard requires daily metrics data to be ready by 6:00 AM UTC. If a task execution exceeds this window, we need automated alerting. Which Airflow task parameter and callback mechanism should be configured to detect and report these timing violations? | fail→fail | — | — | — | — | — | — | — | — | — | — | — | — |