Skip to content

Expose queueing/scheduled time in the Gantt Chart#63372

Open
smyjt wants to merge 2 commits intoapache:mainfrom
smyjt:feature/gantt-queue-time
Open

Expose queueing/scheduled time in the Gantt Chart#63372
smyjt wants to merge 2 commits intoapache:mainfrom
smyjt:feature/gantt-queue-time

Conversation

@smyjt
Copy link

@smyjt smyjt commented Mar 11, 2026

Adds queued_dttm to the Gantt chart so users can see how long tasks spent waiting in the queue before execution started.

Changes:

  • Backend: Added queued_dttm scheduled_dttmfield to GanttTaskInstance datamodel and SQL queries (both TaskInstance and TaskInstanceHistory)
  • Frontend: Renders a separate queue segment on the Gantt bar (from queued_dttm scheduled_dttm to start_date) using the existing queued and scheduled state color
  • OpenAPI spec regenerated via scripts/in_container/run_generate_openapi_spec.py
  • TypeScript types regenerated via npm run codegen
  • Tests updated and a new test added to verify queued_dttm/scheduled_dttm is returned

This mirrors the Airflow 2 Gantt chart behavior which was missing in Airflow 3.

closes: #63356


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Sonnet 4.6

Generated-by: Claude Sonnet 4.6 following the guidelines

@boring-cyborg
Copy link

boring-cyborg bot commented Mar 11, 2026

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Mar 11, 2026
@bbovenzi
Copy link
Contributor

Great! Can you provide screenshots of the queued time showing up in the chart?

@bbovenzi bbovenzi added this to the Airflow 3.1.9 milestone Mar 11, 2026
@bbovenzi bbovenzi added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Mar 11, 2026
@smyjt
Copy link
Author

smyjt commented Mar 11, 2026

Great! Can you provide screenshots of the queued time showing up in the chart?

Screenshot 2026-03-12 at 12 49 02 AM

Does this look good?

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, should we also handle a scheduled segment? Based on scheduled_dttm ?

@smyjt
Copy link
Author

smyjt commented Mar 12, 2026

Nice, should we also handle a scheduled segment? Based on scheduled_dttm ?

Will add that too.

@smyjt smyjt force-pushed the feature/gantt-queue-time branch from 75c51e6 to d3f3c72 Compare March 12, 2026 17:47
@smyjt
Copy link
Author

smyjt commented Mar 12, 2026

@pierrejeambrun @bbovenzi
I have added both scheduled and queued segment in the gantt chart
Visually, scheduled uses zinc and queued uses stone, which was hard for me to differentiate for a task instance considering the durations are in ms, but I have verified that it is working.

Screenshot 2026-03-12 at 11 20 32 PM Screenshot 2026-03-12 at 11 21 02 PM

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good.

Do you mind fixing the CI.

I agree those are hard to dissociate, but are not directly related to this PR. A great follow up PR would be to suggest another default color for those states to have a better contrast. @smyjt

@smyjt smyjt force-pushed the feature/gantt-queue-time branch from d3f3c72 to 12e98b0 Compare March 13, 2026 14:04
@smyjt
Copy link
Author

smyjt commented Mar 13, 2026

Rebased after taking latest pull, all checks are passing locally

@pierrejeambrun pierrejeambrun changed the title Expose queueing time in the Gantt Chart Expose queueing/scheduled time in the Gantt Chart Mar 18, 2026
@smyjt
Copy link
Author

smyjt commented Mar 18, 2026

Hi, can you help me with how to run these CI pipeline tests locally that are failing, I ran the pre-commit command without any error

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but on some task groups / tasks, when transition states are close enough (I think < 1s) all bars are stacked on top of each others and we cannot see schedules or queued.

Image
Screen.Recording.2026-03-18.at.17.51.38.mov

@pierrejeambrun
Copy link
Member

CI failures seems unrelated. Please rebase and lets try again. Usually the github logs show the breeze command that is running for the particular step. (All CI checks are breeze commands locally reproducible)

@smyjt smyjt force-pushed the feature/gantt-queue-time branch from 12e98b0 to 1b0d5b3 Compare March 20, 2026 09:51
@smyjt
Copy link
Author

smyjt commented Mar 20, 2026

Looks good but on some task groups / tasks, when transition states are close enough (I think < 1s) all bars are stacked on top of each others and we cannot see schedules or queued.

Should we change anything for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose queueing time in the Gantt Chart

3 participants