Return each attempt's own URL from operator extra links - #71522
Open
1fanwang wants to merge 1 commit into
Open
Conversation
This was referenced Aug 12, 2026
1fanwang
force-pushed
the
oss/links-c-state-store
branch
2 times, most recently
from
August 13, 2026 00:15
1e4e436 to
e2148a5
Compare
1fanwang
marked this pull request as ready for review
August 13, 2026 00:17
1fanwang
requested review from
amoghrajesh,
ashb,
bolkedebruin and
kaxil
as code owners
August 13, 2026 00:17
1fanwang
force-pushed
the
oss/links-c-state-store
branch
from
August 13, 2026 05:00
e2148a5 to
2283d2b
Compare
An operator extra link is cached as a single XCom row under the link's xcom_key, and XComOperatorLink.get_link ignores the try_number on the TaskInstanceKey it is given. A task's XComs are also cleared before every attempt, so that row only ever holds whichever attempt ran last. Asking for an earlier attempt's link therefore returns the latest attempt's URL, and the original attempt's logs cannot be reached from the UI at all. The worker now also writes each attempt's rendered link to the task state store, which a retry does not clear, and the reader prefers that row. The XCom row is left in place and is still the answer for links written before this change. Reading goes through get_state_backend(), the same resolver the execution API uses, so a deployment pointing [state_store] backend elsewhere keeps working. Closes: apache#71471 Signed-off-by: 1fanwang <1fannnw@gmail.com>
1fanwang
force-pushed
the
oss/links-c-state-store
branch
from
August 13, 2026 05:01
2283d2b to
a7fb9b7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR — operator extra links don't work per attempt. After a retry, the link button under
attempt 1 opens attempt 2's logs, and attempt 1's logs cannot be reached from the UI at all.
Closes #71471.
It hits every bundled provider whose link carries a job id the remote service mints per
submission: EMR, Glue, Databricks, Dataproc, Livy.
Why it happens
A link is one XCom row under
BaseOperatorLink.xcom_key.XComOperatorLink.get_linkreceivesa
TaskInstanceKeyand ignores itstry_number, and that row only ever holds the latestattempt anyway — the run endpoint clears every XCom for the task before each attempt.
#65661 made the extra-links endpoint resolve the requested attempt, which is what makes the
wrong answer reachable from the UI.
The fix
The worker also writes each attempt's rendered link to the task state store, which a retry
does not clear — that is what it exists for, and what the durable-execution work in #69914 and
#71211 relies on. The reader prefers that row and falls back to the XCom row, which stays in
place and is still the answer for links written before this change.
Reading goes through
get_state_backend(), the same resolver the execution API uses, so adeployment pointing
[state_store] backendelsewhere keeps working.TaskScopeis keyed on(dag_id, run_id, task_id, map_index)with no attempt, so the try isencoded in the key. The store buys immunity from the clear, not a per-attempt dimension.
Testing
Red to green on a real 3.3.0 install, store populated per attempt and XCom holding the latest.
Raw output
Before — attempt 1 resolves to attempt 2's URL:
After:
Tests cover both sides, run locally against a source install:
They cover the reader returning the requested attempt, falling back to XCom, preferring the
store over the XCom row that holds the latest attempt, and returning empty when nothing is
stored; plus
finalizewriting the attempt's link to the store.The existing link tests still pass. Two failures in
test_dag_serialization.pyreproduce onunmodified
main, so they are not from this change.Notes for review
Two alternatives are drafted if you would rather not put link data in the state store: a row
per attempt in XCom exempt from the clear (#71518), or the same with the link opting in
(#71521). I'll close both once this direction is confirmed.
One constraint worth naming: the state store landed after 3.2.2, so this cannot be backported
to the 3.2 line. #71518 can, if that matters.
No newsfragment yet — happy to add one if you want it.