Add single-use callback token for deadline callback context fetch - #71192
Open
seanghaeli wants to merge 10 commits into
Open
Add single-use callback token for deadline callback context fetch#71192seanghaeli wants to merge 10 commits into
seanghaeli wants to merge 10 commits into
Conversation
seanghaeli
requested review from
XD-DENG,
amoghrajesh,
ashb,
dheerajturaga,
hussein-awala,
kaxil,
o-nikolas and
pierrejeambrun
as code owners
August 5, 2026 17:11
boring-cyborg
Bot
added
area:API
Contributor
Author
|
@ashb Whoops, closed the last pull request, re-opening it here. About your comment about the token
|
1 task
Deadline callbacks run in a subprocess that needs to read the DagRun context (and connections/variables/xcoms) from the Execution API. PR also accept the long-lived ``workload`` token via ``token:workload`` opt-ins. That over-broadened the workload token's reach (scope creep): a long-lived token could read arbitrary DagRun/connection/variable/xcom data for the whole queue-wait lifetime, and Ash asked for a single-use credential instead. This re-lands the security core of apache#66608 with a tighter design: * New ``callback`` token scope. The executor mints a ``callback``-scoped token whose ``sub`` is the callback id. That token is accepted ONLY on the new ``PATCH /callbacks/{id}/run`` exchange endpoint (pinned by a ``callback:self`` scope) and nowhere else. * Single-use exchange. ``run_callback`` performs an atomic ``QUEUED -> RUNNING`` transition under ``SELECT ... FOR UPDATE`` and returns a fresh short-lived ``execution`` token via the ``Refreshed-API-Token`` header. A second presentation of the same callback token finds the row already RUNNING and gets 409, so the token is exchangeable exactly once. This mirrors the Task Instance ``/run`` pattern. The subprocess calls the exchange once, before any context read, then uses the execution token for the actual reads. * Reverted the apache#66608 ``token:workload`` opt-ins on the DagRun, connections, variables, and xcoms GET endpoints back to execution-only. * ``JWTReissueMiddleware`` skips ``callback`` tokens (as it already does for ``workload``), since they are long-lived and exchanged exactly once. Two shared-helper refactors keep the new code DRY and also clean up pre-existing duplication: * ``require_auth`` now drives the three ``*:self`` scope checks (``ti:self`` / ``ct:self`` / ``callback:self``) from a single ``SELF_SCOPE_PATH_PARAMS`` mapping + loop instead of three near-identical branches. Behavior for ``ti:self`` and ``ct:self`` is unchanged. * New ``issue_execution_token`` helper mints an execution-scope token and sets the ``Refreshed-API-Token`` header. It replaces the duplicated mint block in the Task Instance ``/run`` handler (still guarded by the ``workload`` scope check) and the new callback exchange endpoint.
The new PATCH /callbacks/{callback_id}/run route accepts the single-use
'callback' token, a non-default (non-execution) scope. The token scope
boundary regression guard requires such routes to be declared explicitly
in NON_DEFAULT_TOKEN_POLICY; add it there per the test's documented
maintenance step.
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
The applied suggestion referenced svcs.Container without importing it; this file disables deferred annotations (I002), so import at runtime.
Review feedback from ashb: the scope claim is a property of the workload type, not something callers should supply per call.
seanghaeli
force-pushed
the
callback-token-apache
branch
from
August 12, 2026 20:03
a5b6cd9 to
55b0e20
Compare
potiuk
added
the
ready for maintainer review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds single-use
callbacktoken so a deadline-callback subprocess can fetch its DagRun context from the Execution API without holding a broadworkloadtoken (the scope creep that got #66608 reverted).PATCH /callbacks/{id}/run: single-use exchange gated on an atomicCallback.stateQUEUED→RUNNING transition (409 on reuse), returning a short-livedexecutiontoken viaRefreshed-API-Token. Mirrors the TI/runpattern.callbacktoken scope +callback:selfcheck (token subject must match the callback id).issue_execution_tokenhelper used by both the callback exchange and TI/run, and collapses the*:selfscope checks into one parameterized check.Reopen of #69840