[v3-3-test] Add partition clear support to REST API matching the CLI (#68702)#68907
Merged
Conversation
…68702) * API: Add partition clear support to REST API to match the CLI clearDagRuns now accepts partition_key / partition_date window selectors as an alternative to an explicit run list. Add POST /dags/{dag_id}/clearPartitions to reset partition_key/partition_date on matching runs, with optional task-instance clear — REST parity with `airflow dags clear` / `airflow partitions clear`. * API: Deduplicate partition selector fields across clear request bodies Extract the shared partition_key / partition_date window fields and their date-order check into a PartitionSelectorMixin reused by BulkDAGRunClearBody and ClearPartitionsBody, and replace the repeated partition-selector presence checks with a has_partition_selectors property. No behavior change. * Scope partition-clear task instance queries to the target dag Add a dag_id filter to the task-instance lookups in both the REST clear_partition_fields service and the airflow partitions clear CLI so a run_id shared across dags no longer clears another dag's task instances, and collapse the per-run dry-run task-instance lookups into a single batched count query. * Share the partition date-window filter across clear paths Extract the resolve_day_bound partition_date window resolution duplicated across the REST clear_dag_runs route, the clear_partition_fields service, and the airflow partitions clear CLI into a single DagRun.apply_partition_date_window helper so the three cannot drift. * Share the partition-clear core between the REST API and the CLI Extract the partition column-reset, task-instance batching, and dry-run counting into a single DagRun.clear_partition_runs helper reused by the clearPartitions REST endpoint and the airflow partitions clear CLI, replacing the two parallel implementations. The CLI keeps its per-run output through an optional callback. No behavior change. * Fix partition clear commands widening sub-day windows to the whole day `dags clear` and `partitions clear` passed user-supplied datetimes through `resolve_day_bound(.date())`, which stripped the time component and expanded any sub-day bound to local midnight. On an hourly partitioned Dag, `--partition-date-start 08:00 --partition-date-end 08:00` cleared all 24 partitions instead of just the 08:00 one. Adds `localize_partition_datetime` to the `Timetable` protocol (base: UTC pass-through; CronMixin: wall-clock re-interpreted in the timetable's local timezone, same logic as #68718). Removes the now-redundant private `_localize_wall_clock_to_timetable_timezone` from `CronPartitionTimetable`. Updates `apply_partition_date_window` to use the new method with an inclusive `<=` end bound instead of the old half-open `< next_midnight` form. * Update REST datamodel descriptions to reflect sub-day precision * Share partition selection-mode validation across clear request bodies BulkDAGRunClearBody and ClearPartitionsBody duplicated the same "exactly one selection mode" rule, including the partition-window definition and the selector-enumeration error message, which would drift independently. Move the shared check onto PartitionSelectorMixin so the partition-selector semantics live in one place. * Drop sub-day-precision wording from partition clear CLI help The "sub-day precision is preserved" phrasing framed the help against a since-fixed truncation bug, which is meaningless to a reader seeing the text fresh. The timezone re-interpretation note plus the date-only -> midnight rule already convey that the time of day is honoured. * Remove unused resolve_day_bound * Refactor tests * regen docs * Fix test failure * Fix ruff F402 and docs spelling failing CI on partition-clear branch A loop variable shadowed the imported `task` decorator (ruff F402) and a British-spelled word in a new docstring tripped the en_US docs spell-check. --------- (cherry picked from commit a0805a8) Co-authored-by: Wei Lee <weilee.rx@gmail.com> Co-authored-by: Rahul Vats <rah.sharma11@gmail.com>
1 task
Lee-W
approved these changes
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
clearDagRuns now accepts partition_key / partition_date window selectors
as an alternative to an explicit run list. Add POST /dags/{dag_id}/clearPartitions
to reset partition_key/partition_date on matching runs, with optional
task-instance clear — REST parity with
airflow dags clear/airflow partitions clear.Extract the shared partition_key / partition_date window fields and their date-order check into a PartitionSelectorMixin reused by BulkDAGRunClearBody and ClearPartitionsBody, and replace the repeated partition-selector presence checks with a has_partition_selectors property. No behavior change.
Add a dag_id filter to the task-instance lookups in both the REST clear_partition_fields service and the airflow partitions clear CLI so a run_id shared across dags no longer clears another dag's task instances, and collapse the per-run dry-run task-instance lookups into a single batched count query.
Extract the resolve_day_bound partition_date window resolution duplicated across the REST clear_dag_runs route, the clear_partition_fields service, and the airflow partitions clear CLI into a single DagRun.apply_partition_date_window helper so the three cannot drift.
Extract the partition column-reset, task-instance batching, and dry-run counting into a single DagRun.clear_partition_runs helper reused by the clearPartitions REST endpoint and the airflow partitions clear CLI, replacing the two parallel implementations. The CLI keeps its per-run output through an optional callback. No behavior change.
dags clearandpartitions clearpassed user-supplied datetimesthrough
resolve_day_bound(.date()), which stripped the time componentand expanded any sub-day bound to local midnight. On an hourly
partitioned Dag,
--partition-date-start 08:00 --partition-date-end 08:00cleared all 24 partitions instead of just the 08:00 one.Adds
localize_partition_datetimeto theTimetableprotocol (base:UTC pass-through; CronMixin: wall-clock re-interpreted in the
timetable's local timezone, same logic as #68718). Removes the
now-redundant private
_localize_wall_clock_to_timetable_timezonefromCronPartitionTimetable. Updatesapply_partition_date_windowto usethe new method with an inclusive
<=end bound instead of the oldhalf-open
< next_midnightform.Update REST datamodel descriptions to reflect sub-day precision
Share partition selection-mode validation across clear request bodies
BulkDAGRunClearBody and ClearPartitionsBody duplicated the same
"exactly one selection mode" rule, including the partition-window
definition and the selector-enumeration error message, which would
drift independently. Move the shared check onto PartitionSelectorMixin
so the partition-selector semantics live in one place.
The "sub-day precision is preserved" phrasing framed the help against a
since-fixed truncation bug, which is meaningless to a reader seeing the
text fresh. The timezone re-interpretation note plus the date-only ->
midnight rule already convey that the time of day is honoured.
Remove unused resolve_day_bound
Refactor tests
regen docs
Fix test failure
Fix ruff F402 and docs spelling failing CI on partition-clear branch
A loop variable shadowed the imported
taskdecorator (ruff F402) and aBritish-spelled word in a new docstring tripped the en_US docs spell-check.
(cherry picked from commit a0805a8)
Co-authored-by: Wei Lee weilee.rx@gmail.com
Co-authored-by: Rahul Vats rah.sharma11@gmail.com