Skip to content

Expose accumulator state to allow prefix scanning - #24035

Open
avantgardnerio wants to merge 2 commits into
apache:mainfrom
avantgardnerio:brent/bwag-finalized-state-observer
Open

Expose accumulator state to allow prefix scanning#24035
avantgardnerio wants to merge 2 commits into
apache:mainfrom
avantgardnerio:brent/bwag-finalized-state-observer

Conversation

@avantgardnerio

@avantgardnerio avantgardnerio commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Expose state of aggregate streams within BWAG so downstream prefix scanning can take place.

API

// physical-plan/src/windows/bounded_window_agg_exec.rs
pub type FinalizedWindowStateObserver = Arc<
    dyn Fn(usize, &PartitionKey, &[Option<Vec<ScalarValue>>]) -> Result<()>
        + Send + Sync,
>;

impl BoundedWindowAggExec {
    pub fn with_finalized_state_observer(mut self, obs: FinalizedWindowStateObserver) -> Self {}
}

// physical-expr/src/window/window_expr.rs
impl WindowState {
    /// `Accumulator::state()` if this is an aggregate window function, `None` otherwise.
    pub fn aggregate_state(&mut self) -> Result<Option<Vec<ScalarValue>>> {}
}

🤖 Generated with Claude Code

@github-actions github-actions Bot added physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate labels Jul 31, 2026
@avantgardnerio
avantgardnerio marked this pull request as draft July 31, 2026 18:55
@avantgardnerio avantgardnerio changed the title feat(physical-plan): FinalizedWindowStateObserver on BoundedWindowAggExec Expose accumulator state to allow prefix scanning Jul 31, 2026
@avantgardnerio
avantgardnerio force-pushed the brent/bwag-finalized-state-observer branch from ae37c0b to 65014d5 Compare July 31, 2026 18:58
@avantgardnerio
avantgardnerio requested a review from Dandandan July 31, 2026 18:58
@codecov-commenter

codecov-commenter commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.82353% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.85%. Comparing base (dbcb5c0) to head (0767f73).
⚠️ Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
...ysical-plan/src/windows/bounded_window_agg_exec.rs 83.96% 8 Missing and 13 partials ⚠️
datafusion/physical-expr/src/window/window_expr.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #24035    +/-   ##
========================================
  Coverage   80.85%   80.85%            
========================================
  Files        1101     1101            
  Lines      374933   375580   +647     
  Branches   374933   375580   +647     
========================================
+ Hits       303166   303692   +526     
- Misses      53671    53781   +110     
- Partials    18096    18107    +11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@avantgardnerio
avantgardnerio marked this pull request as ready for review August 1, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants