GFQL implementation shrink: pass manager and pushdown residue

[lmeyerov]

Goal

Shrink GFQL pass manager / pushdown implementation code without changing behavior.

Target 300-600 LOC of production GFQL implementation deletion/DRY if safely available. Tests/fixtures/changelog do not count toward the headline target; report them separately.

Context

Refs:

• perf: graphistry wheel size (625 kB) — GFQL engine files are large, investigate modularization #1058 cleanup/deletion meta
• GFQL cleanup: add CPU/GPU coverage audit for dead-code candidates #1517 coverage audit infrastructure
• GFQL deletion tranche: planner/IR private helper and fixture dedupe bundle #1542 planner/IR cleanup
• GFQL deletion: remove residual unplanned Cypher chain fallback after bucket closeout #1486 fallback deletion
• perf: graphistry wheel size (625 kB) — GFQL engine files are large, investigate modularization #1058 implementation-LOC-first policy update

The pass manager and pushdown modules were added around planner evolution. This lane should audit whether there is remaining duplicated or stale implementation code now that native planning/fallback deletion has settled.

Candidate Surface

Start from latest master and audit this ownership area:

• graphistry/compute/gfql/passes/*.py
• graphistry/compute/gfql/ir/pushdown_safety.py
• graphistry/compute/gfql/passes/predicate_pushdown.py
• graphistry/compute/gfql/passes/unnest_apply.py
• pass manager / pushdown tests only as validation/support

Avoid overlapping with physical executor cleanup and typed-schema work.

Required Protocol

1. Build an inventory before editing: production symbol/file, callers, coverage signal, and expected implementation LOC deletion.
2. Prioritize production GFQL implementation shrink. Test/comment deletion is secondary and must be reported separately.
3. If implementation deletion opportunity is <300 LOC after a full audit, stop with an audit receipt rather than shipping a mostly-test cleanup PR.
4. Preserve current planner/pass behavior and public APIs.
5. Use GFQL cleanup: add CPU/GPU coverage audit for dead-code candidates #1517 coverage as triage only; require static caller proof and focused validation.

Validation Expectations

• Focused pass manager / predicate pushdown / unnest apply tests.
• Planner/runtime tests if pass behavior is touched.
• Broad non-cuDF GFQL selection if production paths are touched.
• Ruff, typecheck, py_compile, surface guard if applicable, diff check.
• Review protocol before marking ready.

Merge Policy

Open a PR when ready, but do not auto-merge. Coordinator must approve after reviewing production-code deletion, test/support delta, and validation.

[lmeyerov]

Audit receipt for #1565 pass-manager/pushdown shrink: started fresh from origin/master @ 73e057d on branch feature/issue-1565-gfql-pass-pushdown-shrink and used the plan/review protocols. Note: the prompt referenced #1567, but the pasted title/scope match #1565; #1567 is the separate IR metadata/verifier lane.

Inventory: target production surface is 672 LOC total across passes/init.py, manager.py, predicate_pushdown.py, unnest_apply.py, and ir/pushdown_safety.py. Static caller proof found PassManager/default pass tuples are production-called from gfql_unified._run_logical_pass_pipeline(); PredicatePushdownPass and UnnestApply are covered directly and through the default pass integration; is_null_rejecting() and with_barrier_blocks_pushdown() are production-used by PredicatePushdownPass; is_null_safe() is an exported inverse wrapper with tests.

Candidate deletion: predicate_pushdown.py still has text-level top-level-AND splitting/ref-inference residue that may be obsolete for MATCH/WHERE because binder now emits one BoundPredicate per top-level AND. Expected safe production deletion is only about 120-150 LOC and touches direct helper compatibility. PassManager removal/tiering collapse would not be safe because it removes exported/internal API behavior covered by ordering, verifier, convergence, metadata, and legacy positional-arg tests.

Decision: no PR opened. After full audit and two review waves, no safe 300-600 production LOC deletion tranche was found while preserving behavior/public APIs. This follows #1565 protocol item 3: stop with audit receipt rather than shipping a mostly-test/tiny cleanup PR.

Validation: focused baseline passed: python3 -m pytest -q graphistry/tests/compute/gfql/test_pass_manager.py graphistry/tests/compute/gfql/test_predicate_pushdown_pass.py graphistry/tests/compute/gfql/test_pushdown_safety.py graphistry/tests/compute/gfql/test_unnest_apply.py --tb=short => 81 passed in 2.41s. No DGX/RAPIDS run because no DataFrame/cuDF runtime code changed.

[lmeyerov]

Corrected final audit for #1565 after coordinator scope clarification.

Scope correction applied: df_executor.py and physical_planner.py are compiler-plan core and were not considered deletion targets. I also treated logical/physical planning contracts, route names, dispatch semantics, pass/verifier extension points, structured error/defer-code context, source-span/diagnostic fidelity, schema/type-system hooks, remote-schema hooks, and public compatibility shims as protected surfaces.

Final safe-target scan was limited to unused private delegators/wrappers, duplicated helper bodies, and unreachable fallback-era residue in the pass/pushdown ownership area. Result: no safe 300+ production LOC tranche.

Evidence:

• PredicatePushdownPass helpers are live. Production binder sites currently construct BoundPredicate(expression=...) without references, so predicate pushdown still needs text splitting/reference inference fallback for analysis.
• pushdown_safety.py masking/normalization helpers are live through is_null_rejecting().
• PassManager, PassResult, LogicalPass, UnnestApply, and verifier integration are pass/compiler extension points under the corrected rule, so they are not deletion targets even though current production creation of Apply-family logical nodes is limited.
• df_executor.py and physical_planner.py were not touched.

Closeout: no PR opened. Production-code LOC deleted: 0. Tests/docs LOC deleted: 0. Compiler-plan surface touched: no. Prior broad deletion idea is explicitly rejected under the coordinator correction.

[lmeyerov]

#1565 closeout after PR #1569 merge.

• PR: Shrink GFQL physical dispatch route scaffolding #1569
• Merge commit: 6d7b0c26350792921bd3644b42ab7ff1bcd78a33
• Merged at: 2026-05-20T17:23:48Z

Final implemented scope followed the later physical-dispatch assignment and coordinator boundary correction, not a broad compiler-plan deletion:

• Removed stale runtime coupling to physical-executor wrapper classes in graphistry.compute.gfql_unified.
• Runtime dispatch now keys off the existing PhysicalPlan.route contract.
• Preserved compiler-plan architecture and compatibility surfaces: PhysicalPlan.operators, logical_op_ids, metadata, PhysicalOperator, physical-planner wrapper classes, route names, planner wrapper construction, pass/defer/diagnostic/schema hooks, and public exports remain intact.
• Candidate wrapper deletion was explicitly rejected because those wrappers are exported and may be part of the compiler/type/remote plan.

LOC receipt:

• Production code: +9 / -26, net -17 LOC
• Tests: +37 / -38, net -1 LOC
• Changelog/docs: +1 / -0

Validation:

• Focused local CPU tests: 39 passed, 1 deselected
• Ruff, typecheck, py_compile, and diff --check: clean
• Review skill reconverged after the coordinator correction; wave 4 had no remaining findings after correcting changelog wording.
• DGX RAPIDS 25.02 + 26.02 targeted cuDF route smoke passed before the boundary correction; the correction restored planner metadata/wrappers and did not change cuDF dataframe behavior.
• PR CI was green after rebasing over moved master, including tck-gfql, GFQL core, pandas-compat GFQL, changed-line coverage, docs/ReadTheDocs, and the broad matrix.

Meta note: this is intentionally below the original 300-600 production LOC target because the larger deletion candidate would have removed protected/exported compiler-plan surfaces.

[lmeyerov]

Coordinator reopen: this issue's actual scope (GFQL pass manager and pushdown residue) has not been addressed. PR #1569 close-linked this issue but its actual content was physical dispatch and dataframe executor residue (the real #1564 scope). I'm reopening #1565 because the pass manager / pushdown shrink work remains TODO.

Cross-ref:

• PR Shrink GFQL physical dispatch route scaffolding #1569 (merge 6d7b0c26350792921bd3644b42ab7ff1bcd78a33) is now correctly attributed to GFQL implementation shrink: physical dispatch and dataframe executor residue #1564 (physical dispatch).
• This issue (GFQL implementation shrink: pass manager and pushdown residue #1565) returns to the open shrink-lane queue.
• Caveat: pass manager and pushdown extension points are listed as protected compiler-plan surfaces in the coordinator plan. Future worker assignment to this issue must preserve pass/verifier extension points; only remove unused private wrappers, duplicated helper bodies, or stale fallback-era code proven unreachable.

See coordinator notes on #1058 for the broader lane-map correction across #1564 / #1565 / #1566.