Background
Issue #741 records a negative finding from PR #731: lowering WL_KFUSION_MIN_PARALLEL_K from 4 to 2 regressed CRDT W=8 by +10%. The fixed lower threshold made low-K recursive strata pay parallel-dispatch overhead across many iterations.
Current behavior remains:
- Plan generation emits
WL_PLAN_OP_K_FUSION for recursive strata with K >= 2.
- Runtime parallel dispatch is gated by
WL_KFUSION_MIN_PARALLEL_K == 4 in wirelog/columnar/ops.c.
- K=2 and K=3 still execute through the serial K-fusion evaluator.
Problem
A static threshold is too blunt. CRDT has thousands of recursive iterations where K=2 parallel overhead compounds. Other workloads may have different iteration counts and relation sizes. The failed experiment also showed that adding a stratum_is_recursive bit to wl_plan_op_k_fusion_t would be redundant, because K-fusion is already generated inside recursive strata.
Proposed direction
Investigate an adaptive runtime heuristic instead of a fixed threshold change, for example:
- Learn per-stratum or per-K-fusion-op whether parallel dispatch pays off after the first N iterations.
- Include relation sizes, live branch count, worker count, and observed dispatch/fill time in the decision.
- Keep K=2/K=3 serial by default until the learned signal justifies parallel execution.
- Preserve the existing fixed threshold as a conservative fallback.
Acceptance criteria
- Do not lower
WL_KFUSION_MIN_PARALLEL_K globally from 4 to 2 without an adaptive guard.
- Do not add a redundant
stratum_is_recursive field to wl_plan_op_k_fusion_t; planner context already guarantees recursive origin for K-fusion.
- Validate CRDT W=8 does not regress beyond the existing performance fence.
- Validate DDISASM and at least one additional workload do not regress beyond +/-2%.
- Document the heuristic and its fallback behavior in
docs/THREADING.md if implemented.
References
Background
Issue #741 records a negative finding from PR #731: lowering
WL_KFUSION_MIN_PARALLEL_Kfrom 4 to 2 regressed CRDT W=8 by +10%. The fixed lower threshold made low-K recursive strata pay parallel-dispatch overhead across many iterations.Current behavior remains:
WL_PLAN_OP_K_FUSIONfor recursive strata with K >= 2.WL_KFUSION_MIN_PARALLEL_K == 4inwirelog/columnar/ops.c.Problem
A static threshold is too blunt. CRDT has thousands of recursive iterations where K=2 parallel overhead compounds. Other workloads may have different iteration counts and relation sizes. The failed experiment also showed that adding a
stratum_is_recursivebit towl_plan_op_k_fusion_twould be redundant, because K-fusion is already generated inside recursive strata.Proposed direction
Investigate an adaptive runtime heuristic instead of a fixed threshold change, for example:
Acceptance criteria
WL_KFUSION_MIN_PARALLEL_Kglobally from 4 to 2 without an adaptive guard.stratum_is_recursivefield towl_plan_op_k_fusion_t; planner context already guarantees recursive origin for K-fusion.docs/THREADING.mdif implemented.References
wirelog/columnar/ops.c::WL_KFUSION_MIN_PARALLEL_Kwirelog/exec_plan_gen.c::rewrite_multiway_delta