[WIP – ignore until reviewed by @ChrisRackauckas] Release v1.0.0: constant refutation replaces is_leaf_sig#64
Conversation
2946e59 to
cc36c3e
Compare
Promotes FunctionProperties to a stable 1.0 public API surface -- `hasbranching` and `is_leaf` -- and makes constant-propagation-aware analysis the single mechanism for suppressing value-independent branches. Analysis change (follow-up to SciML#62): the type recursion is the source of truth; when it reports a branch inside a call carrying `Core.Const` arguments, the callee is re-inferred with those constants preserved (no optimizer, so no library/structural branches are inlined into view) and, only if that folds the branch, the finding is refuted. This is a strict refinement -- it can only downgrade a reported branch to branch-free -- so it adds no false positives on arbitrary code (broadcast, ComponentArrays), unlike a "run const-prop everywhere" approach. It replaces the `is_leaf_sig` hook, so no per-container override is needed (e.g. split-mode MTK `getindex(::MTKParameters, ::Int)`). The refutation uses `Base.Compiler`/`Core.Compiler` internals whose API differs across versions, so it is functionally gated: a probe folds a constant-decided branch fixture at first use and only activates if the fold works; otherwise, and on any inference failure, the analysis is exactly the plain type recursion. Verified: lts 1.10.11 (gated off), 1.12.6, 1.13.0-rc1 all green; MTK RHS table correct through the `ODEFunction` with no container-specific overrides; docs build clean. Also fixes the docs config: `deploydocs` pointed at MultiScaleArrays.jl, and the docs environment pinned `FunctionProperties = "0.1.2"`. BREAKING (1.0.0): removes the exported `is_leaf_sig` (public in 0.1.7). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
cc36c3e to
f794dae
Compare
…er inference An adversarial review found two demonstrated false negatives (the dangerous direction for the SciMLSensitivity consumer) and a design-premise violation: 1. Order-dependent false negative from `seen` memoization. Pre-refutation, any `true` terminated the whole analysis, so sigs left in `seen` on a true-returning chain could never be revisited. Refutation broke that invariant: it flips `true` to `false` and continues, so a later call site of the same widened callsig (with different or no constants) short-circuited on `sig in seen` and was silently treated as branch-free. `hasbranching` of `p -> pick(p,1)[1] + pick(p,i)[1]` returned `false` while the statement-reversed twin returned `true`. Fix: pop sigs from `seen` on true-returning paths so `seen` only memoizes proven-branch-free sigs (sound globally: widened-type scans only over-report, so a type-level `false` holds for every constant refinement); refuted results are never memoized. 2. Unbounded refutation recursion. A constant-recursive callee re-entered refutation with a fresh seen-set and an unincremented depth, recursing until stack overflow -- which the error handling then converted into "refuted", a false negative. Fix: refutations share the caller's `seen`, bump `depth` (bounded by RECURSION_LIMIT), and a transient `(:refute, sig, constants)` path marker breaks cycles conservatively, leaving the branch reported. The recursive fixtures now return `true` in under half a second. 3. The "no optimizer" premise was violated on 1.12: under the `:volatile` cache mode `typeinf` runs the optimizer, so refutation scanned post-optimization IR (helpers inlined, dead branches DCE'd). Fix: try the non-caching `:no` mode first -- verified on 1.12 and 1.13 to yield the unoptimized inferred body where the constant-decided branch appears as a `GotoIfNot` with a `Core.Const` condition and nothing is inlined into view (it also writes nothing into the inference cache). `:volatile` remains a fallback and is documented as potentially optimized but refutation-sound (inlined library branches can only make refutation fail, i.e. leave the branch reported). Regression tests added for all three (const-then-dynamic and dynamic-then-const orderings; self- and mutually-recursive constant callees). Verified: 1.10.11 24/24 (gate off), 1.12.6 and 1.13.0-rc1 25/25 (active); review reproducers all correct on both active versions; MTK RHS table 6/6 through the ODEFunction. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Adversarial re-review round (agent, Fable 5): found and fixed two critical soundness bugs plus one design-premise violation, all demonstrated with runnable reproducers before and after. Fixed in 98a733b.
Regression tests added for all three (both call-site orderings; self- and mutually-recursive constant callees). Post-fix verification: lts 1.10.11 24/24 (gate off), 1.12.6 & 1.13.0-rc1 25/25 (active); all review reproducers correct on both active versions; MTK RHS table 6/6 through the Non-blocking review notes, resolved without code change: the capability-probe |
A pathological-case battery against the refutation mechanism found one more
demonstrated false negative: `_hasbranching`'s depth backstop returns `false`
("assume leaf"), and refutation manufactures depth -- one level per
constant-recursion step -- so a refutation cascade over a constant-recursion
tower deeper than RECURSION_LIMIT read the backstop as "branch-free" and
refuted every level. A genuine value-dependent branch sitting below the cutoff
was silently lost:
hidden(q, n, x) = n == 0 ? (x > 0 ? q.a : q.b) : hidden(q, n - 1, x)
hasbranching((q, x) -> hidden(q, 5, x)[1], p, 0.5) # true (correct)
hasbranching((q, x) -> hidden(q, 400, x)[1], p, 0.5) # was false -- lost branch
Fix: `_recurse_sig` reports a branch when the depth budget is exhausted --
"cannot verify" must fail refutation. This is the conservative polarity for
both scan contexts (a normal scan deeper than 256 distinct-sig calls now
over-reports instead of under-reporting).
The rest of the battery (23 cases x {1.10 gate-off, 1.12, 1.13}: call-site
orderings, diamonds, changing/diverging/alternating constant recursion, mixed
const+value branches, Symbol/Bool/Float constants, dead-arm contents, throw-arm
folds, splat/closure/kwarg boundaries, 50 refutation sites, 40-deep forwarding
chains) passes with no hangs; constant-decided recursion below the budget still
folds (countdown-5 refutes) and above it is conservatively reported.
Regression tests added: hidden-branch towers (5 and 400), countdown 5/400, and
a diverging (`n + 1`) constant recursion that must terminate. Suite: 1.10.11
28/28 (gate off), 1.12.6 and 1.13.0-rc1 30/30; MTK RHS table 6/6; Runic clean.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Pathological-case battery round (23 adversarial cases × {lts gate-off, 1.12, 1.13}): found and fixed one more demonstrated false negative, in 0e072a9. The bug — depth-budget polarity. hidden(q, n, x) = n == 0 ? (x > 0 ? q.a : q.b) : hidden(q, n - 1, x)
hasbranching((q,x) -> hidden(q, 5, x)[1], p, 0.5) # true ✓
hasbranching((q,x) -> hidden(q, 400, x)[1], p, 0.5) # was false — branch lostFix: Battery coverage, all green on all three channels, no hangs: call-site ordering/memoization stress (const-dyn-const, dyn-const-dyn, diamonds through shared helpers), constant recursion (countdown-5 legitimately folds to Post-fix verification: lts 1.10.11 28/28 (gate off), 1.12.6 & 1.13.0-rc1 30/30 (active, includes 5 new regression tests for the towers and diverging recursion); MTK RHS table 6/6 through the |
…ys, QA allowlist
CI fixes:
- QA: the new compiler-internal qualified accesses (IR node types, reflection
entry points, and the abstract-interpreter names) are allow-listed in
test/qa/qa.jl with per-category documentation of why each group has no
public equivalent. These are the package's purpose -- compiler
introspection -- and the interpreter dependency is confined behind the
functional capability probe.
- Runic: reformatted with the current release (CI runs a newer Runic than the
previously-used local install).
Pathological-input hardening (each demonstrated before fixing):
- Self-referential constants crashed `hasbranching` with an uncaught
StackOverflowError: the refutation path marker hashed constant *values*,
which recurses into user data (and was O(length) for large constants).
Non-isbits constants are now keyed by object identity, which is exact for
the marker's purpose. Regression test added.
- Deep distinct-signature constant towers (`Val{N}` recursion, 400 levels)
ran for upwards of an hour: every depth-limit-tainted result triggered a
refutation attempt that could never succeed (its own scan exhausts the same
budget), each paying a full re-descent, and failed refutations are (soundly)
not memoized. The scan result is now a tri-state -- NOBRANCH / BRANCH /
LIMITED ("could be branching") -- and refutation is attempted only on
BRANCH: a branch that was actually seen. Val-400 drops to ~6s with zero
inference calls, while countdown-5 still folds and a genuine branch hidden
below the cutoff is still reported.
- Refutations are only started within REFUTATION_DEPTH_LIMIT (32) of the
root, and successful refutations are memoized by (sig, constant-key) --
sound because a successful refutation is provably independent of the cycle
marker (the marker can only inject "branch" verdicts, which would have made
it fail). Constant chains deeper than the cap are conservatively reported.
Verified: suites 1.10.11 29/29 (gate off), 1.12.6 and 1.13.0-rc1 31/31; QA
18/18 locally on 1.12; both adversarial batteries (24 + 13 cases) pass on all
three channels with no case slower than ~7s; Runic clean with the CI version.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
JET's typo mode flagged `local argtypes, ck` as possibly undefined: they were assigned under one `with_consts` guard and read under another, which is defined on every path that reads them but not provable by JET's analysis. Both are now always assigned (`nothing` when constants are not in play) and the reads are guarded by the `nothing` check itself. QA (including JET) is 18/18 locally. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
CI green + pathological round 2 (commits d81dcb6, 132623e). All 11 checks pass: Core on lts/1/pre, QA (incl. JET), Runic, docs, downgrade, downstream SciMLSensitivity. CI fixes:
Two more pathological-input defects found by adversarial fuzzing, both demonstrated then fixed:
Adversarial coverage now: 37 pathological cases across two batteries × three channels — call-site orderings, diamonds, constant recursion (folding/diverging/alternating/exceeding budget), hidden branches under deep towers, Symbol/Bool/Float/String/ |
A generated function whose generator consulted `hasbranching` got a silent
false negative: reflection is restricted during generated-function expansion,
so `code_typed_by_type` throws, and the catch treated the whole query as a
leaf -- the generator then emitted the wrong arm. Demonstrated on 1.12 and
1.13; the analyzer itself never noticed.
The catch is now split by position. If the *entry* IR cannot be obtained, the
query answers `LIMITED` ("could be branching") -- the caller gets `true` and a
generator consulting the analyzer mid-expansion errs on the safe side. For a
*nested* callee whose IR is unobtainable even though `which` resolved it, the
leaf treatment is kept: that is the same tier as the other unresolvable-callee
give-ups, and on older Julia versions some library-adjacent signatures
legitimately fail reflection mid-recursion (a blanket flip regressed the
lts neural-network contract with false positives).
Also from this adversarial round, verified clean with no code change needed:
fib-shaped constant recursion with failing refutations stays flat (no
exponential retry tree) while the fully-constant fib still folds via the
success memo; self-application `hasbranching(hasbranching, ...)` terminates;
24 concurrent threaded analyses agree with serial results; Function- and
Type-valued constants fold; ambiguous methods do not error; analysis does not
corrupt subsequently compiled code; a throwing `is_leaf` override propagates
and leaves the analyzer healthy.
Regression test added (generator consulting `hasbranching` must produce the
branchy arm). Suites: 1.10.11 30/30, 1.12.6 and 1.13.0-rc1 32/32; all four
adversarial batteries pass on active versions; MTK RHS table 6/6; Runic clean.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Two more adversarial findings, both demonstrated before fixing:
- A constant whose type overloads `Base.hash` or `==` to throw escaped
`hasbranching` as an uncaught user exception: the refute marker keyed
isbits constants by value, and `Set` membership runs user `hash`/`isequal`.
Constants are now keyed by `objectid` unconditionally -- egal-based (equal
isbits values and identical mutables map to the same id, so marker
semantics are unchanged), total, and it never runs user code.
- `hasbranching` on an opaque closure silently answered branch-free:
`code_typed_by_type` returns no scannable methods for it (opaque closures
have no method-table entry), and the empty-results path fell through to
the leaf answer. An entry with nothing scannable now answers the
conservative "could be branching", same policy as an unobtainable entry
IR. Mid-recursion the leaf treatment stands (`which` already resolved the
callee there).
Attacks from this round verified safe with no code change: mutating a
const-bound `Ref`/`Vector` after analysis cannot create a false negative
(Julia's effects system refuses to fold loads from mutable constants -- locked
in by a regression test), generators that illegally `eval` during expansion
throw without corrupting the analyzer, exception-driven control flow and
manual `invoke` degrade to the documented consistent limitations, a
300-argument call with a mid-position constant folds, and 16 threaded
analyses racing a thread `eval`ing new methods stay consistent.
Regression tests added: throwing `hash`/`==` overloads, const-bound mutable
stays reported, opaque-closure entry is conservative. Suites: 1.10.11 33/33,
1.12.6 and 1.13.0-rc1 35/35; all five adversarial batteries pass on all
channels; MTK RHS table 6/6; Runic clean.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Adversarial round 4 (attacker's-knowledge fuzzing of the implementation itself; commit ac03bf4, CI 11/11 green). Two more demonstrated defects fixed:
Attacks that failed to land (several locked in as regression tests): mutation attacks — folding on a const-bound Cumulative for this PR: 8 demonstrated defects found → fixed → regression-tested (order-dependent memo, SOE→false-negative recursion, 1.12 optimizer-premise violation, depth-budget polarity, selfref-hash crash, Val-tower blowup, generator-reentrancy false negative, evil-hash crash + OC entry hole). Every one was either a give-up path silently answering branch-free or user-controlled code running inside the analyzer; both classes are now structurally closed ( |
Adversarial round 5: a 120-case differential fuzzer (random helper DAGs with constant-decided and value-dependent branches, checked against an oracle that computes reachability by construction) found the analyzer sound and exactly matching the oracle on every case on 1.10/1.12/1.13 -- zero false negatives, zero precision misses, deterministic. Targeted probing found one more false-negative class (defect 9, pre-existing since before this PR): a user branch captured inside a Base callable wrapper was invisible -- `hasbranching(relu ∘ layer, x)` and `hasbranching(Base.Fix2(cmp, t), x)` reported branch-free. `ComposedFunction` and `Fix` delegate through Base-owned helper methods (kwargs bodies, tuple plumbing), so the library-leaf boundary swallowed the wrapped user function. Known wrappers (`ComposedFunction`, `Base.Fix`, and the pre-1.12 `Fix1`/`Fix2`) are now unwrapped structurally into component signatures, each routed through the normal call policy: a Base component (`sin ∘ f`) stays a library leaf, a user component is scanned, at the entry and nested alike. Also probed clean, no code change: a 5000-element constant-tuple splat through a user forwarder analyzes in ~0.1s; a type used as the callable at the entry answers conservatively; constant rebinding is picked up across queries. Regression tests added for the wrapper matrix (branchy/free x composed/fixed x entry/nested; Base components stay leaves). Suites: 1.10.11 40/40, 1.12.6 and 1.13.0-rc1 42/42; fuzzer sound on all channels; all six adversarial batteries pass; MTK RHS table 6/6; Runic clean. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
|
Adversarial round 5 — differential fuzzing + wrapper smuggling (commits e11a96d, 67568e5; CI 11/11 green). Differential fuzzer. 120 randomly generated helper DAGs per channel — constant-decided branches, value-dependent "poison" branches, passthroughs, literal/propagated/derived constants, diamonds — each checked against an oracle that computes ground truth by construction (abstract interpretation over Known/Unknown constant states in the generator). Result across 1.10 / 1.12.6 / 1.13-rc1: 360/360 sound, zero false negatives, zero precision misses, deterministic — on active channels the analyzer exactly matched the oracle on every random shape. Defect 9 (pre-existing, now fixed): user branches smuggled inside Base callable wrappers. Probed clean, no change needed: 5000-element constant-tuple splat through a user forwarder (~0.1s), a type used as the callable at the entry (conservative), constant rebinding picked up across queries. State: suites 40/40 (lts) · 42/42 · 42/42; fuzzer sound on all channels; six adversarial batteries ≈ 80 cases/channel; MTK RHS table 6/6. Cumulative: 9 demonstrated defects found → fixed → regression-tested. |
Summary
Two things:
hasbranchingandis_leaf(both documented and in the rendered API page).is_leaf_sighook added in 0.1.7 (so no per-container override is needed — e.g. split-mode MTKgetindex(::MTKParameters, ::Int)).Analysis change
The type recursion stays the source of truth. When it reports a branch inside a call carrying
Core.Constarguments, the callee is re-inferred with those constants preserved (no optimizer, so no library/structural branches are inlined into view); only if that folds the branch away is the finding refuted. This is a strict refinement — it can only downgrade a reported branch to branch-free, never the reverse — so it introduces no false positives on arbitrary code. (An earlier "run const-prop on every constant-carrying call" version regressed broadcast /ComponentArrayneural-net RHS: const inference propagates aConstsymbol intogetproperty(::ComponentArray, ::Symbol)'s specialized path and surfaces an unfolded index branch. Refuting only found branches avoids that.)Version robustness (lts / 1 / pre)
The refutation uses
Base.Compiler/Core.Compilerinternals whose API differs across versions (theInferenceStateconstruction and inferred-source location already differ between 1.12 and 1.13). It is functionally gated: a probe folds a constant-decided-branch fixture through the const inference at first use and only activates the refutation if the fold works; otherwise, and on any inference failure, the analysis is exactly the plain type recursion (conservative).MTK RHS table through the
ODEFunction(linear/power branch-free,relubranchy, both split modes) is correct with no container-specific overrides (with the merged unwrap, SciML/SciMLBase.jl#1413). Docs build verified locally (clean; version auto-detected as 1.0.0).Docs config fixes (were broken on
main)deploydocspointed atgithub.com/SciML/MultiScaleArrays.jl.git→ fixed to FunctionProperties.jl.FunctionProperties = "0.1.2"→"1".Trade-off (flagged)
The refutation depends on non-public compiler internals (
InferenceResult/InferenceState/typeinf/retrieve_code_info), which the removedis_leaf_sigdid not. It is gated to degrade safely, but where active it tracks compiler-internal churn each release. The public API committed at v1 (hasbranching,is_leaf) is independent of this and stable; the const-prop path is an internal, self-disabling implementation detail.Semver
Breaking / 1.0 release: removes the exported
is_leaf_sig(public in 0.1.7).0.1.7→1.0.0.