fix(codegen): bound the sensitivity df/dp derivation with a #95-style budget (#90) - #95
Merged
Merged
Conversation
… budget (#90) #95/#187 budget the build-time symbolic derivation of the analytical Jacobian, so a model that does not derive in time falls back to the finite-difference Jacobian instead of hanging the load. The df/dp path #55 added (#65/#66/#67/#68) does the same kind of sympy work and had none: `_codegen.py` contained zero references to `deadline`. Its worst site runs one `sp.diff` per (distinct rate law, parameter it reads) pair — the product of rate-law size and parameter count, the axis #95 found super-linear — so a genome-scale Functional model with a long `sensitivity_params` list presented as a build that HANGS rather than one that declines and says why. The derivation is now bounded by `BNGSIM_SENS_DERIV_BUDGET_S`, resolved by `_sens_derivation_budget_s` off the same base, slope and override grammar as the Jacobian's (one implementation, so the two cannot drift) but read independently, so `BNGSIM_JAC_DERIV_BUDGET_S=inf` does not silently uncap it. The deadline is threaded to every sympy site on the build — the Functional rate laws, both flavours of derived-parameter chain rule, the model path and the .net text path — and checked on entry to each rate law as well as before each partial, so overshoot is bounded to one (law, parameter) pair. Expiry declines through the existing `_warn_functional_sens_rhs_refused`, naming how far it got and the override. It also covers the derivation the issue's three-site table does not list: the `J*yS` half re-derives df/dx through `_functional_jacobian_groups`, which is the same math `attach_functional_jacobian` ran at load under #95 but with no clock of its own — and a model whose load-time attach was itself cut off has no earlier bound to inherit, so bounding only df/dp left the identical hang one call later. The #151 native saturable emitters run no sympy and are untouched. One difference from the Jacobian budget is deliberate: this one never becomes unbounded by size. `_FD_NONVIABLE_SPECIES` exists because past that scale an FD Jacobian does not converge — there is nothing to fall back to. Declining a sensitivity RHS only hands the columns to CVODES' difference quotient, which is what every Functional model used before #55 and is correct at every scale. Not cheap (9-37x per column, and at tight rtol the DQ's ~sqrt(rtol) accuracy collapses the step size outright), which is why the decline warns and names the knob. Corpus A/B over all 585 .net models, default budget vs unbounded: 544 emit an analytic sens RHS in both arms, 0 source hashes differ, total derivation 7.8s with the slowest model at 0.46s — 46x headroom under the 20s base. An explicit override also joins the .net path's in-process memo key and its on-disk model_hash, which key on model content rather than on the generated C.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #90.
The hole
#95/#187 bound the build-time symbolic derivation of the analytical Jacobian:
a model that does not derive in time falls back to the finite-difference Jacobian
instead of hanging the load. The
∂f/∂ppath #55 added (#65/#66/#67/#68) does thesame kind of sympy work and had no budget at all —
python/bngsim/_codegen.pycontained zero references to
deadline, and its threesp.diffsites were allunguarded.
The one that scales badly runs one
sp.diffper (distinct rate law, parameter itreads) pair — the product of rate-law size and parameter count, exactly the axis
#95 found super-linear for the Jacobian. So a genome-scale Functional model with a
long
sensitivity_paramslist presented as a build that hangs rather than onethat declines and says why (and to a
rr_parity-style harness, which times buildand solve together, as an ODE timeout — the #95 symptom).
#55 itself listed this in its "What still has to bail" section; each of the four
stages deferred it because the corpus never came close.
The fix
BNGSIM_SENS_DERIV_BUDGET_S, resolved by_jacobian._sens_derivation_budget_s.Per the issue's step 3, the budget is separate from the Jacobian's, but shares
its implementation — same base, same per-species slope, same override grammar,
one
_budget_env_override— because two spellings of one policy is exactly thedrift #68 already had to clean up. Independent resolution means
BNGSIM_JAC_DERIV_BUDGET_S=inf(the documented way to keep a genome-scale model'sanalytical Jacobian) does not silently uncap this one; they buy different things
and fall back to different paths.
The deadline is threaded to every sympy site on the build and checked on entry to
each rate law as well as before each partial, so overshoot is bounded to one
(law, parameter) pair rather than one whole model:
_functional_rate_law_partials_functional_dfdp_terms— the∂func/∂phalf_derived_param_jacobian_checkedderived_expansionloop of bothgenerate_sens_from_modeland the.netgenerate_sens_rhs_cbuild_per_species_c/differentiate_rate_law_c_functional_jacobian_groups— theJ·yShalf (see below)Expiry declines through the existing
_warn_functional_sens_rhs_refused, so itlands beside every other decline reason rather than in a channel of its own:
Two things beyond the issue's text
1.
J·ySneeded the same bound. The issue's three-site table covers∂f/∂p,but
ySdot = J·yS + ∂f/∂phas two derivations. TheJ·yShalf re-derives∂f/∂xthrough_functional_jacobian_groups, which is the same mathattach_functional_jacobianruns at load under the #95 budget — but are-derivation with no clock of its own, and on a model whose load-time attach was
itself cut off by #95 there is no earlier bound to inherit. Bounding only
∂f/∂pwould have left the identical hang reachable one call later. (The #151native saturable emitters run no sympy and are untouched; only their fallback
takes the deadline.
test_the_jacobian_half_of_the_sens_rhs_is_bounded_tooneedsa
sin()fixture for that reason.)2. The budget had to reach the
.netcache key. The model path iscontent-addressed on the generated C, so a decline naturally lands under a
different hash. The
.netpath is not — it keys on the .net's content — sowithout a tag, a build made under a deliberately tight budget would be served back
to one made without it, and the decline would be invisible.
_sens_budget_cache_tag()joins both the in-process memo key and the on-disk
model_hash, and is empty whenthe env var is unset, so every
.soalready cached still hits. This is the trap#67's A/B hatch already had to sidestep.
Deliberately not in scope
_derived_expr_partials_numeric(the Forward sensitivity does not seed ∂x0/∂p through derived-parameter species initial conditions #43 IC-seed path, site 1 of the issue'stable, "should probably take the same deadline for consistency"). It runs at
solve setup, not on this build, and a partial result there is a silently
wrong
∂x₀/∂prather than a slower-but-correct fallback — so it needs its ownall-or-nothing story, not this one's.
generate_output_sens_from_model(#198'sd func/dθ) is unbudgeted sympy on thesame build and is a real remaining hole, but its declared contract is "fail
loudly rather than fall back to FD", which is a different decline decision than
the one this PR makes everywhere else. Worth its own issue.
∂f/∂pis closed form and calls no sympy — no guard needed, asthe issue says.
Measurements
Corpus A/B, all 585
.netmodels underbenchmarks/suites/ode_fullnet/nets—default budget vs
BNGSIM_SENS_DERIV_BUDGET_S=inf(the pre-change behaviour), sameprocess, same models, so a hash difference is the budget and nothing else:
Nothing on the corpus is anywhere near the cut-off: 7.8 s of derivation across all
585 models, slowest single model 0.459 s (
BaruaBCR_2012, 1122 species, 128parameters) against a 20 s base — 46x headroom.
What a decline actually costs, so the trade is on the record rather than
implied (
ampk-signaling, 3 parameters, t=[0,100], 51 points, best of 3):mxstepThat is #55's cliff, reproduced: the DQ's
~sqrt(rtol)accuracy cannot meet atolerance below it, so the step size collapses. Declining is correct and it beats
hanging the build, but it is not free — which is why the decline is a warning
that names the knob, not a silent downgrade.
Tests
python/tests/test_codegen_sens_budget.py(32 tests, named to match thetest_codegen*.pypath filter, and added to themir.ymlandwindows-tail.ymlrun lists — CI does not otherwise run
python/tests):_derivation_budget_sgoesNone(the design decision, asserted against itstwin); malformed override degrades to the policy, not to "no budget"; the two env
vars are independent in both directions.
clock that advances 1 s per reading (plus a generous-budget control on the same
clock, so the test cannot pass by always being expired); the
J·yShalf raises;deadline=Noneis a no-op..nettext path each decline under a tight budget and emit under the default;infrestores the unbudgeted path; the warning names the env var.prepare_codegenis not handed the analytic.so.the analytic ones.
Full local suite: 2842 passed, 67 skipped.