MIR JIT: a Functional model with sensitivity_params fails to compile (c2mir rejects bngsim_codegen_output_sens)

[wshlavacek]

Summary

Under the MIR JIT backend (BNGSIM_CODEGEN_JIT=mir), a Functional model run with
sensitivity_params fails to compile:

RuntimeError: MirJit: c2mir failed to compile generated RHS source:
bngsim_rhs_module:201:9: syntax error on double (expected '<statement>'):

This is on main (4ab9e72), not introduced by any open PR. It reproduces on
all four MIR legs — ubuntu, macos-14, macos-15-intel, windows.

Reproducer

# python/tests/test_mir_probe_functional_sens.py, run with BNGSIM_CODEGEN_JIT=mir
def test_functional_model_with_sensitivity_params_compiles(tmp_path):
    net = tmp_path / "m.net"
    net.write_text(SIR)              # any Functional rate law; SIR below
    m = bngsim.Model.from_net(net)
    sim = bngsim.Simulator(m, method="ode", sensitivity_params=["gamma"])
    r = sim.run(t_span=(0.0, 5.0), n_points=6)
    assert r.sensitivities is not None

with

begin parameters
    1 S0     2e7  # Constant
    2 I0     1  # Constant
    3 beta   1/S0  # ConstantExpression
    4 gamma  1/7  # Constant
end parameters
begin functions
    1 betaI() beta*I
end functions
begin species
    1 person(state~S) S0
    2 person(state~I) I0
    3 person(state~R) 0
end species
begin reactions
    1 1 2 betaI #_R1
    2 2 3 gamma #_R2
end reactions
begin groups
    1 S                    1
    2 I                    2
    3 R                    3
end groups

CI proof on a branch off clean main carrying only this test plus a one-line
mir.yml addition: run 30290133235
— 1 failed, 155 passed, same error on every leg.

Where it is

sensitivity_params sets _want_output_sens, which appends
bngsim_codegen_output_sens (GH #198) to the JIT module. Mapping the reported
module line back through the generated source puts the failure inside that
function's per-column loop, at or next to:

    for (int _c = 0; _c < n_sens; ++_c) {
        const double* ss = state_sens[_c];
        double* fs = func_sens_out + (size_t)_c * N_FUNC;   /* <- reported line */

Caveat on that localization: I could not confirm the exact statement locally. A
standalone driver built against the vendored third_party/mir picks up the host
SDK's math.h, which c2mir rejects long before reaching the generated body, so
it does not reproduce the CI include environment. The line-number mapping is
consistent across three anchors (the second #define N_PARAMS, the function's
opening brace column, and the error line) but is still inference — worth
confirming against a real MIR build before fixing.

cc and the .so path compile the identical source without complaint, so this is
c2mir-specific, not invalid C.

Why nothing caught it

mir.yml's paths filter is python/tests/test_codegen*.py, but the step it runs
is a hand-maintained list of filenames — and none of the listed tests puts a
Functional model through the JIT with sensitivities. The closest,
test_model_codegen_sensitivity.py::test_rhs_only_codegen_uses_fd_correctly,
pre-sets model._codegen_so_path, which makes the Simulator skip auto-codegen
entirely, so the JIT never sees the module.

Follow-up

Once fixed, add

            python/tests/test_codegen_functional_dfdp.py \
            python/tests/test_codegen_functional_sens_rhs.py \

to mir.yml's run list. Those files cover #66/#67 and currently trigger the job
without executing in it. Reverted out of #84 for exactly this reason
(c520c4c); after #67 lands, the
population this bug affects grows from "Functional + sensitivity_params" to
every Functional model that now gets an analytic sensitivity RHS.

[wshlavacek]

The Follow-up section above is now out of date — it points the wrong way

#87 (12f1dc1) added all three files to mir.yml's run list without waiting for this
bug, and quarantined only what this bug blocks. So the follow-up here is no longer "add the
files"; doing that now would just duplicate lines 133-135. The remaining work when this is
fixed is to delete the markers:

• python/tests/test_codegen_functional_sens_rhs.py — blocked_on_mir_jit at line 78, applied
  per-test at lines 640, 659, 683, 745 (7 items after parametrization)
• python/tests/test_codegen_switch_condition_sens.py — same marker at line 71, applied to the
  whole TestEndToEnd class at line 375 (3 items)

test_codegen_functional_dfdp.py needed no marker at all: it never builds a Simulator, so it
is backend-agnostic and passes 29/29 under the JIT. That narrows this bug's blast radius a
little from how it is described above — it is sensitivity_params on a constructed
Simulator, not the analytic sensitivity RHS as such.

The markers are xfail(strict=True, raises=RuntimeError), so they retire themselves: the day
c2mir compiles these models, all 10 XPASS and the MIR legs go red until the markers come out.
No need to remember this comment.

One case is deliberately not marked, and it usefully bounds the bug:
test_steady_state_dfdp_stops_differencing reaches the sensitivity codegen without ever
setting _want_output_sens, so bngsim_codegen_output_sens is never emitted and it passes
under the JIT today.

Current state on all four MIR legs: 229 passed, 10 xfailed.

Correction to "Where it is"

That section flags its own line-number mapping as inference, and on a real MIR build the
mapping does not hold. Built locally with
-DBNGSIM_ENABLE_MIR=ON -DBNGSIM_ENABLE_KLU=OFF -DBNGSIM_REQUIRE_KLU=OFF (HAS_MIR True,
HAS_KLU False), the SIR reproducer fails as expected, and the two #define N_PARAMS anchors
land at source 25 and 88 but module 41 and 102 — offsets of 16 and 14. Not a constant
prelude, so no anchor-based mapping onto model._codegen_c_source is trustworthy, including
the one used to localize this to the per-column loop. Whatever MirJit hands c2mir needs to be
dumped directly before this is localized.

At 7ebfdfe the error had also moved with the source:

bngsim_rhs_module:102:9: warning -- different macro redefinition of N_PARAMS
bngsim_rhs_module:41:9: warning -- previous definition of N_PARAMS
bngsim_rhs_module:343:9: syntax error on double (expected '<statement>'):
bngsim_rhs_module:328:72: unfinished compound statement

Possibly worth a look on its own: those two N_PARAMS defines carry different values (4
and 5 for SIR — the second counts the function-bound betaI), so everything after the second
define sees 5. That is the redefinition warning, and N_PARAMS is read inside the offending
function.

Reproducing locally is cheap

Contrary to the note above about a standalone c2mir driver picking up the host SDK's math.h:
a real -DBNGSIM_ENABLE_MIR=ON build reproduces this on the first run and matched CI exactly
(same 229/10 counts as all four legs). Roughly 10 minutes with a warm ccache, and it beats a
probe branch for iterating on a fix.