Skip to content

Analytical tests batch 1#67

Open
wiesnerfriedman wants to merge 8 commits into
swmm6_relfrom
analytical_tests_batch_1
Open

Analytical tests batch 1#67
wiesnerfriedman wants to merge 8 commits into
swmm6_relfrom
analytical_tests_batch_1

Conversation

@wiesnerfriedman
Copy link
Copy Markdown
Collaborator

@wiesnerfriedman wiesnerfriedman commented May 8, 2026

Add manufactured benchmark suite (batch 1) with analytical test harness

Summary
Introduces seven manufactured/analytical benchmarks under tests/benchmarks/manufactured/, each with definition.md, reference.csv, and provenance.yaml: Horton infiltration, Green-Ampt saturated trajectory, LID rain-barrel exfiltration, kinematic-wave normal depth, exponential decay ODE, logistic growth ODE, and SIR epidemic ODE (n=3 coupled system).
Adds test_ode_solver.cpp with 12 tests covering the ODE solver, root-finder, and trajectory benchmarks; extends test_infiltration.cpp with two trajectory tests against benchmark CSVs.
Adds tools/generate_sir_reference/ — a Fortran driver that compiles against the genuine Flash-X CashKarp45 integrator to produce the SIR reference CSV, providing independent cross-language, cross-implementation verification.

Bugs fixed
LID.cpp (batchBarrelFlux): stor_ksat was never applied — wb_infil and infil_loss were always zero for rain-barrel storage units. Fixed with an Euler exfiltration step (exact for the constant-rate ODE).
KinematicWave.cpp line 86: getAofS(xs, s_needed * a_full) was passed a dimensionally-wrong argument; corrected to getAofS(xs, s_needed).
Test results

All three test executables pass with no failures and no skips:

test_engine_infiltration: 32/32
test_engine_ode_solver: 12/12
test_engine_lid: 48/48

I have read the CLA Document and I hereby sign the CLA.

wiesnerfriedman and others added 4 commits May 7, 2026 23:15
Seven manufactured benchmarks covering infiltration (Horton, Green-Ampt),
LID exfiltration, kinematic wave, and ODE solver (exponential decay,
logistic growth, SIR epidemic). Each benchmark carries definition.md,
reference.csv, and provenance.yaml with primary literature citations.

Test coverage:
- test_ode_solver.cpp (new): ODE integrator and root-finder unit tests,
  plus trajectory benchmarks for exponential decay, logistic growth, and
  SIR epidemic (conservation + phase-plane invariants + qualitative dynamics);
  SIR CSV trajectory test skips until reference is populated.
- test_infiltration.cpp: Horton and Green-Ampt saturated-branch trajectory
  benchmarks only; ODE/FindRoot tests moved to test_ode_solver.cpp.
- test_routing.cpp: kinematic-wave normal-depth steady-state benchmark.
- test_lid.cpp: constant-area exfiltration benchmark.

Also fixes KinematicWave.cpp getAofS call (s_needed * a_full → s_needed)
that caused incorrect cross-sectional area lookup in the KW solver.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LID.cpp batchBarrelFlux never applied stor_ksat — wb_infil was
unconditionally zero for all rain-barrel storage units.  Added
Euler exfiltration step (exact for constant-rate ODE).

Green-Ampt trajectory test was sub-stepping at dt=1 s (up to 8793
Newton solves), accumulating ~0.035 ft of round-off against a
1e-3 ft criterion.  Replaced with one implicit solve per benchmark
interval — the method is exact for any dt.

Horton RMS tolerance tightened beyond the 8-digit CSV precision;
relaxed from 1e-10 to 1e-9 ft.

SIR epidemic integration extended from t=60 to t=120 — with R0=3
and γ=0.1 the infected fraction does not drop below 1% until ~t=80.

CMake: made OpenMP linkage conditional to fix macOS no-libomp
builds; guarded regression CTest registration against missing data dir.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
These changes were specific to a no-libomp macOS dev environment and
should not be part of the engine build configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… CSV

Writes the genuine cross-code reference trajectory for the
odesolve-sir-epidemic benchmark using Flash-X's independent Fortran
CashKarp45 implementation (eFrac=1e-12), replacing the t=0 placeholder.
SIRTrajectoryMatchesBenchmark now runs (was previously SKIPPED).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a first batch of manufactured/analytical verification benchmarks under tests/benchmarks/ and wires them into new/extended unit tests, while also fixing two solver issues (rain-barrel storage exfiltration and KW inlet area inversion) that the new benchmarks exercise.

Changes:

  • Added seven manufactured benchmark datasets (CSV + definition + provenance) and shared benchmark conventions/docs under tests/benchmarks/.
  • Added a dedicated ODE/root-finder test executable plus new trajectory-based regression tests for infiltration, LID rain-barrel exfiltration, and kinematic-wave normal depth.
  • Fixed (1) rain-barrel storage exfiltration not applying stor_ksat and (2) a dimensional error in KW inlet area computation.

Reviewed changes

Copilot reviewed 38 out of 40 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
.gitignore Ignores two local documentation note files.
src/engine/hydrology/LID.cpp Fixes rain-barrel exfiltration to actually apply stor_ksat and track losses in the water balance.
src/engine/hydraulics/KinematicWave.cpp Fixes KW inlet section-factor inversion by passing dimensionally correct S to getAofS.
tests/unit/engine/CMakeLists.txt Injects BENCHMARK_DATA_DIR into several unit-test targets and adds the new test_engine_ode_solver target.
tests/unit/engine/test_infiltration.cpp Removes basic ODE/root tests (moved to dedicated file) and adds Horton + Green-Ampt trajectory tests against benchmark CSVs.
tests/unit/engine/test_ode_solver.cpp New unit/benchmark test suite for ODE integrator and root-finders (incl. SIR multi-variable tests + optional trajectory comparison).
tests/unit/engine/test_lid.cpp Adds rain-barrel storage exfiltration trajectory test against manufactured benchmark CSV.
tests/unit/engine/test_routing.cpp Adds KW steady-state “normal depth recovered” benchmark test against manufactured CSV.
tests/benchmarks/README.md Documents benchmark subtree layout and how verification datasets should be stored/consumed.
tests/benchmarks/shared/conventions.md Establishes units, CSV formatting, and naming conventions for benchmark datasets.
tests/benchmarks/provenance-template.md Adds a provenance template for externally generated datasets.
tests/benchmarks/generated/.gitkeep Keeps the generated-benchmarks directory in version control.
tests/benchmarks/manufactured/infil-horton-constant-rainfall/definition.md Defines the Horton manufactured benchmark scenario and acceptance criteria.
tests/benchmarks/manufactured/infil-horton-constant-rainfall/provenance.yaml Records parameters and generation details for the Horton benchmark.
tests/benchmarks/manufactured/infil-horton-constant-rainfall/reference.csv Provides Horton reference trajectory values in internal units (plus readable columns).
tests/benchmarks/manufactured/grnampt-saturated-trajectory/definition.md Defines the Green-Ampt saturated-branch manufactured benchmark.
tests/benchmarks/manufactured/grnampt-saturated-trajectory/provenance.yaml Records parameters and derivation basis for the Green-Ampt benchmark.
tests/benchmarks/manufactured/grnampt-saturated-trajectory/reference.csv Provides Green-Ampt saturated-branch reference trajectory table.
tests/benchmarks/manufactured/exfil-storage-constant-area/definition.md Defines constant-rate storage exfiltration manufactured benchmark for rain barrels.
tests/benchmarks/manufactured/exfil-storage-constant-area/provenance.yaml Records parameters and mass-balance identity for the storage exfiltration benchmark.
tests/benchmarks/manufactured/exfil-storage-constant-area/reference.csv Provides constant-rate storage exfiltration reference trajectory table.
tests/benchmarks/manufactured/kinwave-normal-depth-rect-open/definition.md Defines KW steady-state normal-depth benchmark and explains zero-residual property.
tests/benchmarks/manufactured/kinwave-normal-depth-rect-open/provenance.yaml Records channel parameters and tolerances for the KW benchmark.
tests/benchmarks/manufactured/kinwave-normal-depth-rect-open/reference.csv Provides KW normal-depth reference table for multiple depths/flows.
tests/benchmarks/manufactured/odesolve-exponential-decay/definition.md Defines exponential-decay ODE benchmark and acceptance criteria.
tests/benchmarks/manufactured/odesolve-exponential-decay/provenance.yaml Records parameters and closed-form generation notes for exponential-decay benchmark.
tests/benchmarks/manufactured/odesolve-exponential-decay/reference.csv Provides exact exponential-decay reference values.
tests/benchmarks/manufactured/odesolve-logistic-growth/definition.md Defines logistic-growth ODE benchmark and acceptance criteria.
tests/benchmarks/manufactured/odesolve-logistic-growth/provenance.yaml Records parameters and closed-form generation notes for logistic-growth benchmark.
tests/benchmarks/manufactured/odesolve-logistic-growth/reference.csv Provides exact logistic-growth reference values.
tests/benchmarks/manufactured/odesolve-sir-epidemic/definition.md Defines SIR ODE benchmark (coupled system) and intended verification checks.
tests/benchmarks/manufactured/odesolve-sir-epidemic/provenance.yaml Records SIR benchmark metadata and generation workflow notes.
tests/benchmarks/manufactured/odesolve-sir-epidemic/reference.csv Provides SIR reference trajectory values (claimed generated via Flash-X).
tools/generate_sir_reference/.gitignore Ignores local build artifacts for the SIR reference generator.
tools/generate_sir_reference/Driver_interface.F90 Adds minimal Flash-X driver stub interface needed by the integrator module.
tools/generate_sir_reference/Makefile Adds a Makefile to build/run a Flash-X-based Fortran generator for SIR reference.csv.
tools/generate_sir_reference/RuntimeParameters_interface.F90 Adds minimal runtime-parameter stub for Flash-X RK initialization.
tools/generate_sir_reference/Simulation.h Adds minimal Simulation.h stub required by some Flash-X modules.
tools/generate_sir_reference/sir_driver.F90 Fortran driver program that integrates SIR with Flash-X CashKarp45 and writes CSV.
tools/generate_sir_reference/sir_ode.F90 Fortran SIR RHS function compatible with Flash-X RungeKutta interface.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tools/generate_sir_reference/Makefile Outdated
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/provenance.yaml
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/provenance.yaml Outdated
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/definition.md Outdated
Comment thread tests/unit/engine/test_ode_solver.cpp
Comment thread tools/generate_sir_reference/Makefile Outdated
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/definition.md Outdated
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/definition.md Outdated
wiesnerfriedman and others added 2 commits May 8, 2026 20:10
The generator (tools/generate_sir_reference/) requires a local Flash-X
checkout at a hardcoded path and cannot be run by other contributors.
The reference CSV it produced is already committed; generation methodology
is fully documented in provenance.yaml and definition.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
t_end_s 60→120, planned_t_eval_s extended to 120, dataset status
placeholder→populated, generation script replaced with a prose note
describing the Flash-X Fortran driver that produced the CSV.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 33 changed files in this pull request and generated 8 comments.

Comments suppressed due to low confidence (1)

src/engine/hydrology/LID.cpp:526

  • In batchBarrelFlux(), wb_init_vol is initialized using stor_depth * stor_void (volume-equivalent depth), but wb_final_vol is later set to stor_depth only. This makes the rain-barrel water-balance bookkeeping inconsistent whenever stor_void != 1.0 (and it will also misreport final stored volume in ctx.lid_usage.wb_final_vol). wb_final_vol should be stored in the same units/definition as wb_init_vol.
        // Water balance tracking
        g.wb_inflow[ui]     += unit_inflow * dt;
        g.wb_evap[ui]       += 0.0;
        g.wb_infil[ui]      += exfil;
        g.wb_surf_flow[ui]  += overflow * dt;
        g.wb_drain_flow[ui] += drain * dt;
        g.wb_final_vol[ui]   = g.stor_depth[ui];
        g.vol_treated[ui]   += unit_inflow * dt;

Comment thread src/engine/hydrology/LID.cpp
Comment thread tests/unit/engine/test_lid.cpp Outdated
Comment thread tests/unit/engine/test_infiltration.cpp Outdated
Comment thread tests/unit/engine/test_ode_solver.cpp
Comment thread tests/unit/engine/test_routing.cpp
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/definition.md Outdated
Comment thread tests/benchmarks/manufactured/odesolve-sir-epidemic/definition.md Outdated
Comment thread tests/unit/engine/test_ode_solver.cpp
wiesnerfriedman and others added 2 commits May 8, 2026 21:26
t_end 60→120, qualitative dynamics updated to match actual trajectory,
reference dataset section replaced with accurate description of the
committed CSV and where to find rebuild instructions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_ode_solver.cpp: update SIR skip comment to reflect populated
  reference.csv and point to provenance.yaml instead of missing .py script
- test_infiltration.cpp, test_lid.cpp: move benchmark includes
  (fstream/sstream/string/vector) to top include block; remove duplicates
  that were left mid-file
- LID.cpp batchBarrelFlux: apply stor_clog reduction via getStorageExfil
  so rain-barrel exfiltration is consistent with other storage-based LIDs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants