Skip to content

fix(domain): apply the full prescribed flux in the thermal reference solve - #145

Open
singhharsh1708 wants to merge 2 commits into
pasteurlabs:mainfrom
singhharsh1708:fix/thermal-reference-neumann-load
Open

fix(domain): apply the full prescribed flux in the thermal reference solve#145
singhharsh1708 wants to merge 2 commits into
pasteurlabs:mainfrom
singhharsh1708:fix/thermal-reference-neumann-load

Conversation

@singhharsh1708

Copy link
Copy Markdown
Contributor

Fixes #144.

The reference solve applied one corner-weight share per unique marked face node instead of one per face element, so it only ever applied (ny+1)(nz+1) / (4*ny*nz) of the prescribed flux. On the configured 16x8x1 mesh that is 0.5625, and it changes with resolution, so target_temperature was both wrong and not mesh-converged.

Fixed by accumulating per face element, which is what torch-fem-thermal already documents and does. Mirroring it element for element rather than writing an independently correct quadrature keeps the reference consistent with the solvers in hot_spot mode too, where the mask covers only part of the face.

Verification

At uniform density the problem reduces to a 1D slab with a closed form, T_hot = Q*Lx/(k*Ly*Lz). Before and after, at rho=0.5 and Q_total=100:

mesh before after analytic
16x8x1 897.965 1588.878 1588.878
8x4x1 ratio 0.625 1588.878 1588.878
32x16x1 ratio 0.531 1588.878 1588.878
16x8x2 ratio 0.422 1588.878 1588.878

Ratio to analytic is 1.00000 at every resolution now, so the reference is mesh-independent as well as correct.

tests/test_thermal_reference.py pins both properties against the closed form rather than golden values, so it stays meaningful if the mesh or the material model changes. All five fail on main.

Full suite is 555 passed against 550 before, ruff clean.

Results churn

This moves published identification_error and final_ic_error for thermal-mesh optimization/conductivity_recovery_bfgs, so it wants the benchmark:domain treatment and a re-run. It is the same objective-consistency question as #85 and #93, and those two probably want reviewing alongside this.

I would not overstate the damage it was doing. The deficit is close to a pure rescale of the load and SIMP conductivity is a free multiplier, so the optimiser could still approach the stored target by inflating density: the recovered field was biased, not unreachable, and the bias moved with resolution. gradient/source_fd_check and gradient/source_width_sweep are AD-versus-FD consistency checks that need a target rather than a physical one, so they were not invalidated. optimization/conductivity_recovery sits behind if False: at config.py:428 and is unaffected.

…solve

_approx_target_temperature added one corner-weight share to each unique
marked face node, so it applied (ny+1)(nz+1)/(4*ny*nz) of the prescribed
flux: 0.5625 on the configured 16x8x1 mesh, and a different fraction at
every other resolution. The reference therefore solved a different problem
than the solvers it is the ground truth for, and was not mesh-converged.

Accumulate per face element instead, which is what torch-fem-thermal
already documents and does, so an interior face node picks up four shares,
an edge node two and a corner one. Mirroring it element for element also
keeps hot_spot mode consistent, where the mask covers only part of the
face.

At uniform density the problem reduces to a 1D slab with a closed form,
T_hot = Q*Lx/(k*Ly*Lz). The reference now matches it to 1e-6 at 16x8x1,
8x4x1, 32x16x1 and 16x8x2, where before it sat at 0.565 of analytic and
moved with the mesh.

Refs pasteurlabs#144
@singhharsh1708

Copy link
Copy Markdown
Contributor Author

The red check is the label gate, not a test failure: this touches mosaic/, so it needs a maintainer to apply one of the benchmark labels. benchmark:domain looks right here, since it changes a problem definition in a single domain rather than any solver.

Small thing spotted on the way: the missing-label error at benchmark.yml:515 lists only three of the four, leaving out benchmark:domain, though the label itself is handled fine at line 91 and the PR template documents all four. Happy to send a one-line fix separately if useful.

@dionhaefner dionhaefner added the benchmark:domain Benchmark every solver in the domain whose problem definition changed label Aug 18, 2026
@PasteurBot

Copy link
Copy Markdown
Contributor

📊 View the full benchmark results

The rendered docs preview has every plot for this run (forward accuracy, gradients, cost, optimization) merged with existing baseline results on main. The summary below reports pass/fail status.


Coverage: partial run — some cells are shown from the baseline rather than measured this run.

Status diff vs base

Legend · ✅ ok · 🟠 anom · ❌ fail · · missing · 🚫 excluded (permanent — out of score denominator) · ⚪ excluded (work-to-do) · * stale — result predates current benchmark run

0 regression(s) · 0 improvement(s) · 12 metric change(s) · 0 other transition(s) · 0 resource-frontier shift(s) · 0 new row(s) · 0 removed row(s) · score 0.95 → 0.95

📊 Metric changes

  • 🔴 thermal-mesh · gradient/source_fd_check · FEniCS · gradient rel-error 0.0458 → 0.0598 (+30%)
  • 🔴 thermal-mesh · gradient/source_fd_check · Firedrake · gradient rel-error 0.0458 → 0.0598 (+30%)
  • 🔴 thermal-mesh · gradient/source_fd_check · JAX-FEM · gradient rel-error 1.71e-05 → 5.18e-05 (+203%)
  • 🔴 thermal-mesh · gradient/source_fd_check · torch-fem · gradient rel-error 1.71e-05 → 5.18e-05 (+203%)
  • 🟢 thermal-mesh · optimization/conductivity_recovery_bfgs · FEniCS · final/initial loss ratio 0.00502 → 0.00208 (-58%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · FEniCS · final loss 9.45e+06 → 1.26e+07 (+34%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · Firedrake · final/initial loss ratio 0.00309 → 0.00373 (+21%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · Firedrake · final loss 5.83e+06 → 2.26e+07 (+288%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · JAX-FEM · final/initial loss ratio 4.55e-05 → 0.000158 (+248%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · JAX-FEM · final loss 8.57e+04 → 9.6e+05 (+1019%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · torch-fem · final/initial loss ratio 5.85e-05 → 0.000295 (+404%)
  • 🔴 thermal-mesh · optimization/conductivity_recovery_bfgs · torch-fem · final loss 1.1e+05 → 1.79e+06 (+1521%)
Full Mosaic status

Mosaic status

Legend · ✅ ok · 🟠 anom · ❌ fail · · missing · 🚫 excluded (permanent — out of score denominator) · ⚪ excluded (work-to-do) · * stale — result predates current benchmark run

Each solver is run against every experiment in the suite. ok = produced valid results; fail = crashed or returned invalid data; anom = ran successfully but tripped an automated quality check (e.g. poor gradient accuracy, outlier wall-clock time, or diverged optimisation). Thresholds are defined per-problem in the problem config.

problem ok anom fail missing excl (work) excl (perm) stale score
ns-3d-grid 95 1 7 0 0 9 0 🟢 0.93
ns-grid 114 8 2 0 0 16 0 🟢 0.95
structural-mesh 40 0 0 0 0 5 5 🟢 1.00
thermal-mesh 60 0 3 0 0 7 0 🟢 0.95
overall 309 9 12 0 0 37 5 🟢 0.95

Failures & anomalies

  • ns-3d-grid · cost/spatial_cost · PhiFlow — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • ns-3d-grid · cost/temporal_cost · PhiFlow — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • ns-3d-grid · cost/vjp_cost/by_N · PhiFlow — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • ns-3d-grid · cost/vjp_cost/by_steps · PhiFlow — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • 🟠 ns-3d-grid · gradient/fd_check · INS.jl — best-ε median FD rel_err 2.36e-03 > max_rel_err=1e-03
  • ns-3d-grid · gradient/horizon_sweep_limits · PhiFlow — nan
  • ns-3d-grid · gradient/horizon_sweep_limits · PICT — error
  • ns-3d-grid · gradient/horizon_sweep_limits · Warp-NS — nan
  • 🟠 ns-grid · forward/agreement/tgv · PhiFlow — phiflow's double CenteredGrid↔StaggeredGrid resampling gives 4.18% amplitude damping (ratio=0.9582); cosine=0.9999924 (pattern correct); arithmetic-average output conversion fix worsened error 9×; upstream library change required
  • 🟠 ns-grid · forward/agreement/tgv · XLB — automatic k=9 sub-steps reduce Ma 0.88→0.098 (81× Ma² reduction); errors drop from 0.216-0.278 → 0.026-0.031 (11-24× peers); remaining floor is O(dx²) LBM spatial discretization at N=64, not reducible by further sub-stepping (tested k=9..27); valid=True
  • 🟠 ns-grid · forward/baseline · INS.jl — staggered MAC grid double-interpolation: collocated TGV IC -> staggered faces -> collocated output gives sin^2(pi/N) round-trip error at all N; 35-40x above collocated peers
  • 🟠 ns-grid · forward/baseline · jax-cfd — staggered MAC grid double-interpolation: collocated TGV IC -> staggered faces -> collocated output gives sin^2(pi/N) round-trip error at all N; 35-40x above collocated peers
  • 🟠 ns-grid · forward/baseline · XLB — irreducible O(Ma²) LBM compressibility error floor: at fixed dt=0.01, Ma=u·dt/dx grows with N; at N=128 Ma~0.2 giving ~0.007 error floor (230× peers); anomalous at all N
  • 🟠 ns-grid · forward/tgv_nu_sweep · XLB — same root cause as forward/agreement/tgv — automatic k=9 sub-stepping reduces Ma 0.88→0.098 but residual O(dx²) LBM spatial discretization gives 11-24× peer errors at all nu values (0.0001–0.05); 0.0309 at nu=0.05 is 12.0× peer median; not reducible by further sub-stepping (tested k=9..27); valid=True
  • 🟠 ns-grid · cost/spatial_cost · OpenFOAM — median time 11.0s is 35× peer median (0.31s); threshold k=20.0
  • ns-grid · cost/spatial_cost · PhiFlow — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • 🟠 ns-grid · cost/temporal_cost · PhiFlow — median time 11.7s is 38× peer median (0.31s); threshold k=20.0
  • ns-grid · cost/vjp_cost/by_N · PhiFlow — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • thermal-mesh · cost/spatial_cost · FEniCS — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • thermal-mesh · cost/vjp_cost/by_N · FEniCS — error: RuntimeError: Error 500 from Tesseract: Traceback (most recent call last):
  • thermal-mesh · cost/vjp_cost/by_N · JAX-FEM — error: ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
ns-3d-grid — 16 experiment(s)
experiment Exponax INS.jl OpenFOAM PhiFlow PICT Warp-NS XLB
forward/agreement
forward/baseline
forward/physical_laws/vs_N
forward/physical_laws/vs_nu
forward/physical_laws/vs_steps
cost/spatial_cost
cost/temporal_cost
cost/vjp_cost/by_N 🚫
cost/vjp_cost/by_steps 🚫
gradient/fd_check 🟠 🚫
gradient/horizon_sweep_limits 🚫
gradient/jacobian_svd 🚫
gradient/jacobian_svd_nu01 🚫
gradient/jacobian_svd_steps20 🚫
gradient/jacobian_svd_steps40 🚫
optimization/recovery_constant_ic_bfgs_proj 🚫
ns-grid — 20 experiment(s)
experiment INS.jl jax-cfd OpenFOAM PhiFlow PICT Warp-NS XLB
forward/agreement/multimode
forward/agreement/tgv 🟠 🟠
forward/baseline 🟠 🟠 🟠
forward/cylinder 🚫 🚫 🚫
forward/physical_laws/vs_N
forward/physical_laws/vs_nu
forward/physical_laws/vs_steps
forward/tgv_nu_sweep 🟠
cost/spatial_cost 🟠
cost/temporal_cost 🟠
cost/vjp_cost/by_N 🚫
cost/vjp_cost/by_steps 🚫
gradient/fd_check 🚫
gradient/horizon_sweep 🚫
gradient/jacobian_svd 🚫
gradient/jacobian_svd_nu01 🚫
gradient/jacobian_svd_steps20 🚫
gradient/jacobian_svd_steps40 🚫
gradient/param_sweep 🚫
optimization/drag_opt 🚫 🚫 🚫 🚫
structural-mesh — 10 experiment(s)
experiment deal.II FEniCS Firedrake JAX-FEM TopOpt.jl
forward/agreement ✅*
forward/baseline ✅*
forward/physical_laws ✅*
cost/spatial_cost ✅*
cost/temporal_cost ✅*
cost/vjp_cost/by_N 🚫
cost/vjp_cost/by_steps 🚫
gradient/fd_check 🚫
gradient/param_sweep 🚫
optimization/topopt 🚫
thermal-mesh — 14 experiment(s)
experiment deal.II FEniCS Firedrake JAX-FEM torch-fem
forward/agreement
forward/baseline
forward/physical_laws
forward/source_baseline
forward/source_linearity
cost/spatial_cost
cost/temporal_cost
cost/vjp_cost/by_N 🚫
cost/vjp_cost/by_steps 🚫
gradient/fd_check 🚫
gradient/param_sweep 🚫
gradient/source_fd_check 🚫
gradient/source_width_sweep 🚫
optimization/conductivity_recovery_bfgs 🚫

@singhharsh1708

Copy link
Copy Markdown
Contributor Author

Reading the benchmark run, since the metric column is mostly red.

No status transitions: 0 regressions, 0 improvements, score 0.95 unchanged. All 12 changes are thermal-mesh metrics, which is the churn this PR was always going to cause, since it changes the ground truth those metrics are measured against.

Part of the movement is just magnitude. The corrected target is 1.77x larger, and identification_error is a sum of squares, so a pure rescale predicts 3.13x on final loss. Observed:

solver final loss ratio
FEniCS 9.45e6 -> 1.26e7 1.33x
Firedrake 5.83e6 -> 2.26e7 3.88x
JAX-FEM 8.57e4 -> 9.60e5 11.2x
torch-fem 1.10e5 -> 1.79e6 16.3x

So scaling alone does not account for it, and the residual is worth being explicit about rather than glossing. The two solvers that moved most are the two that previously fit best in absolute terms. My reading is that the old target was close to reachable by a near-uniform conductivity rescale, because the flux deficit was almost a pure scalar on the load and SIMP conductivity is a free multiplier, so an accurate solver could sit very near it without recovering the right density field. The corrected target removes that shortcut. On that reading a higher residual is the benchmark asking a harder and more meaningful question, not the recovery getting worse.

I want to be clear that is an interpretation, not something this run establishes. The measurement that would settle it is recovered rho against rho_truth before and after, rather than the loss against the target. If you would like, I can add that as a diagnostic, though it needs the solver containers so it is not something I can produce from a local run.

On gradient/source_fd_check, JAX-FEM and torch-fem move 1.71e-05 -> 5.18e-05, both still three orders under the 1e-3 gate. FEniCS and Firedrake move 0.0458 -> 0.0598, but those two were already sitting far above the gate on an eps-independent plateau before this PR, which looks like a separate issue in their source adjoint rather than anything this changes.

Happy to hold this until #85 and #93 land if you would rather review the objective-consistency changes together.

@dionhaefner

Copy link
Copy Markdown
Contributor

Thanks for the contribution @singhharsh1708. Your reasoning makes sense but I don't have the bandwidth to dig in here at the moment. Pending further analysis I'm ok with letting this sit, e.g. until @andrinr can have a closer look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark:domain Benchmark every solver in the domain whose problem definition changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

thermal-mesh reference applies 56% of the prescribed heat flux, and the deficit moves with resolution

3 participants