Reduce unconstrained NLP overhead by skipping all-infinite Bounds for L-BFGS-B

[daggbt]

Summary

The NLP overhead shown in the benchmark suite appears to be dominated by how Optyx calls SciPy for unconstrained problems, rather than by autodiff or compiled callback cost.

For the benchmark objective used in benchmarks/run_benchmarks.py, Optyx auto-selects L-BFGS-B for unconstrained NLPs in src/optyx/problem.py, and the SciPy solve path always builds and passes a Bounds object in src/optyx/solvers/scipy_solver.py and src/optyx/solvers/scipy_solver.py.
When every bound is effectively (-inf, inf), passing that Bounds object seems to push SciPy onto a much slower L-BFGS-B path.

Current behavior

For the unconstrained quadratic benchmark x.dot(x) - x.sum() at n=5000:

• Raw SciPy L-BFGS-B with no bounds: about 0.95 ms
• Raw SciPy L-BFGS-B with all-infinite bounds: about 13.49 ms
• Optyx compiled objective/gradient with no bounds: about 0.75 ms
• Optyx compiled objective/gradient with all-infinite bounds: about 13.08 ms
• Full Optyx Problem.solve() warm solve: about 17.0 ms

This suggests the dominant cost is not symbolic compilation or callback evaluation. The biggest regression comes from passing an all-infinite Bounds object.

The same pattern shows up in the overhead chart in benchmarks/results/overhead_breakdown.png, where NLP is a major outlier while LP, CQP, and MILP stay near parity.

Reproduction

Use the benchmark helper in benchmarks/run_benchmarks.py:

• Run benchmark_nlp_vector(5000)
• Compare the existing result against a runtime-patched version that omits bounds= when all lower bounds are -inf and all upper bounds are inf

Observed on the dev container:

• Baseline exact benchmark:
  • cold_solve_ms: about 69.0
  • warm_solve_ms: about 19.8
  • warm_overhead: about 32.8x
• With runtime patch to skip all-infinite bounds:
  • cold_solve_ms: about 14.1
  • warm_solve_ms: about 4.24
  • warm_overhead: about 7.1x

At n=50, the same change reduced:

• cold overhead from about 16.1x to 4.3x
• warm overhead from about 2.7x to 1.85x

Likely root cause

In the SciPy solver path, bounds are constructed and passed even when they do not constrain the problem at all.

For unconstrained NLPs solved with L-BFGS-B, this appears to introduce a large fixed overhead inside SciPy.

Proposed fix

Before calling SciPy, detect whether all variable bounds are effectively unbounded:

• all lower bounds are -inf
• all upper bounds are inf

If so, pass bounds=None instead of a Bounds object.

Expected outcome

This should substantially reduce the NLP overhead outlier in the benchmark suite, especially for trivial unconstrained objectives where SciPy can otherwise solve in sub-millisecond time.

Notes

No source changes were made during the investigation. The improvement numbers above were measured by runtime patching only.