Make multi-response optimization work end to end: sweet-spot region, shared desirability, intervals at the optimum

[kgdunn]

Summary

The library implements Derringer-Suich desirability correctly, but the surrounding multi-response workflow did not hold together. This PR closes that.

• The sweet-spot plot did not show the sweet spot. OverlayPlot turned each response's low/high into an AnnotationType.label, and no adapter has a branch for that annotation type, so the limits were dropped silently at render time. Nothing computed the region where all responses are simultaneously in specification. It now computes a feasibility mask and shades it, draws each bounded response's contours at exactly its two specification limits, and reports sweet_spot_fraction / sweet_spot_empty so a caller can tell when no compromise exists.
• The contour adapter discarded its styling. _contour_trace read only the three grid keys and hard-coded the colorscale, ignoring colorscale, zmin, zmax, contours_coloring, ncontours, layer.opacity, and LayerSpec.color. An overlay therefore rendered as N filled surfaces stacked in one colorscale. Those keys are now honoured, defaulting to the previous values so no existing plot changes.
• The desirability math existed twice and had drifted. Now shared from experiments/_desirability.py, holding the union of the two behaviours: asymmetric target weights from the plotting copy, strict argument checking from the optimizer copy.
• desirability_weights was misnamed: it is the importance vector, and it silently overrode per-goal importance. Renamed to response_importance, with the old name kept as a deprecated alias.
• Goals were matched to models by list position while goal["response"] was documented as the key and never read, so reordering goals silently optimised the wrong thing. Now matched by name when both sides supply one, falling back to position with a warning.
• optimize_responses returned no uncertainty at the optimum. Passing fitted_results alongside fitted_models now attaches a confidence interval and a prediction interval per response. Behaviour is unchanged when omitted.
• The search region was hard-coded to the factorial cube. That is right for a two-level design but understates a central composite design, whose axial runs sit at plus or minus alpha: the optimizer would not consider settings the experiment had actually covered, and inside_design_space could report a legitimately explored point as outside. New search_bounds sets the region, defaulting to the previous (-1, 1).

Also fixed: the desirability contour plot's single-response fallback set the bounds to negative and positive infinity, so the ramp computed inf/inf and the entire grid came out NaN.

Test plan

• uv run pytest on the merged tree: 2163 passed, 2 skipped
• uv run ruff check .
• uv run mypy src/process_improve
• Rendered a two-response overlay and cross-checked the shaded mask against an independently recomputed feasibility set: no disagreements over 2500 grid cells. The existing tests never asserted z_matrix values or rendered output, so this could not be left to the suite.
• New tests assert where the optimum lands (a plane's optimum at its known corner; two conflicting responses settling strictly between their individual optima; raising one response's importance moving the compromise its way), the sweet-spot mask against a hand-computed region, empty-sweet-spot reporting, goal matching by name, the deprecated alias, interval ordering at the optimum, and the widened search region. The previous end-to-end tests asserted only that the composite exceeded zero, which a badly wrong optimum would also satisfy.
• New adapter tests cover each newly honoured contour style key.
• search_bounds checked against the bioreactor example used in the book: its optimum is interior to the cube, so that answer is unchanged.

Checklist

• Version bumped in pyproject.toml (MINOR, 1.61.0 to 1.62.0: new features plus API additions)
• Tests added or updated where relevant
• ruff check . passes
• CHANGELOG.md updated, CITATION.cff kept in sync

Note on the version number

This branch originally claimed 1.61.0. #476 merged to main first and took that number for the score_contributions fix, so both were about to ship under one version. Merging main in renumbered this work to 1.62.0, leaving main's 1.61.0 entry exactly as it stands and adding the matching compare links to the changelog footer.

Related

• Book: Add a section on multi-response optimization: the sweet spot and desirability pid-book#255 adds the section on multi-response optimization whose figures this code produces. Now green and mergeable_state: clean.
• Figures: Add figures for the multi-response optimization subchapter figures#66, merged, holds the generated images and the script, which calls this code.

[codecov]

Codecov Report

❌ Patch coverage is 90.63830% with 22 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/process_improve/experiments/_desirability.py	84.61%	5 Missing and 3 partials ⚠️
src/process_improve/experiments/optimization.py	92.30%	6 Missing and 1 partial ⚠️
...eriments/visualization/plots/optimization_plots.py	89.70%	4 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!