Skip to content

Commit 94fc55f

Browse files
Jammy2211Jammy2211
Jammy2211
authored and
Jammy2211
committed
docs(health): group release-fidelity fix prompts
1 parent c140b76 commit 94fc55f

8 files changed

Lines changed: 250 additions & 0 deletions

‎bug/health_fixes/README.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Release-fidelity health fixes
2+
3+
This folder tracks the script regressions exposed by the first real release-profile
4+
validation in [PyAutoHeart issue #27](https://github.com/PyAutoLabs/PyAutoHeart/issues/27),
5+
GitHub Actions run `28784914443`.
6+
7+
The run installed and verified all five TestPyPI wheels successfully, then reported
8+
37 script failures and 5 timeouts across 42 scripts. Local reproduction used the
9+
shared PyAuto development environment, current `main` library checkouts, each
10+
workspace's `config/build/env_vars_release.yaml`, and the same 300-second cap.
11+
12+
Each failing script is assigned to exactly one prompt:
13+
14+
| Prompt | Scripts | Primary concern |
15+
|---|---:|---|
16+
| [samples_parameter_paths.md](samples_parameter_paths.md) | 9 | PyAutoFit result/sample path resolution |
17+
| [autofit_sampler_database.md](autofit_sampler_database.md) | 9 | Emcee NaNs and database output discovery |
18+
| [aggregator_output_contracts.md](aggregator_output_contracts.md) | 7 | Result/aggregator prerequisites and generated paths |
19+
| [jax_runtime_and_parity.md](jax_runtime_and_parity.md) | 6 | JAX/TFP compatibility and likelihood parity |
20+
| [jit_visualization_outputs.md](jit_visualization_outputs.md) | 4 | Quick-update visualizations not producing images |
21+
| [numerical_inversion_failures.md](numerical_inversion_failures.md) | 2 | Non-positive-definite inversion matrices |
22+
| [release_timeout_policy.md](release_timeout_policy.md) | 5 | 300-second release-surface decisions |
23+
24+
Total: **42 scripts**. Scripts that pass on current `main` remain listed because they
25+
still require a clean-worktree, directory-order reproduction before being declared
26+
fixed. Do not rebaseline assertions or edit tutorials to conceal a library regression.

‎bug/health_fixes/aggregator_output_contracts.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Fix release aggregator and generated-output contracts
2+
3+
## Context
4+
5+
Seven release failures involve results that are absent, shorter than examples assume,
6+
or written to a different location. Several pass in a stateful local checkout, so the
7+
first task is a clean, directory-ordered reproduction. Primary owners are @PyAutoFit,
8+
@autogalaxy_workspace, @autolens_workspace, and @autolens_workspace_test.
9+
10+
## Scripts
11+
12+
- `autogalaxy_workspace/scripts/guides/results/start_here.py`
13+
- `autogalaxy_workspace/scripts/guides/results/aggregator/samples_via_aggregator.py`
14+
- `autolens_workspace/scripts/guides/results/start_here.py`
15+
- `autolens_workspace/scripts/guides/results/aggregator/galaxies_fits.py`
16+
- `autolens_workspace/scripts/guides/results/aggregator/samples_via_aggregator.py`
17+
- `autolens_workspace/scripts/guides/results/workflow/csv_make.py`
18+
- `autolens_workspace_test/scripts/imaging/convolution.py`
19+
20+
## Required work
21+
22+
1. Run each parent directory in CI order from a clean output tree under the release
23+
profile, then rerun each script independently to document prerequisites.
24+
2. Determine whether failures are library output/aggregator regressions, invalid script
25+
assumptions about sample counts, or missing directory creation for generated files.
26+
3. Fix PyAutoFit when valid completed searches are not discoverable. Otherwise update
27+
scripts minimally while preserving their teaching narrative.
28+
4. Ensure scripts create their own output directories and do not depend on unrelated
29+
earlier legs or developer-local artifacts.
30+
5. Add the narrowest appropriate tests and rerun all seven release-profile scripts.

‎bug/health_fixes/autofit_sampler_database.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Fix Autofit release sampler and database regressions
2+
3+
## Context
4+
5+
Release run `28784914443` failed two @autofit_workspace cookbooks and seven
6+
@autofit_workspace_test scripts. The cookbook failures reproduce on current `main`:
7+
Emcee proposes invalid LogUniform values and raises `ValueError: Probability function
8+
returned NaN`. Database failures include empty aggregators, stale/mismatched scraped
9+
search metadata, and changed likelihood assertions.
10+
11+
Primary library: @PyAutoFit.
12+
13+
## Scripts
14+
15+
- `autofit_workspace/scripts/cookbooks/result.py`
16+
- `autofit_workspace/scripts/cookbooks/samples.py`
17+
- `autofit_workspace_test/scripts/database/directory/general.py`
18+
- `autofit_workspace_test/scripts/database/directory/multi_analysis.py`
19+
- `autofit_workspace_test/scripts/database/scrape/general.py`
20+
- `autofit_workspace_test/scripts/database/scrape/grid_search.py`
21+
- `autofit_workspace_test/scripts/database/scrape/multi_analysis.py`
22+
- `autofit_workspace_test/scripts/database/scrape/sensitivity.py`
23+
- `autofit_workspace_test/scripts/features/minimal_output.py`
24+
25+
## Required work
26+
27+
1. Reproduce each script from a clean output/database state using the exact release
28+
profiles; directory and scrape scripts must not consume prior local runs.
29+
2. Fix Emcee initialization/proposal handling so bounded priors cannot emit NaN
30+
probabilities during reduced release-mode sampling.
31+
3. Audit database directory and scrape paths, identifiers, session lifecycle, grid and
32+
sensitivity metadata, and minimal-output expectations against current PyAutoFit.
33+
4. Fix library defects in PyAutoFit. Change script assertions only when the documented
34+
output contract intentionally changed, with an explicit explanation.
35+
5. Add unit/integration coverage, run PyAutoFit pytest, and rerun all nine scripts.

‎bug/health_fixes/jax_runtime_and_parity.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Fix release JAX runtime compatibility and likelihood parity
2+
3+
## Context
4+
5+
Six JAX likelihood scripts failed with the rehearsed release stack. CI included
6+
TensorFlow Probability using removed JAX internals and NumPy/JAX likelihood mismatches.
7+
All six passed locally with current `main` and JAX 0.9.2, so verify whether upstream
8+
library changes already fixed them or whether environment/order sensitivity remains.
9+
10+
Owners: @PyAutoArray, @PyAutoFit, @PyAutoGalaxy, @PyAutoLens,
11+
@autogalaxy_workspace_test, and @autolens_workspace_test.
12+
13+
## Scripts
14+
15+
- `autogalaxy_workspace_test/scripts/jax_likelihood_functions/imaging/delaunay_mge.py`
16+
- `autogalaxy_workspace_test/scripts/jax_likelihood_functions/interferometer/delaunay_mge.py`
17+
- `autogalaxy_workspace_test/scripts/jax_likelihood_functions/multi/delaunay_mge.py`
18+
- `autolens_workspace_test/scripts/jax_likelihood_functions/interferometer/delaunay_mge.py`
19+
- `autolens_workspace_test/scripts/jax_likelihood_functions/multi/rectangular.py`
20+
- `autolens_workspace_test/scripts/jax_likelihood_functions/multi/rectangular_mge.py`
21+
22+
## Required work
23+
24+
1. Reproduce in a clean source environment using the release dependency constraints and
25+
record exact JAX, jaxlib, TFP, NumPy, nufftax, and pynufft versions.
26+
2. Confirm whether current `main` fixes every CI traceback without cached compiled state.
27+
3. If TFP remains incompatible, fix or replace the owning library path rather than
28+
disabling JAX in JAX-specific scripts.
29+
4. Investigate parity differences from data/model construction through inversion and
30+
regularization; do not merely loosen tolerances without a numerical error budget.
31+
5. Add library regression tests and rerun all six scripts in both fresh processes and
32+
their normal directory sequence.

‎bug/health_fixes/jit_visualization_outputs.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Fix JIT quick-update visualization output regressions
2+
3+
## Context
4+
5+
Four test-workspace scripts expect real release-profile searches to invoke the JIT-cached
6+
quick-update visualization path and produce fit images. CI reported missing files. Local
7+
`main` still fails the ellipse, interferometer, and point-source cases, while imaging
8+
passes.
9+
10+
Owners: @PyAutoFit, @PyAutoGalaxy, @PyAutoLens, @autogalaxy_workspace_test, and
11+
@autolens_workspace_test.
12+
13+
## Scripts
14+
15+
- `autogalaxy_workspace_test/scripts/ellipse/modeling_visualization_jit.py`
16+
- `autogalaxy_workspace_test/scripts/imaging/modeling_visualization_jit.py`
17+
- `autogalaxy_workspace_test/scripts/interferometer/modeling_visualization_jit.py`
18+
- `autolens_workspace_test/scripts/point_source/modeling_visualization_jit.py`
19+
20+
## Required work
21+
22+
1. Reproduce from clean output directories with JAX enabled and real release-profile
23+
searches.
24+
2. Trace search update cadence, visualization dispatch, cached fit creation, output-path
25+
routing, and exception handling for all four dataset types.
26+
3. Fix the shared library path where possible; do not add sleeps, weaken assertions, or
27+
fabricate image files in scripts.
28+
4. Add focused tests proving quick updates invoke visualization and write the expected
29+
artifact for ellipse, imaging, interferometer, and point-source analyses.
30+
5. Run owning-library tests and all four scripts under the release profile.

‎bug/health_fixes/numerical_inversion_failures.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Fix release-profile numerical inversion failures
2+
3+
## Context
4+
5+
Two interferometer scripts fail in inversion paths with non-positive-definite matrices.
6+
The Autolens test failure reproduces on current `main`; the Autogalaxy script passed in a
7+
stateful local checkout and needs a clean confirmation.
8+
9+
Owners: @PyAutoArray, @PyAutoGalaxy, @PyAutoLens, @autogalaxy_workspace, and
10+
@autolens_workspace_test.
11+
12+
## Scripts
13+
14+
- `autogalaxy_workspace/scripts/interferometer/features/pixelization/galaxy_reconstruction.py`
15+
- `autolens_workspace_test/scripts/interferometer/model_fit.py`
16+
17+
## Required work
18+
19+
1. Reproduce in clean output/worktrees with deterministic seeds and release settings.
20+
2. Capture the curvature and regularization matrix properties at failure: symmetry,
21+
conditioning, eigenvalue range, dtype, backend, and mapper configuration.
22+
3. Identify whether the defect is invalid sampled parameters, regularization construction,
23+
numerical stabilization, or a script model that permits an undefined inversion.
24+
4. Fix the owning library for valid inputs. Do not catch `LinAlgError` or alter the script
25+
to hide a genuine inversion failure.
26+
5. Add numerical regression tests and rerun both scripts repeatedly under the profile.

‎bug/health_fixes/release_timeout_policy.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Resolve release-profile timeout scripts deliberately
2+
3+
## Context
4+
5+
Five scripts exceeded PyAutoBuild's 300-second per-script cap in release run
6+
`28784914443`. A timeout is a release-surface policy decision, not automatically a code
7+
bug. Stateful local reruns are not authoritative because completed search output can make
8+
chained scripts resume quickly.
9+
10+
Owners: @autogalaxy_workspace, @autolens_workspace, @PyAutoFit, @PyAutoGalaxy,
11+
@PyAutoLens, and @PyAutoBuild where runner evidence is needed.
12+
13+
## Scripts
14+
15+
- `autogalaxy_workspace/scripts/ellipse/multipoles.py`
16+
- `autolens_workspace/scripts/cluster/start_here.py`
17+
- `autolens_workspace/scripts/cluster/modeling.py`
18+
- `autolens_workspace/scripts/imaging/features/advanced/double_einstein_ring/chaining.py`
19+
- `autolens_workspace/scripts/multi/features/slam/simultaneous.py`
20+
21+
## Required work
22+
23+
1. Benchmark each script from a clean output tree with the exact release profile and
24+
record phase-level timing. Confirm whether it completes correctly beyond 300 seconds.
25+
2. Investigate avoidable repeated compilation, plotting, search, dataset, and chaining
26+
costs without reducing the scientific/tutorial contract.
27+
3. For each script choose explicitly between:
28+
- optimize it to fit reliably below the cap; or
29+
- add a documented `SLOW` entry to that workspace's `config/build/no_run.yaml` because
30+
it is unsuitable for automated release validation.
31+
4. Do not silently raise the global cap. Do not use cached outputs as pass evidence.
32+
5. Validate optimized scripts from clean state or validate that the runner reports the
33+
chosen scripts as skipped with their documented reasons.

‎bug/health_fixes/samples_parameter_paths.md‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Fix release result/sample parameter-path regressions
2+
3+
## Context
4+
5+
PyAutoHeart release validation run `28784914443` exposed a family of workspace
6+
failures. Against current `main`, nine scripts converge on
7+
`PyAutoFit/autofit/non_linear/samples/sample.py::parameter_lists_for_paths`
8+
raising `KeyError` for model paths that no longer match stored sample kwargs.
9+
10+
Primary repository: @PyAutoFit. Downstream verification repositories:
11+
@autogalaxy_workspace and @autolens_workspace.
12+
13+
## Scripts
14+
15+
- `autogalaxy_workspace/scripts/imaging/features/pixelization/galaxy_reconstruction.py`
16+
- `autogalaxy_workspace/scripts/imaging/features/shapelets/modeling.py`
17+
- `autogalaxy_workspace/scripts/multi/features/imaging_and_interferometer/modeling.py`
18+
- `autolens_workspace/scripts/group/features/advanced/mass_stellar_dark/chaining.py`
19+
- `autolens_workspace/scripts/guides/modeling/advanced/hierarchical.py`
20+
- `autolens_workspace/scripts/imaging/features/advanced/shapelets/modeling.py`
21+
- `autolens_workspace/scripts/interferometer/features/subhalo/detect/start_here.py`
22+
- `autolens_workspace/scripts/interferometer/features/extra_galaxies/slam.py`
23+
- `autolens_workspace/scripts/multi/features/imaging_and_interferometer/modeling.py`
24+
25+
## Required work
26+
27+
1. Reproduce from clean task worktrees with the release profile and current library
28+
sources; distinguish stale cached output from newly written samples.
29+
2. Trace model path construction, stored kwargs, path aliases, and collection/list
30+
indices through `Samples`, `Sample`, and result loading.
31+
3. Fix the owning PyAutoFit contract if valid current-model paths cannot resolve.
32+
Do not weaken workspace scripts or silently ignore missing parameters.
33+
4. Add focused PyAutoFit regression tests covering single, multi-analysis, chained,
34+
list-profile, and multi-dataset paths represented above.
35+
5. Run PyAutoFit pytest, then rerun all nine scripts under their release profiles.
36+
37+
Preserve tutorial prose and make workspace edits only where a script genuinely uses a
38+
retired API rather than exposing a library defect.

0 commit comments

Comments
 (0)