diff --git a/.claude/skills/af_chain_searches.md b/.claude/skills/af_chain_searches.md new file mode 120000 index 0000000..e7704f9 --- /dev/null +++ b/.claude/skills/af_chain_searches.md @@ -0,0 +1 @@ +../../skills/af_chain_searches.md \ No newline at end of file diff --git a/.claude/skills/af_custom_analysis.md b/.claude/skills/af_custom_analysis.md new file mode 120000 index 0000000..8f823f3 --- /dev/null +++ b/.claude/skills/af_custom_analysis.md @@ -0,0 +1 @@ +../../skills/af_custom_analysis.md \ No newline at end of file diff --git a/.claude/skills/af_debug_fit_failure.md b/.claude/skills/af_debug_fit_failure.md new file mode 120000 index 0000000..f4e81a2 --- /dev/null +++ b/.claude/skills/af_debug_fit_failure.md @@ -0,0 +1 @@ +../../skills/af_debug_fit_failure.md \ No newline at end of file diff --git a/.claude/skills/af_plot_fit.md b/.claude/skills/af_plot_fit.md new file mode 120000 index 0000000..62c8d9b --- /dev/null +++ b/.claude/skills/af_plot_fit.md @@ -0,0 +1 @@ +../../skills/af_plot_fit.md \ No newline at end of file diff --git a/.claude/skills/af_simulate_dataset.md b/.claude/skills/af_simulate_dataset.md new file mode 120000 index 0000000..0e84dd5 --- /dev/null +++ b/.claude/skills/af_simulate_dataset.md @@ -0,0 +1 @@ +../../skills/af_simulate_dataset.md \ No newline at end of file diff --git a/skills/README.md b/skills/README.md index 4e21082..71fd255 100644 --- a/skills/README.md +++ b/skills/README.md @@ -25,8 +25,6 @@ configured) via symlinks; the canonical files live here. ## Index -> A follow-up tranche (`af_chain_searches`, `af_custom_analysis`, `af_simulate_dataset`, -> `af_plot_fit`, `af_debug_fit_failure`) is planned; everything below is complete. ### Meta @@ -57,6 +55,16 @@ configured) via symlinks; the canonical files live here. inspection of the Result; runtime triage. - [`af_load_results.md`](./af_load_results.md) — posterior summaries, errors, evidence, and bulk result loading via the aggregator. +- [`af_chain_searches.md`](./af_chain_searches.md) — pass one fit's result into the + next fit's priors or start points; staged pipelines. +- [`af_custom_analysis.md`](./af_custom_analysis.md) — extra Analysis inputs, custom + Visualizer (live fit figures), custom Result. +- [`af_simulate_dataset.md`](./af_simulate_dataset.md) — simulate from a composed model + with recorded truth; the recovery check. +- [`af_plot_fit.md`](./af_plot_fit.md) — corner/diagnostic plots via `autofit.plot`, + model-vs-data + residuals via matplotlib over instances. +- [`af_debug_fit_failure.md`](./af_debug_fit_failure.md) — the strict-order triage: + likelihood → priors → data → sampler. ### Domain adaptation diff --git a/skills/af_chain_searches.md b/skills/af_chain_searches.md new file mode 100644 index 0000000..fe527dd --- /dev/null +++ b/skills/af_chain_searches.md @@ -0,0 +1,120 @@ +--- +name: af_chain_searches +description: Chain non-linear searches — pass one fit's result into the next fit's priors or start points, so a cheap simplified fit hands a focused parameter space to the expensive full fit. Covers `result.instance` (fix at fitted values), `result.model` (posterior-updated priors), the `model_centred*` variants, and the Initializer start-point classes. Use when the user says "use the first fit to seed the second", "fix these components at the previous result", "chain searches", or wants a staged/pipelined analysis. Not for choosing a single sampler (af_configure_search) or combining datasets (that is the factor graph). +user-invocable: true +--- + +# Chaining searches + +A staged analysis fits a deliberately simplified problem first, then uses its result to +focus the real fit: fewer wasted evaluations, more robust convergence, and each stage +cheap enough to inspect before committing to the next. PyAutoFit makes the hand-off a +one-liner — the design question is *what* to pass, and how tightly. + +## Orient + +A completed `Result` offers three hand-off currencies +(`PyAutoFit:autofit/non_linear/result.py`): + +- **`result.instance.`** — the max-likelihood *values*, passed as **fixed** + (no longer free). Use for components the next stage treats as known. +- **`result.model.`** — the component with **priors updated from the + posterior**. The next search explores around stage 1's answer instead of the cold + prior. +- **`result.model_centred*`** — explicit-width variants when you want to control the + hand-off width rather than inherit the posterior's: `model_centred_absolute(a=...)` + (GaussianPriors of absolute sigma `a` around the fitted values), + `model_centred_relative(r=...)` (sigma = `r` × each value), + `model_centred_max_lh_bounded(b=...)`. + +Canonical runnable example: `autofit_workspace:scripts/features/search_chaining.py`. +Background: `wiki/core/concepts/initialization_and_chaining.md` — including the design +rule that a chain must *narrow* priors, never bias them. + +## Ask + +- What does stage 1 fix or learn, and what stays free in stage 2? (The split *is* the + pipeline design.) +- Fixed values or refreshed priors for each passed component — is stage 1's answer + trustworthy enough to freeze? + +## Branch — pass fixed values vs updated priors + +```python +""" +__Chained Fit__ + +Stage 1 fitted the left Gaussian alone. Stage 2 fixes it at the fitted values +(`result.instance`) while fitting the right Gaussian fresh — the left component costs +zero parameters in stage 2 (`PyAutoFit:autofit/non_linear/result.py`). +""" +model_2 = af.Collection( + gaussian_left=result_1.instance.gaussian_left, # fixed at stage-1 values + gaussian_right=af.Model(Gaussian), # free, cold priors +) + +""" +__Prior Passing__ + +If stage 1's answer should guide but not freeze, pass `result.model` instead: the +component arrives with priors centred on the stage-1 posterior. +""" +model_3 = af.Collection( + gaussian_left=result_1.model.gaussian_left, # free, posterior-updated priors + gaussian_right=af.Model(Gaussian), +) +``` + +Width control when the posterior's own width is not what you want handed on +(anti-lock-in: several × the stage-1 sigma is the usual safe choice): + +```python +model_3 = result_1.model_centred_relative(r=0.5) # sigma = 0.5 × each fitted value +``` + +## Branch — start points (same sampler, warmer start) + +When the model is unchanged and only the *starting region* should improve — e.g. +seeding MCMC after a quick MLE — use an initializer rather than touching priors +(`PyAutoFit:autofit/non_linear/initializer.py`): + +```python +""" +__Start Point__ + +Walkers start inside the given bounds; priors (and therefore the posterior and any +evidence) are unchanged — model.info confirms it. +""" +initializer = af.InitializerParamBounds( + {model.centre: (49.0, 51.0), model.sigma: (9.0, 11.0)} +) +search = af.Emcee(name="warm_start", nwalkers=30, nsteps=500, initializer=initializer) +``` + +`InitializerBall` / `InitializerParamStartPoints` are the tighter variants; parameters +not named fall back to prior draws. Start points change *where sampling begins*, prior +passing changes *the inference itself* — say which you're doing out loud. + +## Design rules (from the chaining wiki page) + +1. Name every link's provider — never assume "something reasonable" seeds a consumer. +2. Chains narrow, never bias: a follow-up prior tight enough to lock in stage 1's + systematics has replaced inference with anchoring. +3. Journal each link (`wiki/project/`): what was passed, in which currency, and why — + a chained result is only reproducible if the seeding is. + +## Combine + +- Stage design usually pairs with `af_configure_search` (cheap sampler for stage 1, + production sampler for stage 2) and `af_load_results` (inspect stage 1 before + trusting it). +- Multi-dataset problems chain *into* the factor graph: fit one dataset alone, pass + `result.model` as the shared components' starting priors + ([[../wiki/core/concepts/graphical_models_and_ep]]). + +## Further reading + +- **Student / new to inference** — HowToFit's search-chaining chapter. +- **General reference** — [RTD: search chaining](https://pyautofit.readthedocs.io/en/latest/features/search_chaining.html). +- **Experienced PyAutoFit user** — `autofit_workspace:scripts/features/search_chaining.py` + and `scripts/searches/start_point.py`. diff --git a/skills/af_custom_analysis.md b/skills/af_custom_analysis.md new file mode 100644 index 0000000..f013595 --- /dev/null +++ b/skills/af_custom_analysis.md @@ -0,0 +1,122 @@ +--- +name: af_custom_analysis +description: Extend an Analysis class beyond plain likelihood wrapping — extra data/config inputs in __init__, a custom Visualizer that plots model fits during and after the search, and a custom Result carrying science-specific attributes. Use when the user wants on-disk visualization of fits as the search runs, richer Result objects, or an Analysis carrying masks/kernels/covariances. Builds on af_wrap_likelihood (do that first if the likelihood isn't wrapped yet); not for the wrapping itself. +user-invocable: true +--- + +# Customizing an Analysis + +`af_wrap_likelihood` gets your likelihood into PyAutoFit; this skill is what you reach +for when the wrapper should *do more* — carry every input your likelihood needs, draw +its own diagnostic figures while the search runs, and return a `Result` that speaks +your science's language. All three extensions are class-attribute overrides on the +same `af.Analysis` you already have (`PyAutoFit:autofit/non_linear/analysis/`). + +## Orient + +The canonical worked example is `autofit_workspace:scripts/cookbooks/analysis.py` — +its sections map one-to-one onto the branches below. The constitution's rule stands +throughout: extra inputs and visualization never alter the likelihood's numerics. + +## Ask + +- What extra inputs does the likelihood genuinely need at call time (mask, kernel, + covariance, instrument config)? Everything else stays out of the class. +- What figure would tell *you* at a glance whether a fit is healthy? That's the + Visualizer's job — not publication plots, health plots. +- What derived quantities do you reach for from every result? Those belong on a + custom `Result`. + +## Branch — extra `__init__` inputs + +Anything the likelihood needs is constructor state — computed once, reused per call: + +```python +""" +__Analysis__ + +The Analysis carries everything the likelihood needs: here a noise covariance, a mask +and a convolution kernel. Expensive derived state (e.g. a Cholesky factor) is computed +once here, not per likelihood call — packaging, never a numerics change +(`autofit_workspace:scripts/cookbooks/analysis.py` __Customization__). +""" +class Analysis(af.Analysis): + def __init__(self, data, noise_covariance_matrix, mask, kernel): + super().__init__() + self.data = data + self.mask = mask + self.kernel = kernel + self.noise_covariance_matrix = noise_covariance_matrix +``` + +## Branch — a custom Visualizer + +Override the `Visualizer` class attribute; PyAutoFit calls its hooks with the +search's current best instance, writing into the search's own `output/` tree: + +```python +""" +__Visualization__ + +`visualize_before_fit` runs once (plot the data being fitted); `visualize` runs at the +search's update intervals and at completion, receiving the current max-likelihood +instance — the live health check for a long fit +(`autofit_workspace:scripts/cookbooks/analysis.py` __Visualization__). +""" +class Visualizer(af.Visualizer): + @staticmethod + def visualize_before_fit(analysis, paths: af.DirectoryPaths, model): + ... # plot analysis.data into paths' image directory + + @staticmethod + def visualize(analysis, paths: af.DirectoryPaths, instance, during_analysis): + ... # plot instance's model against analysis.data + residuals + + +class Analysis(af.Analysis): + Visualizer = Visualizer + ... +``` + +For 1D-profile problems, `af.ex.plot_profile_1d(xvalues=..., profile_1d=..., +output_path=pathlib.Path(...), output_filename=...)` is a ready-made figure helper +(`output_path` must be a `pathlib.Path`, not a str). Quote the output +directory to the user — the point of a Visualizer is that someone looks. + +## Branch — a custom Result + +Override `Result` (and `make_result` when construction needs extra inputs) so every +fit returns your science's objects directly: + +```python +""" +__Custom Result__ + +result.best_model_data becomes a first-class attribute of every fit of this Analysis — +no re-deriving in every notebook (`autofit_workspace:scripts/cookbooks/analysis.py` +__Custom Result__ section; `PyAutoFit:autofit/non_linear/analysis/`). +""" +class Result(af.Result): + @property + def best_model_data(self): + xvalues = np.arange(self.analysis.data.shape[0]) + return self.instance.model_data_from(xvalues=xvalues) + + +class Analysis(af.Analysis): + Result = Result + ... +``` + +## Combine + +- The upgraded Analysis drops straight into `af_run_search` and the aggregator flow of + `af_load_results` — custom Result attributes come back on reload too. +- Fits that misbehave despite good visuals route to `af_debug_fit_failure`. +- Offer (default-yes) a `wiki/project/` entry: what the Analysis now carries, what the + Visualizer draws, what the Result exposes. + +## Further reading + +- **General reference** — [RTD: analysis cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/analysis.html). +- **Experienced PyAutoFit user** — `autofit_workspace:scripts/cookbooks/analysis.py`. diff --git a/skills/af_debug_fit_failure.md b/skills/af_debug_fit_failure.md new file mode 100644 index 0000000..f75f5c5 --- /dev/null +++ b/skills/af_debug_fit_failure.md @@ -0,0 +1,114 @@ +--- +name: af_debug_fit_failure +description: Triage a misbehaving fit — garbage posteriors, stuck samplers, NaN likelihoods, truth not recovered, walls of sampler warnings. A strict-order diagnostic flowchart: likelihood sanity first, then prior coverage, then data pathology, then sampler diagnostics — because each later stage is meaningless if an earlier one is broken. Use when the user says "the fit isn't working", "the posterior looks wrong", "the sampler is stuck", or a simulation recovery failed. Not for slow-but-correct fits (that is a chaining/budget conversation, af_chain_searches / af_configure_search). +user-invocable: true +--- + +# Debugging a failing fit + +Fit failures are diagnosed in strict order: **likelihood → priors → data → sampler**. +The order matters because every stage assumes the ones before it — retuning a sampler +on top of a sign-flipped likelihood just converges to the wrong answer faster. Resist +the reflex to swap samplers first; it's stage 4 for a reason. + +## Orient + +Get the failure's shape before touching anything (one question each): + +- *Instant garbage* (finishes fast, posterior absurd) → almost always stage 1. +- *Stuck / crawling* (log-likelihood plateaus at a bad value) → stages 2–3. +- *Crashes / NaN warnings* → stage 3 (data) or stage 1 (likelihood domain errors). +- *Simulation truth not recovered* → the best failure to have; run all four stages + with the truth in hand (`af_simulate_dataset`). + +## Stage 1 — likelihood sanity + +The wrapper validation trio from `af_wrap_likelihood`, run *again*, now with +suspicion: + +```python +""" +__Likelihood Sanity__ + +A known-good instance must score a finite float, match the user's function called +directly, and beat a known-bad instance. A bad instance scoring HIGHER is the classic +sign flip (chi-squared returned where a log likelihood is expected) — the sampler then +dutifully maximises misfit. +""" +good = MyModel(centre=50.0, amplitude=25.0, width=10.0) +bad = MyModel(centre=-1e3, amplitude=1e-6, width=0.1) +ll_good = analysis.log_likelihood_function(good) +ll_bad = analysis.log_likelihood_function(bad) +assert np.isfinite(ll_good) and ll_bad < ll_good +``` + +Also at this stage: parameter-order bugs (vector-convention wrappers scoring +`[a, b, c]` as `[b, a, c]` — check by perturbing one parameter and watching the +response), and unit mismatches between instance attributes and what the function +expects. + +## Stage 2 — prior coverage + +```python +""" +__Prior Coverage__ + +Draws from the priors must bracket every plausible solution. A truth (or expected +value) outside the drawn range cannot be found by any sampler — that is a prior bug, +not a sampler bug. +""" +for _ in range(5): + print(vars(model.random_instance())) +print(model.info) +``` + +Red flags: a fitted parameter hugging a prior limit (widen it, or justify the bound); +`LogUniformPrior` limits crossing zero; a linked/fixed parameter you forgot is no +longer free (`model.prior_count` says). + +## Stage 3 — data pathology + +The constitution's inspection gate, re-run with hindsight: plot the data *with the +current best model over it* (`af_plot_fit`). NaN/inf in data or noise-map, zero or +negative noise values, outliers the noise model doesn't cover, masked regions leaking +into the likelihood. Fix the **producer** of bad values — a crash you can see beats a +silent guard that biases every fit after it (never add clipping inside the +likelihood; that is the user's science to change or not). + +## Stage 4 — sampler diagnostics + +Only now, with 1–3 clean, per family +([[../wiki/core/concepts/non_linear_search]] and the family pages): + +- **Nested**: evidence wandering between reruns or a known mode missing → raise + `n_live`/`nlive`; check `log_likelihood_vs_iteration` for a plateau that never + turns over (budget too small). +- **Ensemble MCMC**: auto-correlation time ~ chain length → not converged (longer + chains, better start); walkers in disjoint clumps → multi-modal posterior, wrong + tool — use nested sampling. +- **NUTS**: divergences / all-warmup → bad start or non-smooth likelihood; seed from + a provider ([[../wiki/core/concepts/initialization_and_chaining]]). +- Cross-check with `af.Drawer`: if the "fit" barely beats prior draws, the data are + uninformative for this model — a science conclusion, not a bug. + +## The decisive instrument + +When stages disagree, simulate: generate data from the current model at plausible +parameters (`af_simulate_dataset`) and run the identical pipeline. Recovery works → +the pipeline is fine and the real data/model mismatch is the finding. Recovery fails +→ you've reproduced the bug with a known answer, and stages 1–4 will now pin it. + +## Combine + +- Fixes route back to their owners: priors → `af_compose_model`; wrapper → + `af_wrap_likelihood`; budget/chaining → `af_configure_search` / + `af_chain_searches`. +- Journal the diagnosis (`wiki/project/`, default-yes): symptom, stage, root cause — + the next failure in this project will look the same. + +## Further reading + +- **Student / new to inference** — HowToFit's fitting chapters (what healthy + convergence looks like). +- **Experienced PyAutoFit user** — `autofit_workspace:scripts/cookbooks/analysis.py` + (the likelihood contract being checked in stage 1). diff --git a/skills/af_plot_fit.md b/skills/af_plot_fit.md new file mode 100644 index 0000000..8e97f01 --- /dev/null +++ b/skills/af_plot_fit.md @@ -0,0 +1,105 @@ +--- +name: af_plot_fit +description: Visualise fits and posteriors — corner plots and search diagnostics via autofit.plot (aplt), model-vs-data and residual figures via matplotlib over model instances, and the af.ex.plot_profile_1d helper for 1D profiles. Use when the user says "plot the fit", "show me the corner plot", "plot residuals", or wants figures from a completed search. Not for the in-search Visualizer hooks (that is af_custom_analysis) or for loading the results being plotted (af_load_results). +user-invocable: true +--- + +# Plotting fits and posteriors + +Two kinds of figure answer two different questions. **Posterior plots** (corner, +trace) ask *what did the inference conclude and can I trust it*; **fit plots** +(model-vs-data, residuals) ask *does the model actually describe the data*. A healthy +analysis looks at both — a beautiful corner plot over a terrible fit is the classic +trap. + +## Orient + +PyAutoFit ships search-level plotting as `autofit.plot` (imported `aplt`); at the +pinned stack the entry points are `corner_cornerpy`, `corner_anesthetic`, +`log_likelihood_vs_iteration`, `subplot_parameters` and `output_figure` +(`PyAutoFit:autofit/plot/` — verify with `dir(aplt)`, never memory). Domain plots are +deliberately *not* framework-owned: your data is yours, so model-vs-data figures are +plain matplotlib over model instances. + +The plot-output conventions from `_style.md` apply: deterministic paths under +`scripts/scratch//`, print the absolute path, quote it back and offer to +open it. + +## Ask + +- Which question first — posterior health or fit quality? +- One fit or a comparison across fits (→ load via the aggregator first, + `af_load_results`)? + +## Branch — posterior plots + +```python +""" +__Corner Plot__ + +The pairwise posterior — degeneracies, multi-modality and prior-edge pile-ups are all +visible here and invisible in a summary table (`PyAutoFit:autofit/plot/`). +""" +import autofit.plot as aplt + +samples = result.samples +aplt.corner_cornerpy(samples=samples) +``` + +`log_likelihood_vs_iteration(samples=samples)` is the convergence-at-a-glance +diagnostic; `subplot_parameters(samples=samples)` panels the per-parameter behaviour. +Read a corner plot for: contours truncated at a prior limit (prior too tight), +banana/curved degeneracies (consider reparametrising), separated islands (multi-modal +— ensemble MCMC results are suspect, [[../wiki/core/concepts/mcmc_and_hmc]]). + +## Branch — fit plots (model vs data) + +Instances are real instances of your classes, so the model curve is one method call: + +```python +""" +__Fit + Residuals__ + +Model-vs-data and normalised residuals from the max-likelihood instance. Residuals +should look like noise: structure in them is model error, and no posterior statistic +substitutes for looking (`_style.md` "Plot output and path announcement"). +""" +import matplotlib.pyplot as plt +import numpy as np + +best = result.samples.max_log_likelihood() +xvalues = np.arange(data.shape[0]) +model_data = best.model_data_from(xvalues=xvalues) + +fig, (ax0, ax1) = plt.subplots(2, 1, sharex=True, height_ratios=[3, 1]) +ax0.errorbar(xvalues, data, yerr=noise_map, fmt=".k", ms=3, elinewidth=0.5) +ax0.plot(xvalues, model_data, "r-") +ax1.plot(xvalues, (data - model_data) / noise_map, ".k", ms=3) +ax1.axhline(0.0, color="r", lw=0.5) +ax1.set_ylabel("residuals / sigma") + +out = "scripts/scratch/my_fit/fit.png" +plt.savefig(out, dpi=150, bbox_inches="tight") +print(f"Saved to: {__import__('pathlib').Path(out).resolve()}") +``` + +Posterior *uncertainty on the curve*: overplot `model_data_from` for a few dozen +posterior draws at low alpha — the honest band, no Gaussian assumption. For 1D +profiles, `af.ex.plot_profile_1d(xvalues=..., profile_1d=..., +output_path=pathlib.Path(...), output_filename=...)` wraps the same figure in one +call — note `output_path` must be a `pathlib.Path`, not a string (the helper joins +with the `/` operator; a str raises TypeError). + +## Combine + +- The numbers behind the figures: `af_load_results`. In-search live figures: the + Visualizer in `af_custom_analysis`. Residual structure you can't explain: + `af_debug_fit_failure`. +- Publication-bound figures belong in a science project's `results/figures/` with a + run manifest (`start-new-project` Phase 2), not in scratch. + +## Further reading + +- **General reference** — [RTD: result cookbook](https://pyautofit.readthedocs.io/en/latest/cookbooks/result.html) (plotting sections). +- **Experienced PyAutoFit user** — `autofit_workspace:scripts/plot/` (per-sampler + plotter scripts). diff --git a/skills/af_simulate_dataset.md b/skills/af_simulate_dataset.md new file mode 100644 index 0000000..fd3a91b --- /dev/null +++ b/skills/af_simulate_dataset.md @@ -0,0 +1,107 @@ +--- +name: af_simulate_dataset +description: Simulate a dataset from a composed model — draw or fix "true" parameters, generate noiseless model data, add noise, and save it alongside a truth record, so fits can be validated against a known answer before touching real data. Use when the user says "simulate data", "make a test dataset", "check the pipeline recovers known inputs", or before a first fit on an expensive real dataset. Not for preparing real data (that is the data-inspection conversation in af_run_search). +user-invocable: true +--- + +# Simulating a dataset + +A simulation is the only fit where you know the answer. Every new pipeline earns its +first real-data run by first recovering known inputs: it validates the likelihood, the +priors, the sampler budget, and your read of the posterior, all at once. The bundled +`dataset/gaussian_x1/` was made exactly this way — its README records the truth. + +## Orient + +The recipe is three lines of physics and one of bookkeeping: instantiate your model +class with chosen "true" values, evaluate its model data, add noise drawn from the +noise model you claim in the likelihood, save data + noise-map + truth. The workspace's +generators are the canonical reference +(`autofit_workspace:scripts/simulators/simulators.py`). + +## Ask + +- Simulate with *hand-picked* truth (a clean pedagogical recovery) or truth *drawn + from the priors* (a stress test of the whole prior volume)? +- What noise model does the likelihood assume? The simulation must add **that** noise + — simulating Gaussian noise then fitting a Poisson likelihood "validates" nothing. + +## Branch — the simulation script + +```python +""" +Simulate: Gaussian 1D +===================== + +Generate noisy 1D data from a Gaussian with recorded truth, for validating the fit +pipeline before real data. + +__Contents__ + +- **Truth:** The chosen true parameters (recorded, always). +- **Model data:** Noiseless evaluation of the model. +- **Noise:** Gaussian noise matching the likelihood's assumption. +- **Output:** data + noise-map + truth record. +""" +import numpy as np +import autofit as af + +""" +__Truth__ + +Hand-picked here; `model.random_instance()` draws truth from the priors instead when +stress-testing. Either way the values are written down — an unrecorded simulation +truth is a lost validation (`autofit_workspace:scripts/simulators/simulators.py`). +""" +rng = np.random.default_rng(1) +truth = Gaussian(centre=50.0, normalization=25.0, sigma=10.0) + +""" +__Model data + Noise__ + +The noise added is exactly what the likelihood assumes (independent Gaussian with this +noise-map); the noise-map is saved with the data because the fit needs the same one. +""" +xvalues = np.arange(100) +model_data = truth.model_data_from(xvalues=xvalues) +noise_map = np.full(xvalues.shape, 2.0) +data = model_data + rng.normal(0.0, noise_map) + +""" +__Output__ + +JSON via af.util keeps the round-trip trivial (`af.util.numpy_array_from_json` on the +fitting side). The truth goes in a README/info file next to the data — never only in +the script. +""" +af.util.numpy_array_to_json(data, file_path="dataset/my_sim/data.json", overwrite=True) +af.util.numpy_array_to_json(noise_map, file_path="dataset/my_sim/noise_map.json", overwrite=True) +``` + +For teaching sessions, `af.ex.Gaussian` / `af.ex.Exponential` are ready-made model +classes with `model_data_from` — the workspace simulators compose multi-component +datasets from exactly these. + +## The recovery check (the point of it all) + +Fit the simulation with the *production* setup (`af_run_search`) and require: + +1. Truth inside the quoted intervals at the expected sigma (most parameters within + 1σ; systematic misses mean bias, not bad luck). +2. Posterior widths plausible for the noise level. +3. (If evidence matters) ln Z stable across live-point settings. + +A recovery that fails is a gift: the pipeline is broken *and you know it* before the +real data ever saw it. Record the recovery table in `wiki/project/`. + +## Combine + +- Seed and validate priors with `af_compose_model`; wrap real likelihoods with + `af_wrap_likelihood` and validate the wrapper on simulated data first. +- Failed recoveries route to `af_debug_fit_failure` with the enormous advantage of a + known answer. + +## Further reading + +- **General reference** — [RTD: the basics](https://pyautofit.readthedocs.io/en/latest/overview/the_basics.html) (simulate → fit → recover). +- **Experienced PyAutoFit user** — `autofit_workspace:scripts/simulators/simulators.py`.