feat: birth autofit_assistant — the generic inference assistant

[Jammy2211]

Overview

Birth autofit_assistant: the PyAutoFit AI Assistant, following the proven autolens_assistant pattern (AGENTS.md canonical + skills + wiki + science-project machinery), paired to PyAutoFit + autofit_workspace. The defining inversion versus the lensing assistant: the user brings their own scientific domain, so domain adaptation (paper ingestion, likelihood wrapping, model composition) is the first-class first-run experience rather than an afterthought.

Launch decisions (human, 2026-07-10, in-session): repo born public; skeleton hand-authored with the clone-v1 plan as a file-inventory checklist only; worked demos are the canonical 1D Gaussian + an SNe cosmology distance-modulus fit on open data.

Plan

• Phase 0 — birth + skeleton (direct to main; an empty repo cannot take a PR for its first push): README, LICENSE, AGENTS.md/CLAUDE.md, modes/, config/, Makefile, activate.sh, sources.yaml, wiki scaffolding, safety invariants generalised for inference.
• Phase 1 — workspace pairing: port the autoassistant/ package (API audit + gate, refresh_api_docs, to_notebook, citation currency, benchmark runner) with af_ naming; first tranche of af_* skills grounded in autofit_workspace/scripts/.
• Phase 2 — core statistics wiki: priors, model composition, per-sampler pages (matched to the installed PyAutoFit roster), nested sampling, MCMC/HMC, initialization/chaining, EP & graphical models, evidence/model comparison, samples/aggregator.
• Phase 3 — domain adaptation + demos: af_adapt_to_domain onboarding driving paper ingestion, likelihood wrapping (Analysis-class generation around user code), and model composition; start-new-project; the two worked demos.
• Phase 4 — parity + publish gate: benchmarks with card↔README parity, teacher mode anchored to HowToFit, HPC link, llms.txt, smoke/citation CI, Heart newborn_validation checklist.

Phases 1–4 each end at PR-open per the autonomy contract; merge stays human.

Detailed implementation plan

Affected Repositories

• autofit_assistant (primary — new, develops in-place)
• autofit_workspace (read-only grounding for skills)
• PyAutoFit / PyAutoConf (read-only; sources.yaml clones @ main)
• autolens_assistant (read-only reference; its benchmark-calibration worktree claim untouched)

Branch Survey

Repository	Current Branch	Dirty?
./autofit_assistant	(empty, new)	n/a

Branch model: Phase 0 lands directly on main (birth commits — empty repos cannot receive PRs). Phases 1–4 on feature/afa-phase-<n>-<slug> branches, PRs into main, stacked if a prior phase's PR is unmerged.

Implementation Steps

Phase 0 (main):

1. README.md — what the assistant is, quickstart, the domain-adaptation pitch (agentic-AI-first framing per docs doctrine; no llms.txt mechanics in user prose).
2. AGENTS.md — canonical instructions: session-start ritual (maintainer-mode check, wiki/project/profile.md calibration, API drift-check), safety invariants (code gate, never write into output//sources/, wiki/core/ read-only, source-edit boundary, never rewrite history; the real-data gate generalises to "plot/inspect the user's dataset before first fit"), the three-layer model (instructions/skills/wiki).
3. CLAUDE.md (@AGENTS.md stub), .gemini/settings.json pointer.
4. modes/{assistant,teacher,maintainer}.md — teacher anchors to HowToFit.
5. config/, Makefile, activate.sh, sources.yaml (PyAutoFit, PyAutoConf, autofit_workspace @ main), .gitignore (output/, sources/, pycache, .maintainer), LICENSE (family-standard).
6. wiki/{core,literature,project}/ scaffolding — literature ships near-empty by design with schema AGENTS.md (grown per-user during domain adaptation); skills/ dir with _style.md + _bootstrap_skill.md meta-skills.

Phase 1 (PR):

1. Port autoassistant/ → audit_skill_apis.py (+ PreToolUse gate hook + self-enforcement path for non-hook harnesses), refresh_api_docs.py, to_notebook.py, literature.py --check-citations (incl. fifth wiki-currency leg), benchmark.py, tests.
2. First af_* skills grounded against autofit_workspace/scripts/{overview,cookbooks,features,searches,model,plot,simulators}: setup_environment, compose_model, configure_search, run_search, chain_searches, load_results, custom_analysis, simulate_dataset, plot_fit, debug_fit_failure, to_notebook, audit_skill_apis, refresh_api_docs, update_wiki.
3. API audit green against installed stack before PR.

Phase 2 (PR):

1. wiki/core/concepts/: bayesian-inference, model-composition-and-priors (incl. sigma=0 point-mass idiom, latent variables), non-linear-search overview, one page per shipped sampler family (roster audited from installed PyAutoFit at write time), nested-sampling, mcmc-and-hmc, initialization-and-chaining, sampler-benchmarks, graphical-models-and-ep, evidence-and-model-comparison, samples-and-posteriors.
2. EP/graphical + sampler pages are generalised public rewrites of PyAutoMemory/methods_wiki content — PyAutoMemory itself is never referenced (privacy seam); originals remain the private superset.
3. wiki/core/stack/: overview, autoconf, autofit. wiki/core/operations/: installation, sandbox, hpc.

Phase 3 (PR):

1. Skills: af_adapt_to_domain (onboarding interview → wiki/project/profile.md + domain journal), af_ingest_paper (grows wiki/literature per-user), af_wrap_likelihood (user code → Analysis class; data/noise conventions, log_likelihood_function contract, JAX triage), af_compose_model; start-new-project + contribute-upstream project-workflow skills.
2. Demos: 1D Gaussian (canonical) + SNe Ia distance-modulus fit on open data (e.g. Pantheon+ subset) exercising af_wrap_likelihood end-to-end, incl. a model-comparison story.

Phase 4 (PR):

1. benchmarks/ (prompts/, runs/, RESULTS.md, AGENTS.md) with test-enforced card↔README parity; inference-flavoured cards.
2. Teacher mode content wiring to HowToFit chapters; hpc/ batch templates + sync (CPU-first, GPU path where the likelihood is JAX-able); llms.txt; smoke CI via the reusable family workflow; firewall/policy + url_fixups riders.
3. Heart newborn_validation.md checklist run (repo already public — the checklist still validates content/hygiene).

Key Files

• autolens_assistant/AGENTS.md — the reference pattern being generalised
• autolens_assistant/autoassistant/*.py — tooling being ported with af_ naming
• PyAutoMemory/methods_wiki/concepts/*.md — private sources for Phase 2 public rewrites
• PyAutoMind/issued/autofit_assistant_birth.md — the design prompt (four pillars)

Notes

• Feature Agent recommended re-homing as research/; overruled by the judgment tier at launch — scoping was already completed at intake (design in the prompt), and the prior research anchor research/autofit_assistant/autofit_assistant_planning.md is absorbed here (retire it when this ships).
• Autonomy: supervised effective (--auto launch 2026-07-10). No Heart-ack given at launch — any Heart YELLOW at ship parks. Judgment forks become batched questions on this issue.

Original Prompt

Click to expand starting prompt

autolens_assistant is proving to be very good, and is now excelling at various science cases.

It is time to make autofit_assistant, noting that this has the following differences:

• Autolens is tied to a specific scientific domain (lensing), whereas autofit_assistant is a tool to help someone perform inference in their own specific scientific domain. That means, when someone begins using autofit_assistant, one of their first tasks is probably going to be also training or adapting it to their scientific domain. This probably includes paper ingestion, manually providing code with a likelihood function (ideally) and the model composition.

• AutoFit assistant would benefit from a wiki on all the core statistics concepts it uses during inference albeit many are probably there from the general foundation model. Nevertheless, the EP wiki we made would be valuable here, probably some stuff specific to each source code sampler, maybe stuff on priors, have a think.

• like the autolens_assistant pairs to the autolens_workspace via skills, we want to do the exact same with the autofit assistant.

• All the core features of the autolens_assistant (making a science project, open data repository design, benchmarks, assistant and teacher mode, HPC link, etc) should be kept and designed suitable for autofit.

[Jammy2211]

Phase 0 landed — the assistant is born. 🎉

main now holds the birth commit (3667f84, 52 files): the AGENTS.md constitution (with the domain-adaptation section and the generalised safety invariants, including the new likelihood-code-is-the-user's rule), the three interaction modes, the skills framework (_style.md rewritten for domain-agnostic inference — the key change is a new voice rule: never invent domain knowledge; when wiki/literature/ is silent, offer ingestion instead of bluffing), the three-sub-wiki scaffolding with literature/ shipping near-empty by design, the project profile template (now led by a Scientific domain section, plus a likelihood-&-model-code inventory), sources.yaml, and config mirrored from autofit_workspace (minus its build/ notebook-generation configs, which belong to the workspace, not the assistant).

Everything not yet built is marked 🚧 in-repo with a pointer back to this issue, so the public repo is honest about its state from day one.

Next: Phase 1 — workspace pairing on feature/afa-phase-1-workspace-pairing: the autoassistant/ tooling port (API audit + gate, literature validation, notebook conversion, benchmark harness) and the first tranche of af_* skills grounded in autofit_workspace/scripts/.

[Jammy2211]

Phase 1 built and verified — parked at ship sign-off (supervised checkpoint-and-continue).

Commit d388ce1 on local branch feature/afa-phase-1-workspace-pairing (34 files, +4,313): the autoassistant/ tooling port (stack roster cut to autoconf+autofit, aliases af/aplt) and the first nine af_* skills.

Verification, all green:

• Test suite: 50/50 pass (the API-gate tests shell out and import autofit per case, so the run is slow by design).
• Symbol audit --scope all: 0 misses · citation paths: 0 missing · idiom lint: clean · --check-version: clean against the freshly written baseline (autofit 2026.7.6.649).
• Every skill code recipe executed against the installed stack. The audit earned its keep during authoring: it caught me writing af.Aggregator.from_directory from memory — the directory walker is autofit.aggregator.aggregator.Aggregator; af.Aggregator is the database variant. The skill now teaches the distinction.

Two scope judgment calls for your review (the batched questions):

1. Benchmark harness deferred to Phase 4. benchmark.py + its tests include repo-level card↔README parity tests that cannot pass before benchmarks/prompts/ exists (2 failures when included). I moved both files to Phase 4 where benchmarks/ lands, rather than shipping tests that fail or weakening them. OK?
2. Skill tranche split. Phase 1 ships 9 of the planned 14 skills (4 maintenance + 5 core inference). The remaining five — chain_searches, custom_analysis, simulate_dataset, plot_fit, debug_fit_failure — are deferred: custom_analysis folds naturally into Phase 3's af_wrap_likelihood work, the rest into a Phase 1b tranche alongside it. OK, or want any of them pulled forward into this PR?

On your answer: approve (or answer the two questions) → I push the branch and open the PR with this evidence in the body. Until then the branch stays local; Phase 2 (core statistics wiki) continues meanwhile on a stacked branch — no pending answer can invalidate it.

[Jammy2211]

Phase 2 built — the core statistics wiki is written (stacked on the parked Phase 1 branch as feature/afa-phase-2-core-wiki, commit afa0ffa, 17 files).

16 pages: 10 concepts (bayesian inference · model composition & priors · non-linear search · nested sampling · MCMC/HMC · MLE/optimizers · initialization & chaining · graphical models & EP · evidence & model comparison · samples & posteriors), 3 stack pages, 3 operations pages. The inference-methods material is a generalised public rewrite for a generic PyAutoFit user, grounded against the installed stack and the workspace cookbooks — the sampler roster documented is exactly what dir(af) exposes at the pinned version (Nautilus, DynestyStatic/Dynamic, Emcee, Zeus, BlackJAXNUTS, LBFGS, Drawer).

Currency discipline from day one — all four legs green: symbol audit 0 misses, citation paths 0 missing (75 citations resolve), idiom lint clean, and provenance --strict 0 errors/0 warnings: every page pins the release-tag commit shas (PyAutoFit 31537d5, PyAutoConf 5e784ca — the ^{commit} peel of the 2026.7.6.649 tags; the tag object shas fail reachability, a nice validation that the checker works) and carries a content_sha256 stamp. No legacy unpinned-main pages in this wiki — it is born fully pinned.

Ship of Phases 1+2 remains parked on the sign-off question two comments up. Continuing with Phase 3 — domain adaptation + demos (af_wrap_likelihood, af_ingest_paper, af_adapt_to_domain, start-new-project; 1D Gaussian + SNe cosmology worked demos) on a further stacked branch.

[Jammy2211]

Phase 3 built — domain adaptation + validated demos (commit 6e1426c on feature/afa-phase-3-domain-adaptation, stacked on Phases 1–2; 12 files, +2,745).

The defining skills, authored fresh (no lensing equivalent existed):

• af_wrap_likelihood — user code → Analysis, with a mandatory three-check pre-fit validation (finite, matches-direct-call, sign-convention). Building it caught a real recipe bug: instance_from_prior_medians() raises ConfigException on user classes without prior config, so validation uses plain hand-built instances and runs before the prior conversation — the skill now teaches exactly that.
• af_adapt_to_domain — the onboarding orchestrator over the three channels, writing durable knowledge to profile.md.
• af_ingest_paper — adapted so ingestion creates the near-empty-by-design domain wiki.
• start-new-project (full lifecycle, refer-back, manifests, publish gate — with the user's likelihood code as a first-class project citizen) + contribute-upstream.

Demos — both validated end-to-end with real inference on this machine:

• dataset/gaussian_x1: DynestyStatic (nlive=75) recovers centre=49.98, norm=24.85, sigma=9.84 vs truth (50, 25, 10).
• dataset/sne_cosmology (Pantheon+SH0ES public release, 1701 SNe): flat-ΛCDM Hubble-flow fit gives Ωm=0.344 +0.021/−0.019, H0=73.14 +0.27/−0.32 — consistent with Brout et al. 2022 (Ωm=0.334±0.018, H0=73.30±1.04); the diagonal-errors simplification is stated prominently in the dataset README with the required citations.

README example prompts now exercise the bundled demos (a fourth prompt covers domain adaptation). All validation legs green again (audit 0 · citations 0 missing · idioms clean · provenance --strict clean).

Run state: Phases 1–3 are built, verified, and stacked locally behind the one ship sign-off question (this comment). Phase 4 (benchmarks + teacher wiring + hpc/ + llms.txt + CI) is deliberately not started: its benchmark cards and parity test bind to the README and demos, which your review may amend — building it now would stack rework risk. On your sign-off I push the branches, open the PR(s), and continue Phase 4.

[Jammy2211]

Phase 4 built and shipped to PR-open — construction complete. 🎉

PR #3 (6782984, stacked on #2; retarget to main after #2 merges): benchmarks (harness + three inference cards with the test-enforced card↔README parity), hpc/ (sync CLI, CPU/GPU SLURM array templates, the autofit fit-script template), llms.txt, and the five-leg wiki-currency CI.

Heart newborn checklist (repo born public by human choice, so the checklist validates rather than gates): leg 1 symbol audit ✅ 0 misses · leg 2 link sweep ✅ (single intentional per-clone profile.md reference) · leg 3 wiki-currency legs all green locally, and the ported workflow runs on PR #3 itself · leg 4 (chat-surface smoke) remains — it is post-publish, human-side (modes/maintainer.md procedure against the public URL).

What the assistant now is, four phases after an empty repo this morning: a constitution with domain adaptation as the first-run experience; 14 skills (incl. af_wrap_likelihood and af_adapt_to_domain); a 16-page provenance-stamped statistics wiki; two demos validated with real inference (the SNe fit reproduces published Pantheon+ cosmology); benchmarks; HPC; CI.

Human acts remaining: merge #2, then retarget + merge #3; the chat-surface smoke; and two filed follow-ups — PyAutoBuild release-workflow wiring for the wiki-currency workflow_call, and the deferred skill tranche (af_chain_searches, af_custom_analysis, af_simulate_dataset, af_plot_fit, af_debug_fit_failure). This issue stays open as the tracker until #3 merges.

[Jammy2211]

Merged and complete. PR #2 and PR #3 are both in main (28fa6de, df4a34b); the four phase branches are deleted.

One post-sign-off fix rode in first: the wiki-currency CI on PR #3 correctly failed its version-drift leg — the baseline pinned the local source stack (2026.7.6.649) while PyPI's live release is 2026.7.9.1 (yesterday's armed nightly). No API-surface hash drift and all content legs were green against the newer release, so 62c34d2 did the honest re-pin (baseline rewritten against the released wheel in a clean venv, version.txt bumped, page pins moved to the 2026.7.9.1 tag commits); CI then passed. A satisfying first catch for the newborn's own immune system.

Remaining threads, all staged:

• Chat-surface smoke (newborn leg 4) → chore: chat-surface compatibility smoke (newborn checklist leg 4) #4 with the ready-to-paste per-surface checklist (public fetchability of llms.txt/AGENTS.md already verified).
• PyAutoBuild release wiring → filed as PyAutoMind/feature/pyautobuild/autofit_assistant_wiki_currency_wiring.md (safe/easy).
• Deferred skill tranche → filed as PyAutoMind/feature/autofit_assistant/deferred_skill_tranche.md (supervised/medium).

Closing this tracker — the assistant is born, built, validated, and live.