From f0309b84d1e3b7590b56e9f5cd4a3abfaae670d7 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Wed, 15 Jul 2026 15:50:02 +0100 Subject: [PATCH 1/2] fix: remove the .mode override; mode selection falls through to inference .mode was gitignored, so PR #34's removal of agent mode could not reach it. A stale sentinel kept winning the selection chain and resolving to a modes/agent.md that no longer exists, so inference never ran. Durable mode preference now lives only in profile.md, which is read and announced each session. AGENTS.md now also states that an unrecognised mode name falls through to inference instead of being improvised on. That rule is the load-bearing half: deleting the local .mode fixes one clone, but other clones have their own gitignored copy that no commit can reach. Co-Authored-By: Claude Opus 4.8 --- .gitignore | 4 ---- AGENTS.md | 10 ++++++---- README.md | 4 ++-- modes/maintainer.md | 2 +- 4 files changed, 9 insertions(+), 11 deletions(-) diff --git a/.gitignore b/.gitignore index f3a0699..5f65e59 100644 --- a/.gitignore +++ b/.gitignore @@ -47,10 +47,6 @@ papers/ # no newcomer-mode defaults). See AGENTS.md and modes/maintainer.md. .maintainer -# Interaction-mode override — optional per-clone file containing `teacher` or -# `assistant`. See AGENTS.md "Modes". `.maintainer` outranks it. -.mode - # Claude Code agent worktrees — ephemeral, machine-local. .claude/worktrees/ diff --git a/AGENTS.md b/AGENTS.md index 28ae82b..7ccb31f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -102,10 +102,12 @@ how it paces the work, not which workflows exist: execute with checkpoints — proactive but not silent; state in `wiki/project/`. The dial is in [`modes/assistant.md`](./modes/assistant.md) "The autonomy dial". -Select (first match): explicit instruction → `.mode` file → `profile.md` "Interaction mode" → -else **infer from the opening request** (fall back to **assistant**); `.maintainer` outranks -`.mode`. State an inferred mode in one line and invite correction; acknowledge an explicit -one only if it changes behavior. Read `modes/.md`; depth still follows +Select (first match): explicit instruction → `profile.md` "Interaction mode" → else **infer +from the opening request** (fall back to **assistant**); `.maintainer` outranks both. There +are exactly two mode names — a value that isn't one of them (e.g. `agent`, removed in July +2026) is **not** a mode: ignore it, say so in one line, and fall through to inference rather +than improvising. State an inferred mode in one line and invite correction; acknowledge an +explicit one only if it changes behavior. Read `modes/.md`; depth still follows `skills/_style.md` "Adaptive depth". --- diff --git a/README.md b/README.md index 7c180f8..f8a5784 100644 --- a/README.md +++ b/README.md @@ -49,8 +49,8 @@ manage an end-to-end lens modeling project directly on your machine. See The assistant works in two modes, and you never have to choose one — it **infers the mode from your first message and tells you which it picked** (e.g. *"Mode: teacher — I'll explain as we go."*). If it guesses wrong, just say so. To set the mode yourself, start your message -with it (the examples below do exactly that); to make a choice permanent, drop a `.mode` file -in the repo containing `teacher` or `assistant`. +with it (the examples below do exactly that); to make a choice permanent, record it under +"Interaction mode" in `wiki/project/profile.md`. - **Teacher** — *learn the workflow.* `Teacher mode: I'm new to PyAutoLens — how do I model this image?` - **Assistant** — *do the workflow.* `Assistant mode: set up a project for this dataset and write the first script.` diff --git a/modes/maintainer.md b/modes/maintainer.md index e536f6d..72c2e91 100644 --- a/modes/maintainer.md +++ b/modes/maintainer.md @@ -64,7 +64,7 @@ generalise anything pre-emptively; just avoid entangling the two sides. **Generic assistant infrastructure** (clones to any domain assistant near-verbatim): `AGENTS.md`'s skeleton (session start, safety invariants, three-layer model, mode selection, source-of-truth resolution, commit cadence), the Teacher/Assistant mode model -and `modes/` machinery (`.mode`, `.maintainer` sentinels), the skills framework +and `modes/` machinery (the `.maintainer` sentinel), the skills framework (`_style.md`, `_bootstrap_skill.md`, the README index conventions), the `core`/`literature`/`project` wiki split and its read-only/update rules, the science-project lifecycle (`start-new-project`, `contribute-upstream`), `sources.yaml` + the source From 536791883626e03f20bf466b1a76c1ef69e7eb78 Mon Sep 17 00:00:00 2001 From: Jammy2211 Date: Wed, 15 Jul 2026 15:50:08 +0100 Subject: [PATCH 2/2] =?UTF-8?q?docs:=20WIP=20paper=20revisions=20=E2=80=94?= =?UTF-8?q?=20do=20not=20merge=20as-is?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Snapshot of in-progress paper.md edits, carried along on this branch rather than left dirty in the working tree. Incomplete: two placeholder stubs ("The first are the wiki pages...", "The harness."), the Benchmark examples and Software design sections removed pending rewrite, an unresolved affiliation note, and typos. Retitles PyAutoLens as PyAutoLens-JAX in the summary against the existing Nightingale2021 citation. Unrelated to the .mode fix in the parent commit; drop this commit if the branch is merged before the rewrite lands. Co-Authored-By: Claude Opus 4.8 --- paper/paper.md | 104 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 77 insertions(+), 27 deletions(-) diff --git a/paper/paper.md b/paper/paper.md index fe4fd82..633b65c 100644 --- a/paper/paper.md +++ b/paper/paper.md @@ -5,6 +5,7 @@ tags: - astronomy - gravitational lensing - artificial intelligence + - agentic ai - large language models - natural language interfaces authors: @@ -13,7 +14,7 @@ authors: affiliation: 1 corresponding: true affiliations: - - name: Institute for Computational Cosmology, Durham University, United Kingdom + - name: Institute for Computational Cosmology, Durham University, United Kingdom # Update to My Newcastle affiliation index: 1 date: 15 July 2026 bibliography: paper.bib @@ -21,15 +22,76 @@ bibliography: paper.bib # Summary -Stage IV weak-lensing surveys, such as Euclid [@EuclidCollaboration2025] and the Vera C. Rubin Observatory [@LSSTDarkEnergyScienceCollaboration2012], are measuring increasingly large samples of galaxies, while strong-lensing searches are discovering rapidly growing numbers of galaxy-, group-, and cluster-scale lenses. These systems are observed through optical and infrared imaging, radio interferometry, point-source measurements of lensed quasars and supernovae, and weak-lensing shear catalogues, enabling studies of cosmology, dark matter, galaxy formation, and the early Universe. Mature open-source software such as PyAutoLens [@Nightingale2021] supports simulations, lensing calculations, and strong- and weak-lensing modelling across these datasets, but constructing a bespoke analysis can still require substantial effort to locate, adapt, and combine the relevant examples using the correct Python API and syntax. - -PyAutoLens-Assistant allows scientists to use natural language to describe the gravitational-lens analysis they want to perform. It provides a domain-specific interface to the documented and tested capabilities of PyAutoLens, supporting simulations, ray-tracing calculations, probabilistic modelling, data preparation, result interpretation, and visualization. Researchers can use it through a conversational AI assistant, such as ChatGPT, to ask questions and develop workflows interactively, or through agentic coding tools, such as Claude Code or Codex, which can inspect data, write and execute scripts, diagnose errors, analyse outputs, and iteratively refine an analysis. PyAutoLens-Assistant is grounded in curated, version-controlled documentation, examples, scientific reference material, and task-specific instructions, and produces explicit Python code and inspectable analysis products. +Stage IV weak-lensing surveys, such as Euclid [@EuclidCollaboration2025] and the Vera C. Rubin +Observatory [@LSSTDarkEnergyScienceCollaboration2012], are mapping the distribution of mass across the Universe on +an unprecedented scale, while strong-lensing searches are rapidly expanding samples of galaxy-, group-, and +cluster-scale lenses. Lensing analyses draw on optical and infrared imaging, submm and radio interferometry, strongly lensed +point sources and transients (e.g. quasars and supernovae), and weak-lensing shear catalogues. Together, these datasets +enable studies of cosmology, dark matter, galaxy formation, star formation, and the early Universe. +PyAutoLens-JAX [@Nightingale2021] provides open-source software for GPU-native, autodifferentiable joint lensing +analyses across these datasets and scales. However, combining datasets and performing inference across vast different +lensing scales is inherently complex, time consuming and error-prone, requiring substantial effort from the scientist +to find and adapt the PyAutoLens API and Python syntax to build their specific analysis pipelines. + +PyAutoLens-Assistant allows scientists to use natural language to construct complex, bespoke scripts for +gravitational-lens analysis. With agentic coding tools such as Claude Code or Codex, users describe the desired +analysis and the agent collates the data, writes and executes Python scripts and brings together the results. The scientist +can then use natural language to visualize, investigate and interpret the results. Alternatively, through web-based c +onversational AI assistants such as ChatGPT, users can ask PyAutoLens-Assistant questions and obtain fully documented +code needed to perform lens analysis task, which they then execute themselves manually. PyAutoLens-Assistant also +includes a Teacher Mode for users new to the software or gravitational lensing, which can explain all the core domain specific +concepts whilest directing them to documentation and Jupyter Notebook guides to help them further build their understanding. # Statement of need -Experienced PyAutoLens users often know exactly which scientific analysis they want to perform, but implementing it still requires substantial time assembling the appropriate Python workflow. An expert can quickly specify: "Perform multi-wavelength lens modelling of the F115W, F150W, F277W, and F444W JWST imaging of the COSMOS-Web Ring [@Casey2023] using a multi-Gaussian expansion lens-light model, a singular isothermal ellipsoid plus external shear mass model, and a Delaunay pixelized source reconstruction." Translating this concise scientific specification into executable code requires locating and combining several examples, loading and configuring each dataset, composing the model components with the correct API, and adapting the workflow to the system being analysed. As models incorporate more datasets, cluster-scale mass distributions, or joint strong- and weak-lensing constraints, this implementation burden increases even when the underlying scientific choices are already clear. PyAutoLens-Assistant reduces this overhead by translating natural-language specifications into explicit, executable, and reproducible PyAutoLens workflows. +Scientists can often inspect a dataset and know exactly which analysis they want to perform. For example, an experienced +strong-lensing scientist might examine multi-wavelength James Webb Space Telescope (JWST) observations of the +COSMOS-Web Ring [@Casey2023] and say: + +> I want to model the F115W, F150W, F277W, and F444W JWST imaging of the COSMOS-Web Ring simultaneously, which are in +> my local folder dataset/cosmos_web_ring. Model the lens light with a multi-Gaussian expansion and its mass with a +> singular isothermal ellipsoid plus external shear, and reconstruct the source on an adaptive rectangular mesh. For +> speed, run the analysis on my laptop GPU using a JAX optimizer that estimates only the maximum-likelihood solution. +> Plot the observed image at each wavelength in the top row, its lensed source model in the middle row, and its source +> reconstruction in the bottom row. + +The example above is a natural-language workflow: the scientist specifies the analysis in scientific terms, and the +assistant translates it into executable code. Complex modelling concepts can therefore be expressed clearly even +when implementing them manually would require substantial effort. By handling the Python syntax and PyAutoLens API, +the assistant allows scientists to focus on what the analysis should do and why, rather than facing the implementation burden of +how to implement it in code. PyAutoLens-Assistant includes several benchmarks that illustrate this further. One uses a +three-paragraph prompt to reproduce the well-known detection of a dark matter subhalo in the strong lens SDSS J0946+1006; +another simulates CCD imaging and interferometric observations of a group-scale strong lens and then models both datasets jointly. + +Through Teacher Mode, PyAutoLens-Assistant supports scientists new to gravitational lensing by explaining concepts in +context and directing them to relevant documentation, tutorials, and open-source lectures. It guides undergraduates and +early-stage PhD students through the foundations of scientific data analysis and Bayesian inference, while helping +experienced lensing researchers new to PyAutoLens navigate its API, syntax, and workflows. This support is increasingly +valuable as datasets grow larger and more diverse, spanning multiple observing techniques, galaxy-, group-, and +cluster-scale systems, and both strong and weak lensing. By reducing the need to engage directly with software syntax, +PyAutoLens-Assistant allows users to learn by performing real analyses and focusing fully on the core scientific concepts +they need to learn. + +# How It Works + +PyAutoLens-Assistant leverages the general knowledge and reasoning capabilities of the underlying foundation model, +available whenever a user starts a new conversation with a coding agent or web-based assistant. However, it adds +two layers on top of this, both of which comprise a library of markdown files structured and formatted in a way +that can be easily and quickly read by an AI assistant or agent, providing them with the full context of PyAutoLens +before the user inputs a prompt. + +The first are the wiki pages, which provide the underlying + +skills, autolens worksapce. + -New users face a complementary challenge: they may not yet know which modelling approach, software abstractions, or examples are appropriate for the task they are learning. PyAutoLens has grown from galaxy-scale imaging analyses to support point-source lenses, group- and cluster-scale systems, weak lensing, interferometry, simulations, and joint analyses, accompanied by well over one hundred worked examples across the `autolens_workspace`. Navigating this material while simultaneously learning gravitational-lensing science, Bayesian inference, and the PyAutoLens API can be overwhelming. PyAutoLens-Assistant enables users to describe their immediate goal in natural language and receive targeted explanations, example code, and pointers to the relevant documentation. Its teaching mode also explains the underlying science and numerical methods, encourages follow-up questions, and supports learning rather than simply returning code. +PyAutoLens-Assistant can be used through a browser-based conversational assistant or a local agentic coding tool. +For systems such as ChatGPT or Claude, `llms.txt` acts as the machine-readable entry point: it asks the assistant to +verify repository access and directs it through the canonical read order of instructions, skills, relevant wiki pages, and +runnable workspace examples. In this mode, users can ask questions, receive scientific explanations, locate examples, +interpret errors and figures, and generate draft end-to-end scripts, although the assistant cannot normally inspect local files or execute code. + +The harness. # State of the field @@ -43,37 +105,25 @@ tested example corpus and the installed API, and inspectable, reproducible outputs. Verify every comparison before submission. --> -# Software design -PyAutoLens-Assistant is a version-controlled knowledge and workflow layer that enables general-purpose AI systems to use PyAutoLens reliably. Its architecture separates three components: instructions define assistant behaviour, skills describe how to perform specific tasks, and wiki pages provide the underlying technical and scientific knowledge. For a given request, the assistant selects the relevant skill, consults the associated wiki material, and adapts tested examples from the `autolens_workspace` rather than generating PyAutoLens code from memory. Generated scripts follow the established workspace structure and can be checked against the installed API, reducing the risk of outdated or invented syntax. ## Reference wikis -Two reference wikis provide complementary context. The core wiki organizes the PyAutoLens API, modelling concepts, datasets, inference methods, and operational guidance, linking these to procedural skills and relevant workspace examples. The literature wiki provides scientific context through pages on lensing concepts, named surveys and systems, and bibliographies of published papers. Users can also ingest papers relevant to a project, after which they become part of the assistant's persistent scientific context. - -## Access modes +Two reference wikis provide complementary context. The core wiki organizes the PyAutoLens API, modelling concepts, datasets, +inference methods, and operational guidance, linking these to procedural skills and relevant workspace examples. The +literature wiki provides scientific context through pages on lensing concepts, named surveys and systems, and bibliographies of published papers. Users can also ingest papers relevant to a project, after which they become part of the assistant's persistent scientific context. -PyAutoLens-Assistant can be used through a browser-based conversational assistant or a local agentic coding tool. For systems such as ChatGPT or Claude, `llms.txt` acts as the machine-readable entry point: it asks the assistant to verify repository access and directs it through the canonical read order of instructions, skills, relevant wiki pages, and runnable workspace examples. In this mode, users can ask questions, receive scientific explanations, locate examples, interpret errors and figures, and generate draft end-to-end scripts, although the assistant cannot normally inspect local files or execute code. -For full computational workflows, PyAutoLens-Assistant can instead be used with agentic tools such as Claude Code or Codex. These tools load the repository instructions directly and can inspect datasets, write and run scripts, generate diagnostic plots, debug failures, and iteratively refine an analysis. The resulting Python code, configuration, outputs, and modelling decisions remain explicit and inspectable. ## Interaction modes and project structure -The assistant operates in two interaction modes. **Assistant mode** is intended for users who want a task completed efficiently, with concise explanations and support ranging from interactive coding to phased end-to-end analysis. **Teacher mode** prioritizes learning by explaining what each stage does and why, making assumptions explicit, and directing users to relevant documentation and examples. Both modes use the same scientific capabilities, reproducibility requirements, and safety checks. - -For agentic work, each analysis can be stored in a separate project repository containing its data, configuration, scripts, results, and project journal. This separates the shared assistant knowledge base from the scientific project while preserving a complete record that can be shared with collaborators or released alongside a publication. - -# Benchmark examples - -PyAutoLens-Assistant is evaluated using a suite of frozen benchmark prompts distributed with the repository. We describe three representative examples here, which span progressively more demanding scientific workflows and are run using multiple conversational and agentic AI systems. Each benchmark records the full interaction, generated code, executed analysis where applicable, scientific outputs, and a rubric-based score, enabling direct comparison between different models, tools, and interaction modes. - -The first benchmark uses **Teacher mode** to simulate Euclid-like imaging of a simple strong lens, fit the simulated data, and recover the lens model. The assistant must explain the purpose of each stage, including model composition, simulation, masking, non-linear inference, and interpretation of the recovered parameters. This benchmark tests whether the assistant can provide scientifically accurate guidance while helping a new user understand an end-to-end PyAutoLens workflow. - -The second benchmark uses **Assistant mode** to model JWST imaging of the COSMOS-Web Ring [@Casey2023]. The assistant must inspect the supplied dataset, perform the required data-preparation steps, construct an appropriate lens-light and mass model with a pixelized source reconstruction, run the analysis, and present the reconstructed source and fit residuals. This benchmark tests the assistant's ability to convert a concise scientific request into a complete and reproducible modelling workflow with limited user intervention. - -The third benchmark requests a more autonomous analysis of the strong lens SDSSJ0946+1006. The assistant must reproduce a reported dark-matter subhalo detection [@Vegetti2010] through Bayesian model comparison, compare alternative subhalo mass profiles to test the reported high concentration of the perturber [@Minor2021], preserve all intermediate models and results for inspection, and determine whether the analysis should run locally or on high-performance computing resources. This benchmark tests long-horizon planning, scientific decision-making, project-state management, and the ability to execute a complex analysis across multiple stages. +The assistant operates in two interaction modes. **Assistant mode** is intended for users who want a task completed +efficiently, with concise explanations and support ranging from interactive coding to phased end-to-end analysis. +**Teacher mode** prioritizes learning by explaining what each stage does and why, making assumptions explicit, and +directing users to relevant documentation and examples. Both modes use the same scientific capabilities, reproducibility requirements, and safety checks. -The benchmark suite is run across different AI systems and access modes, including browser-based conversational assistants and local agentic coding tools. Results will be reported using metrics such as task completion, scientific correctness, API validity, reproducibility, degree of autonomy, number of user interventions, wall-clock time, and computational cost. Together, the benchmarks test the two principal use cases of PyAutoLens-Assistant: teaching new users how to perform gravitational-lens analyses and enabling experienced users to execute complex workflows efficiently from natural-language specifications. +For agentic work, each analysis can be stored in a separate project repository containing its data, configuration, +scripts, results, and project journal. This separates the shared assistant knowledge base from the scientific project while preserving a complete record that can be shared with collaborators or released alongside a publication. # Research impact statement