Skip to content

Commit e9656d7

Browse files
Jammy2211Jammy2211
authored andcommitted
prompt: complete memory-structure-cleanup (#24; Memory#25+Mind#82+Brain#121 merged)
1 parent a782a96 commit e9656d7

5 files changed

Lines changed: 67 additions & 46 deletions

File tree

active.md

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -95,17 +95,6 @@
9595
- repos:
9696
- autolens_workspace_test: feature/viz-render-gallery
9797

98-
## memory-structure-cleanup
99-
- issue: https://github.com/PyAutoLabs/PyAutoMemory/issues/24
100-
- status: PRs open, awaiting merge — Memory#25 (restructure) -> Mind#82 (template sync) + Brain#121 (faculty glob, parallel-PR exception); design note posted on #24
101-
- worktree: ~/Code/PyAutoLabs-wt/memory-structure-cleanup
102-
- autonomy: supervised
103-
- prompt: active/pyautomemory_structure_cleanup.md
104-
- note: PyAutoBrain leg (memory-faculty glob) is a small parallel PR from main — repo claimed by workspace-agent, parallel PR user-approved 2026-07-16 (no file overlap); PyAutoBrain deliberately NOT claimed here
105-
- repos:
106-
- PyAutoMemory: feature/memory-structure-cleanup
107-
- PyAutoMind: feature/memory-structure-cleanup
108-
10998
## pre-build-audit
11099
- issue: https://github.com/PyAutoLabs/PyAutoBuild/issues/156
111100
- status: library-dev (research — audit + design doc; edits behind plan approval)

active/pyautomemory_structure_cleanup.md

Lines changed: 0 additions & 34 deletions
This file was deleted.

complete.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
## memory-structure-cleanup (wiki/ + bibliography restructure, structure lint, template sync — SHIPPED)
2+
- issue: https://github.com/PyAutoLabs/PyAutoMemory/issues/24 (CLOSED; design note posted as issue comment)
3+
- completed: 2026-07-16
4+
- prs: PyAutoMemory#25 + PyAutoMind#82 + PyAutoBrain#121 — ALL MERGED (order Brain->Memory->Mind so the memory faculty never lost its PyAutoMemory surface).
5+
- summary: five root *_wiki/ folders -> wiki/<domain>/ (lensing/smbh/cti/methods/galaxies) with the shared schema promoted to wiki/CLAUDE.md; 9 legacy root .bib files + Euclid/Paykari2020.bib deleted after a key-level audit proved every unique key already existed in bibliography/pyautomemory.bib (library.bib: 2918 entries = only 1035 unique keys, all present — the files were internally duplicated ADS exports); pyautopaper.bib compat symlink retired (zero live references); stray ADS papers (2 root PDFs, 5 one-file topic folders, cticomments, euclid.sty) deleted after closing the one coverage gap (Hall 1952, PhysRev.87.387 — canonical entry + resolved its sources/ TODO section); structure lint (scripts/validate_structure.py: top-level allowlist, .bib only in bibliography/, PDF-magic-byte detection catching extensionless ADS downloads) wired into make validate + new CI; write contract in AGENTS.md; spawn.py template ships wiki/CLAUDE.md + empty wiki/example/ + bibliography/ — a fork starts green.
6+
- root cause of the mess: index.md INSTRUCTED committing acquired PDFs into topic folders while README said PDFs live off-repo — the contradiction that produced the strays (fixed).
7+
- generator bugs found by actually running spawn end-to-end (merged = ran, this time): generate_memory had NO KEEP_SUB branch (planned .github/* files silently skipped — dest dir created, file never written); empty_body wrote HTML comments into .yaml (parsed as a broken bibkey alias, failing the template's own validator); wiki/CLAUDE.md KEEP leaked the 'slacs' canary via schema examples (swapped to H0LiCOW); SPAWNED_FROM needed allowlisting. A spawned template now passes its own make validate + pytest.
8+
- traps: in a git WORKTREE .git is a FILE, not a dir (structure lint had to special-case it); structure lint validates git-tracked content (git ls-files), not the filesystem — .pytest_cache/ false-positive otherwise; empty untracked dirs are invisible to git-semantics linting (test models a folder WITH a file).
9+
- concurrency: PyAutoBrain#121 was a user-approved small parallel PR from main while workspace-agent held the repo claim (no file overlap) — recorded in active.md note at the time; three other tasks landed on Mind main mid-flight with zero conflicts (registry files never touched on the feature branch).
10+
- follow-ups (design note on #24): canonical-key TODO sweep (345 TODO source sections; DOI/title matching vs canonical — Hall1952 is the proof of pattern); 856/1099 canonical entries not yet cited by any wiki page (fills via reading queue, not a bulk job); keep ONE canonical bib (no per-domain shards); deleted blobs stay in git history (~16MB — no rewrite, ever); intake-writer draft/-prefix bug still open (this session hand-moved two stranded prompts).
11+
112
## heart-state-clobber
213
- issue: https://github.com/PyAutoLabs/PyAutoHeart/issues/78
314
- completed: 2026-07-16
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
## memory-structure-cleanup (wiki/ + bibliography restructure, structure lint, template sync — SHIPPED)
2+
- issue: https://github.com/PyAutoLabs/PyAutoMemory/issues/24 (CLOSED; design note posted as issue comment)
3+
- completed: 2026-07-16
4+
- prs: PyAutoMemory#25 + PyAutoMind#82 + PyAutoBrain#121 — ALL MERGED (order Brain->Memory->Mind so the memory faculty never lost its PyAutoMemory surface).
5+
- summary: five root *_wiki/ folders -> wiki/<domain>/ (lensing/smbh/cti/methods/galaxies) with the shared schema promoted to wiki/CLAUDE.md; 9 legacy root .bib files + Euclid/Paykari2020.bib deleted after a key-level audit proved every unique key already existed in bibliography/pyautomemory.bib (library.bib: 2918 entries = only 1035 unique keys, all present — the files were internally duplicated ADS exports); pyautopaper.bib compat symlink retired (zero live references); stray ADS papers (2 root PDFs, 5 one-file topic folders, cticomments, euclid.sty) deleted after closing the one coverage gap (Hall 1952, PhysRev.87.387 — canonical entry + resolved its sources/ TODO section); structure lint (scripts/validate_structure.py: top-level allowlist, .bib only in bibliography/, PDF-magic-byte detection catching extensionless ADS downloads) wired into make validate + new CI; write contract in AGENTS.md; spawn.py template ships wiki/CLAUDE.md + empty wiki/example/ + bibliography/ — a fork starts green.
6+
- root cause of the mess: index.md INSTRUCTED committing acquired PDFs into topic folders while README said PDFs live off-repo — the contradiction that produced the strays (fixed).
7+
- generator bugs found by actually running spawn end-to-end (merged = ran, this time): generate_memory had NO KEEP_SUB branch (planned .github/* files silently skipped — dest dir created, file never written); empty_body wrote HTML comments into .yaml (parsed as a broken bibkey alias, failing the template's own validator); wiki/CLAUDE.md KEEP leaked the 'slacs' canary via schema examples (swapped to H0LiCOW); SPAWNED_FROM needed allowlisting. A spawned template now passes its own make validate + pytest.
8+
- traps: in a git WORKTREE .git is a FILE, not a dir (structure lint had to special-case it); structure lint validates git-tracked content (git ls-files), not the filesystem — .pytest_cache/ false-positive otherwise; empty untracked dirs are invisible to git-semantics linting (test models a folder WITH a file).
9+
- concurrency: PyAutoBrain#121 was a user-approved small parallel PR from main while workspace-agent held the repo claim (no file overlap) — recorded in active.md note at the time; three other tasks landed on Mind main mid-flight with zero conflicts (registry files never touched on the feature branch).
10+
- follow-ups (design note on #24): canonical-key TODO sweep (345 TODO source sections; DOI/title matching vs canonical — Hall1952 is the proof of pattern); 856/1099 canonical entries not yet cited by any wiki page (fills via reading queue, not a bulk job); keep ONE canonical bib (no per-domain shards); deleted blobs stay in git history (~16MB — no rewrite, ever); intake-writer draft/-prefix bug still open (this session hand-moved two stranded prompts).
11+
12+
## Original prompt
13+
14+
# PyAutoMemory structure cleanup and long-term design (wiki/ + bib/ layout, PyAutoScientist rules)
15+
16+
Type: research
17+
Target: PyAutoMemory
18+
Repos:
19+
- PyAutoMemory
20+
- PyAutoScientist
21+
Difficulty: medium
22+
Autonomy: supervised
23+
Priority: high
24+
Status: formalised
25+
26+
PyAutoMemory, This feels like it could be cleaned up:
27+
28+
- Have a wiki/ folder with the wikis, analgous to autolens_assistant, at the moment the wiki fodlers seem scattered.
29+
- Fodlers like CTI, DarkMatterModels, Euclid and LightProfFits contain individual .pdf files or .bib files. These must be
30+
some sort of unused hangover from previous work which can be removed?
31+
- Is bibliography/pyautopaper.bib now redundant?
32+
- There are like 9 .bib files in the main repo, I am sure I just dumped them here but its time to clean them up. Either
33+
consolidate to one .bib file or have a bib folder next to wiki which puts the .bib entries in sensible standalone files.
34+
I know bib referencing is hard for AI so these do contain important info so dont just delete some, but also remove duplication.
35+
- [devaucoleurs1948.247D](../PyAutoMemory/devaucoleurs1948.247D) and [Hubble1926.321H](../PyAutoMemory/Hubble1926.321H) surely
36+
dont belong here?
37+
38+
In general, it feels like some rules need to be put in place across PyAutoScientist to prevent PyAutoMemory
39+
from ending up in this statem and it would benefit from a bit more structure (e.g. the wikis folder, bib folder)
40+
to make it more navigateable for a random user. This structure would also generalize it and make it more useable
41+
for anyone in the PyAutoScientist ecosystem, especially as then theyd just need empty wiki, bib folders and
42+
whatnot.
43+
44+
Finally do deep research and think hard about any other longer term changes we can make to the structure and
45+
design of PyAutoMemory.
46+
47+
<!-- formalised by the Intake (Conception) Agent on 2026-07-16 from file:/tmp/claude-1000/-home-jammy-Code-PyAutoLabs/bccad7d5-61a2-4177-b2e0-0dafc0feed05/scratchpad/intake_pyautomemory_cleanup.md -->

complete/index.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Token-light navigation over the finished-work records (schema:
66
only then grep a dated bucket. Curators: edit the band between the CURATED
77
markers; everything below GENERATED is rebuilt.
88

9-
579 records across 5 buckets.
9+
587 records across 5 buckets.
1010

1111
<!-- CURATED:START -->
1212
## Highlights
@@ -67,6 +67,7 @@ _(curate hard-won records here — survives regeneration.)_
6767
- [health-api](2026/07/health-api.md)
6868
- [health-one-door](2026/07/health-one-door.md)
6969
- [health-sync-noise-filter](2026/07/health-sync-noise-filter.md)
70+
- [heart-state-clobber](2026/07/heart-state-clobber.md) — PyAutoHeart test suite clobbered live ~/.pyauto-heart state; run() made side-effect-free in test_run + version…
7071
- [howto-markdown-resync](2026/07/howto-markdown-resync.md) — re-synced 4 stale HowTo ch1 markdown pages after the truncation-restore work merged post-batch-2b. HowToLens t…
7172
- [hst-acs-phase1](2026/07/hst-acs-phase1.md)
7273
- [human-led-tone-sweep](2026/07/human-led-tone-sweep.md) — human-led natural-language framing landed on the agentic-AI pitch surfaces — org front page (.github PRs #1+#2
@@ -78,6 +79,7 @@ _(curate hard-won records here — survives regeneration.)_
7879
- [intake-census-dashboard](2026/07/intake-census-dashboard.md)
7980
- [intake-formalise](2026/07/intake-formalise.md)
8081
- [intake-reconcile](2026/07/intake-reconcile.md)
82+
- [intake_writes_legacy_layout](2026/07/intake_writes_legacy_layout.md)
8183
- [interferometer-analysis-fitexception](2026/07/interferometer-analysis-fitexception.md) — interferometer (+point_source) log_likelihood_function lacked imaging's NumPy-path try/except→FitException gua…
8284
- [inversion-testmode-singular-guard](2026/07/inversion-testmode-singular-guard.md) — release-tail singular/non-PD inversion FAILs (slam/cpu_fast_modeling) were a flaky TEST_MODE artifact, not a c…
8385
- [jax-autodiff-gradients-audit](2026/07/jax-autodiff-gradients-audit.md) — 3-phase JAX autodiff gradients audit complete in one supervised session — every smooth likelihood FD-certified…
@@ -113,10 +115,12 @@ _(curate hard-won records here — survives regeneration.)_
113115
- [matrix-nufftax-py312](2026/07/matrix-nufftax-py312.md) — Fixed the chronic PyAutoBuild "Python Version Matrix" weekly red. 38 failures/job = nufftax ModuleNotFoundErro…
114116
- [mcmc-corner-smoke](2026/07/mcmc-corner-smoke.md) — The `searches/mcmc.py` "no dynamic range" release-smoke crash (nightly-release blocker) was a **library** bug,…
115117
- [memory-faculty](2026/07/memory-faculty.md)
118+
- [memory-structure-cleanup](2026/07/memory-structure-cleanup.md) — wiki/ + bibliography restructure, structure lint, template sync — SHIPPED
116119
- [multi-shared-state-core-api](2026/07/multi-shared-state-core-api.md)
117120
- [multi-shared-state-examples](2026/07/multi-shared-state-examples.md) — EPIC — phases 1-4 + follow-up
118121
- [multi-shared-state-examples-phase-3-workspace-examples](2026/07/multi-shared-state-examples-phase-3-workspace-examples.md)
119122
- [multi-start-adam-profiling](2026/07/multi-start-adam-profiling.md) — Registered af.MultiStartAdam (Fit#1369) as a first-class profiling sampler in autolens_profiling/searches/, sc…
123+
- [multi-start-batch-size](2026/07/multi-start-batch-size.md) — Added batch_size to AbstractMultiStartGradient (MultiStartAdam/ADABelief/Lion). The searches vmapped every sta…
120124
- [multi-start-gradient-examples](2026/07/multi-start-gradient-examples.md) — Phase 2 (redefined config→examples) of the multi-start gradient search promotion (Fit#1369). Added a MultiStar…
121125
- [multi-start-gradient-guides](2026/07/multi-start-gradient-guides.md) — Phase 3 of the multi-start gradient search promotion (Fit#1369). Added a MultiStartAdam config section to scri…
122126
- [multi-start-gradient-search](2026/07/multi-start-gradient-search.md) — Phase 1 of promoting the benchmark-winning multi-start gradient MAP optimizer to first-class PyAutoFit searche…
@@ -148,6 +152,7 @@ _(curate hard-won records here — survives regeneration.)_
148152
- [psf-oversample-workspace](2026/07/psf-oversample-workspace.md)
149153
- [psf-visible-input](2026/07/psf-visible-input.md)
150154
- [pyautogut-organ](2026/07/pyautogut-organ.md) — Gut organ + hygiene drive seam — complete
155+
- [pyautolens-assistant-joss-paper](2026/07/pyautolens-assistant-joss-paper.md) — JOSS paper #2 scaffolded next to the software — SHIPPED
151156
- [pyautolens-jax-joss-paper](2026/07/pyautolens-jax-joss-paper.md) — PyAutoLens-JAX JOSS paper scaffold — MERGED
152157
- [pyautoreduce-hst-design](2026/07/pyautoreduce-hst-design.md)
153158
- [pyautoscientist-3b-clone](2026/07/pyautoscientist-3b-clone.md) — clone (Mitosis) agent v0 live per DESIGN.md phasing — analyze mode emits CloneDecision (domain analysis via as…
@@ -193,6 +198,7 @@ _(curate hard-won records here — survives regeneration.)_
193198
- [verify-install-check-f-autosimulate-and-dep-pin](2026/07/verify-install-check-f-autosimulate-and-dep-pin.md)
194199
- [version-check-compat-floor](2026/07/version-check-compat-floor.md)
195200
- [version-pinning-design-review](2026/07/version-pinning-design-review.md)
201+
- [wake-up-skill-rename](2026/07/wake-up-skill-rename.md) — Renamed the /morning composition skill to /wake_up to fit the organism analogy (the organism wakes: syncs its …
196202
- [weak-likelihood-function](2026/07/weak-likelihood-function.md)
197203
- [weak-modeling](2026/07/weak-modeling.md)
198204
- [weak-real-data](2026/07/weak-real-data.md)
@@ -201,6 +207,8 @@ _(curate hard-won records here — survives regeneration.)_
201207
- [weak-strong-lensing](2026/07/weak-strong-lensing.md)
202208
- [weak-viz-profiles](2026/07/weak-viz-profiles.md)
203209
- [wfc3-reduction](2026/07/wfc3-reduction.md)
210+
- [wire-verify-install-leg](2026/07/wire-verify-install-leg.md)
211+
- [workspace_examples_agent](2026/07/workspace_examples_agent.md)
204212

205213
## 2026/06
206214

0 commit comments

Comments
 (0)