Factory experiment 53. Implements issue #173.
What to Build
A 5-phase implementation that closes all 4 data gaps in the ACE self-improvement pipeline:
Phase 1 — Core Data Infrastructure
- Add 5 new Pydantic models to
factory/models.py: AgentVerdict, Observation, PerformanceReport, ProjectEntry, ProjectRegistry
- New
factory/registry.py: atomic load/save of ~/.factory/registry.json, idempotent register_project(), update_project_stats(), get_project_paths() replacing discover_projects() with fallback
- New
factory/report.py: parse_ceo_verdicts(), build_performance_report(), observation append, file I/O
Phase 2 — Store & CLI Integration
store.py: begin() calls register_project(), finalize() calls update_project_stats() (both try/except non-blocking)
cli.py: new report-update and registry-list commands
- Update
cmd_ace(), cmd_insights(), cmd_study(): make --projects-dir optional, prefer registry
Phase 3 — Reflector Enhancement
reflector.py: _load_from_reports() with TSV fallback, _verdict_bullets(), _observation_bullets(), use get_project_paths()
Phase 4 — Prompt Updates (Vault Removal)
archivist.md: remove ALL Obsidian references, .factory/archive/ only, add factory report-update responsibility
ceo.md: remove --projects-dir, update archivist task strings, meta mode vault removal
researcher.md: replace vault references with .factory/archive/ paths
Phase 5 — Tests & Migration
tests/test_registry.py, tests/test_report.py, tests/test_reflector_enhanced.py
- Update existing tests that mock
discover_projects
- Migration: auto-populate registry from directory scan on first use
Acceptance Criteria
Constraints
- Read CLAUDE.md before starting
- Do NOT touch files outside declared scope
- Do NOT modify eval/score.py or .factory/ data files
- All registry/report calls in store.py must be try/except non-blocking
- All new models use ConfigDict(strict=True, extra="forbid")
- Maintain backward compatibility — TSV fallback, discover_projects() fallback
Factory experiment 53. Implements issue #173.
What to Build
A 5-phase implementation that closes all 4 data gaps in the ACE self-improvement pipeline:
Phase 1 — Core Data Infrastructure
factory/models.py:AgentVerdict,Observation,PerformanceReport,ProjectEntry,ProjectRegistryfactory/registry.py: atomic load/save of~/.factory/registry.json, idempotentregister_project(),update_project_stats(),get_project_paths()replacingdiscover_projects()with fallbackfactory/report.py:parse_ceo_verdicts(),build_performance_report(), observation append, file I/OPhase 2 — Store & CLI Integration
store.py:begin()callsregister_project(),finalize()callsupdate_project_stats()(both try/except non-blocking)cli.py: newreport-updateandregistry-listcommandscmd_ace(),cmd_insights(),cmd_study(): make--projects-diroptional, prefer registryPhase 3 — Reflector Enhancement
reflector.py:_load_from_reports()with TSV fallback,_verdict_bullets(),_observation_bullets(), useget_project_paths()Phase 4 — Prompt Updates (Vault Removal)
archivist.md: remove ALL Obsidian references,.factory/archive/only, addfactory report-updateresponsibilityceo.md: remove--projects-dir, update archivist task strings, meta mode vault removalresearcher.md: replace vault references with.factory/archive/pathsPhase 5 — Tests & Migration
tests/test_registry.py,tests/test_report.py,tests/test_reflector_enhanced.pydiscover_projectsAcceptance Criteria
factory report-update /pathgenerates.factory/performance_report.jsonwith experiments, verdicts, observationsfactory registry-listshows registered projectsfactory ace /pathworks without--projects-dir(uses registry)factory ace /pathreads performance reports for qualitative signalsfactory report-updateinstead of vaultstore.begin()andstore.finalize()auto-update the registryruff check .clean,mypy factory/cleanConstraints