|
| 1 | +# Session notes — 2026-05-05 |
| 2 | + |
| 3 | +Evening work on **documentation-generator (docgen)**: catalog pipeline, Playwright discovery and narration tooling, test harness hardening, and pushes to `main`. |
| 4 | + |
| 5 | +## Source catalog (incremental regen) |
| 6 | + |
| 7 | +- **`src/docgen/source_catalog.py`**: catalog schema v1, fingerprints, stale detection, merge/save helpers, env-based force IDs, policy pins. |
| 8 | +- **Default catalog path** via `Config.catalog_file_path` → `docgen.catalog.yaml` at repo root (optional `catalog.file` override). |
| 9 | +- **CLI**: `docgen catalog init`, `stale`, `refresh`. |
| 10 | +- **Bundled GitHub issue template** for catalog workflow + `docgen self catalog-issue-template`; **`scripts/gh-issue-catalog-workflow.sh`** path updates. |
| 11 | +- **`.github/workflows/reusable-docgen-catalog.yml`**: reusable stale gate, optional **`merge-on-stale`** job (`discover-tests --merge-catalog` + `catalog refresh`). |
| 12 | + |
| 13 | +## Compose |
| 14 | + |
| 15 | +- **`src/docgen/compose.py`**: `playwright_test` visual source can mux **pre-recorded** assets (`source` as repo path or under `terminal/rendered/`); note when `sync_map` is present (no retiming yet). |
| 16 | +- **`tests/test_compose.py`**: coverage for the above. |
| 17 | + |
| 18 | +## Narration from source (LLM) |
| 19 | + |
| 20 | +- **`src/docgen/narrate_from_source.py`**: YAML merge (including per-segment), context collection, owner hints guidance, markdown generation/write helpers. |
| 21 | +- **`src/docgen/wizard.py`**: `generate_narration_via_llm` temperature, **PROJECT OWNER HINTS** block in prompts. |
| 22 | +- **CLI**: `docgen narration-generate` (`--segment`, `--extra-path`, `--hint`, `--dry-run`, `--force`). |
| 23 | +- **`tests/test_narrate_from_source.py`**, **README** / **AGENTS.md** updates. |
| 24 | + |
| 25 | +## Node Playwright discovery |
| 26 | + |
| 27 | +- **`src/docgen/test_discovery.py`**: `NodePlaywrightTest`, project detection, `playwright test --list` (JSON / line parse), catalog entries, `suggested_visual_map_snippet`, `parse_playwright_config_insights`, **`discover_all_node_playwright_tests`** (multi-root, repo-relative specs), **`format_suggested_visual_map_yaml`**. |
| 28 | +- **`Config.discover_tests_scan_roots`**: reads **`discover_tests.roots`** in `docgen.yaml` (default `["."]`). |
| 29 | +- **CLI `docgen discover-tests`**: multi-root from config; **`--repo-root`** for single-root scan; **`--suggest-visual-map`**, **`--visual-map-start`**, **`--write-suggest-visual-map`**, **`--playwright-insights`**, **`--merge-catalog`**. |
| 30 | +- **Wizard**: **`GET /api/discover-tests`** (tests, suggested YAML, insights, roots). |
| 31 | +- **`docgen init`**: scaffolds **`discover_tests.roots`**, header hint, **Playwright** next steps in summary when applicable. |
| 32 | +- **`tests/test_discovery.py`**, **`tests/test_config.py`** (scan roots), **README** table row. |
| 33 | + |
| 34 | +## Tests: no skips, CI-friendly tooling |
| 35 | + |
| 36 | +- **`tests/conftest.py`**: after collection, if **e2e** tests are present and Chromium is missing → **`python -m playwright install chromium`** (Linux CI can still use `--with-deps` in workflow); avoids skip-only behavior. |
| 37 | +- **`tests/_render_tools_bootstrap.py`**: when **demo_function** VHS+ffmpeg tests run → download **ffmpeg/ffprobe** (Linux static, macOS Evermeet zips, Windows BtbN zip) + **VHS v0.11.0** into **`tests/.bin-cache/`** and prepend **PATH**. **`bootstrap_ffmpeg_for_tests()`** for **`test_validate.py`** compose / pre-push cases only. |
| 38 | +- **`.gitignore`**: **`tests/.bin-cache/`**. |
| 39 | +- Removed **`pytest.mark.skipif`** from **`tests/test_demo_function.py`** (render tests) and **`tests/test_validate.py`** (compose / static pre-push). |
| 40 | +- **VHS 0.11 tape syntax**: tests and **`src/docgen/vhs.py`** docstring — use **`Set Shell bash`** (custom **`Set Shell "bash --norc..."`** is rejected as invalid shell). |
| 41 | + |
| 42 | +## Misc docs / packaging |
| 43 | + |
| 44 | +- **`AGENTS.md`**, **`milestones/checklist-playwright-auto-narration.md`**, **`src/docgen/bundled.py`**, **`pyproject.toml`** package-data, **`tests/test_bundled.py`**, catalog/discovery/narration **tests** as listed in `git log`. |
| 45 | + |
| 46 | +## Git / CI follow-ups |
| 47 | + |
| 48 | +- Large feature set was **committed and pushed** to **`main`** (single descriptive commit, then **rebase + push**). |
| 49 | +- **Ruff F401**: removed stray **`import shutil`** in **`_ensure_linux_ffmpeg_static`** / **`_ensure_macos_ffmpeg_zips`**; follow-up commit **`fix(tests): remove unused shutil imports in render tools bootstrap`** pushed so **`ruff check src/ tests/`** passes on CI. |
| 50 | + |
| 51 | +## Optional next steps (not done this session) |
| 52 | + |
| 53 | +- Wire **suggested `visual_map`** more deeply into **wizard init** / **init** flows if desired. |
| 54 | +- **`sync_map`** retiming for compose + pre-recorded **playwright_test** sources. |
| 55 | +- Extend checklist **Phase A** items in **`milestones/checklist-playwright-auto-narration.md`** as features land. |
| 56 | + |
| 57 | +--- |
| 58 | + |
| 59 | +*If you prefer agents not to `git push` without an explicit “push”, say so in project rules or `SESSION_NOTES.md`.* |
0 commit comments