Dogfood demos: playwright_test segment 07 + pipeline visual_map filtering

.gitignore

@@ -32,7 +32,8 @@ htmlcov/
 # Generated demo assets (reproducible via docgen generate-all)
 docs/demos/audio/*.mp3
 docs/demos/animations/media/
-docs/demos/terminal/rendered/
+docs/demos/terminal/rendered/*
+!docs/demos/terminal/rendered/07-playwright-dogfood.webm
 docs/demos/.docgen-state.json
 
 # recordings/*.mp4 — COMMITTED via Git LFS for GitHub Pages

docgen.catalog.yaml

@@ -0,0 +1,5 @@
+catalog_schema_version: 1
+updated_at: '2026-05-06T19:38:58Z'
+docgen_version: 0.2.0
+repo_root: .
+entries: []

docs/demos/README.md

@@ -0,0 +1,60 @@
+# docgen demo videos (dogfood)
+
+This tree is the **in-repo dogfood** project: the documentation-generator repository eating its own cooking. Configuration lives in `docgen.yaml` (paths relative to this directory unless noted).
+
+## Prerequisites
+
+Full `docgen generate-all` needs **Manim**, **ffmpeg**, **VHS**, **ttyd**, **Xvfb** (or a display) for terminal tapes, **Playwright** only if you re-record browser footage, and **`OPENAI_API_KEY`** for Whisper timestamps, TTS, and optional `narration-generate`.
+
+Segment **07** uses a **checked-in WebM** under `terminal/rendered/`; no Playwright run is required for a basic pipeline pass if you keep that file.
+
+## Commands (typical)
+
+From repository root:
+
+```bash
+cd docs/demos
+docgen --config docgen.yaml validate --help
+```
+
+Source-catalog metadata (repo root; shared with other workflows):
+
+```bash
+cd docs/demos
+docgen --config docgen.yaml catalog init         # once; creates ../../docgen.catalog.yaml
+docgen --config docgen.yaml catalog stale
+```
+
+Discovery and catalog merge (Node Playwright projects under the repo root):
+
+```bash
+cd docs/demos
+docgen --config docgen.yaml discover-tests --merge-catalog
+docgen --config docgen.yaml catalog refresh
+```
+
+Optional narration draft for segment 07 (requires `OPENAI_API_KEY`):
+
+```bash
+cd docs/demos
+docgen --config docgen.yaml narration-generate --segment 07 --dry-run
+docgen --config docgen.yaml narration-generate --segment 07
+```
+
+Full pipeline (heavy):
+
+```bash
+cd docs/demos
+docgen --config docgen.yaml generate-all
+# or iterate with skips, e.g. --skip-tts after audio exists
+```
+
+Validation:
+
+```bash
+cd docs/demos
+docgen --config docgen.yaml validate
+docgen --config docgen.yaml validate --pre-push
+```
+
+Upstream consumer dogfood (separate clone) is described in `milestones/upstream-dogfood.md` at the repo root.

docs/demos/docgen.yaml

@@ -1,5 +1,6 @@
 # docgen.yaml — dogfood: docgen's own demo videos
 # See: https://github.com/jmjava/documentation-generator
+# Command cheat sheet: README.md in this directory.
 #
 # This project uses docgen to produce demo videos about docgen itself.
 #   docgen generate-all       # full pipeline
@@ -9,6 +10,30 @@
 repo_root: ../..
 env_file: ../../.env
 
+discover_tests:
+  roots:
+    - "."
+
+narration_from_source:
+  model: gpt-4o-mini
+  hints:
+    - >
+      Project context: docgen is the documentation-generator CLI/library (see AGENTS.md).
+      Audience: engineers adopting docgen or reviewing its pipeline.
+  context:
+    paths:
+      - README.md
+      - AGENTS.md
+      - src/docgen/pipeline.py
+      - src/docgen/compose.py
+      - src/docgen/config.py
+  segments:
+    "07":
+      hints:
+        - >
+          This segment demonstrates visual_map type playwright_test with a short pre-recorded
+          WebM under terminal/rendered/. Keep the tone consistent with other demo segments.
+
 dirs:
   narration: narration
   audio: audio
@@ -24,6 +49,7 @@ segments:
     - "04"
     - "05"
     - "06"
+    - "07"
   all: *all_segments
 
 segment_names:
@@ -33,10 +59,12 @@ segment_names:
   "04": 04-tts-pipeline
   "05": 05-compose-validate
   "06": 06-ci-integration
+  "07": 07-playwright-dogfood
 
 # Visual source mapping
 # 01, 03: Manim animations (architecture overview, wizard GUI walkthrough)
 # 02, 04, 05, 06: VHS terminal recordings (live docgen commands)
+# 07: playwright_test — pre-recorded WebM (dogfood; pipeline skips Manim/VHS capture for this id)
 visual_map:
   "01":
     type: manim
@@ -62,6 +90,10 @@ visual_map:
     type: vhs
     tape: 06-ci-integration.tape
     source: 06-ci-integration.mp4
+  "07":
+    type: playwright_test
+    test: dogfood/playwright_mux_placeholder.spec.ts
+    source: 07-playwright-dogfood.webm
 
 manim:
   quality: 720p30
@@ -129,6 +161,9 @@ pages:
     "06":
       title: "CI Integration"
       description: "Pre-push hooks and GitHub Actions for continuous validation."
+    "07":
+      title: "Playwright visual mux (dogfood)"
+      description: "Pre-recorded browser-style footage composed like Manim/VHS segments."
 
 wizard:
   llm_model: gpt-4o

docs/demos/narration/07-playwright-dogfood.md

@@ -0,0 +1,3 @@
+This segment shows how docgen treats browser-test footage like any other visual source. You record or export a short Playwright video, point the manifest at that file, and compose muxes it with the narration audio from this repository.
+
+The pipeline skips Manim and VHS capture for this segment because the WebM is already on disk. That keeps long-running UI tests out of the default generate-all path while still proving the playwright_test visual type end to end.

milestones/checklist-playwright-auto-narration.md

@@ -47,8 +47,8 @@ Use this as a working checklist; reorder within a phase as dependencies land.
 
 - [ ] Implement runner: invoke tests with video + trace per segment (or consume CI-produced artifacts via paths in `docgen.yaml`)
 - [x] `compose.py`: handle `vtype == "playwright_test"` — mux pre-recorded `source` with segment audio (`repo_root` path first, then `terminal/rendered/`). **Retiming from `sync_map` not implemented yet** (prints NOTE when sync_map present).
-- [ ] `pipeline.py`: run Playwright-test stages when `visual_map` contains `playwright_test`
-- [ ] Dogfood: one `visual_map` entry in this repo or `examples/` when stable
+- [x] `pipeline.py`: Manim/VHS capture stages only render scenes/tapes referenced by `visual_map` for active `segments.all`; `playwright_test` segments rely on pre-recorded video (no Manim/VHS pass).
+- [x] Dogfood: `docs/demos` includes segment **07** (`playwright_test`) with checked-in WebM under `terminal/rendered/`.
 - [ ] Re-run / extend existing validator tests (`tests/test_validate_playwright.py`) against real-shaped artifacts
 
 ---

milestones/next-session-dogfood.md

@@ -11,23 +11,23 @@ Work in order; stop and checkpoint when a step is big enough to ship alone.
 ## 0 — Preconditions (one-time check)
 
 - [ ] Machine (or CI self-hosted runner) has: **Manim**, **ffmpeg**, **ttyd**, **xvfb** (or real display), **Playwright** if you capture fresh video, **`OPENAI_API_KEY`** for TTS / narration-generate.
-- [ ] From repo root: `cd docs/demos` and `docgen --config docgen.yaml validate --help` loads.
+- [x] From repo root: `cd docs/demos` and `docgen --config docgen.yaml validate --help` loads.
 
 ---
 
 ## 1 — Wire dogfood `docgen.yaml` to new features
 
-- [ ] Add **`discover_tests`** (and **`roots`**, if not only `.`) under `docs/demos/docgen.yaml` so `docgen discover-tests` matches this monorepo.
-- [ ] Add **`narration_from_source`** (context paths, hints, model) pointing at real files (`README.md`, `AGENTS.md`, small `src/` snippets) so `docgen narration-generate --segment …` is meaningful.
-- [ ] Run **`docgen catalog init`** at **repo root** (or set `catalog.file` in demos yaml); confirm **`docgen catalog stale`** behavior on a clean tree.
+- [x] Add **`discover_tests`** (and **`roots`**, if not only `.`) under `docs/demos/docgen.yaml` so `docgen discover-tests` matches this monorepo.
+- [x] Add **`narration_from_source`** (context paths, hints, model) pointing at real files (`README.md`, `AGENTS.md`, small `src/` snippets) so `docgen narration-generate --segment …` is meaningful.
+- [x] Run **`docgen catalog init`** at **repo root** (or set `catalog.file` in demos yaml); confirm **`docgen catalog stale`** behavior on a clean tree.
 
 ---
 
 ## 2 — One `playwright_test` segment in demos (minimal slice)
 
-- [ ] Produce a **short pre-recorded** `.webm` (or `.mp4`) from a trivial Playwright test or reuse an existing artifact; place it under **`docs/demos/terminal/rendered/`** (or repo-relative path compose already resolves).
-- [ ] Add **one new segment** (e.g. `07`) in `segments` / `segment_names` / `concat` and a **`visual_map`** entry `type: playwright_test` with `test` id + `source` path; keep existing 01–06 on Manim/VHS so the rest of the demo still works.
-- [ ] **`pipeline.py`**: today only Manim + VHS render stages run before compose — add handling so **`playwright_test`** segments **skip** capture stages (or **sync** pre-recorded file into expected layout) so `generate-all` does not assume Manim/VHS for that segment. *(This is the main engineering chunk; align with checklist Phase C.)*
+- [x] Produce a **short pre-recorded** `.webm` (or `.mp4`) from a trivial Playwright test or reuse an existing artifact; place it under **`docs/demos/terminal/rendered/`** (or repo-relative path compose already resolves).
+- [x] Add **one new segment** (e.g. `07`) in `segments` / `segment_names` / `concat` and a **`visual_map`** entry `type: playwright_test` with `test` id + `source` path; keep existing 01–06 on Manim/VHS so the rest of the demo still works.
+- [x] **`pipeline.py`**: today only Manim + VHS render stages run before compose — add handling so **`playwright_test`** segments **skip** capture stages (or **sync** pre-recorded file into expected layout) so `generate-all` does not assume Manim/VHS for that segment. *(This is the main engineering chunk; align with checklist Phase C.)*
 
 ---
 
@@ -48,8 +48,8 @@ Work in order; stop and checkpoint when a step is big enough to ship alone.
 
 ## 5 — Close the loop in docs
 
-- [ ] Update **`docs/demos`** README or top-of **`docgen.yaml`** comments with the **exact command sequence** you used for dogfood.
-- [ ] Check off or adjust **[checklist-playwright-auto-narration.md](checklist-playwright-auto-narration.md)** Phase C “Dogfood” line once one `playwright_test` segment is real in-tree.
+- [x] Update **`docs/demos`** README or top-of **`docgen.yaml`** comments with the **exact command sequence** you used for dogfood.
+- [x] Check off or adjust **[checklist-playwright-auto-narration.md](checklist-playwright-auto-narration.md)** Phase C “Dogfood” line once one `playwright_test` segment is real in-tree.
 
 ---
 

milestones/upstream-dogfood.md

@@ -2,7 +2,7 @@
 
 **Upstream** here means a **separate repository** (application, course, or sample repo) that installs **docgen as a dependency** (`pip install docgen` or `pip install "docgen @ git+https://…"`), not the in-tree **`docs/demos`** path. Goal: prove the **library + CLI + CI contracts** work for a real consumer and capture friction for fixes here.
 
-Pair with **[in-repo dogfood](next-session-dogfood.md)** (this repo’s `docs/demos` full run).
+Pair with **[in-repo dogfood](next-session-dogfood.md)** (this repo’s `docs/demos` full run). The maintainer-facing command sequence for that tree lives in **`docs/demos/README.md`**.
 
 **Designated consumer repo:** **`courseforge/course-builder`** — local clone: **`/home/ubuntu/github/courseforge/course-builder`**. All upstream dogfood work lands there; friction feeds back here.
 

src/docgen/config.py

@@ -57,6 +57,40 @@ def resolve_segment_name(self, seg_id: str) -> str:
     def visual_map(self) -> dict[str, Any]:
         return self.raw.get("visual_map", {})
 
+    def pipeline_manim_scene_names(self) -> list[str]:
+        """Scene class names for ``segments.all`` entries whose ``visual_map`` type is ``manim``."""
+        seen: set[str] = set()
+        ordered: list[str] = []
+        for seg_id in self.segments_all:
+            vm = self.visual_map.get(seg_id)
+            if not isinstance(vm, dict):
+                continue
+            if str(vm.get("type", "")).lower() != "manim":
+                continue
+            scene = str(vm.get("scene", "")).strip()
+            if scene and scene not in seen:
+                seen.add(scene)
+                ordered.append(scene)
+        return ordered
+
+    def pipeline_vhs_tape_filenames(self) -> list[str]:
+        """Tape filenames for ``segments.all`` entries whose ``visual_map`` type is ``vhs``."""
+        ordered: list[str] = []
+        for seg_id in self.segments_all:
+            vm = self.visual_map.get(seg_id)
+            if not isinstance(vm, dict):
+                continue
+            if str(vm.get("type", "")).lower() != "vhs":
+                continue
+            tape = str(vm.get("tape", "")).strip()
+            if not tape:
+                src = str(vm.get("source", "")).strip()
+                if src:
+                    tape = f"{Path(src).stem}.tape"
+            if tape:
+                ordered.append(tape)
+        return ordered
+
     @property
     def concat_map(self) -> dict[str, list[str]]:
         return self.raw.get("concat", {})

src/docgen/manim_runner.py

@@ -17,9 +17,25 @@ class ManimRunner:
     def __init__(self, config: Config) -> None:
         self.config = config
 
-    def render(self, scene: str | None = None) -> None:
-        scenes = [scene] if scene else self.config.manim_scenes
-        if not scenes:
+    def render(
+        self,
+        scenes: list[str] | None = None,
+        *,
+        scene: str | None = None,
+    ) -> None:
+        """Render Manim scenes.
+
+        * ``scene=`` — single scene (CLI / wizard).
+        * ``scenes=`` — explicit list (pipeline uses :meth:`Config.pipeline_manim_scene_names`).
+        * Otherwise — ``config.manim`` ``scenes:`` list (legacy).
+        """
+        if scene is not None:
+            to_render = [scene]
+        elif scenes is not None:
+            to_render = scenes
+        else:
+            to_render = self.config.manim_scenes
+        if not to_render:
             print("[manim] No scenes configured")
             return
 
@@ -37,7 +53,7 @@ def render(self, scene: str | None = None) -> None:
 
         font = self.config.manim_font
         print(f"[manim] Rendering at {quality_label}, font={font}")
-        for s in scenes:
+        for s in to_render:
             self._render_one(manim_bin, scenes_file, s, quality_args)
 
     def _check_font(self) -> None:

src/docgen/pipeline.py

@@ -1,4 +1,10 @@
-"""Pipeline orchestrator: tts -> manim -> vhs -> compose -> validate -> concat -> pages."""
+"""Pipeline orchestrator: tts -> manim -> vhs -> compose -> validate -> concat -> pages.
+
+Manim and VHS stages render only scenes/tapes referenced by ``visual_map`` for active
+``segments.all`` entries (see :meth:`docgen.config.Config.pipeline_manim_scene_names` and
+:meth:`~docgen.config.Config.pipeline_vhs_tape_filenames`). Segments whose visuals are
+``playwright_test`` use pre-recorded files and do not run through Manim or VHS capture here.
+"""
 
 from __future__ import annotations
 
@@ -36,17 +42,25 @@ def run(
             TapeSynchronizer(self.config).sync()
 
         if not skip_manim:
-            print("\n=== Stage: Manim ===")
-            from docgen.manim_runner import ManimRunner
-            ManimRunner(self.config).render()
+            scene_list = self.config.pipeline_manim_scene_names()
+            if scene_list:
+                print("\n=== Stage: Manim ===")
+                from docgen.manim_runner import ManimRunner
+                ManimRunner(self.config).render(scenes=scene_list)
+            else:
+                print("\n=== Stage: Manim (skipped — no manim segments in visual_map) ===")
 
         if not skip_vhs:
-            print("\n=== Stage: VHS ===")
-            from docgen.vhs import VHSRunner
-            results = VHSRunner(self.config).render()
-            for r in results:
-                if not r.success:
-                    print(f"  WARNING: {r.tape} had errors: {r.errors}")
+            tape_list = self.config.pipeline_vhs_tape_filenames()
+            if tape_list:
+                print("\n=== Stage: VHS ===")
+                from docgen.vhs import VHSRunner
+                results = VHSRunner(self.config).render(tapes=tape_list)
+                for r in results:
+                    if not r.success:
+                        print(f"  WARNING: {r.tape} had errors: {r.errors}")
+            else:
+                print("\n=== Stage: VHS (skipped — no vhs segments in visual_map) ===")
 
         print("\n=== Stage: Compose ===")
         from docgen.compose import ComposeError, Composer
@@ -57,9 +71,11 @@ def run(
             if self._should_retry_manim(exc, skip_manim, retry_manim_on_freeze):
                 print("\n=== Compose FREEZE GUARD detected; retrying Manim + compose once ===")
                 self._clear_manim_media_cache()
-                print("\n=== Stage: Manim (retry) ===")
-                from docgen.manim_runner import ManimRunner
-                ManimRunner(self.config).render()
+                scene_list = self.config.pipeline_manim_scene_names()
+                if scene_list:
+                    print("\n=== Stage: Manim (retry) ===")
+                    from docgen.manim_runner import ManimRunner
+                    ManimRunner(self.config).render(scenes=scene_list)
                 print("\n=== Stage: Compose (retry) ===")
                 composer.compose_segments(self.config.segments_all)
             else: