Pipeline idea — CI evidence gate: green means proven

[fenar]

The gap: convention 6 says claims carry evidence, and 101/201 honor it with full QA/ packs — but the packs are hand-run. 202/301/302 are "being brought to the same bar" manually. Evidence that isn't regenerated on every change goes stale silently.

Proposal: turn the QA convention into CI machinery.

Design:

• PR replay gate: GitHub Actions replays every course's --offline scripted episode on each pull request — no model endpoint needed, that's exactly what offline mode is for. CPU-only runners, minutes per course.
• Golden-trace diff: compare the replayed episode against the committed golden trace structurally — tool-call sequence, arguments schema, final verdicts — not verbatim text, so prompt wording changes don't break the build while behavior changes do.
• Evidence pack as artifact: every green run publishes the regenerated QA pack as a build artifact; the repo badge means "the logs in the repo were reproduced by the last merge," not "someone ran this once in July."
• Tool smoke tests: each course's MCP servers get a start-call-verify smoke job (101's QA pack already defines the pattern; lift it into the workflow).
• Nightly (optional, later): scheduled live smoke against a real endpoint using the llm-credentials secret, kept out of the PR path so forks and contributors never need credentials.

Effort: small — the hard part (deterministic offline episodes) is already a repo convention. This is wiring, not new capability.

Why it matters: it upgrades "QA-passed" from a snapshot claim to a standing guarantee, and it's the prerequisite for accepting outside contributions without eroding the evidence bar.