Part of #244 (full-codebase eval testbed). No hard dependency — the fixture repo can be published
today; the init scaffolding lands once the codebase block exists.
Why
#244 asks for this directly:
We should provide a default fixture for a simple codebase (maybe just by creating it on github),
and suggest using the eval-magic codebase itself as a complex fixture. Instructions for what
fixtures should be used should be part of the committed eval files in some way.
Without a default, the first thing every new user hits after the codebase feature ships is "…so what
do I point this at?" A published fixture makes the feature usable on day one; documenting eval-magic
itself gives people a genuinely complex target to graduate to.
Scope
- Publish a small default fixture project on GitHub — enough structure for a realistic task, small
enough to clone quickly.
- Scaffold a
codebase block from eval-magic init pointing at it, with a pinned ref.
- New shipped guide
docs/guides/codebases.md → eval-magic docs codebases, covering how to source
a codebase, how to pin it, how the resolved SHA is recorded, and when to reach for a complex
fixture. build.rs discovers docs/guides/*.md at compile time, so the topic registers
automatically; the filename stem is the topic name and must be ASCII kebab-case.
- Document eval-magic itself as the complex fixture.
Per CLAUDE.md, shipped output refers to guides as eval-magic docs <topic>, never repo-relative
paths.
Files
docs/guides/codebases.md (new)
src/cli/commands/init.rs
src/cli/help.rs
tests/cli/docs.rs
Acceptance criteria
eval-magic docs lists codebases, and eval-magic docs codebases prints the guide.
eval-magic init scaffolds a runnable eval with a pinned codebase block.
- That scaffolded eval runs end to end against the published fixture.
- The guide covers both the simple fixture and eval-magic-as-fixture.
- Guide-contract tests in
tests/cli/docs.rs cover the new topic.
Cross-cutting requirements
- Mode B parity. Acceptance must hold for both
Mode::NewSkill and Mode::Revision
(src/core/types.rs:225), and the snapshot/promote path (src/workspace/snapshot.rs,
src/workspace/promote.rs) must keep working against codebase-backed iterations.
- Provenance reaches the report. Source + resolved SHA must land in
conditions.json, each
run.json, benchmark.json, and the BASELINE.md built by promote.rs:231 — not just in the
workspace.
Verification
cargo fmt --check && cargo build && cargo test && cargo clippy --all-targets -- -D warnings
Implementation requires the slow-powers:working-with-tdd skill.
Parent: #244
Part of #244 (full-codebase eval testbed). No hard dependency — the fixture repo can be published
today; the
initscaffolding lands once thecodebaseblock exists.Why
#244 asks for this directly:
Without a default, the first thing every new user hits after the codebase feature ships is "…so what
do I point this at?" A published fixture makes the feature usable on day one; documenting eval-magic
itself gives people a genuinely complex target to graduate to.
Scope
enough to clone quickly.
codebaseblock fromeval-magic initpointing at it, with a pinnedref.docs/guides/codebases.md→eval-magic docs codebases, covering how to sourcea codebase, how to pin it, how the resolved SHA is recorded, and when to reach for a complex
fixture.
build.rsdiscoversdocs/guides/*.mdat compile time, so the topic registersautomatically; the filename stem is the topic name and must be ASCII kebab-case.
Per
CLAUDE.md, shipped output refers to guides aseval-magic docs <topic>, never repo-relativepaths.
Files
docs/guides/codebases.md(new)src/cli/commands/init.rssrc/cli/help.rstests/cli/docs.rsAcceptance criteria
eval-magic docslistscodebases, andeval-magic docs codebasesprints the guide.eval-magic initscaffolds a runnable eval with a pinnedcodebaseblock.tests/cli/docs.rscover the new topic.Cross-cutting requirements
Mode::NewSkillandMode::Revision(
src/core/types.rs:225), and the snapshot/promote path (src/workspace/snapshot.rs,src/workspace/promote.rs) must keep working against codebase-backed iterations.conditions.json, eachrun.json,benchmark.json, and theBASELINE.mdbuilt bypromote.rs:231— not just in theworkspace.
Verification
Implementation requires the
slow-powers:working-with-tddskill.Parent: #244