feat(evals): SkillGym-inspired DX layer (authoring façade, reporter lifecycle, workspace presets, token snapshots)#37

Merged

ThomasK33 merged 36 commits into

mainfrom

Apr 19, 2026

Commits on Apr 18, 2026

feat(evals): add evals/authoring façade (builders + raw escape hatches)
ThomasK33
committed
test(evals): unit tests for authoring façade compilation
ThomasK33
committed
refactor(evals): migrate 4 pilot cases to authoring façade
ThomasK33
committed
test(evals): golden parity tests for migrated pilot cases
ThomasK33
committed
test(evals): CLI-level integration tests for migrated pilot cases
ThomasK33
committed
feat(evals): add reporter lifecycle types and dispatcher
ThomasK33
committed
feat(evals): add console jsonl and final report adapters
ThomasK33
committed
feat(evals): add scheduler reporter hooks and case runtime tracker
ThomasK33
committed
feat(evals): emit reporter lifecycle from lane runners
ThomasK33
committed
feat(evals): wire reporter selection into eval run loop
ThomasK33
committed
test(evals): cover reporter CLI and integration flows
ThomasK33
committed
style(evals): format files touched by reporter phases
ThomasK33
committed
style(evals): format reporter foundation files
ThomasK33
committed
style(evals): format files touched by earlier eval DX phases
ThomasK33
committed
feat(evals): add workspace preset types + registry + resolver
ThomasK33
committed
feat(evals): integrate workspace presets into runners
ThomasK33
committed
feat(evals): add .workspace() builder, register agent-tty-smoke preset, apply to pilots
ThomasK33
committed
feat(evals): surface resolved workspace plan in case.start reporter events
ThomasK33
committed
test(evals): unit + integration tests for workspace presets
ThomasK33
committed
feat(evals): add optional token usage to normalized output
ThomasK33
committed
feat(evals): normalize token usage in claude and codex providers
ThomasK33
committed
feat(evals): support deterministic token usage in fixtures provider
ThomasK33
committed
docs(evals): add dogfood proof bundle for Phase 5 token usage
ThomasK33
committed
feat(evals): add token-usage sidecar artifact writer
ThomasK33
committed
feat(evals): emit token-usage sidecars from prompt/execution/dogfood runners
ThomasK33
committed
feat(evals): add evals/snapshots package (fingerprint, schema, store, compare)
ThomasK33
committed
test(evals): unit tests for snapshots fingerprint/store/compare
ThomasK33
committed
feat(evals): add token report summary to reports
ThomasK33
committed
feat(evals): extend run finish event with tokenReport
ThomasK33
committed
feat(evals): wire --snapshot-update/--snapshot-check/--snapshot-threshold/--snapshot-dir flags
ThomasK33
committed
feat(evals): aggregate per-trial token usage into TokenReportSummary
ThomasK33
committed
test(evals): integration coverage for token snapshots + CLI flag validation
ThomasK33
committed
style(evals): format Phase 5 token-usage + snapshots files
ThomasK33
committed
docs(evals): document eval DX authoring, reporters, workspaces, and snapshots
ThomasK33
committed
docs(skills): update eval-guide for eval DX surfaces
ThomasK33
committed
docs(dogfood): reference phase 5 eval proof bundle
ThomasK33
committed