Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified qa/scores.db
Binary file not shown.
4 changes: 3 additions & 1 deletion qa/scores_ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,12 @@

> **Ruler** = `scoring_config_version` (a content hash of the rubric + schema + gate files, RRI gate included). **Lens ruler** = `lens_config_version` (the 8 files that produce the story/mech/angry LENS numbers — full ruler minus `release_readiness.py`; blank = recorded before lens stamping, #725). Rows under DIFFERENT Ruler values are **NOT directly comparable as a quality trend** — the ruler changed (a rubric recalibration or a new gate moves the number with no change in play quality). Use `python3 qa/scores_db.py --compare` for a lens-fenced engine-duo trend (add `--compare-rc-surface` for the GUI-built-app RC blocks); comparing across rulers requires re-scoring an archived transcript under the current ruler. **RC** = the release candidate a run scored (e.g. `v1.0.4-rc1`).

> Rows: **77** · rendered 2026-06-16T18:36:09+00:00
> Rows: **79** · rendered 2026-06-17T06:05:06+00:00

| Run | When | SHA | Build date | Surface | DM model | Actor model | Scorer | Ruler | Lens ruler | RC | Methodology | Story | Mech | AngryDM | Behav | Sat | RRI | Crit | Img% | s/beat | cold-open s | turns/beat | Acts | Structural coverage | Pass | Source | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| gs-ledger-deep | 2026-06-17T08:30:00Z | 4f88d3a | 2026-06-17 | engine-duo | opus | sonnet | claude | sc_d5011ee20801 | lc_ec48e53a4726 | v1.0.4-rc5 | authored-spine-fulldepth | 4.8 | 3.9 | 2.6 | GREEN | | | | | | | | 3 | all ✓ + camp_scene; Ondine 0→80, all 4 gates; 2 quests resolved (incl. personal arc) | PASS | | BG golden spine 'The Ledger of Mercy' full-depth proof (24b): Ondine all 4 gates incl betrayal, camp_scene ran, 2 quests resolved+evolved, structural PASS. story 4.8. |
| gs-ember-deep | 2026-06-17T08:00:00Z | 4f88d3a | 2026-06-17 | engine-duo | opus | sonnet | claude | sc_d5011ee20801 | lc_ec48e53a4726 | v1.0.4-rc5 | authored-spine-fulldepth | 4.8 | 3.8 | 2.9 | GREEN | | | | | | | | 3 | recruit camp approval combat quest-resolved evolved (all ✓); Toll 0→47 loyalty@25 unlocked | PASS | | Golden-spine full-depth proof (Embergloom 24b): companion approval climbed, arc gate unlocked, quest resolved+evolved, structural_completeness PASS. story 4.8 best-yet. |
| RRI-8f20d71 | 2026-06-16T13:59:39Z | 8f20d71 | 2026-06-16 | GUI-headless-proxy | sonnet | sonnet | claude | sc_2e98030df329 | lc_4a5237765450 | v1.0.4-rc3 | gate | 4.2 | 3.5 | 3.5 | GREEN | 7 | 6.4 | 0 | | | | | | | FAIL | | CLEAN GREEN-behavioral (party traveled, no false-cap). Planning layer #951 VALIDATED: optimizer 5->6, crit 1->0 (zero_critical passes), cross_persona_sat passes (avg 7.0). Personas newbie8/veteran7/adversarial6/narrative8/optimizer6. story 4.2 (near 4.3 bar), mech 3.5 (coverage-cap). Fails: native_gate+image_render (VM-only; Mac handoff -> ~8/9-gates) + story 4.2<4.3 + mech. |
| RRI-15a49cb-rc3 | 2026-06-16T11:06:44Z | 15a49cb | 2026-06-16 | GUI-headless-proxy | sonnet | sonnet | claude | sc_fa0c567637ad | lc_4a5237765450 | v1.0.4-rc3 | gate | 2.5 | 2.1 | 2.1 | RED | 7 | 3.6 | 1 | | | | | | | FAIL | | *PARTIAL/HARNESS* behavioral RED false-cap (party_traveled, emergent single-scene) caps story/mech <=2.5; true baseline ~6.4 (0a8a3fc GREEN). optimizer crit=1 = half-caster FALSE ALARM (engine SRD-correct: L10 Paladin=L3 slots). 4/5 personas: newbie8 veteran7 adversarial7 narrative8 optimizer5. Real lever: min-maxer planning layer. |
| combat-sprint-median-af5ba1f | 2026-06-15T22:59:13+00:00 | af5ba1f | 2026-06-16 | engine-duo | opus | | claude | sc_fa0c567637ad | lc_4a5237765450 | v1.0.4-rc2 | combat-sprint median n>=2 (Angry-DM 5e-fidelity) — the canonical mech gate | | 3.6 | 3.6 | GREEN | | | 0 | | | | | | | FAIL | | Combat-sprint median ~3.6 (3.7,3.5; behavioral GREEN) — AGREES with the duo's mech 3.6, so mech is NOT a single-duo undersample; it's the known coverage-cap on short pre-seeded fights (a single combat-sprint Angry-DM is capped ~3). Engine combat is SRD-correct/feature-complete; the lift to >=4.5 is richer combat COVERAGE (multi-round, conditions, tactical depth, broader seeds) = DM-craft + scenario coverage, NOT an engine fix. |
Expand Down
Loading