Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified qa/scores.db
Binary file not shown.
3 changes: 2 additions & 1 deletion qa/scores_ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@

> **Ruler** = `scoring_config_version` (a content hash of the rubric + schema + gate files). Rows under DIFFERENT Ruler values are **NOT directly comparable as a quality trend** — the ruler changed (a rubric recalibration or a new gate moves the number with no change in play quality). Use `python3 qa/scores_db.py --compare` for a ruler-fenced trend; comparing across rulers requires re-scoring an archived transcript under the current ruler. **RC** = the release candidate a run scored (e.g. `v1.0.4-rc1`).

> Rows: **65** · rendered 2026-06-09T18:10:34+00:00
> Rows: **66** · rendered 2026-06-09T23:47:29+00:00

| Run | When | SHA | Build date | Surface | DM model | Actor model | Scorer | Ruler | RC | Methodology | Story | Mech | AngryDM | Behav | Sat | RRI | Crit | Img% | Pass | Source | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| v1.0.4-rc2-c92a393 | 2026-06-09T23:47:29+00:00 | c92a393 | 2026-06-09 | GUI-built-app | opus | opus | claude | sc_df34ecd02b4f | v1.0.4-rc2 | 11-gate RRI rc2: VM part-B 5-persona @c92a393 (post 7 Sprint-1 reliability fixes); Mac part-A SKIPPED (disk 3.9Gi) → native_gate=no-handoff gap | 4 | 3 | 3 | GREEN | 5.8 | 2.7 | 1 | 0% | FAIL | VM:/root/worldos-qa/results/RRI-5canon.json @c92a393 | POST-Sprint-1-reliability (7 fixes). PRODUCT IMPROVED vs rc1: criticals 4->1 (the #735 active-PC keystone validated — gone from the 4 personas that had it), sat 4.8->5.8, behavioral GREEN, 4/5 personas clean (adv/narr/vet sat=7,0-crit,no-giveup). RRI 3.6->2.7 is DEPRESSED by 2 non-product effects: (1) -1 native_gate = Mac handoff SKIPPED (disk), not a regression — with handoff ~3.7; (2) -1 no_give_up = newbie's mid-stream STALL (beat streamed partially then froze mid-word, controls locked, no retry/cancel = the #623 DEEPER recovery layer; the heartbeat fixed perceived-hang but not a genuinely-stalled beat). newbie's 1 crit IS that same stall. mech 3.0 = one-duo social-sampling (engine-duo ~3.8). ui_audit FAIL (rc=1, separate from #306/#744 title-bar which is fixed). Same ruler sc_5ac7a1d9103c as rc1 (apples-to-apples). NEXT DOMINO: fix newbie mid-stream-stall recovery -> flips no_give_up + zero_critical. |
| v1.0.4-rc1-fa97b34 | 2026-06-09T18:10:34+00:00 | fa97b34 | 2026-06-09 | GUI-built-app | opus | opus | claude | sc_5ac7a1d9103c | v1.0.4-rc1 | 11-gate RRI RC1: Mac part-A handoff 100/100 + VM part-B 5-persona sweep (lean-ON); rollup via fixed release_readiness.py (#728) | 3.9 | 3.5 | 3.2 | GREEN | 4.8 | 3.6 | 4 | 0% | FAIL | /Volumes/LEXAR/Codex/worldos-qa-results-fa97b34/RRI-full.json | RC1 honest 11-gate RRI @fa97b34 (post #719/#720). PASS: native_gate(signal; Mac handoff 100/100), arc, no_give_up, behavioral GREEN, palette. FAIL: cross_persona_sat 4.8<7, zero_critical(4), story 3.9<4.3, mech 3.2<4.5(one-duo sampling, understated), ui_audit FAIL(axe). native_gate withheld credit pending --support-preflight (split VM+Mac contract) — handoff itself is valid 100/100. image_render 0% = VM-gateway gap (Mac/gateway lane measures it). vs Tuesday 3.6: no_give_up RED->PASS (input-lock give-up gone), sat 4.4->4.8, behavioral GREEN. Caught+fixed build_shas rollup regression (#728) that had crashed EVERY release_readiness.py rollup since #723. |
| reorient-leanON-e2c0da4 | 2026-06-09T13:53:41+00:00 | e2c0da4 | 2026-06-09 | engine-duo | opus | opus | claude | sc_52a5cbd26775 | | 3-lens duo 8-beat lean-ON (CLAWDND_LEAN_BEATS); reorient post-fix baseline | 4.1 | 3.7 | 3.4 | GREEN | | | | | | /Volumes/LEXAR/Codex/session-notes/2026-06-05/worldos-release-grind/reorient-leanON-duo-e2c0da4.log | First ruler-stamped baseline (#722). Lean-ON Opus healthy → confirms the lean-ON dip was transient (patched lean re-ground bug). Lens files identical e2c0da4↔52e74e2; only release_readiness.py changed (#723), which does NOT affect engine-duo lens scoring. |
| handoff-20260607-da05101-current-main-clean | 2026-06-07 | da05101 | | GUI-built-app | scripted+codex | scripted | | | | hybrid handoff gate (web smoke 5 + built smoke 5 + built Codex 1) | | | | | | | | | PASS | /Volumes/LEXAR/Codex/worldos-agent-grade-app-testability/handoff-20260607-da05101-current-main-clean/ | handoff_score=100/100 on current origin/main da05101; clean same-disk worktree, dirty=false, release_verdict=false. Supersedes 9545383 as GUI velocity proof. Full #466 five-persona RRI still required. No quality lens. |
Expand Down
Loading