qa(scores): amend false rc1/rc2 ledger claims, guard the claim class, land the lens-only ruler fence (#725)

[100yenadmin]

What

P1 ledger/ruler integrity batch (3 small related fixes in qa/), from the post-Sprint-1 adversarial audit. Closes #725.

1. Amend the false ledger claims (qa/scores_amend_rc1_rc2_notes.py — idempotent one-shot, db + ledger re-rendered)

• v1.0.4-rc2-c92a393 notes claimed "Same ruler sc_5ac7a1d9103c as rc1 (apples-to-apples)" — false: the row itself is stamped sc_df34ecd02b4f. Verified by recomputing the ruler hash from git history: rc1's stamp matches the ruler state at a104f1e (post-fix(rri): restore main()'s build_shas — #723 regression that crashed every RRI rollup #728), and fix(engine): companion location/XP sync — de-facto companions co-locate AND co-earn (#353) #739 changed qa/assert_behavioral.py (a SCORING_CONFIG_FILES member) before the rc2 recording. Amended to: CROSS-RULER, the LENS prose comparison is still informative, but the BEHAVIORAL-capped lens comparability is a re-baseline.
• v1.0.4-rc1-fa97b34 notes claimed "ui_audit FAIL(axe)" — false: the VM log (worldos-qa-results-fa97b34/results/ui_audit.log lines 32–33) shows axe never ran (browser-driver-manager missing → silent WARN-skip). The actual FAILs were launcher play_reachable (no Resume/Continue CTA @1366+@1512) and merchant art_present (placeholders=4 > cap 2). Amended accordingly.

Both amendments carry an explicit [AMENDED 2026-06-10: …] tag — the ledger shows the correction, not a silent rewrite.

2. Guard the false-claim class (scores_db.add_run)

add_run now raises ValueError when notes cite a ruler hash (sc_/lc_ + 12 hex) that differs from the row's stamped version. The exact rc2 incident is a test case. To reference another run's ruler, cite its run_id, not its hash.

3. Land #725 — the lens-only ruler fence

• qa/scoring_config_version.py: LENS_CONFIG_FILES = the 8 lens-relevant files (3 rubrics incl. rubric_angry_dm.src.md, 3 schemas, assert_behavioral.py) = SCORING_CONFIG_FILES minus release_readiness.py; lens_config_version() with the same hashing (distinct lc_ prefix so a lens hash can never be confused with — or false-trip the guard against — a full-ruler sc_ hash); CLI --lens.
• qa/scores_db.py: additive lens_config_version column (existing _ensure_schema ALTER path; old rows read NULL). add_run stamps both versions; when the full ruler is pinned (backfill), the lens stamp stays NULL unless also pinned — stamping today's lens hash onto an old run would itself be a false claim.
• compare_rc now fences the engine-duo trend on the LENS ruler (falls back to the full ruler for pre-lens rows — conservative: may split more than needed, never falsely merges). An RRI-gate-only edit (the fix(rri): scope native_gate build-SHA contract to the canonical release personas #723/fix(rri): restore main()'s build_shas — #723 regression that crashed every RRI rollup #728 case) no longer false-fences the lens trend.
• New --compare-rc-surface: shows GUI-built-app RC rows (the RRI sweeps, incl. rc1/rc2) in their own blocks, fenced on the FULL ruler — the RRI is produced by release_readiness.py, which IS in that ruler.

Verification

• TDD: 14 new tests written first (red), then implementation (green).
• uv run --directory servers/engine python -m pytest ../../qa/test_scores_db_comparability.py ../../qa/test_release_readiness.py ../../qa/test_scores_db.py -q -p no:xdist → 65 passed.
• bash qa/fast_gate.sh → PASS (188 deterministic engine tests).
• Migration run twice → second run is a no-op (idempotency proven).
• Full ruler hash unchanged by this PR (sc_df34ecd02b4f before and after — none of the changed files is a SCORING_CONFIG_FILES member). Current lens ruler: lc_4f7b718f3eb1.
• Two negative-disclosure tests now run against the committed qa/scores.db itself: no row's notes may cite a hash differing from its stamp, and the rc1/rc2 rows must carry the amended truth.

Invariants

Additive only (new column via ALTER, old rows read NULL); no wire contracts touched; engine untouched.

Summary by CodeRabbit

• Documentation

  • Updated QA ledger documentation to explain new Lens ruler concept for tracking configuration changes.

• New Features

  • Added lens-aware release candidate comparison capabilities with additional filtering options.

• Bug Fixes

  • Amended incorrect release candidate notes with corrected comparison explanations and failure details.
  • Enhanced consistency checks to ensure notes accurately reference configuration ruler versions.

• Tests

  • Added comprehensive test coverage for ruler comparability and configuration tracking.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a lens ruler—a quality-trend-fencing mechanism that excludes release-readiness-gate-only changes—alongside the existing full scoring ruler. It updates hashing, database persistence, run stamping, RC comparison fencing, CLI, documentation, and test coverage to implement lens-aware tracking and prevent non-apples-to-apples comparisons.

Changes

Lens ruler: foundation through testing

Layer / File(s)	Summary
Lens ruler hash functions qa/scoring_config_version.py	LENS_CONFIG_FILES excludes release_readiness.py; content_hash() generalizes hashing with prefix support; scoring_config_version() and new lens_config_version() return sc- and lc_-prefixed hashes respectively; --lens CLI flag added.
Database schema and imports for lens ruler qa/scores_db.py	lens_config_version imported and added as nullable ledger column; documentation clarifies fencing semantics and NULL values for pre-lens rows.
Lens ruler stamping and validation in add_run qa/scores_db.py	add_run() now auto-stamps both scoring_config_version and lens_config_version; stricter notes validation rejects mismatched sc_/lc_ hashes embedded in notes.
Lens-aware compare_rc with RC surface option qa/scores_db.py	compare_rc() refactored to accept include_rc_surface; groups engine-duo runs by lens_config_version (with scoring_config_version fallback for pre-lens rows); marks current blocks by lens or full ruler match; optionally appends RC-surface section fenced by full ruler; markdown preamble updated with ruler/lens-ruler semantics and comparison instructions.
CLI integration: --compare-rc-surface flag qa/scores_db.py	--compare-rc-surface flag added to CLI parser; threaded through main() into compare_rc() include_rc_surface parameter.
One-shot amendment script for rc1/rc2 notes qa/scores_amend_rc1_rc2_notes.py	Standalone script connects to scores.db, performs idempotent substring replacements in run notes for defined amendments, validates consistency (skip/no-op/warn+fail), commits, re-renders ledger, reports amended count.
Ledger documentation and rc1/rc2 note amendments qa/scores_ledger.md	Ledger docs introduce Lens ruler concept; table schema adds Lens ruler column; rc2 notes amended with corrected cross-ruler explanation; rc1 notes updated with specific UI-audit failure evidence; Lens ruler values populated for all rows.
Comprehensive lens ruler test coverage qa/test_scores_db_comparability.py	Tests verify legacy schema migration adds lens_config_version (NULL for pre-lens); lens hashing stable under RRI-only edits but changes under rubric/gate edits; add_run stamps both rulers and respects pinning; notes/stamp consistency rejects mismatched hashes; compare_rc fences by lens ruler with pre-lens fallback and RC-surface opt-in; ledger integrity: notes cite matching rulers and rc1/rc2 contain amended truth.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• electricsheephq/WorldOS#725: This PR implements the code-level lens ruler tracking feature: introduces lens_config_version hash and updates compare_rc to fence engine-duo trends by lens ruler instead of full scoring ruler, directly addressing the issue's objectives.

Possibly related PRs

• electricsheephq/WorldOS#747: Both PRs amend the rc2 run (v1.0.4-rc2-c92a393) notes in scores_ledger.md for cross-ruler comparability; this PR extends with lens ruler context.
• electricsheephq/WorldOS#722: Both PRs refactor compare_rc to be ruler-aware and fence quality trends; this PR extends the fencing logic to lens_config_version.

Poem

🐰 A lens splits the ruler in two,
One lens-pure, one scoring through and through,
So RRI changes won't cloud the view,
Apples to apples—the trends ring true!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the three main changes: amending false ledger claims, guarding against false claims via validation, and implementing the lens-only ruler fence.
Description check	✅ Passed	The description covers all required sections, provides detailed explanations of the three changes with specific evidence and test results, but the CLA checkbox section is not completed.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

qa/test_scores_db_comparability.py (1)

128-128: 💤 Low value

Prefer next() over list slice for single-element lookup.

Both lines use list comprehension with [0] indexing to find a single row. next() is more idiomatic and avoids building an intermediate list.

♻️ Proposed refactor

-        legacy = [r for r in scores_db.fetch_rows(db) if r["run_id"] == "legacy"][0]
+        legacy = next(r for r in scores_db.fetch_rows(db) if r["run_id"] == "legacy")

-        r2 = [x for x in scores_db.fetch_rows(db) if x["run_id"] == "old2"][0]
+        r2 = next(x for x in scores_db.fetch_rows(db) if x["run_id"] == "old2")

Also applies to: 196-196

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@qa/test_scores_db_comparability.py` at line 128, Replace the
list-comprehension + [0] single-element lookups with next() generator
expressions to avoid building an intermediate list: change the assignment to
legacy (currently using [r for r in scores_db.fetch_rows(db) if r["run_id"] ==
"legacy"][0]) to use next(r for r in scores_db.fetch_rows(db) if r["run_id"] ==
"legacy") and make the analogous replacement for the other single-row lookup at
the second occurrence (line ~196); if you want to handle missing rows
gracefully, pass a default to next(...) or wrap it in a clear error message.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@qa/test_scores_db_comparability.py`:
- Line 128: Replace the list-comprehension + [0] single-element lookups with
next() generator expressions to avoid building an intermediate list: change the
assignment to legacy (currently using [r for r in scores_db.fetch_rows(db) if
r["run_id"] == "legacy"][0]) to use next(r for r in scores_db.fetch_rows(db) if
r["run_id"] == "legacy") and make the analogous replacement for the other
single-row lookup at the second occurrence (line ~196); if you want to handle
missing rows gracefully, pass a default to next(...) or wrap it in a clear error
message.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a33836c3-b816-4904-ab19-16b12ca8a6e0

📥 Commits

Reviewing files that changed from the base of the PR and between f24a102 and 53cbea5.

⛔ Files ignored due to path filters (1)

• qa/scores.db is excluded by !**/*.db

📒 Files selected for processing (5)

• qa/scores_amend_rc1_rc2_notes.py
• qa/scores_db.py
• qa/scores_ledger.md
• qa/scoring_config_version.py
• qa/test_scores_db_comparability.py