chore(qa): RRI scorer + GUI runbook/workbook + RRI gates (operating goal & scorecard) - #413
Merged
Conversation
…oal & scorecard) Release Readiness Index (qa/release_readiness.py): 11 hard gates -> 0-10, reads disk artifacts (robust to tool-channel fabrication). Adds image-render-rate + palette-live gates (the two owner-visible defects the old gate could pass while broken). WorldOS-GUI-RUNBOOK.md: two-surface look-and-wire loop (iterate 8799-from-canonical, gate on built .app). qa/GUI_WORKBOOK.md: living punch-list. OPERATING-GOAL section 4 + SCORECARD extended with RRI. Docs/tooling only; engine + wire contracts untouched.
📝 WalkthroughWalkthroughThis PR establishes a Release Readiness Index (RRI) framework for the WorldOS GUI: defining hard-gated success criteria in policy documents, implementing a Python script that aggregates on-disk QA artifacts into a single readiness signal, and integrating RRI scoring into the daily release and fix loop. ChangesRelease Readiness Index (RRI) Framework, Script, and Operational Integration
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@qa/release_readiness.py`:
- Line 140: The tuple entry for "arc_completed" uses an unnecessary f-string
literal f"completed_intro_flow on >=1 persona" which triggers Ruff F541; change
that literal to a plain string "completed_intro_flow on >=1 persona" in the
mapping where "arc_completed" is defined (the tuple containing any_completed and
the message) so the code uses a regular string rather than an f-string with no
placeholders.
- Around line 67-70: The status parsing in image_render_rate (variables net,
img, ok calculation) can raise when n.get("status") is non-numeric; change the
ok computation to safely coerce status to int (or treat as non-OK) by wrapping
int conversion in a try/except (ValueError, TypeError) or using a small helper
like safe_int(status, default=0) and only count entries where the parsed int is
>0 and <400; ensure you reference the same variables (img, ok, total) and retain
the existing logic that malformed statuses are treated as failures (not
incrementing ok).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c9f511c6-9ee9-4969-ad79-45b9f26dc7b6
📒 Files selected for processing (5)
WorldOS-GUI-RUNBOOK.mdWorldOS-OPERATING-GOAL.mdqa/GUI_WORKBOOK.mdqa/SCORECARD.mdqa/release_readiness.py
Comment on lines
+67
to
+70
| img = [n for n in net if "/image" in str(n.get("url", ""))] | ||
| if img: | ||
| ok = sum(1 for n in img if int(n.get("status", 0) or 0) and int(n.get("status")) < 400) | ||
| total = len(img) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the relevant portion around the reported lines
sed -n '1,120p' qa/release_readiness.py | nl -ba | sed -n '55,95p'
# Locate the function definition and show its full body (if not too large)
rg -n "def image_render_rate" -S qa/release_readiness.py
python3 - <<'PY'
import re, itertools
path="qa/release_readiness.py"
with open(path,'r',encoding='utf-8') as f:
lines=f.readlines()
# find def image_render_rate
start=None
for i,l in enumerate(lines):
if re.search(r'^\s*def\s+image_render_rate\s*\(', l):
start=i
break
if start is None:
raise SystemExit("image_render_rate not found")
# print until next def/class at same indent (best-effort)
indent=re.match(r'^(\s*)def', lines[start]).group(1)
end=len(lines)
for j in range(start+1,len(lines)):
if re.match(r'^'+re.escape(indent)+r'(def|class)\s', lines[j]):
end=j
break
for k in range(start,end):
print(f"{k+1:4d}:{lines[k].rstrip()}")
PY
# Check for any existing safe parsing helpers or similar logic in this file
rg -n "int\\(.*status" qa/release_readiness.py
rg -n "status.*int\\(" qa/release_readiness.py
rg -n "ValueError|TypeError" qa/release_readiness.pyRepository: electricsheephq/WorldOS
Length of output: 110
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Snippet around lines ~55-95 =="
awk 'NR>=55 && NR<=95 {printf "%4d:%s\n", NR, $0}' qa/release_readiness.py
echo "== Find image_render_rate definition =="
rg -n "def image_render_rate" qa/release_readiness.py
echo "== Show image_render_rate function body (best-effort) =="
python3 - <<'PY'
import re
path="qa/release_readiness.py"
with open(path,'r',encoding='utf-8') as f:
lines=f.readlines()
start=None
for i,l in enumerate(lines):
if re.search(r'^\s*def\s+image_render_rate\s*\(', l):
start=i
break
if start is None:
raise SystemExit("image_render_rate not found")
indent = re.match(r'^(\s*)def', lines[start]).group(1)
end=len(lines)
for j in range(start+1, len(lines)):
if re.match(r'^'+re.escape(indent)+r'(def|class)\s', lines[j]):
end=j
break
for k in range(start, end):
print(f"{k+1:4d}:{lines[k].rstrip()}")
PY
echo "== Search for status int parsing in this file =="
rg -n "int\(.{0,40}status|status.{0,40}int\(" qa/release_readiness.py
echo "== Search for ValueError/TypeError handling =="
rg -n "ValueError|TypeError" qa/release_readiness.pyRepository: electricsheephq/WorldOS
Length of output: 3502
Harden status parsing in image_render_rate to avoid scorer crashes on non-numeric status
Line 69 calls int(n.get("status")) without guarding, so malformed/non-numeric status values in network.ndjson can raise and terminate scoring.
💡 Proposed fix
def image_render_rate(run: Path, score: dict) -> tuple[float, int, int]:
@@
- if img:
- ok = sum(1 for n in img if int(n.get("status", 0) or 0) and int(n.get("status")) < 400)
+ if img:
+ def _status_code(v: object) -> int:
+ try:
+ return int(v or 0)
+ except (TypeError, ValueError):
+ return 0
+
+ ok = sum(
+ 1
+ for n in img
+ if (code := _status_code(n.get("status"))) and code < 400
+ )
total = len(img)
return (ok / total if total else 1.0), ok, total🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@qa/release_readiness.py` around lines 67 - 70, The status parsing in
image_render_rate (variables net, img, ok calculation) can raise when
n.get("status") is non-numeric; change the ok computation to safely coerce
status to int (or treat as non-OK) by wrapping int conversion in a try/except
(ValueError, TypeError) or using a small helper like safe_int(status, default=0)
and only count entries where the parsed int is >0 and <400; ensure you reference
the same variables (img, ok, total) and retain the existing logic that malformed
statuses are treated as failures (not incrementing ok).
| # ---- the 11 gates (each contributes to RRI; all must hold for 10/10) ---- | ||
| gates = { | ||
| "native_gate": (native == "PASS", f"part_a={native or 'n/a'}"), | ||
| "arc_completed": (any_completed, f"completed_intro_flow on >=1 persona"), |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the exact referenced line with context
nl -ba qa/release_readiness.py | sed -n '120,160p'
# Search for the specific string usage in the repo for consistency
rg -n 'completed_intro_flow on >=1 persona' -S .
rg -n 'F541|f"[^"]*{[^}]*}' qa/release_readiness.py -S || trueRepository: electricsheephq/WorldOS
Length of output: 110
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path("qa/release_readiness.py")
lines = p.read_text(encoding="utf-8").splitlines()
start, end = 120, 160
for i in range(start, end+1):
if 1 <= i <= len(lines):
print(f"{i:4d}: {lines[i-1]}")
PY
python3 - <<'PY'
from pathlib import Path
p = Path("qa/release_readiness.py")
text = p.read_text(encoding="utf-8")
needle = 'completed_intro_flow on >=1 persona'
print("needle occurrences:", text.count(needle))
print("line with needle(s):")
for i, line in enumerate(text.splitlines(), 1):
if needle in line:
print(f"{i}: {line}")
PYRepository: electricsheephq/WorldOS
Length of output: 2725
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find Ruff config (common names)
rg -n --hidden --no-ignore-vcs "ruff" pyproject.toml setup.cfg .ruff.toml ruff.toml 2>/dev/null || true
ls -la | head
# Locate Ruff config files
fd -tf ".ruff.toml" -d 4 . || true
fd -tf "pyproject.toml" -d 6 . || true
fd -tf "setup.cfg" -d 6 . || true
fd -tf "ruff.toml" -d 6 . || true
# Print relevant Ruff sections if pyproject.toml exists
if [ -f "pyproject.toml" ]; then
echo "----- pyproject.toml ruff sections -----"
python3 - <<'PY'
import re
from pathlib import Path
p=Path("pyproject.toml")
t=p.read_text(encoding="utf-8")
# crude extraction around ruff blocks
m=re.search(r"\[tool\.ruff[^\]]*\]", t)
print("Found tool.ruff:", bool(m))
PY
rg -n "tool\.ruff|select|ignore|F541" pyproject.toml || true
fi
# Also directly search for F541 in repo config
rg -n "F541" -S . || trueRepository: electricsheephq/WorldOS
Length of output: 711
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show ruff config
echo "---- ruff.toml ----"
sed -n '1,200p' ruff.toml
echo "---- end preview ----"
# Search for F541 / ignore rules
rg -n "F541|f-string|F5(41|[0-9]{2})|select|ignore|extend-select|extend-ignore" ruff.toml || true
# Also locate any tool.ruff config in pyproject.toml files
for f in servers/*/pyproject.toml; do
echo "---- $f ----"
rg -n "tool\.ruff|F541|select|ignore|extend-select|extend-ignore" "$f" || true
doneRepository: electricsheephq/WorldOS
Length of output: 1595
Remove unused f-string prefix (F541)
qa/release_readiness.py:140 uses f"completed_intro_flow on >=1 persona" without placeholders; Ruff F541 will flag it.
💡 Proposed fix
- "arc_completed": (any_completed, f"completed_intro_flow on >=1 persona"),
+ "arc_completed": (any_completed, "completed_intro_flow on >=1 persona"),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| "arc_completed": (any_completed, f"completed_intro_flow on >=1 persona"), | |
| "arc_completed": (any_completed, "completed_intro_flow on >=1 persona"), |
🧰 Tools
🪛 Ruff (0.15.14)
[error] 140-140: f-string without any placeholders
Remove extraneous f prefix
(F541)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@qa/release_readiness.py` at line 140, The tuple entry for "arc_completed"
uses an unnecessary f-string literal f"completed_intro_flow on >=1 persona"
which triggers Ruff F541; change that literal to a plain string
"completed_intro_flow on >=1 persona" in the mapping where "arc_completed" is
defined (the tuple containing any_completed and the message) so the code uses a
regular string rather than an f-string with no placeholders.
This was referenced May 31, 2026
This was referenced Jun 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release Readiness Index tooling + the GUI look-and-wire runbook, from the 2026-05-31 GUI reorientation. qa/release_readiness.py (11 hard gates → 0-10, disk-artifact reader). WorldOS-GUI-RUNBOOK.md (two-surface loop). qa/GUI_WORKBOOK.md (punch-list). OPERATING-GOAL §4 + SCORECARD extended with RRI (adds image-render-rate + palette-live gates — the two owner-visible defects). Docs/tooling only; engine + wire contracts untouched.
Summary by CodeRabbit
Documentation
Tests