Skip to content

Harden RRI handoff evidence bridge #505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
100yenadmin merged 1 commit into from
Jun 1, 2026
Merged

Harden RRI handoff evidence bridge #505

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 30 additions & 6 deletions WorldOS-GUI-RUNBOOK.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
> `qa/release_readiness.py` (the RRI scorer), `qa/SCORECARD.md` (the ledger).
>
> Takeover routing, 2026-06-01: `/Users/lume/ClawDnD-val` is the synced local app/private-art checkout
> (`f7ab6d7 == origin/main` after #475/#494/#495/#496/#498/#499/#500) and the default place to build/run/test the GUI and native app.
> (`4a0efe1 == origin/main` after #475/#494/#495/#496/#498/#499/#500/#501/#504) and the default place to build/run/test the GUI and native app.
> Lexar is for evidence/snapshots/logs, not the default runtime tree, because macOS permission prompts
> can break AI/browser tests when assets live on the external drive. For tracked GUI edits, prefer a
> same-disk local worktree; use Lexar worktrees only for non-GUI slices that will not launch against art.
Expand Down Expand Up @@ -64,6 +64,14 @@
accessibility review showed the chronicle with one opening narration row and one follow-up narration row,
not duplicate chat/event prose. This closes the #479 diagnostic blocker, but release still requires #466's
full non-partial RRI gate.
- The post-#504 merged-main handoff gate
(`/Volumes/LEXAR/Codex/worldos-agent-grade-app-testability/handoff-20260601T081016Z-4a0efe1/`, build
`4a0efe1`) is the current fastest GUI trust proof. It scored `handoff_score=100` with web scripted smoke
5 moves, built `dist/WorldOS.app` scripted smoke 5 moves, and built `dist/WorldOS.app` Codex-provider
playtest 1 move. All three evidence manifests passed with zero gaps, private art present, screenshots,
app-status/session-surface snapshots, move logs, provider trace, console/network/action logs, and failure
bucket fields. The Codex trace summary reported `trace_exists=true`, `line_count=175`, and
`failed_or_error_count=0`. This is the fast GUI velocity gate, not the release verdict.

## Agent-facing app contract

Expand All @@ -74,11 +82,12 @@
campaign, can the player act, where is the move sink, and is private art configured?"
- `qa/ui_playtest_app.sh` captures launcher and minted-provider `app-status` JSON into the native evidence
folder. A built-app proof that cannot produce this status object is a harness/product observability failure.
- Agent-grade testing progress as of `f7ab6d7`: #481 app-status is closed, #482 deterministic scripted
provider is merged, #483 failure buckets are merged, and #484 stable accessibility/DOM hooks are merged.
#485 evidence bundle completion and #486 gate-split follow-through remain active. A scripted `:8899`
harness surface can prove app observability, but it is not built-app release proof unless it came from
`dist/WorldOS.app` / `qa/ui_playtest_app.sh`.
- Agent-grade testing progress as of `4a0efe1`: #481 app-status is closed, #482 deterministic scripted
provider is merged, #483 failure buckets are merged, #484 stable accessibility/DOM hooks are merged,
and #504's hybrid handoff gate is merged and green on `main`. #485 evidence bundle completion and
#486 gate-split follow-through remain active. A scripted `:8899` harness surface can prove app
observability, but it is not built-app release proof unless it came from `dist/WorldOS.app` /
`qa/ui_playtest_app.sh`.

## Stand up the iteration surface (8799, playable, from canonical)
```bash
Expand Down Expand Up @@ -183,10 +192,25 @@ release truth still requires `qa/ui_playtest_app.sh` Part A+B and the full RRI s
auth/profile status, `uv`, Node/npm/Playwright availability, private-art availability or explicit
backend-only/no-art classification, env vars, budget/concurrency cap, teardown commands, and the artifact
return path under `/Volumes/LEXAR/Codex`.
- Current preflight status (2026-06-01): this local Codex Desktop session can parse an SSH alias for
`support-vm-1`, but `ssh -o BatchMode=yes support-vm-1 ...` failed DNS resolution (`nodename nor servname
provided`). Do not start #466 there until operator routing is restored and the preflight fields above are
written to Lexar evidence.
- Read-only VM scout (2026-06-01): an operator-only endpoint note can reach `evaos-support` without printing
the endpoint. Capacity/tooling look suitable for heavy sweeps: ~32 GB RAM, 16 CPUs, ~537 GB free disk, `git`,
`python3`, `uv 0.11.17`, Node `v22.22.1`, npm `10.9.4`, `codex-cli 0.120.0`, Playwright modules, and private
art. The VM WorldOS checkout at `/root/worldos-qa/WorldOS` is clean but stale at `4524b3e`, 19 commits behind
current `main@4a0efe1`; Codex auth/config is not proven; `/Volumes/LEXAR/Codex` does not exist on the VM.
Before #466, approve/sync the VM checkout, prove Codex auth, set a remote staging path, and copy artifacts
back to local Lexar.
- RRI rollup rule: Mac/local evidence supplies native Part A and built-app screenshots; VM artifacts can supply
persona, behavior, image/network, palette-live, and score evidence only when `run.json`, `score.json`,
`session_surface.final.json`, `network.ndjson`, and build SHA are present. Missing or mixed-SHA artifacts
must remain `partial` / `harness_contaminated`.
- Split Mac/VM rollup command shape: pass the Mac proof into RRI as
`--handoff-json /Volumes/LEXAR/Codex/worldos-agent-grade-app-testability/<handoff>/handoff.json`
alongside the VM persona run dirs. RRI should then satisfy the native gate from the Mac handoff bundle
only if all required handoff gates and manifests are same-SHA, clean, private-art-present, and gap-free.

## Release (when RRI = 10/10 on a fresh .app build)
Bump `.claude-plugin/plugin.json` → 1.0.4, tag `v1.0.4`, GitHub release + CHANGELOG. Then MAINTAIN:
Expand Down
78 changes: 48 additions & 30 deletions WorldOS-OPERATING-GOAL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
Post-compaction agents: this 6-line block is ground truth. Do NOT reconstruct
state from scattered docs or old plans; trust this, verify the sha, then act.
──────────────────────────────────────────────────────────────────────────
AS OF: 2026-06-01T08:05:48+07:00 #479 trace-clean merged-main proof archived
AS OF: 2026-06-01T15:35:00+07:00 #504 handoff gate merged + post-merge fast gate passed
MAIN BASELINE:
f7ab6d7 (PRs #475, #494, #495, #496, #498, #499, #500 merged; verified
`/Users/lume/ClawDnD-val` was fast-forwarded after #500).
4a0efe1 (PRs #475, #494, #495, #496, #498, #499, #500, #501, #504
merged; verified `/Users/lume/ClawDnD-val` was fast-forwarded after #504).
Re-verify current `origin/main` before acting.
CANONICAL: /Users/lume/ClawDnD-val is now the synced local app/private-art checkout and
the default place to build/run/test the Mac app. Keep GUI/runtime tests on this
Expand All @@ -20,34 +20,41 @@
in local operator-only evidence/runbooks, not tracked repo docs. Use it for
heavy backend/persona sweeps only after Codex/config/credentials are intentionally
installed. Mac-built `.app` smoke/play proof stays on this Mac or macOS CI.
Current local preflight note: `ssh -G support-vm-1` resolves only to the alias;
`ssh -o BatchMode=yes support-vm-1 ...` could not resolve the hostname in this
Codex Desktop session. A read-only operator-endpoint scout reached `evaos-support`
(~32 GB RAM, 16 CPUs) with WorldOS at `/root/worldos-qa/WorldOS`, but that checkout
was stale (`4524b3e`, 19 commits behind `4a0efe1`) and Codex auth/config was not
proven. Restore/verify operator routing, fast-forward the VM repo, verify Codex auth,
and define artifact return before running #466 there.
LAST MEASURED GATE BUILD:
f5500ac produced qa/RRI.json = 2.7/10, but this is PARTIAL /
HARNESS-CONTAMINATED evidence: only newbie wrote score.json; the other
personas failed around port/backend harness setup; behavioral/UI/palette/image
evidence was not a valid five-persona release verdict.
LAST BUILT-APP PLAY PROOF:
Last real-provider built-app proof on merged main is `f7ab6d7`
(`codex-main-f7ab6d7-proof-20260601T010058Z`): private BG art loaded,
Codex provider minted the live session, Alfira seated, visible DM
narration, five enabled actions, and one `/move` resolved. It is
trace-clean merged-main diagnostic evidence: no app-evidence gaps,
provider trace summary records 0 parse errors and 0 failed/error tool
calls, chat roles reached `dm, player, dm`, `/session-surface` stayed
actionable, and native accessibility review showed one opening narration
row plus one post-move narration row, not duplicate chat/event prose.
Latest deterministic built-app smoke proof is PR #495 app-code
`698ed65` (`ui-hooks-app-proof-20260601T054934`): scripted provider,
private art, Abby seated, five enabled actions, writable `/move`, one
deterministic follow-up, and real image bytes. A current-session
`080497e` scripted harness proof exists at `scripted-smoke-20260531T233022Z-080497e`,
but ad-hoc harness ports are transient and are not built `.app` release proof.
Last merged-main handoff proof is `4a0efe1`
(`/Volumes/LEXAR/Codex/worldos-agent-grade-app-testability/handoff-20260601T081016Z-4a0efe1/`):
`qa/app_handoff_gate.py` scored `handoff_score=100` with web scripted smoke
5 moves, built `dist/WorldOS.app` scripted smoke 5 moves, and built
`dist/WorldOS.app` Codex-provider playtest 1 move. Private BG art was present,
Alfira was seated, visible narration and five enabled actions were present,
`/move` resolved, all three manifests had zero evidence gaps, and the Codex
provider trace reported `trace_exists=true`, `line_count=175`, and
`failed_or_error_count=0`.
Prior trace-clean real-provider built-app proof on merged main is `f7ab6d7`
(`codex-main-f7ab6d7-proof-20260601T010058Z`); it remains useful diagnostic
#479 evidence but is superseded as the latest merged-main app proof by the
`4a0efe1` handoff gate.
LAST VALID RELEASE GATE:
none after the RRI contract hardening. A release verdict requires expected
persona count, disk-backed palette/image/behavioral evidence, and built .app play.
NEXT ACTION: Close/update #479 against the merged-main proof, then run #466 for a trustworthy
clean RRI failure list/result: Mac/local or macOS CI supplies built
`.app` proof, while the 32GB support VM runs heavy backend/persona
sweeps after explicit VM preflight/auth setup. Continue #485/#486 for
NEXT ACTION: #479 is closed and #504 gives a 100/100 merged-main fast GUI velocity gate.
Do not claim release. Run #466 for a trustworthy clean RRI failure list/result:
Mac/local or macOS CI supplies built `.app` proof, while the 32GB support VM
runs heavy backend/persona sweeps after explicit VM routing/auth/config
preflight. If the VM route is still unavailable, record that as the blocker and
file/fix repo-side RRI harness gaps only if found. Continue #485/#486 for
evidence export and gate split follow-through; #481/#482/#483/#484 are closed.
Keep sprint work UX-first (#467): first-turn playability, clickability/chrome,
launcher clarity, live-response feel, and CRPG depth before more hardening/proxy/security work.
Expand Down Expand Up @@ -139,7 +146,8 @@ by an average). **RRI 10/10 = every gate holds on one fresh build:**
> The RRI output contract also records `required_release_personas`, `expected_personas`,
> `completed_personas`, `missing_personas`, `missing_release_personas`, `partial`,
> `harness_contaminated`, explicit `evidence_gaps`, image source/denominator, behavioral evidence path,
> UI-audit log, palette-live source, per-run Part B pass status, and per-run build SHA. Smoke-sized
> UI-audit log, palette-live source, optional `--handoff-json` Mac app proof, per-run Part B pass status,
> and per-run build SHA. Smoke-sized
> persona sets, mixed-build evidence, failed app-persona loops, missing persona scores, or missing
> artifact denominators can never silently produce a release-ready result.

Expand Down Expand Up @@ -188,7 +196,7 @@ verifier; can revert the goal to "fix" anytime.

---

## 9. CURRENT STATUS (2026-06-01T08:05:48+07:00 — #479 trace-clean merged-main proof archived)
## 9. CURRENT STATUS (2026-06-01T15:35:00+07:00 — #504 handoff gate passed post-merge)

- Repo truth stabilization merged in PR #465, UX-first doc sync merged in PR #468, first-minute
click/title chrome proof merged in PR #470, local/Lexar/support-VM routing merged in PR #471,
Expand All @@ -197,9 +205,10 @@ verifier; can revert the goal to "fix" anytime.
`/app-status`, evidence export baseline, and docs; PR #494 added the dev-gated scripted provider;
PR #495 added stable OpenWorlds accessibility / `data-worldos-testid` hooks; PR #496 added app
playtest failure buckets plus RRI gate-split metadata. PR #498 synced takeover docs after those
merges. PR #499 then recorded the current-main built-app proof, and PR #500 fixed the Codex-DM
provider trace cancellations. The local app/private-art checkout `/Users/lume/ClawDnD-val`
was fast-forwarded to `f7ab6d7 == origin/main` after #500.
merges. PR #499 then recorded the current-main built-app proof, PR #500 fixed the Codex-DM
provider trace cancellations, PR #501 recorded that proof in the runbooks/scorecard, and PR #504
added the hybrid 100/100 app handoff gate. The local app/private-art checkout
`/Users/lume/ClawDnD-val` was fast-forwarded to `4a0efe1 == origin/main` after #504.
- The stale local pre-sync artifacts were preserved before the fast-forward at
`/Volumes/LEXAR/Codex/worldos-local-checkout-snapshot-20260531T223923` and in `stash@{0}`
(`pre-sync local takeover docs 2026-05-31`). Treat those as evidence, not current release truth.
Expand All @@ -221,10 +230,14 @@ verifier; can revert the goal to "fix" anytime.
macOS TCC attribution contamination: `responsible=dev.clawdnd.app`, but the actual accessor was
`/usr/bin/find` launched by the test/diagnostic environment. Treat that screenshot prompt as harness
contamination unless a clean run shows `WorldOSApp`/WebKit itself accessing a protected library path.
- The next gate evidence step is issue #466: a clean non-partial five-persona RRI from `f7ab6d7` or newer.
- The next gate evidence step is issue #466: a clean non-partial five-persona RRI from `4a0efe1` or newer.
Heavy backend/persona sweeps belong on the owner-provided 32GB support VM (`support-vm-1`) once auth/config
are intentionally installed there; connection details are kept outside tracked docs. Mac-only built-app
launch/play proof stays on this Mac or macOS CI.
are intentionally installed there; connection details are kept outside tracked docs. In this Codex Desktop
session the local SSH alias for `support-vm-1` did not resolve; a read-only operator-endpoint scout reached
the VM and confirmed `evaos-support` has ~32 GB RAM, 16 CPUs, `git`, `python3`, `uv`, Node/npm, Codex CLI,
Playwright, and private art, but its WorldOS checkout is `4524b3e` (19 commits behind `4a0efe1`) and Codex
auth/config was not proven. Restore/verify VM routing, fast-forward the VM checkout, verify auth, and define
artifact return before the heavy sweep. Mac-only built-app launch/play proof stays on this Mac or macOS CI.
- Built-app diagnostic evidence exists, but release truth is still absent. The PR #475 pre-merge app-code
proof `8bd833f` (`codex-app-headproof-20260601T043909`) was trace-clean. The post-merge main proof
`32ca561` (`post475-main-app-proof-20260601T051230`) was playable with private art, Alfira, five enabled
Expand All @@ -245,6 +258,11 @@ verifier; can revert the goal to "fix" anytime.
screenshot archived, `app-evidence/manifest.json` with no gaps, and `provider-errors.after-move.json`
reporting zero parse errors plus zero failed/error tool calls. This is sufficient to close #479 as a
merged-main diagnostic; it is still not an RRI release verdict.
- The post-#504 merged-main handoff gate `handoff-20260601T081016Z-4a0efe1` then proved the fast GUI
velocity loop on the current `main`: web scripted smoke 5 moves, built-app scripted smoke 5 moves,
built-app Codex playtest 1 move, private art present, Alfira active, five enabled actions, zero evidence
gaps across all three manifests, and Codex trace `failed_or_error_count=0`. This is the current best
app-wiring proof, but it remains diagnostic and cannot replace the full five-persona RRI.
- The agent-grade testability layer now has real code merged: `GET /app-status` exposes the live run,
campaign, provider, private-art presence, move sink, actor, enabled actions, readiness, and failure buckets
without mutating state; the scripted provider can prove wiring behind a dev/test gate; and stable a11y/DOM
Expand Down
Loading
Loading