Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion WorldOS-GUI-RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,10 @@ release truth still requires `qa/ui_playtest_app.sh` Part A+B and the full RRI s
- Target: owner-provided **32GB support VM** (`support-vm-1`); connection/auth details live in local
operator-only runbooks/evidence, not tracked repo docs.
- Do not assume it is ready for Codex runs until credentials/config are intentionally installed and verified.
The default support-VM persona lane is Codex DM plus Codex UI player; Claude is only required when
the preflight is run with `--provider claude` or `--player-agent claude`. The Codex lane requires
Codex CLI `>=0.120.0` because it uses per-invocation `codex exec -c mcp_servers.*` overrides rather
than mutating `CODEX_HOME` with `codex mcp add`.
- Use it for heavy backend/persona release sweeps and parallel QA once configured.
- Do **not** use it as proof for Mac-only surfaces: `WorldOS.app` build/launch, native #356, and built-app
UI play evidence stay on this Mac or macOS CI.
Expand All @@ -249,14 +253,18 @@ release truth still requires `qa/ui_playtest_app.sh` Part A+B and the full RRI s
python3 qa/support_vm_preflight.py \
--repo /root/worldos-qa/WorldOS \
--expected-sha 9545383 \
--provider codex \
--player-agent codex \
--art-root /root/worldos-qa/WorldOS \
--private-art-mode required \
--artifact-dir /tmp/worldos-support-vm-preflight-9545383 \
--artifact-return-target /Volumes/LEXAR/Codex/worldos-support-vm-rri/9545383-preflight
```
The script is read-only with respect to WorldOS state; it writes `support_vm_preflight.json` and
`support_vm_preflight.md`, redacts secrets, and exits non-zero if same-SHA/origin/tool/auth/private-art
blockers would make the RRI sweep untrustworthy.
blockers would make the RRI sweep untrustworthy. Its generated persona commands must include both
`WOS_APP_SELECTED_PROVIDER` and `WOS_APP_PLAYER_AGENT`; otherwise the VM sweep lane is not defined
tightly enough to count toward #466.
- Read-only VM scout (2026-06-01): an operator-only endpoint note can reach `evaos-support` without printing
the endpoint. Capacity/tooling look suitable for heavy sweeps: ~32 GB RAM, 16 CPUs, ~537 GB free disk, `git`,
`python3`, `uv 0.11.17`, Node `v22.22.1`, npm `10.9.4`, `codex-cli 0.120.0`, Playwright modules, and private
Expand Down
9 changes: 6 additions & 3 deletions WorldOS-OPERATING-GOAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
Post-compaction agents: this 6-line block is ground truth. Do NOT reconstruct
state from scattered docs or old plans; trust this, verify the sha, then act.
──────────────────────────────────────────────────────────────────────────
AS OF: 2026-06-01T17:49:00+07:00 support-VM origin-readiness pass; latest handoff build remains 9545383
AS OF: 2026-06-01T18:02:45+07:00 repo Codex persona-lane patch; latest handoff build remains 9545383
MAIN BASELINE:
Latest same-SHA app-proof target is `9545383` (PR #508 merged the repo-owned
support-VM preflight artifact gate, and the app handoff gate was rerun on that
Expand All @@ -21,7 +21,9 @@
SUPPORT VM: 32GB owner-provided support VM (`support-vm-1`). Connection/auth details live
in local operator-only evidence/runbooks, not tracked repo docs. Use it for
heavy backend/persona sweeps only after Codex/config/credentials are intentionally
installed. Mac-built `.app` smoke/play proof stays on this Mac or macOS CI.
installed. The default VM persona lane is Codex DM plus Codex UI player and requires
Codex CLI `>=0.120.0`; Claude is only required when deliberately selected. Mac-built
`.app` smoke/play proof stays on this Mac or macOS CI.
Current local preflight note: a read-only operator-endpoint scout reached
`evaos-support` (~32 GB RAM, 16 CPUs) with WorldOS at `/root/worldos-qa/WorldOS`,
but that checkout was stale (`4524b3e`) and behind the `9545383` app-proof baseline.
Expand Down Expand Up @@ -57,7 +59,8 @@
Mac handoff proof through `--handoff-json`.
Do not claim release. Run #466 for a trustworthy clean RRI failure list/result.
For same-SHA RRI, sync the support VM checkout to `9545383`, run
`python3 qa/support_vm_preflight.py --expected-sha 9545383 ...`, and pair
`python3 qa/support_vm_preflight.py --expected-sha 9545383 --provider codex --player-agent codex ...`,
and pair
the VM persona artifacts with the `9545383` Mac handoff JSON above. If a
newer release-candidate SHA is used instead, rerun the Mac handoff on that
same SHA before rollup. The 32GB support VM runs heavy backend/persona sweeps
Expand Down
5 changes: 4 additions & 1 deletion qa/QA_TOOLS.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,12 +49,15 @@ Stable app failure buckets are:
|---|---|---|---|
| `qa/release_readiness.py` | The Release Readiness Index rollup and only release verdict | Complete same-SHA app/persona evidence, including optional `--handoff-json` Mac proof | You only need fast GUI wiring confidence |
| `qa/release_gate.sh` | Orchestrate the release sweep over the canonical persona set | Built app, persona runs, behavior/UI/image/palette evidence | The support VM or Mac proof preflight is incomplete |
| `qa/support_vm_preflight.py` | Read-only readiness artifact before a support-VM persona sweep | VM identity, repo SHA, `origin/main` queryability, tool/auth/art status, return path, teardown plan | You are trying to fix or sync the VM; get operator approval first |
| `qa/support_vm_preflight.py` | Read-only readiness artifact before a support-VM persona sweep | VM identity, repo SHA, selected provider/player lane, `origin/main` queryability, tool/auth/art status, return path, teardown plan | You are trying to fix or sync the VM; get operator approval first |

RRI requires a non-partial five-persona result on one build SHA. A handoff gate can feed the native
app proof through `--handoff-json`, but it cannot fill in missing persona artifacts. The support-VM
preflight must pass before a VM sweep can count toward #466; if `origin/main` is not queryable from
the VM, fix the VM repo credentials/sync lane before running personas.
The default VM lane is Codex DM plus Codex UI player; Claude is only a readiness dependency when
`--provider claude` or `--player-agent claude` is selected. The Codex lane requires Codex CLI
`>=0.120.0` for per-invocation MCP server overrides.

## Browser And Persona Diagnostics

Expand Down
107 changes: 92 additions & 15 deletions qa/support_vm_preflight.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,14 @@
SCHEMA = "worldos.support-vm-preflight.v1"
CANONICAL_PERSONAS = ["newbie", "veteran", "adversarial", "narrative", "optimizer"]
MIN_SHA_MATCH_CHARS = 7
REQUIRED_TOOLS = [
MIN_CODEX_MCP_OVERRIDE_VERSION = (0, 120, 0)
BASE_REQUIRED_TOOLS = [
"git",
"python3",
"uv",
"node",
"npm",
"npx",
"codex",
"claude",
"jq",
"curl",
"lsof",
Expand All @@ -45,6 +44,8 @@
"pgrep",
"ps",
]
PERSONA_PROVIDERS = ("codex", "claude")
PLAYER_AGENTS = ("codex", "claude")
INTERESTING_ENV_PREFIXES = ("WORLDOS_", "CLAWDND_", "CODEX_", "OPENAI_", "ANTHROPIC_")
SAFE_PATH_ENV_NAMES = {
"WORLDOS_ART_REPO_ROOT",
Expand All @@ -71,6 +72,17 @@ class PreflightConfig:
budget: str
concurrency: int
port: int
provider: str = "codex"
player_agent: str = "codex"


def required_tools_for(config: PreflightConfig) -> list[str]:
tools = list(BASE_REQUIRED_TOOLS)
if config.provider == "codex" or config.player_agent == "codex":
tools.append("codex")
if config.provider == "claude" or config.player_agent == "claude":
tools.append("claude")
return tools


def utc_timestamp() -> str:
Expand Down Expand Up @@ -144,6 +156,18 @@ def has_auth_marker(text: str, markers: Sequence[str]) -> bool:
return any(re.search(rf"\b{re.escape(marker)}\b", text) for marker in markers)


def parse_semver(text: str) -> tuple[int, int, int] | None:
match = re.search(r"\b(\d+)\.(\d+)\.(\d+)\b", text or "")
if not match:
return None
return tuple(int(part) for part in match.groups())


def supports_codex_mcp_overrides(version_text: str) -> bool:
version = parse_semver(version_text)
return bool(version and version >= MIN_CODEX_MCP_OVERRIDE_VERSION)


def run_command(cmd: Sequence[str], cwd: Path | None = None, timeout: int = 8) -> dict:
try:
proc = subprocess.run(
Expand Down Expand Up @@ -350,7 +374,12 @@ def inspect_tool(
return info


def inspect_tools(repo: Path, runner: CommandRunner, which: WhichFn) -> tuple[dict, list[str], list[str]]:
def inspect_tools(
repo: Path,
runner: CommandRunner,
which: WhichFn,
required_tools: Sequence[str],
) -> tuple[dict, list[str], list[str]]:
blockers: list[str] = []
warnings: list[str] = []
tool_specs = {
Expand All @@ -371,7 +400,7 @@ def inspect_tools(repo: Path, runner: CommandRunner, which: WhichFn) -> tuple[di
"ps": ("ps", ["--version"]),
}
tools = {name: inspect_tool(name, exe, args, repo, runner, which) for name, (exe, args) in tool_specs.items()}
for name in REQUIRED_TOOLS:
for name in required_tools:
if not tools.get(name, {}).get("available"):
blockers.append(f"required VM tool missing: {name}")

Expand Down Expand Up @@ -421,9 +450,18 @@ def inspect_tools(repo: Path, runner: CommandRunner, which: WhichFn) -> tuple[di
blockers.append("Playwright Chromium executable is not installed; run (cd qa/playwright && npx playwright install chromium)")
tools["playwright_chromium"] = chromium

codex = {"available": bool(tools.get("codex", {}).get("available")), "auth_status": "not_proven"}
codex_required = "codex" in required_tools
codex = {"available": bool(tools.get("codex", {}).get("available")), "auth_status": "not_required"}
codex_path = tools.get("codex", {}).get("path")
codex_version = tools.get("codex", {}).get("version") or ""
codex["mcp_override_min_version"] = ".".join(str(part) for part in MIN_CODEX_MCP_OVERRIDE_VERSION)
codex["mcp_override_supported"] = supports_codex_mcp_overrides(codex_version) if codex_path else False
if codex_required and codex_path and not codex["mcp_override_supported"]:
blockers.append(
"Codex CLI version does not prove support for codex exec -c mcp_servers.* overrides; require >= 0.120.0"
)
if codex_path:
codex["auth_status"] = "not_proven"
result = runner([codex_path, "auth", "status"], repo, 10)
combined = f"{result.get('stdout') or ''}\n{result.get('stderr') or ''}".strip()
lower = combined.lower()
Expand All @@ -433,17 +471,27 @@ def inspect_tools(repo: Path, runner: CommandRunner, which: WhichFn) -> tuple[di
"auth_probe_timed_out": bool(result.get("timed_out")),
}
)
negative_auth = ("not authenticated", "unauthenticated", "not logged in", "signed out", "inactive")
negative_auth = (
"not authenticated",
"unauthenticated",
"not logged in",
"not signed in",
"not signed",
"signed out",
"inactive",
)
positive_auth = ("authenticated", "logged in", "signed in")
if has_auth_marker(lower, negative_auth):
codex["auth_status"] = "not_proven"
blockers.append("Codex CLI auth/profile status is not proven")
if codex_required:
blockers.append("Codex CLI auth/profile status is not proven")
elif result.get("ok") and has_auth_marker(lower, positive_auth):
codex["auth_status"] = "proven"
elif result.get("ok"):
codex["auth_status"] = "command_ok_unclassified"
blockers.append("Codex CLI auth/profile status is not proven")
else:
if codex_required:
blockers.append("Codex CLI auth/profile status is not proven")
elif codex_required:
blockers.append("Codex CLI auth/profile status is not proven")
tools["codex_auth"] = codex
return tools, blockers, warnings
Expand Down Expand Up @@ -481,7 +529,12 @@ def inspect_private_art(art_root: Path, mode: str) -> tuple[dict, list[str], lis
return info, blockers, warnings


def inspect_required_repo_files(repo: Path, personas: list[str]) -> tuple[dict, list[str], list[str]]:
def inspect_required_repo_files(
repo: Path,
personas: list[str],
provider: str,
_player_agent: str,
) -> tuple[dict, list[str], list[str]]:
blockers: list[str] = []
warnings: list[str] = []
required = [
Expand All @@ -492,9 +545,11 @@ def inspect_required_repo_files(repo: Path, personas: list[str]) -> tuple[dict,
"qa/release_readiness.py",
"qa/play_player_duo.txt",
"qa/playwright/palette_server.js",
"scripts/play.sh",
"scripts/play_party.sh",
]
if provider == "codex":
required.append("scripts/play_codex_dm.sh")
elif provider == "claude":
required.extend(["scripts/play.sh", "scripts/play_party.sh"])
required.extend(f"qa/play_player_browser_{persona}.txt" for persona in personas)
files = {}
for rel in required:
Expand Down Expand Up @@ -539,6 +594,8 @@ def build_vm_persona_commands(config: PreflightConfig) -> list[str]:
"WOS_APP_PART=B",
"WOS_APP_SKIP_BUILD=1",
"WOS_APP_NO_GLOBAL_KILL=1",
f"WOS_APP_SELECTED_PROVIDER={q(config.provider)}",
f"WOS_APP_PLAYER_AGENT={q(config.player_agent)}",
f"WOS_APP_PREFERRED_PORT={q(config.port)}",
"qa/ui_playtest_app.sh",
q(f"{run_prefix}-{persona}"),
Expand All @@ -562,11 +619,21 @@ def build_report(
blockers: list[str] = []
warnings: list[str] = []
config.artifact_dir.mkdir(parents=True, exist_ok=True)
if config.provider not in PERSONA_PROVIDERS:
blockers.append(f"unsupported support-VM persona provider: {config.provider}")
if config.player_agent not in PLAYER_AGENTS:
blockers.append(f"unsupported support-VM player agent: {config.player_agent}")

required_tools = required_tools_for(config)
repo, repo_blockers, repo_warnings = inspect_repo(config.repo, config.expected_sha, runner)
tools, tool_blockers, tool_warnings = inspect_tools(config.repo, runner, which)
tools, tool_blockers, tool_warnings = inspect_tools(config.repo, runner, which, required_tools)
art, art_blockers, art_warnings = inspect_private_art(config.art_root, config.private_art_mode)
repo_files, file_blockers, file_warnings = inspect_required_repo_files(config.repo, config.personas)
repo_files, file_blockers, file_warnings = inspect_required_repo_files(
config.repo,
config.personas,
config.provider,
config.player_agent,
)
blockers.extend(repo_blockers + tool_blockers + art_blockers + file_blockers)
warnings.extend(repo_warnings + tool_warnings + art_warnings + file_warnings)

Expand Down Expand Up @@ -599,6 +666,9 @@ def build_report(
"budget": config.budget,
"concurrency_cap": config.concurrency,
"port": config.port,
"provider": config.provider,
"player_agent": config.player_agent,
"required_tools": required_tools,
"same_sha_required": True,
"expected_sha": config.expected_sha,
"support_vm_scope": "backend/persona artifacts only; Mac built-app/native handoff evidence is supplied separately",
Expand Down Expand Up @@ -663,6 +733,9 @@ def markdown_report(report: dict) -> str:
f"- Personas: `{','.join(report['rri_plan']['expected_personas'])}`",
f"- Budget: `{report['rri_plan']['budget']}`",
f"- Port: `{report['rri_plan']['port']}`",
f"- Provider: `{report['rri_plan']['provider']}`",
f"- Player agent: `{report['rri_plan']['player_agent']}`",
f"- Required tools: `{','.join(report['rri_plan'].get('required_tools', []))}`",
f"- Support VM scope: `{report['rri_plan']['support_vm_scope']}`",
f"- Do not run on support VM: `{report['rri_plan']['do_not_run_on_support_vm']}`",
f"- First persona command: `{(report['rri_plan'].get('vm_persona_sweep_commands') or [''])[0]}`",
Expand Down Expand Up @@ -693,6 +766,8 @@ def parse_args(argv: Sequence[str]) -> argparse.Namespace:
parser.add_argument("--budget", default="12.00")
parser.add_argument("--concurrency", type=int, default=1)
parser.add_argument("--port", type=int, default=8785)
parser.add_argument("--provider", choices=PERSONA_PROVIDERS, default="codex")
parser.add_argument("--player-agent", choices=PLAYER_AGENTS, default="codex")
parser.add_argument("--no-fail", action="store_true", help="Write the report and exit 0 even if blockers exist")
return parser.parse_args(argv)

Expand All @@ -711,6 +786,8 @@ def main(argv: Sequence[str] | None = None) -> int:
budget=args.budget,
concurrency=args.concurrency,
port=args.port,
provider=args.provider,
player_agent=args.player_agent,
)
report = build_report(config)
json_path = config.artifact_dir / "support_vm_preflight.json"
Expand Down
16 changes: 16 additions & 0 deletions qa/test_macos_app_static.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ def test_built_app_playtest_can_keep_minted_backend_for_manual_gameplay(self):

self.assertIn("WOS_APP_KEEP_MINTED_BACKEND=1", harness)
self.assertIn("WOS_APP_SELECTED_PROVIDER=codex|scripted|claude|openclaw", harness)
self.assertIn("WOS_APP_PLAYER_AGENT=claude|codex", harness)
self.assertIn('PLAYER_AGENT="${WOS_APP_PLAYER_AGENT:-claude}"', harness)
self.assertIn('PART_B_PROVIDER="${SELECTED_PROVIDER:-claude}"', harness)
self.assertIn('defaults write dev.clawdnd.app selectedProvider "$SELECTED_PROVIDER"', harness)
self.assertIn("requires WOS_APP_PART=A", harness)
self.assertIn('KEEP_MINTED_BACKEND="${WOS_APP_KEEP_MINTED_BACKEND:-0}"', harness)
Expand All @@ -73,6 +76,19 @@ def test_built_app_playtest_can_keep_minted_backend_for_manual_gameplay(self):
self.assertNotIn('play_party.sh $WORLD $minted_run', harness)
self.assertNotIn('play.sh $WORLD $minted_run', harness)

def test_built_app_part_b_supports_codex_provider_and_player_agent(self):
harness = self.read("qa/ui_playtest_app.sh")

self.assertIn("scripts/play_codex_dm.sh", harness)
self.assertIn("CLAWDND_PROVIDER=codex", harness)
self.assertIn("codex exec", harness)
self.assertIn("codex_supports_mcp_override_config", harness)
self.assertIn("Codex CLI >= 0.120.0", harness)
self.assertIn("mcp_servers.clawdnd-uiplayer.command", harness)
self.assertIn("palette_server.js", harness)
self.assertIn("player_agent", harness)
self.assertIn("provider", harness)

def test_scripted_provider_is_dev_gated_and_model_free(self):
models = self.read("macos/WorldOSApp/Sources/WorldOSApp/Models/ProviderModels.swift")
providers = self.read("macos/WorldOSApp/Sources/WorldOSApp/Services/ProviderAdapters.swift")
Expand Down
Loading
Loading