test(e2e): add state validation phase fixture

[cv]

Summary

Adds a typed state-validation phase fixture that consumes the expected-state registry and calls existing host, gateway, and sandbox clients. This gives the Vitest E2E path typed probes for CLI installed, gateway healthy/absent, and sandbox running/absent.

Related Issue

Refs #4941
Refs #4990

Changes

• Adds StateValidationPhaseFixture and result types for expected-state probe execution.
• Exposes stateValidation on the shared Vitest E2E scenario context.
• Allows gateway and sandbox clients to accept shell probe run options so live checks can inherit the host env.
• Adds tests for positive ready state, preflight-negative absent state, failure behavior, optional expected-state dimensions, and context typing.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• npx prek run --all-files passes
• npm test passes
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

Additional verification after rebasing on current main:

• npx vitest run --project e2e-scenario-framework test/e2e-scenario/framework-tests/e2e-phase-state-validation.test.ts passes
• npx vitest run --project e2e-scenario-framework passes
• npx vitest run --project cli test/cli/list-share-live-inference.test.ts test/e2e-scenario/framework-tests/e2e-fixture-context.test.ts passes
• Pre-push checks pass

Note: full local pre-commit test-cli retries hit unrelated timing flakes in test/cli/list-share-live-inference.test.ts, test/cli/sandbox-mutations.test.ts, test/cli/snapshot-shields.test.ts, and test/e2e-scenario/framework-tests/e2e-fixture-context.test.ts; the affected files passed when rerun directly.

---

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

• Tests
  • Added comprehensive end-to-end tests validating gateway and sandbox client probe option preservation.
  • Introduced new state-validation test phase implementation with extensive scenario coverage, including gateway health verification, sandbox detection, and failure condition handling.
  • Extended test client APIs to support configurable probe execution options across gateway and sandbox operations.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 25af2211-c8ce-424b-80f3-54dc7c0cc72d

📥 Commits

Reviewing files that changed from the base of the PR and between 4286152 and 9e3f224.

📒 Files selected for processing (8)

• test/e2e-scenario/framework-tests/e2e-clients.test.ts
• test/e2e-scenario/framework-tests/e2e-phase-state-validation.test.ts
• test/e2e-scenario/framework/clients/gateway.ts
• test/e2e-scenario/framework/clients/sandbox.ts
• test/e2e-scenario/framework/e2e-test.ts
• test/e2e-scenario/framework/phases/index.ts
• test/e2e-scenario/framework/phases/onboarding.ts
• test/e2e-scenario/framework/phases/state-validation.ts

---

📝 Walkthrough

Walkthrough

This PR adds a new e2e state-validation phase to the test framework. GatewayClient and SandboxClient now accept ShellProbeRunOptions to configure probe execution. StateValidationPhaseFixture verifies CLI availability, gateway health, and sandbox status through multiple probe handlers, with support for provider-specific fallbacks and strict environment isolation.

Changes

E2E State Validation Phase

Layer / File(s)	Summary
Client method signatures and option forwarding test/e2e-scenario/framework/clients/gateway.ts, test/e2e-scenario/framework/clients/sandbox.ts, test/e2e-scenario/framework-tests/e2e-clients.test.ts	GatewayClient.status() and SandboxClient.list()/status()/expectRunning() accept optional ShellProbeRunOptions parameters and forward them to underlying nemoclaw/openshell commands. Tests verify options propagate correctly through the invocation chain.
State validation phase implementation test/e2e-scenario/framework/phases/state-validation.ts, test/e2e-scenario/framework/phases/onboarding.ts, test/e2e-scenario/framework/phases/index.ts	StateValidationPhaseFixture runs five typed probe handlers: cli-installed, gateway-healthy (with Ollama localhost fallback), gateway-absent, sandbox-running, and sandbox-absent. NemoClawInstance fields widened to unions to support hermes/openclaw agents, nvidia/ollama providers, and cloud/local environments. Types re-exported via phases module.
Fixture wiring and test integration test/e2e-scenario/framework/e2e-test.ts, test/e2e-scenario/framework-tests/e2e-phase-state-validation.test.ts	stateValidation fixture added to E2EScenarioFixtures and wired via Vitest base.extend. Comprehensive test suite with FakeRunner mock validates ready/absent states, fallback behavior, error cases, environment isolation, and substring-safe sandbox detection across 22 scenarios.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• NVIDIA/NemoClaw#4966: Updates e2e framework clients to accept and forward ShellProbeRunOptions when constructing nemoclaw/openshell shell commands, establishing the option-handling foundation for this PR.
• NVIDIA/NemoClaw#4965: Introduces ShellProbeRunOptions type definition and shell-probe framework that this PR consumes for configurable probe execution.

Suggested labels

area: e2e

Suggested reviewers

• jyaunches
• prekshivyas

Poem

🐰 A phase to validate, with probes in place,
From gateway health to sandbox's embrace,
Options now flow through each client call,
Mocking and testing, we've covered them all!
State-by-state, we verify right. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'test(e2e): add state validation phase fixture' directly and clearly describes the main change: introducing a new state validation phase fixture for e2e testing.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/e2e-4990-stack-03-state-validation-phase

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

PR Review Advisor

Findings: 0 needs attention, 0 worth checking, 0 nice ideas
Since last review: 1 prior item resolved, 0 still apply, 0 new items found

Consider writing more tests for

• **Acceptance clause:** Refs Adopt Vitest fixtures as the E2E scenario execution model #4941 — add test evidence or identify existing coverage. The PR body references Adopt Vitest fixtures as the E2E scenario execution model #4941, but the deterministic context did not include issue Adopt Vitest fixtures as the E2E scenario execution model #4941 body or comments, so literal issue acceptance clauses could not be extracted.
• **Acceptance clause:** Refs Adopt phase fixtures + registry-driven test discovery for Vitest E2E scenarios #4990 — add test evidence or identify existing coverage. The PR body references Adopt phase fixtures + registry-driven test discovery for Vitest E2E scenarios #4990, but the deterministic context did not include issue Adopt phase fixtures + registry-driven test discovery for Vitest E2E scenarios #4990 body or comments, so literal issue acceptance clauses could not be extracted.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

[github-actions]

E2E Advisor Recommendation

Required E2E: None
Optional E2E: e2e-vitest-scenarios:live-scenarios:ubuntu-repo-cli-smoke, ubuntu-no-docker-preflight-negative, ubuntu-repo-cloud-openclaw

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• None. No merge-blocking E2E is required: the PR is confined to E2E scenario framework/test code and does not change installer, onboarding implementation, sandbox lifecycle implementation, credentials handling, security policy, inference routing, deployment, or real assistant runtime behavior. Optional E2E runs are listed only for harness-integration confidence.

Optional E2E

• e2e-vitest-scenarios:live-scenarios:ubuntu-repo-cli-smoke (low): Optional smoke confidence that the Vitest E2E fixture extension still loads after adding the stateValidation fixture and changing client exports. It exercises the live Vitest scenario harness with a low-cost CLI boundary.
• ubuntu-no-docker-preflight-negative (medium): Optional scenario-level confidence for the adjacent negative side-effect contract that the new state-validation fixture mirrors: after an onboarding preflight failure, gateway and sandbox should remain absent.
• ubuntu-repo-cloud-openclaw (medium): Optional positive-path scenario confidence for the analogous ready-state contract: installed CLI, healthy gateway, and running sandbox after OpenClaw onboarding.

New E2E recommendations

• state-validation-harness (medium): No existing live Vitest E2E scenario appears to call StateValidationPhaseFixture.from() against a real onboarded NemoClaw instance; current coverage is framework/unit-level. If this fixture becomes the typed live scenario post-onboarding gate, add a focused live scenario that onboards a sandbox and invokes stateValidation.from('cloud-openclaw-ready', instance).
  • Suggested test: Add a live Vitest E2E scenario for StateValidationPhaseFixture against a real cloud OpenClaw onboarding result.

[github-actions]

E2E Scenario Advisor Recommendation

Required scenario E2E: e2e-scenarios-all
Optional scenario E2E: None

Dispatch required scenario E2E:

• gh workflow run e2e-scenarios-all.yaml --ref <pr-head-ref>

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

• e2e-scenarios-all: Changes modify shared scenario framework fixtures, clients, onboarding types, and add the state-validation phase used by scenario execution/state checks. Shared scenario framework changes can affect all scenario IDs, so run the full scenario fan-out.
  • Dispatch: gh workflow run e2e-scenarios-all.yaml --ref <pr-head-ref>

Optional scenario E2E

• None.

Relevant changed files

• test/e2e-scenario/framework-tests/e2e-clients.test.ts
• test/e2e-scenario/framework-tests/e2e-phase-state-validation.test.ts
• test/e2e-scenario/framework/clients/gateway.ts
• test/e2e-scenario/framework/clients/sandbox.ts
• test/e2e-scenario/framework/e2e-test.ts
• test/e2e-scenario/framework/phases/index.ts
• test/e2e-scenario/framework/phases/onboarding.ts
• test/e2e-scenario/framework/phases/state-validation.ts