Skip to content

refactor(onboard): extract provider selection failure reporting#5176

Merged
cv merged 13 commits into
mainfrom
codex/onboard-provider-selection-failure-reporting
Jun 11, 2026
Merged

refactor(onboard): extract provider selection failure reporting#5176
cv merged 13 commits into
mainfrom
codex/onboard-provider-selection-failure-reporting

Conversation

@cv

@cv cv commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Extract the non-interactive provider-selection failure reporting from setupNim into a focused helper. This keeps the selection exit point in setupNim while giving the existing failure messages and Windows-host rejection delegation direct tests.

Related Issue

Part of #3802.

Changes

  • Added reportProviderSelectionFailure for provider recovery/request failure messages.
  • Replaced the inline setupNim failure switch with the new helper.
  • Added focused coverage for every provider-selection failure reason.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • Bug Fixes
    • Improved error reporting during provider selection in onboarding with clearer, scenario-specific error messages and guidance for different failure types, including Windows-host scenarios and provider unavailability issues.

cv added 7 commits June 10, 2026 12:06
@cv cv added refactor PR restructures code without intended behavior change area: cli Command line interface, flags, terminal UX, or output area: architecture Architecture, design debt, major refactors, or maintainability labels Jun 10, 2026
@cv cv self-assigned this Jun 10, 2026
@copy-pr-bot

copy-pr-bot Bot commented Jun 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9998740a-bb44-41e3-a74c-8003c203266e

📥 Commits

Reviewing files that changed from the base of the PR and between 8eeacff and 1910f4e.

📒 Files selected for processing (3)
  • src/lib/onboard.ts
  • src/lib/onboard/provider-selection-failure.test.ts
  • src/lib/onboard/provider-selection-failure.ts

📝 Walkthrough

Walkthrough

The PR extracts provider-selection failure reporting into a dedicated utility function with a typed input interface. The reportProviderSelectionFailure function replaces inline error-message branching in src/lib/onboard.ts, dispatching on failure reason kinds to emit tailored user guidance. Comprehensive tests verify all failure scenarios.

Changes

Provider Selection Failure Reporting

Layer / File(s) Summary
Failure reporting utility contract and implementation
src/lib/onboard/provider-selection-failure.ts
ReportProviderSelectionFailureInput interface and reportProviderSelectionFailure function switch on failure reason.kind to emit user-facing error messages for unavailable recorded Ollama, recorded providers, Windows-host Ollama, Hermes providers, and generic requested providers, with environment-specific hints and callback delegation for Windows-host Ollama rejection.
Integration into onboard provider selection
src/lib/onboard.ts
Imports reportProviderSelectionFailure and wires it into the non-interactive provider-selection failure path, passing the failure reason, Windows-host state, rejection callback, and console.error logger in place of the previous inline switch-based error branching.
Test coverage for failure reporting
src/lib/onboard/provider-selection-failure.test.ts
Vitest suite with a test helper that captures error and rejection outputs via callback overrides, then asserts exact error messages and rejection behavior for all failure scenarios: WSL Ollama unavailability, recorded-provider unavailability with Windows-host hints, unsupported Windows-host Ollama delegation, Hermes-provider unavailability, and generic requested-provider unavailability.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#5173: Main PR adds reportProviderSelectionFailure and wires src/lib/onboard.ts to use it when resolveRequestedProviderSelection() (from the extracted resolver in the retrieved PR) returns specific failure reason.kind values, so the two PRs directly connect at the shared failure-reason handling layer.

Poem

🐰 Errors once scattered, now gathered with care,
A utility born to dispatch them with flair,
Windows and Hermes, each case now divine,
Tests prove each path through the failure design. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely summarizes the main change: extracting provider selection failure reporting into a dedicated helper function from setupNim.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/onboard-provider-selection-failure-reporting

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 0 needs attention, 0 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 0 still apply, 0 new items found

Consider writing more tests for
  • **Runtime validation** — Non-interactive unavailable requested provider exits after helper reporting: validate setupNim emits the requested-provider unavailable stderr and stops through the existing exit path.. The new unit tests are sufficient for the extracted helper, but the changed call site is onboarding/provider-selection host glue where runtime integration coverage would improve confidence that stderr, rejection, and exit behavior remain wired correctly.
  • **Runtime validation** — Non-interactive unsupported Windows-host Ollama aborts instead of falling back: validate the resolver/helper/rejection integration delegates to abortNonInteractive and does not select another provider.. The new unit tests are sufficient for the extracted helper, but the changed call site is onboarding/provider-selection host glue where runtime integration coverage would improve confidence that stderr, rejection, and exit behavior remain wired correctly.
  • **Runtime validation** — Recorded provider recovery unavailable emits the Windows-host hint only when windowsHostKey is present in the full setup path.. The new unit tests are sufficient for the extracted helper, but the changed call site is onboarding/provider-selection host glue where runtime integration coverage would improve confidence that stderr, rejection, and exit behavior remain wired correctly.
  • **Runtime validation** — Recorded-provider-unavailable with windowsHostKey null omits the Windows-host hint in reportProviderSelectionFailure.. The new unit tests are sufficient for the extracted helper, but the changed call site is onboarding/provider-selection host glue where runtime integration coverage would improve confidence that stderr, rejection, and exit behavior remain wired correctly.
  • **Runtime validation** — Adding a new ProviderSelectionFailureReason variant fails compile-time or test-time exhaustiveness until reportProviderSelectionFailure handles it.. The new unit tests are sufficient for the extracted helper, but the changed call site is onboarding/provider-selection host glue where runtime integration coverage would improve confidence that stderr, rejection, and exit behavior remain wired correctly.
  • **Acceptance clause:** Part of Umbrella: refactor onboarding into a serializable FSM #3802. — add test evidence or identify existing coverage. The deterministic context did not include issue Umbrella: refactor onboarding into a serializable FSM #3802 body or comments and reported no linkedIssues, so no literal issue acceptance clauses were available to verify. The diff is consistent with a broad refactor slice: provider-selection failure reporting is extracted from src/lib/onboard.ts into src/lib/onboard/provider-selection-failure.ts with tests.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: onboard-negative-paths-vitest
Optional E2E: onboard-negative-paths-e2e, wsl-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • onboard-negative-paths-vitest (medium): Runs the built CLI through a real non-interactive onboard negative path and asserts clean non-zero behavior without JavaScript stack traces. This is the highest-signal PR-sized E2E for changes in onboard error handling.

Optional E2E

  • onboard-negative-paths-e2e (high): Broader shell E2E coverage for onboard negative and edge cases, including provider/model environment selection and clean credential validation failures. Useful because the PR changes the central onboard failure-reporting path, but it is longer-running and overlaps with the required Vitest negative-path job.
  • wsl-e2e (high): Optional confidence for WSL-adjacent behavior because the refactored helper includes the recorded WSL Ollama / Windows-host Ollama failure message path. Run only if the PR is suspected to affect WSL provider recovery beyond message extraction.

New E2E recommendations

  • onboarding / provider selection failure (high): Existing E2E coverage exercises invalid credentials and successful provider selection, but does not appear to directly run the real CLI with an unavailable NEMOCLAW_PROVIDER to hit reportProviderSelectionFailure's requested-provider-unavailable branch.
    • Suggested test: Add an onboard negative-path E2E that runs non-interactive onboard with NEMOCLAW_PROVIDER=missing-provider and asserts a non-zero exit, the friendly "Requested provider 'missing-provider' is not available" message, no stack trace, and no gateway/sandbox side effects.
  • WSL / Windows-host Ollama recorded-provider recovery (medium): The helper has WSL-specific recorded-provider failure branches that are unit-tested but not covered by an existing E2E scenario visible in the repository.
    • Suggested test: Add a WSL/provider-recovery negative E2E that seeds a recorded WSL Ollama provider state, runs non-interactive onboard in an environment where that provider is unavailable, and verifies the Windows-host Ollama guidance and fail-closed exit behavior.

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: ubuntu-repo-cloud-openclaw
Optional Vitest E2E scenarios: None

Dispatch required Vitest E2E scenarios:

  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: medium

Required Vitest E2E scenarios

  • ubuntu-repo-cloud-openclaw: Onboarding provider-selection failure handling was refactored into a new helper and wired into src/lib/onboard.ts. The live-supported Ubuntu cloud OpenClaw scenario is the smallest typed Vitest scenario that exercises the affected onboarding path and validates the CLI still builds and onboards successfully.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw

Optional Vitest E2E scenarios

  • None.

Relevant changed files

  • src/lib/onboard.ts
  • src/lib/onboard/provider-selection-failure.ts

@cv

cv commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

Validation update for this onboard slimming stack slice:

  • Scope: extracted provider-selection failure reporting into reportProviderSelectionFailure; setupNim still owns the final process.exit(1) control-flow decision.
  • Local validation passed: Biome format/lint on touched files, npm run build:cli, npm run typecheck:cli, focused Vitest provider/onboard suite, and git diff --check.
  • Standard PR CI is green and merge state is clean. PR Review Advisor reported 0 needs attention / 0 worth checking / 0 nice ideas; its runtime-validation suggestions are covered by the required live E2E runs below for this draft slice. CodeRabbit skipped because this PR is draft and has no actionable feedback.
  • Required live validation passed:

Leaving this as draft while the lower stack settles: #5165 ready, #5171 draft, #5173 draft, #5175 draft, #5176 draft.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 27306608753
Target ref: codex/onboard-provider-selection-failure-reporting
Requested jobs: onboard-negative-paths-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
onboard-negative-paths-e2e ✅ success

@cv cv added the v0.0.64 Release target label Jun 10, 2026
cv added 5 commits June 10, 2026 23:48
…nu-merge-update

# Conflicts:
#	test/e2e/test-onboard-inference-smoke.sh
…' into codex/tmp-provider-host-state-merge-update
…e-flow' into codex/tmp-provider-selection-merge-update
…-flow' into codex/tmp-provider-prompt-merge-update
…ow' into codex/tmp-provider-failure-reporting-merge-update
@cv cv added the area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow label Jun 11, 2026
Base automatically changed from codex/onboard-provider-prompt-flow to main June 11, 2026 09:07
# Conflicts:
#	src/lib/onboard.ts
#	src/lib/onboard/provider-host-state.test.ts
#	src/lib/onboard/provider-host-state.ts
@cv cv marked this pull request as ready for review June 11, 2026 09:14
@cv cv merged commit cf1e701 into main Jun 11, 2026
37 of 39 checks passed
@cv cv deleted the codex/onboard-provider-selection-failure-reporting branch June 11, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: architecture Architecture, design debt, major refactors, or maintainability area: cli Command line interface, flags, terminal UX, or output area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow refactor PR restructures code without intended behavior change v0.0.64 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant