Skip to content

refactor(messaging): migrate credential providers to manifest plans#4937

Draft
sandl99 wants to merge 45 commits into
mainfrom
refactor/phase-4b-credential-provider-plan
Draft

refactor(messaging): migrate credential providers to manifest plans#4937
sandl99 wants to merge 45 commits into
mainfrom
refactor/phase-4b-credential-provider-plan

Conversation

@sandl99

@sandl99 sandl99 commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Stacked on #4908, this PR migrates messaging credential/provider setup from legacy token definitions to manifest-backed SandboxMessagingPlan.credentialBindings. It keeps provider names, env keys, placeholders, and credential hashes plan-driven so create, reuse, rebuild, add, remove, rollback, and rotation checks all consume the compiled plan.

Related Issue

Closes #4393. Part of #3896

Changes

  • Thread credential hash side-channel data through the manifest compiler and workflow planner without storing raw secrets in plans.
  • Add plan-based OpenShell provider application support for create/update/reuse/delete-recreate and external credential resolution.
  • Update onboard create/reuse/rebuild to derive active channels, provider attachments, missing-provider migration, and rotation checks from the compiled messaging plan.
  • Remove the rebuild-time buildMessagingPlanForSandbox fallback; createSandbox now reads the setup-staged env plan or the persisted registry plan only.
  • Update channels add/remove and rollback to use plan credential bindings and provider names instead of legacy channel token metadata.
  • Remove the legacy token-definition rotation path, stale env-key-to-channel mapper, and synthesized pre-plan provider-name fallback.
  • Rename the remaining generic provider batch helper and extra-placeholder provider shapes so Brave/extra placeholders are not framed as the old messaging token path.
  • Add focused coverage for provider reuse/replacement, plan-based rotation, channel conflict tests, channel add rollback, and onboarding provider behavior.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

Focused checks run locally:

  • npm run build:cli passed

  • npx vitest run --project cli src/lib/onboard/providers.test.ts src/lib/onboard/extra-placeholder-keys.test.ts src/lib/messaging/applier/setup-applier.test.ts src/lib/onboard/messaging-channel-setup.test.ts src/lib/actions/sandbox/policy-channel-conflict.test.ts test/credential-rotation.test.ts test/channels-add-preset.test.ts test/channels-remove-full-teardown.test.ts test/onboard-messaging.test.ts passed: 154 tests

  • npm run typecheck:cli passed

  • npm run test-size:check passed

  • npx prek run --files ... did not pass because the broad Test (CLI) hook still fails in unrelated timeout/assertion cases outside these changed files (test/nemoclaw-start.test.ts, test/policies.test.ts, test/secret-redaction.test.ts, test/cli/connect-recovery.test.ts, test/cli/doctor-gateway-token.test.ts)

  • npx prek run --all-files passes

  • npm test passes

  • Tests added or updated for new or changed behavior

  • No secrets, API keys, or credentials committed

  • Docs updated for user-facing behavior changes

  • npm run docs builds without warnings (doc changes only)

  • Doc pages follow the style guide (doc changes only)

  • New doc pages include SPDX header and frontmatter (new pages only)


Signed-off-by: San Dang sdang@nvidia.com

sandl99 and others added 29 commits June 7, 2026 11:49
Stores the compiled SandboxMessagingPlan in the onboard session so that
resume runs can restore the plan to env without re-running enrollment
hooks (token paste, QR pairing). Fixes the gap where the registry entry
lost its `messaging` field on rebuild because the plan was only held in
a process env var that didn't survive across process restarts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…el-setup

Exports readMessagingPlanFromEnv and writePlanToEnv from
messaging-channel-setup.ts (which already owns MessagingSetupApplier)
to keep src/lib/onboard.ts from growing. Collapses the one-name
messaging-channel-setup require into a single line to free headroom.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…an persistence

- Extract parseSandboxMessagingPlan to messaging-plan-session.ts to
  keep onboard-session.ts growth under the monolith threshold
- Guard plan restoration with sandbox-name + agent identity check so
  stale plans from renamed sandboxes or agent switches are not reused
- Add three behavior assertions in sandbox.test.ts: fresh setup
  persists env plan to session; matching plan is restored to env on
  non-interactive resume; mismatched sandbox name skips restoration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tive resume

rebuild.ts stages an authoritative plan from the registry (which reflects
post-stop/-start channel mutations) before calling onboard --resume.
Previously, the session plan restoration was unconditionally overwriting
that staged plan, causing stopped channels to reappear as active after
rebuild.

Now the handler checks the env first: if a plan is already staged
(rebuild path), it is used as-is. The session plan is only restored when
the env is empty, covering the plain process-restart resume case this PR
was originally targeting.

Also adds a test asserting the rebuild-path preference.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- On non-interactive resume, restore plan from registry (always current
  after stop/start/add/remove) instead of stale session snapshot;
  env-first priority preserved so rebuild.ts staging still wins

- In rebuild.ts, persist the staged plan to the session alongside
  messagingChannels/disabledChannels/messagingChannelConfig so the
  session is fully consistent during the rebuild window

- Add getChannelsFromPlan, getDisabledChannelsFromPlan, and
  getMessagingChannelConfigFromPlan helpers in messaging-plan-session.ts
  so the next PR can replace the three individual session fields with
  plan-derived reads

- Move MessagingHostStateApplier re-export to messaging-channel-setup
  and getRegistrySandboxMessagingPlan helper to keep onboard.ts net-neutral

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…itecture (phase 4a)

- Add src/lib/messaging/applier/conflict-detection.ts with all core logic:
  ConflictRegistryEntry minimal interface, createMessagingConflictProbe factory
  (consolidates the duplicated tri-state probe pattern from onboard.ts,
  policy-channel.ts, and status-command-deps.ts), plan-to-request helpers
  (getActiveChannelIdsFromPlan, getCredentialHashesFromPlan,
  planToConflictChannelRequests), plan-preferred/legacy-fallback entry
  resolution (resolveActiveChannelsFromEntry, resolveCredentialHashesFromEntry),
  and pure detection functions (findConflictsInEntries, detectAllOverlapsInEntries,
  backfillLegacyEntryChannels)

- Rewrite messaging-conflict.ts as a thin public adapter: no detection logic
  inside, all three public exports delegate to applier functions, re-exports
  createMessagingConflictProbe so callers don't need to import from applier
  directly; removes getChannelDef/getChannelTokenKeys import (stored hashes
  are self-describing — no channel-constant lookup needed)

- Update onboard.ts conflict-check block: remove makeConflictProbe() and the
  MESSAGING_CHANNELS/hashCredential block; use createMessagingConflictProbe
  with injected openshell deps and findChannelConflictsFromPlan from plan

- Add plan-path tests: planToConflictChannelRequests grouping and exclusions,
  findChannelConflictsFromPlan matching/disabled/no-hash cases, plan-only
  registry entries in findChannelConflicts and findAllOverlaps

Closes #4392

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Remove redundant createMessagingConflictProbe named import from
  messaging-conflict.ts (symbol is already re-exported on the line below;
  flagged by CodeQL unused-import rule)
- Fix checkGatewayLiveness in onboard.ts to use runOpenshell exit status
  instead of stdout length; a healthy gateway with no sandboxes returns
  empty output with status 0, which the previous check incorrectly treated
  as "down"

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Fix credentialHash gate: gate on credentialAvailable (set by the compiler
  for all real onboarding flows) instead of credentialHash (not yet populated),
  which previously caused the conflict check to be silently skipped for all
  manifest-plan onboarding; planToConflictChannelRequests now includes channels
  with credentialAvailable=true and no hash, falling through to unknown-token
  conservative detection
- Fix cross-channel hash contamination: getCredentialHashesFromPlan now accepts
  an optional channelId filter; conflictReasonForRequest and
  conflictReasonForPair use the new resolveChannelHashesFromEntry helper so
  Slack or Discord hashes cannot produce false positive unknown-token results
  when checking a Telegram request
- Fix legacy hash fallback: when a plan-backed entry exists but carries no
  credentialHash for the requested channel (migration in flight), fall back to
  providerCredentialHashes so matching-token detection is not silently lost
- Align active-channel semantics with plan-filter.ts: getActiveChannelIdsFromPlan
  now also checks channel.active && !channel.disabled, not only disabledChannels
- Split plan-driven tests into src/lib/messaging/applier/conflict-detection.test.ts
  to offset the messaging-conflict.test.ts monolith growth; adds WhatsApp no-op
  test, cross-channel isolation, credentialAvailable-gate, and legacy-fallback cases

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…e and populate credentialHash in plan bindings

Remove hardcoded PROVIDER_SUFFIXES/KNOWN_CHANNEL_IDS from conflict-detection.ts; backfillLegacyEntryChannels now receives the suffix map as an injected parameter. The adapter layer (messaging-conflict.ts) derives it from BUILT_IN_CHANNEL_MANIFESTS credentials so adding a channel to the manifest registry propagates automatically.

Also fix credential-binding-engine to hash the env var value into credentialHash when credentialAvailable is true, enabling matching-token conflict detection on the plan-driven path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove legacy providerCredentialHashes fallback from resolveChannelHashesFromEntry — entries without a plan now return {} and fall through to unknown-token (conservative). Fix PROVIDER_SUFFIXES to collect all credential suffixes per manifest rather than only credentials[0], so multi-credential channels like Slack probe all their provider names during backfill (channel active if any present). Split 473-line conflict-detection.test.ts monolith into conflict-detection-plan.test.ts (plan helpers) and conflict-detection-entry.test.ts (detection functions) to stay under the monolith growth threshold.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…omments

Reject the env-staged plan when its sandboxName does not match the current
sandbox being created, preventing a stale plan from a prior run from gating
or bypassing conflict detection for a different sandbox.

Update stale comments that described the credential-binding engine as not yet
emitting credentialHash — it does now. Rephrase to reflect the remaining
no-hash case (registry-only resume without a compiler re-run).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… test fixtures

Restore the providerCredentialHashes fallback in resolveChannelHashesFromEntry
for entries whose plan carries no channel hashes (e.g. registry-only resume
without a compiler re-run), preserving safety coverage during migration.

Fix tgChannel(false, true) → tgChannel(true, true) in disabled-channel test
cases so assertions isolate plan.disabledChannels as the only gate rather than
passing via the inactive-channel path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dings with active channels

Address PR Review Advisor findings on phase-4a conflict detection:

- resolveChannelHashesFromEntry now filters legacy providerCredentialHashes to
  only the providerEnvKey values declared for the requested channel in
  BUILT_IN_CHANNEL_MANIFESTS, preventing unrelated Slack hashes from producing
  false Telegram conflicts and vice versa. Unknown channels retain the full map.

- planToConflictChannelRequests now intersects credentialBindings with
  getActiveChannelIdsFromPlan so stale bindings for inactive or absent channels
  cannot generate false conflict requests.

- Fix misleading JSDoc on findChannelConflictsFromPlan: available bindings with
  no credentialHash are included (not excluded) and fall through to conservative
  unknown-token detection.

- Add regression tests: cross-channel legacy hash scoping, active-channel binding
  intersection (absent channel, active=false), and slackChannel fixture for
  plan test file.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…rison and add no-plan fallback

Address second round of PR Review Advisor findings:

- conflictReasonForRequest and conflictReasonForPair now use manifest-declared
  CHANNEL_CREDENTIAL_ENV_KEYS as the primary comparison set. For multi-credential
  channels like Slack (SLACK_BOT_TOKEN + SLACK_APP_TOKEN), a differing bot token
  with a missing app token previously returned null; it now returns unknown-token
  because the missing manifest key marks the comparison as incomplete.

- onboard.ts conflict gate: when hasPlanCredentials is false but token-backed
  channels were selected (enabledChannels contains non-QR channels), fall back to
  findChannelConflicts with just the channel names to preserve conservative
  unknown-token warnings even when the compiled plan has no credential data
  (e.g. credential-store-backed availability or stale env plan).

- policy-channel.ts: add explicit comment scoping the add-channel path as a
  known limitation — no compiled plan is available at channels-add time, so
  findChannelConflicts is used with caller-built hashes; plan-driven migration
  is follow-up work.

- Tests: four Slack partial-hash suppression cases covering the false-negative
  regression (conflictReasonForRequest with missing app token, both differ,
  conflictReasonForPair with missing app token from both sides, both differ).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…nifest-driven hashing

Address third round of PR Review Advisor findings:

- messaging-conflict.ts: add findChannelConflictsForOnboarding() that unions
  the plan-based hash-precise check with a channel-name-only fallback for any
  selectedTokenChannels not covered by the plan's credentialAvailable bindings.
  This ensures a partial or stale same-sandbox plan cannot silently skip channels
  it omits. onboard.ts now delegates to this single function rather than
  inlining the union logic.

- onboard.ts conflict gate: replace the multi-branch plan/fallback block with a
  single call to findChannelConflictsForOnboarding(sandboxName, currentPlan,
  selectedTokenChannels, registry), keeping all conflict logic in messaging-conflict.ts.

- policy-channel.ts checkChannelAddConflict: replace raw acquired-map iteration
  with manifest-driven hash building via createBuiltInChannelManifestRegistry().
  Iterates channelManifest.credentials[].providerEnvKey to build credentialHashes,
  mirroring planToConflictChannelRequests without requiring a compiled plan.
  QR-only channels (WhatsApp: credentials=[]) exit early with no conflict possible.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…tection

Legacy entries no longer carry hash metadata into conflict comparisons.
resolveChannelHashesFromEntry returns {} for entries without a plan, which
falls through to conservative unknown-token detection in all callers.

- ConflictRegistryEntry: drop providerCredentialHashes field
- resolveChannelHashesFromEntry: plan path only; legacy entries → {}
- CHANNEL_CREDENTIAL_ENV_KEYS comment updated (now solely for manifest-key
  comparison set, not legacy scoping)
- Section comment: "plan-preferred, legacy-fallback" → "Entry resolution"

- Remove findChannelConflictsForOnboarding and fallback logic; onboard.ts
  calls findChannelConflictsFromPlan directly (rationale in issue #4392)

Test updates:
- conflict-detection-entry.test.ts: remove legacy cross-channel hash scoping
  describe block; migrate Slack pair tests from providerCredentialHashes
  fixtures to plan-backed entries
- messaging-conflict.test.ts: remove hash-comparison tests (covered by
  plan-entry tests); update remaining fixtures to drop providerCredentialHashes;
  update file header comment

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…aths

Completes the providerCredentialHashes deprecation started in the phase
4a conflict-detection migration. The field was written by onboard.ts,
policy-channel.ts, and rebuild.ts, and read by messaging-credentials.ts
and workflow-planner.ts for rotation detection and credential-availability
inference.

All write paths are removed. Reads are replaced with plan-backed
equivalents: detectMessagingCredentialRotation now reads
credentialBindings.credentialHash from the compiled SandboxMessagingPlan,
and credentialAvailabilityFromSandboxEntry reads
plan.credentialBindings[].credentialAvailable. The field is removed from
SandboxEntry and MessagingWorkflowPlannerSandboxEntry entirely; existing
sandboxes.json files with stale providerCredentialHashes entries are
unaffected at runtime (unknown JSON fields are ignored).

Tests updated to use plan-backed fixtures for matching-token conflict
scenarios; credential-rotation tests use messaging.plan.credentialBindings
fixtures.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…-messaging

Both files shrank after removing providerCredentialHashes fixtures and
assertions in the preceding refactor commit.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Take nemoclaw-start budget reduction from main (5310→5300) and keep
onboard-messaging reduction from this branch (2122→2110).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@sandl99 sandl99 self-assigned this Jun 8, 2026
@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 56d2a4c1-6034-4b09-ab00-9606d0383aea

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/phase-4b-credential-provider-plan

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: messaging-providers-e2e, channels-add-remove-e2e, channels-stop-start-e2e, token-rotation-e2e, cloud-onboard-e2e, network-policy-e2e
Optional E2E: credential-migration-e2e, rebuild-openclaw-e2e, ubuntu-repo-cloud-openclaw-token-rotation

Dispatch hint: messaging-providers-e2e,channels-add-remove-e2e,channels-stop-start-e2e,token-rotation-e2e,cloud-onboard-e2e,network-policy-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/feat/phase-4a-messaging-conflict-plan-driven
Head: HEAD
Confidence: high

Required E2E

  • messaging-providers-e2e (high): Required because the PR changes manifest/plan-driven credential bindings and OpenShell provider application. This E2E validates Telegram, Discord, Slack, WeChat, and WhatsApp provider/placeholder/L7-proxy behavior, credential isolation, config patching, and no-provider QR-only parity.
  • channels-add-remove-e2e (high): Required because policy-channel.ts, provider-name resolution for remove, plan validation, and channel preset tests are changed. This is the closest real user-flow coverage for channels add, channels remove, rebuild after add/remove, policy preset application, and gateway credential reuse.
  • channels-stop-start-e2e (high): Required because the plan validation and channel state changes include disabled-channel semantics and provider reattachment/reuse behavior. This E2E exercises stop/start lifecycle across Telegram, Discord, WeChat, Slack, and WhatsApp for OpenClaw and Hermes.
  • token-rotation-e2e (medium): Required because credential binding, provider update/create logic, onboarding credential reuse, and rotation tests are changed. This E2E verifies rotating Telegram, Discord, and Slack tokens propagates to OpenShell providers and only rebuilds the affected bridge.
  • cloud-onboard-e2e (high): Required because onboarding, sandbox machine handlers, messaging channel setup, provider reuse, and deployment verification are touched. This validates the install/onboard/deployment health path with a real sandbox and policy presets.
  • network-policy-e2e (high): Required because channel add/remove policy handling and policy-channel behavior changed. This validates deny-by-default, live policy add, dry-run, hot reload, inference exemption, SSRF validation, and approved egress behavior in a real sandbox.

Optional E2E

  • credential-migration-e2e (medium): Optional confidence for credential-sensitive paths because this PR changes provider application/reuse and placeholder handling, but it does not primarily modify legacy credential migration code.
  • rebuild-openclaw-e2e (high): Optional confidence for plan persistence across rebuild because messaging plans are stored in registry state and consumed by sandbox rebuild/apply paths; required coverage is already provided by channel lifecycle jobs.
  • ubuntu-repo-cloud-openclaw-token-rotation (high): Optional scenario-runner confidence for the same token-rotation user flow through the typed scenario framework, complementary to the nightly shell E2E.

New E2E recommendations

  • plan-backed channel removal security (high): Existing channel add/remove coverage exercises normal lifecycle, but the PR adds fail-closed validation of persisted messaging plans before exposing provider names for cleanup. Add a real E2E that tampers a stored plan providerName, runs channels remove, verifies the command refuses unsafe cleanup, and verifies no unrelated OpenShell provider is deleted.
    • Suggested test: Add a channel-remove stored-plan tamper E2E covering provider-name validation and safe failure before provider deletion.
  • plan-driven provider cleanup parity (medium): The new provider-name resolver has legacy registry fallback and plan-backed provider lookup paths. A targeted E2E should prove both legacy rows and plan-backed rows remove all expected OpenShell providers for multi-provider channels such as Slack.
    • Suggested test: Add an E2E that creates/removes Slack from both legacy-style and plan-backed sandbox registry states and asserts both bridge/app providers are fully removed from OpenShell.

Dispatch hint

  • Workflow: nightly-e2e.yaml
  • jobs input: messaging-providers-e2e,channels-add-remove-e2e,channels-stop-start-e2e,token-rotation-e2e,cloud-onboard-e2e,network-policy-e2e

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

E2E Scenario Advisor Recommendation

Required scenario E2E: ubuntu-repo-cloud-openclaw-telegram, ubuntu-repo-cloud-openclaw-slack, ubuntu-repo-cloud-openclaw-token-rotation
Optional scenario E2E: ubuntu-repo-cloud-openclaw-discord, ubuntu-repo-cloud-hermes-slack, ubuntu-repo-cloud-hermes-discord

Dispatch required scenario E2E:

  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-telegram
  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-slack
  • gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-token-rotation

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/feat/phase-4a-messaging-conflict-plan-driven
Head: HEAD
Confidence: high

Required scenario E2E

  • ubuntu-repo-cloud-openclaw-telegram: Primary targeted scenario for the changed messaging onboarding, workflow planner, credential binding, OpenShell provider application, and channel lifecycle paths. It exercises provider attachment, placeholder configuration, no-secret-leak checks, bridge reachability, and Telegram-specific messaging safety on the default OpenClaw onboarding path.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-telegram
  • ubuntu-repo-cloud-openclaw-slack: Required because the PR changes provider-name resolution, plan validation, credential bindings, and OpenShell provider create/update behavior; Slack is the smallest routed scenario that covers the multi-credential/multi-provider messaging surface.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-slack
  • ubuntu-repo-cloud-openclaw-token-rotation: Required for the credential rotation and provider rewrite surface touched by messaging applier, provider bindings, onboarding credential reuse, and channel add/remove cleanup changes.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-token-rotation

Optional scenario E2E

  • ubuntu-repo-cloud-openclaw-discord: Optional adjacent coverage for the same common messaging suite through the Discord manifest; useful if maintainers want broader channel-manifest confidence beyond Telegram and Slack.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-openclaw-discord
  • ubuntu-repo-cloud-hermes-slack: Optional adjacent coverage for the same Slack multi-provider messaging path on the Hermes onboarding profile rather than OpenClaw.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-hermes-slack
  • ubuntu-repo-cloud-hermes-discord: Optional adjacent coverage for the shared messaging applier/planner path on Hermes with a different channel manifest and agent profile.
    • Dispatch: gh workflow run e2e-scenarios.yaml --ref <pr-head-ref> --field scenarios=ubuntu-repo-cloud-hermes-discord

Relevant changed files

  • src/lib/actions/sandbox/channel-remove-provider-names.ts
  • src/lib/actions/sandbox/openshell-output.ts
  • src/lib/actions/sandbox/policy-channel.ts
  • src/lib/messaging/applier/index.ts
  • src/lib/messaging/applier/openshell-provider.ts
  • src/lib/messaging/applier/plan-validation.ts
  • src/lib/messaging/applier/types.ts
  • src/lib/messaging/compiler/engines/credential-binding-engine.ts
  • src/lib/messaging/compiler/types.ts
  • src/lib/messaging/compiler/workflow-planner.ts
  • src/lib/messaging/index.ts
  • src/lib/messaging/provider-bindings.ts
  • src/lib/onboard.ts
  • src/lib/onboard/extra-placeholder-keys.ts
  • src/lib/onboard/machine/handlers/sandbox.ts
  • src/lib/onboard/messaging-channel-setup.ts
  • src/lib/onboard/messaging-credentials.ts
  • src/lib/onboard/messaging-reuse.ts
  • src/lib/onboard/providers.ts
  • src/lib/verify-deployment.ts

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor

Findings: 2 needs attention, 8 worth checking, 0 nice ideas
Since last review: 3 prior items resolved, 5 still apply, 1 new item found

Review findings

🛠️ Needs attention

  • Plan-backed provider lifecycle parity is still only partially pinned (src/lib/actions/sandbox/policy-channel.ts:1474): Issue Phase 4b: migrate messaging credential/provider binding to manifest plans #4393 requires provider upsert, attach, detach, delete, and create/rebuild attachment behavior to match current externally visible behavior. The PR now covers useful pieces, including provider get/create/update/delete-recreate shapes, plan tampering rejection, and pre-plan Telegram/Slack remove cleanup. I still could not find direct coverage for valid stored-plan remove across the main provider shapes, NotFound/not-attached success-equivalent cleanup, rollback cleanup parity, WeChat cleanup, or rebuild attachment parity from a validated stored plan.
    • Recommendation: Add focused tests for valid stored-plan remove and rollback detach/delete argv for Telegram, Discord, Slack bot/app, and WeChat. Include OpenShell NotFound/not-attached detach/delete outputs and a create/rebuild path assertion that a validated stored plan attaches the same provider names as initial create.
    • Evidence: setup-applier.test.ts asserts applyCredentialsAtOpenShell command shapes and sandboxCreateProviderArgs; channels-remove-full-teardown.test.ts covers pre-plan Telegram/Slack detach/delete and tampered-plan rejection. Comparable valid stored-plan remove, WeChat cleanup, NotFound/not-attached, rollback, and rebuild attachment parity coverage was not evident.
  • Large high-risk lifecycle validation code grew as a new monolith (src/lib/messaging/applier/plan-validation.ts:1): This PR adds a large new validator and continues expanding existing oversized messaging lifecycle/test files. The changed code controls stored-plan trust, OpenShell provider names, credential hashes, network policy entries, hooks, render plans, and registry source-of-truth binding. Keeping that much security-critical validation in one large module makes review and future hardening harder.
    • Recommendation: Split plan validation into smaller focused modules, such as channel state validation, credential binding validation, policy validation, and generated-entry validation. Avoid further expanding policy-channel.ts and oversized provider lifecycle tests; extract reusable provider cleanup and stored-plan source-of-truth helpers before adding more behavior.
    • Evidence: Deterministic size analysis flags new src/lib/messaging/applier/plan-validation.ts at about 499 lines, src/lib/actions/sandbox/policy-channel.ts growing further, and existing large workflow/setup tests growing substantially.

🔎 Worth checking

  • Source-of-truth review needed: Legacy provider-name compatibility fallback: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: getManifestProviderNamesForChannel() is documented as a compatibility helper and resolveChannelProviderNamesForRemove() falls back to it when no plan exists.
  • Source-of-truth review needed: Channel add conflict backfill recovery: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: checkChannelAddConflict() catches and ignores backfillMessagingChannels() errors before findChannelConflicts().
  • Source-of-truth review needed: Best-effort OpenShell provider cleanup/replacement: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: openshell-provider.ts accumulates failures under options.bestEffort and throws an aggregate error after iterating.
  • Source-of-truth review needed: Ambient process.env credential-hash fallback: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: resolveCredentialHash() falls back from context.credentialHashes to hashCredential(process.env[envKey]).
  • Credential hash planning can fall back to ambient process.env (src/lib/messaging/compiler/engines/credential-binding-engine.ts:56): The credential binding compiler now accepts credentialHashes in context, but resolveCredentialHash() falls back to hashing process.env[envKey] when context hashes are absent. That does not store raw secrets, but it creates hidden host-environment coupling in plan generation and can make credential-derived registry conflict state depend on ambient shell values rather than the explicit compiler inputs.
    • Recommendation: Prefer making credential hashes an explicit compiler input only, or add a clearly named option for ambient-env fallback. Add a regression test proving plan compilation cannot accidentally derive credentialHash from ambient process.env when the caller supplied explicit empty/no hash context.
    • Evidence: resolveCredentialHash() checks context.credentialHashes first and then returns hashCredential(process.env[envKey]) ?? undefined.
  • Legacy provider-name fallback still lacks a migration or removal guard (src/lib/messaging/provider-bindings.ts:19): The manifest-derived provider-name fallback is useful for pre-plan registry rows, but the code still cannot prove that a row is truly pre-plan versus a new row missing a plan due to drift. Without a versioned migration marker or other provenance guard, new source-of-truth drift can continue relying on compatibility behavior.
    • Recommendation: Make the compatibility boundary explicit with a registry version, migration marker, or equivalent provenance. Use manifest-derived names only for rows proven to predate messaging.plan. Document or enforce the exact condition under which getManifestProviderNamesForChannel() can be removed.
    • Evidence: getManifestProviderNamesForChannel() says it is a compatibility helper to remove once supported entries are migrated; resolveChannelProviderNamesForRemove() still falls back to manifest-derived names when entry.messaging.plan is absent and entry.messagingChannels contains the channel.
  • Channel-add conflict backfill recovery still has no explicit removal condition (src/lib/actions/sandbox/policy-channel.ts:429): The channel-add conflict path intentionally swallows backfillMessagingChannels() failures so recorded conflict state can still be used when the gateway/backfill probe is transiently unavailable. That behavior is tested, but the source-of-truth boundary and the condition for removing this compatibility recovery remain implicit.
    • Recommendation: Document the exact invalid state handled by the swallowed backfill failure and the point at which it can be removed, such as after registry plans and credential hashes are guaranteed for all supported rows and gateway probing is no longer part of conflict backfill.
    • Evidence: checkChannelAddConflict() catches and ignores backfillMessagingChannels() errors before findChannelConflicts(); policy-channel-conflict.test.ts covers swallowed backfill failure with a pre-recorded matching hash and proceeding when no pre-recorded conflict exists.
  • Best-effort provider replacement failure contracts are not pinned (src/lib/messaging/applier/openshell-provider.ts:45): bestEffort mode continues past provider delete/create/update failures and throws an aggregate error after iterating. That is appropriate for rollback/cleanup only if callers can rely on redaction, failed-provider exclusion from results, and continued unrelated-provider processing, but I did not find tests covering best-effort failure semantics.
    • Recommendation: Add tests for applyCredentialsAtOpenShell(..., { bestEffort: true }) covering replace delete failure and create/update failure. Assert errors are aggregated and redacted, failed providers are excluded from providerNames/upserted, and unrelated successful providers are preserved.
    • Evidence: openshell-provider.ts pushes failure messages under options.bestEffort at delete and create/update sites, then throws failures.join('; '); grep found no bestEffort coverage in setup-applier.test.ts.

🌱 Nice ideas

  • None.
Consider writing more tests for
  • **Runtime validation** — applyCredentialsAtOpenShell bestEffort replace delete failure aggregates a redacted error and continues unrelated providers. The changed behavior crosses OpenShell provider lifecycle, sandbox create/rebuild attachment, registry plan validation, credential rotation, network policy cleanup, CLI redaction, and compatibility fallback boundaries. Unit and subprocess-style coverage improved, but several externally visible provider lifecycle and cleanup paths remain mocked or unpinned.
  • **Runtime validation** — applyCredentialsAtOpenShell bestEffort create failure excludes the failed provider from upserted/providerNames and preserves successful providers. The changed behavior crosses OpenShell provider lifecycle, sandbox create/rebuild attachment, registry plan validation, credential rotation, network policy cleanup, CLI redaction, and compatibility fallback boundaries. Unit and subprocess-style coverage improved, but several externally visible provider lifecycle and cleanup paths remain mocked or unpinned.
  • **Runtime validation** — channels remove with a valid stored Slack plan detaches and deletes both plan-derived Slack providers. The changed behavior crosses OpenShell provider lifecycle, sandbox create/rebuild attachment, registry plan validation, credential rotation, network policy cleanup, CLI redaction, and compatibility fallback boundaries. Unit and subprocess-style coverage improved, but several externally visible provider lifecycle and cleanup paths remain mocked or unpinned.
  • **Runtime validation** — channels remove with a valid stored WeChat plan detaches and deletes the plan-derived WeChat provider. The changed behavior crosses OpenShell provider lifecycle, sandbox create/rebuild attachment, registry plan validation, credential rotation, network policy cleanup, CLI redaction, and compatibility fallback boundaries. Unit and subprocess-style coverage improved, but several externally visible provider lifecycle and cleanup paths remain mocked or unpinned.
  • **Runtime validation** — channels remove treats OpenShell NotFound and not-attached detach/delete output as success and still updates registry/policy cleanup. The changed behavior crosses OpenShell provider lifecycle, sandbox create/rebuild attachment, registry plan validation, credential rotation, network policy cleanup, CLI redaction, and compatibility fallback boundaries. Unit and subprocess-style coverage improved, but several externally visible provider lifecycle and cleanup paths remain mocked or unpinned.
  • **Best-effort provider replacement failure contracts are not pinned** — Add tests for applyCredentialsAtOpenShell(..., { bestEffort: true }) covering replace delete failure and create/update failure. Assert errors are aggregated and redacted, failed providers are excluded from providerNames/upserted, and unrelated successful providers are preserved.
  • **Acceptance clause:** OpenShell command shapes for provider upsert, attach, detach, and delete match current externally visible behavior. — add test evidence or identify existing coverage. setup-applier.test.ts asserts provider get/create/update/delete-recreate argv and sandboxCreateProviderArgs; channels-remove-full-teardown.test.ts asserts pre-plan Telegram/Slack detach/delete argv. Valid stored-plan remove, rollback, WeChat, and NotFound/not-attached cleanup parity coverage was not evident.
  • **Acceptance clause:** Sandbox create/rebuild receives the same messaging provider attachments as before. — add test evidence or identify existing coverage. applyCredentialsAtOpenShell() returns sandboxCreateProviderArgs/providerNames and onboard.ts appends providerNames into createArgs. A direct create/rebuild path assertion for provider attachment parity from a validated stored registry plan was not evident.
Since last review details

Current findings:

  • Source-of-truth review needed: Legacy provider-name compatibility fallback: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: getManifestProviderNamesForChannel() is documented as a compatibility helper and resolveChannelProviderNamesForRemove() falls back to it when no plan exists.
  • Source-of-truth review needed: Channel add conflict backfill recovery: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: checkChannelAddConflict() catches and ignores backfillMessagingChannels() errors before findChannelConflicts().
  • Source-of-truth review needed: Best-effort OpenShell provider cleanup/replacement: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: openshell-provider.ts accumulates failures under options.bestEffort and throws an aggregate error after iterating.
  • Source-of-truth review needed: Ambient process.env credential-hash fallback: The advisor marked localized patch analysis as needs_followup.
    • Recommendation: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
    • Evidence: resolveCredentialHash() falls back from context.credentialHashes to hashCredential(process.env[envKey]).
  • Plan-backed provider lifecycle parity is still only partially pinned (src/lib/actions/sandbox/policy-channel.ts:1474): Issue Phase 4b: migrate messaging credential/provider binding to manifest plans #4393 requires provider upsert, attach, detach, delete, and create/rebuild attachment behavior to match current externally visible behavior. The PR now covers useful pieces, including provider get/create/update/delete-recreate shapes, plan tampering rejection, and pre-plan Telegram/Slack remove cleanup. I still could not find direct coverage for valid stored-plan remove across the main provider shapes, NotFound/not-attached success-equivalent cleanup, rollback cleanup parity, WeChat cleanup, or rebuild attachment parity from a validated stored plan.
    • Recommendation: Add focused tests for valid stored-plan remove and rollback detach/delete argv for Telegram, Discord, Slack bot/app, and WeChat. Include OpenShell NotFound/not-attached detach/delete outputs and a create/rebuild path assertion that a validated stored plan attaches the same provider names as initial create.
    • Evidence: setup-applier.test.ts asserts applyCredentialsAtOpenShell command shapes and sandboxCreateProviderArgs; channels-remove-full-teardown.test.ts covers pre-plan Telegram/Slack detach/delete and tampered-plan rejection. Comparable valid stored-plan remove, WeChat cleanup, NotFound/not-attached, rollback, and rebuild attachment parity coverage was not evident.
  • Large high-risk lifecycle validation code grew as a new monolith (src/lib/messaging/applier/plan-validation.ts:1): This PR adds a large new validator and continues expanding existing oversized messaging lifecycle/test files. The changed code controls stored-plan trust, OpenShell provider names, credential hashes, network policy entries, hooks, render plans, and registry source-of-truth binding. Keeping that much security-critical validation in one large module makes review and future hardening harder.
    • Recommendation: Split plan validation into smaller focused modules, such as channel state validation, credential binding validation, policy validation, and generated-entry validation. Avoid further expanding policy-channel.ts and oversized provider lifecycle tests; extract reusable provider cleanup and stored-plan source-of-truth helpers before adding more behavior.
    • Evidence: Deterministic size analysis flags new src/lib/messaging/applier/plan-validation.ts at about 499 lines, src/lib/actions/sandbox/policy-channel.ts growing further, and existing large workflow/setup tests growing substantially.
  • Credential hash planning can fall back to ambient process.env (src/lib/messaging/compiler/engines/credential-binding-engine.ts:56): The credential binding compiler now accepts credentialHashes in context, but resolveCredentialHash() falls back to hashing process.env[envKey] when context hashes are absent. That does not store raw secrets, but it creates hidden host-environment coupling in plan generation and can make credential-derived registry conflict state depend on ambient shell values rather than the explicit compiler inputs.
    • Recommendation: Prefer making credential hashes an explicit compiler input only, or add a clearly named option for ambient-env fallback. Add a regression test proving plan compilation cannot accidentally derive credentialHash from ambient process.env when the caller supplied explicit empty/no hash context.
    • Evidence: resolveCredentialHash() checks context.credentialHashes first and then returns hashCredential(process.env[envKey]) ?? undefined.
  • Legacy provider-name fallback still lacks a migration or removal guard (src/lib/messaging/provider-bindings.ts:19): The manifest-derived provider-name fallback is useful for pre-plan registry rows, but the code still cannot prove that a row is truly pre-plan versus a new row missing a plan due to drift. Without a versioned migration marker or other provenance guard, new source-of-truth drift can continue relying on compatibility behavior.
    • Recommendation: Make the compatibility boundary explicit with a registry version, migration marker, or equivalent provenance. Use manifest-derived names only for rows proven to predate messaging.plan. Document or enforce the exact condition under which getManifestProviderNamesForChannel() can be removed.
    • Evidence: getManifestProviderNamesForChannel() says it is a compatibility helper to remove once supported entries are migrated; resolveChannelProviderNamesForRemove() still falls back to manifest-derived names when entry.messaging.plan is absent and entry.messagingChannels contains the channel.
  • Channel-add conflict backfill recovery still has no explicit removal condition (src/lib/actions/sandbox/policy-channel.ts:429): The channel-add conflict path intentionally swallows backfillMessagingChannels() failures so recorded conflict state can still be used when the gateway/backfill probe is transiently unavailable. That behavior is tested, but the source-of-truth boundary and the condition for removing this compatibility recovery remain implicit.
    • Recommendation: Document the exact invalid state handled by the swallowed backfill failure and the point at which it can be removed, such as after registry plans and credential hashes are guaranteed for all supported rows and gateway probing is no longer part of conflict backfill.
    • Evidence: checkChannelAddConflict() catches and ignores backfillMessagingChannels() errors before findChannelConflicts(); policy-channel-conflict.test.ts covers swallowed backfill failure with a pre-recorded matching hash and proceeding when no pre-recorded conflict exists.
  • Best-effort provider replacement failure contracts are not pinned (src/lib/messaging/applier/openshell-provider.ts:45): bestEffort mode continues past provider delete/create/update failures and throws an aggregate error after iterating. That is appropriate for rollback/cleanup only if callers can rely on redaction, failed-provider exclusion from results, and continued unrelated-provider processing, but I did not find tests covering best-effort failure semantics.
    • Recommendation: Add tests for applyCredentialsAtOpenShell(..., { bestEffort: true }) covering replace delete failure and create/update failure. Assert errors are aggregated and redacted, failed providers are excluded from providerNames/upserted, and unrelated successful providers are preserved.
    • Evidence: openshell-provider.ts pushes failure messages under options.bestEffort at delete and create/update sites, then throws failures.join('; '); grep found no bestEffort coverage in setup-applier.test.ts.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

@sandl99 sandl99 marked this pull request as draft June 8, 2026 04:43
@copy-pr-bot

copy-pr-bot Bot commented Jun 8, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@wscurran wscurran added area: messaging Messaging channels, bridges, manifests, or channel lifecycle area: providers Inference provider integrations and provider behavior refactor PR restructures code without intended behavior change labels Jun 8, 2026
Base automatically changed from feat/phase-4a-messaging-conflict-plan-driven to main June 8, 2026 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: messaging Messaging channels, bridges, manifests, or channel lifecycle area: providers Inference provider integrations and provider behavior refactor PR restructures code without intended behavior change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants