improvement(self-host): simplify capability setup configuration (#6230)

* feat(self-host): add capability-aware setup

* fix(self-host): preserve capability compatibility

* fix(copilot): honor preview availability server-side

* improvement(self-host): centralize capability resolution

* fix(self-host): preserve integration availability paths

* fix(testing): align capability-aware config mocks

* improvement(self-host): simplify capability setup configuration

* fix(setup): preserve unowned storage overrides

* fix(self-host): reconcile storage and allowlists

* fix(integrations): preserve connect deep links

Author: TheodoreSpeaks

.agents/skills/add-block/SKILL.md

@@ -163,6 +163,32 @@ export const {ServiceName}Block: BlockConfig = {
 
 Optional companions: `credentialLabels` (override the picker's section/connect-row copy) and `allowServiceAccounts: true` (trigger-mode only — list service accounts, which triggers otherwise exclude; set only when the trigger's polling path can resolve a service-account token). The connect modal, provider families (Google JSON key, Atlassian token, token-paste, client-credential, Slack bot), and the preview gate are all resolved from `serviceAccountProviderId` — you don't wire them per block.
 
+### OAuth deployment availability (required for integration blocks)
+
+A visible tools-category block with OAuth is deployment-gated. Its `oauth-input.serviceId` is
+projected into `apps/sim/lib/integrations/integrations.json`, then resolved through
+`resolveOAuthClientCapabilityId()` in `apps/sim/lib/core/config/env-capabilities.ts`.
+
+When adding or changing an OAuth integration block:
+
+1. Keep exactly one distinct OAuth `serviceId` across the block's `oauth-input` subBlocks.
+2. Confirm that service ID resolves to an entry in `OAUTH_CLIENT_CAPABILITIES`. Google and
+   Microsoft service IDs intentionally share their provider-level capability; do not add duplicate
+   entries for those aliases.
+3. For a new capability, add its required client fields to `OAUTH_CLIENT_CAPABILITIES` and ensure
+   every referenced field exists in the env schema in `apps/sim/lib/core/config/env.ts`. Then add
+   the matching `text` or `secret` input modes to `OAUTH_CLIENT_SETUP_FIELDS` in
+   `scripts/setup/capability-config.ts`. The CLI catalog is exhaustively typed and checked against
+   the runtime field list; do not infer secrecy from the field name.
+4. If the canonical OAuth service declares `serviceAccountProviderId`, keep
+   `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in
+   `apps/sim/lib/integrations/service-account-metadata.ts` aligned. Set
+   `deploymentRequirement` only when the service-account path is preview-gated or depends on the
+   OAuth client fields; otherwise omit it.
+
+Missing capability metadata is a runtime configuration error, not a reason to make the integration
+silently available.
+
 ### Selectors (with dynamic options)
 ```typescript
 // Channel selector (Slack, Discord, etc.)
@@ -919,12 +945,25 @@ Derive templates from the service's real use cases. Each prompt should name a co
 - **Ground every skill in operations the block actually exposes** — cross-check each skill's steps against `tools.access`. Never describe an action the integration cannot perform.
 - **Derive skills from real, popular use cases found online — never invent them.** Web-search the service's documented use cases (vendor use-case/solutions pages, official docs describing the workflow, reputable "top automations for X" articles) and only add a skill you can source as something people genuinely do with the service. Do not hallucinate skills.
 
-## Generated tool metadata
+## Generated artifacts
 
-Adding a block on its own needs **no** regeneration — a block references existing tool IDs through `tools.access` and does not change any tool's shape.
+Adding a block on its own needs no **tool metadata** regeneration — a block references existing
+tool IDs through `tools.access` and does not change any tool's shape.
 
 But if the same change also adds, edits **or removes** a tool, run `bun run tool-metadata:generate` and commit the result, or CI fails on stale artifacts. That matters here because a block's `outputs` are authored to match its tools' outputs, and the UI now reads those from the generated metadata rather than the executable registry — an unregenerated tool change makes the block's outputs disagree with what the panel renders. See `.agents/skills/tool-registry-boundary/SKILL.md`.
 
+A visible integration block does require the generated integration catalog and docs to be refreshed.
+After adding or changing one, run:
+
+```bash
+bun run scripts/generate-docs.ts
+bun run integration-catalog:check
+```
+
+The catalog check independently derives deployment metadata from the executable block registry and
+compares it with the committed `apps/sim/lib/integrations/integrations.json`. Review the generated
+diff and keep only intentional changes.
+
 ## Checklist Before Finishing
 
 - [ ] `integrationType` is set to the correct `IntegrationType` enum value
@@ -934,12 +973,17 @@ But if the same change also adds, edits **or removes** a tool, run `bun run tool
 - [ ] DependsOn set for fields that need other values
 - [ ] Required fields marked correctly (boolean or condition)
 - [ ] OAuth inputs have correct `serviceId` and `requiredScopes: getScopesForService(serviceId)`
+- [ ] Every OAuth `serviceId` resolves through `resolveOAuthClientCapabilityId()` to the correct `OAUTH_CLIENT_CAPABILITIES` entry
+- [ ] Any new OAuth capability fields exist in `apps/sim/lib/core/config/env.ts`
+- [ ] If the OAuth service supports service accounts, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` matches its canonical `serviceAccountProviderId` and deployment requirement
 - [ ] Scope descriptions added to `SCOPE_DESCRIPTIONS` in `lib/oauth/utils.ts` for any new scopes
 - [ ] Tools.access lists all tool IDs (snake_case)
 - [ ] Tools.config.tool returns correct tool ID (snake_case)
 - [ ] Outputs match tool outputs
 - [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
 - [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts
+- [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes
+- [ ] `bun run integration-catalog:check` passes
 - [ ] If icon missing: asked user to provide SVG
 - [ ] If triggers exist: `triggers` config set, trigger subBlocks spread
 - [ ] Optional/rarely-used fields set to `mode: 'advanced'`

.agents/skills/add-integration/SKILL.md

@@ -17,7 +17,8 @@ Adding an integration involves these steps in order:
 4. **Add Icon** - Add the service's brand icon
 5. **Create Triggers** (optional) - If the service supports webhooks
 6. **Register** - Register tools, block, and triggers in their registries
-7. **Generate Docs** - Run the docs generation script
+7. **Configure Deployment Availability** - Wire OAuth client and service-account metadata
+8. **Generate and Validate the Catalog** - Regenerate docs/catalog artifacts and run drift checks
 
 ## Step 1: Research the API
 
@@ -465,15 +466,48 @@ export const TRIGGER_REGISTRY: TriggerRegistry = {
 }
 ```
 
-## Step 7: Generate Docs
+## Step 7: Configure Deployment Availability
+
+Do this for every visible OAuth integration. API-key and unauthenticated integrations do not need
+an OAuth client capability.
+
+The block's `oauth-input.serviceId` is the canonical link between the generated integration catalog,
+the OAuth service configuration, deployment availability, and the setup CLI.
+
+1. Ensure the block has exactly one distinct OAuth `serviceId` and that it matches the canonical
+   service entry in `apps/sim/lib/oauth/oauth.ts`.
+2. Confirm `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended provider entry in
+   `OAUTH_CLIENT_CAPABILITIES` in `apps/sim/lib/core/config/env-capabilities.ts`. Google and
+   Microsoft service IDs deliberately share provider-level capabilities.
+3. For a new OAuth provider, add the required client fields to `OAUTH_CLIENT_CAPABILITIES`, add
+   every referenced field to the env schema in `apps/sim/lib/core/config/env.ts`, and add the
+   matching `text` or `secret` entries to `OAUTH_CLIENT_SETUP_FIELDS` in
+   `scripts/setup/capability-config.ts`. Do not create integration-specific setup logic or infer
+   secret fields from naming; the CLI mapping is exhaustively checked against the runtime fields.
+4. If the canonical OAuth service has `serviceAccountProviderId`, add the matching projection to
+   `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in
+   `apps/sim/lib/integrations/service-account-metadata.ts`. Use:
+   - no `deploymentRequirement` when the service-account path works independently of OAuth client fields;
+   - `'oauth-client'` when it requires the same deployment OAuth client fields;
+   - `'preview-gated'` when availability is controlled by the service-account preview block.
+
+Never add a permissive fallback for missing capability metadata. A visible OAuth integration without
+a resolvable capability must fail validation.
+
+## Step 8: Generate and Validate the Catalog
 
 Run the documentation generator:
 ```bash
 bun run scripts/generate-docs.ts
+bun run integration-catalog:check
 ```
 
 This creates `apps/docs/content/docs/en/integrations/{service}.mdx` — one page per service carrying the block's Actions and, if it has one, its Triggers section. Never hand-edit generated pages; the only editable region is the `{/* MANUAL-CONTENT */}` block (see `scripts/README.md`).
 
+The same generator refreshes `apps/sim/lib/integrations/integrations.json`. The catalog check then
+derives the deployment-relevant fields from the executable block registry and compares them with the
+committed projection. Review the generated diff and keep only intentional changes.
+
 ## V2 Integration Pattern
 
 If creating V2 versions (API-aligned outputs):
@@ -524,6 +558,13 @@ If creating V2 versions (API-aligned outputs):
 - [ ] Used `getCanonicalScopesForProvider()` in `auth.ts` (never hardcode)
 - [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode)
 
+### Deployment Availability (if OAuth service)
+- [ ] Block declares exactly one distinct `oauth-input.serviceId`
+- [ ] `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry
+- [ ] Every new OAuth capability field exists in `apps/sim/lib/core/config/env.ts`
+- [ ] Runtime OAuth fields live in `OAUTH_CLIENT_CAPABILITIES`; matching CLI input modes live in the exhaustively checked `OAUTH_CLIENT_SETUP_FIELDS`
+- [ ] If `serviceAccountProviderId` is configured, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` has the matching projection and deployment requirement
+
 ### Icon
 - [ ] Asked user to provide SVG
 - [ ] Added icon to `components/icons.tsx`
@@ -542,6 +583,8 @@ If creating V2 versions (API-aligned outputs):
 ### Docs
 - [ ] Ran `bun run scripts/generate-docs.ts`
 - [ ] Verified docs file created
+- [ ] Reviewed and committed the generated `apps/sim/lib/integrations/integrations.json` change
+- [ ] `bun run integration-catalog:check` passes
 
 ### Final Validation (Required)
 - [ ] Read every tool file and cross-referenced inputs/outputs against the API docs
@@ -886,3 +929,5 @@ requiredScopes: getScopesForService('{service}'),
 10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled
 11. **Never hardcode scopes** - Use `getScopesForService()` in blocks and `getCanonicalScopesForProvider()` in auth.ts
 12. **Always add scope descriptions** - New scopes must have entries in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts`
+13. **OAuth service IDs need deployment capabilities** - Every visible OAuth integration must resolve through `OAUTH_CLIENT_CAPABILITIES`; shared Google/Microsoft aliases map to their provider capability
+14. **Keep runtime and presentation separate** - Runtime OAuth fields live in `env-capabilities.ts`; CLI input modes live in the exhaustively checked `scripts/setup/capability-config.ts` mapping

.agents/skills/ship/SKILL.md

@@ -54,17 +54,23 @@ When the user runs `/ship`:
   ```
   Then `git status --short` to see what regenerated — those files must be staged in step 7 alongside your own changes.
 
-  **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present.
+  **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. `scripts/generate-docs.ts` rewrites the integration docs and client-safe catalog; run it when this PR changes their block/icon/landing-content inputs or when `integration-catalog:check` reports drift, then review its broad generated diff. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present.
 
-  **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then fan the rest out and collect exit codes. This is exactly the read-only audit set from CI's `Lint and Test` job (all in-repo, runnable in any worktree):
+  **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/test-build.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes:
   ```bash
   # autofix formatting first (mutating; not parallel-safe with the audits). Gate its exit too —
   # a non-zero lint (unfixable errors) must abort before the audits run, not be ignored.
   bun run lint || { echo "❌ lint failed — do not ship"; exit 1; }
+  bun run apps/sim/scripts/check-block-registry.ts origin/staging || {
+    echo "❌ block registry audit failed — do not ship"
+    exit 1
+  }
   rm -f /tmp/ship-audit-results
-  for s in check:boundaries check:api-validation:strict check:utils check:zustand-v5 \
+  for s in check:boundaries check:api-validation:strict check:desktop-bridge check:desktop-ipc \
+           check:utils check:zustand-v5 \
            check:react-query check:client-boundary check:bare-icons check:icon-paths \
-           check:realtime-prune skills:check agent-stream-docs:check; do
+           check:realtime-prune check:tool-registry-boundary tool-metadata:check \
+           integration-catalog:check skills:check agent-stream-docs:check; do
     ( bun run "$s" >"/tmp/ship-audit-${s//:/-}.log" 2>&1; echo "$? $s" >>/tmp/ship-audit-results ) &
   done
   wait
@@ -150,4 +156,3 @@ gh pr create --base staging --title "COMMIT_MESSAGE" --body "PR_BODY"
 - "Tested manually" is acceptable for testing section; include lint, boundary validation, and (when migrations changed) `check:migrations` results when run
 - Checkboxes filled in appropriately
 - No screenshots section unless UI changes
-