Skip to content

Commit 69a768b

Browse files
Bill LeoutsakosBill Leoutsakos
authored andcommitted
Merge remote-tracking branch 'origin/staging' into feat/quickbooks-integration
# Conflicts: # apps/sim/.env.example # apps/sim/tools/generated/tool-metadata.ts # apps/sim/tools/generated/tool-outputs.ts
2 parents 82c779d + a7d6b96 commit 69a768b

984 files changed

Lines changed: 24997 additions & 13936 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/add-block/SKILL.md

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,32 @@ export const {ServiceName}Block: BlockConfig = {
163163

164164
Optional companions: `credentialLabels` (override the picker's section/connect-row copy) and `allowServiceAccounts: true` (trigger-mode only — list service accounts, which triggers otherwise exclude; set only when the trigger's polling path can resolve a service-account token). The connect modal, provider families (Google JSON key, Atlassian token, token-paste, client-credential, Slack bot), and the preview gate are all resolved from `serviceAccountProviderId` — you don't wire them per block.
165165

166+
### OAuth deployment availability (required for integration blocks)
167+
168+
A visible tools-category block with OAuth is deployment-gated. Its `oauth-input.serviceId` is
169+
projected into `apps/sim/lib/integrations/integrations.json`, then resolved through
170+
`resolveOAuthClientCapabilityId()` in `apps/sim/lib/core/config/env-capabilities.ts`.
171+
172+
When adding or changing an OAuth integration block:
173+
174+
1. Keep exactly one distinct OAuth `serviceId` across the block's `oauth-input` subBlocks.
175+
2. Confirm that service ID resolves to an entry in `OAUTH_CLIENT_CAPABILITIES`. Google and
176+
Microsoft service IDs intentionally share their provider-level capability; do not add duplicate
177+
entries for those aliases.
178+
3. For a new capability, add its required client fields to `OAUTH_CLIENT_CAPABILITIES` and ensure
179+
every referenced field exists in the env schema in `apps/sim/lib/core/config/env.ts`. Then add
180+
the matching `text` or `secret` input modes to `OAUTH_CLIENT_SETUP_FIELDS` in
181+
`scripts/setup/capability-config.ts`. The CLI catalog is exhaustively typed and checked against
182+
the runtime field list; do not infer secrecy from the field name.
183+
4. If the canonical OAuth service declares `serviceAccountProviderId`, keep
184+
`SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in
185+
`apps/sim/lib/integrations/service-account-metadata.ts` aligned. Set
186+
`deploymentRequirement` only when the service-account path is preview-gated or depends on the
187+
OAuth client fields; otherwise omit it.
188+
189+
Missing capability metadata is a runtime configuration error, not a reason to make the integration
190+
silently available.
191+
166192
### Selectors (with dynamic options)
167193
```typescript
168194
// Channel selector (Slack, Discord, etc.)
@@ -919,12 +945,25 @@ Derive templates from the service's real use cases. Each prompt should name a co
919945
- **Ground every skill in operations the block actually exposes** — cross-check each skill's steps against `tools.access`. Never describe an action the integration cannot perform.
920946
- **Derive skills from real, popular use cases found online — never invent them.** Web-search the service's documented use cases (vendor use-case/solutions pages, official docs describing the workflow, reputable "top automations for X" articles) and only add a skill you can source as something people genuinely do with the service. Do not hallucinate skills.
921947

922-
## Generated tool metadata
948+
## Generated artifacts
923949

924-
Adding a block on its own needs **no** regeneration — a block references existing tool IDs through `tools.access` and does not change any tool's shape.
950+
Adding a block on its own needs no **tool metadata** regeneration — a block references existing
951+
tool IDs through `tools.access` and does not change any tool's shape.
925952

926953
But if the same change also adds, edits **or removes** a tool, run `bun run tool-metadata:generate` and commit the result, or CI fails on stale artifacts. That matters here because a block's `outputs` are authored to match its tools' outputs, and the UI now reads those from the generated metadata rather than the executable registry — an unregenerated tool change makes the block's outputs disagree with what the panel renders. See `.agents/skills/tool-registry-boundary/SKILL.md`.
927954

955+
A visible integration block does require the generated integration catalog and docs to be refreshed.
956+
After adding or changing one, run:
957+
958+
```bash
959+
bun run scripts/generate-docs.ts
960+
bun run integration-catalog:check
961+
```
962+
963+
The catalog check independently derives deployment metadata from the executable block registry and
964+
compares it with the committed `apps/sim/lib/integrations/integrations.json`. Review the generated
965+
diff and keep only intentional changes.
966+
928967
## Checklist Before Finishing
929968

930969
- [ ] `integrationType` is set to the correct `IntegrationType` enum value
@@ -934,12 +973,17 @@ But if the same change also adds, edits **or removes** a tool, run `bun run tool
934973
- [ ] DependsOn set for fields that need other values
935974
- [ ] Required fields marked correctly (boolean or condition)
936975
- [ ] OAuth inputs have correct `serviceId` and `requiredScopes: getScopesForService(serviceId)`
976+
- [ ] Every OAuth `serviceId` resolves through `resolveOAuthClientCapabilityId()` to the correct `OAUTH_CLIENT_CAPABILITIES` entry
977+
- [ ] Any new OAuth capability fields exist in `apps/sim/lib/core/config/env.ts`
978+
- [ ] If the OAuth service supports service accounts, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` matches its canonical `serviceAccountProviderId` and deployment requirement
937979
- [ ] Scope descriptions added to `SCOPE_DESCRIPTIONS` in `lib/oauth/utils.ts` for any new scopes
938980
- [ ] Tools.access lists all tool IDs (snake_case)
939981
- [ ] Tools.config.tool returns correct tool ID (snake_case)
940982
- [ ] Outputs match tool outputs
941983
- [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
942984
- [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts
985+
- [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes
986+
- [ ] `bun run integration-catalog:check` passes
943987
- [ ] If icon missing: asked user to provide SVG
944988
- [ ] If triggers exist: `triggers` config set, trigger subBlocks spread
945989
- [ ] Optional/rarely-used fields set to `mode: 'advanced'`

.agents/skills/add-enrichment/SKILL.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ Why it matters: the cascade runner only bills (and only reads `output.cost.total
6363
Create `apps/sim/enrichments/{name}/{name}.ts` and a barrel `index.ts`. Mirror the existing entries (`work-email`, `phone-number`, `company-domain`, `company-info`).
6464

6565
```typescript
66-
import { SomeIcon } from 'lucide-react'
66+
import { SomeIcon } from '@sim/emcn/icons'
6767
import { filterUndefined } from '@sim/utils/object'
6868
import { normalizeDomain, splitName, str, toolProvider } from '@/enrichments/providers'
6969
import type { EnrichmentConfig } from '@/enrichments/types'
@@ -109,7 +109,7 @@ export { myEnrichment } from './my-enrichment'
109109
```
110110

111111
Rules:
112-
- Keep the file **client-safe**: import only `lucide-react`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call.
112+
- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call.
113113
- `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs).
114114
- Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config).
115115

Lines changed: 34 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,38 +1,46 @@
11
---
22
name: add-feature-flag
3-
description: Add a runtime gated feature flag (AppConfig-backed on prod, secret fallback off-prod), gated by org id, user id, or admin
3+
description: Add a runtime feature flag (AppConfig-backed on prod, secret fallback off-prod), global by default or optionally gated by org id, user id, or platform admin
44
argument-hint: <flag-name>
55
---
66

77
# Add Feature Flag Skill
88

9-
You add a **runtime, gated feature flag** to Sim — one that can be turned on for specific orgs, users, or admins and changed on prod with no redeploy (AWS AppConfig). When AppConfig isn't the source of truth, the flag falls back to a single **secret** (on/off only).
9+
You add a **runtime feature flag** to Sim that can change on prod with no redeploy (AWS AppConfig). Prefer a global on/off flag unless the rollout actually needs per-organization, per-user, or platform-admin targeting. When AppConfig isn't the source of truth, the flag falls back to a single **secret** (on/off only).
1010

1111
## When to use this vs `env-flags.ts`
1212

13-
- **Feature flag** (`@/lib/core/config/feature-flags.ts`): per-request, gated by `userId`/`orgId`/admin, changeable at runtime. This skill.
13+
- **Feature flag** (`@/lib/core/config/feature-flags.ts`): runtime global on/off by default, optionally scoped by `userId`/`orgId`/admin. This skill.
1414
- **Env flag** (`@/lib/core/config/env-flags.ts`): deploy-time capability/environment detection (`isProd`, `isHosted`, `isBillingEnabled`). A module-load boolean. **Do not add gated flags here.**
1515

1616
If the user wants a fixed per-deployment toggle, send them to `env-flags.ts` instead.
1717

1818
## The flag model
1919

20-
A flag's **gating rule lives only in the hosted AppConfig document**. It is ON for a context when any clause matches:
20+
A flag's **gating rule lives only in the hosted AppConfig document**. It is ON for a context when any configured clause matches:
2121

2222
```ts
2323
interface FeatureFlagRule {
2424
enabled?: boolean // global default for everyone
2525
orgIds?: string[] // allowlisted organization ids
2626
userIds?: string[] // allowlisted user ids
27-
admins?: boolean // platform admins (user.role === 'admin')
27+
adminEnabled?: boolean // platform admins (user.role === 'admin')
2828
}
2929
```
3030

31-
Critically, **none of this is expressible in code** — gating (especially `admins`) can only be set through AppConfig, so no environment can grant access from a code literal. Off-AppConfig (self-hosted/OSS/local), a flag is simply on or off, derived from its fallback secret.
31+
Critically, **none of this is expressible in code** — gating (especially `adminEnabled`) can only be set through AppConfig, so no environment can grant access from a code literal. Off-AppConfig (self-hosted/OSS/local), a flag is simply on or off, derived from its fallback secret.
3232

3333
## Steps
3434

35-
1. **Define the flag.** Add one entry to the `FEATURE_FLAGS` registry in `apps/sim/lib/core/config/feature-flags.ts`. Each entry is the flag's whole definition — name (kebab-case key), `description`, and the `fallback` secret consulted when AppConfig isn't the source of truth (truthy ⇒ on globally):
35+
1. **Confirm the granularity before editing code.** If the user has not already specified it, stop and ask:
36+
37+
> Should `<flag-name>` be a global on/off flag (recommended), or does it need rollout targeting by organization, user, and/or platform admin?
38+
39+
- Recommend **global**. Do not infer scoped gating merely because the call site already has a user or organization id.
40+
- If the user chooses scoped gating but does not name the dimensions, ask which of organization, user, and platform admin it needs. Wire only the selected dimensions.
41+
- If the user wants a fixed per-deployment toggle rather than a runtime AppConfig flag, use `env-flags.ts` instead.
42+
43+
2. **Define the flag.** Add one entry to the `FEATURE_FLAGS` registry in `apps/sim/lib/core/config/feature-flags.ts`. Each entry is the flag's whole definition — name (kebab-case key), `description`, and the `fallback` secret consulted when AppConfig isn't the source of truth (truthy ⇒ on globally):
3644

3745
```ts
3846
const FEATURE_FLAGS = {
@@ -45,7 +53,19 @@ Critically, **none of this is expressible in code** — gating (especially `admi
4553

4654
`fallback` is the env/secret key (typed as `keyof typeof env`), so add `<FLAG_SECRET>` to `apps/sim/lib/core/config/env.ts` first (and the deployment's secret store) — it won't typecheck otherwise. Do **not** add org/user/admin defaults here — that gating exists only in AppConfig. Adding the entry makes `<flag-name>` a valid `FeatureFlagName`.
4755

48-
2. **Gate the call site.** Call `isFeatureEnabled` with whatever ids you have — admin status is resolved internally, so callers never pass it:
56+
3. **Gate the call site at the chosen granularity.** For the recommended global mode, pass no context:
57+
58+
```ts
59+
import { isFeatureEnabled } from '@/lib/core/config/feature-flags'
60+
61+
if (await isFeatureEnabled('<flag-name>')) {
62+
// gated behavior
63+
}
64+
```
65+
66+
Do not fetch, resolve, or thread through user or organization context solely for a global flag.
67+
68+
For scoped rollout, pass only the dimensions the user selected. Admin status is resolved internally, so ordinary callers pass `userId`, not a role:
4969

5070
```ts
5171
import { isFeatureEnabled } from '@/lib/core/config/feature-flags'
@@ -55,19 +75,21 @@ Critically, **none of this is expressible in code** — gating (especially `admi
5575
}
5676
```
5777

78+
- Organization targeting uses `orgId`; user and platform-admin targeting require `userId`.
5879
- Missing ids are fine — a clause with no matching id is skipped; with no `userId`, the admin clause resolves to `false` without a DB read.
5980
- Admin routes that already know the caller is an admin may pass `{ userId, isAdmin: true }` to skip the role lookup.
6081
- **Client/UI flags:** resolve server-side (in a server component, route, or loader) and pass the boolean down as a prop. There is no client AppConfig.
6182

62-
3. **(Prod) configure in AppConfig.** The infra `feature-flags` profile schema is permissive, so a new flag needs **no infra change**. Operators add the flag under `flags` in the hosted `feature-flags` document — including any `orgIds`/`userIds`/`admins` gating — and start a `sim-<env>-fast` deployment (see the AppConfig runbook in the infra README — same flow as `access-control`). The fallback secret only applies when AppConfig is disabled.
83+
4. **(Prod) configure in AppConfig.** The infra `feature-flags` profile schema is permissive, so a new flag needs **no infra change**. Operators add the flag to the hosted `feature-flags` document using `enabled` for global rollout or only the selected `orgIds`/`userIds`/`adminEnabled` clauses for scoped rollout, then start a `sim-<env>-fast` deployment (see the AppConfig runbook in the infra README — same flow as `access-control`). The fallback secret only applies when AppConfig is disabled.
6384

64-
4. **Test.** Add a case to `apps/sim/lib/core/config/feature-flags.test.ts`: use `withAppConfig({ flags: { ... } })` to cover the gating rule (mock `isPlatformAdmin` for the `admins` clause), and toggle the fallback secret to cover the off-AppConfig path.
85+
5. **Test.** Add a case to `apps/sim/lib/core/config/feature-flags.test.ts` that matches the chosen granularity. For a global flag, exercise `isFeatureEnabled('<flag-name>')` with an AppConfig `enabled` rule and toggle the fallback secret for the off-AppConfig path. For scoped rollout, cover only the selected clauses and mock `isPlatformAdmin` when testing `adminEnabled`.
6586

66-
5. **Clean up after rollout.** When the feature ships to everyone, delete the flag's entry from `FEATURE_FLAGS`, the `<FLAG_SECRET>` env entry, the AppConfig document, the call sites, and the test. Leaving dead flags around is the main failure mode of flag systems.
87+
6. **Clean up after rollout.** When the feature ships to everyone, delete the flag's entry from `FEATURE_FLAGS`, the `<FLAG_SECRET>` env entry, the AppConfig document, the call sites, and the test. Leaving dead flags around is the main failure mode of flag systems.
6788

6889
## Notes
6990

7091
- Flag keys are `kebab-case`.
7192
- Never read flags via raw `fetch` or a new AppConfig client — always go through `isFeatureEnabled` / `getFeatureFlags`.
7293
- Never bake gating into code. The fallback is a single boolean secret; org/user/admin scoping is AppConfig-only.
73-
- The admin check reads the DB **replica** (`dbReplica`) and is resolved lazily, so an admin-gated flag adds at most one cheap replica read, and only when `admins` is the deciding clause.
94+
- Never add or propagate request context unless the user chose scoped rollout.
95+
- The admin check reads the DB **replica** (`dbReplica`) and is resolved lazily, so an admin-gated flag adds at most one cheap replica read, and only when `adminEnabled` is the deciding clause.

0 commit comments

Comments
 (0)