Skip to content

feat(orgs): bulk org/site import + organization_external_links (#3242) - #3273

Merged
ToddHebebrand merged 4 commits into
mainfrom
feat/3242-org-external-links-bulk-import
Aug 8, 2026
Merged

feat(orgs): bulk org/site import + organization_external_links (#3242)#3273
ToddHebebrand merged 4 commits into
mainfrom
feat/3242-org-external-links-bulk-import

Conversation

@ToddHebebrand

@ToddHebebrand ToddHebebrand commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Closes #3243. Part of epic #3249.

Implements the plan in docs/superpowers/plans/open/2026-08-08-partner-api-provisioning-writes.md (on the docs/rmm-migration-guides branch): lets a partner service principal create organizations, sites, and enrollment keys unattended, so RMM-migration scripts and integrations can provision tenancy without an MFA'd human JWT.

What changed

Scopes (services/partnerServicePrincipalScopes.ts)

  • New organizations:write, sites:write, enrollment-keys:write, validated by the existing validatePartnerServicePrincipalScopes and mintable only through the MFA-gated partner principal management routes.
  • The Weavestream default scope set is now pinned to the read scopes explicitly — write scopes are opt-in at principal creation, never implicit.
  • hasSatisfiedMfa is untouched, per the issue thread's explicit rejection of principal-type branching.

Write-surface allowlist test (routes/partnerApi/writeSurface.test.ts) — written FIRST, per the condition attached to the #3243 decision

  • Enumerates every route under partnerApiRoutes; the set of non-GET routes must equal an explicit allowlist constant. Seeded empty, verified green on the pre-change tree; its first run against the new routes failed with exactly POST /organizations, POST /sites, POST /enrollment-keys before they were allowlisted (evidence the guard works). A canary test proves a fourth unlisted write route fails the suite.

Auth middleware (middleware/partnerApiAuth.ts)

  • Non-GET requests take a new branch: principal + org discovery resolve in a short bounded system context, and the handler runs with no ambient DB context — handlers open their own bounded contexts per operation (mirroring the human /orgs/* write routes).
  • Why: the read path holds the partner-export advisory lock shared for the whole request, while an organizations INSERT trigger takes the same lock exclusive (2026-07-21-partner-export-canonical-org-mutations.sql). An insert from a nested system transaction on a second pooled connection would wait on our own request's shared lock — an application-level self-deadlock. Writes also take no export snapshot, so the held-transaction consistency machinery doesn't apply to them.
  • Writes get a tighter per-principal rate bucket: min(key limit, 120)/hour, separate Redis key, 429 + Retry-After on exhaustion. recordMachineUse continues to audit every write request (success and failure) as the service principal.

Routes (routes/partnerApi/provisioning.ts)

  • POST /partner-api/organizationsorganizations:write. partnerId comes only from the principal (body value stripped). Runs in a bounded system context (a new org's id can't pass the id-keyed RLS insert policy under partner scope — same escape as the human route), takes the partner lock exclusive up front so the maxOrganizations quota check is race-free, maps slug 23505 to 409, and enforces the cap with a specific 409 (partner_provisioning_org_limit_reached). Status restricted to active/trial (creating suspended/churned tenants unattended is not a workload).
  • POST /partner-api/sitessites:write. orgId outside the principal's accessible set → 403. Insert runs in a partner-scoped bounded context (userId: null), so RLS enforces org access a second time.
  • POST /partner-api/enrollment-keysenrollment-keys:write. Mirrors the human schema (.strict(), ttlMinutes XOR expiresAt, maxUsage 1–100000), rejects TTLs above the partner cap on both expiry paths via the shared assertTtlWithinCap, verifies siteId belongs to the org, stores only the hash, createdBy: null. Raw key returned exactly once at the top level of the 201 body.
  • Created-object responses reuse the read-side export DTO contracts: same strict record schemas, revision, and safelyExportDefinition inspection, so dtoSafety/exportSafety apply to writes exactly as to reads.
  • No DELETE routes — deletion stays human + MFA on the main API (plan Task 4).
  • Audit: organization.create / site.create / enrollment_key.create events attributed to the service principal (actorType: 'api_key', key id), never a human.

UI (apps/web/.../PartnerServicePrincipalsPage.tsx) — write scopes selectable but excluded from the default selection.

Docsreference/api.mdx gains a Partner API section (all routes + scopes); reference/api-keys.mdx gains the partner-principal scope table with the opt-in warning.

Tests

  • writeSurface.test.ts — allowlist + canary (first run failed pre-allowlist, see above).
  • provisioning.test.ts — 21 cases: scope/auth (401/403), body-partnerId ignored, org-access 403s, maxOrganizations boundary (at cap 409 / below cap 201), TTL-cap 400, XOR 400, .strict() unknown-key 400, site-mismatch 400, raw-key-once + hash-stored + createdBy null, DTO revision shape, partner-context userId: null, audit attribution for all three routes.
  • partnerApiAuth.test.ts — 5 new cases for the write branch: no held context during next(), write bucket key/limit, write 429 + Retry-After, machine-use audit on write success and failure. All 34 pre-existing cases unchanged and green.
  • Affected suites green locally (single worker): 19 files, 490 tests (partnerApiAuth, all routes/partnerApi/*, partnerServicePrincipals, enrollmentKeys*, enrollmentKeySecurity). tsc --noEmit clean for apps/api.

Not in this PR

🤖 Generated with Claude Code

ToddHebebrand and others added 3 commits August 8, 2026 15:20
…3242)

- New organization_external_links table: shape-1 RLS (enabled+forced, same
  migration), composite (org_id, partner_id) FK, per-partner unique
  (partner_id, system, external_id), backfill from accounting_* columns.
- Registered in CORE_ORG_CASCADE_DELETE_ORDER and CORE_TENANT_EXPORT_POLICY.
- services/orgImport: preview/commit pipeline with source seam, name-match
  acknowledgement, soft-deleted reactivation opt-in, per-row partial success.
- QuickBooks importer DUAL-WRITES the link row alongside the legacy columns
  and reads the union, so post-PR orgs keep matching on re-import.
- POST /orgs/import/preview and POST /orgs/import (partner/system + orgs:write
  + MFA), 1000-row cap, audit fan-out.
- Unit tests (service, routes, QB importer) + RLS/backfill integration suite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CSV upload (picker + drag-and-drop) -> client-side RFC-4180 parse (hand-rolled,
no new dependency) -> column mapping with auto-guess -> preview table with
per-row status badges -> commit through runAction with partial-success
reporting. name-match rows unchecked by default; soft-deleted matches are an
explicit reactivate opt-in; conflict rows not selectable. Mounted on the
Organizations settings page behind a Bulk import toggle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 8, 2026

Copy link
Copy Markdown

Deploying breeze with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0e5bdd0
Status: ✅  Deploy successful!
Preview URL: https://4f852c5e.breeze-9te.pages.dev
Branch Preview URL: https://feat-3242-org-external-links.breeze-9te.pages.dev

View logs

- Use shared isPgUniqueViolation (utils/pgErrors) in orgImport and the QB
  importer: Drizzle wraps postgres.js errors so a top-level .code check missed
  every race-recovery path.
- Identity pinning: commit rows carry expectedOrganizationId from preview and
  are rejected if the re-derived match resolves to a different organization.
- Web select-all only toggles create/link-match rows — never bulk-acknowledges
  name-matches or bulk-reactivates soft-deleted orgs.
- i18n: mark dynamic t() keys with /* i18n-dynamic */, translate remaining
  bulkOrgImport strings, and bump settings.json duplicate baselines for the
  genuine cognates (Status de/pt, Site fr) with justification comments.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ToddHebebrand

Copy link
Copy Markdown
Collaborator Author

Review run: /code-review at high effort (8 finder angles: 3 correctness — line-by-line diff scan, removed-behavior audit, cross-file trace; reuse; simplification; efficiency; altitude; conventions).

Findings: 3 consequential raised → all addressed in 0e5bdd0; 0 outstanding.

  1. New local isUniqueViolation only read the top-level err.code, but Drizzle wraps postgres.js errors in DrizzleQueryError (code lives on .cause) — the concurrent-duplicate recovery paths would never fire. → Both the org-import pipeline and the QuickBooks importer now use the shared isPgUniqueViolation from utils/pgErrors.ts.
  2. Name-match identity gap: commit re-derived the match by name only, so an acknowledgement of org X could silently transfer to a different org Y that took over the name between preview and commit. → Commit rows now pin expectedOrganizationId from preview and are rejected if the re-derived match resolves elsewhere.
  3. Web select-all selected every non-conflict row, bulk-acknowledging name-matches and bulk-opting soft-deleted rows into reactivation. → Select-all now only toggles create/link-match rows; name-match and reactivate stay strictly per-row.

Conventions angle came back clean (migration idempotency, cascade/export registrations, runAction usage, i18n all verified compliant). Remaining reuse/altitude notes (third slug-helper copy in aiToolsOrgs.ts, QB importer's own union-read copy pending its migration onto the shared seam) are the plan's explicit Phase 2 items — deferred with #3246 / the QB-seam follow-up, not regressions of this PR.

Tests: apps/api affected suites green single-run (orgImport 27, quickbooksCustomerImport 23, routes/orgs 199, autoMigrate 60, routes/accounting 16, export-policy/tenancy statics 56); apps/web green (BulkOrgImport 10, csvParse, no-silent-mutations, QuickbooksCustomerImport, i18n keyUsage + translationCoverage — 132 across the affected set). tsc --noEmit clean for both packages on touched files.

Live-DB contract suites (rls-coverage, tenantCascade, tenant-export-policy, tenantExportErasureRoundtrip, and the new orgExternalLinksRls forge/backfill suite) cannot run on this machine (no Docker) — CI was dispatched on the branch (workflow_dispatch run 31265785264) to cover them; the previous dispatch's only failures were the web i18n contract tests, fixed in 0e5bdd0. Both the PR run and the dispatched run were still in progress at time of writing — check them before merge.

Status: review-clean, awaiting CI + maintainer merge. The accounting_* column drop, PSA source (#3246), and QB migration onto the shared seam are deliberate follow-ups per the plan.

@ToddHebebrand
ToddHebebrand merged commit 1d18808 into main Aug 8, 2026
99 checks passed
@ToddHebebrand
ToddHebebrand deleted the feat/3242-org-external-links-bulk-import branch August 8, 2026 16:11
@ToddHebebrand

Copy link
Copy Markdown
Collaborator Author

CI update: both runs finished green — PR run 31265784182 (required jobs incl. Test API / Test Web) and dispatched run 31265785264 (workflow_dispatch, covering the live-DB contract suites: rls-coverage, tenantCascade, tenant-export-policy, erasure roundtrip, and the new orgExternalLinksRls forge/backfill suite). Nothing outstanding from my side — awaiting maintainer merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[API][Security] No machine-to-machine credential can provision tenancy — org/site writes are interactive-JWT + MFA only

1 participant