docs(plan): P4.1 — meetings-first orgs & multi-client calendar brief

docs/plan/p4.1-meetings-first-orgs.md

@@ -0,0 +1,211 @@
+# Delegation Brief — P4.1: Meetings-first orgs & multi-client calendar
+
+> Self-contained brief for a fresh, low-context Claude Code session. Read §0 first.
+> **Platform:** macOS, Node 24, pnpm 10.33.0. **This EXTENDS the merged P4 Calendar** (`docs/plan/p4-calendar.md`, PR #7, commit `1f5c36a`) — you are reshaping how meetings become client records, not rebuilding the Graph/Recall plumbing.
+> **Branch + PR for review. Do NOT push to `main`.**
+
+---
+
+## 0. Read first (cold-start context)
+
+- `docs/plan/p4-calendar.md` — the **original P4 brief**. This document is the delta on top of it.
+- `docs/HANDOFF.md` / `docs/09-build-phases.md` — product state (P1–P6 + P6B + SSO + P4 done; live in prod at gracie.graceandassociates.com).
+- `docs/04-database-schema.sql` — `clients`, `client_aliases`, `meetings`, `users`, `settings` (the tables you touch).
+- `docs/07-integrations.md` §6 (MS Graph) · `docs/08-design-system.md` §M7 (Calendar UI).
+
+### What the orchestrator ALREADY did (do not redo)
+- **Wiped all seed/test data** from the shared Supabase: `clients` (12), `documents` (24), `tasks` (23), folders/notes/pipeline_runs/master_record/embeddings/notifications — all cascaded to 0. **Kept** the 9 `users`, the 2 `integration_credentials` (OpenAI + Recall keys), and `settings`. So the DB is a **clean slate**: empty client roster, empty meetings.
+- Confirmed P4 is infra-healthy: the scan runs (business hours Mon–Fri 7–19 ET), both group members (`jgrace@`, `dvelez@`) are `calendar_connected=true`, the bot kill-switch (`settings.calendar_bot_dispatch_enabled`) is **`false`** (fail-safe OFF — leave it OFF), and per-user opt-out works (`dvelez` has `auto_join_meetings=false`).
+
+### Why this phase exists (the problem)
+The original P4 scan **skips** any calendar event that doesn't match a client, and matches only via `client_aliases` + `primary_contact_email` domain against a single `meetings.client_id`. The operator wants the calendar to be the **client-discovery surface**: show *every* meeting, drive client identity off attendees' **email domains**, support **multi-client** meetings, and let a user **create a client (or a lead/prospect) directly from a meeting** with an unknown domain.
+
+---
+
+## 1. Goal (one paragraph)
+
+Turn the calendar into a meetings-first workspace. The scan ingests **every** real meeting (not just matched ones). A meeting's client(s) are derived from the **external email domains** of its attendees (everything except Grace & Associates' own domain and free-email providers). Multiple external domains ⇒ a **multi-client** meeting (many-to-many). Unknown domains surface on the meeting with one-click **"Create client / Create lead / Link existing"**, and creating/linking an org **retroactively connects** past meetings on that domain. Purely-internal meetings (only GA staff) are shown, tagged **Internal**, filed under a first-class **Grace & Associates** internal org (their home for notes/transcripts/files), and — like client meetings — **get an auto-join bot**.
+
+---
+
+## 2. Operator's design decisions (the authority — build to these)
+
+1. **Non-client parties = a `type` on `clients`** (`client` | `prospect` | `lead` | `partner` | `internal`), default `client`. One domain-keyed table for all parties; "promote a lead → client" is just flipping `type`. Non-client types are excluded from client-only surfaces (the client roster list, cadence, fees, the ambiguous-assign picker) but are fully linkable to meetings.
+2. **Internal meetings** (only `@graceandassociates.com` attendees): **show + tag "Internal"**, **and auto-join a bot** (same kill-switch + opt-out gating as clients). Grace & Associates itself becomes a first-class **`internal`-type org** — internal meetings link to it so it's the home for team notes/transcripts/files. "Treat GA as a client in some respects."
+3. **Domain-driven matching**, multi-client via a junction. `graceandassociates.com` and free-email domains never count as client domains.
+4. **Bots**: auto-join for **client** meetings and **internal** meetings only. Leads / prospects / partners / still-unassigned meetings → **no auto-join** (manual dispatch is a later nicety, out of scope here).
+
+---
+
+## 3. Data model — migration `0004` (write it, then apply)
+
+Create `packages/db/migrations/0004_meetings_first_orgs.sql`, mirroring the style of `0002`/`0003` (idempotent: `IF NOT EXISTS`, guarded `create type`). After applying, **regenerate the TS types** the repo uses (`packages/db` generated `Database` types + the `packages/shared` hand types in §5).
+
+```sql
+-- 0004_meetings_first_orgs.sql
+-- P4.1: client type (client/prospect/lead/partner/internal), domain-keyed
+-- matching, many-to-many meeting↔client, external-attendee capture, GA internal org.
+
+-- 1. Party type on clients.
+do $$ begin
+  create type client_type as enum ('client','prospect','lead','partner','internal');
+exception when duplicate_object then null; end $$;
+alter table clients add column if not exists type client_type not null default 'client';
+create index if not exists idx_clients_type on clients (type);
+
+-- 2. Domain → org. THE match key. A domain maps to exactly one org (global unique).
+create table if not exists client_domains (
+  id         uuid primary key default gen_random_uuid(),
+  client_id  uuid not null references clients(id) on delete cascade,
+  domain     text not null,
+  created_at timestamptz not null default now()
+);
+create unique index if not exists uq_client_domains_domain on client_domains (lower(domain));
+create index if not exists idx_client_domains_client on client_domains (client_id);
+
+-- 3. Many-to-many meeting ↔ client (multi-client meetings).
+create table if not exists meeting_clients (
+  meeting_id uuid not null references meetings(id) on delete cascade,
+  client_id  uuid not null references clients(id) on delete cascade,
+  created_at timestamptz not null default now(),
+  primary key (meeting_id, client_id)
+);
+create index if not exists idx_meeting_clients_client on meeting_clients (client_id);
+
+-- 4. Meetings: internal flag + external-attendee capture (external emails/domains
+--    are NOT persisted today — attendee_user_ids holds only INTERNAL user uuids).
+alter table meetings add column if not exists is_internal boolean not null default false;
+alter table meetings add column if not exists external_attendees jsonb not null default '[]'::jsonb;
+-- external_attendees shape: [{ "email": string, "name": string|null, "domain": string }]
+
+-- 5. Seed the Grace & Associates internal org (the home for internal meetings).
+insert into clients (name, initials, type, cadence, description)
+select 'Grace & Associates', 'GA', 'internal', 'ad_hoc',
+       'Internal workspace — team meetings, notes, and files.'
+where not exists (select 1 from clients where type = 'internal');
+
+-- 6. Internal email domains live in settings (configurable, no deploy needed).
+insert into settings (key, value)
+values ('internal_email_domains', 'graceandassociates.com')
+on conflict (key) do nothing;
+```
+
+Design notes:
+- **Keep `meetings.client_id`** as the denormalized **primary org** (nullable). The junction `meeting_clients` is the source of truth for *all* linked orgs; `client_id` = the primary one (internal ⇒ the GA org; external ⇒ the first matched org deterministically by `created_at`, else `null` when unassigned). This keeps every existing `meetings.client_id` read working (client detail, docs, tasks) — minimal blast radius.
+- Do **NOT** register `graceandassociates.com` in `client_domains`; internal domains live only in `settings.internal_email_domains`, and the GA org is found by `type='internal'`. This prevents GA from being matched as a "client" on every meeting.
+- **Apply the migration** to the shared dev Supabase via the same postgres-meta `POST {SUPABASE_URL}/pg/query` path used for `0002`/`0003` (service-role key from `apps/worker/.env.local`; never print/commit it). ⚠️ **Dev and prod share one Supabase** — the migration is additive + idempotent, but **confirm with the orchestrator before applying**.
+
+---
+
+## 4. Worker — scan rewrite (`apps/worker/src`)
+
+### 4a. Matching (`lib/calendar-match.ts` + new `lib/internal-domains.ts` or config)
+Replace client-candidate resolution with **domain-first** logic. Add a free-email blocklist and internal-domain handling.
+
+- **Free-email blocklist** (constant): `gmail.com, googlemail.com, outlook.com, hotmail.com, live.com, msn.com, yahoo.com, ymail.com, icloud.com, me.com, mac.com, aol.com, proton.me, protonmail.com, gmx.com, zoho.com` (extend as needed). Free-email domains are **never** org domains and are **never** offered for "create client from domain".
+- **Internal domains**: read from `settings.internal_email_domains` (comma-separated; default `graceandassociates.com`), lower-cased set.
+- Keep `emailDomain()`, `normalizeText()`, `meetingDedupKey()` as-is. You can keep subject/alias matching as a *secondary* signal, but **domain is now primary**.
+- New per-meeting resolver returns:
+  - `externalOrgDomains` = attendee+organizer domains − internal − freemail.
+  - `matchedClientIds` = orgs whose `client_domains.domain` ∈ externalOrgDomains (join `clients` filtered to `type <> 'internal'`).
+  - `unknownOrgDomains` = externalOrgDomains not found in `client_domains`.
+  - `isInternal` = **every** attendee domain is internal (no external org domain AND no free-email attendee).
+  - `externalAttendees` = `[{email, name, domain}]` for every non-internal attendee (org + freemail) — persisted to `meetings.external_attendees`.
+
+### 4b. Scan processor (`processors/calendar-scan.processor.ts`)
+- **Ingest every real meeting.** Remove the "0 candidates → skip" rule. Guard: still skip `isCancelled`/null-start (as today) and skip **solo calendar blocks** (no `joinUrl` **and** ≤1 attendee) so personal holds/focus time don't become meetings.
+- On insert/update, set: `is_internal`, `external_attendees`, `meeting_clients` links (all `matchedClientIds`; for internal, link the GA org), and `client_id` = primary (per §3). Leave the existing non-destructive rules intact (never disturb a `bot_dispatched`/in-flight/non-calendar row; never overwrite an Admin-assigned link — see below).
+- **Re-scan link rule:** the scan may **add** links for currently-matched domains; it must **never remove** a link (Admin/manual links and previously-created orgs are sticky). Simplest: upsert `meeting_clients` for matched orgs, delete nothing.
+- Keep `loadMatchers()`/roster loads filtered to **`type <> 'internal'`** (never match GA as a client).
+
+### 4c. Bot-dispatch (`processors/bot-dispatch.processor.ts`)
+Keep kill-switch (`settings.calendar_bot_dispatch_enabled`), per-user opt-out (`users.auto_join_meetings` on `meeting_lead_user_id`), the join-URL requirement, the ≤5-min lead / 60-min grace window, and the atomic `false→true` claim. **Change eligibility:** dispatch when the meeting is `is_internal = true` **OR** has ≥1 linked org of `type='client'` (join `meeting_clients`→`clients`). Leads/prospects/partners/unassigned ⇒ no bot. (Today it keys off `client_id != null`; that's now insufficient because an unassigned meeting has a null `client_id` and a lead-only meeting must NOT dispatch.)
+
+---
+
+## 5. Shared types (`packages/shared`)
+
+- `src/constants/enums.ts`: add `CLIENT_TYPES = ['client','prospect','lead','partner','internal']`.
+- `src/types/client.ts`: add `type: ClientType` to `Client`; add a `ClientDomain` type.
+- `src/types/meeting.ts` + `src/types/calendar.ts`: add `isInternal`, `orgs: { id, name, type }[]` (from the junction), `externalAttendees`, and `unknownOrgDomains: string[]` to the calendar view-model (`CalendarMeeting`). Keep `clientId`/`clientName` as the primary-org convenience fields.
+- `packages/db` generated `Database` types: regenerate after the migration so `client_domains`, `meeting_clients`, and the new columns type-check.
+
+---
+
+## 6. Web — API + data layer (`apps/web`)
+
+Modify `lib/data/calendar.ts` and the routes under `app/api/calendar/`:
+
+- **`GET /api/calendar`** (`listCalendarMeetings`): include each meeting's linked `orgs` (join `meeting_clients`→`clients` for id/name/type), `isInternal`, `externalAttendees`, and computed `unknownOrgDomains` (external_attendees domains − internal − freemail − already-linked). The last is computed at read time so it stays correct as orgs get created.
+- **New `POST /api/calendar/meetings/[id]/orgs`** — link/unlink an existing org to a meeting (writes `meeting_clients`; recompute primary `client_id`). Admin/standard; role-gate per existing convention.
+- **New `POST /api/calendar/meetings/[id]/create-org`** — create a `client|prospect|lead|partner` from an unknown domain on this meeting: insert `clients` (with `type`, `initials` derived from the name, `primary_contact`/`primary_contact_email` from the chosen external attendee) + `client_domains` (the domain), link it to this meeting, then **retroactively link** all other meetings whose `external_attendees` contain that domain (insert `meeting_clients`, set `client_id` where null & not internal). Reject free-email domains with a clear error.
+- **`GET/POST /api/calendar/ambiguous`** and **`/assign`**: keep, but the assign picker (`listClients()` options) must filter **`type='client'`** (plus `prospect` if you want). "Ambiguous" now means "has ≥1 unknown org domain or no linked client" — broaden `listAmbiguousMeetings()` accordingly, or lean on the per-meeting flow and keep this as an Admin catch-all.
+- **`POST /api/clients`** (`app/api/clients/route.ts` → `lib/data/clients.ts` `createClient`): accept `type` (default `client`) and optional `domains[]`.
+
+---
+
+## 7. Web — Calendar UI (`app/(app)/calendar/page.tsx`, §M7)
+
+The page is currently fairly monolithic — extract components as needed. Add:
+- **Show every meeting** in the month grid + day detail with clear states: **assigned** (one org chip), **multi-client** (multiple chips), **internal** (grey "Internal" tag), **unassigned / needs-attention** (amber, e.g. `acme.com · no client`).
+- **Meeting detail**: linked-org chips (removable), the **external attendees** list (name/email), and for each **unknown org domain**: **"Create client"**, **"Create lead / prospect"**, **"Link existing"**. The create action opens a small modal pre-filled from the domain (name = title-cased second-level domain) + the attendee (contact name/email), with a **type** selector; on save it calls `create-org` and the meeting re-renders linked.
+- **GA internal workspace**: internal meetings link to the GA `internal` org, so its existing **client-detail page already works** as the notes/files home. Make the GA org **reachable** (e.g. a pinned "Internal · Grace & Associates" entry in the clients nav or a link from the calendar's Internal filter) while keeping it **out of the normal client roster list**.
+- Standard loading / error / empty states; keep the connection panel, auto-join toggle, cadence tracker, and Admin kill-switch that P4 shipped.
+
+---
+
+## 8. `clients.type` blast radius — add a type filter at these EXACT sites
+
+The orchestrator mapped every place `clients` is consumed. Add `type='client'` (or the appropriate set) so leads/prospects/internal don't leak into client surfaces:
+
+| File | Change |
+|---|---|
+| `apps/web/app/api/clients/route.ts` (GET list) | filter `type='client'` (public roster) |
+| `apps/web/lib/data/calendar.ts` → `listClientCadence()` | filter `type='client'` (cadence is client-only) |
+| `apps/web/lib/data/calendar.ts` → ambiguous/assign options | picker shows `type='client'` (+ `prospect` optional) |
+| `apps/web/app/api/calendar/ambiguous/route.ts` | ensure the client options are filtered |
+| `apps/worker/src/processors/calendar-scan.processor.ts` → `loadMatchers()` | match only `type <> 'internal'` (never match GA); domain match against real orgs |
+| `apps/web/app/(app)/clients/page.tsx` | roster list = `type='client'`; consider tabs/filter for leads/prospects; GA (internal) reachable but not in the default list |
+
+`getClient(id)` single-fetch and all `clientId`-scoped queries (documents, tasks, folders, client-detail) need **no** filter — if a row exists, serve it.
+
+---
+
+## 9. Rules & edge cases (get these right)
+
+- **Internal domain(s)** come from `settings.internal_email_domains` (default `graceandassociates.com`). Never treat them as client domains; they drive the `isInternal` decision only.
+- **Free-email domains** are never orgs and never offered for "create from domain." A meeting with only free-email externals is **unassigned** (surface the people so a user can create a **lead** and attach them manually), not internal.
+- **`isInternal`** = every attendee/organizer domain is internal (no external org domain, no free-email external). Internal ⇒ link GA org, `is_internal=true`, bot eligible.
+- **Multi-client**: 2+ distinct external org domains ⇒ link all matched orgs; primary `client_id` = deterministic first.
+- **Retroactive linking** happens on `create-org` **and** on linking an existing org whose domain is newly added — backfill past meetings by domain.
+- **Dedup unchanged**: `calendar_event_id` = `meetingDedupKey(...)`; one row per logical meeting across attendees.
+- **Idempotency / stickiness**: scan adds links, never removes; never overwrite an Admin-assigned/manual link; never disturb a dispatched/in-flight/non-calendar meeting.
+- **`initials` is `NOT NULL`** on `clients` — derive it in every create path (e.g. first letters of the name).
+
+---
+
+## 10. Out of scope (do NOT build)
+- Manual "dispatch a bot now" / auto-join for leads/prospects.
+- Calendar write-back to Outlook; Resend email (P7); any change to the merged P5b generation/webhook.
+- Fuzzy/NLP matching. Merging/deduping orgs. A full CRM for leads (just the `type` + create/link/promote basics).
+- Deleting orphaned MinIO blobs left by the seed-data wipe (storage cruft; ignore).
+
+---
+
+## 11. Acceptance (all must pass before opening the PR)
+- `pnpm -w typecheck` + `pnpm -w lint` + `pnpm --filter web build` all pass.
+- Migration `0004` applied to the shared dev DB (coordinated with orchestrator); `clients.type`, `client_domains`, `meeting_clients`, `meetings.is_internal`, `meetings.external_attendees`, the GA `internal` org, and `settings.internal_email_domains` all present.
+- With the worker running against a real (or crafted-test) calendar:
+  - An **external** meeting is ingested and linked to the org whose `client_domains` domain matches an attendee; a **two-external-domain** meeting links to **both** orgs (multi-client).
+  - A meeting whose external domain matches **no** org is ingested as **unassigned** with the unknown domain surfaced.
+  - **Create client / lead from a domain** creates the org + domain, links this meeting, and **retroactively links** another past meeting on the same domain.
+  - A **GA-only** meeting is ingested, tagged **Internal**, linked to the GA org, and is **bot-eligible**.
+  - **Bot eligibility**: with the kill-switch ON in a test, only **client** and **internal** meetings with a join URL and a non-opted-out lead dispatch; leads/prospects/unassigned do **not**. (Leave the prod kill-switch **OFF** when done.)
+- Leads/prospects/internal do **not** appear in the client roster list, cadence, or the ambiguous-assign picker.
+- Branch + **PR for review** (not `main`); `git status` shows no secrets staged (`git check-ignore` the env/SECRETS files).
+
+## 12. Escalate (stop + ask the orchestrator) if
+- Applying `0004` to the shared Supabase is risky/unclear, or a type regen churns unrelated types.
+- The many-to-many change would force a change to the merged P5b generation/webhook or the worker queue factories.
+- Free-email/internal-domain policy or the primary-`client_id` denormalization conflicts with something you find in `docs/04`/`docs/09` — confirm, don't guess.
+- Real Graph calendar reads 403 after policy propagation (pre-existing P4 escalation path).