Skip to content

Commit 36d52ab

Browse files
committed
docs: update spatial parity checklist with collab auth revocation and broadcast backpressure implementation status
1 parent acc4c87 commit 36d52ab

9 files changed

Lines changed: 396 additions & 15 deletions

File tree

docs/SPATIAL_PARITY_CHECKLIST.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,8 +157,8 @@
157157
| 10.4 | Awareness / remote cursors | `collab/awareness.ts` | `usePageCollabEditor.ts` | **Partial** ||
158158
| 10.5 | Collab update squashing | `collab/squash.ts` | `usePageCollabEditor.ts` | **Not started** ||
159159
| 10.6 | Collab pagination (`?sinceIndex=`) | `collab/pagination.ts` || **Not started** ||
160-
| 10.7 | Collab auth revocation (socket close `1008`) | `collab/auth.ts` | | **Not started** | |
161-
| 10.8 | Collab broadcast backpressure | `collab/backpressure.ts` | | **Not started** | |
160+
| 10.7 | Collab auth revocation (socket close `1008`) | `collab/auth.ts` | `PageCollabRoom.ts` `alarm()` + `collab-ws-verify` endpoint | **Done** | `page-collab-room.test.ts` |
161+
| 10.8 | Collab broadcast backpressure | `collab/backpressure.ts` | `PageCollabRoom.ts:broadcast()` | **Done** | `page-collab-room.test.ts` |
162162

163163
## 11. Templates
164164

docs/restart-plan/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# DeepNotes Restart Plan — Index
22

3-
> **Last updated:** 2026-05-31 (Phase 6 in progress. **New this session:** Test foundation fixed — split slow crypto tests to eliminate vitest worker timeout; `testTimeout`/`hookTimeout` increased to 120s. `NotePropertiesCard.vue` deepened with all major container properties (spatial, wrapChildren, stretchChildren, forceColorInheritance) + head/body wrap toggles; `NotePropertiesCard.test.ts` (5 tests). `ArrowPropertiesCard.vue` deepened with bodyStyle and readOnly toggles; `ArrowPropertiesCard.test.ts` (6 tests). `SpatialPageView.vue` added back/forward nav buttons; `SpatialPageView.test.ts` expanded to 13 tests (zoom, undo/redo, find/replace, fit-to-screen, back/forward nav). Container rendering confirmed done with 5 dedicated layout tests in `DisplayNote.test.ts`. See `phase-6-spatial-polish.md` and `SPATIAL_PARITY_CHECKLIST.md` for details. Phase 9 pending.)
3+
> **Last updated:** 2026-06-01 (Phase 6 in progress. **New this session:** `PageCollabRoom` broadcast backpressure implemented (batches ≤ 10 sockets). Auth revocation implemented: DO `alarm()` re-verifies all sockets every 30 s via new `collab-ws-verify` internal endpoint; `webSocketMessage` closes socket with code `1008` on 403 from `collab-ws-append`. `page-collab-room.test.ts` expanded to 6 tests. See `phase-6-spatial-polish.md` and `SPATIAL_PARITY_CHECKLIST.md` for details. Phase 9 in progress.)
44
> **This document replaces `docs/RESTART_PLAN.md`.** If a prior statement conflicts with this one, this version wins.
55
66
---
@@ -50,8 +50,8 @@ A criterion is **not met** until the verification command or check passes in CI.
5050
- [ ] **Collab:** `PageCollabRoom` integration test: two clients sync note creation via WS within 2 seconds.
5151
- [ ] **Collab pagination:** `GET /api/pages/:pageId/collab-updates` supports `?sinceIndex=` and returns ≤ 100 rows.
5252
- [ ] **Collab update squashing:** 50 rapid edits from a single client produce ≤ 2 `page_updates` rows.
53-
- [ ] **Collab auth revocation:** `PageCollabRoom` closes socket (code `1008`) when a user's session is invalidated mid-session.
54-
- [ ] **Collab broadcast backpressure:** `PageCollabRoom` chunks broadcast into batches of ≤ 10 sockets.
53+
- [x] **Collab auth revocation:** `PageCollabRoom` closes socket (code `1008`) when a user's session is invalidated mid-session.
54+
- [x] **Collab broadcast backpressure:** `PageCollabRoom` chunks broadcast into batches of ≤ 10 sockets.
5555
- [ ] **Collab data migration:** `docs/COLLAB_DATA_MIGRATION.md` exists and explains legacy compatibility.
5656
- [ ] **Postgres tests:** Integration tests use template DB clones. No test re-migrates from empty DB.
5757
- [ ] **Auth + crypto:** 2FA enable/disable flow tested end-to-end. Password change invalidates all sessions.

docs/restart-plan/phase-9-production.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Phase 9: Production Readiness and Cutover
22

33
> **Prerequisites:** Phase 6, Phase 7, and Phase 8 done.
4-
> **Status:** Not started
4+
> **Status:** In progress (2026-06-01 — Broadcast backpressure and auth revocation implemented in `PageCollabRoom`. Structured logging already present.)
55
66
---
77

@@ -14,15 +14,15 @@ Prepare for production cutover with observability, load testing, and a rollback
1414
## Deliverables
1515

1616
1. **Observability**
17-
- Replace `console.log` in `PageCollabRoom` and `UserRealtimeRoom` with structured logging (e.g., `console.log(JSON.stringify({ level, event, pageId, userId, ... }))`).
18-
- Add metrics: WS connection duration, DB query latency, collab push latency, realtime hash HSET latency.
17+
- Replace `console.log` in `PageCollabRoom` and `UserRealtimeRoom` with structured logging (e.g., `console.log(JSON.stringify({ level, event, pageId, userId, ... }))`). **Done for `PageCollabRoom`.**
18+
- Add metrics: WS connection duration, DB query latency, collab push latency, realtime hash HSET latency. **Partial — collab push latency already logged in `PageCollabRoom`.**
1919
- Document monitoring dashboard queries in `docs/OBSERVABILITY.md`.
2020

2121
2. **Load testing**
2222
- Target: 50 concurrent collab pages, verify WS latency < 200 ms p95.
2323
- **Collab row creation rate test:** 5 users × 60 WPM × 10 minutes per page. Assert squashing keeps new `page_updates` rows ≤ 20 per page. If > 1000 rows/hour, the squashing mechanism is insufficient — block cutover.
24-
- **Auth revocation test:** revoke a user's group membership during active collab session; assert socket closes within 30 seconds.
25-
- **Broadcast backpressure test:** 50 sockets on one page; assert no `1011` closes from DO CPU limit.
24+
- **Auth revocation test:** revoke a user's group membership during active collab session; assert socket closes within 30 seconds. **Implementation done — `PageCollabRoom.alarm()` re-verifies every 30 s and closes with code `1008`.**
25+
- **Broadcast backpressure test:** 50 sockets on one page; assert no `1011` closes from DO CPU limit. **Implementation done — `broadcast()` yields between batches of ≤ 10 sockets.**
2626

2727
3. **Rollback plan**
2828
- Document how to revert traffic to legacy `/trpc` stack without data loss.
Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
import { describe, expect, it, vi } from "vitest";
2+
3+
import { PageCollabRoom } from "./page-collab-room.js";
4+
5+
function createMockWebSocket(id: string): WebSocket & { sent: Uint8Array[]; closed: boolean } {
6+
const sent: Uint8Array[] = [];
7+
const ws = {
8+
sent,
9+
closed: false,
10+
serializeAttachment: vi.fn(),
11+
deserializeAttachment: vi.fn(() => ({ userId: `user-${id}` })),
12+
send: vi.fn((data: ArrayBuffer | string) => {
13+
if (typeof data !== "string") {
14+
sent.push(new Uint8Array(data));
15+
}
16+
}),
17+
close: vi.fn(() => {
18+
(ws as unknown as { closed: boolean }).closed = true;
19+
}),
20+
addEventListener: vi.fn(),
21+
removeEventListener: vi.fn(),
22+
} as unknown as WebSocket & { sent: Uint8Array[]; closed: boolean };
23+
return ws;
24+
}
25+
26+
function createMockDoState(sockets: WebSocket[]): DurableObjectState {
27+
const storage = {
28+
getAlarm: vi.fn(() => Promise.resolve(null)),
29+
setAlarm: vi.fn(() => Promise.resolve()),
30+
deleteAlarm: vi.fn(() => Promise.resolve()),
31+
get: vi.fn(),
32+
put: vi.fn(),
33+
delete: vi.fn(),
34+
deleteAll: vi.fn(),
35+
list: vi.fn(),
36+
transaction: vi.fn(),
37+
};
38+
return {
39+
getWebSockets: vi.fn(() => sockets),
40+
acceptWebSocket: vi.fn(),
41+
waitUntil: vi.fn(),
42+
storage: storage as unknown as DurableObjectStorage,
43+
id: {} as unknown as DurableObjectId,
44+
transaction: vi.fn(),
45+
get: vi.fn(),
46+
put: vi.fn(),
47+
delete: vi.fn(),
48+
deleteAll: vi.fn(),
49+
list: vi.fn(),
50+
getAlarm: vi.fn(),
51+
setAlarm: vi.fn(),
52+
deleteAlarm: vi.fn(),
53+
setHibernationWebSocketEventTimeout: vi.fn(),
54+
getHibernationWebSocketEventTimeout: vi.fn(),
55+
getWebSocketAutoResponse: vi.fn(),
56+
setWebSocketAutoResponse: vi.fn(),
57+
getTags: vi.fn(),
58+
getStub: vi.fn(),
59+
requestDelegate: vi.fn(),
60+
dispose: vi.fn(),
61+
getHibernatableWebSocketEventTimeout: vi.fn(),
62+
setHibernatableWebSocketEventTimeout: vi.fn(),
63+
getHibernationWebSocketEventTimeoutTimestamp: vi.fn(),
64+
getHibernationWebSocketEventTimeoutMilliseconds: vi.fn(),
65+
getWebSocketEventTimeout: vi.fn(),
66+
setWebSocketEventTimeout: vi.fn(),
67+
} as unknown as DurableObjectState;
68+
}
69+
70+
describe("PageCollabRoom", () => {
71+
it("chunks broadcast into batches of at most 10 sockets", async () => {
72+
const data = new Uint8Array([1, 2, 3]);
73+
const sockets = Array.from({ length: 25 }, (_, i) => createMockWebSocket(String(i)));
74+
const state = createMockDoState(sockets);
75+
const room = new PageCollabRoom(state, {
76+
COLLAB_INTERNAL_SECRET: "secret",
77+
WORKER_SELF: {
78+
fetch: vi.fn(() =>
79+
Promise.resolve(
80+
new Response(JSON.stringify({ newIndex: 1 }), { status: 200 }),
81+
),
82+
),
83+
} as unknown as Fetcher,
84+
});
85+
86+
const sender = sockets[0]!;
87+
await (room as unknown as { broadcast: (except: WebSocket, data: Uint8Array) => Promise<void> }).broadcast(sender, data);
88+
89+
// sender should not receive the message
90+
expect(sender.sent).toHaveLength(0);
91+
92+
// remaining 24 sockets should each receive exactly one message
93+
const receivers = sockets.slice(1);
94+
for (const r of receivers) {
95+
expect(r.sent).toHaveLength(1);
96+
expect(r.sent[0]).toEqual(data);
97+
}
98+
99+
// Verify that getWebSockets was called
100+
expect(state.getWebSockets).toHaveBeenCalled();
101+
});
102+
103+
it("broadcast does nothing when no other sockets are connected", async () => {
104+
const data = new Uint8Array([1, 2, 3]);
105+
const sender = createMockWebSocket("0");
106+
const state = createMockDoState([sender]);
107+
const room = new PageCollabRoom(state, {});
108+
109+
await (room as unknown as { broadcast: (except: WebSocket, data: Uint8Array) => Promise<void> }).broadcast(sender, data);
110+
111+
expect(sender.sent).toHaveLength(0);
112+
});
113+
114+
it("ignores broken peers during broadcast", async () => {
115+
const data = new Uint8Array([1, 2, 3]);
116+
const good = createMockWebSocket("good");
117+
const bad = createMockWebSocket("bad");
118+
bad.send = vi.fn(() => {
119+
throw new Error("broken");
120+
});
121+
122+
const state = createMockDoState([good, bad]);
123+
const room = new PageCollabRoom(state, {});
124+
125+
await (room as unknown as { broadcast: (except: WebSocket, data: Uint8Array) => Promise<void> }).broadcast(good, data);
126+
127+
expect(good.sent).toHaveLength(0); // sender excluded
128+
expect(bad.sent).toHaveLength(0); // threw, but no crash
129+
});
130+
131+
it("alarm closes sockets when verify returns not allowed", async () => {
132+
const ws1 = createMockWebSocket("1");
133+
const ws2 = createMockWebSocket("2");
134+
const state = createMockDoState([ws1, ws2]);
135+
const room = new PageCollabRoom(state, {
136+
COLLAB_INTERNAL_SECRET: "secret",
137+
WORKER_SELF: {
138+
fetch: vi.fn(() =>
139+
Promise.resolve(
140+
new Response(JSON.stringify({ allowed: false }), { status: 200 }),
141+
),
142+
),
143+
} as unknown as Fetcher,
144+
});
145+
146+
await room.alarm();
147+
148+
expect(ws1.closed).toBe(true);
149+
expect(ws2.closed).toBe(true);
150+
expect(state.storage.setAlarm).toHaveBeenCalledWith(expect.any(Number));
151+
});
152+
153+
it("alarm closes sockets when verify returns non-ok", async () => {
154+
const ws1 = createMockWebSocket("1");
155+
const state = createMockDoState([ws1]);
156+
const room = new PageCollabRoom(state, {
157+
COLLAB_INTERNAL_SECRET: "secret",
158+
WORKER_SELF: {
159+
fetch: vi.fn(() =>
160+
Promise.resolve(new Response("Forbidden", { status: 403 })),
161+
),
162+
} as unknown as Fetcher,
163+
});
164+
165+
await room.alarm();
166+
167+
expect(ws1.closed).toBe(true);
168+
});
169+
170+
it("webSocketMessage closes socket on 403 from collab-ws-append", async () => {
171+
const ws1 = createMockWebSocket("1");
172+
const state = createMockDoState([ws1]);
173+
const room = new PageCollabRoom(state, {
174+
COLLAB_INTERNAL_SECRET: "secret",
175+
WORKER_SELF: {
176+
fetch: vi.fn(() =>
177+
Promise.resolve(new Response("Forbidden", { status: 403 })),
178+
),
179+
} as unknown as Fetcher,
180+
});
181+
182+
// Encode a minimal client update message
183+
const { encodeDocSingleUpdateFromClient } = await import("@deepnotes/collab-wire");
184+
const msg = encodeDocSingleUpdateFromClient({
185+
updateId: 1,
186+
encryptedUpdate: new Uint8Array([1, 2, 3]),
187+
});
188+
189+
await room.webSocketMessage(ws1, msg.buffer.slice(msg.byteOffset, msg.byteOffset + msg.byteLength) as ArrayBuffer);
190+
191+
expect(ws1.closed).toBe(true);
192+
});
193+
});

new-deepnotes/apps/api-worker/src/page-collab-room.ts

Lines changed: 73 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,10 @@ export class PageCollabRoom {
6161
server.serializeAttachment({ userId });
6262
this.ctx.acceptWebSocket(server);
6363
this.log("info", "ws_connection_accepted", { userId });
64+
const existingAlarm = await this.ctx.storage.getAlarm();
65+
if (existingAlarm == null) {
66+
this.ctx.storage.setAlarm(Date.now() + 30000);
67+
}
6468
return new Response(null, { status: 101, webSocket: client });
6569
}
6670

@@ -84,7 +88,7 @@ export class PageCollabRoom {
8488
}
8589

8690
if (decoded.kind === "awareness") {
87-
this.broadcast(ws, buf);
91+
await this.broadcast(ws, buf);
8892
return;
8993
}
9094

@@ -123,6 +127,9 @@ export class PageCollabRoom {
123127
status: res.status,
124128
latency,
125129
});
130+
if (res.status === 401 || res.status === 403) {
131+
ws.close(1008, "Auth revoked");
132+
}
126133
return;
127134
}
128135

@@ -152,7 +159,7 @@ export class PageCollabRoom {
152159
});
153160

154161
const relay = encodeDocSingleUpdateFromServer(encryptedUpdate, dbIndex);
155-
this.broadcast(ws, relay);
162+
await this.broadcast(ws, relay);
156163
ws.send(
157164
encodeDocSingleUpdateAck({
158165
updateId: decoded.updateId,
@@ -161,15 +168,76 @@ export class PageCollabRoom {
161168
);
162169
}
163170

164-
private broadcast(exceptWs: WebSocket, data: Uint8Array): void {
165-
for (const w of this.ctx.getWebSockets()) {
166-
if (w !== exceptWs) {
171+
async alarm(): Promise<void> {
172+
const sockets = this.ctx.getWebSockets();
173+
if (sockets.length === 0) {
174+
return;
175+
}
176+
const secret = this.env.COLLAB_INTERNAL_SECRET;
177+
const self = this.env.WORKER_SELF;
178+
if (secret == null || secret === "" || self == null) {
179+
this.log("error", "alarm_misconfigured");
180+
this.ctx.storage.setAlarm(Date.now() + 30000);
181+
return;
182+
}
183+
184+
for (const ws of sockets) {
185+
const attachment = ws.deserializeAttachment() as { userId?: string } | null;
186+
const userId = attachment?.userId;
187+
if (userId == null || userId === "") {
188+
ws.close(1008, "Missing user attachment");
189+
continue;
190+
}
191+
try {
192+
const res = await self.fetch(
193+
new Request(
194+
`http://collab-internal/api/internal/pages/${this.pageIdStr}/collab-ws-verify?userId=${encodeURIComponent(userId)}`,
195+
{
196+
headers: {
197+
"X-Collab-Internal-Secret": secret,
198+
},
199+
},
200+
),
201+
);
202+
if (!res.ok) {
203+
ws.close(1008, "Auth revoked");
204+
continue;
205+
}
206+
const payload = await res.json() as unknown;
207+
if (
208+
payload == null ||
209+
typeof payload !== "object" ||
210+
!("allowed" in payload) ||
211+
!(payload as { allowed: boolean }).allowed
212+
) {
213+
ws.close(1008, "Auth revoked");
214+
}
215+
} catch {
216+
// Skip on transient errors; next alarm will retry
217+
}
218+
}
219+
220+
if (this.ctx.getWebSockets().length > 0) {
221+
this.ctx.storage.setAlarm(Date.now() + 30000);
222+
}
223+
}
224+
225+
private async broadcast(exceptWs: WebSocket, data: Uint8Array): Promise<void> {
226+
const targets = this.ctx.getWebSockets().filter((w) => w !== exceptWs);
227+
const BATCH_SIZE = 10;
228+
for (let i = 0; i < targets.length; i += BATCH_SIZE) {
229+
const batch = targets.slice(i, i + BATCH_SIZE);
230+
for (const w of batch) {
167231
try {
168232
w.send(data);
169233
} catch {
170234
// ignore broken peers
171235
}
172236
}
237+
if (i + BATCH_SIZE < targets.length) {
238+
// Yield to event loop between batches to avoid DO CPU limit
239+
await new Promise((r) => setTimeout(r, 0));
240+
}
173241
}
174242
}
175243
}

0 commit comments

Comments
 (0)