Merge v0.2.2-1

.env.default

@@ -41,6 +41,8 @@ POSTGRES_STATEMENT_TIMEOUT=100000
 
 DELETE_AFTER_DAYS=30
 
+YJS_ENABLED=true
+
 DEV_BUILD_CONTEXT=
 
 JWT_SECRET=

docker-compose.yml

@@ -31,6 +31,7 @@ services:
       AI_LLM_TPM: ${DOCKER_COMPOSE_APP_ENV_AI_LLM_API_TPM}
       AI_LLM_TPD: ${DOCKER_COMPOSE_APP_ENV_AI_LLM_API_TPD}
       JWT_SECRET: ${JWT_SECRET}
+      YJS_ENABLED: ${YJS_ENABLED:-true}
 
       TESTING_PLAYWRIGHT_WS_ENDPOINT: "ws://playwright:9323"
       TESTING_PLAYWRIGHT_BASE_URL: "http://app:4200"
@@ -59,7 +60,7 @@ services:
       - postgres_data:/var/lib/postgresql/data/pgdata
 
   playwright:
-    image: mcr.microsoft.com/playwright:v1.57.0-noble
+    image: mcr.microsoft.com/playwright:v1.58.2-noble
     container_name: playwright
     depends_on:
       - app

openspec/changes/yjs-introduction/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-02-15

openspec/changes/yjs-introduction/proposal.md

@@ -0,0 +1,71 @@
+## Why
+
+The current real-time collaboration system uses individual Socket.io messages per operation (addNode, updateNode, removeNode, etc.), each validated against the PostgreSQL database. In larger teams, clients frequently drift out of sync — e.g., a client sends an update for a node that another client already deleted, causing FK constraint violations, full-map-state reloads, and a degraded experience. There are many edge cases where local state diverges from server state, and the current last-write-wins approach with per-operation DB validation cannot resolve them reliably. Introducing Yjs (a CRDT library) as a sync layer eliminates these conflicts by design — concurrent changes merge deterministically without coordination.
+
+Additionally, the new Yjs WebSocket backend needs hardening for production stability. An audit revealed critical gaps: no error handler on individual connections (risking process crashes), no heartbeat mechanism (causing zombie connection accumulation), no message size or connection limits (enabling resource exhaustion), and several race conditions in connection lifecycle management.
+
+## What Changes
+
+### Yjs Introduction
+- **Replace Socket.io data sync with Yjs sync protocol**: All node/map operations (addNodes, updateNode, removeNode, applyMapChangesByDiff) stop being individual Socket.io messages. Instead, clients share a Y.Doc that syncs via y-websocket over a standard WebSocket connection.
+- **Replace Socket.io presence with Yjs Awareness**: Client list, colors, and node selection tracking move from custom Socket.io events to the Yjs Awareness API.
+- **Add server-side Y.Doc management**: The backend hosts Y.Doc instances per active map, loaded from DB on first client connect, evicted after last client disconnects.
+- **Add decode-on-save persistence**: A debounced persistence service reads the Y.Doc, extracts nodes, and writes them to the existing MmpNode/MmpMap tables in a transaction.
+- **Rewrite MapSyncService as a Y.Doc bridge**: The frontend MapSyncService becomes a two-way bridge — MMP events write to Y.Doc, Y.Doc observations apply remote changes to MMP.
+- **Remove per-operation server-side validation**: The DB is no longer the real-time arbiter. Yjs handles merge correctness. The DB becomes a persistence target written to periodically.
+- **Remove Socket.io dependency for data operations**: Socket.io is fully replaced by y-websocket (data sync) and Yjs Awareness (presence). **BREAKING** for any clients relying on the current Socket.io event protocol.
+- **Auth moves to WebSocket handshake**: The modification secret is verified when the WebSocket connection is established, not per-message. Read-only clients can receive state but not send updates.
+- **MMP library stays unchanged**: MMP keeps its internal state, rendering, and event system. It is not aware of Yjs.
+- **DB schema stays unchanged**: MmpMap and MmpNode tables, all migrations, and the REST API remain as-is.
+- **History/undo stays as-is (Phase 1)**: MMP's snapshot-based history continues to work locally. Undo/redo diffs are no longer broadcast over the network. Replacing with Y.UndoManager is deferred to a future change.
+
+### WebSocket Hardening
+- Add `ws.on('error')` handler to prevent unhandled errors from crashing the process
+- Implement ping/pong heartbeat to detect and terminate zombie connections
+- Set `maxPayload` on WebSocketServer to prevent memory exhaustion from oversized messages
+- Add per-IP and global connection limits on WebSocket upgrades
+- Add connection setup timeout for slow/unresponsive database scenarios
+- Implement `OnModuleDestroy` in `YjsPersistenceService` to clear pending timers on shutdown
+- Fix unawaited `decrementClientCount` in connection close handler
+- Fix grace timer / client count increment race condition in `handleConnection`
+- Unify client count tracking to derive from the connection set (single source of truth)
+- Make `deleteMap` properly async with awaited database calls
+
+## Non-goals
+
+- Document compaction / CRDT tombstone cleanup (separate optimization effort)
+- Awareness cleanup race on REST-triggered map deletion (existing defensive handling is sufficient)
+- Explicit listener cleanup replacing implicit `doc.destroy()` behavior (low risk)
+- Stale snapshot on rapid reconnect during grace period (self-correcting via debounced persistence)
+- Frontend WebSocket client changes (beyond Yjs migration)
+- Load balancing or horizontal scaling
+- Replacing MMP's internal state management or history system (Phase 2)
+- Storing Y.Doc binary state in the database (future optimization)
+- Offline-first support or local-first persistence
+
+## Capabilities
+
+### New Capabilities
+- `yjs-sync`: Y.Doc-based real-time synchronization — server-side Y.Doc lifecycle (load, sync, evict), y-websocket integration with NestJS, and the frontend WebsocketProvider setup.
+- `yjs-persistence`: Decode-on-save persistence — debounced extraction of Y.Doc state into existing MmpNode/MmpMap tables, and hydration of Y.Doc from DB rows on first connect.
+- `yjs-awareness`: Presence and selection tracking via Yjs Awareness API — client colors, connected client list, and node selection highlighting.
+- `yjs-bridge`: MapSyncService rewrite — two-way bridge between MMP events and Y.Doc, replacing all Socket.io event handlers.
+- `yjs-ws-connection-resilience`: Error handling on individual connections, ping/pong heartbeat for zombie detection, and connection setup timeout.
+- `yjs-ws-resource-protection`: Message size limits (`maxPayload`), per-IP and global connection rate limiting, and concurrent connection caps.
+- `yjs-ws-lifecycle-integrity`: Persistence service graceful shutdown, awaited async operations, grace timer race fix, unified client count tracking, and async `deleteMap`.
+
+### Modified Capabilities
+<!-- No existing specs to modify — openspec/specs/ is empty -->
+
+## Impact
+
+- **Frontend MapSyncService** (`map-sync.service.ts`): Heavy rewrite. All Socket.io event setup, error handling, and operation methods replaced with Y.Doc observation and writes. Socket.io client dependency removed.
+- **Backend MapsGateway** (`maps.gateway.ts`): Heavy rewrite. All `@SubscribeMessage` handlers for node operations replaced by Yjs WebSocket handler. Socket.io server dependency removed.
+- **Backend MapsService** (`maps.service.ts`): Simplified. Per-operation validation logic largely removed. `deleteMap` made async with awaited repository call.
+- **Backend YjsGateway** (`yjs-gateway.service.ts`): Error handler, heartbeat, connection limits, timeout, await fixes.
+- **Backend YjsDocManager** (`yjs-doc-manager.service.ts`): Client count refactored to derive from connection set.
+- **Backend YjsPersistence** (`yjs-persistence.service.ts`): `OnModuleDestroy` added for graceful shutdown.
+- **Dependencies**: Add `yjs`, `y-websocket`, `y-protocols`. Remove `socket.io`, `socket.io-client` (for data ops). Remove `cache-manager` usage for client tracking (Awareness replaces it). No new runtime dependencies for hardening.
+- **REST API**: Unchanged. Map CRUD, export/import endpoints continue to work against the same DB tables.
+- **APIs**: WebSocket connections may be rejected with HTTP 429/503 when limits are exceeded. Connections will receive periodic pings. Oversized messages will cause connection termination.
+- **Testing**: New unit tests for error handling, heartbeat, connection limits, timeout, shutdown, and lifecycle race conditions. E2E tests updated for WebSocket transport change.

openspec/changes/yjs-introduction/specs/yjs-awareness/spec.md

@@ -0,0 +1,64 @@
+## ADDED Requirements
+
+### Requirement: Client presence via Yjs Awareness
+Each connected client SHALL announce its presence using the Yjs Awareness protocol. The awareness state SHALL include the client's assigned color and current node selection. The awareness state SHALL be set when the client connects and updated when the selection changes.
+
+#### Scenario: Client sets initial awareness state on connect
+- **WHEN** a client connects to a map via the WebsocketProvider
+- **THEN** the client SHALL set its awareness state with `{ color: <assigned-color>, selectedNodeId: null }`
+
+#### Scenario: Client updates awareness on node selection
+- **WHEN** a user selects a node in the map
+- **THEN** the client SHALL update its awareness state to `{ color: <color>, selectedNodeId: <node-id> }`
+
+#### Scenario: Client updates awareness on node deselection
+- **WHEN** a user deselects a node
+- **THEN** the client SHALL update its awareness state to `{ color: <color>, selectedNodeId: null }`
+
+### Requirement: Client color assignment
+Each client SHALL be assigned a random color from the existing `COLORS` palette on connection. If the chosen color collides with another connected client's color (detected via awareness states), a random fallback color SHALL be generated.
+
+#### Scenario: Client picks a unique color
+- **WHEN** a client connects and its randomly chosen color is not used by any other client
+- **THEN** the client SHALL use that color in its awareness state
+
+#### Scenario: Client color collision
+- **WHEN** a client connects and its randomly chosen color matches another client's awareness color
+- **THEN** the client SHALL generate a random hex color as a fallback
+
+### Requirement: Client list derived from awareness states
+The frontend SHALL derive the list of connected clients and their colors from `awareness.getStates()`. There SHALL be no separate server-side client tracking (the `cache-manager` client cache is removed).
+
+#### Scenario: New client appears in the list
+- **WHEN** a new client connects and sets its awareness state
+- **THEN** all other clients SHALL receive an awareness change event and update their client list to include the new client's color
+
+#### Scenario: Client disconnects and disappears from list
+- **WHEN** a client disconnects
+- **THEN** the Yjs Awareness protocol SHALL automatically remove the client's state and all other clients SHALL receive a change event to update their client list
+
+### Requirement: Node selection highlighting from awareness
+The frontend SHALL observe awareness state changes and highlight nodes that other clients have selected. When a remote client's `selectedNodeId` changes, the local client SHALL call MMP's `highlightNode` method with the remote client's color.
+
+#### Scenario: Remote client selects a node
+- **WHEN** a remote client's awareness state changes to `{ selectedNodeId: "node-1" }`
+- **THEN** the local client SHALL highlight "node-1" with the remote client's color via `mmpService.highlightNode()`
+
+#### Scenario: Remote client deselects a node
+- **WHEN** a remote client's awareness state changes from `{ selectedNodeId: "node-1" }` to `{ selectedNodeId: null }`
+- **THEN** the local client SHALL remove the highlight from "node-1" (or apply the next client's color if another client still has it selected)
+
+#### Scenario: Remote client selects a node that no longer exists locally
+- **WHEN** a remote client's awareness state references a node ID that does not exist in the local MMP instance
+- **THEN** the local client SHALL ignore the highlight update without error
+
+### Requirement: Connection status from WebSocket provider
+The frontend SHALL derive the connection status (`connected` / `disconnected`) from the `WebsocketProvider`'s connection state, replacing the previous Socket.io-based connection tracking.
+
+#### Scenario: Connection established
+- **WHEN** the WebsocketProvider successfully connects
+- **THEN** the connection status observable SHALL emit `'connected'`
+
+#### Scenario: Connection lost
+- **WHEN** the WebSocket connection is interrupted
+- **THEN** the connection status observable SHALL emit `'disconnected'`

openspec/changes/yjs-introduction/specs/yjs-bridge/spec.md

@@ -0,0 +1,93 @@
+## ADDED Requirements
+
+### Requirement: Local MMP events write to Y.Doc
+When the user performs an action in MMP (create, update, remove node), the MapSyncService bridge SHALL write the change to the local Y.Doc. The bridge SHALL NOT send individual network messages — Yjs handles synchronization automatically.
+
+#### Scenario: User creates a node
+- **WHEN** MMP fires a `nodeCreate` event with the new node's `ExportNodeProperties`
+- **THEN** the bridge SHALL create a new Y.Map entry in `yDoc.getMap('nodes')` with the node's ID as key and all properties as values
+
+#### Scenario: User updates a node property
+- **WHEN** MMP fires a `nodeUpdate` event with the updated property and value
+- **THEN** the bridge SHALL update the corresponding property on the node's Y.Map entry in the `nodes` map
+
+#### Scenario: User removes a node
+- **WHEN** MMP fires a `nodeRemove` event with the removed node's properties
+- **THEN** the bridge SHALL delete the node's entry from `yDoc.getMap('nodes')`
+
+#### Scenario: User pastes multiple nodes
+- **WHEN** MMP fires a `nodePaste` event with an array of node properties
+- **THEN** the bridge SHALL add all nodes to the Y.Doc `nodes` map within a single `yDoc.transact()` call
+
+#### Scenario: User updates map options
+- **WHEN** the user changes map options (e.g., map name)
+- **THEN** the bridge SHALL update the corresponding fields in `yDoc.getMap('mapOptions')`
+
+### Requirement: Remote Y.Doc changes apply to MMP
+The bridge SHALL observe the Y.Doc for remote changes and apply them to MMP using its existing API. Remote changes SHALL be applied with `notifyWithEvent: false` to prevent echo loops.
+
+#### Scenario: Remote node added
+- **WHEN** the Y.Doc `nodes` map emits an add event from a remote transaction
+- **THEN** the bridge SHALL call `mmpService.addNode()` with the node properties and `notifyWithEvent: false`
+
+#### Scenario: Remote node property updated
+- **WHEN** the Y.Doc `nodes` map emits an update event for an existing node from a remote transaction
+- **THEN** the bridge SHALL call `mmpService.updateNode()` with the changed property, value, `notifyWithEvent: false`, and the node ID
+
+#### Scenario: Remote node removed
+- **WHEN** the Y.Doc `nodes` map emits a delete event from a remote transaction
+- **THEN** the bridge SHALL call `mmpService.removeNode()` with the node ID and `notifyWithEvent: false`, but only if the node exists in MMP
+
+#### Scenario: Remote map options updated
+- **WHEN** the Y.Doc `mapOptions` map emits a change event from a remote transaction
+- **THEN** the bridge SHALL call `mmpService.updateAdditionalMapOptions()` with the updated options
+
+### Requirement: Echo prevention
+The bridge SHALL prevent echo loops where a local MMP event writes to Y.Doc, which then triggers a Y.Doc observation that would re-apply the same change to MMP. The bridge SHALL use `transaction.local` on Y.Doc observations to distinguish local from remote changes and only apply remote changes to MMP.
+
+#### Scenario: Local change does not echo back
+- **WHEN** the user creates a node locally (MMP event → Y.Doc write)
+- **THEN** the Y.Doc observer SHALL detect the change as `transaction.local === true` and SHALL NOT apply it back to MMP
+
+#### Scenario: Remote change is applied
+- **WHEN** a remote client creates a node (Y.Doc sync update received)
+- **THEN** the Y.Doc observer SHALL detect the change as `transaction.local === false` and SHALL apply it to MMP
+
+### Requirement: Map import via Y.Doc transaction
+When a user imports a map, the bridge SHALL clear the Y.Doc `nodes` map and repopulate it with the imported data inside a single `yDoc.transact()` call. Yjs SHALL sync the new state to all connected clients automatically.
+
+#### Scenario: User imports a map
+- **WHEN** the user triggers a map import with new map data
+- **THEN** the bridge SHALL execute a Y.Doc transaction that deletes all entries from the `nodes` map and adds all imported nodes as new entries
+- **AND** all connected clients SHALL receive the Y.Doc update and re-render via their bridge observers
+
+### Requirement: Map deletion handling
+When a map is deleted via the REST API, the server SHALL destroy the Y.Doc and close all WebSocket connections for that map. The frontend SHALL detect the WebSocket close and handle cleanup.
+
+#### Scenario: Admin deletes a map
+- **WHEN** the admin calls the map deletion REST endpoint
+- **THEN** the server SHALL destroy the in-memory Y.Doc for that map and close all associated WebSocket connections
+
+#### Scenario: Client detects map deletion
+- **WHEN** the client's WebSocket connection is closed by the server due to map deletion
+- **THEN** the frontend SHALL handle the disconnection (e.g., redirect to home or show a notification)
+
+### Requirement: Initial map load via Y.Doc
+When a user opens a map, the frontend SHALL receive the map state through the Yjs sync protocol instead of a Socket.io `join` response. After the Y.Doc syncs, the bridge SHALL extract all nodes and initialize MMP.
+
+#### Scenario: User opens an existing map
+- **WHEN** the WebsocketProvider connects and the Y.Doc syncs with the server
+- **THEN** the bridge SHALL read all entries from the `nodes` map, convert them to `ExportNodeProperties[]`, and call `mmpService.new(snapshot)` to initialize the map
+
+### Requirement: Local undo/redo stays local
+MMP's existing undo/redo (snapshot-based history) SHALL continue to function for local changes. Undo/redo diffs SHALL NOT be broadcast to other clients. Other clients will see the result of the undo/redo as normal Y.Doc changes.
+
+#### Scenario: User performs undo
+- **WHEN** the user triggers undo in MMP
+- **THEN** MMP SHALL redraw from its history snapshot and the bridge SHALL write the resulting node changes to the Y.Doc (adds, updates, deletes from the undo diff)
+- **AND** other clients SHALL receive these as normal node changes, not as an undo operation
+
+#### Scenario: User performs redo
+- **WHEN** the user triggers redo in MMP
+- **THEN** MMP SHALL redraw from its history snapshot and the bridge SHALL write the resulting node changes to the Y.Doc
+- **AND** other clients SHALL receive these as normal node changes, not as a redo operation