-
Notifications
You must be signed in to change notification settings - Fork 1
Feat 83 recording (socket profiler recording & replay) #200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
theultimatewasteofspace
wants to merge
155
commits into
dev
Choose a base branch
from
feat-83-recording
base: dev
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
155 commits
Select commit
Hold shift + click to select a range
7bdc7dc
feat: add recording dashboard UI and nav element migration
2dbd5a1
feat: add recording dashboard component
b788500
feat: rename recording to socket profiler
3117dae
feat: add recording and trace tables with models
be58e84
feat: update socket profiler dashboard with subscribeTable and correc…
de3f283
feat: implement socket recorder with onAny listeners and real-time tr…
6e19b77
feat: add deleted and deletedAt columns to trace table
add5478
fix: format timestamps, remove stop button from table rows
535a693
feat: add persistent recording bar to dashboard
d98c81d
feat: add save recording modal with trace selection
862885c
fix: recordings persist after reload, set deleted false on create
7bb50f1
refactor: remove catch-all listeners by reference instead of nuking all
becee70
fix: default deleted to false on recording and trace models
c24ac5a
chore: stop tracking root logs directory
cf906f8
refactor: collapse trace migrations and use proper timestamps
daf9445
fix: set publicTable false on recording and trace, move deleted defau…
17c9b8d
refactor: remove redundant sendToast calls and unused return from sta…
a1fb6e2
refactor: use BasicButton in recording modal for behavior tracking
41c6a27
feat: add edit recording modal with trace pruning
38b0bda
feat: capture multi-user events across all connected sockets
c5d691c
feat: multi-user recording with participant selection and online indi…
b1966c3
Merge remote-tracking branch 'origin/dev' into feat-83-recording
6bbf384
feat: add replay engine with results modal
e9780eb
feat: add event exclusion filter and update modals to use BasicTable
ddc98ac
feat: rename 'Level' to 'Iteration' in replay UI
89339c0
docs: note coupling between replay HMAC and session secret
a5f866d
fix: recover interrupted recordings, await save operations
8e991d9
feat: tag traces with socketId for per-session granularity
2376e83
fix: correct selected count, show session count in start modal
ccd234a
feat: replay groups traces by socketId for per-session granularity
ecac28f
feat: pure session-based recording selection
cd41288
feat: persistent recording icon replaces global recording bar
dae7a5b
feat: replay configuration modal with pooled scaling
7aa5b2d
Merge branch 'dev' into feat-83-recording
d796191
fix: move Socket Profiler nav element into Manage group
8bd3b00
feat: live elapsed-time display on recording icon + restore click nav…
e7f6a7b
chore: ignore .DS_Store files
6fc56cb
fix: close mounted() method properly in Topbar.vue
58fb9c2
feat: add Time and Elapsed columns to recording traces modal
a11a8e0
style: fix Vue attribute-order lint warnings
e50f23f
chore: remove leftover debug console.log from replay worker
13e13e4
Merge remote-tracking branch 'origin/dev' into feat-83-recording
f3ff342
feat: export and import recordings as JSON
bc66320
feat: configurable ack timeout for replay
a550780
style: clean up Server.js indentation, JSDoc, and trailing whitespace
df75b95
chore: sync backend package-lock with nodemon devDependency
88f24ef
feat: auto-download replay results as JSON
acc0eb7
fix: warm-up delay before replay to avoid first-trace race
31bcbae
refactor: use BasicForm for replay config fields
5bdb0c1
feat: strip dbChanges from downloaded replay results
8c86eaa
fix: show recording icon only on sessions being recorded
0541846
feat: chronological sequence column in recording traces modal
96e6d52
fix: populate Connected time and mark current tab in session picker
a171e8b
feat: help tooltips for replay config fields
5819b40
fix: replace replay warm-up delay with server ready signal
f4ee14c
fix: filter stale sockets from online session list
8afcab4
feat: flag recording disconnected on participant drop; debounce uncap…
b87add7
feat: live-update session picker as connections come and go
00b255c
fix: include payload in getTraces so exports carry it
f66753d
fix: set participantSocketIds on imported recordings
b5a6f3c
fix: name exported recording file after the recording
b0e5150
Add integrity guard to recording export
f03d43f
Add validators to recording and trace models
b62cdd3
Document trace direction boolean in migration
f26d260
Require admin for start/stop recording
1f6e5c4
Avoid N+1 query when building replay session pool
c5eecef
Log session cleanup failures instead of swallowing
ea4f4a1
Fix direction arrow in recording trace label
3e41e64
Fix crash in unsubscribeAppData on unknown subscription id
3518f34
Add progress bar to replay runs
e8513b2
Recorder: per-socket recordings instead of one server-wide recording
6f6fd29
Socket profiler: broadcast stopped recordings + frontend per-socket stop
9065276
Socket profiler: fix recording icon for concurrent per-socket recordings
d33102e
Replay results: collapsible iteration overview + exclude recorder con…
45bafa0
Recorder: require admin for trace and session read handlers
502eb00
Replay results: two-level table navigation (iterations -> sessions ->…
a0f5f10
Replay progress: per-trace granularity instead of per-iteration
a66a121
Replay progress: throttle emits via lodash
0c27fc2
perf: CLI load-test scaffold — auth, recording resolution, regression…
9c6e82c
Replay engine: latency-threshold stop + single-level mode
98b1582
perf: add ramp, ceiling, and inspect modes
695994c
perf: add soak mode
255c49b
perf metrics: expose backend health + pg-stats via socket handlers
392dccb
perf metrics: sampler + wire into soak
7aede14
perf metrics: add DB pool-waiting stop to ceiling
263c045
perf regression: folder suite support + per-story verdict
e4d61a2
Merge remote-tracking branch 'origin/dev' into feat-83-recording
12ff306
perf: per-trace-type diagnostic stats across all modes
b0a6faa
perf: memory-vs-trace correlation in soak + vitals in ramp
bd7d252
perf: surface Postgres stats in all load modes + memory in ceiling
e764013
perf: save results to disk (JSON + readable text) across all modes
a3aca96
feat: replay recordings from files without db import
0049745
feat: read perf admin password from flag, env, or prompt
823ce43
refactor: inspect shows plain trace distribution without classification
c2afb4a
feat: make replay target url configurable via env
f21c3cc
feat: report node heap alongside rss in load modes
8ce4263
feat: nest replay results drill-down as child modals
c7cf59f
refactor: address review findings in replayer
d35f3df
refactor: address review findings in recorder
da9ce48
fix: correct scope, session cleanup and secret handling in replay int…
167a1be
fix: restrict recording table access and correct schema definitions
6ba489d
fix: correct recording status validation, foreign keys and start claims
9a07d8d
fix: address re-review findings across recorder, replayer and replay …
1935b84
fix: address review findings in socket profiler frontend
1956b5e
fix: remove dead import path and guard file loading in perf CLI
0a2350b
fix: refresh recordings table on import via remount workaround
641a1b2
chore: untrack committed .DS_Store files and fix gitignore
a1102d9
fix: reject replay runs whose peak concurrency exceeds the limit
079eb1c
refactor: consolidate socket profiler nav migration into manage group
8ffda3c
fix: validate session secret before inserting session row
fc255a1
refactor: rename verifyAdmin to verifyAuthenticatedSession
4f305d6
fix: surface readable report write failures
e623126
fix: continue inspection when one recording fetch fails
d8ef2fa
fix: stop output capture on early return and key stories by stable id
e150a33
fix: clean up progress listener and sampler on all exit paths
78412cc
fix: clean up sampler and output capture on all exit paths
02f8a04
fix: guarantee sampler cleanup, wire pool baseline, reuse printCulprit
fffc0f3
fix: validate latency threshold and read ceiling exit code
bcf6f83
fix: reset modal open flag on hide event
8eb85d8
Merge remote-tracking branch 'origin/dev' into feat-83-recording
b42fcdf
refactor: rename loginAsAdmin to login
492ae87
chore: move perf scripts into perf subdirectory
b70c2bb
chore: restructure perf scripts into modes and utils
b69327f
fix: save perf results under the backend log directory
612d90a
docs: describe expected object contents in perf jsdoc
3597cad
docs: complete missing perf jsdoc params
728fcf7
refactor: move replay helpers into backend utils
e6f77b8
refactor: return trace rows directly instead of remapping
4bde2e6
fix: restrict session change broadcast to admins
c98d823
refactor: move recording recovery helpers into utils
e9b1413
fix: await first metric sample so pool baseline is populated
5d9f96b
docs: add missing perf jsdoc
0a7b3b6
docs: correct parseDuration return type
226e6b8
feat: replay regression sessions sequentially for deterministic ids
7467bca
fix: accept --dir as a standalone recording source in perf cli
c2d0197
feat: show replay progress during regression runs
5ab3d69
feat: replay file sessions as their recorded user
0c40f61
fix: report recorded user per story and hide unused maxIterations
4570905
refactor: move replay session helpers into backend utils
4bb209b
refactor: reuse a single raw recordings computed in socket profiler
4ba35bd
feat: import recordings through a single transactional handler
decbeae
fix: fail regression when a story is skipped for a missing user
dd646a4
fix: only echo maxIterations for modes that use it
67b6b08
refactor: drop duplicate recording broadcast on disconnect
730c34c
fix: restore buffers in trace payloads before replay
a295d62
fix: read session identity from this.userId when saving a study template
e207bad
fix: remap recorded hashes to replayed ones during replay
e4544e2
fix: share observed hashes across sessions and map them from recorded…
2fc3a3e
chore: remove hash remapping debug output
e9a31c5
feat: report unacknowledged traces separately from failures
9024c82
feat: wire up unused perf cli options
de2892c
test: add recorded user story regression suite
eda41fb
chore: mark regression stories as generated
355b4cf
docs: add socket profiler documentation
31e2f15
Merge remote-tracking branch 'origin/dev' into feat-83-recording
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.createTable('recording', { | ||
| id: { | ||
| type: Sequelize.INTEGER, | ||
| primaryKey: true, | ||
| autoIncrement: true, | ||
| allowNull: false, | ||
| }, | ||
| name: { | ||
| type: Sequelize.STRING, | ||
| allowNull: true, | ||
| }, | ||
| status: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| defaultValue: 'recording', | ||
| }, | ||
| startTime: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| }, | ||
| endTime: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| // Recordings are soft-deleted only, so a user delete must never cascade | ||
| // them away. RESTRICT forces whoever adds user deletion to soft-delete | ||
| // the recordings first rather than silently destroying recorded data. | ||
| userId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: false, | ||
| references: { | ||
| model: 'user', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'RESTRICT', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| participantSocketIds: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| excludeEvents: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| deleted: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| deletedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| createdAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| updatedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| }); | ||
| // publicTable is false, so getAutoTable filters these rows by userId. | ||
| await queryInterface.addIndex('recording', ['userId'], { | ||
| name: 'recording_userId_idx', | ||
| }); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.dropTable('recording'); | ||
| }, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| 'use strict'; | ||
|
|
||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.createTable('trace', { | ||
| id: { | ||
| type: Sequelize.INTEGER, | ||
| primaryKey: true, | ||
| autoIncrement: true, | ||
| allowNull: false, | ||
| }, | ||
| recordingId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: false, | ||
| references: { | ||
| model: 'recording', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'CASCADE', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| // SET NULL would leave the recording intact but silently unreplayable | ||
| // (groupTracesBySocket skips traces with no userId). RESTRICT keeps trace | ||
| // ownership honest; recordings are soft-deleted, never orphaned. | ||
| userId: { | ||
| type: Sequelize.INTEGER, | ||
| allowNull: true, | ||
| references: { | ||
| model: 'user', | ||
| key: 'id', | ||
| }, | ||
| onDelete: 'RESTRICT', | ||
| onUpdate: 'CASCADE', | ||
| }, | ||
| socketId: { | ||
| type: Sequelize.STRING, | ||
| allowNull: true, | ||
| }, | ||
| action: { | ||
| type: Sequelize.STRING, | ||
| allowNull: false, | ||
| }, | ||
| payload: { | ||
| type: Sequelize.JSONB, | ||
| allowNull: true, | ||
| }, | ||
| // true = frontend -> backend (onAny), false = backend -> frontend (onAnyOutgoing) | ||
| direction: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| }, | ||
| startTime: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| }, | ||
| endTime: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| }, | ||
| deleted: { | ||
| type: Sequelize.BOOLEAN, | ||
| allowNull: false, | ||
| defaultValue: false, | ||
| }, | ||
| deletedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: true, | ||
| defaultValue: null, | ||
| }, | ||
| createdAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| updatedAt: { | ||
| type: Sequelize.DATE, | ||
| allowNull: false, | ||
| defaultValue: Sequelize.fn('NOW'), | ||
| }, | ||
| }); | ||
|
|
||
| // Every trace lookup filters by recordingId (getTraces, buildSessionPool, | ||
| // stopRecording), so that's the index that matters. | ||
| await queryInterface.addIndex('trace', ['recordingId'], { | ||
| name: 'trace_recordingId_idx', | ||
| }); | ||
|
|
||
| await queryInterface.addIndex('trace', ['socketId'], { | ||
| name: 'trace_socketId_idx', | ||
| }); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.dropTable('trace'); | ||
| }, | ||
| }; | ||
50 changes: 50 additions & 0 deletions
50
backend/db/migrations/20260506220447-extend-nav-socket-profiler.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| 'use strict'; | ||
|
|
||
| const navElements = [ | ||
| { | ||
| name: "Socket Profiler", | ||
| groupId: "Manage", | ||
| icon: "record-circle", | ||
| order: 16, | ||
| admin: true, | ||
| path: "Socket_Profiler", | ||
| component: "SocketProfiler", | ||
| } | ||
| ]; | ||
|
|
||
| /** @type {import('sequelize-cli').Migration} */ | ||
| module.exports = { | ||
| async up(queryInterface, Sequelize) { | ||
| await queryInterface.bulkInsert( | ||
| "nav_element", | ||
| await Promise.all( | ||
| navElements.map(async (t) => { | ||
| const groupId = await queryInterface.rawSelect( | ||
| "nav_group", | ||
| { | ||
| where: { name: t.groupId }, | ||
| }, | ||
| ["id"] | ||
| ); | ||
|
|
||
| t["createdAt"] = new Date(); | ||
| t["updatedAt"] = new Date(); | ||
| t["groupId"] = groupId; | ||
|
|
||
| return t; | ||
| }), | ||
| {} | ||
| ) | ||
| ); | ||
| }, | ||
|
|
||
| async down(queryInterface, Sequelize) { | ||
| await queryInterface.bulkDelete( | ||
| "nav_element", | ||
| { | ||
| name: navElements.map((t) => t.name), | ||
| }, | ||
| {} | ||
| ); | ||
| }, | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| "use strict"; | ||
|
theultimatewasteofspace marked this conversation as resolved.
|
||
| const MetaModel = require("../MetaModel.js"); | ||
|
|
||
| module.exports = (sequelize, DataTypes) => { | ||
| // Allowed recording lifecycle states. Single source of truth for the | ||
| // status validator below; add new states here rather than scattering | ||
| // string literals. "interrupted" is set on startup for recordings whose | ||
| // server died mid-capture. | ||
| const RECORDING_STATUSES = ["recording", "finished", "disconnected", "interrupted"]; | ||
|
|
||
| class Recording extends MetaModel { | ||
| // MetaModel flags. autoTable registers this model for generic CRUD | ||
| // over sockets. fields=[] means no column whitelist (all columns | ||
| // returned). publicTable=false keeps recordings out of unauthenticated | ||
| // reads: they are admin-only artifacts, and the rows expose who was | ||
| // recorded, their session ids and their activity times. | ||
| static autoTable = true; | ||
| static fields = []; | ||
| static publicTable = false; | ||
|
|
||
| static associate(models) { | ||
| Recording.belongsTo(models["user"], { | ||
| foreignKey: "userId", | ||
| as: "user", | ||
| }); | ||
| Recording.hasMany(models["trace"], { | ||
| foreignKey: "recordingId", | ||
| as: "traces", | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| Recording.init( | ||
| { | ||
| name: DataTypes.STRING, | ||
| status: { | ||
| type: DataTypes.STRING, | ||
| allowNull: false, | ||
| validate: { isIn: [RECORDING_STATUSES] }, | ||
| }, | ||
| startTime: { | ||
| type: DataTypes.DATE, | ||
| allowNull: false, | ||
| }, | ||
| endTime: DataTypes.DATE, | ||
| userId: { | ||
| type: DataTypes.INTEGER, | ||
| allowNull: false, | ||
| }, | ||
| participantSocketIds: DataTypes.JSONB, | ||
| excludeEvents: DataTypes.JSONB, | ||
| deleted: DataTypes.BOOLEAN, | ||
| deletedAt: DataTypes.DATE, | ||
| createdAt: DataTypes.DATE, | ||
| updatedAt: DataTypes.DATE, | ||
| }, | ||
| { | ||
| sequelize, | ||
| modelName: "recording", | ||
| tableName: "recording", | ||
| } | ||
| ); | ||
|
|
||
| return Recording; | ||
| }; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| "use strict"; | ||
| const MetaModel = require("../MetaModel.js"); | ||
|
|
||
| module.exports = (sequelize, DataTypes) => { | ||
| class Trace extends MetaModel { | ||
| // MetaModel flags. autoTable registers this model for generic CRUD | ||
| // over sockets. fields=[] means no column whitelist. publicTable=false | ||
| // keeps traces out of the generic subscription layer, since payloads | ||
| // can hold sensitive event data — they're fetched explicitly instead. | ||
| static autoTable = true; | ||
| static fields = []; | ||
| static publicTable = false; | ||
|
|
||
| static associate(models) { | ||
| Trace.belongsTo(models["recording"], { | ||
| foreignKey: "recordingId", | ||
| as: "recording", | ||
| }); | ||
| Trace.belongsTo(models["user"], { | ||
| foreignKey: "userId", | ||
| as: "user", | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| Trace.init( | ||
| { | ||
| recordingId: { | ||
| type: DataTypes.INTEGER, | ||
| allowNull: false, | ||
| }, | ||
| userId: DataTypes.INTEGER, | ||
| socketId: DataTypes.STRING, | ||
| action: { | ||
| type: DataTypes.STRING, | ||
| allowNull: false, | ||
| }, | ||
| payload: DataTypes.JSONB, | ||
| direction: { | ||
| type: DataTypes.BOOLEAN, // true = frontend -> backend, false = backend -> frontend | ||
| allowNull: false, | ||
| }, | ||
| startTime: { | ||
| type: DataTypes.DATE, | ||
| allowNull: false, | ||
| }, | ||
| endTime: DataTypes.DATE, | ||
| deleted: DataTypes.BOOLEAN, | ||
| deletedAt: DataTypes.DATE, | ||
| createdAt: DataTypes.DATE, | ||
| updatedAt: DataTypes.DATE, | ||
| }, | ||
| { | ||
| sequelize, | ||
| modelName: "trace", | ||
| tableName: "trace", | ||
| } | ||
| ); | ||
|
|
||
| return Trace; | ||
| }; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.