feat(dashboard): cross-link candidates/rollouts and consolidate the overlapping diff & file tabs - #253
feat(dashboard): cross-link candidates/rollouts and consolidate the overlapping diff & file tabs#253OsherElhadad wants to merge 3 commits into
Conversation
…diff/file tabs Closes #139. The DeepDive had ten top-level tabs and no cross-linking: four separate file/diff surfaces the reader had to disambiguate before knowing which one answered "what did this edit change", and no way to get from "candidate cand_0002 regressed" to the edit or the failing task. Consolidation (10 tabs -> 7). Iterations / Git diffs / Memory / Files are now sub-modes of one "Changes & files" area, ordered by distance from the edit itself: the candidate diff, the real commit, the optimizer's memory of it, the raw run dir. Every panel still mounts, so no backend field loses its reader -- including rejected.jsonl / history.jsonl via MemoryPanel, which #212 confirmed are live consumers. "Overview" became "Fitness" and now carries both candidate-selection charts, since those are the cross-link sources rather than a tab spent on one chart. Cross-links. Selecting a candidate on the fitness curve goes to its rollouts, and one click further to its diff, preselected to the same id; a heatmap cell opens that exact rollout's drawer, resolved through the rollouts index rather than by guessing a filename. All of it is URL state (?tab / ?candidate / ?task / ?mode), so a cross-link is one navigation, the back button undoes it, and "look at this candidate" is a shareable link. New: TaskHeatmap. The tasks x iterations grid has existed in the single-file static dashboard since day one but never in the SPA, which left the backend's GraphNode.per_task and GraphNode.feedback with no SPA reader at all. This is that reader, and the cross-link source the issue asks for. Built as a table of buttons rather than SVG rects so cell activation is keyboard-native. Accessibility, given the two failures this epic already caught: - Tabs follow the WAI-ARIA pattern -- one tab stop per tablist (roving tabindex), Left/Right/Home/End between tabs, native Enter/Space. - The active tab is bold + aria-selected, not colour alone; heatmap cells carry a glyph and a worded aria-label ("pass"/"fail"/"partial"/"not run"), not colour alone. - The rollout drawer's backdrop is aria-hidden and unfocusable (it was a focusable <button>, which is exactly how focus lands behind a panel, #196), Tab cycles inside the panel, Escape closes, and focus returns to the opener -- or to the surrounding tabpanel when a cross-link already unmounted the opener, rather than dropping to <body>.
…d tabs The consolidation test pinned the exact top-level tab array, which failed the merge with #204 for adding an Events tab -- a reason unrelated to whether the four diff/file surfaces were consolidated. Assert the subset that #139 owns and that the four merged surfaces are absent; arrow-key nav is asserted by position rather than by tab name for the same reason.
skillberry-bot
added
dashboard
|
🏷️ Automatic Labeling I've analyzed this pull request and added the following labels:
These labels were selected based on the PR title, description, and changed files. If you believe any labels are incorrect or missing, feel free to adjust them manually. |
🔬 EvidenceEvery command and its complete output. Worktree 1. Setup$ git worktree add /tmp/wt-139 -b feat/issue-139-spa-crosslink origin/main
Preparing worktree (new branch 'feat/issue-139-spa-crosslink')
HEAD is now at 21fe3e49 fix(ci): keep the plan job's log line out of $GITHUB_OUTPUT (#242)2. A real zero-API toy_calc + mock run$ REPO=/tmp/wt-139
$ export CAPEVOLVE_CORE=$REPO/core PYTHONPATH=$REPO/core CAPEVOLVE_SKILLS_DIR=$REPO/skills
$ export CAPEVOLVE_TOY_DATA=$REPO/examples/toy_calc CAPEVOLVE_MOCK_SCRIPT=$REPO/examples/toy_calc/mock_script.json
$ D=/tmp/toy139; mkdir -p $D/.capevolve/project/adapters
$ cp $REPO/examples/toy_calc/adapter.py $D/.capevolve/project/adapters/
$ cp -R $REPO/examples/toy_calc/capability $D/seed_capability
$ cp $REPO/templates/project/capevolve.yaml $D/.capevolve/project/capevolve.yaml
$ cd $D && python -m cap_evolve.cli run --spec $D/.capevolve/project/capevolve.yaml --project $D/.capevolve/project --run-ts demo
{"dashboard": "http://127.0.0.1:7878"}
{
"run_dir": ".capevolve/run_demo",
"best_id": "cand_0001",
"baseline_val": 0.0,
"test_reward": 1.0,
"test_baseline_reward": 0.0,
"test_delta": 1.0,
"test_pass_k": {"1": 1.0, "2": 0.0},
"iterations": 3,
"dashboard": ".capevolve/run_demo/dashboard.html",
"dashboard_server": "http://127.0.0.1:7878"
}The run graph — note $ python -m capevolve_dashboard.export_static --base .capevolve --run-id run_demo --out /tmp/toy139/data
wrote 92 JSON files to /tmp/toy139/data
$ python -c "...print each node..."
tasks: ['a1', 'a4']
seed 0 seed 0.0 {'a1': 0.0, 'a4': 0.0} {'a1': "expected '7' but agent produced 'I think 3 + 4 is roughly some number.'; the prompt likely lacks an explicit instruction to compute and output only the number", 'a4': "expected '10' but agent produced 'I think 9 + 1 is roughly some number.'; the prompt likely lacks an explicit instruction to compute and output only the number"}
cand_0001 1 accepted 1.0 {'a1': 1.0, 'a4': 1.0} {'a1': 'correct', 'a4': 'correct'}
cand_0002 2 rejected 1.0 {'a1': 1.0, 'a4': 1.0} {'a1': 'correct', 'a4': 'correct'}
cand_0003 3 rejected 1.0 {'a1': 1.0, 'a4': 1.0} {'a1': 'correct', 'a4': 'correct'}3. Serving the built SPA against that real run dir$ (cd dashboard/frontend && npm run build) # verify only; dist reverted afterwards
$ CAPEVOLVE_BASE_DIR=/tmp/toy139/.capevolve CAPEVOLVE_STATIC_DIR=$REPO/dashboard/frontend/dist \
python -m uvicorn capevolve_dashboard.asgi:app --host 127.0.0.1 --port 7979
$ curl -s http://127.0.0.1:7979/api/health
{"ok":true,"base_dir":"/tmp/toy139/.capevolve"}
$ curl -s http://127.0.0.1:7979/api/runs
[{"run_id":"run_demo","path":"/tmp/toy139/.capevolve/run_demo","algorithm":null,"status":"done","best_val":1.0,"baseline_val":0.0,"delta_pct":null,"iterations":3,"total_usd":0.0,"mtime":1785458150.909568}]4. Every view and every cross-link, driven headlessly (playwright/chromium) against that serverfull rendered output per view (click)5. Keyboard navigation, in fullRoving tabindex, Arrow/Home/End moving selection and focus, the URL following, the back button undoing a cross-link, Space activating a heatmap cell, Shift+Tab staying inside the drawer: $ node kbd139.mjs
ArrowRight -> selected=Cost focused=Cost url=?tab=cost
ArrowRight -> selected=Phases focused=Phases url=?tab=phases
End -> selected=Insights focused=Insights url=?tab=insights
Home -> selected=Fitness focused=Fitness url=?tab=fitness
ArrowLeft -> selected=Insights focused=Insights url=?tab=insights
roving tabindex now: ["-1","-1","-1","-1","-1","-1","0"]
back button undoes the cross-link:
after link : ?tab=trajectories&candidate=cand_0001 | tab = Trajectories
after back : ?tab=fitness | tab = Fitness
Space activates a heatmap cell (no mouse):
drawer open: 1 | file: a4__seed__t0.json
Shift+Tab stays inside drawer:
{ inDrawer: true, tag: 'BUTTON' }Visible focus ring, verified by a real $ node ring139.mjs
first heatmap cell reached by Tab:
{
"label": "a1 at iteration 0 (seed): fail, reward 0.000 — exp",
"matchesFocusVisible": true,
"outline": "2px solid rgb(59, 130, 246)",
"offset": "2px"
}
first tab button reached by Tab:
{
"tab": "Fitness",
"matchesFocusVisible": true,
"outline": "2px solid rgb(59, 130, 246)",
"fontWeight": "600"
}
6. This branch: full suites$ PYTHONPATH=/tmp/wt-139/core python -m pytest core/tests -q
........................................................................ [ 40%]
........................................................................ [ 80%]
................................... [100%]
179 passed in 63.95s (0:01:03)
$ python -m compileall -q core dashboard
compileall=0 # no output = clean
$ npx tsc -b --noEmit
tsc exit=0
$ npm run build
dist/index.html 0.54 kB │ gzip: 0.34 kB
dist/assets/index-Dk2GgY8b.css 17.33 kB │ gzip: 4.69 kB
dist/assets/index-BPHKpzS8.js 838.39 kB │ gzip: 252.44 kB
✓ built in 739ms
$ npm test
Test Files 14 passed (14)
Tests 54 passed (54)
$ git status --porcelain -- dashboard/frontend/dist
$ git status --porcelain -- dashboard/frontend/dist | wc -l
07. Merged tree — #204 + #234 + #218 + #221 appliedControl first (the four PRs merged together without this one), so the number means something: $ git worktree add /tmp/wt-139-ctl -b tmp/ctl-139 origin/main
$ for b in 117-event-ticker 138-phase-pipeline 118-stall-detection 130-plateau-detection; do git merge origin/feat/issue-$b; done
feat/issue-117-event-ticker => clean
feat/issue-138-phase-pipeline => CONFLICT: core/cap_evolve/cli.py
feat/issue-118-stall-detection => CONFLICT: CHANGELOG.md core/cap_evolve/cli.py
core/cap_evolve/eventstream.py
dashboard/backend/capevolve_dashboard/app.py
docs/GETTING_STARTED.md
feat/issue-130-plateau-detection => clean (after resolving the above)
$ PYTHONPATH=/tmp/wt-139-ctl/core python -m pytest core/tests -q
32 failed, 274 passed, 2 warnings in 87.71s (0:01:27)Now with #139 on top of the same four: $ git worktree add /tmp/wt-139-merged -b tmp/merged-139 feat/issue-139-spa-crosslink
$ for b in 117-event-ticker 138-phase-pipeline 118-stall-detection 130-plateau-detection; do git merge origin/feat/issue-$b; done
feat/issue-117-event-ticker => CONFLICT: dashboard/frontend/src/routes/RunDeepDive.tsx
(one hunk: the TABS array — #204 adds Events, #139
renames overview->fitness. Resolved by keeping BOTH.)
feat/issue-138-phase-pipeline => CONFLICT: core/cap_evolve/cli.py <- same as control
feat/issue-118-stall-detection => CONFLICT: CHANGELOG.md core/cap_evolve/cli.py
core/cap_evolve/eventstream.py
dashboard/backend/capevolve_dashboard/app.py
docs/GETTING_STARTED.md <- same as control
feat/issue-130-plateau-detection => clean
$ PYTHONPATH=/tmp/wt-139-merged/core python -m pytest core/tests -q
32 failed, 274 passed, 2 warnings in 88.47s (0:01:28)
$ cd dashboard/frontend && npx tsc -b --noEmit
tsc=0
$ npm run build
✓ built in 503ms
$ npm test
Test Files 17 passed (17)
Tests 89 passed (89)Read: The merged tree's seven-plus-Events tab bar, after resolving: const TABS: TabDef[] = [
{ id: 'fitness', label: 'Fitness' }, // #139
{ id: 'events', label: 'Events' }, // #204
{ id: 'cost', label: 'Cost' },
{ id: 'phases', label: 'Phases' },
{ id: 'lineage', label: 'Lineage' },
{ id: 'trajectories', label: 'Trajectories' },
{ id: 'changes', label: 'Changes & files' }, // #139: iterations+git+memory+files
{ id: 'insights', label: 'Insights' },
]#234's 8. Orphaned-data check$ grep -rn "rejected\|history" dashboard/frontend/src/lib/insights.ts dashboard/frontend/src/components/MemoryPanel.tsx | head
lib/insights.ts: export function deadEnds(rejected: MemoryRejected[]) <- #212's live reader, untouched
components/MemoryPanel.tsx: data?.history.map(...) <- still mounted (mode=memory)
components/MemoryPanel.tsx: data?.rejected.map(...) <- still mounted (mode=memory)
# per_task / feedback before this PR: read ONLY by the single-file static dashboard
$ grep -rn "per_task" dashboard/frontend/src/ # on origin/main
(no matches)
$ grep -rn "per_task" core/cap_evolve/dashboard.py | head -2
1020: const iters=G.nodes.filter(n=>Object.keys(n.per_task||{}).length)...
# after this PR
$ grep -rn "per_task" dashboard/frontend/src/
components/TaskHeatmap.tsx: it.per_task?.[t] <- the new reader9. Files touched$ git diff --stat origin/main...HEAD
.../frontend/src/components/BestCurveChart.tsx | 47 +++-
dashboard/frontend/src/components/ChangesPanel.tsx | 64 +++++
.../frontend/src/components/IterationsDiff.tsx | 25 +-
dashboard/frontend/src/components/MemoryPanel.tsx | 18 +-
dashboard/frontend/src/components/TaskHeatmap.tsx | 167 +++++++++++++
dashboard/frontend/src/components/Trajectories.tsx | 163 +++++++++++--
dashboard/frontend/src/components/ui/Tabs.tsx | 71 +++++-
dashboard/frontend/src/routes/RunDeepDive.tsx | 108 +++++++--
dashboard/frontend/src/test/CrossLink.test.tsx | 262 +++++++++++++++++++++
9 files changed, 865 insertions(+), 60 deletions(-) |
🔍 Review — PR #253Verdict: CHANGES REQUESTED — one blocking defect. The consolidation claim holds: I independently verified all four merged panels mount and render real data against a live run dir, and no field lost a reader. The a11y claims all reproduced verbatim in a real headless Chromium. The merged-tree control is byte-identical without #139. But Blocking1.
Three concrete consequences:
Note the sibling paths are already right: Fix — make close a URL operation, same as open. // RunDeepDive.tsx, trajectories branch
<Trajectories … onCloseRollout={() => goto({ task: null })} />
// Trajectories.tsx:166
{openFile && <RolloutDrawer … onClose={() => { setOpenFile(null); onCloseRollout?.() }} />}
Non-blocking2. 2500 tab presses to pass the widget. It is not literally a trap (Tab does eventually exit, landing on Fix: the WAI-ARIA grid pattern the 3. Sub-mode switch has no accessible name and announces nothing beyond the tab label. Both tablists carry no naming attributes: Two nested tablists with identical roles and no 4. 5.
Nits6. 7. 8. Did consolidation lose anything?No. I did not take the PR's list; I enumerated every removed tab's panel component, grepped for its importers in the new tree, and drove each sub-mode against a real zero-API
Independent orphan audit. I extracted all 94 fields declared in Nothing became orphaned. The two newly-read fields. 8/8 exact. Rows worst-first, columns Discoverability verdict — "one extra click" is honest for Iterations/Git diffs, understated for Memory and Files. For the diff surfaces the merge is a clear win: they genuinely overlapped, the sub-mode ordering (diff → commit → memory → bytes, URL stateDeep links restore the exact view — all four probed against the served SPA:
Back button undoes exactly one navigation. Two chained cross-links, then two No per-keystroke history stacking. Three tab clicks → Hostile params degrade safely — zero injection, zero crashes, zero console errors: Path traversal in #254 — the PR was right to file it, but its "shareable" claim is not accurate as written today. Reproduced on the live backend: Fixing it means adding an SPA catch-all to A11y — what I measured myselfReal headless Chromium against the served SPA, not asserted. Every claim reproduced: Selection is bold ( #196 fix confirmed. The backdrop is now a Focus never lands on My extra probes:
Merged-tree controlYes — byte-identical. I built both trees myself with the same conflict-resolution strategy and diffed the failure names, not just the counts. The four, identical in both: Note: I measured 4/302, not the PR's 32/274. That gap is a resolution artefact, not a disagreement about #139 — I resolved the The merged Merge-order noteTrial-merged the proposed order end to end ( Concur — #139 last. It is the only branch in the set whose sole conflict is one frontend file, and putting it last means every sibling's Two additions for whoever merges: (a) the
|
…p a real grid Review fixes for #139. BLOCKING — `?task` was written on drawer open and never cleared on close, so the URL recorded a state the user had dismissed: a link copied after closing reopened the drawer for the recipient, `?task` leaked onto every unrelated tab, and closing produced no history entry. `onCloseRollout={() => goto({ task: null })}` makes closing a URL operation, exactly as `IterationsDiff`/`MemoryPanel` already treat `?candidate`. - TaskHeatmap: WAI-ARIA grid instead of one tab stop per cell. A 50x50 run was 2500 tab stops (2500 presses to pass the widget); now `role="grid"`/`role="gridcell"`, one roving tab stop, and Arrow/Home/End/PageUp/PageDown move the active cell. Cells use aria-disabled rather than disabled so the arrow walk has no holes. - Tabs: an accessible `label` on the tablist and `aria-labelledby` on the panel, so the two nested tablists are distinguishable to a screen reader ("Run views" / "Change surfaces"). - Tabs: an unknown `?tab`/`?mode` is now normalised out of the URL instead of rendering the fallback while the bad param sits in the link. - RunDeepDive: the cross-link headers validate `?candidate` against the graph before naming it, so a stale link no longer claims a candidate the panel isn't showing. - Renamed the consolidated tab to "Changes, memory & files" — Memory and Files were never diff surfaces, so a label saying only "changes" hid them. - BestCurveChart: a `<details>` that is never closed is a heading wearing a disclosure triangle; use an actual heading when the table must stay open. Tests: 61 vitest (was 54) — the drawer-close lifecycle, `?mode` deep-linking, unknown `?tab` normalisation, a nonexistent `?candidate`, the 600-cell single-tab-stop assertion, arrow movement, and both tablist names.
🔧 Review fixesCommit The blocking defect — all three symptoms fixed
1. A copied link no longer reopens what the sharer dismissed — Other params audited for the same write-without-clear asymmetry —
Nit 6 ( The 50×50 heatmap — measured
2500 tab stops → 1. Escape count 2500 → 1. And the keyboard cross-link still works end to end on the real run dir, from an arrowed-to cell: Numbered response to all 8 findings1. BLOCKING — 2. 50×50 heatmap keyboard trap. Fixed as a WAI-ARIA grid, measurements above. Two tests: a 30×20 (600-cell) assertion that 3. Both tablists unnamed.
4. Unknown 5. Header named a candidate the panel wasn't showing. Both cross-link headers now render 6. 7. 8. Stale Taking the labelling suggestion: the consolidated tab is renamed "Changes, memory & files". The reviewer is right that "one extra click" was understated for Memory and Files — they were never diff surfaces, and a label naming only "changes" hid 22KB of optimizer memory two levels down. The docstring and the four test references were updated with it.
|
Closes #139.
The DeepDive had ten top-level tabs and no cross-linking. Four of those tabs were
different file/diff surfaces the reader had to disambiguate before knowing which one
answered "what did this edit change", and there was no path from "cand_0002 regressed"
to either the edit or the failing task.
Tab decisions
?tab=fitness—BestCurveChart+ the newTaskHeatmap?tab=changes&mode=candidate—IterationsDiff, unchanged,+candidateprop?tab=changes&mode=commit—GitDiff, unchanged?tab=changes&mode=memory—MemoryPanel, unchanged,+candidateprop?tab=changes&mode=files—FileTree, unchanged?tab=fitness— newTaskHeatmapNothing was removed, and no tab lost its panel. All four surfaces still mount, as
sub-modes, in increasing distance from the edit itself: the diff → the commit → the
optimizer's memory of it → the bytes on disk. Ten tabs → seven, with four sub-modes.
Why consolidate rather than delete
Deleting any of the four would orphan a backend field. Consolidating orphans nothing —
verified below. In particular
MemoryPanelis the live reader ofrejected.jsonlandhistory.jsonlthat #212 confirmed must not lose its consumer, so it is a sub-mode,not a casualty. The one thing #234's evidence header genuinely made redundant is a
top-level tab spent on a single chart — so "Overview" became "Fitness" and earned its
place by carrying the two candidate-selection charts, which the header cannot replace.
No orphaned data — every field still has a reader
/diff/{candidate}IterationsDiffmode=candidate/git/log,/git/diffGitDiffmode=commit/memory→history/rejected(history.jsonl,rejected.jsonl)MemoryPanelandinsights.ts::deadEnds(#212)mode=memoryand the Insights tab/candidate/{cid}/filesMemoryPanelmode=memory/tree,/fileFileTreemode=files/rollouts,/rollout/{file}Trajectories,Insights/rolloutsgraph.nodes[].per_task,graph.nodes[].feedbackdashboard.pyTaskHeatmapsummary.tasksdashboard.pyonlyTaskHeatmapNet: zero fields orphaned, two previously-orphaned fields gained a reader. No backend
field is marked for removal by this PR.
Cross-link map
All cross-links are URL state, so one link = one navigation, the back button undoes it,
and "look at this candidate" is a shareable URL.
BestCurveChartscatter dot?tab=trajectories&candidate=<id>— that candidate's rollouts onlyBestCurveChartcandidate table<id>changed →"?tab=changes&mode=candidate&candidate=<id>— diff preselected<id>scored per task"?tab=trajectories&candidate=<id>TaskHeatmapcell?tab=trajectories&candidate=<cid>&task=<t>→ opens that rollout's drawerThe heatmap cell resolves
(task, candidate)through the/rolloutsindex rather thanguessing a filename, and switches the split selector to wherever the rollout actually
lives. When a per-task score exists but its trajectory file wasn't kept, it says so
instead of opening an empty drawer.
Accessibility
This epic has already caught colour-only accept/reject chips (#204) and focus escaping
behind a drawer (#196), so:
measured
["0","-1","-1","-1","-1","-1","-1"]— then Left/Right/Home/End move selectionand focus together. Enter/Space are native
<button>activation.aria-selected, not colour. Everyheatmap cell carries a glyph (
✓ ✗ ~ ·) and a wordedaria-label(
"a1 at iteration 0 (seed): fail, reward 0.000 — expected '7' but agent produced …"),and the legend names each outcome in words. Same icon+word contract as
StatusBadge.outline: 2px solid rgb(59,130,246)with a 2pxoffset on tabs, heatmap cells and cross-links, reached by real
Tab(
matches(':focus-visible') === true).<button>— exactlyhow focus lands behind an open panel. It is now
aria-hiddenand unfocusable; focusmoves into the panel on open, Tab/Shift+Tab cycle inside it, Escape closes it, and focus
returns to the opener — or, when a cross-link already unmounted the opener, to the
surrounding
tabpanelrather than dropping to<body>.<table>of<button>cells with a<caption>, scoped headersand per-cell labels — not SVG rects — so cell activation is keyboard-native rather than
re-implemented.
Merge order & merged-tree result
Expected order — this PR is happiest last among the SPA PRs:
#194 → #212 → #218 → #221 → #204 → #234 → #139.
Merged locally with #204 + #234 + #218 + #221 all applied:
pytest core/teststsc -b --noEmitvite buildvitestThe 32 python failures are pre-existing in the sibling merge and identical without this
PR — they come from resolving the #204↔#218 conflict in
eventstream.py/app.py/cli.pyby hand (those PRs disagree about the SSE offset and the tail exit codes, andeach owns half). #139 contributes zero conflicts and zero regressions: its only
merge conflict is one hunk of the
TABSarray against #204's Events tab, resolved bykeeping both.
On this branch alone: 179 passed / 179, 0 failed and 54 vitest passed / 54.
dashboard/frontend/dist/is NOT committed (#188) — built to verify, then reverted;git status -- dashboard/frontend/distis empty.Verification
Exercised against a real zero-API
examples/toy_calc+mockrun(
baseline_val 0.0 → test_reward 1.0, 3 iterations,cand_0001accepted,cand_0002/cand_0003rejected), served by the real FastAPI backend against the real run dir withthe freshly-built SPA. Full command log and complete output in the 🔬 Evidence
comment; the load-bearing excerpts:
Known pre-existing issue (not this PR)
A hard load of a deep path (
/runs/<id>) 404s on the live backend:app.pymountsStaticFileswith no SPA fallback, so client-side routes only resolve when entered from/. Unrelated to this change (the static export usesHashRouterand is unaffected), soit is left alone here — the evidence run enters via the Hub, which also exercises that
link. Worth its own issue.