feat(overflow): interactive overflow analysis tab#124
Merged
Conversation
Embed the alphaDeesp interactive HTML viewer in the Overflow Analysis tab and augment it with Co-Study4Grid–specific behaviour: semantic layer toggles, action pins synchronised with the Action Overview NAD, drill-down to the SLD on double-click, edge / line re-routing in geo mode, and a per-pin filter widget that mirrors the Action Overview filters. See docs/features/interactive-overflow-analysis.md for the full feature contract. Backend: - main.py replaces the static /results/pdf StaticFiles mount with a dynamic GET that injects the overlay block into HTML responses (PDF passthrough preserved). - services/overflow_overlay.py: new module owning the overlay payload — splices a single <style>+<script> block before </body>, inlines the SHARED pinGlyph.js factory (ESM exports stripped), renders pins with anchor priority + CTM-correct coordinate projection, owns the iframe-side filter panel and message router. - services/analysis/overflow_geo_transform.py: tapered-edge body preservation in geo mode (graphviz emits TWO polygons + no path for swapped-flow lines) so swapped flow keeps its tapered body AND arrowhead. - services/network_service.py: voltage-level → substation map helper used by the pin payload builder. - tests/test_overflow_overlay.py + test_overflow_html_dim_logic.py + test_overflow_geo_transform.py: full coverage of the overlay injection, pin click/dblclick listeners, single-click debounce, shared-glyph inlining, no-duplicate-const-SVG_NS guard, action pins filter section placement, pin counter, dim-vs-hide rule, edge-id alignment guard, is_overload ⊂ is_monitored semantics, unsimulated pins (dashed stroke + dim opacity, distinct dblclick route, multi-line pin.title tooltip), and tapered-edge geo rewrite. Frontend: - utils/svg/pinGlyph.js: SHARED plain-JS pin-glyph factory consumed by both actionPinRender.ts AND the iframe overlay. Ships with .d.ts declarations and a parity test against the design tokens. - utils/svg/overflowPinPayload.ts: builds the OverflowPin[] payload with the same anchor priority as the Action Overview NAD pins (load-shed/curtail → VL early-return; primary lines → edge midpoint; VL targets → VL node; max_rho_line last resort) and applies the ActionOverviewFilters (categories / threshold / actionType). buildOverflowUnsimulatedPinPayload renders score- only pins with a multi-line tooltip (buildUnsimulatedPinTitle, exported from actionPinData.ts for cross-module reuse). - components/VisualizationPanel.tsx: 📌 Pins toolbar toggle (disabled until step 2 finishes), iframe ref + message router (cs4g:overlay-ready handshake, cs4g:pins, cs4g:filters, cs4g:pin-clicked, cs4g:pin-double-clicked, cs4g:overflow-unsimulated-pin-double-clicked, cs4g:overflow-filter-changed, cs4g:overflow-layer-toggled, cs4g:overflow-node-double-clicked). - components/SldOverlay.tsx: action sub-tab gating relaxed to (actionDiagram OR vlOverlay.actionId) so the sub-tab is offered when the SLD is opened from an overflow pin double-click. - hooks/useSldOverlay.ts: forceTab override path. - hooks/useDiagrams.ts: re-broadcast of the iframe URL when the overflow graph regenerates. - App.tsx wires overflowPinsEnabled, overflowPins + overflowUnsimulatedPins (the latter gated on overviewFilters.showUnsimulated), allOverflowPins, the pin-click feed-focus + popover handler, the pin-double-click → SLD overlay handler, and the unsimulated-pin double-click → manual simulation handler. - Comprehensive Vitest coverage across pinGlyph, overflowPinPayload (including the unsimulated builder + tooltip), VisualizationPanel message-router, useSldOverlay forceTab, and the spec-conformance table. Parity & docs: - scripts/check_standalone_parity.py declares the new 📌 Pins toggle, the overflow_pins_toggled / overflow_layout_mode_toggled / overflow_pin_clicked / overflow_pin_double_clicked / overflow_layer_toggled / overflow_select_all_layers / overflow_node_double_clicked / overview_unsimulated_pin_simulated / overview_filter_changed gestures, and confirms presence in both the React source and the auto-generated standalone bundle. - docs/features/interactive-overflow-analysis.md: 12-section feature contract covering architecture, source-of-truth tagging, layer-toggle UI, geo-mode SVG transform, overlay injection, action-pin overlay (anchor priority, glyph factory, click semantics, SLD action sub-tab, unsimulated pin layer with multi-line tooltip), filter sync, forwarded interaction events, file map, tests, reset / lifecycle invariants, and verification. Removes the obsolete frontend/dist-standalone/standalone_v0.9.html snapshot — the auto-generated single-file bundle is what ships now. Companion to: - ExpertOp4Grid #75 (taggers + grouped layer toggles upstream). - Expert_op4grid_recommender #88 (wires the lists into the taggers). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the gestures introduced by the interactive overflow analysis tab (PR landed in 4af7bbd) so the replay-agent contract and the session save/load contract stay in sync with the InteractionType union and specConformance.test.ts: - Add the 7 overflow-tab gestures (overflow_layout_mode_toggled, overflow_pins_toggled, overflow_pin_clicked, overflow_pin_double_clicked, overflow_layer_toggled, overflow_select_all_layers, overflow_node_double_clicked) under a new "Overflow Analysis Tab" section, with their details payloads and replay actions. - Document the cross-surface filter events (overview_filter_changed kinds, overview_unsimulated_toggled, overview_unsimulated_pin_simulated) — emitted by both the Action Overview NAD and the Overflow Analysis iframe sidebar. - Document vl_names_toggled (the 🏷 VL labels toggle). - Add wait-point entries for overflow_layout_mode_toggled (iframe refresh) and overview_unsimulated_pin_simulated (pin replacement). - Note in "What is NOT persisted" that overflow-tab UI state (pins toggle, layout mode, layer checkboxes, overviewFilters) is ephemeral and must be replayed via the corresponding events. Save-results doc: switch the example overflow_graph entry to .html to match the current VISUALIZATION_FORMAT default and clarify that pdf_url / pdf_path keep their names for backwards compatibility.
Document Overflow Analysis tab interactions and ephemeral UI state
Tests in test_overflow_html_dim_logic.py shell out to `dot` via pydot; the binary was previously a silent system prerequisite. - Add expert_backend/install_graphviz.py — best-effort installer for Linux (apt/dnf/yum/pacman/apk/zypper), macOS (brew/port), and Windows (choco/winget/scoop). - Add setup.py with post-install/develop hooks invoking the helper. - Expose `costudy4grid-install-graphviz` console script for users whose wheel-based pip install bypasses setup.py hooks. - Add `pydot` to runtime dependencies (used by the overflow graph rendering pipeline). - Install graphviz in GitHub Actions and CircleCI before pytest so the failing overflow-graph tests run with `dot` on PATH. - Document the system prerequisite + manual fallbacks in CONTRIBUTING.md.
Add Graphviz auto-installer and pydot dependency for graph rendering
The previous commit's edit accidentally landed the `quality` group under `[project.scripts]`, which expects string values and broke `pip install ".[quality]"` with a pyproject validation error.
fix(build): restore quality extras under optional-dependencies
….com/marota/Co-Study4Grid into claude/interactive-overflow-analysis
The hard-coded absolute path under ``/home/marotant/...`` made ``TestSmallGridOverflowGraphLayers`` fail on any other checkout (CI, container, contributor laptop). Derive the path from ``Path(__file__).resolve().parents[2]`` so it tracks the actual project root, and add the missing ``pytest.skip`` guard to ``test_svg_data_attrs_consistent_with_titles`` so a fresh checkout without a generated overflow graph skips instead of erroring with ``FileNotFoundError``. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The four parity scripts (Layers 1, 2, 3a, 4) auto-detect their
target standalone in this order:
1. ``frontend/dist-standalone/standalone.html`` (gitignored — only
exists after ``npm run build:standalone``)
2. ``frontend/dist-standalone/standalone_v*.html`` (also gitignored
by the umbrella ``dist-standalone`` rule)
3. ``standalone_interface_legacy.html`` (frozen by design)
In CI, steps 1 and 2 are absent, so every parity check has been
silently scoring the React source against the frozen legacy file —
which lags behind by 11 InteractionType events
(``overflow_layer_toggled``, ``overflow_layout_mode_toggled``, …)
and 4 API paths (``/api/n1-diagram-patch``, etc.). That triggered
``[FAIL]`` in the layer1 report despite the React source being
correct.
Add a Node.js setup + ``npm run build:standalone`` step ahead of
each parity job so the freshly built bundle is the audit target,
matching the local-dev resolution order.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
``check_gesture_sequence.py``'s handler walker only matched the bare ``useCallback(`` wrapper when locating an arrow-function body. The auto-generated standalone bundle emits the namespaced form ``reactExports.useCallback(`` (vite/rollup imports the hook through a namespace object), so handlers like ``handleRunAnalysis`` that the React source defines via ``useCallback`` were not detected on the standalone side. The script then fell back to call-site scanning, whose +2000-char forward window is too narrow to span the ``runAnalysisStep2Stream`` body — missing the ``analysis_step2_completed`` ``recordCompletion`` and producing a spurious gesture-#5 failure. Accept an optional ``<ident>.`` namespace prefix in front of ``useCallback`` so both the bare and dotted forms resolve to the arrow-function body. Local run now reports 30/30 gestures passing (was 29/30) and gestures 2, 4, 6, 7, 9–14 also resolve via the stricter ``handler:*`` route instead of the looser ``callsite:*`` fallback. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ceiling VisualizationPanel.tsx had grown to 1654 lines, breaching the 1500-line frontend-component ceiling enforced by `scripts/check_code_quality.py`. Split out three self-contained concerns; the panel is now 1342 lines and remains the orchestration shell for the four diagram tabs + overflow iframe. Extractions: * `components/InspectSearchField.tsx` — the custom-rendered Inspect text field + suggestions dropdown that replaces the unreliable native `<input list>` / `<datalist>` pair when the host subtree is portaled into a detached window. * `components/DetachedPlaceholder.tsx` — the in-place stand-in shown inside the panel when a tab is open in a separate browser window (Focus / Reattach affordances). * `hooks/useOverflowIframe.ts` — the entire iframe ↔ parent-window plumbing for the overflow-graph tab: `cs4g:overlay-ready` handshake, message routing for pin / filter / layer / node events, pin-popover state with outside-click + Escape dismissal, and the pin / filter broadcast effects. Exposes the iframe ref, popover ref + state, and the popover setters needed by the parent JSX. No behaviour changes — Vitest suite (1250 tests) + code-quality gate both green; standalone bundle rebuilt and all four parity layers pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend the layer-coverage fixture with four nodes carrying the
upstream ``prod_or_load`` + ``value`` attributes (one prod, one
load above the 1 MW floor, and two below it — a 0.4 MW load and
the conventional zero-balance ``value="0.0"`` load). Add
``TestProdLoadValueLayers`` to lock down the new contract:
* ``node:prod`` contains exactly the prod node above 1 MW
* ``node:load`` excludes both the 0.4 MW sub-floor node and the
zero-balance node by construction
* both layers are emitted under the *Individual entities
properties* sidebar section
Pairs with the upstream alphaDeesp change that added the
``_VALUE_NODE_LAYERS`` machinery + ``_PROD_LOAD_VALUE_FLOOR_MW``
threshold.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Picks up the new Production / Consumption node filter layers shipped in the upstream release (https://github.com/marota/ExpertOp4Grid/ releases/tag/v0.3.2.post1): * ``node:prod`` / ``node:load`` toggle chips in the overflow-graph viewer's *Individual entities properties* sidebar section. * 1 MW absolute-value floor so zero-balance load placeholders don't flood the consumption layer. Also refresh the cached ``Overflow_Graph/Overflow_Graph_P.SAOL31RONCI_*.html`` reference fixture so the layer-coverage test class ``TestSmallGridOverflowGraphLayers`` runs against an HTML viewer that carries the new layer machinery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Section 2.1 (node attributes) gains rows for ``prod_or_load``+``value`` with the simulator wiring (``build_nodes``/``build_nodes_v2`` across grid2op / powsybl / pypownet) and the 1 MW floor rationale that keeps placeholder zero-balance nodes out of the Consumption layer. Section 3 (layer-toggle UI) extends the *Individual entities properties* row to list ``Production nodes`` / ``Consumption nodes`` and notes that both are pure node-only filters with coral / lightblue circle swatches matching the upstream fillcolor convention. Pairs with the upstream ExpertOp4Grid 0.3.2.post1 release. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ``test-backend`` job was wedged on ``apt-get update`` for ~8 minutes because the Azure apt mirror keeps timing out and the runner falls back to the slower archive.ubuntu.com repeatedly. Restructure the workflow so that: * ``test-backend`` (fast lane) runs every backend test that does NOT need the graphviz ``dot`` binary — i.e. every file except ``test_overflow_html_dim_logic.py``. No system-package install at all; ~720 tests finish in ~15 s. Adds ``cache: 'pip'`` to ``setup-python`` so the pip resolver is warm across runs too. * ``test-backend-graphviz`` (slow lane) only runs the small subset of tests that drive the upstream ``alphaDeesp.core.interactive_html`` viewer through pydot → ``dot``. Gated behind both ``test-backend`` and ``test-frontend`` via ``needs:``, so a regression elsewhere is reported in ~1 minute and only the prod/load-layer fixtures pay the apt install cost. * The graphviz install itself uses ``awalsh128/cache-apt-pkgs-action`` to cache the resolved .deb files. First run is ~30 s, subsequent runs restore in ~5 s — vs. the previous ~8 min ``apt-get update``. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Embed the alphaDeesp interactive HTML viewer in the Overflow
Analysis tab and augment it with Co-Study4Grid–specific behaviour:
semantic layer toggles, action pins synchronised with the Action
Overview NAD, drill-down to the SLD on double-click, edge / line
re-routing in geo mode, and a per-pin filter widget that mirrors
the Action Overview filters.
Full feature contract:
docs/features/interactive-overflow-analysis.md(12 sections — architecture, source-of-truth tagging, layer-toggle
UI, geo-mode SVG transform, overlay injection, action-pin overlay
including anchor priority + glyph factory + click semantics + SLD
action sub-tab + un-simulated pin layer with multi-line tooltip,
filter sync, forwarded interaction events, file map, tests,
reset / lifecycle invariants, verification).
Key additions
/results/pdfstatic mount with adynamic GET that injects the overlay block into HTML responses.
New
services/overflow_overlay.pyowns the overlay payload andinlines the SHARED
pinGlyph.jsfactory (ESM exports stripped).Geo mode now preserves the tapered-edge body for swapped-flow
lines.
📌 Pinstoolbar toggle (disabled until step2 finishes), an
OverflowPinpayload builder mirroring theAction Overview anchor priority, popover-on-single-click vs
SLD-on-double-click semantics, and bidirectional filter sync
with the iframe sidebar (categories / threshold / showUnsimulated
/ actionType).
Show unsimulatedticked oneither surface renders dashed grey "?" pins; hovering reveals a
multi-line tooltip (id / type / score / rank / MW-start);
double-clicking kicks off the manual simulation flow.
frontend/src/utils/svg/pinGlyph.jsisconsumed by BOTH the React Action Overview pin renderer
(
actionPinRender.ts) and the iframe overlay (read as text bythe backend, ESM exports stripped). Single source of truth for
shape, palette, and status symbols.
Companion PRs
— semantic source-of-truth tagging + grouped layer toggles in
the alphaDeesp viewer.
— wires the constrained-path / red-loop / overloads lists into
the new taggers.
Test plan
pytest expert_backend/tests/test_overflow_overlay.py—overlay injection, pin click/dblclick listeners, single-click
debounce, shared-glyph inlining, no-duplicate-
const SVG_NSguard, action pins filter section placement, pin counter,
unsimulated pin dashed/dim render, distinct
cs4g:overflow-unsimulated-pin-double-clickedroute, andpin.titleconsumption.pytest expert_backend/tests/test_overflow_html_dim_logic.py— section grouping, membership-based dim model, edge-id
alignment guard,
is_overload ⊂ is_monitoredsemantics.pytest expert_backend/tests/test_overflow_geo_transform.py— geo-mode rewrites including the tapered-edge body /
arrowhead split.
cd frontend && npm run test— full Vitest suite (1250passed, 2 skipped) including the new
pinGlyph.test.ts,overflowPinPayload.test.ts(coversbuildOverflowUnsimulatedPinPayload+ multi-line title),VisualizationPanel.test.tsx,useSldOverlay.test.ts, andspecConformance.test.ts.cd frontend && npm run build:standalone— auto-generatedbundle inherits the changes.
python scripts/check_standalone_parity.py— declaredgestures present in both React source and standalone.
data/bare_env_small_grid_test(steps 1–9in
docs/features/interactive-overflow-analysis.md§12).🤖 Generated with Claude Code