diff --git a/.claude/playwright-mcp-config.json b/.claude/playwright-mcp-config.json new file mode 100644 index 00000000..ce57fcb3 --- /dev/null +++ b/.claude/playwright-mcp-config.json @@ -0,0 +1,7 @@ +{ + "browser": { + "launchOptions": { + "chromiumSandbox": true + } + } +} diff --git a/.claude/skills/gently-session-postmortem/SKILL.md b/.claude/skills/gently-session-postmortem/SKILL.md new file mode 100644 index 00000000..d0e31f1b --- /dev/null +++ b/.claude/skills/gently-session-postmortem/SKILL.md @@ -0,0 +1,94 @@ +--- +name: gently-session-postmortem +description: Use when analyzing how a user actually operated the gently UI — replaying a recorded session, walking the semantic action log, correlating UI actions with agent/device behavior at the same timestamps, or rendering what the screen showed at a specific moment. +--- + +# Gently session postmortem + +Every gently session records its UI interactions (always-on, rrweb-based; +kill switch `GENTLY_REPLAY=0`). You are the intended reader: work text-first +through the action log, and render pixels only for the moments that matter. + +## Where the artifacts live + +Per session, under the storage root (`D:\Gently3` on the Windows production +machine; on Linux dev runs the default is the *literal* directory `D:/Gently3` +under the cwd the server launched from, unless `GENTLY_STORAGE_PATH` is set): + +``` +sessions/{folder}/ui-replay/ + actions.jsonl # semantic action log — READ THIS FIRST + rrweb-{tab}.jsonl # full DOM event stream, one file per browser tab + meta.yaml # tabs seen, first_seen, user agents +ui-replay/unassigned-{YYYYMMDD}/ # batches that arrived with no active session +``` + +Session id → folder mapping: `sessions/_index.yaml`. + +## The walk + +1. **Read `actions.jsonl`** (one JSON object per line). It is **arrival-ordered, + not time-ordered** (batches from multiple tabs interleave) — sort by `t` + before reconstructing a timeline. Non-browser traffic (curl probes, harness + runs) records too; check the tab's `user_agent` in `meta.yaml` before + treating a tab as a human. Kinds you'll see: + - `page-load` — with url + viewport; a new `tab` id per browser tab/reload. + - `tab:embryos`, `view:board`, `bz:-10`, `button#op-detect`, + `click:marking-action-btn:Done` — clicks, named from the UI's semantic + `data-*` vocabulary, element ids, or class+text fallback. `target` carries + tag/id/dataset/label. + - `submit`, `navigate` — forms and SPA route/hash changes. + - `bus-summary` — per-flush counts of ClientEventBus traffic + (`BOTTOM_CAMERA_FRAME×240` means live frames were streaming; token + streaming shows as high `AGENT_*` counts). This is how you know what the + *system* was doing between clicks. + - `gap` — the recorder dropped events under pressure (params say how many). +2. **Join with the rest of the file store on timestamps.** Everything else is + already on disk: agent logs (`logs/gently_*.log`), perception traces and + `predictions.jsonl` per embryo, `timelapse.yaml`, events. See the + `gently-debugging` skill for that map. + **Clock semantics:** `actions.jsonl` `t` is the *browser's* clock as UTC + ISO; rrweb `timestamp` is epoch ms; `meta.yaml` `first_seen` is *naive + server-local* time (IST on the production rig — expect it to differ from + the same tab's UTC `page-load` by the UTC offset); server logs print + server-local time. Convert everything to epoch before joining, and expect + modest client/server skew (same machine ⇒ sub-second; remote browser ⇒ + whatever their clock is). + **Names vs labels:** action names use the UI's *internal* vocabulary + (`tab:events`), while `target.label` is what the user actually saw + ("Logs") — quote labels when narrating for humans. +3. **Render the moments that matter** (needs the repo venv with Playwright): + + ```bash + python tools/session_replay/render_frame.py \ + --session 81865db3 --t 2026-07-13T07:05:38Z --out /tmp/frame.png \ + [--tab 04dce1bd] [--url http://localhost:8080] [--storage PATH] + ``` + + `--t` takes an ms offset, `52s`, `mm:ss`, or an absolute ISO timestamp + (paste straight from an action's `t`). `--url` pointing at a *running* + gently makes stored images inside the frame resolve; without it you get + structure + inlined CSS only. Default tab is the largest stream. +4. **Human scrubbing**: `http://localhost:8080/replay` lists recordings; + `/replay/{session_id}` plays one (tab picker, speed, click an action to + seek to it). + +## Caveats + +- The live camera `` is deliberately blocked from + capture (base64 frame storms) — replays show a placeholder box there. What + the camera saw lives in the file store (volumes/projections); whether it + was streaming is in `bus-summary`. +- Replays disable CSS animations/transitions (`animation: none` injected) so + final styles apply — otherwise paused replays freeze entrance animations at + their invisible from-state. Consequence: spinners/pulses appear static, and + a frame rendered mid-animation shows the settled state, not the tween. +- A rendered frame shows the DOM *at that instant*: click a nav button and + the previous panel is still on screen — the response mutations land in the + following ~200 ms. Render a second frame slightly later to see the effect. +- A session's recording can span multiple `rrweb-{tab}` files (reloads, second + windows). `meta.yaml` + `page-load` actions give the timeline of tabs. +- The final ≤4s batch of a tab can be missing if the browser/app closed + uncleanly (see spec: quit-flush race). +- Recording only exists for sessions run with `settings.ui.replay` on + (default on; `GENTLY_REPLAY=0` disables). diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml new file mode 100644 index 00000000..a050e298 --- /dev/null +++ b/.github/workflows/lint.yml @@ -0,0 +1,69 @@ +name: Lint + +on: + push: + branches: [main, development] + pull_request: + +jobs: + lint: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.10" + + - name: Install ruff + # Pinned so CI never floats to a new ruff release mid-review. Keep in + # sync with the ruff-pre-commit rev in .pre-commit-config.yaml and the + # pin in pyproject.toml. + run: pip install ruff==0.16.0 + + - name: ruff check + run: ruff check . + + - name: ruff format check + run: ruff format --check . + + # Fast baseline mypy without project deps. With ignore_missing_imports, + # third-party imports fall back to Any, so this is the weaker check — kept + # as the required gate because it needs no heavy install and stays green. + # The stronger, deps-installed run lives in the mypy-strict job below. + # Pin to match the pre-commit mirrors-mypy rev and the pyproject dev + # group so CI, local commits, and `uv sync` never run different mypys. + - name: Install mypy + run: pip install mypy==2.1.0 + + - name: mypy (deps-less) + run: mypy . + + # Deps-installed mypy: with the real third-party packages present (numpy, + # anthropic, …) mypy resolves their actual types instead of Any, surfacing + # genuine mismatches the deps-less run above cannot see. This is what a + # contributor gets from `uv run mypy .` after `uv sync` (see CONTRIBUTING.md). + # Required gate: the deps-only errors have been cleared (issue #63), so a + # regression here now fails the workflow like the deps-less run above. + mypy-strict: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + # gently-perception is not on PyPI; it is installed editable from a sibling + # clone (see [tool.uv.sources]), so `uv sync` fails without it. + - name: Clone gently-perception sibling + run: git clone --depth 1 https://github.com/gently-project/gently-perception.git ../gently-perception + + - name: Install uv + uses: astral-sh/setup-uv@v5 + with: + python-version: "3.10" + enable-cache: true + + - name: uv sync + run: uv sync + + - name: mypy (deps-installed) + run: uv run mypy . diff --git a/.gitignore b/.gitignore index cce868ec..d66d6e7b 100644 --- a/.gitignore +++ b/.gitignore @@ -145,3 +145,17 @@ electron/ /stage_definitions_for_review.txt gently/ui/tui/node_modules/ gently/ui/tui/dist/ + +# Stray local storage: GENTLY_STORAGE_PATH default (D:\Gently3) resolves +# literally to ./D:/ under the repo on Linux. Not data we track. +/D:/ +.superpowers/ + +# Runtime operator overrides (written by the Settings panel) +config/config.local.yml +config/settings.local.yml +config/dashboard_defaults.json + +# UI crawler generated output +tools/ui_crawler/out*/ + diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 00000000..4cab484e --- /dev/null +++ b/.mcp.json @@ -0,0 +1,14 @@ +{ + "mcpServers": { + "playwright": { + "type": "stdio", + "command": "npx", + "args": [ + "@playwright/mcp@latest", + "--config", + ".claude/playwright-mcp-config.json" + ], + "env": {} + } + } +} \ No newline at end of file diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 00000000..4c627f68 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,16 @@ +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.16.0 # keep in sync with the ruff pins in pyproject.toml + lint.yml; bump via `pre-commit autoupdate` + hooks: + - id: ruff + args: [--fix] + - id: ruff-format + + - repo: https://github.com/pre-commit/mirrors-mypy + # Keep in sync with the mypy pin in pyproject.toml dev group and + # .github/workflows/lint.yml. Run `pre-commit autoupdate` to bump. + rev: v2.1.0 + hooks: + - id: mypy + pass_filenames: false + args: ["."] diff --git a/.python-version b/.python-version new file mode 100644 index 00000000..2c073331 --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.11 diff --git a/CHANGELOG.md b/CHANGELOG.md index 30669817..0097853a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -394,6 +394,52 @@ Net: ~3,500 lines removed across P6–P7. --- +## v0.22.0 + +File-based storage, a redesigned web UI, new hardware, and the tooling to +keep it all honest. + +**File-based storage (Gently3)** +Retired the SQLite databases. All state now lives as human-browsable files +under `D:\Gently3\` — sessions, embryos, volumes, projections, traces, +campaigns, learnings, agent memory, all YAML/JSONL/TIFF. +- `FileStore` replaces `GentlyStore`; `FileContextStore` replaces the + `agent_mind.db` `ContextStore`. Drop-in API replacements. +- A root `gently.yaml` manifest documents the layout for humans and agents. +- YAML parses are cached in `FileContextStore` — fixes slow Plans/campaign + loading. + +**Web UI redesign** +- Agent chat became a docked, sliding side panel (overlay + pin-to-dock) + instead of owning the screen. +- Added a Home landing tab; the chat no longer auto-runs the startup wizard. +- Login is non-blocking — a "Continue in view-only" escape hatch. +- Recent images aggregate across previous sessions. + +**Hardware** +- Integrated the ACUITYnano temperature controller (config, web control, + SDKs) with a live HiveMQ cloud SIM for hardware-free testing. +- Added the SPIM-head F-drive device, hard limits, and focus/align plans. +- Room-light toggle and a device-layer terminal UI. + +**Agent + perception** +- Integrated the agent with perception: pull tool, prompt context, event + bridge, wake-router. +- Live acquisition control with observable, permissioned autonomy and a + refreshed prompt. +- Retired napari from the agent; added web-chat autocomplete and pruned + dead tools. + +**Tooling and environment** +- Added ruff lint/format tooling and fixed all violations. +- Adopted incremental mypy typing — config, CI, pre-commit wiring, and a + documented policy in `CONTRIBUTING.md`; pinned mypy to 2.1.0. +- Switched environment setup to uv with an offline/UI-only launch path; + pinned pymmcore to device-interface 70. +- Relicensed and updated the author list. + +--- + ## Notes on how we think about this Things we've learned building this, roughly in order: diff --git a/CLAUDE.md b/CLAUDE.md index 15301c4b..4f114143 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,13 +1,89 @@ # Gently — Microscopy Agent +## Working on this repo + +**README.md** covers environment setup and how to run things (`uv sync`, +`uv run pytest`, `uv run python launch_gently.py` and its flags). +**CONTRIBUTING.md** covers the lint/type toolchain — ruff, the two mypy runs, +and pre-commit. This section is only for what those two don't say — the things +that are easy to get wrong here. + +### Where work lands + +Branch off `development`; open the PR against `development` in +**`gently-project/gently`**. Only the push differs by access level: + +```bash +# With write access — branches go straight to the org repo +git clone git@github.com:gently-project/gently.git && cd gently +git checkout -b feature/ origin/development +git push -u origin feature/ + +# Without write access — fork on GitHub, then keep the org repo as `upstream` +git clone git@github.com:/gently.git && cd gently +git remote add upstream git@github.com:gently-project/gently.git +git fetch upstream +git checkout -b feature/ upstream/development +git push -u origin feature/ # your fork + +# Either way +gh pr create --repo gently-project/gently --base development +``` + +Check `git remote -v` before pushing: in a direct clone `origin` *is* the org +repo; in a fork-based clone `origin` is your fork and `upstream` is the org repo. +Two `gh` defaults will misfile a PR — the repo's default branch is `main` while +PRs target `development`, and with no `gh` default repo set it resolves from +whichever remote it finds. Pass `--base development --repo gently-project/gently`. + +### Before committing + +CONTRIBUTING.md is canonical for the toolchain. The trap it doesn't mention: +**the pre-commit hook is not installed in a fresh clone**, so commits silently +bypass ruff and mypy. Run `pre-commit install` once per clone, then +`pre-commit run --all-files` before opening a PR. + +`.github/workflows/lint.yml` is the source of truth for which checks gate a PR, +and that changes — read it rather than trusting a list here. What stays true: + +- **There are two mypy runs and they can disagree.** One runs `mypy .` with no + project deps, so third-party imports fall back to `Any`; the other runs + `uv run mypy .` with the real packages and resolves their actual types. A + green run of one says nothing about the other. `uv sync && uv run mypy .` + reproduces the deps-installed run locally; the pre-commit hook reproduces the + deps-less one. +- **A job stops at its first failing step.** A ruff failure means the mypy step + in that job never ran, so a green re-run after fixing ruff is not evidence + that mypy passed. +- **CI runs no JavaScript.** A change under `gently/ui/web/static/js/` is not + covered by CI, so verify it by running the app and exercising the UI by hand. +- CI runs on pull requests and on pushes to `main`/`development`, so a feature + branch with no PR open gets no signal at all. + +### Running the app off-Windows + +The storage paths throughout this file (`D:\Gently3\...`) are the Windows +microscope PCs, where `D:` is the dedicated data drive. Off-Windows that default +is **not an absolute path**: it resolves against the cwd and silently creates a +junk directory literally named `D:` with session data inside it +(`gently/settings.py`). Always set an explicit path when running on Linux or +macOS: + +```bash +GENTLY_STORAGE_PATH=/tmp/gently-dev uv run python launch_gently.py --no-api --no-auth --no-browser +``` + +Those three flags are the usual agent/dev combination: no Anthropic key needed, +no login gate, and no browser window. The UI is then on `http://localhost:8080`. + ## Storage Architecture (Gently3 — File-Based) All data lives under `D:\Gently3\` (env: `GENTLY_STORAGE_PATH`). **No SQLite databases.** Everything is human-browsable files. ### Key store classes -- **`FileStore`** (`gently/core/file_store.py`) — replaces `GentlyStore`. Manages sessions, embryos, volumes, projections, predictions, traces. Drop-in API replacement. -- **`FileContextStore`** (`gently/harness/memory/file_store.py`) — replaces `ContextStore` / `agent_mind.db`. Manages campaigns, plans, learnings, observations, agent state. Drop-in API replacement. -- **Root manifest**: `D:\Gently3\gently.yaml` — documents the structure for humans and agents. +- **`FileStore`** (`gently/core/file_store.py`) — sessions, embryos, volumes, projections, predictions, traces. +- **`FileContextStore`** (`gently/harness/memory/file_store.py`) — campaigns, plans, learnings, observations, agent state. +- **Root manifest**: `gently.yaml` at the storage root — documents the structure for humans and agents. ### Directory layout ``` @@ -63,46 +139,92 @@ D:/Gently3/ incoming/{uuid}.tif # transient device staging ``` -### Legacy stores (D:\Gently2\ — read-only reference) -The old SQLite-based stores are preserved but no longer written to: -- `gently.db` (GentlyStore) — replaced by FileStore -- `context/agent_mind.db` (ContextStore) — replaced by FileContextStore -- `D:\gently\dataset.db` — legacy benchmarking DB +### Superseded stores — do not wire new code to these + +The SQLite-era classes are **still in the package and still have passing tests**, +so they look live. They have no production callers; only `tests/` instantiate +them. Use the file stores above instead. + +- `GentlyStore` (`gently/core/store.py`) → use `FileStore` +- `ContextStore` (`gently/harness/memory/store.py`, `agent_mind.db`) → use `FileContextStore` +- `gently/dataset/` still defaults to `D:/gently/dataset.db` (legacy benchmarking DB) + +Their data lives under the old `D:\Gently2\` root, read-only reference only. ## Logging -Both the agent and device layer write logs to `D:\Gently3\logs\`: +Both the agent and device layer write logs to `/logs/`: - **Agent**: `gently_YYYYMMDD_HHMMSS.log` — INFO+ to file, console level configurable via `-v` flag - **Device layer**: `device_layer_YYYYMMDD_HHMMSS.log` — INFO level -To check logs during a session: +To check logs during a session (the expansion keeps these working off-Windows, +where the `D:/Gently3` default does not apply — see above): + ```bash -# Latest agent log -tail -f D:/Gently3/logs/$(ls -t D:/Gently3/logs/gently_*.log | head -1) +LOGS="${GENTLY_STORAGE_PATH:-D:/Gently3}/logs" + +# Latest agent log (ls prints the full path — do not prefix $LOGS again) +tail -f "$(ls -t "$LOGS"/gently_*.log | head -1)" # Latest device layer log -tail -f D:/Gently3/logs/$(ls -t D:/Gently3/logs/device_layer_*.log | head -1) +tail -f "$(ls -t "$LOGS"/device_layer_*.log | head -1)" # Filter for errors -grep -E "ERROR|Traceback" D:/Gently3/logs/gently_*.log +grep -E "ERROR|Traceback" "$LOGS"/gently_*.log ``` ## Perception -Perception is handled by `gently-perception` (separate repo: `pskeshu/gently-perception`), installed as a pip dependency. The timelapse orchestrator uses `Perceiver()` from `gently_perception` — a self-contained system that loads its own examples and accumulates per-embryo context through sequential calls. +Perception is handled by `gently-perception` (separate repo: +`gently-project/gently-perception` — this is what CI clones and what +`pyproject.toml` expects beside this repo), installed as a pip dependency. The timelapse orchestrator uses `Perceiver()` from `gently_perception` — a self-contained system that loads its own examples and accumulates per-embryo context through sequential calls. ## Device Layer The device layer runs as a separate process (`python start_device_layer.py`). It communicates with the agent via HTTP. Bluesky plans require ophyd device name kwargs (e.g. `xy_stage='xy_stage'`, `volume_scanner='volume_scanner'`) — these must match the device names registered in `device_factory.py`. -## Debugging Data Sources +## Desktop App (Tauri) + +Gently can run as a Windows desktop app — a thin Tauri (WebView2) shell in +`desktop/` that OWNS the Python backend. It spawns `launch_gently.py --no-browser`, +shows a splash, then renders the live web UI (`http://localhost:8080`) in a native +window. The Python-served web UI stays the single source of truth — no app logic +lives in the shell. Full detail: `desktop/README.md`. -- **Agent logs**: `D:\Gently3\logs\gently_*.log` -- **Device layer logs**: `D:\Gently3\logs\device_layer_*.log` -- **Perception traces**: `D:\Gently3\sessions\{session}\embryos\{embryo}\traces\` — per-timepoint JSON -- **Predictions**: `D:\Gently3\sessions\{session}\embryos\{embryo}\predictions.jsonl` -- **Volume staging**: `D:\Gently3\incoming\` -- **Agent memory**: `D:\Gently3\agent\` — campaigns, learnings, observations (all YAML) -- **Session state**: `D:\Gently3\sessions\{session}\session.yaml` -- **Timelapse state**: `D:\Gently3\sessions\{session}\timelapse.yaml` +### Build / run +``` +cd desktop +npm install # once — restores the Tauri CLI +npm run dev # dev: build + launch (spawns the backend for you) +npm run build # release: NSIS installer under src-tauri/target/release/bundle/ +``` +Prereqs: Rust (MSVC toolchain), the repo's uv `.venv`, WebView2 runtime (inbox on +Win 11). The **release** exe (`src-tauri/target/release/gently-desktop.exe`) is what +the Desktop shortcut points at; rebuild with `npm run build` to refresh it. + +### Key pieces +- `desktop/src-tauri/src/main.rs` — spawns the backend, waits for the server, + navigates the window to it, owns teardown. +- **No orphans:** the spawned Python (and its device-layer grandchild) run in a + Windows Job Object with `KILL_ON_JOB_CLOSE`, so quitting/crashing the shell reaps + the whole tree. +- **No console window:** the backend is spawned with `CREATE_NO_WINDOW` in release + (the device-layer supervisor does the same); `tauri dev` keeps the console so + logs stay visible while developing. + +### Editing code — what reflects +- **Web UI** (`gently/ui/web/templates`, `static/js`, `static/css`): refresh the + window (Ctrl+R) — served live by Python, no rebuild. +- **Python backend**: restart it, or run with `--reload` + (`launch_gently.py --reload`, or `GENTLY_LAUNCH_ARGS="--reload …"`) to auto-restart + on `gently/*.py` changes, then Ctrl+R. Whole-backend restart — not for live hardware. +- **Rust shell / `tauri.conf.json`**: `npm run dev` auto-rebuilds and relaunches. + +### Shell env config (read by `main.rs`) +`GENTLY_HOME` (repo dir), `GENTLY_PYTHON` (interpreter), `GENTLY_LAUNCH_ARGS` +(extra `launch_gently` args), `VIZ_PORT` (default 8080), `GENTLY_DEVICE_LAYER_SCRIPT`. + +### Deferred +Bundling the Python env (torch/anthropic/perception) for a redistributable +installer — today's build launches the repo's `.venv`, so it's single-machine. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..1b869a4b --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,69 @@ +# Contributing to Gently + +## Code quality toolchain + +This project uses [ruff](https://docs.astral.sh/ruff/) for linting and formatting, and +[mypy](https://mypy-lang.org/) for type checking, enforced automatically before every +commit via [pre-commit](https://pre-commit.com/). + +### First-time setup + +Install the dev dependencies (includes ruff, mypy, and pre-commit): + +```bash +uv sync +``` + +Then install the pre-commit hooks: + +```bash +pre-commit install +``` + +From this point on, ruff runs on staged files and mypy runs across the whole +project whenever you `git commit`. + +### Running manually + +To check all files at once (useful before opening a PR): + +```bash +pre-commit run --all-files +``` + +Or run the tools directly: + +```bash +ruff check . # lint +ruff format . # format in-place +``` + +### Keeping hooks up to date + +To update hook versions to their latest releases: + +```bash +pre-commit autoupdate +``` + +### Type checking + +mypy runs two ways, and they can disagree: + +- `mypy .` — no project dependencies installed. `[tool.mypy]` sets + `ignore_missing_imports = true`, so third-party imports fall back to `Any`. + This is the form the pre-commit hook runs. +- `uv run mypy .` — after `uv sync`, with the real packages present, so mypy + checks their actual types and can surface mismatches the deps-less run cannot. + +A green run of one says nothing about the other, so run `uv run mypy .` before +pushing if you touched code that uses a typed third-party library. New code must +type-clean under both. + +### CI + +`.github/workflows/lint.yml` is the source of truth for what gates a pull +request, and it changes — read it rather than trusting a summary here. +`pre-commit run --all-files` reproduces the ruff checks and the deps-less +`mypy .` locally, but **not** the deps-installed run; reproduce that with +`uv run mypy .` after `uv sync`. diff --git a/LICENSE b/LICENSE index 486ead3a..f288702d 100644 --- a/LICENSE +++ b/LICENSE @@ -1,21 +1,674 @@ -MIT License - -Copyright (c) 2023 Kesavan Subburam - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/README.md b/README.md index 5dca56db..360b27fb 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Agentic harness for microscopy. -**Status**: v0.11.0 — actively developed at Shroff Lab, Janelia. +**Status**: 1.0.0.dev0 — actively developed at Shroff Lab, Janelia. ![Safety Architecture](docs/images/safety_architecture.png) @@ -67,55 +67,187 @@ Currently, the sample abstraction is the `Embryo` object for *C. elegans* work. ### Prerequisites -- Python 3.11+ -- [Node.js](https://nodejs.org/) 18+ (for the Ink TUI) -- An `ANTHROPIC_API_KEY` environment variable +- Python 3.10+ +- An `ANTHROPIC_API_KEY` — either exported in your shell + (`export ANTHROPIC_API_KEY=your-key`) or placed in a `.env` file in the + project root (`ANTHROPIC_API_KEY=your-key`), which is loaded automatically + on launch and is gitignored. *(Not required if you launch with `--no-api` + to browse the UI only — see Launch below.)* +- *(Optional)* `GENTLY_STORAGE_PATH` — where sessions and data live (default `D:/Gently3`) + +Gently is **web-first**: the agent is driven from an in-page chat in your +browser. There is no TUI to build (Node.js is only needed for the paper +diagrams, not the app). ### Setup +This project uses [uv](https://docs.astral.sh/uv/) for environment and +dependency management. If you don't have it yet, install it following the +[uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) +(e.g. `curl -LsSf https://astral.sh/uv/install.sh | sh` on macOS/Linux). + +Gently depends on **`gently-perception`** (the VLM perception harness, repo +`gently-project/gently-perception`), which is not published to PyPI. For development, it is +installed as an **editable sibling clone**, so clone both repos side by side: + ```bash -# Clone and install Python dependencies -git clone https://github.com/pskeshu/gently.git +git clone git@github.com:gently-project/gently.git +git clone git@github.com:gently-project/gently-perception.git + +# Layout: +# / +# gently/ <- you run commands from here +# gently-perception/ <- editable, resolved via [tool.uv.sources] + cd gently -pip install -r requirements.txt +uv sync # base env (add --extra ... for torch etc., see below) +``` + +> The `git@github.com:` URLs use **SSH**, which needs an +> [SSH key configured with GitHub](https://docs.github.com/en/authentication/connecting-to-github-with-ssh). +> If you don't use SSH, clone over HTTPS instead +> (`https://github.com/gently-project/.git`). + +`[tool.uv.sources]` resolves `gently-perception` to the sibling as an editable +install, so your perception edits are live immediately and survive `uv sync`. If +the sibling isn't cloned, `uv sync` fails by design — clone it first. + +You get a `.venv` in the project directory with the runtime + dev dependencies +pinned in `uv.lock`. Activate it with `source .venv/bin/activate`, or prefix +commands with `uv run` (e.g. `uv run python ...`) to use it without activating. + +#### The `device` group and optional extras + +The hardware-control stack (`pymmcore`, `bluesky`, `ophyd`) plus the accessory +transports (BLE/serial/MQTT — SwitchBot room light, ACUITYnano thermal +controller) live in the `device` dependency group, which is **installed by +default**. None of it is imported by the agent or by `--offline` launch — only +by the device layer — so a machine with no microscope can drop the whole stack: -# Build the TUI (one-time, rebuild after TUI code changes) -cd gently/tui -npm install -npm run build -cd ../.. +```bash +# Laptop / review / CI — no microscope, skip the hardware-control stack +uv sync --no-group device +``` + +PyTorch is **not** in the base install — the CUDA build is machine-specific, so +it lives in mutually-exclusive extras wired to the right PyTorch index: + +```bash +# PyTorch (needed for SAM detection and the ML pipeline) +# NOTE: the GPU and CPU builds are mutually exclusive, so they can't be combined. +uv sync --extra torch-gpu # CUDA 11.8 build (GPU box, e.g. the microscope PC) +uv sync --extra torch-cpu # CPU-only build (dev laptop / CI) +``` + +#### Running tests + +```bash +uv run pytest ``` ### Launch +> The commands below use `uv run` so they work without activating the env. If you've activated it first (`source .venv/bin/activate`), the `uv run` prefix isn't necessary. + +To verify the install, you can start gently without an API key or hardware. The +web UI boots and is browsable, though the agent itself (chat, perception, plan +mode) stays disabled until you add a key: + ```bash -# 1. Start the device layer (hardware control + SAM detection) -python start_device_layer.py +uv run python launch_gently.py --offline --no-api +``` + +For the full launch: + +```bash +# 1. Device layer (hardware control + SAM detection) — separate process, own terminal +uv run python start_device_layer.py + +# 2. Agent + web UI (starts the in-process server and opens your browser) +uv run python launch_gently.py + +# Run without hardware (development / review) +uv run python launch_gently.py --offline -# 2. Launch the agent -python launch_gently.py +# UI-only — boot the web UI with no API key (chat/perception disabled) +uv run python launch_gently.py --no-api -# Or launch without hardware (for development / review) -python launch_gently.py --offline +# Don't auto-open a browser — open the printed URL yourself +uv run python launch_gently.py --no-browser + +# Skip login — disable accounts (localhost-control mode; same as GENTLY_NO_AUTH=1) +uv run python launch_gently.py --no-auth # Resume a previous session -python launch_gently.py --resume # interactive picker -python launch_gently.py --resume latest # most recent session -python launch_gently.py --resume # specific session +uv run python launch_gently.py --resume # interactive picker +uv run python launch_gently.py --resume latest # most recent session +uv run python launch_gently.py --resume # specific session # Verbose / debug logging -python launch_gently.py -v # INFO level -python launch_gently.py --debug # DEBUG level +uv run python launch_gently.py -v # INFO level +uv run python launch_gently.py --debug # DEBUG level +``` + +The launcher prints a banner with the URL (default `http://localhost:8080`), +device status, storage path, and log location. Open that URL in any browser on +the LAN. + +### First sign-in (accounts) + +**Viewing is open** — the dashboard loads read-only for anyone, no login. +Signing in *elevates* you to control (driving hardware, taking the +single-operator lock); it isn't a gate on the page. + +On the **first run**, Gently creates one `admin` account and prints a one-time +random password in the startup banner: + +``` +First-run admin account created — sign in at the URL above: + username: admin + password: ``` +- **Save it now** — the password is printed to the console once and never + written to the log (only a PBKDF2 hash is stored). +- After signing in, add accounts (roles `viewer` / `operator` / `admin`) via the + admin-only `POST /api/auth/users`. +- **Lost it?** There's no reset command yet — delete + `/auth/users.yaml` and restart to bootstrap a fresh + `admin` (this clears all accounts). +- **Just trying it locally?** Launch with `--no-auth` (or set `GENTLY_NO_AUTH=1`) + to disable accounts entirely (localhost gets control, remote callers need + `X-Gently-Token`). Handy if you've lost the admin password. + +Accounts live under `/auth/` (`users.yaml` + `secret.key`), +outside the repo. + +## Make your first plan + +You don't need a microscope to try the core loop — **plan mode is pure agent reasoning and works under `--offline`**. The path from launch to an inspectable plan: + +1. **Open the agent chat.** Click **Agent** in the header (or press `Ctrl`/`Cmd`+`J`). New here? The **Home** tab's *Start an experiment* button runs a short setup wizard (also available anytime via `/wizard` — it sets the organism, the campaign, and what you're trying to learn). +2. **Enter plan mode** — type `/plan` in the chat. The agent switches from *operator* to *scientific collaborator*: it won't touch hardware, it helps you design an experiment. +3. **Describe what you want, in plain language.** For example: + > *"Follow GFP-tagged embryos from bean stage through elongation, imaging every 10 minutes, with a no-laser control — three embryos per condition."* + + The agent drafts a **campaign**: a sequence of typed **plan items** — imaging 📷, bench 🧪, genetics 🧬, analysis 📊, decision points 🚦 — each with concrete specs (strain, interval, laser power, Z-slices, target window, success criteria). Keep replying to refine it; `/plan status` shows progress and `/plan exit` returns to run mode. +4. **Inspect it in the plan viewer.** Open the **Plans** tab. Your campaign appears as a card — click it to open the **plan document**. Each item shows its status (○ planned · ◑ in progress · ● done) and specs; click one to see full details in the inspector. Switch layouts (document / board / graph / timeline) from the view controls, and browse plan **versions** as it evolves. (Typing `/campaign` in chat lists campaigns too.) + +That's the loop: **talk → plan → inspect.** With hardware connected (drop `--offline` and start the device layer), the same campaign drives acquisition — and perception events can wake the agent to adjust it as the embryos develop. + ## Guides | Guide | Audience | What you'll learn | |-------|----------|-------------------| +| [Documentation Home](docs/index.md) | Everyone | Browse the generated-docs structure for Gently | +| [Full Stack Microscopy](docs/full-stack-microscopy.md) | Everyone | How intent, samples, hardware, perception, data, and operators fit together | | [Try Without Hardware](docs/guides/try-offline.md) | Everyone | Run the agent in 10 minutes — conversation, plan mode, perception | | [What Gently Can Do](docs/guides/capabilities.md) | Everyone | Perception, detection, plan mode, memory, mesh, safety | | [Build a Plugin](docs/guides/build-a-plugin.md) | Developers | Create organism and hardware plugins for other modalities | | [Hardware Setup](docs/guides/hardware-setup.md) | Labs | Connect a diSPIM, start the device layer, first acquisition | +| [Sample & Hardware Model](docs/architecture/sample-hardware-domains.md) | Developers | Generalize samples, hardware operations, and device-profile boundaries | +| [Sample Tracking Metrics](docs/architecture/sample-tracking-metrics.md) | Developers | Define reusable sample state, exposure, focus, and perception metrics | +| [Hardware Profile Template](docs/architecture/hardware-profile-template.md) | Developers | Checklist for adding or documenting a new microscope profile | ## Architecture @@ -125,7 +257,7 @@ Four layers with strict downward-only dependencies. The **harness** (reusable ag gently/ ├── core/ # Layer 1: Foundation — zero domain knowledge │ ├── event_bus.py # Async pub/sub messaging -│ ├── store.py # GentlyStore (SQLite + files) +│ ├── file_store.py # FileStore (file-based: YAML / JSONL / TIF) │ ├── imaging.py # Projection, normalization, encoding │ └── coordinates.py # Pixel/stage transforms │ @@ -220,4 +352,6 @@ These papers provide theoretical background for gently's approach: ## License -See [LICENSE](LICENSE) file. +Copyright © 2026 Howard Hughes Medical Institute. + +Gently is licensed under the GNU General Public License v3.0 or later (GPL-3.0-or-later) — see the [LICENSE](LICENSE) file. diff --git a/benchmarks/agent/evaluator.py b/benchmarks/agent/evaluator.py index 190b2c8b..3442e79f 100644 --- a/benchmarks/agent/evaluator.py +++ b/benchmarks/agent/evaluator.py @@ -9,7 +9,7 @@ from dataclasses import dataclass, field from datetime import datetime from pathlib import Path -from typing import Any, Dict, List, Optional, Union +from typing import Any logger = logging.getLogger(__name__) @@ -17,17 +17,18 @@ @dataclass class EvalResult: """Result of evaluating a single test case""" + test_id: str query: str - expected_tool: Union[str, List[str]] - actual_tool: Optional[str] + expected_tool: str | list[str] + actual_tool: str | None tool_correct: bool params_correct: bool - param_errors: List[str] = field(default_factory=list) + param_errors: list[str] = field(default_factory=list) input_tokens: int = 0 output_tokens: int = 0 latency_ms: float = 0 - error: Optional[str] = None + error: str | None = None @property def passed(self) -> bool: @@ -37,6 +38,7 @@ def passed(self) -> bool: @dataclass class BenchmarkReport: """Summary report for a benchmark run""" + timestamp: str num_cases: int num_passed: int @@ -45,10 +47,10 @@ class BenchmarkReport: total_input_tokens: int total_output_tokens: int avg_latency_ms: float - results: List[EvalResult] - metadata: Dict[str, Any] = field(default_factory=dict) + results: list[EvalResult] + metadata: dict[str, Any] = field(default_factory=dict) - def to_dict(self) -> Dict: + def to_dict(self) -> dict: return { "timestamp": self.timestamp, "summary": { @@ -61,7 +63,10 @@ def to_dict(self) -> Dict: "tokens": { "total_input": self.total_input_tokens, "total_output": self.total_output_tokens, - "avg_per_query": (self.total_input_tokens + self.total_output_tokens) / self.num_cases if self.num_cases > 0 else 0, + "avg_per_query": (self.total_input_tokens + self.total_output_tokens) + / self.num_cases + if self.num_cases > 0 + else 0, }, "latency": { "avg_ms": self.avg_latency_ms, @@ -92,7 +97,7 @@ class AgentEvaluator: print(f"Tool accuracy: {report.tool_accuracy:.1%}") """ - def __init__(self, test_cases_path: Optional[Path] = None): + def __init__(self, test_cases_path: Path | None = None): """ Parameters ---------- @@ -111,8 +116,8 @@ def __init__(self, test_cases_path: Optional[Path] = None): async def run_benchmark( self, agent, - tags: Optional[List[str]] = None, - max_cases: Optional[int] = None, + tags: list[str] | None = None, + max_cases: int | None = None, ) -> BenchmarkReport: """ Run benchmark against agent @@ -162,7 +167,7 @@ async def run_benchmark( metadata={"version": self.version, "tags": tags}, ) - async def _evaluate_case(self, agent, case: Dict) -> EvalResult: + async def _evaluate_case(self, agent, case: dict) -> EvalResult: """Evaluate a single test case""" test_id = case["id"] query = case["query"] @@ -172,6 +177,7 @@ async def _evaluate_case(self, agent, case: Dict) -> EvalResult: try: # Get tool call from agent (without executing) import time + start = time.perf_counter() tool_call = await self._get_tool_call(agent, query) @@ -211,7 +217,9 @@ async def _evaluate_case(self, agent, case: Dict) -> EvalResult: elif key not in actual_params: param_errors.append(f"missing param: {key}") elif actual_params[key] != expected_value: - param_errors.append(f"{key}: expected {expected_value}, got {actual_params[key]}") + param_errors.append( + f"{key}: expected {expected_value}, got {actual_params[key]}" + ) return EvalResult( test_id=test_id, @@ -238,7 +246,7 @@ async def _evaluate_case(self, agent, case: Dict) -> EvalResult: error=str(e), ) - async def _get_tool_call(self, agent, query: str) -> Optional[Dict]: + async def _get_tool_call(self, agent, query: str) -> dict | None: """ Get the tool call Claude would make for a query @@ -248,7 +256,7 @@ async def _get_tool_call(self, agent, query: str) -> Optional[Dict]: return await agent.get_tool_call(query) -def compare_reports(before: BenchmarkReport, after: BenchmarkReport) -> Dict: +def compare_reports(before: BenchmarkReport, after: BenchmarkReport) -> dict: """ Compare two benchmark reports @@ -271,8 +279,8 @@ def compare_reports(before: BenchmarkReport, after: BenchmarkReport) -> Dict: "tokens": { "before": before.total_input_tokens + before.total_output_tokens, "after": after.total_input_tokens + after.total_output_tokens, - "delta": (after.total_input_tokens + after.total_output_tokens) - - (before.total_input_tokens + before.total_output_tokens), + "delta": (after.total_input_tokens + after.total_output_tokens) + - (before.total_input_tokens + before.total_output_tokens), }, "latency_ms": { "before": before.avg_latency_ms, @@ -280,11 +288,13 @@ def compare_reports(before: BenchmarkReport, after: BenchmarkReport) -> Dict: "delta": after.avg_latency_ms - before.avg_latency_ms, }, "regressions": [ - r.test_id for r in after.results + r.test_id + for r in after.results if not r.passed and any(br.test_id == r.test_id and br.passed for br in before.results) ], "improvements": [ - r.test_id for r in after.results + r.test_id + for r in after.results if r.passed and any(br.test_id == r.test_id and not br.passed for br in before.results) ], } diff --git a/benchmarks/perception/__init__.py b/benchmarks/perception/__init__.py index c811d881..f26bbea4 100644 --- a/benchmarks/perception/__init__.py +++ b/benchmarks/perception/__init__.py @@ -5,9 +5,9 @@ """ from .ground_truth import GroundTruth -from .testset import OfflineTestset, TestCase -from .runner import PerceptionBenchmark, BenchmarkConfig, EmbryoResult from .metrics import PerceptionMetrics +from .runner import BenchmarkConfig, EmbryoResult, PerceptionBenchmark +from .testset import OfflineTestset, TestCase __all__ = [ "GroundTruth", diff --git a/benchmarks/perception/ground_truth.py b/benchmarks/perception/ground_truth.py index d3ce2a0a..71187596 100644 --- a/benchmarks/perception/ground_truth.py +++ b/benchmarks/perception/ground_truth.py @@ -7,11 +7,18 @@ import json from dataclasses import dataclass, field from pathlib import Path -from typing import Dict, List, Optional - # Stage progression order -STAGE_ORDER = ["early", "bean", "comma", "1.5fold", "2fold", "pretzel", "hatching", "hatched"] +STAGE_ORDER = [ + "early", + "bean", + "comma", + "1.5fold", + "2fold", + "pretzel", + "hatching", + "hatched", +] @dataclass @@ -24,14 +31,14 @@ class GroundTruth: """ # {embryo_id: {stage: start_timepoint}} - transitions: Dict[str, Dict[str, int]] = field(default_factory=dict) + transitions: dict[str, dict[str, int]] = field(default_factory=dict) # Metadata - session_id: Optional[str] = None - annotator: Optional[str] = None - notes: Optional[str] = None + session_id: str | None = None + annotator: str | None = None + notes: str | None = None - def get_stage_at(self, embryo_id: str, timepoint: int) -> Optional[str]: + def get_stage_at(self, embryo_id: str, timepoint: int) -> str | None: """ Get the ground truth stage for a given embryo at a given timepoint. @@ -63,15 +70,13 @@ def get_stage_at(self, embryo_id: str, timepoint: int) -> Optional[str]: return current_stage - def get_transition_timepoint( - self, embryo_id: str, stage: str - ) -> Optional[int]: + def get_transition_timepoint(self, embryo_id: str, stage: str) -> int | None: """Get the timepoint when a stage starts for a given embryo.""" if embryo_id not in self.transitions: return None return self.transitions[embryo_id].get(stage) - def get_stages_for_embryo(self, embryo_id: str) -> List[str]: + def get_stages_for_embryo(self, embryo_id: str) -> list[str]: """Get list of stages (in order) for a given embryo.""" if embryo_id not in self.transitions: return [] @@ -79,10 +84,7 @@ def get_stages_for_embryo(self, embryo_id: str) -> List[str]: embryo_transitions = self.transitions[embryo_id] # Sort by start timepoint - sorted_stages = sorted( - embryo_transitions.keys(), - key=lambda s: embryo_transitions[s] - ) + sorted_stages = sorted(embryo_transitions.keys(), key=lambda s: embryo_transitions[s]) return sorted_stages def get_timepoint_range(self, embryo_id: str) -> tuple: @@ -101,11 +103,11 @@ def get_timepoint_range(self, embryo_id: str) -> tuple: return (min(starts), max(starts)) @property - def embryo_ids(self) -> List[str]: + def embryo_ids(self) -> list[str]: """Get list of all embryo IDs with ground truth.""" return list(self.transitions.keys()) - def to_dict(self) -> Dict: + def to_dict(self) -> dict: """Serialize to dictionary for JSON storage.""" return { "session_id": self.session_id, @@ -115,7 +117,7 @@ def to_dict(self) -> Dict: } @classmethod - def from_dict(cls, data: Dict) -> "GroundTruth": + def from_dict(cls, data: dict) -> "GroundTruth": """Load from dictionary.""" return cls( transitions=data.get("transitions", {}), @@ -127,7 +129,7 @@ def from_dict(cls, data: Dict) -> "GroundTruth": @classmethod def from_json(cls, path: Path) -> "GroundTruth": """Load ground truth from JSON file.""" - with open(path, "r") as f: + with open(path) as f: data = json.load(f) return cls.from_dict(data) @@ -139,9 +141,9 @@ def save_json(self, path: Path) -> None: def create_ground_truth_from_email_format( - annotations: Dict[str, str], - session_id: Optional[str] = None, - annotator: Optional[str] = None, + annotations: dict[str, str], + session_id: str | None = None, + annotator: str | None = None, ) -> GroundTruth: """ Create GroundTruth from email-style annotations. diff --git a/benchmarks/perception/live_viewer.py b/benchmarks/perception/live_viewer.py index a6ea2456..0a311786 100644 --- a/benchmarks/perception/live_viewer.py +++ b/benchmarks/perception/live_viewer.py @@ -13,22 +13,19 @@ import argparse import asyncio -import base64 import json import logging import sys import webbrowser -from dataclasses import asdict from datetime import datetime, timedelta from pathlib import Path -from typing import Any, Dict, List, Optional +from typing import Any # FastAPI and websockets try: + import uvicorn from fastapi import FastAPI, WebSocket, WebSocketDisconnect from fastapi.responses import HTMLResponse - from fastapi.staticfiles import StaticFiles - import uvicorn except ImportError: print("Please install: pip install fastapi uvicorn websockets") sys.exit(1) @@ -39,8 +36,8 @@ logger = logging.getLogger(__name__) # Global state for websocket connections -connected_clients: List[WebSocket] = [] -benchmark_state: Dict[str, Any] = { +connected_clients: list[WebSocket] = [] +benchmark_state: dict[str, Any] = { "status": "idle", "current_embryo": None, "current_timepoint": None, @@ -49,10 +46,10 @@ "verification_active": False, } is_paused: bool = False -pause_event: asyncio.Event = None # Will be initialized on startup +pause_event: asyncio.Event | None = None # Will be initialized on startup -async def broadcast(message: Dict): +async def broadcast(message: dict): """Broadcast message to all connected clients.""" if not connected_clients: return @@ -396,7 +393,10 @@ async def broadcast(message: Dict):

Perception Benchmark Live Viewer

- +
Connecting...
@@ -555,8 +555,10 @@ async def broadcast(message: Dict): // Display three-view combined image (XY+YZ+XZ orthogonal projections) container.innerHTML = `
-
THREE-VIEW (XY | YZ / XZ)
- Three-View T${timepoint} +
+ THREE-VIEW (XY | YZ / XZ)
+ Three-View T${timepoint}
`; // Also get embryoId and groundTruth from stored data if not provided embryoId = embryoId || imgData.embryoId; @@ -628,10 +630,13 @@ async def broadcast(message: Dict): // Update header document.querySelector('.trace-section h2').innerHTML = - `Reasoning Trace - T${timepoint} `; + `Reasoning Trace - T${timepoint} `; if (traceSteps.length === 0) { - list.innerHTML = '
No trace recorded for T' + timepoint + '
'; + list.innerHTML = '
No trace recorded for T' + + timepoint + '
'; return; } @@ -685,12 +690,16 @@ async def broadcast(message: Dict): 'verified' : ''; html += ` -
+
T${pred.timepoint}
-
${pred.predicted}
-
${pred.ground_truth}
+
${pred.predicted}
+
${pred.ground_truth}
${(pred.confidence * 100).toFixed(0)}%
-
${pred.phase_count > 1 ? pred.phase_count + '-phase' : ''}${verifiedBadge}
+
${pred.phase_count > 1 + ? pred.phase_count + '-phase' : ''}${verifiedBadge}
`; } @@ -749,10 +758,14 @@ async def websocket_endpoint(websocket: WebSocket): connected_clients.append(websocket) # Send current state - await websocket.send_text(json.dumps({ - "type": "status", - "status": benchmark_state["status"], - })) + await websocket.send_text( + json.dumps( + { + "type": "status", + "status": benchmark_state["status"], + } + ) + ) try: while True: @@ -789,8 +802,8 @@ def __init__( embryo_id: str, enable_verification: bool = True, start_timepoint: int = 0, - max_timepoints: Optional[int] = None, - trace_dir: Optional[Path] = None, + max_timepoints: int | None = None, + trace_dir: Path | None = None, ): self.testset = testset self.embryo_id = embryo_id @@ -798,7 +811,7 @@ def __init__( self.start_timepoint = start_timepoint self.max_timepoints = max_timepoints - self.predictions: List[Dict] = [] + self.predictions: list[dict] = [] self.correct_count = 0 self.adjacent_count = 0 self.verified_count = 0 @@ -808,7 +821,7 @@ def __init__( self.trace_dir = trace_dir or Path("benchmarks/results/traces") self.run_dir = self.trace_dir / f"{self.run_id}_{embryo_id}" self.run_dir.mkdir(parents=True, exist_ok=True) - self.traces: Dict[int, List[Dict]] = {} # timepoint -> trace steps + self.traces: dict[int, list[dict]] = {} # timepoint -> trace steps logger.info(f"Trace persistence enabled: {self.run_dir}") @@ -861,23 +874,23 @@ async def run(self): benchmark_state["current_timepoint"] = test_case.timepoint # Send image(s) - await broadcast({ - "type": "image", - "embryo_id": self.embryo_id, - "timepoint": test_case.timepoint, - "ground_truth": test_case.ground_truth_stage, - "image": test_case.image_b64, # Combined for backward compat - "top_image": test_case.top_image_b64, - "side_image": test_case.side_image_b64, - }) + await broadcast( + { + "type": "image", + "embryo_id": self.embryo_id, + "timepoint": test_case.timepoint, + "ground_truth": test_case.ground_truth_stage, + "image": test_case.image_b64, # Combined for backward compat + "top_image": test_case.top_image_b64, + "side_image": test_case.side_image_b64, + } + ) # Clear trace for new prediction await broadcast({"type": "clear_trace"}) # Run perception with trace streaming - result = await self._run_perception_with_streaming( - engine, session, test_case - ) + result = await self._run_perception_with_streaming(engine, session, test_case) # Check accuracy is_correct = result.stage == test_case.ground_truth_stage @@ -915,23 +928,29 @@ async def run(self): await broadcast(pred_msg) # Save trace for this timepoint - self._save_timepoint_trace(test_case.timepoint, result, test_case, is_correct, is_adjacent) + self._save_timepoint_trace( + test_case.timepoint, result, test_case, is_correct, is_adjacent + ) # Send updated stats total = len(self.predictions) - await broadcast({ - "type": "stats", - "accuracy": self.correct_count / total if total > 0 else None, - "adjacent": self.adjacent_count / total if total > 0 else None, - "total": total, - "verified": self.verified_count, - }) + await broadcast( + { + "type": "stats", + "accuracy": self.correct_count / total if total > 0 else None, + "adjacent": self.adjacent_count / total if total > 0 else None, + "total": total, + "verified": self.verified_count, + } + ) # Add observation to session with simulated timestamp # Typical diSPIM acquisition interval is ~4 minutes per timepoint - simulated_timestamp = datetime.now() - timedelta( - minutes=(self.max_timepoints or 100) * 4 - ) + timedelta(minutes=test_case.timepoint * 4) + simulated_timestamp = ( + datetime.now() + - timedelta(minutes=(self.max_timepoints or 100) * 4) + + timedelta(minutes=test_case.timepoint * 4) + ) session.add_observation( timepoint=test_case.timepoint, @@ -953,7 +972,6 @@ async def run(self): async def _run_perception_with_streaming(self, engine, session, test_case): """Run perception and stream trace steps.""" - from gently.harness.perception.session import ReasoningStep # We need to hook into the reasoning trace # For now, run perception and stream the trace after @@ -985,7 +1003,9 @@ async def _run_perception_with_streaming(self, engine, session, test_case): return result - def _save_timepoint_trace(self, timepoint: int, result, test_case, is_correct: bool, is_adjacent: bool): + def _save_timepoint_trace( + self, timepoint: int, result, test_case, is_correct: bool, is_adjacent: bool + ): """Save trace for a single timepoint to disk.""" trace_data = { "timepoint": timepoint, @@ -1035,7 +1055,7 @@ async def run_benchmark_background( embryo_id: str, enable_verification: bool, start_timepoint: int = 0, - max_timepoints: Optional[int] = None, + max_timepoints: int | None = None, ): """Run benchmark in background after server starts.""" print("[DEBUG] run_benchmark_background starting", flush=True) @@ -1049,7 +1069,10 @@ async def run_benchmark_background( # Load data ground_truth = GroundTruth.from_json(ground_truth_path) - print(f"[DEBUG] Loaded ground truth: {len(ground_truth.transitions)} embryos", flush=True) + print( + f"[DEBUG] Loaded ground truth: {len(ground_truth.transitions)} embryos", + flush=True, + ) testset = OfflineTestset( session_path=session_path, ground_truth=ground_truth, @@ -1071,6 +1094,7 @@ async def run_benchmark_background( except Exception as e: print(f"[DEBUG] ERROR in run_benchmark_background: {e}", flush=True) import traceback + traceback.print_exc() @@ -1142,9 +1166,9 @@ def main(): trace_dir = Path("benchmarks/results/traces") run_id = datetime.now().strftime("%Y%m%d_%H%M%S") - print(f"\n{'='*60}") + print(f"\n{'=' * 60}") print("Perception Benchmark Live Viewer") - print(f"{'='*60}") + print(f"{'=' * 60}") print(f"Session: {session_path}") print(f"Embryo: {args.embryo}") print(f"Start timepoint: T{args.start_timepoint}") @@ -1152,7 +1176,7 @@ def main(): print(f"Verification: {'disabled' if args.no_verification else 'enabled'}") print(f"Traces: {trace_dir / f'{run_id}_{args.embryo}'}") print(f"URL: http://localhost:{args.port}") - print(f"{'='*60}\n") + print(f"{'=' * 60}\n") # Open browser if not args.no_browser: @@ -1162,14 +1186,16 @@ def main(): @app.on_event("startup") async def startup_event(): print("[DEBUG] Startup event fired", flush=True) - asyncio.create_task(run_benchmark_background( - session_path=session_path, - ground_truth_path=gt_path, - embryo_id=args.embryo, - enable_verification=not args.no_verification, - start_timepoint=args.start_timepoint, - max_timepoints=args.max_timepoints, - )) + asyncio.create_task( + run_benchmark_background( + session_path=session_path, + ground_truth_path=gt_path, + embryo_id=args.embryo, + enable_verification=not args.no_verification, + start_timepoint=args.start_timepoint, + max_timepoints=args.max_timepoints, + ) + ) print("[DEBUG] Background task created", flush=True) # Run server diff --git a/benchmarks/perception/metrics.py b/benchmarks/perception/metrics.py index f69c054f..97d5c0f9 100644 --- a/benchmarks/perception/metrics.py +++ b/benchmarks/perception/metrics.py @@ -6,14 +6,23 @@ from collections import defaultdict from dataclasses import dataclass, field -from typing import Any, Dict, List, Optional, Tuple, TYPE_CHECKING +from typing import TYPE_CHECKING, Any if TYPE_CHECKING: from .runner import BenchmarkReport # Stage order for metrics -STAGE_ORDER = ["early", "bean", "comma", "1.5fold", "2fold", "pretzel", "hatching", "hatched"] +STAGE_ORDER = [ + "early", + "bean", + "comma", + "1.5fold", + "2fold", + "pretzel", + "hatching", + "hatched", +] @dataclass @@ -25,30 +34,30 @@ class PerceptionMetrics: adjacent_accuracy: float = 0.0 # Within 1 stage # Per-stage accuracy - stage_accuracy: Dict[str, float] = field(default_factory=dict) - stage_counts: Dict[str, int] = field(default_factory=dict) + stage_accuracy: dict[str, float] = field(default_factory=dict) + stage_counts: dict[str, int] = field(default_factory=dict) # Confusion matrix: confusion[gt_stage][pred_stage] = count - confusion_matrix: Dict[str, Dict[str, int]] = field(default_factory=dict) + confusion_matrix: dict[str, dict[str, int]] = field(default_factory=dict) # Confidence calibration mean_confidence: float = 0.0 confidence_when_correct: float = 0.0 confidence_when_wrong: float = 0.0 - calibration_bins: List[Tuple[float, float, int]] = field(default_factory=list) + calibration_bins: list[tuple[float, float, int]] = field(default_factory=list) # (confidence_bin_center, accuracy_in_bin, count) expected_calibration_error: float = 0.0 # ECE # Temporal metrics backward_transitions: int = 0 # Errors where stage went backward - stage_transition_delay: Dict[str, float] = field(default_factory=dict) + stage_transition_delay: dict[str, float] = field(default_factory=dict) # How many timepoints after GT transition until prediction caught up # Tool usage total_tool_calls: int = 0 tool_call_rate: float = 0.0 # Avg tool calls per prediction - tool_use_by_stage: Dict[str, float] = field(default_factory=dict) + tool_use_by_stage: dict[str, float] = field(default_factory=dict) # When tools were used vs not accuracy_with_tools: float = 0.0 @@ -59,7 +68,7 @@ class PerceptionMetrics: transitional_rate: float = 0.0 transitional_accuracy: float = 0.0 # Accuracy when marked transitional - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "accuracy": self.accuracy, "adjacent_accuracy": self.adjacent_accuracy, @@ -102,8 +111,7 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: # Collect all predictions all_preds = [ - p for r in report.embryo_results for p in r.predictions - if p.ground_truth_stage is not None + p for r in report.embryo_results for p in r.predictions if p.ground_truth_stage is not None ] if not all_preds: @@ -116,8 +124,8 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: metrics.adjacent_accuracy = adjacent_correct / len(all_preds) # Per-stage accuracy - stage_correct = defaultdict(int) - stage_total = defaultdict(int) + stage_correct: dict = defaultdict(int) + stage_total: dict = defaultdict(int) for p in all_preds: gt = p.ground_truth_stage @@ -130,13 +138,11 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: metrics.stage_accuracy[stage] = stage_correct[stage] / stage_total[stage] # Confusion matrix - confusion = defaultdict(lambda: defaultdict(int)) + confusion: dict = defaultdict(lambda: defaultdict(int)) for p in all_preds: confusion[p.ground_truth_stage][p.predicted_stage] += 1 - metrics.confusion_matrix = { - gt: dict(preds) for gt, preds in confusion.items() - } + metrics.confusion_matrix = {gt: dict(preds) for gt, preds in confusion.items()} # Confidence statistics confidences = [p.confidence for p in all_preds] @@ -156,10 +162,7 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: bin_high = (i + 1) / num_bins bin_center = (bin_low + bin_high) / 2 - bin_preds = [ - p for p in all_preds - if bin_low <= p.confidence < bin_high - ] + bin_preds = [p for p in all_preds if bin_low <= p.confidence < bin_high] if bin_preds: bin_accuracy = sum(1 for p in bin_preds if p.is_correct) / len(bin_preds) @@ -198,6 +201,8 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: stage_tool_calls[p.ground_truth_stage].append(p.tool_calls) for stage, calls in stage_tool_calls.items(): + if stage is None: + continue metrics.tool_use_by_stage[stage] = sum(calls) / len(calls) # Accuracy with vs without tools @@ -207,7 +212,9 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: if with_tools: metrics.accuracy_with_tools = sum(1 for p in with_tools if p.is_correct) / len(with_tools) if without_tools: - metrics.accuracy_without_tools = sum(1 for p in without_tools if p.is_correct) / len(without_tools) + metrics.accuracy_without_tools = sum(1 for p in without_tools if p.is_correct) / len( + without_tools + ) # Transitional observations transitional_preds = [p for p in all_preds if p.is_transitional] @@ -215,16 +222,16 @@ def compute_metrics(report: "BenchmarkReport") -> PerceptionMetrics: metrics.transitional_rate = len(transitional_preds) / len(all_preds) if transitional_preds: - metrics.transitional_accuracy = sum( - 1 for p in transitional_preds if p.is_correct - ) / len(transitional_preds) + metrics.transitional_accuracy = sum(1 for p in transitional_preds if p.is_correct) / len( + transitional_preds + ) return metrics def format_confusion_matrix( - confusion: Dict[str, Dict[str, int]], - stages: Optional[List[str]] = None, + confusion: dict[str, dict[str, int]], + stages: list[str] | None = None, ) -> str: """Format confusion matrix as ASCII table.""" if stages is None: @@ -278,34 +285,38 @@ def format_metrics_summary(metrics: PerceptionMetrics) -> str: count = metrics.stage_counts[stage] lines.append(f" {stage:>10}: {acc:.1%} (n={count})") - lines.extend([ - "", - "CONFIDENCE CALIBRATION", - f" Mean confidence: {metrics.mean_confidence:.2f}", - f" Confidence (correct): {metrics.confidence_when_correct:.2f}", - f" Confidence (wrong): {metrics.confidence_when_wrong:.2f}", - f" Expected Cal. Error: {metrics.expected_calibration_error:.3f}", - "", - "TOOL USAGE", - f" Total tool calls: {metrics.total_tool_calls}", - f" Avg calls per pred: {metrics.tool_call_rate:.2f}", - f" Accuracy with tools: {metrics.accuracy_with_tools:.1%}", - f" Accuracy without tools: {metrics.accuracy_without_tools:.1%}", - "", - "TEMPORAL", - f" Backward transitions: {metrics.backward_transitions}", - "", - "TRANSITIONAL OBSERVATIONS", - f" Count: {metrics.transitional_count}", - f" Rate: {metrics.transitional_rate:.1%}", - f" Accuracy: {metrics.transitional_accuracy:.1%}", - ]) + lines.extend( + [ + "", + "CONFIDENCE CALIBRATION", + f" Mean confidence: {metrics.mean_confidence:.2f}", + f" Confidence (correct): {metrics.confidence_when_correct:.2f}", + f" Confidence (wrong): {metrics.confidence_when_wrong:.2f}", + f" Expected Cal. Error: {metrics.expected_calibration_error:.3f}", + "", + "TOOL USAGE", + f" Total tool calls: {metrics.total_tool_calls}", + f" Avg calls per pred: {metrics.tool_call_rate:.2f}", + f" Accuracy with tools: {metrics.accuracy_with_tools:.1%}", + f" Accuracy without tools: {metrics.accuracy_without_tools:.1%}", + "", + "TEMPORAL", + f" Backward transitions: {metrics.backward_transitions}", + "", + "TRANSITIONAL OBSERVATIONS", + f" Count: {metrics.transitional_count}", + f" Rate: {metrics.transitional_rate:.1%}", + f" Accuracy: {metrics.transitional_accuracy:.1%}", + ] + ) if metrics.confusion_matrix: - lines.extend([ - "", - "CONFUSION MATRIX", - format_confusion_matrix(metrics.confusion_matrix), - ]) + lines.extend( + [ + "", + "CONFUSION MATRIX", + format_confusion_matrix(metrics.confusion_matrix), + ] + ) return "\n".join(lines) diff --git a/benchmarks/perception/runner.py b/benchmarks/perception/runner.py index 33d93fe3..70ac32fd 100644 --- a/benchmarks/perception/runner.py +++ b/benchmarks/perception/runner.py @@ -12,11 +12,11 @@ from dataclasses import dataclass, field from datetime import datetime from pathlib import Path -from typing import Any, Dict, List, Optional +from typing import Any from .ground_truth import GroundTruth -from .testset import OfflineTestset, TestCase from .metrics import PerceptionMetrics, compute_metrics +from .testset import OfflineTestset logger = logging.getLogger(__name__) @@ -39,20 +39,20 @@ class BenchmarkConfig: # Test settings start_timepoint: int = 0 - max_timepoints_per_embryo: Optional[int] = None - embryo_ids: Optional[List[str]] = None # None = all + max_timepoints_per_embryo: int | None = None + embryo_ids: list[str] | None = None # None = all # Ablation toggles include_temporal_context: bool = True include_previous_observations: bool = True # Custom system prompt override - system_prompt_override: Optional[str] = None + system_prompt_override: str | None = None # Metadata description: str = "" - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "model": self.model, "temperature": self.temperature, @@ -78,20 +78,20 @@ class PredictionResult: timepoint: int predicted_stage: str - ground_truth_stage: Optional[str] + ground_truth_stage: str | None confidence: float is_transitional: bool - transition_between: Optional[List[str]] + transition_between: list[str] | None reasoning: str - reasoning_trace: Optional[Dict[str, Any]] # Serialized ReasoningTrace + reasoning_trace: dict[str, Any] | None # Serialized ReasoningTrace tool_calls: int - tools_used: List[str] + tools_used: list[str] # Multi-phase verification fields verification_triggered: bool = False phase_count: int = 1 - verification_result: Optional[Dict[str, Any]] = None - candidate_stages: Optional[List[Dict[str, Any]]] = None + verification_result: dict[str, Any] | None = None + candidate_stages: list[dict[str, Any]] | None = None @property def is_correct(self) -> bool: @@ -113,7 +113,7 @@ def is_adjacent_correct(self) -> bool: except ValueError: return False - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "timepoint": self.timepoint, "predicted_stage": self.predicted_stage, @@ -139,9 +139,9 @@ class EmbryoResult: """Results for a single embryo run.""" embryo_id: str - predictions: List[PredictionResult] = field(default_factory=list) + predictions: list[PredictionResult] = field(default_factory=list) duration_seconds: float = 0.0 - error: Optional[str] = None + error: str | None = None @property def accuracy(self) -> float: @@ -159,7 +159,7 @@ def adjacent_accuracy(self) -> float: correct = sum(1 for p in self.predictions if p.is_adjacent_correct) return correct / len(self.predictions) - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "embryo_id": self.embryo_id, "predictions": [p.to_dict() for p in self.predictions], @@ -175,11 +175,11 @@ class BenchmarkReport: """Complete benchmark report.""" config: BenchmarkConfig - embryo_results: List[EmbryoResult] = field(default_factory=list) - metrics: Optional[PerceptionMetrics] = None + embryo_results: list[EmbryoResult] = field(default_factory=list) + metrics: PerceptionMetrics | None = None started_at: datetime = field(default_factory=datetime.now) - completed_at: Optional[datetime] = None - session_id: Optional[str] = None + completed_at: datetime | None = None + session_id: str | None = None @property def total_predictions(self) -> int: @@ -192,7 +192,7 @@ def overall_accuracy(self) -> float: return 0.0 return sum(1 for p in all_preds if p.is_correct) / len(all_preds) - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "config": self.config.to_dict(), "embryo_results": [r.to_dict() for r in self.embryo_results], @@ -222,7 +222,7 @@ def __init__( self, testset: OfflineTestset, config: BenchmarkConfig, - engine: Optional[Any] = None, # PerceptionEngine + engine: Any | None = None, # PerceptionEngine ): """ Parameters @@ -246,7 +246,6 @@ async def _get_engine(self): # Lazy import to avoid circular dependencies import anthropic from gently.harness.perception.engine import PerceptionEngine - from gently.harness.perception.example_store import ExampleStore client = anthropic.Anthropic() @@ -465,7 +464,8 @@ async def main(): help="Description for this benchmark run", ) parser.add_argument( - "-v", "--verbose", + "-v", + "--verbose", action="store_true", help="Verbose logging", ) @@ -481,6 +481,7 @@ async def main(): # The perception engine reads stage definitions etc. from the active # organism module, which is normally loaded by launch_gently.py. from gently.organisms import load_organism + load_organism("celegans") # Find session path diff --git a/benchmarks/perception/testset.py b/benchmarks/perception/testset.py index 658cc011..c3514ffe 100644 --- a/benchmarks/perception/testset.py +++ b/benchmarks/perception/testset.py @@ -6,18 +6,19 @@ import base64 import io +from collections.abc import Iterator from dataclasses import dataclass from datetime import datetime from pathlib import Path -from typing import Iterator, List, Optional, Tuple, Dict +from typing import Any import numpy as np from .ground_truth import GroundTruth # Lazy imports -tifffile = None -PIL_Image = None +tifffile: Any = None +PIL_Image: Any = None def _ensure_dependencies(): @@ -26,10 +27,12 @@ def _ensure_dependencies(): if tifffile is None: import tifffile as _tifffile + tifffile = _tifffile if PIL_Image is None: from PIL import Image as _Image + PIL_Image = _Image @@ -40,27 +43,29 @@ class TestCase: embryo_id: str timepoint: int image_b64: str # Combined view (for backward compatibility) - top_image_b64: Optional[str] # TOP view only - side_image_b64: Optional[str] # SIDE view only - volume: Optional[np.ndarray] - ground_truth_stage: Optional[str] - acquired_at: Optional[datetime] = None + top_image_b64: str | None # TOP view only + side_image_b64: str | None # SIDE view only + volume: np.ndarray | None + ground_truth_stage: str | None + acquired_at: datetime | None = None def _discover_volumes( - session_dir: Path, embryo_id: Optional[str] = None -) -> Dict[str, List[Tuple[datetime, Path]]]: + session_dir: Path, embryo_id: str | None = None +) -> dict[str, list[tuple[datetime, Path]]]: """Discover volume files (with parsed acquisition timestamps) in a session directory.""" if not session_dir.exists(): return {} tif_files = ( - list(session_dir.glob("*.tif")) + list(session_dir.glob("*.tiff")) - + list(session_dir.glob("**/*.tif")) + list(session_dir.glob("**/*.tiff")) + list(session_dir.glob("*.tif")) + + list(session_dir.glob("*.tiff")) + + list(session_dir.glob("**/*.tif")) + + list(session_dir.glob("**/*.tiff")) ) # Deduplicate (flat + recursive may overlap) tif_files = list({f.resolve(): f for f in tif_files}.values()) - embryo_volumes = {} + embryo_volumes: dict = {} for f in tif_files: parts = f.stem.split("_") @@ -89,6 +94,7 @@ def _discover_volumes( def _load_volume(path: Path) -> np.ndarray: """Load a volume from TIFF file.""" from gently.core.imaging import load_volume + return load_volume(path) @@ -122,9 +128,9 @@ def _create_three_view_image(volume: np.ndarray, max_dim: int = 1500) -> str: _ensure_dependencies() from gently.core.imaging import ( - projection_three_view, - compute_crop_bounds, apply_crop_bounds, + compute_crop_bounds, + projection_three_view, ) # Auto-crop to embryo region @@ -150,7 +156,7 @@ def _create_three_view_image(volume: np.ndarray, max_dim: int = 1500) -> str: return base64.b64encode(buffer.getvalue()).decode("utf-8") -def _create_separate_view_images(volume: np.ndarray, max_dim: int = 1000) -> Tuple[str, str]: +def _create_separate_view_images(volume: np.ndarray, max_dim: int = 1000) -> tuple[str, str]: """Create separate TOP and SIDE view images from volume, return base64 tuple. Parameters @@ -244,7 +250,7 @@ def __init__( self._embryo_volumes = _discover_volumes(self.session_path) @property - def embryo_ids(self) -> List[str]: + def embryo_ids(self) -> list[str]: """Get list of embryo IDs with both volumes and ground truth.""" gt_embryos = set(self.ground_truth.embryo_ids) vol_embryos = set(self._embryo_volumes.keys()) @@ -258,7 +264,7 @@ def iter_embryo( self, embryo_id: str, start_timepoint: int = 0, - end_timepoint: Optional[int] = None, + end_timepoint: int | None = None, ) -> Iterator[TestCase]: """ Iterate through timepoints for an embryo sequentially. @@ -318,7 +324,7 @@ def iter_embryo( acquired_at=acquired_at, ) - def iter_all(self) -> Iterator[Tuple[str, Iterator[TestCase]]]: + def iter_all(self) -> Iterator[tuple[str, Iterator[TestCase]]]: """ Iterate through all embryos in the testset. diff --git a/benchmarks/perception/trace_viewer.py b/benchmarks/perception/trace_viewer.py index 5a3ced2d..69ea5fa6 100644 --- a/benchmarks/perception/trace_viewer.py +++ b/benchmarks/perception/trace_viewer.py @@ -7,12 +7,9 @@ import argparse import json import sys -from datetime import datetime from pathlib import Path -from typing import Any, Dict, List, Optional - -from .metrics import format_metrics_summary, PerceptionMetrics +from .metrics import PerceptionMetrics, format_metrics_summary HTML_TEMPLATE = """ @@ -293,16 +290,16 @@ """ -def generate_embryo_section(embryo_id: str, predictions: List[Dict]) -> str: +def generate_embryo_section(embryo_id: str, predictions: list[dict]) -> str: """Generate HTML for one embryo's predictions.""" rows = [ '
', - '
Timepoint
', - '
Predicted
', - '
Ground Truth
', - '
Confidence
', - '
Details
', - '
', + "
Timepoint
", + "
Predicted
", + "
Ground Truth
", + "
Confidence
", + "
Details
", + "
", ] for i, pred in enumerate(predictions): @@ -320,15 +317,15 @@ def generate_embryo_section(embryo_id: str, predictions: List[Dict]) -> str: row_id = f"{embryo_id}-{timepoint}" rows.append(f'
') - rows.append(f'
T{timepoint}
') + rows.append(f"
T{timepoint}
") rows.append(f'
{pred_stage}
') rows.append(f'
{gt_stage}
') - rows.append(f'''
+ rows.append(f"""
{confidence:.0%} -
''') +
""") # Details column with expand button tool_calls = pred.get("tool_calls", 0) @@ -351,31 +348,31 @@ def generate_embryo_section(embryo_id: str, predictions: List[Dict]) -> str: if phase_count > 1: badges += f'{phase_count}-phase' - rows.append(f'''
+ rows.append(f"""
{details_str} {badges} [show trace] -
''') - rows.append('
') +
""") + rows.append("") # Reasoning trace (hidden by default) trace_html = format_reasoning_trace(pred.get("reasoning_trace")) - rows.append(f'''
+ rows.append(f"""
Reasoning: {reasoning} {trace_html} -
''') +
""") - return f''' + return f"""

{embryo_id}

{"".join(rows)}
- ''' + """ -def format_reasoning_trace(trace: Optional[Dict]) -> str: +def format_reasoning_trace(trace: dict | None) -> str: """Format reasoning trace as HTML.""" if not trace: return "" @@ -393,56 +390,66 @@ def format_reasoning_trace(trace: Optional[Dict]) -> str: if step_type == "tool_call": tool_name = step.get("tool_name", "") tool_input = step.get("tool_input", {}) - html_parts.append(f''' + html_parts.append(f"""
Tool Call: {tool_name}
Input: {json.dumps(tool_input, indent=2)}
- ''') + """) elif step_type == "tool_result": summary = step.get("tool_result_summary", content) - html_parts.append(f''' + html_parts.append(f"""
Tool Result: {summary}
- ''') + """) elif step_type == "final_decision": - html_parts.append(f''' + html_parts.append(f"""
Final Decision:
{content[:500]}...
- ''') + """) elif step_type == "verification_requested": - html_parts.append(f''' + html_parts.append(f"""
Verification Requested:
{content}
- ''') + """) elif step_type == "verification_subagent": tool_input = step.get("tool_input", {}) summary = step.get("tool_result_summary", content) - html_parts.append(f''' + html_parts.append(f"""
- Subagent: {tool_input.get("stage_a", "?")} vs {tool_input.get("stage_b", "?")}
+ Subagent: {tool_input.get("stage_a", "?")} vs + {tool_input.get("stage_b", "?")}
Result: {summary}
- ''') + """) elif step_type == "verification_result": - html_parts.append(f''' + html_parts.append(f"""
Verification Result:
{content}
- ''') + """) return "".join(html_parts) -def generate_confusion_matrix_html(confusion: Dict[str, Dict[str, int]]) -> str: +def generate_confusion_matrix_html(confusion: dict[str, dict[str, int]]) -> str: """Generate HTML table for confusion matrix.""" - stages = ["early", "bean", "comma", "1.5fold", "2fold", "pretzel", "hatching", "hatched"] + stages = [ + "early", + "bean", + "comma", + "1.5fold", + "2fold", + "pretzel", + "hatching", + "hatched", + ] # Filter to stages present in data present = set() @@ -478,7 +485,7 @@ def generate_confusion_matrix_html(confusion: Dict[str, Dict[str, int]]) -> str: return "".join(rows) -def generate_html_report(report_data: Dict) -> str: +def generate_html_report(report_data: dict) -> str: """Generate complete HTML report from benchmark data.""" # Extract summary metrics metrics = report_data.get("metrics", {}) @@ -493,9 +500,7 @@ def generate_html_report(report_data: Dict) -> str: # Generate embryo sections embryo_sections = [] for er in embryo_results: - embryo_sections.append( - generate_embryo_section(er["embryo_id"], er["predictions"]) - ) + embryo_sections.append(generate_embryo_section(er["embryo_id"], er["predictions"])) # Generate confusion matrix confusion = metrics.get("confusion_matrix", {}) @@ -569,8 +574,7 @@ def main(): # Filter embryo if specified if args.embryo: report_data["embryo_results"] = [ - er for er in report_data.get("embryo_results", []) - if er["embryo_id"] == args.embryo + er for er in report_data.get("embryo_results", []) if er["embryo_id"] == args.embryo ] # Generate HTML diff --git a/benchmarks/runner.py b/benchmarks/runner.py index 4d6d847a..7dd31236 100644 --- a/benchmarks/runner.py +++ b/benchmarks/runner.py @@ -13,8 +13,6 @@ import json import logging import sys -from datetime import datetime -from pathlib import Path logging.basicConfig(level=logging.INFO, format="%(message)s") logger = logging.getLogger(__name__) @@ -66,7 +64,6 @@ async def run_agent_benchmark(args): def compare_reports(args): """Compare two benchmark reports""" - from .agent.evaluator import BenchmarkReport, compare_reports as _compare with open(args.before) as f: before_data = json.load(f) @@ -101,7 +98,10 @@ def compare_reports(args): delta_str = f"+{fmt.format(delta)}" if delta > 0 else fmt.format(delta) status = "improved" if delta > 0 else ("regressed" if delta < 0 else "unchanged") - logger.info(f" {name}: {fmt.format(before_val)} -> {fmt.format(after_val)} ({delta_str}) [{status}]") + logger.info( + f" {name}: {fmt.format(before_val)} -> {fmt.format(after_val)}" + f" ({delta_str}) [{status}]" + ) # Token comparison before_tokens = before_data.get("tokens", {}) @@ -111,7 +111,7 @@ def compare_reports(args): after_total = after_tokens.get("total_input", 0) + after_tokens.get("total_output", 0) token_delta = after_total - before_total - logger.info(f"\nTokens:") + logger.info("\nTokens:") logger.info(f" Total: {before_total:,} -> {after_total:,} ({token_delta:+,})") return 0 diff --git a/config/config.yml b/config/config.yml index e59e9f6d..9c05c274 100644 --- a/config/config.yml +++ b/config/config.yml @@ -1,4 +1,38 @@ organism: "celegans" hardware: "dispim" mmconfig: "MMConfig_tracking_screening.cfg" -mmdirectory: "C:/Program Files/Micro-Manager-1.4" \ No newline at end of file +mmdirectory: "C:/Program Files/Micro-Manager-1.4" + +# SwitchBot Bot — physical button-pusher mounted on the diSPIM room light +# switch. Talks BLE direct (no SwitchBot Hub / cloud). Plans address it by +# name, e.g. `bps.mv(room_light, 'on')`. Remove this block to skip +# registration; the device layer is tolerant of either state. +switchbot: + name: room_light + address: "EC:6F:04:06:5B:23" + timeout: 20.0 + +# ACUITYnano Precision Thermal Controller (Peltier/TEC, 0.0–99.9 °C). When this +# block is present the device layer registers a `temperature` device and the +# Devices tab shows a setpoint control; plans can also block on it via +# `bps.mv(temperature, 20.0)`. Remove/comment the block to skip registration. +# +# backend: mqtt talks to the controller over the vendor's MQTT bridge +# (acuitynano_precision_thermalizer_api). With no broker/port/user/password +# keys it uses the vendor package's embedded HiveMQ Cloud defaults — set them +# here only to point at a different broker. The vendor package must be +# installed on the device-layer machine (not on PyPI). +temperature: + name: temperature + backend: serial # serial | mqtt | mock — USB serial is the working + # link on this machine (MQTT cloud is firewalled) + com_port: "COM8" + baud_rate: 115200 + stabilize_timeout: 600 # seconds to wait for "[ SYSTEM LOCKED ]" on a blocking set + feedback_peltier: false # true = control off the peltier sensor instead of water + # MQTT alternative (vendor HiveMQ Cloud defaults; needs outbound TLS :8883): + # backend: mqtt + # broker: "your-broker.example.com" # optional — overrides the embedded default + # port: 8883 + # user: "username" + # password: "secret" \ No newline at end of file diff --git a/desktop/.gitignore b/desktop/.gitignore new file mode 100644 index 00000000..6b3daa59 --- /dev/null +++ b/desktop/.gitignore @@ -0,0 +1,14 @@ +# Node / Tauri CLI +node_modules/ + +# Rust / Tauri build output +src-tauri/target/ +src-tauri/gen/ + +# Generated mobile / MS-Store icon variants (regenerate with `tauri icon`). +# The desktop icons referenced by tauri.conf.json (32x32, 128x128, 128x128@2x, +# icon.ico, icon.icns, icon.png) ARE committed so a clean checkout builds. +src-tauri/icons/android/ +src-tauri/icons/ios/ +src-tauri/icons/Square*Logo.png +src-tauri/icons/StoreLogo.png diff --git a/desktop/README.md b/desktop/README.md new file mode 100644 index 00000000..a18e4c44 --- /dev/null +++ b/desktop/README.md @@ -0,0 +1,159 @@ +# Gently Desktop (Tauri shell) + +A thin [Tauri](https://v2.tauri.app) desktop wrapper that turns gently into a +double-click Windows app. It **owns** the Python backend: on launch it spawns +`launch_gently.py --no-browser`, shows a splash while the server boots, then +renders the existing web UI in a native WebView2 window. No application logic +lives here — the web UI served by Python stays the single source of truth. + +This is the desktop-packaging fold-in described in RFC #78 +(`docs/superpowers/specs/2026-07-02-unified-launcher-design.md`). + +## Why Tauri (and what it does / doesn't solve) + +- **Tiny footprint** — uses the OS WebView2 (already present on Win 11), not a + bundled Chromium. +- **Robust process ownership** — the shell puts the spawned Python into a + Windows **Job Object** with `KILL_ON_JOB_CLOSE`. Python's own device-layer + grandchild (via `DeviceLayerSupervisor`) inherits the job, so when the app + quits *or crashes* the OS reaps the **entire** process tree — no orphaned + device layer holding COM ports. +- **Not solved here:** bundling the Python environment (torch, anthropic, + perception) into a redistributable installer. This build launches the repo's + existing `.venv`. Shipping a self-contained `.msi`/`.exe` to another machine + needs an embeddable-Python / PyInstaller sidecar — see *Bundling* below. + +## Architecture + +``` +Tauri shell (Rust, WebView2 window) + └─ spawns: python launch_gently.py --no-browser [in a kill-on-close Job] + └─ spawns: start_device_layer.py (DeviceLayerSupervisor) + ├─ shows splash/index.html while uvicorn boots + └─ navigate() → http://localhost:8080 (the live gently UI) +``` + +- `src-tauri/src/main.rs` — spawn + Job Object teardown + wait-for-server + navigate. +- `splash/index.html` — boot splash (Rust updates its status line via `window.__gentlyStatus`). +- `src-tauri/tauri.conf.json` — one window, `frontendDist: ../splash`, NSIS bundle. + +## Quitting (graceful handshake, issue #85) + +Closing the window does **not** kill the backend outright. The shell first runs +a graceful handshake: it intercepts the close and `POST`s the backend's +loopback-only `/api/shutdown`, which stops a managed device layer via its clean +SIGTERM path and lets the backend drain state (e.g. session-replay final +batches) before exiting on its own; the shell waits a few seconds for the port +to go down, then quits. If an acquisition is running the backend answers `409` +and the shell asks for confirmation first — cancel and the window stays open. +If the backend is dead or hung, the shell just exits. Either way the Job-Object +kill-on-close (and best-effort `child.kill()`) remains the unchanged crash-safe +floor underneath. + +## Prerequisites + +- **Rust** (MSVC toolchain) — `rustup default stable-x86_64-pc-windows-msvc` +- **MSVC C++ build tools** (Visual Studio 2022 / Build Tools) +- **WebView2 runtime** (inbox on Windows 11) +- **Node** (only for the Tauri CLI: `npm install` in this folder) +- A working gently checkout with its Python **`.venv`** at the repo root + +### Linux (dev machines) + +The shell is cross-platform — the Job-Object code is `#[cfg(windows)]`, so on +Linux teardown is the best-effort `child.kill()` only (fine for dev; the +kill-on-close guarantee is a Windows-production feature). Verified on Manjaro +(KDE/Wayland, webkit2gtk 2.52): + +- **Rust** — `rustup default stable` +- **WebKitGTK + GTK3** — `webkit2gtk-4.1` and `gtk3` dev packages + (Arch/Manjaro: `pacman -S webkit2gtk-4.1`; Debian/Ubuntu: + `apt install libwebkit2gtk-4.1-dev build-essential`) +- **Node** — as above +- The venv is found at **`.venv/bin/python`** (Unix layout) automatically; + `GENTLY_PYTHON` overrides as usual + +Same commands, bash syntax: + +```bash +cd desktop && npm install +GENTLY_LAUNCH_ARGS="--no-api --offline --no-auth" npm run dev +``` + +`npm run build` is Windows-only as configured (`bundle.targets: ["nsis"]`) — +on Linux use `npm run dev`, or override targets if a Linux bundle is ever needed. + +## Run (dev) + +```powershell +cd desktop +npm install # once — restores the Tauri CLI +npm run dev # build + launch the shell (spawns the backend for you) +``` + +`npm run dev` (= `tauri dev`) compiles the shell, opens the window, spawns +`launch_gently.py --no-browser`, and navigates to the live UI once it's up. + +UI-only (no hardware / API key / login) — handy for pure UI work: + +```powershell +$env:GENTLY_LAUNCH_ARGS="--no-api --offline --no-auth"; npm run dev +``` + +## Making code changes — what reflects, and how + +The window is just a WebView pointed at the Python server, so it splits in three: + +| You edit… | Reflects by… | Rebuild? | +|---|---|---| +| **Web UI** (`gently/ui/web/templates`, `static/js`, `static/css`) | **Refresh the window** (Ctrl+R) | no — served live by Python | +| **Python backend** (`gently/**/*.py`, `launch_gently.py`) | restart the backend — or run with **`--reload`** (below), then Ctrl+R | process restart | +| **Rust shell / config** (`src-tauri/`, `splash/`) | `npm run dev` **auto-rebuilds + relaunches** | automatic (watched) | + +**Python hot-reload.** Pass `--reload` and the backend auto-restarts whenever a +`gently/*.py` file changes (watchfiles) — then just Ctrl+R the window: + +```powershell +$env:GENTLY_LAUNCH_ARGS="--reload --no-api --offline --no-auth"; npm run dev +# or, iterating in a plain browser instead of the shell: +uv run python launch_gently.py --reload +``` + +It restarts the *whole* backend (a few seconds), so use it for UI / backend dev, +not live hardware sessions. + +## Build (installer) + +```powershell +cd desktop +npm run build # = tauri build — NSIS installer under src-tauri/target/release/bundle/ +``` + +Then Gently installs like any Windows app (Start-Menu entry, double-click — no +terminal). The built app launches the **repo's** `.venv` + `launch_gently.py` +(paths baked at compile time / overridable — see env vars); it's a single-machine +build until Python bundling lands. + +## Configuration (env vars read by the shell) + +| Var | Default | Purpose | +|---|---|---| +| `GENTLY_HOME` | compile-time repo root | Directory to run the backend from | +| `GENTLY_PYTHON` | `/.venv/Scripts/python.exe` | Python interpreter to launch | +| `GENTLY_LAUNCH_ARGS` | *(none)* | Extra args appended to `launch_gently.py --no-browser` | +| `VIZ_PORT` | `8080` | Port the shell waits on and navigates to | +| `GENTLY_DEVICE_LAYER_SCRIPT` | `start_device_layer.py` | Alternate device-layer entry point (read by `DeviceLayerSupervisor`) | + +## Bundling Python (the remaining long pole) + +`tauri build` today does **not** package Python. To make a redistributable app: + +1. Produce a self-contained backend (PyInstaller one-folder of `launch_gently`, + or an embeddable-Python tree with the deps installed). +2. Ship it as a Tauri **sidecar** (or under the app's resource dir) and point + `GENTLY_PYTHON` / `GENTLY_HOME` at it. +3. Expect a multi-GB artifact (torch/CUDA dominate) and plan auto-update + accordingly. + +The Job-Object ownership and boot/navigate flow are unchanged by bundling — only +*where Python comes from* changes. diff --git a/desktop/icons/source.png b/desktop/icons/source.png new file mode 100644 index 00000000..8f2af46c Binary files /dev/null and b/desktop/icons/source.png differ diff --git a/desktop/package-lock.json b/desktop/package-lock.json new file mode 100644 index 00000000..63621f9e --- /dev/null +++ b/desktop/package-lock.json @@ -0,0 +1,232 @@ +{ + "name": "gently-desktop", + "version": "0.1.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "gently-desktop", + "version": "0.1.0", + "devDependencies": { + "@tauri-apps/cli": "^2.11.4" + } + }, + "node_modules/@tauri-apps/cli": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli/-/cli-2.11.4.tgz", + "integrity": "sha512-R8xGtMpwyetawSqm9kYOuMmEqkhUbvcUy8n0aNXIxollKBLESUu5f4Fx+64hgASYm1H+jSWq6jCW6zqTnH6hqQ==", + "dev": true, + "license": "Apache-2.0 OR MIT", + "bin": { + "tauri": "tauri.js" + }, + "engines": { + "node": ">= 10" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/tauri" + }, + "optionalDependencies": { + "@tauri-apps/cli-darwin-arm64": "2.11.4", + "@tauri-apps/cli-darwin-x64": "2.11.4", + "@tauri-apps/cli-linux-arm-gnueabihf": "2.11.4", + "@tauri-apps/cli-linux-arm64-gnu": "2.11.4", + "@tauri-apps/cli-linux-arm64-musl": "2.11.4", + "@tauri-apps/cli-linux-riscv64-gnu": "2.11.4", + "@tauri-apps/cli-linux-x64-gnu": "2.11.4", + "@tauri-apps/cli-linux-x64-musl": "2.11.4", + "@tauri-apps/cli-win32-arm64-msvc": "2.11.4", + "@tauri-apps/cli-win32-ia32-msvc": "2.11.4", + "@tauri-apps/cli-win32-x64-msvc": "2.11.4" + } + }, + "node_modules/@tauri-apps/cli-darwin-arm64": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-arm64/-/cli-darwin-arm64-2.11.4.tgz", + "integrity": "sha512-1ryOF3ZhpZ/nemHV5zVwBQBz9jDGKmKPvWPADOhc83ig0P4bMc2iER4NbC6r9sjeIZ6RVQ4g3RZIYvezhcl4TQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-darwin-x64": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-darwin-x64/-/cli-darwin-x64-2.11.4.tgz", + "integrity": "sha512-uFsGQAAfuyz1k/yGLmkWfkBlgKAqZfxqlHmLWx81QU27RJWfmbNHCIq8T8w1e+VClleIuZUjpHWfoE4E3DLo3A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-linux-arm-gnueabihf": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm-gnueabihf/-/cli-linux-arm-gnueabihf-2.11.4.tgz", + "integrity": "sha512-IaHZn5CdBL21oUmjiVOS1ctw6Ip1O0pjp70FwOWmYz1myWe0SY96ZIj2FYf7pT0m8bI2h/hrs5ZbEXXh44/MkQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-linux-arm64-gnu": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-gnu/-/cli-linux-arm64-gnu-2.11.4.tgz", + "integrity": "sha512-N41/ukTRVe6XSuUTESuFdGeOW2i7k62tK+6gHK5Kd5/q5RPvvi19GaWAVPPb9u95HSGmTChSolBfzynUsssFaA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-linux-arm64-musl": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.11.4.tgz", + "integrity": "sha512-v277UnT/fB64xAfSroL5N3Km3tLmvATWqJJw/wRI+g6o+HkeD0slyE7gOhNs1MbjE41R7bQOTxMVoL3aomUJmw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-linux-riscv64-gnu": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-riscv64-gnu/-/cli-linux-riscv64-gnu-2.11.4.tgz", + "integrity": "sha512-qqgNkQ2u1yZHxjhxsZaxUtRDW8dIqIYm33rx/mzwQv0SfY9x1B+iraj8vWeFiXjjSVVhEMepXSOts1TqPzvXNQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-linux-x64-gnu": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-gnu/-/cli-linux-x64-gnu-2.11.4.tgz", + "integrity": "sha512-2VRNWl84FOH0m2giiDkO2h0QXlcMJeX+zJDpI5kDIQAx6s+geF3v48F4DXfJez4GS/FdoDGnPnw1C2iYGbQ7bQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-linux-x64-musl": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-linux-x64-musl/-/cli-linux-x64-musl-2.11.4.tgz", + "integrity": "sha512-o9GyhYor/nc7xarmwDE3ka2szuW3uuZzXjHWh64Q8YX5AtSgxdQkFWzrY4O8KiGtVNvFBI14H3Q49Qj5TOIP/A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-win32-arm64-msvc": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-arm64-msvc/-/cli-win32-arm64-msvc-2.11.4.tgz", + "integrity": "sha512-ld5Ehb598m0VkYyylRPNeCFsBe/km0jxis6KgMpl3IGY6I/i1RwQXO05I1AsXUXO2WC6AvB/Lw4qTf/asiuEiQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-win32-ia32-msvc": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-ia32-msvc/-/cli-win32-ia32-msvc-2.11.4.tgz", + "integrity": "sha512-12Hxi0XX/H5VFxO/bGgHkFWhml9VMgEOu9CidjeCeTNQ1l6fpUlbiGgSP7CLI3PFtW9/FfbeHieZ+kyWK5H7CA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + }, + "node_modules/@tauri-apps/cli-win32-x64-msvc": { + "version": "2.11.4", + "resolved": "https://registry.npmjs.org/@tauri-apps/cli-win32-x64-msvc/-/cli-win32-x64-msvc-2.11.4.tgz", + "integrity": "sha512-+vDiqBIU5dMISg/wNvX3sF+ZHfgJGJ5T0AcO+EHNXV9GGAG+P5fzodlDXD3QdKCRgZxMoCm5PPvj3BqLNjBthw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "Apache-2.0 OR MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 10" + } + } + } +} diff --git a/desktop/package.json b/desktop/package.json new file mode 100644 index 00000000..0173efb1 --- /dev/null +++ b/desktop/package.json @@ -0,0 +1,14 @@ +{ + "name": "gently-desktop", + "version": "0.1.0", + "private": true, + "description": "Tauri desktop shell for gently (owns the Python backend + device layer)", + "scripts": { + "tauri": "tauri", + "dev": "tauri dev", + "build": "tauri build" + }, + "devDependencies": { + "@tauri-apps/cli": "^2.11.4" + } +} diff --git a/desktop/splash/index.html b/desktop/splash/index.html new file mode 100644 index 00000000..f7bb3357 --- /dev/null +++ b/desktop/splash/index.html @@ -0,0 +1,51 @@ + + + + + +Gently + + + + +
+
Gently
+
+
Starting the backend…
+
microscopy agent · desktop
+ + + diff --git a/desktop/src-tauri/Cargo.lock b/desktop/src-tauri/Cargo.lock new file mode 100644 index 00000000..27a8fbb4 --- /dev/null +++ b/desktop/src-tauri/Cargo.lock @@ -0,0 +1,4433 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "adler2" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa" + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "alloc-no-stdlib" +version = "2.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc7bb162ec39d46ab1ca8c77bf72e890535becd1751bb45f64c597edb4c8c6b3" + +[[package]] +name = "alloc-stdlib" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0e76a019e91224d279006ff972f1e984179a6e9feb050adba6ce8274aef23195" +dependencies = [ + "alloc-no-stdlib", +] + +[[package]] +name = "android_system_properties" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311" +dependencies = [ + "libc", +] + +[[package]] +name = "anyhow" +version = "1.0.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a4385e2e34eb35d6b3efe798b9eb88096925d87726c0798709bf56d9ed84af3" + +[[package]] +name = "atk" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "241b621213072e993be4f6f3a9e4b45f65b7e6faad43001be957184b7bb1824b" +dependencies = [ + "atk-sys", + "glib", + "libc", +] + +[[package]] +name = "atk-sys" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c5e48b684b0ca77d2bbadeef17424c2ea3c897d44d566a1617e7e8f30614d086" +dependencies = [ + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + +[[package]] +name = "atomic-waker" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "base64" +version = "0.21.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567" + +[[package]] +name = "base64" +version = "0.22.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" + +[[package]] +name = "bit-set" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3" +dependencies = [ + "bit-vec", +] + +[[package]] +name = "bit-vec" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + +[[package]] +name = "bitflags" +version = "2.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8" +dependencies = [ + "serde_core", +] + +[[package]] +name = "block-buffer" +version = "0.10.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71" +dependencies = [ + "generic-array", +] + +[[package]] +name = "block2" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdeb9d870516001442e364c5220d3574d2da8dc765554b4a617230d33fa58ef5" +dependencies = [ + "objc2", +] + +[[package]] +name = "brotli" +version = "8.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5cc91aac060a7a1e25823bdccbfb6af1875b88f17c6daac97894eed8207166b3" +dependencies = [ + "alloc-no-stdlib", + "alloc-stdlib", + "brotli-decompressor", +] + +[[package]] +name = "brotli-decompressor" +version = "5.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a32acac15fe1967bc3986b2a6347dffc965602354ea6f450ad07e8bfd253583" +dependencies = [ + "alloc-no-stdlib", + "alloc-stdlib", +] + +[[package]] +name = "bs58" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf88ba1141d185c399bee5288d850d63b8369520c1eafc32a0430b5b6c287bf4" +dependencies = [ + "tinyvec", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "bytemuck" +version = "1.25.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6aedf8ae72766347502cf3cb4f41cf5e9cc37d28bee90f1fdaaae15f9cf9424" + +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + +[[package]] +name = "bytes" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" +dependencies = [ + "serde", +] + +[[package]] +name = "cairo-rs" +version = "0.18.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ca26ef0159422fb77631dc9d17b102f253b876fe1586b03b803e63a309b4ee2" +dependencies = [ + "bitflags 2.13.0", + "cairo-sys-rs", + "glib", + "libc", + "once_cell", + "thiserror 1.0.69", +] + +[[package]] +name = "cairo-sys-rs" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "685c9fa8e590b8b3d678873528d83411db17242a73fccaed827770ea0fedda51" +dependencies = [ + "glib-sys", + "libc", + "system-deps", +] + +[[package]] +name = "camino" +version = "1.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f2d30e4173c4026932d51d31d6b0613b1fd3014bf3f9f8943d4ba139c437ba0" +dependencies = [ + "serde_core", +] + +[[package]] +name = "cargo-platform" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e35af189006b9c0f00a064685c727031e3ed2d8020f7ba284d78cc2671bd36ea" +dependencies = [ + "serde", +] + +[[package]] +name = "cargo_metadata" +version = "0.19.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dd5eb614ed4c27c5d706420e4320fbe3216ab31fa1c33cd8246ac36dae4479ba" +dependencies = [ + "camino", + "cargo-platform", + "semver", + "serde", + "serde_json", + "thiserror 2.0.18", +] + +[[package]] +name = "cargo_toml" +version = "0.22.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "374b7c592d9c00c1f4972ea58390ac6b18cbb6ab79011f3bdc90a0b82ca06b77" +dependencies = [ + "serde", + "toml 0.9.12+spec-1.1.0", +] + +[[package]] +name = "cc" +version = "1.2.67" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e17dd265a7d0f31ef544e1b20e03add05d3b45b491b633b10d67145d2acc1a38" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "cesu8" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6d43a04d8753f35258c91f8ec639f792891f748a1edbd759cf1dcea3382ad83c" + +[[package]] +name = "cfb" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d38f2da7a0a2c4ccf0065be06397cc26a81f4e528be095826eee9d4adbb8c60f" +dependencies = [ + "byteorder", + "fnv", + "uuid", +] + +[[package]] +name = "cfg-expr" +version = "0.15.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d067ad48b8650848b989a59a86c6c36a995d02d2bf778d45c3c5d57bc2718f02" +dependencies = [ + "smallvec", + "target-lexicon", +] + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "chrono" +version = "0.4.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327" +dependencies = [ + "iana-time-zone", + "num-traits", + "serde", + "windows-link 0.2.1", +] + +[[package]] +name = "combine" +version = "4.6.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ba5a308b75df32fe02788e748662718f03fde005016435c444eea572398219fd" +dependencies = [ + "bytes", + "memchr", +] + +[[package]] +name = "cookie" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ddef33a339a91ea89fb53151bd0a4689cfce27055c291dfa69945475d22c747" +dependencies = [ + "time", + "version_check", +] + +[[package]] +name = "core-foundation" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2a6cd9ae233e7f62ba4e9353e81a88df7fc8a5987b8d445b4d90c879bd156f6" +dependencies = [ + "core-foundation-sys", + "libc", +] + +[[package]] +name = "core-foundation-sys" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" + +[[package]] +name = "core-graphics" +version = "0.25.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "064badf302c3194842cf2c5d61f56cc88e54a759313879cdf03abdd27d0c3b97" +dependencies = [ + "bitflags 2.13.0", + "core-foundation", + "core-graphics-types", + "foreign-types", + "libc", +] + +[[package]] +name = "core-graphics-types" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3d44a101f213f6c4cdc1853d4b78aef6db6bdfa3468798cc1d9912f4735013eb" +dependencies = [ + "bitflags 2.13.0", + "core-foundation", + "libc", +] + +[[package]] +name = "cpufeatures" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" +dependencies = [ + "libc", +] + +[[package]] +name = "crc32fast" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9481c1c90cbf2ac953f07c8d4a58aa3945c425b7185c9154d67a65e4230da511" +dependencies = [ + "cfg-if", +] + +[[package]] +name = "crossbeam-channel" +version = "0.5.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" + +[[package]] +name = "crypto-common" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" +dependencies = [ + "generic-array", + "typenum", +] + +[[package]] +name = "cssparser" +version = "0.36.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dae61cf9c0abb83bd659dab65b7e4e38d8236824c85f0f804f173567bda257d2" +dependencies = [ + "cssparser-macros", + "dtoa-short", + "itoa", + "phf", + "smallvec", +] + +[[package]] +name = "cssparser-macros" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13b588ba4ac1a99f7f2964d24b3d896ddc6bf847ee3855dbd4366f058cfcd331" +dependencies = [ + "quote", + "syn 2.0.118", +] + +[[package]] +name = "ctor" +version = "0.8.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "352d39c2f7bef1d6ad73db6f5160efcaed66d94ef8c6c573a8410c00bf909a98" +dependencies = [ + "ctor-proc-macro", + "dtor", +] + +[[package]] +name = "ctor-proc-macro" +version = "0.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52560adf09603e58c9a7ee1fe1dcb95a16927b17c127f0ac02d6e768a0e25bc1" + +[[package]] +name = "darling" +version = "0.23.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "25ae13da2f202d56bd7f91c25fba009e7717a1e4a1cc98a76d844b65ae912e9d" +dependencies = [ + "darling_core", + "darling_macro", +] + +[[package]] +name = "darling_core" +version = "0.23.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9865a50f7c335f53564bb694ef660825eb8610e0a53d3e11bf1b0d3df31e03b0" +dependencies = [ + "ident_case", + "proc-macro2", + "quote", + "strsim", + "syn 2.0.118", +] + +[[package]] +name = "darling_macro" +version = "0.23.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3984ec7bd6cfa798e62b4a642426a5be0e68f9401cfc2a01e3fa9ea2fcdb8d" +dependencies = [ + "darling_core", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "dbus" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ab69f03cc8c4340c9c8e315114e1658e6775a9b16a04357973aa21cec22b32e" +dependencies = [ + "libc", + "libdbus-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "deranged" +version = "0.5.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c" +dependencies = [ + "serde_core", +] + +[[package]] +name = "derive_more" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d751e9e49156b02b44f9c1815bcb94b984cdcc4396ecc32521c739452808b134" +dependencies = [ + "derive_more-impl", +] + +[[package]] +name = "derive_more-impl" +version = "2.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "799a97264921d8623a957f6c3b9011f3b5492f557bbb7a5a19b7fa6d06ba8dcb" +dependencies = [ + "proc-macro2", + "quote", + "rustc_version", + "syn 2.0.118", +] + +[[package]] +name = "digest" +version = "0.10.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" +dependencies = [ + "block-buffer", + "crypto-common", +] + +[[package]] +name = "dirs" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3e8aa94d75141228480295a7d0e7feb620b1a5ad9f12bc40be62411e38cce4e" +dependencies = [ + "dirs-sys", +] + +[[package]] +name = "dirs-sys" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e01a3366d27ee9890022452ee61b2b63a67e6f13f58900b651ff5665f0bb1fab" +dependencies = [ + "libc", + "option-ext", + "redox_users", + "windows-sys 0.61.2", +] + +[[package]] +name = "dispatch2" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38" +dependencies = [ + "bitflags 2.13.0", + "block2", + "libc", + "objc2", +] + +[[package]] +name = "displaydoc" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ac70aa55017e108007fbaf5aa0f54b021c98f92ff8af59d42eda9da96e3dd4f" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "dlopen2" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e2c5bd4158e66d1e215c49b837e11d62f3267b30c92f1d171c4d3105e3dc4d4" +dependencies = [ + "dlopen2_derive", + "libc", + "once_cell", + "winapi", +] + +[[package]] +name = "dlopen2_derive" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fbbb781877580993a8707ec48672673ec7b81eeba04cfd2310bd28c08e47c8f" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "dom_query" +version = "0.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "521e380c0c8afb8d9a1e83a1822ee03556fc3e3e7dbc1fd30be14e37f9cb3f89" +dependencies = [ + "bit-set", + "cssparser", + "foldhash", + "html5ever", + "precomputed-hash", + "selectors", + "tendril", +] + +[[package]] +name = "dpi" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76" +dependencies = [ + "serde", +] + +[[package]] +name = "dtoa" +version = "1.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590" + +[[package]] +name = "dtoa-short" +version = "0.3.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87" +dependencies = [ + "dtoa", +] + +[[package]] +name = "dtor" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1057d6c64987086ff8ed0fd3fbf377a6b7d205cc7715868cd401705f715cbe4" +dependencies = [ + "dtor-proc-macro", +] + +[[package]] +name = "dtor-proc-macro" +version = "0.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f678cf4a922c215c63e0de95eb1ff08a958a81d47e485cf9da1e27bf6305cfa5" + +[[package]] +name = "dunce" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813" + +[[package]] +name = "dyn-clone" +version = "1.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555" + +[[package]] +name = "embed-resource" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fbfdaacccebec3b28e4866b8973543c7647797db5ada1bdab552e48fe665fbbd" +dependencies = [ + "cc", + "memchr", + "rustc_version", + "toml 1.1.2+spec-1.1.0", + "vswhom", + "winreg", +] + +[[package]] +name = "embed_plist" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ef6b89e5b37196644d8796de5268852ff179b44e96276cf4290264843743bb7" + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "erased-serde" +version = "0.4.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2add8a07dd6a8d93ff627029c51de145e12686fbc36ecb298ac22e74cf02dec" +dependencies = [ + "serde", + "serde_core", + "typeid", +] + +[[package]] +name = "fastrand" +version = "2.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6" + +[[package]] +name = "fdeflate" +version = "0.3.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e6853b52649d4ac5c0bd02320cddc5ba956bdb407c4b75a2c6b75bf51500f8c" +dependencies = [ + "simd-adler32", +] + +[[package]] +name = "field-offset" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38e2275cc4e4fc009b0669731a1e5ab7ebf11f469eaede2bab9309a5b4d6057f" +dependencies = [ + "memoffset", + "rustc_version", +] + +[[package]] +name = "find-msvc-tools" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582" + +[[package]] +name = "flate2" +version = "1.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c" +dependencies = [ + "crc32fast", + "miniz_oxide", +] + +[[package]] +name = "fnv" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1" + +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + +[[package]] +name = "foreign-types" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d737d9aa519fb7b749cbc3b962edcf310a8dd1f4b67c91c4f83975dbdd17d965" +dependencies = [ + "foreign-types-macros", + "foreign-types-shared", +] + +[[package]] +name = "foreign-types-macros" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1a5c6c585bc94aaf2c7b51dd4c2ba22680844aba4c687be581871a6f518c5742" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "foreign-types-shared" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aa9a19cbb55df58761df49b23516a86d432839add4af60fc256da840f66ed35b" + +[[package]] +name = "form_urlencoded" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf" +dependencies = [ + "percent-encoding", +] + +[[package]] +name = "futures-channel" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d" +dependencies = [ + "futures-core", +] + +[[package]] +name = "futures-core" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d" + +[[package]] +name = "futures-executor" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d" +dependencies = [ + "futures-core", + "futures-task", + "futures-util", +] + +[[package]] +name = "futures-io" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718" + +[[package]] +name = "futures-macro" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "futures-sink" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893" + +[[package]] +name = "futures-task" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393" + +[[package]] +name = "futures-util" +version = "0.3.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6" +dependencies = [ + "futures-core", + "futures-io", + "futures-macro", + "futures-sink", + "futures-task", + "memchr", + "pin-project-lite", + "slab", +] + +[[package]] +name = "gdk" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9f245958c627ac99d8e529166f9823fb3b838d1d41fd2b297af3075093c2691" +dependencies = [ + "cairo-rs", + "gdk-pixbuf", + "gdk-sys", + "gio", + "glib", + "libc", + "pango", +] + +[[package]] +name = "gdk-pixbuf" +version = "0.18.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "50e1f5f1b0bfb830d6ccc8066d18db35c487b1b2b1e8589b5dfe9f07e8defaec" +dependencies = [ + "gdk-pixbuf-sys", + "gio", + "glib", + "libc", + "once_cell", +] + +[[package]] +name = "gdk-pixbuf-sys" +version = "0.18.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f9839ea644ed9c97a34d129ad56d38a25e6756f99f3a88e15cd39c20629caf7" +dependencies = [ + "gio-sys", + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + +[[package]] +name = "gdk-sys" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c2d13f38594ac1e66619e188c6d5a1adb98d11b2fcf7894fc416ad76aa2f3f7" +dependencies = [ + "cairo-sys-rs", + "gdk-pixbuf-sys", + "gio-sys", + "glib-sys", + "gobject-sys", + "libc", + "pango-sys", + "pkg-config", + "system-deps", +] + +[[package]] +name = "gdkwayland-sys" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "140071d506d223f7572b9f09b5e155afbd77428cd5cc7af8f2694c41d98dfe69" +dependencies = [ + "gdk-sys", + "glib-sys", + "gobject-sys", + "libc", + "pkg-config", + "system-deps", +] + +[[package]] +name = "gdkx11" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3caa00e14351bebbc8183b3c36690327eb77c49abc2268dd4bd36b856db3fbfe" +dependencies = [ + "gdk", + "gdkx11-sys", + "gio", + "glib", + "libc", + "x11", +] + +[[package]] +name = "gdkx11-sys" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e2e7445fe01ac26f11601db260dd8608fe172514eb63b3b5e261ea6b0f4428d" +dependencies = [ + "gdk-sys", + "glib-sys", + "libc", + "system-deps", + "x11", +] + +[[package]] +name = "generic-array" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a" +dependencies = [ + "typenum", + "version_check", +] + +[[package]] +name = "gently-desktop" +version = "0.1.0" +dependencies = [ + "tauri", + "tauri-build", + "windows 0.62.2", +] + +[[package]] +name = "getrandom" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff2abc00be7fca6ebc474524697ae276ad847ad0a6b3faa4bcb027e9a4614ad0" +dependencies = [ + "cfg-if", + "libc", + "wasi", +] + +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "libc", + "r-efi 5.3.0", + "wasip2", +] + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "libc", + "r-efi 6.0.0", +] + +[[package]] +name = "gio" +version = "0.18.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d4fc8f532f87b79cbc51a79748f16a6828fb784be93145a322fa14d06d354c73" +dependencies = [ + "futures-channel", + "futures-core", + "futures-io", + "futures-util", + "gio-sys", + "glib", + "libc", + "once_cell", + "pin-project-lite", + "smallvec", + "thiserror 1.0.69", +] + +[[package]] +name = "gio-sys" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "37566df850baf5e4cb0dfb78af2e4b9898d817ed9263d1090a2df958c64737d2" +dependencies = [ + "glib-sys", + "gobject-sys", + "libc", + "system-deps", + "winapi", +] + +[[package]] +name = "glib" +version = "0.18.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "233daaf6e83ae6a12a52055f568f9d7cf4671dabb78ff9560ab6da230ce00ee5" +dependencies = [ + "bitflags 2.13.0", + "futures-channel", + "futures-core", + "futures-executor", + "futures-task", + "futures-util", + "gio-sys", + "glib-macros", + "glib-sys", + "gobject-sys", + "libc", + "memchr", + "once_cell", + "smallvec", + "thiserror 1.0.69", +] + +[[package]] +name = "glib-macros" +version = "0.18.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bb0228f477c0900c880fd78c8759b95c7636dbd7842707f49e132378aa2acdc" +dependencies = [ + "heck 0.4.1", + "proc-macro-crate 2.0.2", + "proc-macro-error", + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "glib-sys" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "063ce2eb6a8d0ea93d2bf8ba1957e78dbab6be1c2220dd3daca57d5a9d869898" +dependencies = [ + "libc", + "system-deps", +] + +[[package]] +name = "glob" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280" + +[[package]] +name = "gobject-sys" +version = "0.18.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0850127b514d1c4a4654ead6dedadb18198999985908e6ffe4436f53c785ce44" +dependencies = [ + "glib-sys", + "libc", + "system-deps", +] + +[[package]] +name = "gtk" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fd56fb197bfc42bd5d2751f4f017d44ff59fbb58140c6b49f9b3b2bdab08506a" +dependencies = [ + "atk", + "cairo-rs", + "field-offset", + "futures-channel", + "gdk", + "gdk-pixbuf", + "gio", + "glib", + "gtk-sys", + "gtk3-macros", + "libc", + "pango", + "pkg-config", +] + +[[package]] +name = "gtk-sys" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f29a1c21c59553eb7dd40e918be54dccd60c52b049b75119d5d96ce6b624414" +dependencies = [ + "atk-sys", + "cairo-sys-rs", + "gdk-pixbuf-sys", + "gdk-sys", + "gio-sys", + "glib-sys", + "gobject-sys", + "libc", + "pango-sys", + "system-deps", +] + +[[package]] +name = "gtk3-macros" +version = "0.18.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ff3c5b21f14f0736fed6dcfc0bfb4225ebf5725f3c0209edeec181e4d73e9d" +dependencies = [ + "proc-macro-crate 1.3.1", + "proc-macro-error", + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "hashbrown" +version = "0.12.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888" + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "heck" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "95505c38b4572b2d910cecb0281560f54b440a19336cbbcb27bf6ce6adc6f5a8" + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "html5ever" +version = "0.38.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1054432bae2f14e0061e33d23402fbaa67a921d319d56adc6bcf887ddad1cbc2" +dependencies = [ + "log", + "markup5ever", +] + +[[package]] +name = "http" +version = "1.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6970f50e31d6fc17d3fa27329444bfa74e196cf62e95052a3f6fee181dba6425" +dependencies = [ + "bytes", + "itoa", +] + +[[package]] +name = "http-body" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184" +dependencies = [ + "bytes", + "http", +] + +[[package]] +name = "http-body-util" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a" +dependencies = [ + "bytes", + "futures-core", + "http", + "http-body", + "pin-project-lite", +] + +[[package]] +name = "httparse" +version = "1.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87" + +[[package]] +name = "hyper" +version = "1.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55281c53a1894c864990125767da440a4e630446785086f52523b20033b74498" +dependencies = [ + "atomic-waker", + "bytes", + "futures-channel", + "futures-core", + "http", + "http-body", + "httparse", + "itoa", + "pin-project-lite", + "smallvec", + "tokio", + "want", +] + +[[package]] +name = "hyper-util" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0" +dependencies = [ + "base64 0.22.1", + "bytes", + "futures-channel", + "futures-util", + "http", + "http-body", + "hyper", + "ipnet", + "libc", + "percent-encoding", + "pin-project-lite", + "socket2", + "tokio", + "tower-service", + "tracing", +] + +[[package]] +name = "iana-time-zone" +version = "0.1.65" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470" +dependencies = [ + "android_system_properties", + "core-foundation-sys", + "iana-time-zone-haiku", + "js-sys", + "log", + "wasm-bindgen", + "windows-core 0.62.2", +] + +[[package]] +name = "iana-time-zone-haiku" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f" +dependencies = [ + "cc", +] + +[[package]] +name = "ico" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e795dff5605e0f04bff85ca41b51a96b83e80b281e96231bcaaf1ac35103371" +dependencies = [ + "byteorder", + "png 0.17.16", +] + +[[package]] +name = "icu_collections" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2984d1cd16c883d7935b9e07e44071dca8d917fd52ecc02c04d5fa0b5a3f191c" +dependencies = [ + "displaydoc", + "potential_utf", + "utf8_iter", + "yoke", + "zerofrom", + "zerovec", +] + +[[package]] +name = "icu_locale_core" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92219b62b3e2b4d88ac5119f8904c10f8f61bf7e95b640d25ba3075e6cac2c29" +dependencies = [ + "displaydoc", + "litemap", + "tinystr", + "writeable", + "zerovec", +] + +[[package]] +name = "icu_normalizer" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c56e5ee99d6e3d33bd91c5d85458b6005a22140021cc324cea84dd0e72cff3b4" +dependencies = [ + "icu_collections", + "icu_normalizer_data", + "icu_properties", + "icu_provider", + "smallvec", + "zerovec", +] + +[[package]] +name = "icu_normalizer_data" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da3be0ae77ea334f4da67c12f149704f19f81d1adf7c51cf482943e84a2bad38" + +[[package]] +name = "icu_properties" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bee3b67d0ea5c2cca5003417989af8996f8604e34fb9ddf96208a033901e70de" +dependencies = [ + "icu_collections", + "icu_locale_core", + "icu_properties_data", + "icu_provider", + "zerotrie", + "zerovec", +] + +[[package]] +name = "icu_properties_data" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e2bbb201e0c04f7b4b3e14382af113e17ba4f63e2c9d2ee626b720cbce54a14" + +[[package]] +name = "icu_provider" +version = "2.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "139c4cf31c8b5f33d7e199446eff9c1e02decfc2f0eec2c8d71f65befa45b421" +dependencies = [ + "displaydoc", + "icu_locale_core", + "writeable", + "yoke", + "zerofrom", + "zerotrie", + "zerovec", +] + +[[package]] +name = "ident_case" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9e0384b61958566e926dc50660321d12159025e767c18e043daf26b70104c39" + +[[package]] +name = "idna" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b0875f23caa03898994f6ddc501886a45c7d3d62d04d2d90788d47be1b1e4de" +dependencies = [ + "idna_adapter", + "smallvec", + "utf8_iter", +] + +[[package]] +name = "idna_adapter" +version = "1.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb68373c0d6620ef8105e855e7745e18b0d00d3bdb07fb532e434244cdb9a714" +dependencies = [ + "icu_normalizer", + "icu_properties", +] + +[[package]] +name = "indexmap" +version = "1.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bd070e393353796e801d209ad339e89596eb4c8d430d18ede6a1cced8fafbd99" +dependencies = [ + "autocfg", + "hashbrown 0.12.3", + "serde", +] + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown 0.17.1", + "serde", + "serde_core", +] + +[[package]] +name = "infer" +version = "0.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a588916bfdfd92e71cacef98a63d9b1f0d74d6599980d11894290e7ddefffcf7" +dependencies = [ + "cfb", +] + +[[package]] +name = "ipnet" +version = "2.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2" + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "javascriptcore-rs" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ca5671e9ffce8ffba57afc24070e906da7fc4b1ba66f2cabebf61bf2ea257fcc" +dependencies = [ + "bitflags 1.3.2", + "glib", + "javascriptcore-rs-sys", +] + +[[package]] +name = "javascriptcore-rs-sys" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af1be78d14ffa4b75b66df31840478fef72b51f8c2465d4ca7c194da9f7a5124" +dependencies = [ + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + +[[package]] +name = "jni" +version = "0.21.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1a87aa2bb7d2af34197c04845522473242e1aa17c12f4935d5856491a7fb8c97" +dependencies = [ + "cesu8", + "cfg-if", + "combine", + "jni-sys 0.3.1", + "log", + "thiserror 1.0.69", + "walkdir", + "windows-sys 0.45.0", +] + +[[package]] +name = "jni-sys" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41a652e1f9b6e0275df1f15b32661cf0d4b78d4d87ddec5e0c3c20f097433258" +dependencies = [ + "jni-sys 0.4.1", +] + +[[package]] +name = "jni-sys" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6377a88cb3910bee9b0fa88d4f42e1d2da8e79915598f65fb0c7ee14c878af2" +dependencies = [ + "jni-sys-macros", +] + +[[package]] +name = "jni-sys-macros" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38c0b942f458fe50cdac086d2f946512305e5631e720728f2a61aabcd47a6264" +dependencies = [ + "quote", + "syn 2.0.118", +] + +[[package]] +name = "js-sys" +version = "0.3.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53b44bfcdb3f8d5837a46dae1ca9660a837176eee74a28b229bc626816589102" +dependencies = [ + "cfg-if", + "futures-util", + "wasm-bindgen", +] + +[[package]] +name = "json-patch" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "863726d7afb6bc2590eeff7135d923545e5e964f004c2ccf8716c25e70a86f08" +dependencies = [ + "jsonptr", + "serde", + "serde_json", + "thiserror 1.0.69", +] + +[[package]] +name = "jsonptr" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5dea2b27dd239b2556ed7a25ba842fe47fd602e7fc7433c2a8d6106d4d9edd70" +dependencies = [ + "serde", + "serde_json", +] + +[[package]] +name = "keyboard-types" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b750dcadc39a09dbadd74e118f6dd6598df77fa01df0cfcdc52c28dece74528a" +dependencies = [ + "bitflags 2.13.0", + "serde", + "unicode-segmentation", +] + +[[package]] +name = "libappindicator" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03589b9607c868cc7ae54c0b2a22c8dc03dd41692d48f2d7df73615c6a95dc0a" +dependencies = [ + "glib", + "gtk", + "gtk-sys", + "libappindicator-sys", + "log", +] + +[[package]] +name = "libappindicator-sys" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e9ec52138abedcc58dc17a7c6c0c00a2bdb4f3427c7f63fa97fd0d859155caf" +dependencies = [ + "gtk-sys", + "libloading", + "once_cell", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "libdbus-sys" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "328c4789d42200f1eeec05bd86c9c13c7f091d2ba9a6ea35acdf51f31bc0f043" +dependencies = [ + "pkg-config", +] + +[[package]] +name = "libloading" +version = "0.7.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b67380fd3b2fbe7527a606e18729d21c6f3951633d0500574c4dc22d2d638b9f" +dependencies = [ + "cfg-if", + "winapi", +] + +[[package]] +name = "libredox" +version = "0.1.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c943259e342f1e06ff2da7a83eabdfe7f92ce10262688dbf1895ff0b3e6e4652" +dependencies = [ + "libc", +] + +[[package]] +name = "litemap" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0" + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "log" +version = "0.4.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad" + +[[package]] +name = "markup5ever" +version = "0.38.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8983d30f2915feeaaab2d6babdd6bc7e9ed1a00b66b5e6d74df19aa9c0e91862" +dependencies = [ + "log", + "tendril", + "web_atoms", +] + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + +[[package]] +name = "mime" +version = "0.3.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a" + +[[package]] +name = "miniz_oxide" +version = "0.8.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316" +dependencies = [ + "adler2", + "simd-adler32", +] + +[[package]] +name = "mio" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02bd0af71c67b473010cbbc60715ee815645a4dc942899111f494b4b737d6fda" +dependencies = [ + "libc", + "wasi", + "windows-sys 0.61.2", +] + +[[package]] +name = "muda" +version = "0.19.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1dd04e60bc0b07438a6771710ee1698f98f6ebbc7f89b61264af1563b8aeb878" +dependencies = [ + "crossbeam-channel", + "dpi", + "gtk", + "keyboard-types", + "objc2", + "objc2-app-kit", + "objc2-core-foundation", + "objc2-foundation", + "once_cell", + "png 0.18.1", + "serde", + "thiserror 2.0.18", + "windows-sys 0.61.2", +] + +[[package]] +name = "ndk" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3f42e7bbe13d351b6bead8286a43aac9534b82bd3cc43e47037f012ebfd62d4" +dependencies = [ + "bitflags 2.13.0", + "jni-sys 0.3.1", + "log", + "ndk-sys", + "num_enum", + "raw-window-handle", + "thiserror 1.0.69", +] + +[[package]] +name = "ndk-sys" +version = "0.6.0+11769913" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee6cda3051665f1fb8d9e08fc35c96d5a244fb1be711a03b71118828afc9a873" +dependencies = [ + "jni-sys 0.3.1", +] + +[[package]] +name = "new_debug_unreachable" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086" + +[[package]] +name = "num-conv" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441" + +[[package]] +name = "num-traits" +version = "0.2.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841" +dependencies = [ + "autocfg", +] + +[[package]] +name = "num_enum" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d0bca838442ec211fa11de3a8b0e0e8f3a4522575b5c4c06ed722e005036f26" +dependencies = [ + "num_enum_derive", + "rustversion", +] + +[[package]] +name = "num_enum_derive" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "680998035259dcfcafe653688bf2aa6d3e2dc05e98be6ab46afb089dc84f1df8" +dependencies = [ + "proc-macro-crate 3.5.0", + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "objc2" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a12a8ed07aefc768292f076dc3ac8c48f3781c8f2d5851dd3d98950e8c5a89f" +dependencies = [ + "objc2-encode", + "objc2-exception-helper", +] + +[[package]] +name = "objc2-app-kit" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d49e936b501e5c5bf01fda3a9452ff86dc3ea98ad5f283e1455153142d97518c" +dependencies = [ + "bitflags 2.13.0", + "block2", + "objc2", + "objc2-core-foundation", + "objc2-foundation", +] + +[[package]] +name = "objc2-cloud-kit" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73ad74d880bb43877038da939b7427bba67e9dd42004a18b809ba7d87cee241c" +dependencies = [ + "bitflags 2.13.0", + "objc2", + "objc2-foundation", +] + +[[package]] +name = "objc2-core-data" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b402a653efbb5e82ce4df10683b6b28027616a2715e90009947d50b8dd298fa" +dependencies = [ + "objc2", + "objc2-foundation", +] + +[[package]] +name = "objc2-core-foundation" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536" +dependencies = [ + "bitflags 2.13.0", + "dispatch2", + "objc2", +] + +[[package]] +name = "objc2-core-graphics" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e022c9d066895efa1345f8e33e584b9f958da2fd4cd116792e15e07e4720a807" +dependencies = [ + "bitflags 2.13.0", + "dispatch2", + "objc2", + "objc2-core-foundation", + "objc2-io-surface", +] + +[[package]] +name = "objc2-core-image" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5d563b38d2b97209f8e861173de434bd0214cf020e3423a52624cd1d989f006" +dependencies = [ + "objc2", + "objc2-foundation", +] + +[[package]] +name = "objc2-core-location" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ca347214e24bc973fc025fd0d36ebb179ff30536ed1f80252706db19ee452009" +dependencies = [ + "objc2", + "objc2-foundation", +] + +[[package]] +name = "objc2-core-text" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0cde0dfb48d25d2b4862161a4d5fcc0e3c24367869ad306b0c9ec0073bfed92d" +dependencies = [ + "bitflags 2.13.0", + "objc2", + "objc2-core-foundation", + "objc2-core-graphics", +] + +[[package]] +name = "objc2-encode" +version = "4.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33" + +[[package]] +name = "objc2-exception-helper" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7a1c5fbb72d7735b076bb47b578523aedc40f3c439bea6dfd595c089d79d98a" +dependencies = [ + "cc", +] + +[[package]] +name = "objc2-foundation" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3e0adef53c21f888deb4fa59fc59f7eb17404926ee8a6f59f5df0fd7f9f3272" +dependencies = [ + "bitflags 2.13.0", + "block2", + "objc2", + "objc2-core-foundation", +] + +[[package]] +name = "objc2-io-surface" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "180788110936d59bab6bd83b6060ffdfffb3b922ba1396b312ae795e1de9d81d" +dependencies = [ + "bitflags 2.13.0", + "objc2", + "objc2-core-foundation", +] + +[[package]] +name = "objc2-quartz-core" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96c1358452b371bf9f104e21ec536d37a650eb10f7ee379fff67d2e08d537f1f" +dependencies = [ + "bitflags 2.13.0", + "objc2", + "objc2-core-foundation", + "objc2-foundation", +] + +[[package]] +name = "objc2-ui-kit" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d87d638e33c06f577498cbcc50491496a3ed4246998a7fbba7ccb98b1e7eab22" +dependencies = [ + "bitflags 2.13.0", + "block2", + "objc2", + "objc2-cloud-kit", + "objc2-core-data", + "objc2-core-foundation", + "objc2-core-graphics", + "objc2-core-image", + "objc2-core-location", + "objc2-core-text", + "objc2-foundation", + "objc2-quartz-core", + "objc2-user-notifications", +] + +[[package]] +name = "objc2-user-notifications" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9df9128cbbfef73cda168416ccf7f837b62737d748333bfe9ab71c245d76613e" +dependencies = [ + "objc2", + "objc2-foundation", +] + +[[package]] +name = "objc2-web-kit" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b2e5aaab980c433cf470df9d7af96a7b46a9d892d521a2cbbb2f8a4c16751e7f" +dependencies = [ + "bitflags 2.13.0", + "block2", + "objc2", + "objc2-app-kit", + "objc2-core-foundation", + "objc2-foundation", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "option-ext" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" + +[[package]] +name = "pango" +version = "0.18.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ca27ec1eb0457ab26f3036ea52229edbdb74dee1edd29063f5b9b010e7ebee4" +dependencies = [ + "gio", + "glib", + "libc", + "once_cell", + "pango-sys", +] + +[[package]] +name = "pango-sys" +version = "0.18.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "436737e391a843e5933d6d9aa102cb126d501e815b83601365a948a518555dc5" +dependencies = [ + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + +[[package]] +name = "parking_lot" +version = "0.12.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-link 0.2.1", +] + +[[package]] +name = "percent-encoding" +version = "2.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220" + +[[package]] +name = "phf" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf" +dependencies = [ + "phf_macros", + "phf_shared", + "serde", +] + +[[package]] +name = "phf_codegen" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49aa7f9d80421bca176ca8dbfebe668cc7a2684708594ec9f3c0db0805d5d6e1" +dependencies = [ + "phf_generator", + "phf_shared", +] + +[[package]] +name = "phf_generator" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737" +dependencies = [ + "fastrand", + "phf_shared", +] + +[[package]] +name = "phf_macros" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "812f032b54b1e759ccd5f8b6677695d5268c588701effba24601f6932f8269ef" +dependencies = [ + "phf_generator", + "phf_shared", + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "phf_shared" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266" +dependencies = [ + "siphasher", +] + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "pkg-config" +version = "0.3.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e" + +[[package]] +name = "plist" +version = "1.10.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da1d65da6dd5d1e44199ac0f58712d241c0f439f80adea8924d832384087f85" +dependencies = [ + "base64 0.22.1", + "indexmap 2.14.0", + "quick-xml", + "serde", + "time", +] + +[[package]] +name = "png" +version = "0.17.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82151a2fc869e011c153adc57cf2789ccb8d9906ce52c0b39a6b5697749d7526" +dependencies = [ + "bitflags 1.3.2", + "crc32fast", + "fdeflate", + "flate2", + "miniz_oxide", +] + +[[package]] +name = "png" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60769b8b31b2a9f263dae2776c37b1b28ae246943cf719eb6946a1db05128a61" +dependencies = [ + "bitflags 2.13.0", + "crc32fast", + "fdeflate", + "flate2", + "miniz_oxide", +] + +[[package]] +name = "potential_utf" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0103b1cef7ec0cf76490e969665504990193874ea05c85ff9bab8b911d0a0564" +dependencies = [ + "zerovec", +] + +[[package]] +name = "powerfmt" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391" + +[[package]] +name = "precomputed-hash" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c" + +[[package]] +name = "proc-macro-crate" +version = "1.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f4c021e1093a56626774e81216a4ce732a735e5bad4868a03f3ed65ca0c3919" +dependencies = [ + "once_cell", + "toml_edit 0.19.15", +] + +[[package]] +name = "proc-macro-crate" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b00f26d3400549137f92511a46ac1cd8ce37cb5598a96d382381458b992a5d24" +dependencies = [ + "toml_datetime 0.6.3", + "toml_edit 0.20.2", +] + +[[package]] +name = "proc-macro-crate" +version = "3.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f" +dependencies = [ + "toml_edit 0.25.12+spec-1.1.0", +] + +[[package]] +name = "proc-macro-error" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da25490ff9892aab3fcf7c36f08cfb902dd3e71ca0f9f9517bea02a73a5ce38c" +dependencies = [ + "proc-macro-error-attr", + "proc-macro2", + "quote", + "syn 1.0.109", + "version_check", +] + +[[package]] +name = "proc-macro-error-attr" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1be40180e52ecc98ad80b184934baf3d0d29f979574e439af5a55274b35f869" +dependencies = [ + "proc-macro2", + "quote", + "version_check", +] + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quick-xml" +version = "0.41.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1" +dependencies = [ + "memchr", +] + +[[package]] +name = "quote" +version = "1.0.46" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dfbc457d0c7a0759a614551b11a6409e5951f6c7537be1f1b7682b9ae9230368" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "raw-window-handle" +version = "0.6.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20675572f6f24e9e76ef639bc5552774ed45f1c30e2951e1e99c59888861c539" + +[[package]] +name = "redox_syscall" +version = "0.5.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" +dependencies = [ + "bitflags 2.13.0", +] + +[[package]] +name = "redox_users" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a4e608c6638b9c18977b00b475ac1f28d14e84b27d8d42f70e0bf1e3dec127ac" +dependencies = [ + "getrandom 0.2.17", + "libredox", + "thiserror 2.0.18", +] + +[[package]] +name = "ref-cast" +version = "1.0.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f354300ae66f76f1c85c5f84693f0ce81d747e2c3f21a45fef496d89c960bf7d" +dependencies = [ + "ref-cast-impl", +] + +[[package]] +name = "ref-cast-impl" +version = "1.0.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7186006dcb21920990093f30e3dea63b7d6e977bf1256be20c3563a5db070da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "regex" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a0e75113e14dc5acb068cd0786884f214f1312650a3d36d269f5c4f3cdee8a2" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f388202e4b80542a0921078cc23b6333bcf1409c1e3f86404cae4766a6131db" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "reqwest" +version = "0.13.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "219c5811de6525e5416c7d5d53bb656d3afdbc6c5af816e0802bcfa42dbdc1c3" +dependencies = [ + "base64 0.22.1", + "bytes", + "futures-core", + "futures-util", + "http", + "http-body", + "http-body-util", + "hyper", + "hyper-util", + "js-sys", + "log", + "percent-encoding", + "pin-project-lite", + "serde", + "serde_json", + "sync_wrapper", + "tokio", + "tokio-util", + "tower", + "tower-http", + "tower-service", + "url", + "wasm-bindgen", + "wasm-bindgen-futures", + "wasm-streams", + "web-sys", +] + +[[package]] +name = "rustc-hash" +version = "2.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d" + +[[package]] +name = "rustc_version" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" +dependencies = [ + "semver", +] + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "same-file" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "schemars" +version = "0.8.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3fbf2ae1b8bc8e02df939598064d22402220cd5bbcca1c76f7d6a310974d5615" +dependencies = [ + "dyn-clone", + "indexmap 1.9.3", + "schemars_derive", + "serde", + "serde_json", + "url", + "uuid", +] + +[[package]] +name = "schemars" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4cd191f9397d57d581cddd31014772520aa448f65ef991055d7f61582c65165f" +dependencies = [ + "dyn-clone", + "ref-cast", + "serde", + "serde_json", +] + +[[package]] +name = "schemars" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2b42f36aa1cd011945615b92222f6bf73c599a102a300334cd7f8dbeec726cc" +dependencies = [ + "dyn-clone", + "ref-cast", + "serde", + "serde_json", +] + +[[package]] +name = "schemars_derive" +version = "0.8.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32e265784ad618884abaea0600a9adf15393368d840e0222d101a072f3f7534d" +dependencies = [ + "proc-macro2", + "quote", + "serde_derive_internals", + "syn 2.0.118", +] + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "selectors" +version = "0.36.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c5d9c0c92a92d33f08817311cf3f2c29a3538a8240e94a6a3c622ce652d7e00c" +dependencies = [ + "bitflags 2.13.0", + "cssparser", + "derive_more", + "log", + "new_debug_unreachable", + "phf", + "phf_codegen", + "precomputed-hash", + "rustc-hash", + "servo_arc", + "smallvec", +] + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" +dependencies = [ + "serde", + "serde_core", +] + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde-untagged" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f9faf48a4a2d2693be24c6289dbe26552776eb7737074e6722891fadbe6c5058" +dependencies = [ + "erased-serde", + "serde", + "serde_core", + "typeid", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "serde_derive_internals" +version = "0.29.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18d26a20a969b9e3fdf2fc2d9f21eda6c40e2de84c9408bb5d3b05d499aae711" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "serde_json" +version = "1.0.150" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_repr" +version = "0.1.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "175ee3e80ae9982737ca543e96133087cbd9a485eecc3bc4de9c1a37b47ea59c" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "serde_spanned" +version = "0.6.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3" +dependencies = [ + "serde", +] + +[[package]] +name = "serde_spanned" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26" +dependencies = [ + "serde_core", +] + +[[package]] +name = "serde_with" +version = "3.21.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "76a5c54c7310e7b8b9577c286d7e399ddd876c3e12b3ed917a8aabc4b96e9e8c" +dependencies = [ + "base64 0.22.1", + "bs58", + "chrono", + "hex", + "indexmap 1.9.3", + "indexmap 2.14.0", + "schemars 0.9.0", + "schemars 1.2.1", + "serde_core", + "serde_json", + "serde_with_macros", + "time", +] + +[[package]] +name = "serde_with_macros" +version = "3.21.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "84d57bc0c8b9a17920c178daa6bb924850d54a9c97ab45194bb8c17ad66bb660" +dependencies = [ + "darling", + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "serialize-to-javascript" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "04f3666a07a197cdb77cdf306c32be9b7f598d7060d50cfd4d5aa04bfd92f6c5" +dependencies = [ + "serde", + "serde_json", + "serialize-to-javascript-impl", +] + +[[package]] +name = "serialize-to-javascript-impl" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "772ee033c0916d670af7860b6e1ef7d658a4629a6d0b4c8c3e67f09b3765b75d" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "servo_arc" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "170fb83ab34de17dc69aa7c67482b22218ddb85da56546f9bd6b929e32a05930" +dependencies = [ + "stable_deref_trait", +] + +[[package]] +name = "sha2" +version = "0.10.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" +dependencies = [ + "cfg-if", + "cpufeatures", + "digest", +] + +[[package]] +name = "shlex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" + +[[package]] +name = "simd-adler32" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214" + +[[package]] +name = "siphasher" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649" + +[[package]] +name = "slab" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" + +[[package]] +name = "smallvec" +version = "1.15.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" + +[[package]] +name = "socket2" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52d1cfed4120b4d927bf7c0f86d2087a4a7d6027c906d9f9d525a80573b9be51" +dependencies = [ + "libc", + "windows-sys 0.61.2", +] + +[[package]] +name = "softbuffer" +version = "0.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aac18da81ebbf05109ab275b157c22a653bb3c12cf884450179942f81bcbf6c3" +dependencies = [ + "bytemuck", + "js-sys", + "ndk", + "objc2", + "objc2-core-foundation", + "objc2-core-graphics", + "objc2-foundation", + "objc2-quartz-core", + "raw-window-handle", + "redox_syscall", + "tracing", + "wasm-bindgen", + "web-sys", + "windows-sys 0.61.2", +] + +[[package]] +name = "soup3" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "471f924a40f31251afc77450e781cb26d55c0b650842efafc9c6cbd2f7cc4f9f" +dependencies = [ + "futures-channel", + "gio", + "glib", + "libc", + "soup3-sys", +] + +[[package]] +name = "soup3-sys" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7ebe8950a680a12f24f15ebe1bf70db7af98ad242d9db43596ad3108aab86c27" +dependencies = [ + "gio-sys", + "glib-sys", + "gobject-sys", + "libc", + "system-deps", +] + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "string_cache" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a18596f8c785a729f2819c0f6a7eae6ebeebdfffbfe4214ae6b087f690e31901" +dependencies = [ + "new_debug_unreachable", + "parking_lot", + "phf_shared", + "precomputed-hash", +] + +[[package]] +name = "string_cache_codegen" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "585635e46db231059f76c5849798146164652513eb9e8ab2685939dd90f29b69" +dependencies = [ + "phf_generator", + "phf_shared", + "proc-macro2", + "quote", +] + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "swift-rs" +version = "1.0.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4057c98e2e852d51fdcfca832aac7b571f6b351ad159f9eda5db1655f8d0c4d7" +dependencies = [ + "base64 0.21.7", + "serde", + "serde_json", +] + +[[package]] +name = "syn" +version = "1.0.109" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72b64191b275b66ffe2469e8af2c1cfe3bafa67b529ead792a6d0160888b4237" +dependencies = [ + "proc-macro2", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "2.0.118" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1b9ae57f904213ebb649ce6895b8a66c66f0203b9319718f69a5612a065b1422" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "sync_wrapper" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263" +dependencies = [ + "futures-core", +] + +[[package]] +name = "synstructure" +version = "0.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "system-deps" +version = "6.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3e535eb8dded36d55ec13eddacd30dec501792ff23a0b1682c38601b8cf2349" +dependencies = [ + "cfg-expr", + "heck 0.5.0", + "pkg-config", + "toml 0.8.2", + "version-compare", +] + +[[package]] +name = "tao" +version = "0.35.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1c93047acf68669466a34690ac58cca7010bd1b201e1ec86f1fd0a75d3dd4a9" +dependencies = [ + "bitflags 2.13.0", + "block2", + "core-foundation", + "core-graphics", + "crossbeam-channel", + "dbus", + "dispatch2", + "dlopen2", + "dpi", + "gdkwayland-sys", + "gdkx11-sys", + "gtk", + "jni", + "libc", + "log", + "ndk", + "ndk-sys", + "objc2", + "objc2-app-kit", + "objc2-foundation", + "objc2-ui-kit", + "once_cell", + "parking_lot", + "percent-encoding", + "raw-window-handle", + "tao-macros", + "unicode-segmentation", + "url", + "windows 0.61.3", + "windows-core 0.61.2", + "windows-version", + "x11-dl", +] + +[[package]] +name = "tao-macros" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4e16beb8b2ac17db28eab8bca40e62dbfbb34c0fcdc6d9826b11b7b5d047dfd" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "target-lexicon" +version = "0.12.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1" + +[[package]] +name = "tauri" +version = "2.11.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "667b20e2726d572dea2de7370da16e188eb06008faf9a92fab7cdc46791190b5" +dependencies = [ + "anyhow", + "bytes", + "cookie", + "dirs", + "dunce", + "embed_plist", + "getrandom 0.3.4", + "glob", + "gtk", + "heck 0.5.0", + "http", + "jni", + "libc", + "log", + "mime", + "muda", + "objc2", + "objc2-app-kit", + "objc2-foundation", + "objc2-ui-kit", + "objc2-web-kit", + "percent-encoding", + "plist", + "raw-window-handle", + "reqwest", + "serde", + "serde_json", + "serde_repr", + "serialize-to-javascript", + "swift-rs", + "tauri-build", + "tauri-macros", + "tauri-runtime", + "tauri-runtime-wry", + "tauri-utils", + "thiserror 2.0.18", + "tokio", + "tray-icon", + "url", + "webkit2gtk", + "webview2-com", + "window-vibrancy", + "windows 0.61.3", +] + +[[package]] +name = "tauri-build" +version = "2.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc9ce40b16101cb6ea63d3e221567affd1c3a9205f95d7bc574941a10636b632" +dependencies = [ + "anyhow", + "cargo_toml", + "dirs", + "glob", + "heck 0.5.0", + "json-patch", + "schemars 0.8.22", + "semver", + "serde", + "serde_json", + "tauri-utils", + "tauri-winres", + "walkdir", +] + +[[package]] +name = "tauri-codegen" +version = "2.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08279169ff42f8fc45a1dbc9dcae888893ba95288142e5880c59b93a26d2cfc5" +dependencies = [ + "base64 0.22.1", + "brotli", + "ico", + "json-patch", + "plist", + "png 0.17.16", + "proc-macro2", + "quote", + "semver", + "serde", + "serde_json", + "sha2", + "syn 2.0.118", + "tauri-utils", + "thiserror 2.0.18", + "time", + "url", + "uuid", + "walkdir", +] + +[[package]] +name = "tauri-macros" +version = "2.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e8b394794f399a421811d06966343e7933fcae92d59f5180b9388d1174497a45" +dependencies = [ + "heck 0.5.0", + "proc-macro2", + "quote", + "syn 2.0.118", + "tauri-codegen", + "tauri-utils", +] + +[[package]] +name = "tauri-runtime" +version = "2.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b0b4bc95aed361b0019067d189a1174a603d460d0f6c72606512d59fc9c12ec8" +dependencies = [ + "cookie", + "dpi", + "gtk", + "http", + "jni", + "objc2", + "objc2-ui-kit", + "objc2-web-kit", + "raw-window-handle", + "serde", + "serde_json", + "tauri-utils", + "thiserror 2.0.18", + "url", + "webkit2gtk", + "webview2-com", + "windows 0.61.3", +] + +[[package]] +name = "tauri-runtime-wry" +version = "2.11.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e6fac707727b7a2f48e4ded90976324267371073edbb415ffb73bb0458d203f" +dependencies = [ + "gtk", + "http", + "jni", + "log", + "objc2", + "objc2-app-kit", + "once_cell", + "percent-encoding", + "raw-window-handle", + "softbuffer", + "tao", + "tauri-runtime", + "tauri-utils", + "url", + "webkit2gtk", + "webview2-com", + "windows 0.61.3", + "wry", +] + +[[package]] +name = "tauri-utils" +version = "2.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3e176a18e67764923c4f1ce66f25ae4abe5f688384d5eb1a0fa6c77f3d90f887" +dependencies = [ + "anyhow", + "brotli", + "cargo_metadata", + "ctor", + "dom_query", + "dunce", + "glob", + "http", + "infer", + "json-patch", + "log", + "memchr", + "phf", + "plist", + "proc-macro2", + "quote", + "regex", + "schemars 0.8.22", + "semver", + "serde", + "serde-untagged", + "serde_json", + "serde_with", + "swift-rs", + "thiserror 2.0.18", + "toml 1.1.2+spec-1.1.0", + "url", + "urlpattern", + "uuid", + "walkdir", +] + +[[package]] +name = "tauri-winres" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc65d45c68858bfe420dd29e834b5d15dbecf8a07a8a16cf4d532c7b1f69d4b6" +dependencies = [ + "dunce", + "embed-resource", + "toml 1.1.2+spec-1.1.0", +] + +[[package]] +name = "tendril" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5fed54709c5b3a53d09bb1c113ea4f5ceafd1e772ddcb0030a82e1d56c087b08" +dependencies = [ + "new_debug_unreachable", +] + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl 1.0.69", +] + +[[package]] +name = "thiserror" +version = "2.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4288b5bcbc7920c07a1149a35cf9590a2aa808e0bc1eafaade0b80947865fbc4" +dependencies = [ + "thiserror-impl 2.0.18", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebc4ee7f67670e9b64d05fa4253e753e016c6c95ff35b89b7941d6b856dec1d5" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "time" +version = "0.3.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18dfaaeddcb932337b5e7866ee7d0ce9b76d2fd092997146f187ec09b4558a50" +dependencies = [ + "deranged", + "num-conv", + "powerfmt", + "serde_core", + "time-core", + "time-macros", +] + +[[package]] +name = "time-core" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e1c906769ad99c88eaa54e728060edef082f8e358ff32030cb7c7d315e81109" + +[[package]] +name = "time-macros" +version = "0.2.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c431b87111666e491a90baa837f914fb45cd5dc3c268591b0220ff5057f2085f" +dependencies = [ + "num-conv", + "time-core", +] + +[[package]] +name = "tinystr" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8323304221c2a851516f22236c5722a72eaa19749016521d6dff0824447d96d" +dependencies = [ + "displaydoc", + "zerovec", +] + +[[package]] +name = "tinyvec" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bb4ebadaa0af04fab11ae01eb5f9fdb5f9c5b875506e210e71c07873528baa7f" +dependencies = [ + "tinyvec_macros", +] + +[[package]] +name = "tinyvec_macros" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" + +[[package]] +name = "tokio" +version = "1.52.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fc7f01b389ac15039e4dc9531aa973a135d7a4135281b12d7c1bc79fd57fffe" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-util" +version = "0.7.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ae9cec805b01e8fc3fd2fe289f89149a9b66dd16786abd8b19cfa7b48cb0098" +dependencies = [ + "bytes", + "futures-core", + "futures-sink", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "toml" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "185d8ab0dfbb35cf1399a6344d8484209c088f75f8f68230da55d48d95d43e3d" +dependencies = [ + "serde", + "serde_spanned 0.6.9", + "toml_datetime 0.6.3", + "toml_edit 0.20.2", +] + +[[package]] +name = "toml" +version = "0.9.12+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf92845e79fc2e2def6a5d828f0801e29a2f8acc037becc5ab08595c7d5e9863" +dependencies = [ + "indexmap 2.14.0", + "serde_core", + "serde_spanned 1.1.1", + "toml_datetime 0.7.5+spec-1.1.0", + "toml_parser", + "toml_writer", + "winnow 0.7.15", +] + +[[package]] +name = "toml" +version = "1.1.2+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee" +dependencies = [ + "indexmap 2.14.0", + "serde_core", + "serde_spanned 1.1.1", + "toml_datetime 1.1.1+spec-1.1.0", + "toml_parser", + "toml_writer", + "winnow 1.0.3", +] + +[[package]] +name = "toml_datetime" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7cda73e2f1397b1262d6dfdcef8aafae14d1de7748d66822d3bfeeb6d03e5e4b" +dependencies = [ + "serde", +] + +[[package]] +name = "toml_datetime" +version = "0.7.5+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92e1cfed4a3038bc5a127e35a2d360f145e1f4b971b551a2ba5fd7aedf7e1347" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_datetime" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_edit" +version = "0.19.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1b5bb770da30e5cbfde35a2d7b9b8a2c4b8ef89548a7a6aeab5c9a576e3e7421" +dependencies = [ + "indexmap 2.14.0", + "toml_datetime 0.6.3", + "winnow 0.5.40", +] + +[[package]] +name = "toml_edit" +version = "0.20.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "396e4d48bbb2b7554c944bde63101b5ae446cff6ec4a24227428f15eb72ef338" +dependencies = [ + "indexmap 2.14.0", + "serde", + "serde_spanned 0.6.9", + "toml_datetime 0.6.3", + "winnow 0.5.40", +] + +[[package]] +name = "toml_edit" +version = "0.25.12+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2153edc6955a6c354fad8f5efd38b6a8769bdccf9fe50f8e1329f81b0baa5d7" +dependencies = [ + "indexmap 2.14.0", + "toml_datetime 1.1.1+spec-1.1.0", + "toml_parser", + "winnow 1.0.3", +] + +[[package]] +name = "toml_parser" +version = "1.1.2+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526" +dependencies = [ + "winnow 1.0.3", +] + +[[package]] +name = "toml_writer" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db" + +[[package]] +name = "tower" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4" +dependencies = [ + "futures-core", + "futures-util", + "pin-project-lite", + "sync_wrapper", + "tokio", + "tower-layer", + "tower-service", +] + +[[package]] +name = "tower-http" +version = "0.6.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840" +dependencies = [ + "bitflags 2.13.0", + "bytes", + "futures-util", + "http", + "http-body", + "pin-project-lite", + "tower", + "tower-layer", + "tower-service", + "url", +] + +[[package]] +name = "tower-layer" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "121c2a6cda46980bb0fcd1647ffaf6cd3fc79a013de288782836f6df9c48780e" + +[[package]] +name = "tower-service" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3" + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-core", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "tray-icon" +version = "0.24.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "65ba1e5f6b9ef9fd87e21b9c6f351554dbd717960089168fcfdef854686961dc" +dependencies = [ + "crossbeam-channel", + "dirs", + "libappindicator", + "muda", + "objc2", + "objc2-app-kit", + "objc2-core-foundation", + "objc2-core-graphics", + "objc2-foundation", + "once_cell", + "png 0.18.1", + "serde", + "thiserror 2.0.18", + "windows-sys 0.61.2", +] + +[[package]] +name = "try-lock" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" + +[[package]] +name = "typeid" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc7d623258602320d5c55d1bc22793b57daff0ec7efc270ea7d55ce1d5f5471c" + +[[package]] +name = "typenum" +version = "1.20.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20" + +[[package]] +name = "unic-char-property" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a8c57a407d9b6fa02b4795eb81c5b6652060a15a7903ea981f3d723e6c0be221" +dependencies = [ + "unic-char-range", +] + +[[package]] +name = "unic-char-range" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0398022d5f700414f6b899e10b8348231abf9173fa93144cbc1a43b9793c1fbc" + +[[package]] +name = "unic-common" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "80d7ff825a6a654ee85a63e80f92f054f904f21e7d12da4e22f9834a4aaa35bc" + +[[package]] +name = "unic-ucd-ident" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e230a37c0381caa9219d67cf063aa3a375ffed5bf541a452db16e744bdab6987" +dependencies = [ + "unic-char-property", + "unic-char-range", + "unic-ucd-version", +] + +[[package]] +name = "unic-ucd-version" +version = "0.9.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96bd2f2237fe450fcd0a1d2f5f4e91711124f7857ba2e964247776ebeeb7b0c4" +dependencies = [ + "unic-common", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unicode-segmentation" +version = "1.13.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c6f5d3c3b1bf09027a88a6bc961fc00497d651009560b5463668dc81b0fa87a8" + +[[package]] +name = "url" +version = "2.5.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff67a8a4397373c3ef660812acab3268222035010ab8680ec4215f38ba3d0eed" +dependencies = [ + "form_urlencoded", + "idna", + "percent-encoding", + "serde", + "serde_derive", +] + +[[package]] +name = "urlpattern" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70acd30e3aa1450bc2eece896ce2ad0d178e9c079493819301573dae3c37ba6d" +dependencies = [ + "regex", + "serde", + "unic-ucd-ident", + "url", +] + +[[package]] +name = "utf8_iter" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be" + +[[package]] +name = "uuid" +version = "1.23.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf80a72845275afea99e7f2b434723d3bc7e38470fcd1c7ed39a599c73319a53" +dependencies = [ + "getrandom 0.4.3", + "js-sys", + "serde_core", + "wasm-bindgen", +] + +[[package]] +name = "version-compare" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "03c2856837ef78f57382f06b2b8563a2f512f7185d732608fd9176cb3b8edf0e" + +[[package]] +name = "version_check" +version = "0.9.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a" + +[[package]] +name = "vswhom" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "be979b7f07507105799e854203b470ff7c78a1639e330a58f183b5fea574608b" +dependencies = [ + "libc", + "vswhom-sys", +] + +[[package]] +name = "vswhom-sys" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb067e4cbd1ff067d1df46c9194b5de0e98efd2810bbc95c5d5e5f25a3231150" +dependencies = [ + "cc", + "libc", +] + +[[package]] +name = "walkdir" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b" +dependencies = [ + "same-file", + "winapi-util", +] + +[[package]] +name = "want" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bfa7760aed19e106de2c7c0b581b509f2f25d3dacaf737cb82ac61bc6d760b0e" +dependencies = [ + "try-lock", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasip2" +version = "1.0.4+wasi-0.2.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b67efb37e106e55ce722a510d6b5f9c17f083e5fc79afc2badeb12cc313d9487" +dependencies = [ + "wit-bindgen", +] + +[[package]] +name = "wasm-bindgen" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b067c0c11094aef6b7a801c1e34a26affafdf3d051dba08456b868789aaf9a4" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-futures" +version = "0.4.76" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c62df1340f32221cb9c54d6a27b030e3dba64361d4a95bed55f9aacb44da291d" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "167ce5e579f6bcf889c4f7175a8a5a585de84e8ff93976ce393efa5f2837aab1" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3997c7839262f4ef12cf90b818d6340c18e80f263f1a94bf157d0ec4420380e" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 2.0.118", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.126" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc1b4cb0cc549fcf58d7dfc081778139b3d283a081644e833e84682ad71cea24" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "wasm-streams" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9d1ec4f6517c9e11ae630e200b2b65d193279042e28edd4a2cda233e46670bbb" +dependencies = [ + "futures-util", + "js-sys", + "wasm-bindgen", + "wasm-bindgen-futures", + "web-sys", +] + +[[package]] +name = "web-sys" +version = "0.3.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8622dcb61c0bcc9fffa6938bed81210af2da9a7e4a1a834b2e37a59b6dfb6141" +dependencies = [ + "js-sys", + "wasm-bindgen", +] + +[[package]] +name = "web_atoms" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "075474b12bcb3d2e3d4546580e9de478eeeead668a1761e2a8860c836b7ef297" +dependencies = [ + "phf", + "phf_codegen", + "string_cache", + "string_cache_codegen", +] + +[[package]] +name = "webkit2gtk" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1027150013530fb2eaf806408df88461ae4815a45c541c8975e61d6f2fc4793" +dependencies = [ + "bitflags 1.3.2", + "cairo-rs", + "gdk", + "gdk-sys", + "gio", + "gio-sys", + "glib", + "glib-sys", + "gobject-sys", + "gtk", + "gtk-sys", + "javascriptcore-rs", + "libc", + "once_cell", + "soup3", + "webkit2gtk-sys", +] + +[[package]] +name = "webkit2gtk-sys" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "916a5f65c2ef0dfe12fff695960a2ec3d4565359fdbb2e9943c974e06c734ea5" +dependencies = [ + "bitflags 1.3.2", + "cairo-sys-rs", + "gdk-sys", + "gio-sys", + "glib-sys", + "gobject-sys", + "gtk-sys", + "javascriptcore-rs-sys", + "libc", + "pkg-config", + "soup3-sys", + "system-deps", +] + +[[package]] +name = "webview2-com" +version = "0.38.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7130243a7a5b33c54a444e54842e6a9e133de08b5ad7b5861cd8ed9a6a5bc96a" +dependencies = [ + "webview2-com-macros", + "webview2-com-sys", + "windows 0.61.3", + "windows-core 0.61.2", + "windows-implement", + "windows-interface", +] + +[[package]] +name = "webview2-com-macros" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67a921c1b6914c367b2b823cd4cde6f96beec77d30a939c8199bb377cf9b9b54" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "webview2-com-sys" +version = "0.38.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "381336cfffd772377d291702245447a5251a2ffa5bad679c99e61bc48bacbf9c" +dependencies = [ + "thiserror 2.0.18", + "windows 0.61.3", + "windows-core 0.61.2", +] + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + +[[package]] +name = "winapi-util" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "window-vibrancy" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9bec5a31f3f9362f2258fd0e9c9dd61a9ca432e7306cc78c444258f0dce9a9c" +dependencies = [ + "objc2", + "objc2-app-kit", + "objc2-core-foundation", + "objc2-foundation", + "raw-window-handle", + "windows-sys 0.59.0", + "windows-version", +] + +[[package]] +name = "windows" +version = "0.61.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9babd3a767a4c1aef6900409f85f5d53ce2544ccdfaa86dad48c91782c6d6893" +dependencies = [ + "windows-collections 0.2.0", + "windows-core 0.61.2", + "windows-future 0.2.1", + "windows-link 0.1.3", + "windows-numerics 0.2.0", +] + +[[package]] +name = "windows" +version = "0.62.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "527fadee13e0c05939a6a05d5bd6eec6cd2e3dbd648b9f8e447c6518133d8580" +dependencies = [ + "windows-collections 0.3.2", + "windows-core 0.62.2", + "windows-future 0.3.2", + "windows-numerics 0.3.1", +] + +[[package]] +name = "windows-collections" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3beeceb5e5cfd9eb1d76b381630e82c4241ccd0d27f1a39ed41b2760b255c5e8" +dependencies = [ + "windows-core 0.61.2", +] + +[[package]] +name = "windows-collections" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23b2d95af1a8a14a3c7367e1ed4fc9c20e0a26e79551b1454d72583c97cc6610" +dependencies = [ + "windows-core 0.62.2", +] + +[[package]] +name = "windows-core" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c0fdd3ddb90610c7638aa2b3a3ab2904fb9e5cdbecc643ddb3647212781c4ae3" +dependencies = [ + "windows-implement", + "windows-interface", + "windows-link 0.1.3", + "windows-result 0.3.4", + "windows-strings 0.4.2", +] + +[[package]] +name = "windows-core" +version = "0.62.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb" +dependencies = [ + "windows-implement", + "windows-interface", + "windows-link 0.2.1", + "windows-result 0.4.1", + "windows-strings 0.5.1", +] + +[[package]] +name = "windows-future" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc6a41e98427b19fe4b73c550f060b59fa592d7d686537eebf9385621bfbad8e" +dependencies = [ + "windows-core 0.61.2", + "windows-link 0.1.3", + "windows-threading 0.1.0", +] + +[[package]] +name = "windows-future" +version = "0.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e1d6f90251fe18a279739e78025bd6ddc52a7e22f921070ccdc67dde84c605cb" +dependencies = [ + "windows-core 0.62.2", + "windows-link 0.2.1", + "windows-threading 0.2.1", +] + +[[package]] +name = "windows-implement" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "windows-interface" +version = "0.59.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "windows-link" +version = "0.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5e6ad25900d524eaabdbbb96d20b4311e1e7ae1699af4fb28c17ae66c80d798a" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-numerics" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9150af68066c4c5c07ddc0ce30421554771e528bde427614c61038bc2c92c2b1" +dependencies = [ + "windows-core 0.61.2", + "windows-link 0.1.3", +] + +[[package]] +name = "windows-numerics" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e2e40844ac143cdb44aead537bbf727de9b044e107a0f1220392177d15b0f26" +dependencies = [ + "windows-core 0.62.2", + "windows-link 0.2.1", +] + +[[package]] +name = "windows-result" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56f42bd332cc6c8eac5af113fc0c1fd6a8fd2aa08a0119358686e5160d0586c6" +dependencies = [ + "windows-link 0.1.3", +] + +[[package]] +name = "windows-result" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5" +dependencies = [ + "windows-link 0.2.1", +] + +[[package]] +name = "windows-strings" +version = "0.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "56e6c93f3a0c3b36176cb1327a4958a0353d5d166c2a35cb268ace15e91d3b57" +dependencies = [ + "windows-link 0.1.3", +] + +[[package]] +name = "windows-strings" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091" +dependencies = [ + "windows-link 0.2.1", +] + +[[package]] +name = "windows-sys" +version = "0.45.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "75283be5efb2831d37ea142365f009c02ec203cd29a3ebecbc093d52315b66d0" +dependencies = [ + "windows-targets 0.42.2", +] + +[[package]] +name = "windows-sys" +version = "0.59.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e38bc4d79ed67fd075bcc251a1c39b32a1776bbe92e5bef1f0bf1f8c531853b" +dependencies = [ + "windows-targets 0.52.6", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link 0.2.1", +] + +[[package]] +name = "windows-targets" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e5180c00cd44c9b1c88adb3693291f1cd93605ded80c250a75d472756b4d071" +dependencies = [ + "windows_aarch64_gnullvm 0.42.2", + "windows_aarch64_msvc 0.42.2", + "windows_i686_gnu 0.42.2", + "windows_i686_msvc 0.42.2", + "windows_x86_64_gnu 0.42.2", + "windows_x86_64_gnullvm 0.42.2", + "windows_x86_64_msvc 0.42.2", +] + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm 0.52.6", + "windows_aarch64_msvc 0.52.6", + "windows_i686_gnu 0.52.6", + "windows_i686_gnullvm", + "windows_i686_msvc 0.52.6", + "windows_x86_64_gnu 0.52.6", + "windows_x86_64_gnullvm 0.52.6", + "windows_x86_64_msvc 0.52.6", +] + +[[package]] +name = "windows-threading" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b66463ad2e0ea3bbf808b7f1d371311c80e115c0b71d60efc142cafbcfb057a6" +dependencies = [ + "windows-link 0.1.3", +] + +[[package]] +name = "windows-threading" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3949bd5b99cafdf1c7ca86b43ca564028dfe27d66958f2470940f73d86d75b37" +dependencies = [ + "windows-link 0.2.1", +] + +[[package]] +name = "windows-version" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e4060a1da109b9d0326b7262c8e12c84df67cc0dbc9e33cf49e01ccc2eb63631" +dependencies = [ + "windows-link 0.2.1", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "597a5118570b68bc08d8d59125332c54f1ba9d9adeedeef5b99b02ba2b0698f8" + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e08e8864a60f06ef0d0ff4ba04124db8b0fb3be5776a5cd47641e942e58c4d43" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + +[[package]] +name = "windows_i686_gnu" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c61d927d8da41da96a81f029489353e68739737d3beca43145c8afec9a31a84f" + +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + +[[package]] +name = "windows_i686_msvc" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "44d840b6ec649f480a41c8d80f9c65108b92d89345dd94027bfe06ac444d1060" + +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8de912b8b8feb55c064867cf047dda097f92d51efad5b491dfb98f6bbb70cb36" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26d41b46a36d453748aedef1486d5c7a85db22e56aff34643984ea85514e94a3" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.42.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + +[[package]] +name = "winnow" +version = "0.5.40" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f593a95398737aeed53e489c785df13f3618e41dbcd6718c6addbf1395aa6876" +dependencies = [ + "memchr", +] + +[[package]] +name = "winnow" +version = "0.7.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945" + +[[package]] +name = "winnow" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0592e1c9d151f854e6fd382574c3a0855250e1d9b2f99d9281c6e6391af352f1" +dependencies = [ + "memchr", +] + +[[package]] +name = "winreg" +version = "0.55.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cb5a765337c50e9ec252c2069be9bf91c7df47afb103b642ba3a53bf8101be97" +dependencies = [ + "cfg-if", + "windows-sys 0.59.0", +] + +[[package]] +name = "wit-bindgen" +version = "0.57.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e" + +[[package]] +name = "writeable" +version = "0.6.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ffae5123b2d3fc086436f8834ae3ab053a283cfac8fe0a0b8eaae044768a4c4" + +[[package]] +name = "wry" +version = "0.55.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "186f9871daa55fd9c016578b810d149de58367113db7fb72b462d2323ce19514" +dependencies = [ + "base64 0.22.1", + "block2", + "cookie", + "crossbeam-channel", + "dirs", + "dom_query", + "dpi", + "dunce", + "gdkx11", + "gtk", + "http", + "javascriptcore-rs", + "jni", + "libc", + "ndk", + "objc2", + "objc2-app-kit", + "objc2-core-foundation", + "objc2-foundation", + "objc2-ui-kit", + "objc2-web-kit", + "once_cell", + "percent-encoding", + "raw-window-handle", + "sha2", + "soup3", + "tao-macros", + "thiserror 2.0.18", + "url", + "webkit2gtk", + "webkit2gtk-sys", + "webview2-com", + "windows 0.61.3", + "windows-core 0.61.2", + "windows-version", + "x11-dl", +] + +[[package]] +name = "x11" +version = "2.21.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "502da5464ccd04011667b11c435cb992822c2c0dbde1770c988480d312a0db2e" +dependencies = [ + "libc", + "pkg-config", +] + +[[package]] +name = "x11-dl" +version = "2.21.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "38735924fedd5314a6e548792904ed8c6de6636285cb9fec04d5b1db85c1516f" +dependencies = [ + "libc", + "once_cell", + "pkg-config", +] + +[[package]] +name = "yoke" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "709fe23a0424b6a435d82152b1bd3fdfb0833487d5fa90d05d42762a9891fef5" +dependencies = [ + "stable_deref_trait", + "yoke-derive", + "zerofrom", +] + +[[package]] +name = "yoke-derive" +version = "0.8.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", + "synstructure", +] + +[[package]] +name = "zerofrom" +version = "0.1.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ec05a11813ea801ff6d75110ad09cd0824ddba17dfe17128ea0d5f68e6c5272" +dependencies = [ + "zerofrom-derive", +] + +[[package]] +name = "zerofrom-derive" +version = "0.1.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", + "synstructure", +] + +[[package]] +name = "zerotrie" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0f9152d31db0792fa83f70fb2f83148effb5c1f5b8c7686c3459e361d9bc20bf" +dependencies = [ + "displaydoc", + "yoke", + "zerofrom", +] + +[[package]] +name = "zerovec" +version = "0.11.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "90f911cbc359ab6af17377d242225f4d75119aec87ea711a880987b18cd7b239" +dependencies = [ + "yoke", + "zerofrom", + "zerovec-derive", +] + +[[package]] +name = "zerovec-derive" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + +[[package]] +name = "zmij" +version = "1.0.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa" diff --git a/desktop/src-tauri/Cargo.toml b/desktop/src-tauri/Cargo.toml new file mode 100644 index 00000000..6d85d560 --- /dev/null +++ b/desktop/src-tauri/Cargo.toml @@ -0,0 +1,29 @@ +[package] +name = "gently-desktop" +version = "0.1.0" +edition = "2021" +description = "Gently desktop shell — owns the Python backend + device layer" +authors = ["gently"] +default-run = "gently-desktop" + +[build-dependencies] +tauri-build = { version = "2", features = [] } + +[dependencies] +tauri = { version = "2" } + +# Windows Job Object: put the spawned Python (and its device-layer grandchild) +# in a kill-on-close job so the whole tree dies when the shell exits or crashes. +[target.'cfg(windows)'.dependencies] +windows = { version = "0.62", features = [ + "Win32_Foundation", + "Win32_System_JobObjects", + "Win32_System_Threading", + "Win32_Security", +] } + +[profile.release] +codegen-units = 1 +lto = true +opt-level = "s" +strip = true diff --git a/desktop/src-tauri/build.rs b/desktop/src-tauri/build.rs new file mode 100644 index 00000000..d860e1e6 --- /dev/null +++ b/desktop/src-tauri/build.rs @@ -0,0 +1,3 @@ +fn main() { + tauri_build::build() +} diff --git a/desktop/src-tauri/capabilities/default.json b/desktop/src-tauri/capabilities/default.json new file mode 100644 index 00000000..c1da0cbb --- /dev/null +++ b/desktop/src-tauri/capabilities/default.json @@ -0,0 +1,7 @@ +{ + "$schema": "../gen/schemas/desktop-schema.json", + "identifier": "default", + "description": "Thin shell — the UI is served by the Python backend and makes no Tauri IPC calls. Navigation is driven from Rust, so no window/shell permissions are needed.", + "windows": ["main"], + "permissions": ["core:default"] +} diff --git a/desktop/src-tauri/icons/128x128.png b/desktop/src-tauri/icons/128x128.png new file mode 100644 index 00000000..28f09f10 Binary files /dev/null and b/desktop/src-tauri/icons/128x128.png differ diff --git a/desktop/src-tauri/icons/128x128@2x.png b/desktop/src-tauri/icons/128x128@2x.png new file mode 100644 index 00000000..ceff7feb Binary files /dev/null and b/desktop/src-tauri/icons/128x128@2x.png differ diff --git a/desktop/src-tauri/icons/32x32.png b/desktop/src-tauri/icons/32x32.png new file mode 100644 index 00000000..777c4bb9 Binary files /dev/null and b/desktop/src-tauri/icons/32x32.png differ diff --git a/desktop/src-tauri/icons/64x64.png b/desktop/src-tauri/icons/64x64.png new file mode 100644 index 00000000..f05ad3ab Binary files /dev/null and b/desktop/src-tauri/icons/64x64.png differ diff --git a/desktop/src-tauri/icons/icon.icns b/desktop/src-tauri/icons/icon.icns new file mode 100644 index 00000000..f0a19c15 Binary files /dev/null and b/desktop/src-tauri/icons/icon.icns differ diff --git a/desktop/src-tauri/icons/icon.ico b/desktop/src-tauri/icons/icon.ico new file mode 100644 index 00000000..744ad499 Binary files /dev/null and b/desktop/src-tauri/icons/icon.ico differ diff --git a/desktop/src-tauri/icons/icon.png b/desktop/src-tauri/icons/icon.png new file mode 100644 index 00000000..22963015 Binary files /dev/null and b/desktop/src-tauri/icons/icon.png differ diff --git a/desktop/src-tauri/src/main.rs b/desktop/src-tauri/src/main.rs new file mode 100644 index 00000000..7e7ace4c --- /dev/null +++ b/desktop/src-tauri/src/main.rs @@ -0,0 +1,366 @@ +// Gently desktop shell. +// +// A thin Tauri (WebView2) window that OWNS the Python backend. On launch it +// spawns `launch_gently.py --no-browser`, shows a splash while uvicorn boots, +// then navigates the window to the live UI (http://localhost:). The whole +// UI is served by Python — this shell holds no application logic. +// +// Process ownership: the Python child spawns its own device-layer grandchild +// (via DeviceLayerSupervisor). To guarantee no orphans, on Windows we put the +// spawned Python into a Job Object with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE; +// children inherit the job, so when this shell exits or crashes the OS reaps the +// entire tree. See docs/superpowers/specs/2026-07-02-unified-launcher-design.md. +// +// Graceful shutdown (issue #85): on window-close the shell first ASKS the +// backend to stop (POST /api/shutdown — drains replay ingest, stops the device +// layer via its clean SIGTERM path), waits briefly for the port to go down, +// and only then exits — where the kill + Job-close floor still applies as the +// unchanged crash-safe fallback. +// +// Deferred (documented): bundling the Python environment for a redistributable +// installer (embeddable Python / PyInstaller sidecar). + +#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")] + +use std::path::{Path, PathBuf}; +use std::process::{Child, Command}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Mutex; +use std::time::{Duration, Instant}; + +use tauri::{Manager, RunEvent}; + +/// True once a graceful shutdown handshake is in flight (or done). A second +/// close request while true is allowed straight through to the hard path. +static SHUTTING: AtomicBool = AtomicBool::new(false); + +/// Backend process + (Windows) the job that owns its tree, held for the app's +/// whole lifetime as Tauri managed state. +struct Backend { + child: Mutex>, + #[cfg(windows)] + job: jobkill::Job, +} + +fn main() { + #[cfg(windows)] + let job = jobkill::create_kill_on_close().expect("failed to create Job Object"); + + let backend = Backend { + child: Mutex::new(None), + #[cfg(windows)] + job, + }; + + tauri::Builder::default() + .manage(backend) + .setup(|app| { + let handle = app.handle().clone(); + // Boot the backend off the UI thread so the splash paints immediately. + std::thread::spawn(move || boot_backend(handle)); + Ok(()) + }) + .build(tauri::generate_context!()) + .expect("error building the Gently desktop app") + .run(|app_handle, event| match event { + // Window close → graceful backend handshake first (issue #85). + // Intercept the close, ask the backend to stop over HTTP, and only + // exit once it's down (or the deadline passes). If a handshake is + // already in flight (or the user insists with a second close), let + // the close proceed to the hard path below. + RunEvent::WindowEvent { + event: tauri::WindowEvent::CloseRequested { api, .. }, + .. + } => { + if SHUTTING.load(Ordering::SeqCst) { + return; + } + SHUTTING.store(true, Ordering::SeqCst); + api.prevent_close(); + let handle = app_handle.clone(); + std::thread::spawn(move || graceful_shutdown(handle)); + } + RunEvent::Exit => { + let b = app_handle.state::(); + // Best-effort direct kill of the child we spawned... + // (trailing `;` so the MutexGuard temporary drops before `b` — + // without it this is the block's tail expression on non-Windows, + // where the cfg'd jobkill line below is compiled out: E0597) + if let Some(mut child) = b.child.lock().unwrap().take() { + let _ = child.kill(); + }; + // ...and close the job handle, which kill-on-close uses to reap + // the whole tree (python + device-layer grandchild). Correct even + // if the child kill above missed a grandchild. + #[cfg(windows)] + jobkill::close(&b.job); + } + _ => {} + }); +} + +/// The graceful shutdown handshake (issue #85), run off the UI thread. +/// +/// POST /api/shutdown → 200: backend is draining (device layer stopped via its +/// SIGTERM path, replay batches flushed) — wait for its port to close, then +/// exit. 409: an acquisition is running — ask the operator in the webview; on +/// confirm the PAGE resends with `{"confirm": true}` (same-origin fetch, no +/// second Rust HTTP path). Any exit still runs the RunEvent::Exit arm, so the +/// kill + Job-close floor is unchanged. +fn graceful_shutdown(app: tauri::AppHandle) { + let port = viz_port(); + match post_shutdown(port, false) { + Some(200) => { + // Backend acknowledged — give it a bounded window to drain + exit. + wait_for_port_closed("127.0.0.1", port, 6); + app.exit(0); + } + Some(409) => { + // Mid-run guard tripped. Confirm in the webview; the page itself + // resends the shutdown with {"confirm": true} if the user agrees. + if let Some(win) = app.get_webview_window("main") { + let _ = win.eval( + "if(confirm('An acquisition is running — quit anyway?'))\ + {fetch('/api/shutdown',{method:'POST',\ + headers:{'Content-Type':'application/json'},\ + body:JSON.stringify({confirm:true})});}", + ); + } + if wait_for_port_closed("127.0.0.1", port, 10) { + app.exit(0); + } else { + // Backend still up — the operator cancelled. Keep the window + // open and re-arm the handshake for the next close. + SHUTTING.store(false, Ordering::SeqCst); + } + } + // Anything else (backend dead, hung, or an unexpected status): nothing + // to hand-shake with — exit now; the Exit arm reaps the tree. + _ => { + app.exit(0); + } + } +} + +/// POST /api/shutdown to the local backend over a raw TCP socket — no HTTP +/// client dependency (style precedent: `wait_for_port` below). Returns the +/// response status code, or None if the request failed outright. +fn post_shutdown(port: u16, confirm: bool) -> Option { + use std::io::{Read, Write}; + use std::net::TcpStream; + + let addr: std::net::SocketAddr = format!("127.0.0.1:{port}").parse().ok()?; + let mut stream = TcpStream::connect_timeout(&addr, Duration::from_secs(2)).ok()?; + let _ = stream.set_read_timeout(Some(Duration::from_secs(2))); + let body = if confirm { r#"{"confirm":true}"# } else { "{}" }; + let req = format!( + "POST /api/shutdown HTTP/1.1\r\n\ + Host: 127.0.0.1:{port}\r\n\ + Content-Type: application/json\r\n\ + Content-Length: {}\r\n\ + Connection: close\r\n\r\n{body}", + body.len() + ); + stream.write_all(req.as_bytes()).ok()?; + let mut buf = [0u8; 512]; + let n = stream.read(&mut buf).ok()?; + // Status line: "HTTP/1.1 200 OK" — the second token is the code. + String::from_utf8_lossy(&buf[..n]) + .split_whitespace() + .nth(1)? + .parse() + .ok() +} + +/// Poll a TCP port until nothing accepts (backend gone), or `secs` elapse. +/// Returns true if the port closed within the deadline. +fn wait_for_port_closed(host: &str, port: u16, secs: u64) -> bool { + use std::net::TcpStream; + let addr: std::net::SocketAddr = match format!("{host}:{port}").parse() { + Ok(a) => a, + Err(_) => return true, + }; + let deadline = Instant::now() + Duration::from_secs(secs); + while Instant::now() < deadline { + if TcpStream::connect_timeout(&addr, Duration::from_millis(500)).is_err() { + return true; + } + std::thread::sleep(Duration::from_millis(300)); + } + false +} + +/// Spawn the Python backend, wait for its server, then navigate the window to it. +fn boot_backend(app: tauri::AppHandle) { + let repo = repo_root(); + let python = python_exe(&repo); + + let mut args: Vec = vec!["launch_gently.py".into(), "--no-browser".into()]; + if let Ok(extra) = std::env::var("GENTLY_LAUNCH_ARGS") { + args.extend(extra.split_whitespace().map(str::to_string)); + } + + set_status(&app, "Starting the backend…"); + eprintln!("[gently-desktop] spawning: {} {:?} (cwd={})", python.display(), args, repo.display()); + + let mut cmd = Command::new(&python); + cmd.args(&args).current_dir(&repo); + // Release: run the console-subsystem Python backend WITHOUT a console window + // (a GUI parent would otherwise pop one). Debug (`tauri dev`) inherits the dev + // console so backend logs stay visible while developing. + #[cfg(all(windows, not(debug_assertions)))] + { + use std::os::windows::process::CommandExt; + const CREATE_NO_WINDOW: u32 = 0x0800_0000; + cmd.creation_flags(CREATE_NO_WINDOW); + } + + let child = match cmd.spawn() { + Ok(c) => c, + Err(e) => { + set_status(&app, &format!("Failed to start backend: {e}")); + eprintln!("[gently-desktop] spawn failed: {e}"); + return; + } + }; + + // Put the child in our kill-on-close job so its tree can't outlive us. + #[cfg(windows)] + { + let b = app.state::(); + if let Err(e) = jobkill::assign(&b.job, &child) { + eprintln!("[gently-desktop] job assign failed (continuing): {e:?}"); + } + } + *app.state::().child.lock().unwrap() = Some(child); + + let port = viz_port(); + set_status(&app, "Waiting for the server…"); + if wait_for_port("127.0.0.1", port, 120) { + let url = format!("http://localhost:{port}"); + eprintln!("[gently-desktop] backend up — navigating to {url}"); + if let Some(win) = app.get_webview_window("main") { + match url.parse() { + Ok(u) => { let _ = win.navigate(u); } + Err(e) => set_status(&app, &format!("Bad backend URL: {e}")), + } + } + } else { + set_status(&app, "Backend did not become ready in time. Check the terminal log."); + eprintln!("[gently-desktop] timed out waiting for 127.0.0.1:{port}"); + } +} + +/// Repo root: `GENTLY_HOME` override, else the compile-time project root +/// (desktop/src-tauri/../..). The compile-time path is correct for `tauri dev` +/// and for a build run on this machine; a redistributable build should set +/// `GENTLY_HOME` (or bundle Python — see the module header). +fn repo_root() -> PathBuf { + if let Ok(p) = std::env::var("GENTLY_HOME") { + return PathBuf::from(p); + } + let manifest = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let root = manifest.join("..").join(".."); + std::fs::canonicalize(&root).unwrap_or(root) +} + +/// Python interpreter: `GENTLY_PYTHON` override, else the repo venv, else PATH. +fn python_exe(repo: &Path) -> PathBuf { + if let Ok(p) = std::env::var("GENTLY_PYTHON") { + return PathBuf::from(p); + } + let venv = if cfg!(windows) { + repo.join(".venv").join("Scripts").join("python.exe") + } else { + repo.join(".venv").join("bin").join("python") + }; + if venv.exists() { + venv + } else { + PathBuf::from("python") + } +} + +/// Viz port — mirrors the backend's `VIZ_PORT` env (default 8080). +fn viz_port() -> u16 { + std::env::var("VIZ_PORT") + .ok() + .and_then(|s| s.parse().ok()) + .unwrap_or(8080) +} + +/// Poll a TCP port until something accepts, or `secs` elapse. +fn wait_for_port(host: &str, port: u16, secs: u64) -> bool { + use std::net::TcpStream; + let addr: std::net::SocketAddr = match format!("{host}:{port}").parse() { + Ok(a) => a, + Err(_) => return false, + }; + let deadline = Instant::now() + Duration::from_secs(secs); + while Instant::now() < deadline { + if TcpStream::connect_timeout(&addr, Duration::from_millis(800)).is_ok() { + return true; + } + std::thread::sleep(Duration::from_millis(700)); + } + false +} + +/// Update the splash status line via the JS hook it exposes. +fn set_status(app: &tauri::AppHandle, msg: &str) { + if let Some(win) = app.get_webview_window("main") { + let safe = msg.replace('\\', "\\\\").replace('\'', "\\'"); + let js = format!("window.__gentlyStatus && window.__gentlyStatus('{safe}')"); + let _ = win.eval(js.as_str()); + } +} + +/// Windows Job Object helper — kill-on-close ownership of the backend tree. +#[cfg(windows)] +mod jobkill { + use std::ffi::c_void; + use std::os::windows::io::AsRawHandle; + use std::process::Child; + + use windows::Win32::Foundation::{CloseHandle, HANDLE}; + use windows::Win32::System::JobObjects::{ + AssignProcessToJobObject, CreateJobObjectW, SetInformationJobObject, + JobObjectExtendedLimitInformation, JOBOBJECT_EXTENDED_LIMIT_INFORMATION, + JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, + }; + + /// Owns a job HANDLE. `Send`/`Sync` so it can live in Tauri managed state; + /// the handle is only touched from create/assign/close. + pub struct Job(pub HANDLE); + unsafe impl Send for Job {} + unsafe impl Sync for Job {} + + pub fn create_kill_on_close() -> windows::core::Result { + unsafe { + let job = CreateJobObjectW(None, None)?; + let mut info = JOBOBJECT_EXTENDED_LIMIT_INFORMATION::default(); + info.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE; + SetInformationJobObject( + job, + JobObjectExtendedLimitInformation, + &info as *const _ as *const c_void, + std::mem::size_of::() as u32, + )?; + Ok(Job(job)) + } + } + + /// Assign a freshly-spawned child to the job. Its own children inherit the + /// job (Windows default), so grandchildren are covered too. + pub fn assign(job: &Job, child: &Child) -> windows::core::Result<()> { + unsafe { AssignProcessToJobObject(job.0, HANDLE(child.as_raw_handle())) } + } + + /// Close the job handle. With kill-on-close, this terminates every process + /// still in the job. + pub fn close(job: &Job) { + unsafe { + let _ = CloseHandle(job.0); + } + } +} diff --git a/desktop/src-tauri/tauri.conf.json b/desktop/src-tauri/tauri.conf.json new file mode 100644 index 00000000..d7dc638e --- /dev/null +++ b/desktop/src-tauri/tauri.conf.json @@ -0,0 +1,38 @@ +{ + "$schema": "../gen/schemas/desktop-schema.json", + "productName": "Gently", + "version": "0.1.0", + "identifier": "org.janelia.gently.desktop", + "build": { + "frontendDist": "../splash" + }, + "app": { + "windows": [ + { + "label": "main", + "title": "Gently", + "width": 1440, + "height": 920, + "minWidth": 900, + "minHeight": 600, + "resizable": true, + "visible": true, + "center": true + } + ], + "security": { + "csp": null + } + }, + "bundle": { + "active": true, + "targets": ["nsis"], + "icon": [ + "icons/32x32.png", + "icons/128x128.png", + "icons/128x128@2x.png", + "icons/icon.icns", + "icons/icon.ico" + ] + } +} diff --git a/diagnostics/benchmark_gentlystore_fps.py b/diagnostics/benchmark_gentlystore_fps.py index 43187b9e..2621e2bb 100644 --- a/diagnostics/benchmark_gentlystore_fps.py +++ b/diagnostics/benchmark_gentlystore_fps.py @@ -22,23 +22,22 @@ import argparse import shutil import statistics + +# Add gently to path +import sys import tempfile import time from dataclasses import dataclass, field from datetime import datetime from pathlib import Path -from typing import List, Optional import numpy as np -# Add gently to path -import sys GENTLY_ROOT = Path(__file__).resolve().parent.parent if str(GENTLY_ROOT) not in sys.path: sys.path.insert(0, str(GENTLY_ROOT)) -from gently.core.file_store import FileStore - +from gently.core.file_store import FileStore # noqa: E402 # --------------------------------------------------------------------------- # Default parameters -- typical diSPIM volume dimensions @@ -60,9 +59,9 @@ class BenchmarkResult: approach: str num_slices: int volume_shape: tuple - timings: List[float] = field(default_factory=list) - sizes_mb: List[float] = field(default_factory=list) - errors: List[str] = field(default_factory=list) + timings: list[float] = field(default_factory=list) + sizes_mb: list[float] = field(default_factory=list) + errors: list[str] = field(default_factory=list) @property def mean(self) -> float: @@ -130,7 +129,7 @@ def generate_synthetic_volume( z = np.linspace(0, 1, num_slices)[:, None, None] y = np.linspace(0, 1, height)[None, :, None] x = np.linspace(0, 1, width)[None, None, :] - vol = (z * 0.3 + y * 0.3 + x * 0.4) + vol = z * 0.3 + y * 0.3 + x * 0.4 if np.issubdtype(dtype, np.integer): info = np.iinfo(dtype) vol = (vol * info.max).astype(dtype) @@ -149,14 +148,18 @@ def generate_synthetic_volume( cx = np.random.randint(width // 4, 3 * width // 4) sz, sy, sx = 3, 15, 15 - z_idx = np.clip(np.arange(cz - sz*2, cz + sz*2), 0, num_slices - 1) - y_idx = np.clip(np.arange(cy - sy*2, cy + sy*2), 0, height - 1) - x_idx = np.clip(np.arange(cx - sx*2, cx + sx*2), 0, width - 1) + z_idx = np.clip(np.arange(cz - sz * 2, cz + sz * 2), 0, num_slices - 1) + y_idx = np.clip(np.arange(cy - sy * 2, cy + sy * 2), 0, height - 1) + x_idx = np.clip(np.arange(cx - sx * 2, cx + sx * 2), 0, width - 1) - zz, yy, xx = np.meshgrid(z_idx, y_idx, x_idx, indexing='ij') + zz, yy, xx = np.meshgrid(z_idx, y_idx, x_idx, indexing="ij") d2 = ((zz - cz) / sz) ** 2 + ((yy - cy) / sy) ** 2 + ((xx - cx) / sx) ** 2 blob = np.exp(-d2 / 2) - vol[z_idx[0]:z_idx[-1]+1, y_idx[0]:y_idx[-1]+1, x_idx[0]:x_idx[-1]+1] += blob + vol[ + z_idx[0] : z_idx[-1] + 1, + y_idx[0] : y_idx[-1] + 1, + x_idx[0] : x_idx[-1] + 1, + ] += blob # Add background noise vol += np.random.random(shape) * 0.1 @@ -178,7 +181,7 @@ def generate_synthetic_volume( def benchmark_raw_tiff_write( volume: np.ndarray, output_dir: Path, - compression: Optional[str] = "zlib", + compression: str | None = "zlib", ) -> tuple[float, float]: """ Benchmark raw tifffile write (no FileStore). @@ -255,7 +258,7 @@ def benchmark_register_volume( # Main benchmark sweep # --------------------------------------------------------------------------- def run_benchmark_sweep( - slices_list: List[int], + slices_list: list[int], width: int, height: int, num_repeats: int, @@ -265,9 +268,9 @@ def run_benchmark_sweep( run_put_volume: bool = True, run_register: bool = True, skip_projection: bool = False, -) -> List[BenchmarkResult]: +) -> list[BenchmarkResult]: """Run the full benchmark sweep.""" - results: List[BenchmarkResult] = [] + results: list[BenchmarkResult] = [] # Create temporary directory for benchmark temp_dir = Path(tempfile.mkdtemp(prefix="gently_benchmark_")) @@ -291,10 +294,12 @@ def run_benchmark_sweep( timepoint = 0 for config_idx, num_slices in enumerate(slices_list): - print(f"\n{'='*60}") - print(f"Config {config_idx + 1}/{total_configs}: " - f"slices={num_slices}, shape=({num_slices}, {height}, {width})") - print(f"{'='*60}") + print(f"\n{'=' * 60}") + print( + f"Config {config_idx + 1}/{total_configs}: " + f"slices={num_slices}, shape=({num_slices}, {height}, {width})" + ) + print(f"{'=' * 60}") volume_shape = (num_slices, height, width) @@ -307,22 +312,22 @@ def run_benchmark_sweep( # --- Raw TIFF write (baseline) --- if run_raw: res_raw = BenchmarkResult("raw_tiff_zlib", num_slices, volume_shape) - print(f"\n[Raw TIFF zlib]") + print("\n[Raw TIFF zlib]") # Warmup for w in range(num_warmup): - print(f" Warmup {w+1}/{num_warmup}...", end=" ", flush=True) + print(f" Warmup {w + 1}/{num_warmup}...", end=" ", flush=True) dur, size = benchmark_raw_tiff_write(volume, raw_dir, "zlib") print(f"{dur:.3f}s, {size:.1f} MB") # Timed repeats for r in range(num_repeats): - print(f" Repeat {r+1}/{num_repeats}...", end=" ", flush=True) + print(f" Repeat {r + 1}/{num_repeats}...", end=" ", flush=True) try: dur, size = benchmark_raw_tiff_write(volume, raw_dir, "zlib") res_raw.timings.append(dur) res_raw.sizes_mb.append(size) - print(f"{dur:.3f}s, {size:.1f} MB ({1/dur:.1f} vol/s)") + print(f"{dur:.3f}s, {size:.1f} MB ({1 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") res_raw.errors.append(str(e)) @@ -333,26 +338,38 @@ def run_benchmark_sweep( # --- put_volume (full pipeline) --- if run_put_volume: res_put = BenchmarkResult("put_volume", num_slices, volume_shape) - print(f"\n[FileStore.put_volume]") + print("\n[FileStore.put_volume]") # Warmup for w in range(num_warmup): embryo_id = f"embryo_{w % NUM_EMBRYOS}" - print(f" Warmup {w+1}/{num_warmup} ({embryo_id})...", end=" ", flush=True) - dur, size = benchmark_put_volume(store, session_id, embryo_id, timepoint, volume) + print( + f" Warmup {w + 1}/{num_warmup} ({embryo_id})...", + end=" ", + flush=True, + ) + dur, size = benchmark_put_volume( + store, session_id, embryo_id, timepoint, volume + ) timepoint += 1 print(f"{dur:.3f}s, {size:.1f} MB") # Timed repeats for r in range(num_repeats): embryo_id = f"embryo_{r % NUM_EMBRYOS}" - print(f" Repeat {r+1}/{num_repeats} ({embryo_id})...", end=" ", flush=True) + print( + f" Repeat {r + 1}/{num_repeats} ({embryo_id})...", + end=" ", + flush=True, + ) try: - dur, size = benchmark_put_volume(store, session_id, embryo_id, timepoint, volume) + dur, size = benchmark_put_volume( + store, session_id, embryo_id, timepoint, volume + ) timepoint += 1 res_put.timings.append(dur) res_put.sizes_mb.append(size) - print(f"{dur:.3f}s, {size:.1f} MB ({1/dur:.1f} vol/s)") + print(f"{dur:.3f}s, {size:.1f} MB ({1 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") res_put.errors.append(str(e)) @@ -363,26 +380,38 @@ def run_benchmark_sweep( # --- register_volume (zero-copy path) --- if run_register: res_reg = BenchmarkResult("register_volume", num_slices, volume_shape) - print(f"\n[FileStore.register_volume]") + print("\n[FileStore.register_volume]") # Warmup for w in range(num_warmup): embryo_id = f"embryo_{w % NUM_EMBRYOS}" - print(f" Warmup {w+1}/{num_warmup} ({embryo_id})...", end=" ", flush=True) - dur, size = benchmark_register_volume(store, session_id, embryo_id, timepoint, volume) + print( + f" Warmup {w + 1}/{num_warmup} ({embryo_id})...", + end=" ", + flush=True, + ) + dur, size = benchmark_register_volume( + store, session_id, embryo_id, timepoint, volume + ) timepoint += 1 print(f"{dur:.3f}s, {size:.1f} MB") # Timed repeats for r in range(num_repeats): embryo_id = f"embryo_{r % NUM_EMBRYOS}" - print(f" Repeat {r+1}/{num_repeats} ({embryo_id})...", end=" ", flush=True) + print( + f" Repeat {r + 1}/{num_repeats} ({embryo_id})...", + end=" ", + flush=True, + ) try: - dur, size = benchmark_register_volume(store, session_id, embryo_id, timepoint, volume) + dur, size = benchmark_register_volume( + store, session_id, embryo_id, timepoint, volume + ) timepoint += 1 res_reg.timings.append(dur) res_reg.sizes_mb.append(size) - print(f"{dur:.3f}s, {size:.1f} MB ({1/dur:.1f} vol/s)") + print(f"{dur:.3f}s, {size:.1f} MB ({1 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") res_reg.errors.append(str(e)) @@ -410,20 +439,24 @@ def _print_single_result(res: BenchmarkResult): if not res.timings: print(f" -> {res.approach}: NO SUCCESSFUL RUNS ({len(res.errors)} errors)") return - print(f" -> {res.approach}: {res.vol_per_sec:.2f} vol/s, " - f"mean={res.mean:.3f}s, std={res.std:.3f}s, " - f"avg_size={res.avg_size_mb:.1f} MB") + print( + f" -> {res.approach}: {res.vol_per_sec:.2f} vol/s, " + f"mean={res.mean:.3f}s, std={res.std:.3f}s, " + f"avg_size={res.avg_size_mb:.1f} MB" + ) -def print_results_table(results: List[BenchmarkResult]): +def print_results_table(results: list[BenchmarkResult]): """Print formatted ASCII results table.""" if not results: print("No results to display.") return - header = (f"{'Slices':>6} | {'Approach':>18} | {'Vol/s':>7} | " - f"{'Mean(s)':>7} | {'Std(s)':>6} | {'Min(s)':>6} | " - f"{'Max(s)':>6} | {'Size(MB)':>8} | {'MB/s':>7}") + header = ( + f"{'Slices':>6} | {'Approach':>18} | {'Vol/s':>7} | " + f"{'Mean(s)':>7} | {'Std(s)':>6} | {'Min(s)':>6} | " + f"{'Max(s)':>6} | {'Size(MB)':>8} | {'MB/s':>7}" + ) sep = "-" * len(header) print(f"\n{sep}") @@ -434,29 +467,35 @@ def print_results_table(results: List[BenchmarkResult]): for r in results: if r.timings: - print(f"{r.num_slices:>6} | {r.approach:>18} | " - f"{r.vol_per_sec:>7.2f} | {r.mean:>7.3f} | {r.std:>6.3f} | " - f"{r.min_t:>6.3f} | {r.max_t:>6.3f} | {r.avg_size_mb:>8.1f} | " - f"{r.mb_per_sec:>7.1f}") + print( + f"{r.num_slices:>6} | {r.approach:>18} | " + f"{r.vol_per_sec:>7.2f} | {r.mean:>7.3f} | {r.std:>6.3f} | " + f"{r.min_t:>6.3f} | {r.max_t:>6.3f} | {r.avg_size_mb:>8.1f} | " + f"{r.mb_per_sec:>7.1f}" + ) else: - print(f"{r.num_slices:>6} | {r.approach:>18} | " - f"{'FAIL':>7} | {'---':>7} | {'---':>6} | " - f"{'---':>6} | {'---':>6} | {'---':>8} | {'---':>7}") + print( + f"{r.num_slices:>6} | {r.approach:>18} | " + f"{'FAIL':>7} | {'---':>7} | {'---':>6} | " + f"{'---':>6} | {'---':>6} | {'---':>8} | {'---':>7}" + ) print(sep) -def print_overhead_analysis(results: List[BenchmarkResult]): +def print_overhead_analysis(results: list[BenchmarkResult]): """Print overhead analysis comparing FileStore to raw TIFF.""" from collections import defaultdict - groups = defaultdict(dict) + groups: dict = defaultdict(dict) for r in results: groups[r.num_slices][r.approach] = r - has_data = [(k, v) for k, v in groups.items() - if "raw_tiff_zlib" in v and - ("put_volume" in v or "register_volume" in v)] + has_data = [ + (k, v) + for k, v in groups.items() + if "raw_tiff_zlib" in v and ("put_volume" in v or "register_volume" in v) + ] if not has_data: return @@ -487,7 +526,7 @@ def print_overhead_analysis(results: List[BenchmarkResult]): print(sep) -def save_results_csv(results: List[BenchmarkResult], path: Path, run_params: dict): +def save_results_csv(results: list[BenchmarkResult], path: Path, run_params: dict): """Save results to CSV.""" import csv import json @@ -508,30 +547,44 @@ def save_results_csv(results: List[BenchmarkResult], path: Path, run_params: dic writer.writerow([]) # Summary table - writer.writerow([ - "slices", "approach", "vol_per_sec", "mean_s", "std_s", - "min_s", "max_s", "avg_size_mb", "mb_per_sec", "num_repeats", "errors", - ]) + writer.writerow( + [ + "slices", + "approach", + "vol_per_sec", + "mean_s", + "std_s", + "min_s", + "max_s", + "avg_size_mb", + "mb_per_sec", + "num_repeats", + "errors", + ] + ) for r in results: - writer.writerow([ - r.num_slices, r.approach, - f"{r.vol_per_sec:.4f}" if r.timings else "", - f"{r.mean:.6f}" if r.timings else "", - f"{r.std:.6f}" if r.timings else "", - f"{r.min_t:.6f}" if r.timings else "", - f"{r.max_t:.6f}" if r.timings else "", - f"{r.avg_size_mb:.2f}" if r.sizes_mb else "", - f"{r.mb_per_sec:.2f}" if r.timings else "", - len(r.timings), - "; ".join(r.errors) if r.errors else "", - ]) + writer.writerow( + [ + r.num_slices, + r.approach, + f"{r.vol_per_sec:.4f}" if r.timings else "", + f"{r.mean:.6f}" if r.timings else "", + f"{r.std:.6f}" if r.timings else "", + f"{r.min_t:.6f}" if r.timings else "", + f"{r.max_t:.6f}" if r.timings else "", + f"{r.avg_size_mb:.2f}" if r.sizes_mb else "", + f"{r.mb_per_sec:.2f}" if r.timings else "", + len(r.timings), + "; ".join(r.errors) if r.errors else "", + ] + ) # Per-volume timings writer.writerow([]) writer.writerow(["# Per-volume timings"]) writer.writerow(["slices", "approach", "repeat", "elapsed_s", "size_mb"]) for r in results: - for i, (t, s) in enumerate(zip(r.timings, r.sizes_mb)): + for i, (t, s) in enumerate(zip(r.timings, r.sizes_mb, strict=False)): writer.writerow([r.num_slices, r.approach, i + 1, f"{t:.6f}", f"{s:.2f}"]) print(f"\nResults saved to: {path}") @@ -541,24 +594,45 @@ def save_results_csv(results: List[BenchmarkResult], path: Path, run_params: dic # Main # --------------------------------------------------------------------------- def main(): - parser = argparse.ArgumentParser( - description="Benchmark FileStore volume storage throughput" + parser = argparse.ArgumentParser(description="Benchmark FileStore volume storage throughput") + parser.add_argument( + "--slices", + type=int, + nargs="+", + default=DEFAULT_SLICES, + help=f"Slice counts to test (default: {DEFAULT_SLICES})", + ) + parser.add_argument( + "--width", + type=int, + default=DEFAULT_WIDTH, + help=f"Image width (default: {DEFAULT_WIDTH})", + ) + parser.add_argument( + "--height", + type=int, + default=DEFAULT_HEIGHT, + help=f"Image height (default: {DEFAULT_HEIGHT})", + ) + parser.add_argument( + "--repeats", + type=int, + default=NUM_REPEATS, + help=f"Number of timed repeats (default: {NUM_REPEATS})", + ) + parser.add_argument( + "--warmup", + type=int, + default=NUM_WARMUP, + help=f"Number of warmup runs (default: {NUM_WARMUP})", + ) + parser.add_argument( + "--pattern", + choices=["noise", "gradient", "embryo"], + default="embryo", + help="Volume pattern: noise, gradient, or embryo (default: embryo)", ) - parser.add_argument("--slices", type=int, nargs="+", default=DEFAULT_SLICES, - help=f"Slice counts to test (default: {DEFAULT_SLICES})") - parser.add_argument("--width", type=int, default=DEFAULT_WIDTH, - help=f"Image width (default: {DEFAULT_WIDTH})") - parser.add_argument("--height", type=int, default=DEFAULT_HEIGHT, - help=f"Image height (default: {DEFAULT_HEIGHT})") - parser.add_argument("--repeats", type=int, default=NUM_REPEATS, - help=f"Number of timed repeats (default: {NUM_REPEATS})") - parser.add_argument("--warmup", type=int, default=NUM_WARMUP, - help=f"Number of warmup runs (default: {NUM_WARMUP})") - parser.add_argument("--pattern", choices=["noise", "gradient", "embryo"], - default="embryo", - help="Volume pattern: noise, gradient, or embryo (default: embryo)") - parser.add_argument("--save", action="store_true", - help="Save results to CSV") + parser.add_argument("--save", action="store_true", help="Save results to CSV") args = parser.parse_args() print("FileStore Volume Storage Benchmark") @@ -566,7 +640,7 @@ def main(): print(f" Dimensions: {args.width} x {args.height}") print(f" Pattern: {args.pattern}") print(f" Repeats: {args.repeats} (+ {args.warmup} warmup)") - print(f" Approaches: raw_tiff_zlib / put_volume / register_volume") + print(" Approaches: raw_tiff_zlib / put_volume / register_volume") results = run_benchmark_sweep( slices_list=args.slices, @@ -581,7 +655,10 @@ def main(): print_overhead_analysis(results) if args.save: - csv_path = Path("results") / f"benchmark_gentlystore_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv" + csv_path = ( + Path("results") + / f"benchmark_gentlystore_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv" + ) run_params = { "slices": args.slices, "width": args.width, diff --git a/diagnostics/benchmark_volume_fps.py b/diagnostics/benchmark_volume_fps.py index ff7b2e38..9c070c88 100644 --- a/diagnostics/benchmark_volume_fps.py +++ b/diagnostics/benchmark_volume_fps.py @@ -15,20 +15,21 @@ """ import os +import statistics import sys import time -import statistics -from pathlib import Path from dataclasses import dataclass, field -from typing import List, Optional, Tuple from datetime import datetime +from pathlib import Path -import yaml import numpy as np import pymmcore +import yaml # Add dispim-control to path for ophyd device imports -DISPIM_CONTROL_DIR = Path(__file__).resolve().parent.parent / "UsersdispimDocumentsGitHubdispim-control" +DISPIM_CONTROL_DIR = ( + Path(__file__).resolve().parent.parent / "UsersdispimDocumentsGitHubdispim-control" +) if str(DISPIM_CONTROL_DIR) not in sys.path: sys.path.insert(0, str(DISPIM_CONTROL_DIR)) @@ -41,10 +42,10 @@ PIEZO_DEVICE = "PiezoStage:P:34" # Default scan parameters (same as run_multi_embryo_volumes.py) -DEFAULT_GALVO_AMPLITUDE = 0.5 # degrees -DEFAULT_GALVO_CENTER = 0.0 # degrees -DEFAULT_PIEZO_AMPLITUDE = 25.0 # um -DEFAULT_PIEZO_CENTER = 50.0 # um +DEFAULT_GALVO_AMPLITUDE = 0.5 # degrees +DEFAULT_GALVO_CENTER = 0.0 # degrees +DEFAULT_PIEZO_AMPLITUDE = 25.0 # um +DEFAULT_PIEZO_CENTER = 50.0 # um DEFAULT_LASER_CONFIG = "488 and 561" DEFAULT_CAMERA_ROI = (128, 896, 2048, 512) # (x, y, width, height) @@ -69,10 +70,30 @@ # Simulated embryo calibration profiles for round-robin reconfig test. # Each entry represents a different embryo with distinct galvo/piezo settings. EMBRYO_PROFILES = [ - {"galvo_amplitude": 0.50, "galvo_center": 0.00, "piezo_amplitude": 25.0, "piezo_center": 50.0}, - {"galvo_amplitude": 0.45, "galvo_center": 0.12, "piezo_amplitude": 22.5, "piezo_center": 55.0}, - {"galvo_amplitude": 0.55, "galvo_center": -0.08, "piezo_amplitude": 27.5, "piezo_center": 45.0}, - {"galvo_amplitude": 0.48, "galvo_center": 0.05, "piezo_amplitude": 24.0, "piezo_center": 52.0}, + { + "galvo_amplitude": 0.50, + "galvo_center": 0.00, + "piezo_amplitude": 25.0, + "piezo_center": 50.0, + }, + { + "galvo_amplitude": 0.45, + "galvo_center": 0.12, + "piezo_amplitude": 22.5, + "piezo_center": 55.0, + }, + { + "galvo_amplitude": 0.55, + "galvo_center": -0.08, + "piezo_amplitude": 27.5, + "piezo_center": 45.0, + }, + { + "galvo_amplitude": 0.48, + "galvo_center": 0.05, + "piezo_amplitude": 24.0, + "piezo_center": 52.0, + }, ] @@ -84,9 +105,9 @@ class BenchmarkResult: approach: str num_slices: int exposure_ms: float - timings: List[float] = field(default_factory=list) - image_counts: List[int] = field(default_factory=list) - errors: List[str] = field(default_factory=list) + timings: list[float] = field(default_factory=list) + image_counts: list[int] = field(default_factory=list) + errors: list[str] = field(default_factory=list) @property def mean(self) -> float: @@ -116,13 +137,13 @@ def total_images(self) -> int: # --------------------------------------------------------------------------- # Config loading # --------------------------------------------------------------------------- -def load_config(path: str) -> Tuple[str, str]: +def load_config(path: str) -> tuple[str, str]: """Read config.yml and return (mm_dir, config_file).""" cfg_path = Path(path) if not cfg_path.exists(): raise FileNotFoundError(f"Config file not found: {cfg_path}") - with open(cfg_path, "r") as f: + with open(cfg_path) as f: cfg = yaml.safe_load(f) mm_dir = cfg["mmdirectory"] @@ -236,8 +257,8 @@ def configure_hardware_raw(core: pymmcore.CMMCore, num_slices: int, exposure_ms: def acquire_volume_raw( core: pymmcore.CMMCore, num_slices: int, - save_dir: Optional[Path] = None, -) -> Tuple[int, float]: + save_dir: Path | None = None, +) -> tuple[int, float]: """ Trigger SPIM and collect images from the circular buffer. @@ -260,7 +281,7 @@ def acquire_volume_raw( t0 = time.perf_counter() core.setProperty(GALVO_DEVICE, "SPIMState", "Running") - images = [] if save_dir else None + images: list | None = [] if save_dir else None count = 0 timeout_s = max(num_slices * 0.05 * 2, 10.0) # generous timeout t_start = time.time() @@ -329,10 +350,10 @@ def create_ophyd_devices(core: pymmcore.CMMCore): which imports a theme module that may not be available). """ from dispim_control.devices import ( - DiSPIMScanner, DiSPIMCamera, - DiSPIMPiezo, DiSPIMLaserControl, + DiSPIMPiezo, + DiSPIMScanner, DiSPIMVolumeScanner, ) @@ -356,8 +377,8 @@ def acquire_volume_ophyd( volume_scanner, num_slices: int, exposure_ms: float, - save_dir: Optional[Path] = None, -) -> Tuple[int, float]: + save_dir: Path | None = None, +) -> tuple[int, float]: """ Configure + trigger via ophyd VolumeScanner. @@ -392,8 +413,9 @@ def acquire_volume_ophyd( # --------------------------------------------------------------------------- # Ophyd burst approach -- configure once, skip per-volume reset # --------------------------------------------------------------------------- -def configure_ophyd_burst(volume_scanner, num_slices: int, exposure_ms: float, - core: pymmcore.CMMCore): +def configure_ophyd_burst( + volume_scanner, num_slices: int, exposure_ms: float, core: pymmcore.CMMCore +): """ Configure ophyd devices once for a burst of volumes. @@ -417,8 +439,9 @@ def configure_ophyd_burst(volume_scanner, num_slices: int, exposure_ms: float, time.sleep(0.1) -def acquire_volume_ophyd_burst(volume_scanner, num_slices: int, - core: pymmcore.CMMCore) -> Tuple[int, float]: +def acquire_volume_ophyd_burst( + volume_scanner, num_slices: int, core: pymmcore.CMMCore +) -> tuple[int, float]: """ Acquire one volume using ophyd devices but without per-volume reset. @@ -494,8 +517,9 @@ def cleanup_ophyd_burst(volume_scanner, core: pymmcore.CMMCore): # --------------------------------------------------------------------------- # Burst reconfig approach -- reconfigure galvo/piezo per volume (round-robin) # --------------------------------------------------------------------------- -def configure_burst_reconfig(volume_scanner, num_slices: int, exposure_ms: float, - core: pymmcore.CMMCore): +def configure_burst_reconfig( + volume_scanner, num_slices: int, exposure_ms: float, core: pymmcore.CMMCore +): """ One-time setup for burst_reconfig: camera, scanner X-axis & timing, lasers. @@ -519,9 +543,11 @@ def configure_burst_reconfig(volume_scanner, num_slices: int, exposure_ms: float def acquire_volume_burst_reconfig( - volume_scanner, num_slices: int, profile_idx: int, + volume_scanner, + num_slices: int, + profile_idx: int, core: pymmcore.CMMCore, -) -> Tuple[int, float]: +) -> tuple[int, float]: """ Acquire one volume after reconfiguring galvo/piezo for a specific embryo. @@ -535,7 +561,6 @@ def acquire_volume_burst_reconfig( profile = EMBRYO_PROFILES[profile_idx % len(EMBRYO_PROFILES)] camera_name = volume_scanner.camera.name scanner = volume_scanner.scanner - piezo = volume_scanner.piezo core.clearCircularBuffer() @@ -548,16 +573,12 @@ def acquire_volume_burst_reconfig( t0 = time.perf_counter() # Reconfigure galvo Y-axis for this embryo - core.setProperty(GALVO_DEVICE, "SingleAxisYAmplitude(deg)", - float(profile["galvo_amplitude"])) - core.setProperty(GALVO_DEVICE, "SingleAxisYOffset(deg)", - float(profile["galvo_center"])) + core.setProperty(GALVO_DEVICE, "SingleAxisYAmplitude(deg)", float(profile["galvo_amplitude"])) + core.setProperty(GALVO_DEVICE, "SingleAxisYOffset(deg)", float(profile["galvo_center"])) # Reconfigure piezo for this embryo - core.setProperty(PIEZO_DEVICE, "SingleAxisAmplitude(um)", - float(profile["piezo_amplitude"])) - core.setProperty(PIEZO_DEVICE, "SingleAxisOffset(um)", - float(profile["piezo_center"])) + core.setProperty(PIEZO_DEVICE, "SingleAxisAmplitude(um)", float(profile["piezo_amplitude"])) + core.setProperty(PIEZO_DEVICE, "SingleAxisOffset(um)", float(profile["piezo_center"])) core.setProperty(PIEZO_DEVICE, "SPIMState", "Armed") time.sleep(0.3) @@ -595,9 +616,11 @@ def acquire_volume_burst_reconfig( # Burst reconfig with waitForDevice -- replaces time.sleep() with MMCore API # --------------------------------------------------------------------------- def acquire_volume_burst_reconfig_wfd( - volume_scanner, num_slices: int, profile_idx: int, + volume_scanner, + num_slices: int, + profile_idx: int, core: pymmcore.CMMCore, -) -> Tuple[int, float]: +) -> tuple[int, float]: """ Same as burst_reconfig but uses core.waitForDevice() instead of time.sleep() to wait for hardware readiness. @@ -608,7 +631,6 @@ def acquire_volume_burst_reconfig_wfd( """ profile = EMBRYO_PROFILES[profile_idx % len(EMBRYO_PROFILES)] camera_name = volume_scanner.camera.name - scanner = volume_scanner.scanner core.clearCircularBuffer() @@ -621,17 +643,13 @@ def acquire_volume_burst_reconfig_wfd( t0 = time.perf_counter() # Reconfigure galvo Y-axis for this embryo - core.setProperty(GALVO_DEVICE, "SingleAxisYAmplitude(deg)", - float(profile["galvo_amplitude"])) - core.setProperty(GALVO_DEVICE, "SingleAxisYOffset(deg)", - float(profile["galvo_center"])) + core.setProperty(GALVO_DEVICE, "SingleAxisYAmplitude(deg)", float(profile["galvo_amplitude"])) + core.setProperty(GALVO_DEVICE, "SingleAxisYOffset(deg)", float(profile["galvo_center"])) core.waitForDevice(GALVO_DEVICE) # Reconfigure piezo for this embryo - core.setProperty(PIEZO_DEVICE, "SingleAxisAmplitude(um)", - float(profile["piezo_amplitude"])) - core.setProperty(PIEZO_DEVICE, "SingleAxisOffset(um)", - float(profile["piezo_center"])) + core.setProperty(PIEZO_DEVICE, "SingleAxisAmplitude(um)", float(profile["piezo_amplitude"])) + core.setProperty(PIEZO_DEVICE, "SingleAxisOffset(um)", float(profile["piezo_center"])) core.setProperty(PIEZO_DEVICE, "SPIMState", "Armed") core.waitForDevice(PIEZO_DEVICE) @@ -686,16 +704,16 @@ def _save_volume(volume: np.ndarray, save_dir: Path, approach: str, num_slices: # --------------------------------------------------------------------------- def run_benchmark_sweep( core: pymmcore.CMMCore, - slices_list: List[int], - exposures_list: List[float], + slices_list: list[int], + exposures_list: list[float], num_repeats: int, num_warmup: int, run_raw: bool, run_ophyd: bool, - save_dir: Optional[Path] = None, -) -> List[BenchmarkResult]: + save_dir: Path | None = None, +) -> list[BenchmarkResult]: """Run the full parameter sweep and return results.""" - results: List[BenchmarkResult] = [] + results: list[BenchmarkResult] = [] volume_scanner = None if run_ophyd: @@ -709,15 +727,17 @@ def run_benchmark_sweep( for num_slices in slices_list: for exposure_ms in exposures_list: config_idx += 1 - print(f"\n{'='*60}") - print(f"Config {config_idx}/{total_configs}: " - f"slices={num_slices}, exposure={exposure_ms}ms") - print(f"{'='*60}") + print(f"\n{'=' * 60}") + print( + f"Config {config_idx}/{total_configs}: " + f"slices={num_slices}, exposure={exposure_ms}ms" + ) + print(f"{'=' * 60}") # --- Raw MMCore --- if run_raw: res_raw = BenchmarkResult("raw", num_slices, exposure_ms) - print(f"\n[Raw MMCore] Configuring hardware...") + print("\n[Raw MMCore] Configuring hardware...") try: configure_hardware_raw(core, num_slices, exposure_ms) except Exception as e: @@ -728,7 +748,7 @@ def run_benchmark_sweep( # Warm-up for w in range(num_warmup): - print(f" Warm-up {w+1}/{num_warmup}...", end=" ", flush=True) + print(f" Warm-up {w + 1}/{num_warmup}...", end=" ", flush=True) try: cnt, dur = acquire_volume_raw(core, num_slices) print(f"{cnt} imgs, {dur:.3f}s") @@ -737,15 +757,15 @@ def run_benchmark_sweep( # Timed repeats for r in range(num_repeats): - print(f" Repeat {r+1}/{num_repeats}...", end=" ", flush=True) + print(f" Repeat {r + 1}/{num_repeats}...", end=" ", flush=True) try: cnt, dur = acquire_volume_raw(core, num_slices, save_dir=save_dir) res_raw.timings.append(dur) res_raw.image_counts.append(cnt) - print(f"{cnt} imgs, {dur:.3f}s ({1.0/dur:.1f} vol/s)") + print(f"{cnt} imgs, {dur:.3f}s ({1.0 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") - res_raw.errors.append(f"repeat {r+1}: {e}") + res_raw.errors.append(f"repeat {r + 1}: {e}") # Cleanup after raw batch (lasers off) cleanup_raw(core) @@ -758,27 +778,34 @@ def run_benchmark_sweep( # Warm-up for w in range(num_warmup): - print(f" [Ophyd] Warm-up {w+1}/{num_warmup}...", end=" ", flush=True) + print( + f" [Ophyd] Warm-up {w + 1}/{num_warmup}...", + end=" ", + flush=True, + ) try: - cnt, dur = acquire_volume_ophyd( - volume_scanner, num_slices, exposure_ms) + cnt, dur = acquire_volume_ophyd(volume_scanner, num_slices, exposure_ms) print(f"{cnt} imgs, {dur:.3f}s") except Exception as e: print(f"ERROR: {e}") # Timed repeats for r in range(num_repeats): - print(f" [Ophyd] Repeat {r+1}/{num_repeats}...", end=" ", flush=True) + print( + f" [Ophyd] Repeat {r + 1}/{num_repeats}...", + end=" ", + flush=True, + ) try: cnt, dur = acquire_volume_ophyd( - volume_scanner, num_slices, exposure_ms, - save_dir=save_dir) + volume_scanner, num_slices, exposure_ms, save_dir=save_dir + ) res_ophyd.timings.append(dur) res_ophyd.image_counts.append(cnt) - print(f"{cnt} imgs, {dur:.3f}s ({1.0/dur:.1f} vol/s)") + print(f"{cnt} imgs, {dur:.3f}s ({1.0 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") - res_ophyd.errors.append(f"repeat {r+1}: {e}") + res_ophyd.errors.append(f"repeat {r + 1}: {e}") results.append(res_ophyd) _print_single_result(res_ophyd) @@ -786,10 +813,9 @@ def run_benchmark_sweep( # --- Ophyd burst (configure once, no per-volume reset) --- if run_ophyd and volume_scanner is not None: res_burst = BenchmarkResult("ophyd_burst", num_slices, exposure_ms) - print(f"\n[Ophyd Burst] Configuring once...") + print("\n[Ophyd Burst] Configuring once...") try: - configure_ophyd_burst(volume_scanner, num_slices, - exposure_ms, core) + configure_ophyd_burst(volume_scanner, num_slices, exposure_ms, core) except Exception as e: print(f" ERROR configuring ophyd burst: {e}") res_burst.errors.append(f"configure: {e}") @@ -798,26 +824,24 @@ def run_benchmark_sweep( # Warm-up for w in range(num_warmup): - print(f" Warm-up {w+1}/{num_warmup}...", end=" ", flush=True) + print(f" Warm-up {w + 1}/{num_warmup}...", end=" ", flush=True) try: - cnt, dur = acquire_volume_ophyd_burst( - volume_scanner, num_slices, core) + cnt, dur = acquire_volume_ophyd_burst(volume_scanner, num_slices, core) print(f"{cnt} imgs, {dur:.3f}s") except Exception as e: print(f"ERROR: {e}") # Timed repeats for r in range(num_repeats): - print(f" Repeat {r+1}/{num_repeats}...", end=" ", flush=True) + print(f" Repeat {r + 1}/{num_repeats}...", end=" ", flush=True) try: - cnt, dur = acquire_volume_ophyd_burst( - volume_scanner, num_slices, core) + cnt, dur = acquire_volume_ophyd_burst(volume_scanner, num_slices, core) res_burst.timings.append(dur) res_burst.image_counts.append(cnt) - print(f"{cnt} imgs, {dur:.3f}s ({1.0/dur:.1f} vol/s)") + print(f"{cnt} imgs, {dur:.3f}s ({1.0 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") - res_burst.errors.append(f"repeat {r+1}: {e}") + res_burst.errors.append(f"repeat {r + 1}: {e}") cleanup_ophyd_burst(volume_scanner, core) results.append(res_burst) @@ -826,11 +850,12 @@ def run_benchmark_sweep( # --- Burst reconfig (round-robin galvo/piezo per volume) --- if run_ophyd and volume_scanner is not None: res_reconfig = BenchmarkResult("burst_reconfig", num_slices, exposure_ms) - print(f"\n[Burst Reconfig] Configuring once, " - f"cycling {len(EMBRYO_PROFILES)} embryo profiles...") + print( + f"\n[Burst Reconfig] Configuring once, " + f"cycling {len(EMBRYO_PROFILES)} embryo profiles..." + ) try: - configure_burst_reconfig(volume_scanner, num_slices, - exposure_ms, core) + configure_burst_reconfig(volume_scanner, num_slices, exposure_ms, core) except Exception as e: print(f" ERROR configuring burst reconfig: {e}") res_reconfig.errors.append(f"configure: {e}") @@ -839,12 +864,15 @@ def run_benchmark_sweep( # Warm-up (cycle through profiles) for w in range(num_warmup): - print(f" Warm-up {w+1}/{num_warmup} " - f"(profile {w % len(EMBRYO_PROFILES)})...", - end=" ", flush=True) + print( + f" Warm-up {w + 1}/{num_warmup} (profile {w % len(EMBRYO_PROFILES)})...", + end=" ", + flush=True, + ) try: cnt, dur = acquire_volume_burst_reconfig( - volume_scanner, num_slices, w, core) + volume_scanner, num_slices, w, core + ) print(f"{cnt} imgs, {dur:.3f}s") except Exception as e: print(f"ERROR: {e}") @@ -852,17 +880,21 @@ def run_benchmark_sweep( # Timed repeats (cycle through profiles) for r in range(num_repeats): pidx = r % len(EMBRYO_PROFILES) - print(f" Repeat {r+1}/{num_repeats} " - f"(profile {pidx})...", end=" ", flush=True) + print( + f" Repeat {r + 1}/{num_repeats} (profile {pidx})...", + end=" ", + flush=True, + ) try: cnt, dur = acquire_volume_burst_reconfig( - volume_scanner, num_slices, r, core) + volume_scanner, num_slices, r, core + ) res_reconfig.timings.append(dur) res_reconfig.image_counts.append(cnt) - print(f"{cnt} imgs, {dur:.3f}s ({1.0/dur:.1f} vol/s)") + print(f"{cnt} imgs, {dur:.3f}s ({1.0 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") - res_reconfig.errors.append(f"repeat {r+1}: {e}") + res_reconfig.errors.append(f"repeat {r + 1}: {e}") cleanup_ophyd_burst(volume_scanner, core) results.append(res_reconfig) @@ -871,11 +903,12 @@ def run_benchmark_sweep( # --- Burst reconfig with waitForDevice (no time.sleep) --- if run_ophyd and volume_scanner is not None: res_wfd = BenchmarkResult("reconfig_wfd", num_slices, exposure_ms) - print(f"\n[Reconfig WFD] Configuring once, " - f"using waitForDevice() instead of time.sleep()...") + print( + "\n[Reconfig WFD] Configuring once, " + "using waitForDevice() instead of time.sleep()..." + ) try: - configure_burst_reconfig(volume_scanner, num_slices, - exposure_ms, core) + configure_burst_reconfig(volume_scanner, num_slices, exposure_ms, core) except Exception as e: print(f" ERROR configuring reconfig_wfd: {e}") res_wfd.errors.append(f"configure: {e}") @@ -884,12 +917,15 @@ def run_benchmark_sweep( # Warm-up (cycle through profiles) for w in range(num_warmup): - print(f" Warm-up {w+1}/{num_warmup} " - f"(profile {w % len(EMBRYO_PROFILES)})...", - end=" ", flush=True) + print( + f" Warm-up {w + 1}/{num_warmup} (profile {w % len(EMBRYO_PROFILES)})...", + end=" ", + flush=True, + ) try: cnt, dur = acquire_volume_burst_reconfig_wfd( - volume_scanner, num_slices, w, core) + volume_scanner, num_slices, w, core + ) print(f"{cnt} imgs, {dur:.3f}s") except Exception as e: print(f"ERROR: {e}") @@ -897,17 +933,21 @@ def run_benchmark_sweep( # Timed repeats (cycle through profiles) for r in range(num_repeats): pidx = r % len(EMBRYO_PROFILES) - print(f" Repeat {r+1}/{num_repeats} " - f"(profile {pidx})...", end=" ", flush=True) + print( + f" Repeat {r + 1}/{num_repeats} (profile {pidx})...", + end=" ", + flush=True, + ) try: cnt, dur = acquire_volume_burst_reconfig_wfd( - volume_scanner, num_slices, r, core) + volume_scanner, num_slices, r, core + ) res_wfd.timings.append(dur) res_wfd.image_counts.append(cnt) - print(f"{cnt} imgs, {dur:.3f}s ({1.0/dur:.1f} vol/s)") + print(f"{cnt} imgs, {dur:.3f}s ({1.0 / dur:.1f} vol/s)") except Exception as e: print(f"ERROR: {e}") - res_wfd.errors.append(f"repeat {r+1}: {e}") + res_wfd.errors.append(f"repeat {r + 1}: {e}") cleanup_ophyd_burst(volume_scanner, core) results.append(res_wfd) @@ -922,22 +962,25 @@ def run_benchmark_sweep( def _print_single_result(res: BenchmarkResult): """Print a single intermediate result.""" if not res.timings: - print(f" -> {res.approach}: NO SUCCESSFUL RUNS " - f"({len(res.errors)} errors)") + print(f" -> {res.approach}: NO SUCCESSFUL RUNS ({len(res.errors)} errors)") return - print(f" -> {res.approach}: {res.vol_per_sec:.2f} vol/s, " - f"mean={res.mean:.3f}s, std={res.std:.3f}s") + print( + f" -> {res.approach}: {res.vol_per_sec:.2f} vol/s, " + f"mean={res.mean:.3f}s, std={res.std:.3f}s" + ) -def print_results_table(results: List[BenchmarkResult]): +def print_results_table(results: list[BenchmarkResult]): """Print formatted ASCII results table.""" if not results: print("No results to display.") return - header = (f"{'Slices':>6} | {'Exp(ms)':>7} | {'Approach':>14} | " - f"{'Vol/s':>7} | {'Mean(s)':>7} | {'Std(s)':>7} | " - f"{'Min(s)':>7} | {'Max(s)':>7} | {'Images':>6}") + header = ( + f"{'Slices':>6} | {'Exp(ms)':>7} | {'Approach':>14} | " + f"{'Vol/s':>7} | {'Mean(s)':>7} | {'Std(s)':>7} | " + f"{'Min(s)':>7} | {'Max(s)':>7} | {'Images':>6}" + ) sep = "-" * len(header) print(f"\n{sep}") @@ -948,35 +991,43 @@ def print_results_table(results: List[BenchmarkResult]): for r in results: if r.timings: - print(f"{r.num_slices:>6} | {r.exposure_ms:>7.1f} | {r.approach:>14} | " - f"{r.vol_per_sec:>7.2f} | {r.mean:>7.3f} | {r.std:>7.3f} | " - f"{r.min_t:>7.3f} | {r.max_t:>7.3f} | {r.total_images:>6}") + print( + f"{r.num_slices:>6} | {r.exposure_ms:>7.1f} | {r.approach:>14} | " + f"{r.vol_per_sec:>7.2f} | {r.mean:>7.3f} | {r.std:>7.3f} | " + f"{r.min_t:>7.3f} | {r.max_t:>7.3f} | {r.total_images:>6}" + ) else: - print(f"{r.num_slices:>6} | {r.exposure_ms:>7.1f} | {r.approach:>14} | " - f"{'FAIL':>7} | {'---':>7} | {'---':>7} | " - f"{'---':>7} | {'---':>7} | {'---':>6}") + print( + f"{r.num_slices:>6} | {r.exposure_ms:>7.1f} | {r.approach:>14} | " + f"{'FAIL':>7} | {'---':>7} | {'---':>7} | " + f"{'---':>7} | {'---':>7} | {'---':>6}" + ) print(sep) -def print_summary(results: List[BenchmarkResult]): +def print_summary(results: list[BenchmarkResult]): """Print overhead analysis comparing ophyd and ophyd_burst vs raw.""" from collections import defaultdict - groups = defaultdict(dict) + + groups: dict = defaultdict(dict) for r in results: groups[(r.num_slices, r.exposure_ms)][r.approach] = r # Need at least raw + one ophyd variant - has_data = [(k, v) for k, v in groups.items() - if "raw" in v and ("ophyd" in v or "ophyd_burst" in v - or "burst_reconfig" in v - or "reconfig_wfd" in v)] + has_data = [ + (k, v) + for k, v in groups.items() + if "raw" in v + and ("ophyd" in v or "ophyd_burst" in v or "burst_reconfig" in v or "reconfig_wfd" in v) + ] if not has_data: return print() - header = (f"{'Slices':>6} | {'Exp(ms)':>7} | " - f"{'Approach':>14} | {'vs Raw(ms)':>10} | {'vs Raw(%)':>9}") + header = ( + f"{'Slices':>6} | {'Exp(ms)':>7} | {'Approach':>14} | {'vs Raw(ms)':>10} | {'vs Raw(%)':>9}" + ) sep = "-" * len(header) print(sep) @@ -995,14 +1046,15 @@ def print_summary(results: List[BenchmarkResult]): other_mean = approaches[label].mean overhead_ms = (other_mean - raw_mean) * 1000.0 overhead_pct = ((other_mean - raw_mean) / raw_mean) * 100.0 - print(f"{ns:>6} | {exp:>7.1f} | {label:>14} | " - f"{overhead_ms:>+10.1f} | {overhead_pct:>+8.1f}%") + print( + f"{ns:>6} | {exp:>7.1f} | {label:>14} | " + f"{overhead_ms:>+10.1f} | {overhead_pct:>+8.1f}%" + ) print(sep) -def save_results_csv(results: List[BenchmarkResult], path: Path, - run_params: dict): +def save_results_csv(results: list[BenchmarkResult], path: Path, run_params: dict): """Write benchmark results to a CSV file with full metadata.""" import csv import json @@ -1031,37 +1083,63 @@ def save_results_csv(results: List[BenchmarkResult], path: Path, writer.writerow([]) # --- Summary table --- - writer.writerow([ - "slices", "exposure_ms", "approach", - "vol_per_sec", "mean_s", "std_s", "min_s", "max_s", - "total_images", "num_repeats", "errors", - ]) + writer.writerow( + [ + "slices", + "exposure_ms", + "approach", + "vol_per_sec", + "mean_s", + "std_s", + "min_s", + "max_s", + "total_images", + "num_repeats", + "errors", + ] + ) for r in results: - writer.writerow([ - r.num_slices, r.exposure_ms, r.approach, - f"{r.vol_per_sec:.4f}" if r.timings else "", - f"{r.mean:.6f}" if r.timings else "", - f"{r.std:.6f}" if r.timings else "", - f"{r.min_t:.6f}" if r.timings else "", - f"{r.max_t:.6f}" if r.timings else "", - r.total_images, - len(r.timings), - "; ".join(r.errors) if r.errors else "", - ]) + writer.writerow( + [ + r.num_slices, + r.exposure_ms, + r.approach, + f"{r.vol_per_sec:.4f}" if r.timings else "", + f"{r.mean:.6f}" if r.timings else "", + f"{r.std:.6f}" if r.timings else "", + f"{r.min_t:.6f}" if r.timings else "", + f"{r.max_t:.6f}" if r.timings else "", + r.total_images, + len(r.timings), + "; ".join(r.errors) if r.errors else "", + ] + ) # --- Per-volume raw timings --- writer.writerow([]) writer.writerow(["# Per-volume timings (seconds)"]) - writer.writerow([ - "slices", "exposure_ms", "approach", - "repeat", "elapsed_s", "image_count", - ]) + writer.writerow( + [ + "slices", + "exposure_ms", + "approach", + "repeat", + "elapsed_s", + "image_count", + ] + ) for r in results: - for i, (t, cnt) in enumerate(zip(r.timings, r.image_counts)): - writer.writerow([ - r.num_slices, r.exposure_ms, r.approach, - i + 1, f"{t:.6f}", cnt, - ]) + for i, (t, cnt) in enumerate(zip(r.timings, r.image_counts, strict=False)): + writer.writerow( + [ + r.num_slices, + r.exposure_ms, + r.approach, + i + 1, + f"{t:.6f}", + cnt, + ] + ) print(f"\nResults saved to: {path}") @@ -1076,8 +1154,10 @@ def main(): print(f" Slices series: {SLICES_SERIES}") print(f" Exposure: {EXPOSURE_MS} ms") print(f" Repeats: {NUM_REPEATS} (+ {NUM_WARMUP} warmup)") - print(f" Approaches: raw / ophyd / ophyd_burst / burst_reconfig / reconfig_wfd") - print(f" Embryo profiles: {len(EMBRYO_PROFILES)} (for burst_reconfig & reconfig_wfd round-robin)") + print(" Approaches: raw / ophyd / ophyd_burst / burst_reconfig / reconfig_wfd") + print( + f" Embryo profiles: {len(EMBRYO_PROFILES)} (for burst_reconfig & reconfig_wfd round-robin)" + ) # Load config and initialize mm_dir, config_file = load_config(config_path) @@ -1099,7 +1179,9 @@ def main(): print_summary(results) # Save CSV - csv_path = Path("results") / f"benchmark_volume_fps_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv" + csv_path = ( + Path("results") / f"benchmark_volume_fps_{datetime.now().strftime('%Y%m%d_%H%M%S')}.csv" + ) run_params = { "slices": SLICES_SERIES, "exposure_ms": EXPOSURE_MS, diff --git a/diagnostics/measure_centering_error.py b/diagnostics/measure_centering_error.py index 0f159c79..c8c22ef4 100644 --- a/diagnostics/measure_centering_error.py +++ b/diagnostics/measure_centering_error.py @@ -6,8 +6,9 @@ """ import json -import numpy as np + import matplotlib.pyplot as plt +import numpy as np from PIL import Image # Load the image after moving to embryo @@ -21,6 +22,7 @@ # Store clicked position clicked_pos = [None, None] + def on_click(event): if event.xdata is not None and event.ydata is not None: clicked_pos[0] = event.xdata @@ -47,48 +49,56 @@ def on_click(event): "error_y_direction": "BELOW" if error_y > 0 else "ABOVE", "error_x_um": float(error_x_um), "error_y_um": float(error_y_um), - "um_per_pixel": um_per_pixel + "um_per_pixel": um_per_pixel, } # Write to file - with open(OUTPUT_FILE, 'w') as f: + with open(OUTPUT_FILE, "w") as f: json.dump(result, f, indent=2) - print(f"\n{'='*50}") + print(f"\n{'=' * 50}") print(f"CLICKED POSITION: ({clicked_pos[0]:.1f}, {clicked_pos[1]:.1f})") print(f"CENTER POSITION: ({CENTER_X}, {CENTER_Y})") - print(f"{'='*50}") + print(f"{'=' * 50}") print(f"ERROR X: {error_x:+.1f} pixels ({result['error_x_direction']} of center)") print(f"ERROR Y: {error_y:+.1f} pixels ({result['error_y_direction']} center)") - print(f"{'='*50}") + print(f"{'=' * 50}") print(f"ERROR X: {error_x_um:+.1f} um") print(f"ERROR Y: {error_y_um:+.1f} um") - print(f"{'='*50}") + print(f"{'=' * 50}") print(f"\nSaved to: {OUTPUT_FILE}") # Update the plot with clicked marker - ax.plot(clicked_pos[0], clicked_pos[1], 'go', markersize=15, markeredgewidth=3, - markerfacecolor='none', label='Actual embryo position') + ax.plot( + clicked_pos[0], + clicked_pos[1], + "go", + markersize=15, + markeredgewidth=3, + markerfacecolor="none", + label="Actual embryo position", + ) ax.legend() fig.canvas.draw() + # Load image img = np.array(Image.open(IMAGE_PATH)) # Create figure fig, ax = plt.subplots(figsize=(12, 12)) -ax.imshow(img, cmap='gray') +ax.imshow(img, cmap="gray") # Draw crosshairs at center -ax.axhline(CENTER_Y, color='red', linestyle='--', alpha=0.7, linewidth=1, label='Center') -ax.axvline(CENTER_X, color='red', linestyle='--', alpha=0.7, linewidth=1) -ax.plot(CENTER_X, CENTER_Y, 'r+', markersize=30, markeredgewidth=2) +ax.axhline(CENTER_Y, color="red", linestyle="--", alpha=0.7, linewidth=1, label="Center") +ax.axvline(CENTER_X, color="red", linestyle="--", alpha=0.7, linewidth=1) +ax.plot(CENTER_X, CENTER_Y, "r+", markersize=30, markeredgewidth=2) ax.set_title("Click on where EMBRYO 3 actually is\n(Red crosshairs = center where it SHOULD be)") ax.legend() # Connect click event -fig.canvas.mpl_connect('button_press_event', on_click) +fig.canvas.mpl_connect("button_press_event", on_click) print("\nClick on where embryo_3 actually appears in the image.") print("The red crosshairs show the center (where it should be).\n") diff --git a/diagnostics/plot_benchmark_results.py b/diagnostics/plot_benchmark_results.py index 2eba25de..d4a8a16f 100644 --- a/diagnostics/plot_benchmark_results.py +++ b/diagnostics/plot_benchmark_results.py @@ -10,14 +10,13 @@ python diagnostics/plot_benchmark_results.py results/benchmark_volume_fps_20260127_123405.csv """ -import sys import csv -from pathlib import Path +import sys from collections import defaultdict +from pathlib import Path -import numpy as np import matplotlib.pyplot as plt -import matplotlib.ticker as ticker +import numpy as np # --------------------------------------------------------------------------- @@ -29,7 +28,7 @@ def parse_benchmark_csv(path: Path) -> dict: summary = [] per_volume = [] - with open(path, "r") as f: + with open(path) as f: reader = csv.reader(f) section = "metadata" @@ -55,27 +54,31 @@ def parse_benchmark_csv(path: Path) -> dict: continue if section == "summary": - summary.append({ - "slices": int(row[0]), - "exposure_ms": float(row[1]), - "approach": row[2], - "vol_per_sec": float(row[3]) if row[3] else None, - "mean_s": float(row[4]) if row[4] else None, - "std_s": float(row[5]) if row[5] else None, - "min_s": float(row[6]) if row[6] else None, - "max_s": float(row[7]) if row[7] else None, - "total_images": int(row[8]) if row[8] else 0, - "num_repeats": int(row[9]) if row[9] else 0, - }) + summary.append( + { + "slices": int(row[0]), + "exposure_ms": float(row[1]), + "approach": row[2], + "vol_per_sec": float(row[3]) if row[3] else None, + "mean_s": float(row[4]) if row[4] else None, + "std_s": float(row[5]) if row[5] else None, + "min_s": float(row[6]) if row[6] else None, + "max_s": float(row[7]) if row[7] else None, + "total_images": int(row[8]) if row[8] else 0, + "num_repeats": int(row[9]) if row[9] else 0, + } + ) elif section == "per_volume": - per_volume.append({ - "slices": int(row[0]), - "exposure_ms": float(row[1]), - "approach": row[2], - "repeat": int(row[3]), - "elapsed_s": float(row[4]), - "image_count": int(row[5]), - }) + per_volume.append( + { + "slices": int(row[0]), + "exposure_ms": float(row[1]), + "approach": row[2], + "repeat": int(row[3]), + "elapsed_s": float(row[4]), + "image_count": int(row[5]), + } + ) return {"metadata": metadata, "summary": summary, "per_volume": per_volume} @@ -95,25 +98,25 @@ def plot_throughput(data: dict, output_dir: Path): approaches.append(r["approach"]) # Build matrix - vps = defaultdict(dict) + vps: dict = defaultdict(dict) for r in summary: if r["vol_per_sec"] is not None: vps[r["slices"]][r["approach"]] = r["vol_per_sec"] # Colors and labels color_map = { - "raw": "#2563eb", - "ophyd": "#dc2626", - "ophyd_burst": "#16a34a", - "burst_reconfig": "#ea580c", - "reconfig_wfd": "#7c3aed", + "raw": "#2563eb", + "ophyd": "#dc2626", + "ophyd_burst": "#16a34a", + "burst_reconfig": "#ea580c", + "reconfig_wfd": "#7c3aed", } label_map = { - "raw": "Raw MMCore", - "ophyd": "Ophyd (full)", - "ophyd_burst": "Ophyd burst", - "burst_reconfig": "Reconfig (sleep)", - "reconfig_wfd": "Reconfig (waitForDevice)", + "raw": "Raw MMCore", + "ophyd": "Ophyd (full)", + "ophyd_burst": "Ophyd burst", + "burst_reconfig": "Reconfig (sleep)", + "reconfig_wfd": "Reconfig (waitForDevice)", } fig, ax = plt.subplots(figsize=(10, 5.5)) @@ -124,16 +127,27 @@ def plot_throughput(data: dict, output_dir: Path): for i, approach in enumerate(approaches): vals = [vps[s].get(approach, 0) for s in slices_set] offset = (i - n / 2 + 0.5) * width - bars = ax.bar(x + offset, vals, width * 0.92, - label=label_map.get(approach, approach), - color=color_map.get(approach, "#888"), - edgecolor="white", linewidth=0.5) + bars = ax.bar( + x + offset, + vals, + width * 0.92, + label=label_map.get(approach, approach), + color=color_map.get(approach, "#888"), + edgecolor="white", + linewidth=0.5, + ) # Value labels on bars - for bar, v in zip(bars, vals): + for bar, v in zip(bars, vals, strict=False): if v > 0: - ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 0.02, - f"{v:.2f}", ha="center", va="bottom", fontsize=7, - fontweight="bold") + ax.text( + bar.get_x() + bar.get_width() / 2, + bar.get_height() + 0.02, + f"{v:.2f}", + ha="center", + va="bottom", + fontsize=7, + fontweight="bold", + ) ax.set_xlabel("Slices per volume", fontsize=11) ax.set_ylabel("Volumes per second", fontsize=11) @@ -149,7 +163,7 @@ def plot_throughput(data: dict, output_dir: Path): fig.tight_layout() fig.savefig(output_dir / "benchmark_throughput.png", dpi=180) plt.close(fig) - print(f" Saved: benchmark_throughput.png") + print(" Saved: benchmark_throughput.png") # --------------------------------------------------------------------------- @@ -167,17 +181,17 @@ def plot_overhead(data: dict, output_dir: Path): compare = ["ophyd", "burst_reconfig", "reconfig_wfd"] color_map = { - "ophyd": "#dc2626", - "burst_reconfig": "#ea580c", - "reconfig_wfd": "#7c3aed", + "ophyd": "#dc2626", + "burst_reconfig": "#ea580c", + "reconfig_wfd": "#7c3aed", } label_map = { - "ophyd": "Ophyd (full teardown/setup)", - "burst_reconfig": "Reconfig (time.sleep)", - "reconfig_wfd": "Reconfig (waitForDevice)", + "ophyd": "Ophyd (full teardown/setup)", + "burst_reconfig": "Reconfig (time.sleep)", + "reconfig_wfd": "Reconfig (waitForDevice)", } - overhead = defaultdict(dict) + overhead: dict = defaultdict(dict) for r in summary: if r["approach"] in compare and r["mean_s"] is not None: s = r["slices"] @@ -192,15 +206,26 @@ def plot_overhead(data: dict, output_dir: Path): for i, approach in enumerate(compare): vals = [overhead[s].get(approach, 0) for s in slices_set] offset = (i - n / 2 + 0.5) * width - bars = ax.bar(x + offset, vals, width * 0.92, - label=label_map.get(approach, approach), - color=color_map.get(approach, "#888"), - edgecolor="white", linewidth=0.5) - for bar, v in zip(bars, vals): + bars = ax.bar( + x + offset, + vals, + width * 0.92, + label=label_map.get(approach, approach), + color=color_map.get(approach, "#888"), + edgecolor="white", + linewidth=0.5, + ) + for bar, v in zip(bars, vals, strict=False): if v > 0: - ax.text(bar.get_x() + bar.get_width() / 2, bar.get_height() + 8, - f"{v:.0f}", ha="center", va="bottom", fontsize=8, - fontweight="bold") + ax.text( + bar.get_x() + bar.get_width() / 2, + bar.get_height() + 8, + f"{v:.0f}", + ha="center", + va="bottom", + fontsize=8, + fontweight="bold", + ) ax.set_xlabel("Slices per volume", fontsize=11) ax.set_ylabel("Overhead vs raw MMCore (ms)", fontsize=11) @@ -216,7 +241,7 @@ def plot_overhead(data: dict, output_dir: Path): fig.tight_layout() fig.savefig(output_dir / "benchmark_overhead.png", dpi=180) plt.close(fig) - print(f" Saved: benchmark_overhead.png") + print(" Saved: benchmark_overhead.png") # --------------------------------------------------------------------------- @@ -228,7 +253,7 @@ def plot_wfd_savings(data: dict, output_dir: Path): summary = data["summary"] slices_set = sorted(set(r["slices"] for r in summary)) - means = defaultdict(dict) + means: dict = defaultdict(dict) raw_means = {} for r in summary: if r["mean_s"] is not None: @@ -243,20 +268,35 @@ def plot_wfd_savings(data: dict, output_dir: Path): x = np.arange(len(slices_set)) width = 0.32 - for i, (approach, label, color) in enumerate([ - ("burst_reconfig", "sleep()", "#ea580c"), - ("reconfig_wfd", "waitForDevice()", "#7c3aed"), - ]): + for i, (approach, label, color) in enumerate( + [ + ("burst_reconfig", "sleep()", "#ea580c"), + ("reconfig_wfd", "waitForDevice()", "#7c3aed"), + ] + ): acq_times = [raw_means.get(s, 0) for s in slices_set] overheads = [means[s].get(approach, 0) - raw_means.get(s, 0) for s in slices_set] offset = (i - 0.5) * width - ax.bar(x + offset, acq_times, width * 0.92, - color="#93c5fd", edgecolor="white", linewidth=0.5, - label="Acquisition time" if i == 0 else None) - ax.bar(x + offset, overheads, width * 0.92, - bottom=acq_times, color=color, edgecolor="white", linewidth=0.5, - label=f"Overhead ({label})") + ax.bar( + x + offset, + acq_times, + width * 0.92, + color="#93c5fd", + edgecolor="white", + linewidth=0.5, + label="Acquisition time" if i == 0 else None, + ) + ax.bar( + x + offset, + overheads, + width * 0.92, + bottom=acq_times, + color=color, + edgecolor="white", + linewidth=0.5, + label=f"Overhead ({label})", + ) ax.set_xlabel("Slices per volume", fontsize=11) ax.set_ylabel("Total time per volume (s)", fontsize=11) @@ -282,27 +322,53 @@ def plot_wfd_savings(data: dict, output_dir: Path): savings.append(so - wo) bar_width = 0.55 - bars_sleep = ax2.barh(x + 0.15, sleep_overhead, bar_width * 0.48, - color="#ea580c", label="time.sleep() overhead") - bars_wfd = ax2.barh(x - 0.15, wfd_overhead, bar_width * 0.48, - color="#7c3aed", label="waitForDevice() overhead") - - for bar, val, sav in zip(bars_sleep, sleep_overhead, savings): - ax2.text(bar.get_width() + 8, bar.get_y() + bar.get_height() / 2, - f"{val:.0f}ms", va="center", fontsize=9, color="#ea580c", - fontweight="bold") - for bar, val in zip(bars_wfd, wfd_overhead): - ax2.text(bar.get_width() + 8, bar.get_y() + bar.get_height() / 2, - f"{val:.0f}ms", va="center", fontsize=9, color="#7c3aed", - fontweight="bold") + bars_sleep = ax2.barh( + x + 0.15, + sleep_overhead, + bar_width * 0.48, + color="#ea580c", + label="time.sleep() overhead", + ) + bars_wfd = ax2.barh( + x - 0.15, + wfd_overhead, + bar_width * 0.48, + color="#7c3aed", + label="waitForDevice() overhead", + ) + + for bar, val, _sav in zip(bars_sleep, sleep_overhead, savings, strict=False): + ax2.text( + bar.get_width() + 8, + bar.get_y() + bar.get_height() / 2, + f"{val:.0f}ms", + va="center", + fontsize=9, + color="#ea580c", + fontweight="bold", + ) + for bar, val in zip(bars_wfd, wfd_overhead, strict=False): + ax2.text( + bar.get_width() + 8, + bar.get_y() + bar.get_height() / 2, + f"{val:.0f}ms", + va="center", + fontsize=9, + color="#7c3aed", + fontweight="bold", + ) # Add savings annotation - for i, (s, sav) in enumerate(zip(slices_set, savings)): - ax2.annotate(f"-{sav:.0f}ms", - xy=(sleep_overhead[i], i + 0.15), - xytext=(sleep_overhead[i] + 60, i + 0.35), - fontsize=8.5, fontweight="bold", color="#166534", - arrowprops=dict(arrowstyle="->", color="#166534", lw=1.2)) + for i, (_s, sav) in enumerate(zip(slices_set, savings, strict=False)): + ax2.annotate( + f"-{sav:.0f}ms", + xy=(sleep_overhead[i], i + 0.15), + xytext=(sleep_overhead[i] + 60, i + 0.35), + fontsize=8.5, + fontweight="bold", + color="#166534", + arrowprops=dict(arrowstyle="->", color="#166534", lw=1.2), + ) ax2.set_yticks(x) ax2.set_yticklabels([f"{s} slices" for s in slices_set]) @@ -316,7 +382,7 @@ def plot_wfd_savings(data: dict, output_dir: Path): fig.tight_layout() fig.savefig(output_dir / "benchmark_wfd_savings.png", dpi=180) plt.close(fig) - print(f" Saved: benchmark_wfd_savings.png") + print(" Saved: benchmark_wfd_savings.png") # --------------------------------------------------------------------------- @@ -328,18 +394,18 @@ def plot_consistency(data: dict, output_dir: Path): approaches_order = ["raw", "ophyd_burst", "reconfig_wfd", "burst_reconfig", "ophyd"] label_map = { - "raw": "Raw\nMMCore", - "ophyd": "Ophyd\n(full)", - "ophyd_burst": "Ophyd\nburst", - "burst_reconfig": "Reconfig\n(sleep)", - "reconfig_wfd": "Reconfig\n(wfd)", + "raw": "Raw\nMMCore", + "ophyd": "Ophyd\n(full)", + "ophyd_burst": "Ophyd\nburst", + "burst_reconfig": "Reconfig\n(sleep)", + "reconfig_wfd": "Reconfig\n(wfd)", } color_map = { - "raw": "#2563eb", - "ophyd": "#dc2626", - "ophyd_burst": "#16a34a", - "burst_reconfig": "#ea580c", - "reconfig_wfd": "#7c3aed", + "raw": "#2563eb", + "ophyd": "#dc2626", + "ophyd_burst": "#16a34a", + "burst_reconfig": "#ea580c", + "reconfig_wfd": "#7c3aed", } slices_set = sorted(set(r["slices"] for r in per_volume)) @@ -348,21 +414,28 @@ def plot_consistency(data: dict, output_dir: Path): if len(slices_set) == 1: axes = [axes] - for ax, ns in zip(axes, slices_set): + for ax, ns in zip(axes, slices_set, strict=False): box_data = [] labels = [] colors = [] for approach in approaches_order: - timings = [r["elapsed_s"] for r in per_volume - if r["slices"] == ns and r["approach"] == approach] + timings = [ + r["elapsed_s"] + for r in per_volume + if r["slices"] == ns and r["approach"] == approach + ] if timings: box_data.append(timings) labels.append(label_map.get(approach, approach)) colors.append(color_map.get(approach, "#888")) - bp = ax.boxplot(box_data, patch_artist=True, widths=0.55, - medianprops=dict(color="black", linewidth=1.5)) - for patch, c in zip(bp["boxes"], colors): + bp = ax.boxplot( + box_data, + patch_artist=True, + widths=0.55, + medianprops=dict(color="black", linewidth=1.5), + ) + for patch, c in zip(bp["boxes"], colors, strict=False): patch.set_facecolor(c) patch.set_alpha(0.7) @@ -377,7 +450,7 @@ def plot_consistency(data: dict, output_dir: Path): fig.tight_layout() fig.savefig(output_dir / "benchmark_consistency.png", dpi=180, bbox_inches="tight") plt.close(fig) - print(f" Saved: benchmark_consistency.png") + print(" Saved: benchmark_consistency.png") # --------------------------------------------------------------------------- diff --git a/diagnostics/run_multi_embryo_volumes.py b/diagnostics/run_multi_embryo_volumes.py index d52aeada..90af3c68 100644 --- a/diagnostics/run_multi_embryo_volumes.py +++ b/diagnostics/run_multi_embryo_volumes.py @@ -9,14 +9,15 @@ python run_multi_embryo_volumes.py """ -import time import json -import numpy as np -from pathlib import Path +import time from datetime import datetime, timedelta -from client import get_mmc -import tifffile +from pathlib import Path + +import numpy as np import rpyc +import tifffile +from client import get_mmc from tqdm import tqdm # Device configuration @@ -36,9 +37,11 @@ def load_database(): """Load embryo database.""" if not DATABASE_FILE.exists(): - raise FileNotFoundError(f"Database not found: {DATABASE_FILE}\nRun multi_embryo_calibration.py first!") + raise FileNotFoundError( + f"Database not found: {DATABASE_FILE}\nRun multi_embryo_calibration.py first!" + ) - with open(DATABASE_FILE, 'r') as f: + with open(DATABASE_FILE) as f: return json.load(f) @@ -58,8 +61,8 @@ def move_to_embryo(embryo_data): embryo_data : dict Embryo information from database """ - target_x = embryo_data['stage_position_after_centering_um']['x'] - target_y = embryo_data['stage_position_after_centering_um']['y'] + target_x = embryo_data["stage_position_after_centering_um"]["x"] + target_y = embryo_data["stage_position_after_centering_um"]["y"] print(f" Moving to embryo position: ({target_x:.2f}, {target_y:.2f}) µm") @@ -94,7 +97,7 @@ def configure_hardware_for_volume(calibration, num_slices): # Stop any existing sequence acquisition (from previous embryo or calibration) if core.isSequenceRunning(): - print(f" Stopping previous sequence...") + print(" Stopping previous sequence...") core.stopSequenceAcquisition() time.sleep(0.5) @@ -105,14 +108,14 @@ def configure_hardware_for_volume(calibration, num_slices): try: core.setProperty(GALVO_DEVICE, "SPIMState", "Idle") time.sleep(0.2) - except: + except Exception: pass # Extract calibration parameters - slope = calibration['slope_um_per_deg'] - offset = calibration['offset_um'] - galvo_top = calibration.get('edge_top_deg', calibration['galvo_top_deg']) - galvo_bottom = calibration.get('edge_bottom_deg', calibration['galvo_bottom_deg']) + slope = calibration["slope_um_per_deg"] + offset = calibration["offset_um"] + galvo_top = calibration.get("edge_top_deg", calibration["galvo_top_deg"]) + galvo_bottom = calibration.get("edge_bottom_deg", calibration["galvo_bottom_deg"]) # Calculate galvo parameters galvo_center = (galvo_top + galvo_bottom) / 2.0 @@ -126,8 +129,14 @@ def configure_hardware_for_volume(calibration, num_slices): piezo_range = piezo_bottom - piezo_top piezo_amplitude = piezo_range / 2.0 - print(f" Galvo: center={galvo_center:+.4f}°, amplitude=±{galvo_amplitude:.4f}° (range: {galvo_range:.4f}°)") - print(f" Piezo: center={piezo_center:.1f}µm, amplitude=±{piezo_amplitude:.1f}µm (range: {piezo_range:.1f}µm)") + print( + f" Galvo: center={galvo_center:+.4f}°, amplitude=±{galvo_amplitude:.4f}°" + f" (range: {galvo_range:.4f}°)" + ) + print( + f" Piezo: center={piezo_center:.1f}µm, amplitude=±{piezo_amplitude:.1f}µm" + f" (range: {piezo_range:.1f}µm)" + ) # System startup core.setConfig("System", "Startup") @@ -189,13 +198,13 @@ def configure_hardware_for_volume(calibration, num_slices): core.setProperty(PIEZO_DEVICE, "SPIMState", "Armed") time.sleep(0.3) - print(f" ✓ Hardware configured for hardware-triggered acquisition") + print(" ✓ Hardware configured for hardware-triggered acquisition") return { - 'galvo_center': galvo_center, - 'galvo_amplitude': galvo_amplitude, - 'piezo_center': piezo_center, - 'piezo_amplitude': piezo_amplitude + "galvo_center": galvo_center, + "galvo_amplitude": galvo_amplitude, + "piezo_center": piezo_center, + "piezo_amplitude": piezo_amplitude, } @@ -238,7 +247,7 @@ def acquire_volume_for_embryo(embryo_id, calibration, num_slices=50): # Trigger SPIM state machine core.setProperty(GALVO_DEVICE, "SPIMState", "Running") - print(f" ✓ SPIM triggered") + print(" ✓ SPIM triggered") # Collect images images = [] @@ -296,52 +305,59 @@ def save_volume(volume, embryo_id, embryo_number, output_dir): def main(): """Main multi-embryo volume acquisition workflow.""" - print(f"{'='*70}") + print(f"{'=' * 70}") print("MULTI-EMBRYO VOLUME ACQUISITION") - print(f"{'='*70}") + print(f"{'=' * 70}") try: # Load database - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print("LOADING DATABASE") - print(f"{'='*70}") + print(f"{'=' * 70}") database = load_database() - embryos = database.get('embryos', {}) + embryos = database.get("embryos", {}) num_embryos = len(embryos) print(f" Database: {DATABASE_FILE}") print(f" Found {num_embryos} embryo(s)") if num_embryos == 0: - print(f"\n ⚠ No embryos in database!") - print(f" Run multi_embryo_calibration.py first.") + print("\n ⚠ No embryos in database!") + print(" Run multi_embryo_calibration.py first.") return # List embryos - print(f"\n Embryos:") + print("\n Embryos:") for emb_id, emb_data in embryos.items(): - emb_num = emb_data.get('embryo_number', '?') - pos = emb_data['stage_position_after_centering_um'] + emb_num = emb_data.get("embryo_number", "?") + pos = emb_data["stage_position_after_centering_um"] print(f" {emb_id} (#{emb_num}): ({pos['x']:.1f}, {pos['y']:.1f}) µm") # Acquisition parameters - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print("ACQUISITION PARAMETERS") - print(f"{'='*70}") + print(f"{'=' * 70}") - num_slices = int(input(f" Number of slices per volume (default 50): ").strip() or "50") + num_slices = int(input(" Number of slices per volume (default 50): ").strip() or "50") print(f" ✓ Will acquire {num_slices} slices per embryo") # Timelapse parameters - num_timepoints = int(input(f" Number of timepoints (default 1 for single acquisition): ").strip() or "1") + num_timepoints = int( + input(" Number of timepoints (default 1 for single acquisition): ").strip() or "1" + ) interval_minutes = 0 if num_timepoints > 1: - interval_minutes = float(input(f" Interval between timepoints in minutes (e.g., 2): ").strip() or "2") + interval_minutes = float( + input(" Interval between timepoints in minutes (e.g., 2): ").strip() or "2" + ) total_duration_hours = (num_timepoints - 1) * interval_minutes / 60.0 - print(f" ✓ Timelapse: {num_timepoints} timepoints every {interval_minutes} min ({total_duration_hours:.1f} hours total)") + print( + f" ✓ Timelapse: {num_timepoints} timepoints every {interval_minutes} min" + f" ({total_duration_hours:.1f} hours total)" + ) else: - print(f" ✓ Single acquisition (no timelapse)") + print(" ✓ Single acquisition (no timelapse)") # Create output directory session_dir = OUTPUT_DIR / datetime.now().strftime("%Y%m%d_%H%M%S") @@ -358,8 +374,11 @@ def main(): desc="Timepoints", unit="tp", position=0, - colour='green', - bar_format='{desc}: {percentage:3.0f}%|{bar}| {n_fmt}/{total_fmt} [{elapsed}<{remaining}, {rate_fmt}]' + colour="green", + bar_format=( + "{desc}: {percentage:3.0f}%|{bar}| {n_fmt}/{total_fmt}" + " [{elapsed}<{remaining}, {rate_fmt}]" + ), ) for timepoint in range(num_timepoints): @@ -367,13 +386,15 @@ def main(): elapsed_hours = (timepoint_start_time - session_start_time) / 3600.0 # Update timepoint progress bar - timepoint_pbar.set_description(f"Timepoint {timepoint+1}/{num_timepoints} (Elapsed: {elapsed_hours:.1f}h)") + timepoint_pbar.set_description( + f"Timepoint {timepoint + 1}/{num_timepoints} (Elapsed: {elapsed_hours:.1f}h)" + ) - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print(f"TIMEPOINT {timepoint + 1}/{num_timepoints}") if num_timepoints > 1: print(f"Elapsed: {elapsed_hours:.2f} hours") - print(f"{'='*70}") + print(f"{'=' * 70}") # Acquire volume for each embryo timepoint_results = [] @@ -385,51 +406,53 @@ def main(): unit="embryo", position=1, leave=False, - colour='cyan' + colour="cyan", ) for idx, (emb_id, emb_data) in enumerate(embryos.items(), 1): - emb_num = emb_data.get('embryo_number', idx) + emb_num = emb_data.get("embryo_number", idx) embryo_pbar.set_description(f" Embryo {emb_num} (t{timepoint:04d})") print(f"\n[Embryo {idx}/{num_embryos}] {emb_id} (Embryo #{emb_num})") - print(f"{'─'*70}") + print(f"{'─' * 70}") # Move to embryo move_to_embryo(emb_data) # Configure hardware - calibration = emb_data['calibration'] + calibration = emb_data["calibration"] configure_hardware_for_volume(calibration, num_slices) # Acquire volume volume = acquire_volume_for_embryo(emb_id, calibration, num_slices) if volume is None: - print(f" ✗ Failed to acquire volume") - timepoint_results.append({ - 'embryo_id': emb_id, - 'timepoint': timepoint, - 'success': False - }) + print(" ✗ Failed to acquire volume") + timepoint_results.append( + {"embryo_id": emb_id, "timepoint": timepoint, "success": False} + ) embryo_pbar.update(1) continue # Save volume with timepoint in filename timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") - filename = session_dir / f"{emb_id}_embryo{emb_num:03d}_t{timepoint:04d}_{timestamp}.tif" + filename = ( + session_dir / f"{emb_id}_embryo{emb_num:03d}_t{timepoint:04d}_{timestamp}.tif" + ) tifffile.imwrite(filename, volume) print(f" ✓ Saved: {filename.name}") - timepoint_results.append({ - 'embryo_id': emb_id, - 'embryo_number': emb_num, - 'timepoint': timepoint, - 'success': True, - 'filename': str(filename), - 'shape': volume.shape - }) + timepoint_results.append( + { + "embryo_id": emb_id, + "embryo_number": emb_num, + "timepoint": timepoint, + "success": True, + "filename": str(filename), + "shape": volume.shape, + } + ) print(f" ✓ Complete: {volume.shape}") embryo_pbar.update(1) @@ -447,10 +470,10 @@ def main(): if wait_time > 0: next_timepoint_time = datetime.now() + timedelta(seconds=wait_time) - print(f"\n{'─'*70}") - print(f"Waiting {wait_time/60:.1f} minutes until next timepoint...") + print(f"\n{'─' * 70}") + print(f"Waiting {wait_time / 60:.1f} minutes until next timepoint...") print(f"Next timepoint at: {next_timepoint_time.strftime('%H:%M:%S')}") - print(f"{'─'*70}") + print(f"{'─' * 70}") # Progress bar for waiting wait_pbar = tqdm( @@ -459,7 +482,7 @@ def main(): unit="s", position=1, leave=False, - colour='yellow' + colour="yellow", ) for _ in range(int(wait_time)): time.sleep(1) @@ -469,70 +492,81 @@ def main(): # Sleep remaining fractional seconds time.sleep(wait_time - int(wait_time)) else: - print(f"\n{'─'*70}") - print(f"⚠ Warning: Acquisition took {timepoint_duration/60:.1f} min (longer than {interval_minutes} min interval)") - print(f"Proceeding immediately to next timepoint...") - print(f"{'─'*70}") + print(f"\n{'─' * 70}") + print( + f"⚠ Warning: Acquisition took {timepoint_duration / 60:.1f} min" + f" (longer than {interval_minutes} min interval)" + ) + print("Proceeding immediately to next timepoint...") + print(f"{'─' * 70}") timepoint_pbar.close() # Cleanup - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print("CLEANUP") - print(f"{'='*70}") + print(f"{'=' * 70}") core.setConfig("Laser", "ALL OFF") - print(f" ✓ Lasers OFF") + print(" ✓ Lasers OFF") # Summary - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print("ACQUISITION COMPLETE") - print(f"{'='*70}") + print(f"{'=' * 70}") total_duration = time.time() - session_start_time - successful = sum(1 for r in all_results if r['success']) + successful = sum(1 for r in all_results if r["success"]) total_acquisitions = num_embryos * num_timepoints - print(f"\n Session duration: {total_duration/3600:.2f} hours") + print(f"\n Session duration: {total_duration / 3600:.2f} hours") print(f" Timepoints: {num_timepoints}") print(f" Embryos per timepoint: {num_embryos}") print(f" Successful acquisitions: {successful}/{total_acquisitions}") print(f" Output directory: {session_dir}") - print(f"\n Results:") + print("\n Results:") for result in all_results: - if result['success']: - t = result.get('timepoint', 0) - print(f" ✓ {result['embryo_id']} t{t:04d}: {result['shape']} → {Path(result['filename']).name}") + if result["success"]: + t = result.get("timepoint", 0) + print( + f" ✓ {result['embryo_id']} t{t:04d}: {result['shape']}" + f" → {Path(result['filename']).name}" + ) else: - t = result.get('timepoint', 0) + t = result.get("timepoint", 0) print(f" ✗ {result['embryo_id']} t{t:04d}: Failed") # Save acquisition log log_file = session_dir / "acquisition_log.json" - with open(log_file, 'w') as f: - json.dump({ - 'timestamp': datetime.now().isoformat(), - 'session_duration_hours': total_duration / 3600.0, - 'num_embryos': num_embryos, - 'num_slices': num_slices, - 'num_timepoints': num_timepoints, - 'interval_minutes': interval_minutes, - 'total_acquisitions': total_acquisitions, - 'successful_acquisitions': successful, - 'results': all_results - }, f, indent=2) + with open(log_file, "w") as f: + json.dump( + { + "timestamp": datetime.now().isoformat(), + "session_duration_hours": total_duration / 3600.0, + "num_embryos": num_embryos, + "num_slices": num_slices, + "num_timepoints": num_timepoints, + "interval_minutes": interval_minutes, + "total_acquisitions": total_acquisitions, + "successful_acquisitions": successful, + "results": all_results, + }, + f, + indent=2, + ) print(f"\n ✓ Log saved: {log_file}") - print(f"\n{'='*70}\n") + print(f"\n{'=' * 70}\n") except KeyboardInterrupt: - print(f"\n\nInterrupted\n") + print("\n\nInterrupted\n") except Exception as e: - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print("ERROR") - print(f"{'='*70}") + print(f"{'=' * 70}") print(f"Error: {e}") import traceback + traceback.print_exc() diff --git a/diagnostics/segment_embryo_nuclei.py b/diagnostics/segment_embryo_nuclei.py index a99538c8..b9c0405f 100644 --- a/diagnostics/segment_embryo_nuclei.py +++ b/diagnostics/segment_embryo_nuclei.py @@ -3,11 +3,12 @@ Run with venv_cv: venv_cv/Scripts/python segment_embryo_nuclei.py """ +from pathlib import Path + +import napari import numpy as np import tifffile -from pathlib import Path from cellpose import models -import napari def load_volume(tiff_path: Path) -> np.ndarray: @@ -39,10 +40,10 @@ def segment_nuclei_3d(volume: np.ndarray, diameter: float = 30.0) -> np.ndarray: masks, flows, styles = model.eval( vol_norm, diameter=diameter, - do_3D=False, # 2D per slice (fast!) + do_3D=False, # 2D per slice (fast!) z_axis=0, stitch_threshold=0.5, # stitch 2D masks into 3D - batch_size=64, # larger batch for speed + batch_size=64, # larger batch for speed ) n_nuclei = len(np.unique(masks)) - 1 @@ -76,6 +77,7 @@ def main(): # Downsample by factor of 2 from scipy.ndimage import zoom + volume = zoom(volume, (1, 0.5, 0.5), order=1) print(f" Downsampled 2x: {volume.shape}") @@ -85,7 +87,7 @@ def main(): # Visualize in Napari print("Opening Napari viewer...") viewer = napari.Viewer() - viewer.add_image(volume, name="Volume", colormap='gray') + viewer.add_image(volume, name="Volume", colormap="gray") viewer.add_labels(masks, name="Nuclei segmentation") napari.run() diff --git a/diagnostics/spim_hardware_triggering_reference.py b/diagnostics/spim_hardware_triggering_reference.py index cab5cfb1..87a9f7a4 100644 --- a/diagnostics/spim_hardware_triggering_reference.py +++ b/diagnostics/spim_hardware_triggering_reference.py @@ -70,9 +70,11 @@ """ import time + import numpy as np from client import get_mmc + def configure_camera_for_hardware_trigger(core, camera_name, exposure_ms): """ Configure Hamamatsu camera for external edge triggering in light sheet mode. @@ -132,8 +134,15 @@ def configure_camera_for_hardware_trigger(core, camera_name, exposure_ms): raise Exception(f"Failed to set TRIGGER ACTIVE to EDGE (got: {trigger_active})") -def configure_spim_scanner(core, scanner_name, num_slices, slice_step_um, - scan_duration_ms, camera_duration_ms, laser_duration_ms): +def configure_spim_scanner( + core, + scanner_name, + num_slices, + slice_step_um, + scan_duration_ms, + camera_duration_ms, + laser_duration_ms, +): """ Configure ASI Tiger scanner for SPIM state machine operation. @@ -180,7 +189,7 @@ def configure_spim_scanner(core, scanner_name, num_slices, slice_step_um, core.setProperty(scanner_name, "SingleAxisYPattern", "1 - Triangle") core.setProperty(scanner_name, "SingleAxisYMode", "3 - Enabled with axes synced") - print(f" X-axis (light sheet): Amplitude=2.0°, Pattern=Triangle, Mode=Synced") + print(" X-axis (light sheet): Amplitude=2.0°, Pattern=Triangle, Mode=Synced") print(f" Y-axis (slice step): Amplitude={y_amplitude:.4f}°, Pattern=Triangle, Mode=Synced") print(f" (Calculated for {num_slices} slices × {slice_step_um} μm steps)") @@ -206,7 +215,7 @@ def configure_spim_scanner(core, scanner_name, num_slices, slice_step_um, core.setProperty(scanner_name, "SPIMDelayBeforeScan(ms)", 0.0) core.setProperty(scanner_name, "SPIMDelayBeforeCamera(ms)", 0.5) - print(f" SPIM State Machine:") + print(" SPIM State Machine:") print(f" NumSlices: {num_slices}") print(f" ScanDuration: {scan_duration_ms} ms (total time per slice)") print(f" CameraDuration: {camera_duration_ms} ms (TTL trigger pulse width)") @@ -214,10 +223,16 @@ def configure_spim_scanner(core, scanner_name, num_slices, slice_step_um, # Verify critical timing relationships if camera_duration_ms > scan_duration_ms: - raise Exception(f"CameraDuration ({camera_duration_ms}ms) must be <= ScanDuration ({scan_duration_ms}ms)") + raise Exception( + f"CameraDuration ({camera_duration_ms}ms) must be <=" + f" ScanDuration ({scan_duration_ms}ms)" + ) if laser_duration_ms > camera_duration_ms: - raise Exception(f"LaserDuration ({laser_duration_ms}ms) must be <= CameraDuration ({camera_duration_ms}ms)") + raise Exception( + f"LaserDuration ({laser_duration_ms}ms) must be <=" + f" CameraDuration ({camera_duration_ms}ms)" + ) def arm_spim_state_machine(core, scanner_name): @@ -256,8 +271,9 @@ def trigger_spim_acquisition(core, scanner_name): print(f" SPIMState: {state}") -def acquire_spim_volume(core, camera_name, scanner_name, num_slices, - scan_duration_ms, timeout_extra_sec=5.0): +def acquire_spim_volume( + core, camera_name, scanner_name, num_slices, scan_duration_ms, timeout_extra_sec=5.0 +): """ Perform hardware-triggered SPIM volume acquisition. @@ -337,7 +353,10 @@ def acquire_spim_volume(core, camera_name, scanner_name, num_slices, count = core.getRemainingImageCount() seq_running = core.isSequenceRunning(camera_name) spim_state = core.getProperty(scanner_name, "SPIMState") - print(f" t={elapsed:.1f}s: images={count}/{num_slices}, seq={seq_running}, SPIM={spim_state}") + print( + f" t={elapsed:.1f}s: images={count}/{num_slices}," + f" seq={seq_running}, SPIM={spim_state}" + ) last_print_time = time.time() time.sleep(0.01) @@ -350,13 +369,16 @@ def acquire_spim_volume(core, camera_name, scanner_name, num_slices, print(" Retrieving images from buffer...") import rpyc + images = [] for i in range(count): img = core.popNextImage() img = rpyc.classic.obtain(img) # Transfer from remote to local images.append(img) - print(f" Image {i+1}/{count}: shape={img.shape}, dtype={img.dtype}, " - f"range=[{img.min()}, {img.max()}], mean={img.mean():.1f}") + print( + f" Image {i + 1}/{count}: shape={img.shape}, dtype={img.dtype}, " + f"range=[{img.min()}, {img.max()}], mean={img.mean():.1f}" + ) # Convert to 3D numpy array volume = np.array(images) @@ -380,9 +402,9 @@ def main(): camera_duration_ms = 155.0 # TTL pulse width (should be ~= exposure) laser_duration_ms = 154.0 # Laser on time (slightly less than camera) - print("="*80) + print("=" * 80) print("ASI diSPIM HARDWARE-TRIGGERED VOLUME ACQUISITION") - print("="*80) + print("=" * 80) try: # Apply system configuration @@ -413,8 +435,13 @@ def main(): # Configure SPIM scanner print() configure_spim_scanner( - core, scanner_name, num_slices, slice_step_um, - scan_duration_ms, camera_duration_ms, laser_duration_ms + core, + scanner_name, + num_slices, + slice_step_um, + scan_duration_ms, + camera_duration_ms, + laser_duration_ms, ) # Arm SPIM state machine @@ -423,43 +450,50 @@ def main(): # Acquire volume print() - volume = acquire_spim_volume( - core, camera_name, scanner_name, num_slices, scan_duration_ms - ) + volume = acquire_spim_volume(core, camera_name, scanner_name, num_slices, scan_duration_ms) # Save volume print("\nSaving volume...") from PIL import Image as PILImage + img_list = [PILImage.fromarray(img.astype(np.uint16)) for img in volume] - img_list[0].save('spim_hardware_triggered_volume.tif', - save_all=True, append_images=img_list[1:]) + img_list[0].save( + "spim_hardware_triggered_volume.tif", + save_all=True, + append_images=img_list[1:], + ) print(f" Saved {len(volume)}-slice volume to: spim_hardware_triggered_volume.tif") print(f" Volume shape: {volume.shape} (slices, height, width)") # Display in napari (optional) try: import napari + print("\nDisplaying in napari...") viewer = napari.Viewer() - viewer.add_image(volume, name='SPIM Volume', colormap='gray', - contrast_limits=[np.percentile(volume, 1), - np.percentile(volume, 99)]) - viewer.dims.axis_labels = ['Z', 'Y', 'X'] + viewer.add_image( + volume, + name="SPIM Volume", + colormap="gray", + contrast_limits=[np.percentile(volume, 1), np.percentile(volume, 99)], + ) + viewer.dims.axis_labels = ["Z", "Y", "X"] print(" Close napari window to continue...") napari.run() except ImportError: print(" (napari not available, skipping visualization)") - print("\n" + "="*80) + print("\n" + "=" * 80) print("ACQUISITION COMPLETE!") - print("="*80) + print("=" * 80) except Exception as e: - print("\n" + "="*80) + print("\n" + "=" * 80) print("ACQUISITION FAILED") - print("="*80) + print("=" * 80) print(f"Error: {e}") import traceback + traceback.print_exc() finally: @@ -469,26 +503,26 @@ def main(): if core.isSequenceRunning(camera_name): core.stopSequenceAcquisition(camera_name) print(" Stopped camera sequence") - except: + except Exception: pass try: core.setProperty(scanner_name, "SPIMState", "Idle") print(" Reset SPIM to Idle") - except: + except Exception: pass try: # Reset camera to internal triggering for live mode core.setProperty(camera_name, "TRIGGER SOURCE", "INTERNAL") print(" Reset camera to internal triggering") - except: + except Exception: pass try: core.setConfig("Laser", "ALL OFF") print(" Lasers OFF") - except: + except Exception: pass diff --git a/diagnostics/switchbot_webgui.py b/diagnostics/switchbot_webgui.py new file mode 100644 index 00000000..f139b97c --- /dev/null +++ b/diagnostics/switchbot_webgui.py @@ -0,0 +1,380 @@ +#!/usr/bin/env python3 +""" +Temporary web GUI to play with the SwitchBot Bot that switches the diSPIM room +light (on for bottom-camera/brightfield imaging, off otherwise). + +This is a TEST TOOL, not part of the production device layer. It drives the Bot +directly over BLE using the same command protocol as +``gently.hardware.switchbot.SwitchBot`` (same command bytes + GATT UUIDs), but +over a single *persistent* connection so the buttons feel snappy and the morse +blinker is fast — the device-layer class is connect-per-command (~1-2 s each), +which is fine for a plan step but hopeless for blinking. + +Features: ON / OFF / PRESS buttons, and a morse-code blinker (blinks the real +room light + mirrors the pattern on screen). The Bot is a mechanical switch +pusher, so each toggle is a ~0.5-1 s servo move — morse is deliberately slow. + +Run: + .venv/bin/python diagnostics/switchbot_webgui.py + # then open http://127.0.0.1:8765 + + .venv/bin/python diagnostics/switchbot_webgui.py --address EC:6F:04:06:5B:23 --port 8765 +""" + +from __future__ import annotations + +import argparse +import asyncio +import logging +from contextlib import asynccontextmanager +from typing import Any + +import uvicorn +from fastapi import FastAPI +from fastapi.responses import HTMLResponse, JSONResponse +from pydantic import BaseModel + +# Reuse the device-layer device's protocol definitions (single source of truth). +from gently.hardware.switchbot import _COMMANDS, _CTRL_CHAR + +logger = logging.getLogger("switchbot_webgui") + +DEFAULT_ADDRESS = "EC:6F:04:06:5B:23" + +# ITU morse, letters + digits. Unsupported characters are skipped. +MORSE = { + "A": ".-", + "B": "-...", + "C": "-.-.", + "D": "-..", + "E": ".", + "F": "..-.", + "G": "--.", + "H": "....", + "I": "..", + "J": ".---", + "K": "-.-", + "L": ".-..", + "M": "--", + "N": "-.", + "O": "---", + "P": ".--.", + "Q": "--.-", + "R": ".-.", + "S": "...", + "T": "-", + "U": "..-", + "V": "...-", + "W": ".--", + "X": "-..-", + "Y": "-.--", + "Z": "--..", + "0": "-----", + "1": ".----", + "2": "..---", + "3": "...--", + "4": "....-", + "5": ".....", + "6": "-....", + "7": "--...", + "8": "---..", + "9": "----.", +} + + +class Bot: + """A single persistent BLE connection to the Bot, with serialized access.""" + + def __init__(self, address: str): + self.address = address + self._client: Any = None + self._lock = asyncio.Lock() + self._morse_task: asyncio.Task | None = None + self.state = "unknown" + self.busy = False + + async def _ensure(self): + from bleak import BleakClient + + if self._client is not None and self._client.is_connected: + return + self._client = BleakClient(self.address, timeout=20) + await self._client.connect() + logger.info("connected to %s", self.address) + + async def _write(self, action: str): + """Write one command, reconnecting once if the link dropped.""" + from bleak.exc import BleakError + + for attempt in (1, 2): + try: + await self._ensure() + await self._client.write_gatt_char(_CTRL_CHAR, _COMMANDS[action], response=True) + if action in ("on", "off"): + self.state = action + return + except (BleakError, OSError, asyncio.TimeoutError) as exc: + logger.warning("write %s attempt %d failed: %s", action, attempt, exc) + self._client = None # force reconnect + if attempt == 2: + raise + + async def _cancel_morse(self): + task = self._morse_task + if task and not task.done(): + task.cancel() + await asyncio.gather(task, return_exceptions=True) + self._morse_task = None + + async def command(self, action: str) -> str: + """ON/OFF/PRESS. Interrupts any running morse (manual override).""" + await self._cancel_morse() + async with self._lock: + await self._write(action) + return self.state + + def schedule(self, text: str, unit: float): + """Build an on/off timeline [(state, seconds), ...] for a message.""" + seq = [("off", round(unit, 3))] # settle to a known state first + for ch in text.upper(): + if ch == " ": + seq.append(("off", round(unit * 7, 3))) + continue + code = MORSE.get(ch) + if not code: + continue + for sym in code: + seq.append(("on", round(unit * (3 if sym == "-" else 1), 3))) + seq.append(("off", round(unit, 3))) # intra-letter gap + st, _ = seq[-1] + seq[-1] = (st, round(unit * 3, 3)) # upgrade to inter-letter gap + return seq + + async def start_morse(self, text: str, unit: float): + await self._cancel_morse() + seq = self.schedule(text, unit) + if len(seq) <= 1: + return None + restore = self.state + self._morse_task = asyncio.create_task(self._play(seq, restore)) + return seq + + async def _play(self, seq, restore: str): + async with self._lock: + self.busy = True + try: + for state, dur in seq: + await self._write(state) + await asyncio.sleep(dur) + await self._write(restore if restore in ("on", "off") else "off") + finally: + self.busy = False + + async def stop(self): + await self._cancel_morse() + async with self._lock: + await self._write("off") + return self.state + + async def disconnect(self): + await self._cancel_morse() + if self._client is not None and self._client.is_connected: + await self._client.disconnect() + + +BOT: Any = None + + +@asynccontextmanager +async def lifespan(app: FastAPI): + yield + if BOT is not None: + await BOT.disconnect() + + +app = FastAPI(lifespan=lifespan) + + +class MorseReq(BaseModel): + text: str + unit: float = 1.5 + + +@app.get("/", response_class=HTMLResponse) +async def index(): + return PAGE.replace("__ADDRESS__", BOT.address) + + +@app.get("/status") +async def status(): + return {"state": BOT.state, "busy": BOT.busy, "address": BOT.address} + + +@app.post("/cmd/{action}") +async def cmd(action: str): + if action not in _COMMANDS: + return JSONResponse({"error": f"unknown action {action!r}"}, status_code=400) + try: + state = await BOT.command(action) + except Exception as exc: + return JSONResponse({"error": str(exc)}, status_code=502) + return {"state": state} + + +@app.post("/morse") +async def morse(req: MorseReq): + unit = max(0.3, min(4.0, req.unit)) + text = req.text[:40] + try: + seq = await BOT.start_morse(text, unit) + except Exception as exc: + return JSONResponse({"error": str(exc)}, status_code=502) + if seq is None: + return JSONResponse({"error": "nothing sendable in that text"}, status_code=400) + seconds = round(sum(d for _, d in seq), 1) + return {"schedule": seq, "unit": unit, "seconds": seconds} + + +@app.post("/stop") +async def stop(): + try: + state = await BOT.stop() + except Exception as exc: + return JSONResponse({"error": str(exc)}, status_code=502) + return {"state": state} + + +PAGE = """ + + +diSPIM Room Light + +
+

diSPIM Room Light

+
SwitchBot Bot · __ADDRESS__
+
+
+
+ + + +
+
+ +
+ fast + + slow + 0.7s +
+
+ + +
+
+
+
+ +""" + + +def main(): + ap = argparse.ArgumentParser(description="Temporary SwitchBot room-light web GUI") + ap.add_argument("--address", default=DEFAULT_ADDRESS, help="Bot BLE MAC address") + ap.add_argument("--port", type=int, default=8765) + ap.add_argument("--host", default="127.0.0.1", help="bind host (default: localhost only)") + args = ap.parse_args() + + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s") + global BOT + BOT = Bot(args.address) + print(f"\n diSPIM Room Light GUI → http://{args.host}:{args.port}\n Bot: {args.address}\n") + uvicorn.run(app, host=args.host, port=args.port, log_level="warning") + + +if __name__ == "__main__": + main() diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 00000000..cd2621f9 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,171 @@ +# Gently — Concurrency & Runtime Architecture + +How the system runs temperature telemetry, device-state polling, experiments, +image acquisition, and perception/VLM "at the same time" without stepping on +itself. The short version: **almost nothing runs truly in parallel — the design +quarantines the blocking work and serializes the hardware, then keeps both event +loops responsive by offloading and decoupling everything slow.** + +> Scope: the concurrency model. For storage layout see `CLAUDE.md`; for the +> device/hardware plugin model see `docs/asi-plugin-architecture.md`. + +## 1. Two processes, bridged by HTTP + a shared filesystem + +This split is the load-bearing fact of the whole design. + +``` +┌────────────────────────────────────────┐ ┌────────────────────────────────────────┐ +│ APP / VIZ process (FastAPI :8080) │ │ DEVICE-LAYER process (aiohttp :60610) │ +│ gently/app/agent.py │ │ gently/hardware/dispim/device_layer.py │ +│ │ HTTP │ │ +│ • agent + TimelapseOrchestrator │◄──────►│ • the ONLY code touching hardware: │ +│ • Perceiver (gently_perception, VLM) │ (DiSPIM│ MMCore/pymmcore, Ophyd, Bluesky RE │ +│ • AsyncAnthropic clients │ Client)│ • plan queue + single executor │ +│ • EventBus + WebSocket ConnectionMgr │ │ • 3 state pollers + camera/LS streamers │ +│ • TemperatureSampler, DeviceStateMonitor│ │ │ +└──────────────┬──────────────────────────┘ └───────────────┬──────────────────────────┘ + │ shared filesystem (incoming/ TIFF staging, session store) + └──────────────────────────────────────────────────┘ +``` + +- The **device-layer process** (`device_layer.py`) is the *only* code that ever + touches the microscope: MMCore/pymmcore, the Ophyd devices, and a Bluesky + `RunEngine`. All truly blocking hardware work is quarantined here. +- The **app/viz process** (`app/agent.py`, FastAPI+uvicorn) owns everything + cognitive and user-facing: the agent, the `TimelapseOrchestrator`, the + `Perceiver` (external `gently_perception` VLM package), the Anthropic clients, + the in-process `EventBus`, the WebSocket `ConnectionManager`, and the pollers. +- The app reaches hardware **only** through `DiSPIMClient` (`client.py`), over a + single shared `aiohttp.ClientSession`. It never imports MMCore. + +Because blocking hardware calls live in a separate process, the app-side loop +stays a light cooperative-asyncio world. + +## 2. One asyncio event loop per process + +- **Device loop:** HTTP routes + the RunEngine driver + the plan queue/executor + + three state pollers (XY ~5 Hz, piezo/galvo ~1 Hz, full property cache ~15 s) + + subscriber-gated camera/lightsheet SSE streamers. +- **App loop:** FastAPI + the `TimelapseOrchestrator` acquisition loop + + `TemperatureSampler` (1 Hz) + `DeviceStateMonitor` + `Perceiver` calls + one + coroutine per open browser WebSocket. + +## 3. The core trick: hardware is serialized, not parallelized + +Every Bluesky plan — a move, a snap, a volume/burst acquisition, a focus/calibration +sweep — is submitted as a `PlanRequest` onto **one `asyncio.Queue` +(`self._plan_queue`), drained by one `_plan_executor` task**. Only one plan owns +the hardware at a time. + +- `submit_plan` enqueues and `await`s a per-request `asyncio.Future`; the executor + sets the result/exception, which flows back to the awaiting HTTP handler. On + failure the executor records it in `_plan_execution_log` and continues to the + next queued plan. +- Underneath, **pymmcore's internal `g_core_lock` is the real mutex** serializing + every actual core call across pollers *and* plans. The `DiSPIMSystem` facade is + the single place the process touches MMCore. + +A microscope has one stage and one camera; single-file execution is correct, not +a limitation. + +## 4. How constant polling coexists with long experiments + +Four cooperating mechanisms keep the loops responsive while a plan runs: + +1. **Offload blocking reads.** Every MMCore read, camera grab, SAM call, and + transient temperature probe goes through `asyncio.to_thread`; the camera + sequence runs on its own `threading.Thread` returning an Ophyd `Status`. The + loop itself never sits on I/O. +2. **Split pollers by cadence.** The three device-state pollers are independent + tasks so a slow (~1.5 s) full-state-cache read cannot stall the ~5 Hz XY path. +3. **`pause_state_updates()` — a reference-counted async context manager.** Heavy + plans (a `frozenset` of names) wrap execution in it, incrementing a counter; + every poller/streamer checks `if self._state_pause_counter > 0` and *skips its + MMCore read*, emitting only ~2 s heartbeats. The plan gets full serial/camera + bandwidth instead of the pollers fighting it for `g_core_lock`. Nested heavy + sections stack and unwind cleanly. +4. **Telemetry bypasses the plan queue.** `GET /api/temperature/status` reads the + temperature Ophyd device directly (`temp.read()` over serial/MQTT — a device + wholly separate from MMCore); `GET /api/devices/state` returns the cached + `_state_latest` snapshot. Neither sits behind a running experiment, so status + polls are never blocked by a long acquisition. (MMCore push callbacks also + mirror joystick/property changes into `_state_latest` via + `loop.call_soon_threadsafe` with ~50 ms debouncing.) + +### Temperature specifically +The vendor SDK backend (serial or MQTT) runs its own **background daemon thread** +that ingests the controller's 500 ms telemetry broadcast into a cache +(`self.telemetry`); `get_water_temp()`/`get_system_state()` return the cached +value (non-blocking). `TemperatureController.read()` returns that cache, and +`TemperatureSampler` (`interval_sec=1.0`) polls it at 1 Hz → persists + emits +`TEMPERATURE_UPDATE`. So we ride the telemetry indirectly, resampled at 1 Hz. + +## 5. Image data travels via the filesystem, not JSON + +Arrays over ~1 MB are written as **TIFF into the shared `incoming/` staging dir**; +only a small `{__file_ref__, path, shape, dtype}` dict crosses HTTP (`serialize_value`). +The client resolves the ref with `tifffile` and hands the decoded array to +`register_volume`, which renames the file into the session store and stamps it +with the latest temperature sample — avoiding a multi-GB JSON blob and a double +decode. + +## 6. Perception / VLM / events are decoupled (fire-and-forget) + +- **Perception never gates acquisition.** The `TimelapseOrchestrator` does + `asyncio.create_task(self._run_perception(...))` rather than awaiting it inline, + so a slow VLM call doesn't hold up the next embryo. Inside the task the + `Perceiver` and the Claude client are awaited cooperatively, and every Claude + call is wrapped in `asyncio.wait_for(timeout=30)` returning a safe fallback + instead of raising into the loop. +- **The `EventBus` publishes without awaiting handlers** (`event_bus.py`): sync + handlers run inline, async handlers are scheduled via `asyncio.ensure_future` / + `loop.call_soon_threadsafe`; high-volume telemetry types skip the bounded + history deque, so one slow WebSocket client delays only its own broadcast. + +## 7. Backpressure & failure isolation + +- **SSE streams** use per-subscriber queues bounded at `maxsize=4`; device-state + broadcasts drop slow subscribers, camera/lightsheet streams drop the *oldest* + frame and push the newest so steady clients keep fresh frames. +- **WebSocket fan-out** (`ConnectionManager.broadcast`) sends per client under an + `asyncio.Lock` and drops clients that error. +- **Failure = a gap, not a crash.** A failed temperature poll logs once and backs + off (`1.0 s → min(interval*10, 30 s)`); a stalled SSE forces a watchdog + reconnect after 60 s (chosen to tolerate the quiet windows during heavy plans); + volume acquisition always disables lasers on error to protect the sample. +- **Config safety.** `POST /api/temperature/config` refuses (`409`) while the + RunEngine is not idle or a ramp holds the controller lock; `health_check()` is + read-only and never flips the connected flag, so a transient status-poll timeout + can't disconnect an in-flight acquisition. + +## 8. Known limits & bottlenecks + +- **RunEngine on the loop.** `self.RE(plan)` is invoked synchronously on the + device-layer loop, so while a plan runs the loop is largely occupied — which is + exactly why heavy plans quiet the pollers and the app-side watchdog tolerates + 60 s of silence. Whether the telemetry HTTP handlers stay fully responsive + mid-acquisition depends on Bluesky's internal threading (external package) and + is not verified from repo code. +- **Unbounded in-flight perception.** `_perception_tasks` is a plain set that + self-prunes; sustained fast acquisition against a slow VLM could grow concurrent + Claude calls without an explicit cap. A semaphore is the obvious guard if + cadence is ever pushed. +- **~~Synchronous O(n) prediction writes~~ — FIXED.** `store_prediction` used to + re-parse the entire `predictions.jsonl` on every append to compute the next id. + Now O(1) via a bounded tail read (`_last_jsonl_record`, `file_store.py`). Other + `FileStore` JSONL writes remain synchronous on the app loop but are single-line + appends. +- **External VLM internals.** How `gently_perception`'s `Perceiver` implements its + VLM call (async httpx vs sync-in-thread) is not inspectable from this repo; the + orchestrator awaits it, implying a coroutine. + +## The model in one line + +Two single-threaded event loops; hardware fenced into one process behind a +one-at-a-time plan queue; everything blocking pushed into threads; and the +slow/cognitive work (VLM, persistence, UI) decoupled with fire-and-forget tasks — +so the system *feels* concurrent while the microscope itself stays strictly +serialized. + +--- +*Generated from a code-grounded architecture pass (Claude Opus 4.8), 2026-07-01.* diff --git a/docs/CLOSED_LOOP_PARADIGM.md b/docs/CLOSED_LOOP_PARADIGM.md new file mode 100644 index 00000000..8f1cf80d --- /dev/null +++ b/docs/CLOSED_LOOP_PARADIGM.md @@ -0,0 +1,536 @@ +# Closed-Loop Paradigm: Notes on the Discussion + +This document captures the design conversation that produced everything on the +`paradigm/closed-loop` branch: the schema split, the Map-as-embryo-home work, +the operator-action vocabulary, the eval substrate (capture / replay / +decisions / shadow), and the trajectory the system is on. It is a +distillation, not a transcript — a future-self / new-collaborator reference +for *why* this code looks the way it does and *where it is going*. + +--- + +## 1. The Original Friction + +The conversation started from a small, concrete observation by the operator: + +> "It feels awkward that the operator has to go between the chat in the TUI +> and the viz server… or even to chat about detecting embryos." + +That awkwardness is a symptom, not a defect. It surfaces a deeper design +question: **what is the orchestrator (the agent) actually for?** Today the +orchestrator does at least four jobs at once, and one of them — *tool router* +— is the one creating the friction. + +### The four orchestrator roles + +| Role | Description | Replaceable by a button? | +| --- | --- | --- | +| 1. Tool router | "Detect embryos" → `detect_embryos()` call | **Yes** — this is the friction surface | +| 2. Workflow runner | Timelapses, multi-embryo plans, perception loops | No | +| 3. Domain reasoner | Knows microscopy, embryos, safety constraints | No | +| 4. Session memory | Coherent narrative of what happened and why | No | + +Routing a single click through chat for a routine action is the system +fighting against its own users. Routing a multi-step scientific decision +through Claude is using the right tool for the right job. The paradigm here +is: **shrink role 1 to its essentials, keep roles 2–4 first-class, and let +the UI carry the rest.** + +--- + +## 2. Web ↔ Chat Reconciliation Patterns + +Four ways to relate the web UI and the chat orchestrator. Each has a +distinct world model property: + +### A. Chat-only intent (the old default) + +Every action originates in chat. The web is observation + delegated subtasks +(e.g. the marking canvas is a delegation the orchestrator triggers). + +* Cleanest record. +* Worst friction. +* Orchestrator's world model is "complete" because every change passes + through it. + +### B. Two parallel command surfaces + +Operator clicks in web, web acts directly; orchestrator finds out by polling +state or doesn't find out at all. + +* Lowest friction. +* Orchestrator's world model **drifts from reality** — fatal for role 4 + (session memory) and dangerous for role 3 (safety reasoning). + +### C. Web acts, orchestrator subscribes *(the chosen direction)* + +Operator clicks → web performs the action **and** publishes an event +(`OPERATOR_*`) → orchestrator's session memory ingests it. + +* Chat log shows only human conversation. +* Orchestrator's working context shows chat + events as a single timeline. +* Phase 7 (operator events vocabulary + reactive candidate) is the first + installment of this pattern. + +### D. Cross-pattern hybrid + +Different action classes use different patterns. Heavy / novel / composite +actions use chat (A); routine / clickable / contextual actions use web (C). +This is what the system actually drifts toward; pattern C is the substrate +that makes it possible. + +The orchestrator's job shifts from being **a funnel for action** to being **a +brain that knows what's happening on every surface**. + +--- + +## 3. The "Turn" is Wrong; the "Decision Moment" is Right + +Chat-AI literature reasons in *turns* (user message → assistant response). +That model imports an assumption that does not hold here: the human is at +the keyboard continuously. In a microscopy experiment running 12+ hours, the +human checks in once, twice, maybe ten times. The agent is autonomous in +between. + +The right unit is a **decision moment**, triggered by: + +1. **User message** — rare, interrupting (classic chat turn). +2. **Critical event** — error, safety violation, lost focus, perception + anomaly. Wake immediately; decide to act / abort / escalate. +3. **Phase boundary** — between timepoints, between embryos. Built-in + checkpoint: review state, decide whether to continue. +4. **Periodic checkpoint** — every N minutes if nothing else happened. + Catches slow drifts. + +Between moments the agent is asleep. Plans execute autonomously. Events +accumulate on the bus and in the world model. When the next decision +moment fires, the agent reads: + +* The trigger (why am I waking up?) +* The world snapshot (NOW state) +* The events digest (what happened since last wake) +* The conversation history (which might be hours old and less relevant + than usual) + +This is closer to a **supervisory controller** than a chat partner. The +conversation history matters less than usual; what matters more is the +**flight log** (events) plus the **current state snapshot**. + +### Trigger model — concrete + +A small router (in code, not Claude) sits between the bus and the brain: + +``` + user input ─┐ + event bus ──┼─► wake-router ──► (compose context) ──► claude.messages.create + schedule ──┘ +``` + +The router's responsibilities: +* Subscribe to a whitelist of "wake-worthy" event types. +* Hold a debounce / coalescing buffer (so a burst of events becomes one + wake). +* Keep a heartbeat schedule (every N minutes if no other trigger fired). +* On wake, package: trigger, world snapshot, events digest, recent + conversation tail. +* Surface the package to the brain. + +The brain stays the brain (Claude). The router is cheap, deterministic, +debuggable code. It's the **meta-orchestrator** the operator mentioned — +**not as another LLM**, but as a control surface. + +### Phase boundaries: hand-back vs subscribe + +Two designs for letting the brain look in mid-plan: + +* **Plan hands control back** at well-known points (between embryos, every + 5 timepoints). Cheaper, predictable, slightly less reactive. +* **Plan never pauses; brain subscribes to plan events** ("perception + complete for embryo 3"). More reactive, more plan-coupling. + +The first one composes better with the supervisory-controller framing and +is the recommended starting point. + +### Idle ticks + +If 30 min pass with no event and no user, should the agent wake to verify +everything's OK? Default to **yes — periodic ticks with a high action +threshold.** Most ticks should result in the agent doing nothing. The +purpose is catching slow drifts (focus, sample state, hardware +degradation) that don't trigger their own events. + +--- + +## 4. World Model — Tiered, Not Monolithic + +A common mistake is "summarise everything every turn." Better is a tiered +model where different freshness/density tiers carry different cadence +costs. + +### Tier 1 — World snapshot + +Structured, ~30 lines, computed from in-memory state (not events), every +wake. + +Includes: live stage XY/Z, current session id, embryo list with +calibration state, current plan, acquisition status, recent operator +actions (one-line summary). + +Cheap to build, fresh every time. Already mostly present in the +codebase — `agent.experiment.get_summary()` plus the cached +`DEVICE_STATE_UPDATE` payload is 80% of this. + +### Tier 2 — Recent-events digest + +Hand-written formatter over the events bus, filtered to wake-worthy types, +inserted as a system note at each wake. + +Shape: `"Since last response: operator added embryo 4 via Map at 14:32; calibration completed for embryo 2; one perception trace pending."` + +Hand-written because LLM summarisation here adds latency, cost, and +non-determinism for low value. Events are already structured. + +### Tier 3 — Pull tools + +For when reasoning needs depth: `get_recent_perceptions(embryo_id, n=5)`, +`get_session_timeline()`, `get_learnings(campaign_id)`, etc. The agent +calls these when it wants the detail. + +### Tier 4 — Optional LLM summariser + +Reserved for genuinely natural-language streams that resist rule-based +compression: accumulated CV reasoning chains, narrative observations, +cross-session learnings. Use a smaller, faster Claude model (Haiku is the +natural fit). Run lazily, when a tier-3 tool asks for "summarise the last +30 min for embryo 3." + +### Why this shape + +Decision moments are **rare** in autonomous mode. Token budget per wake +can be generous (it's mostly idle compute). What matters more than budget +is **cadence of waking** — saving 200 tokens per turn doesn't help if +you're waking up at the wrong moments. + +--- + +## 5. Testing — Where Most Projects Fail + +You cannot iterate on agent architecture without a way to compare +architectures. Microscopy makes this hard: + +* Physical, non-deterministic, non-replayable in the trivial sense. +* "Correct" is fuzzy — biological judgements rarely have ground truth. +* Slow feedback (a timelapse takes hours). +* Can't always reset to a clean state (samples are consumed). + +Five testing primitives, ranked by payoff per unit work (this ordering +informed Phase 6's build order): + +### 5.1 Event replay *(built — Phase 6a/6b)* + +Capture the full event stream during a real run. Offline, replay it +through any candidate architecture. Diff its decisions against +production's. **Foundation** — without it, every change to the +orchestrator is a flight test. + +### 5.2 Shadow mode *(built — Phase 6d)* + +During a real experiment, candidate architectures run alongside +production. They see the same events but their decisions are *logged, +not enacted*. Unique value over pure replay: shadow agents experience +real temporal cadence, so timing-sensitive things (drift, races) are +caught. + +### 5.3 Synthetic event sequences + +Hand-crafted streams: cascading errors, ambiguous perception, +contradictory operator actions, focus drift, network drop mid-acquisition. +Stress / chaos testing. The orchestrator is correct if it doesn't do +something catastrophic — much easier to score than biological +correctness. + +Trivially built on top of 5.1 — write a `jsonl` by hand, replay it. + +### 5.4 Decision-level micro-benchmarks + +Specific judgements — "given this perception result and these recent +observations, should the agent re-focus?" — captured as +(input → expected decision) pairs labelled by a biologist. Regression +suite. Cheap with biologist time, expensive to bootstrap, very valuable +once you have a few hundred. + +### 5.5 Multi-agent A/B in production + +Two embryos in the same dish, one supervised by architecture A and one +by B (both honouring the firmware fence). Compare biological outcomes. +Slow (one timelapse per data point), but the **only thing that measures +biological correctness end-to-end.** + +--- + +## 6. Embryo Schema: Coarse vs Fine + +Foundational and quietly important. Each embryo carries: + +* `position_coarse` — set by bottom-camera detection or manual Map + placement. Always present. +* `position_fine` — set later by SPIM-objective alignment (workflow not + yet built). Initially `{}`. +* `stage_position` — a *derived property*: `fine if fine else coarse`. + Downstream motion / perception keeps reading this and stays agnostic + about which calibration stage we're in. + +This is the seed for a broader idea: **measurements have provenance and +calibration state**. The same embryo at the same nominal XY can have +different "true" positions depending on which sensor sighted it. Encode +that explicitly so any downstream decision can ask *"how confident is +this position?"* without needing to know the whole calibration history. + +When the operator drags an embryo on the Map, the PUT clears `fine` — +overriding the sighting invalidates any SPIM-derived fine alignment +derived from the old coarse. `OPERATOR_EDITED_EMBRYO` carries +`fine_position_invalidated` so the candidate / future controller can +schedule a re-alignment without inferring it. + +--- + +## 7. The Map as Collaborative World Model + +The Devices > Map page is more than visualisation. It is the **first +collaborative surface** between operator and orchestrator: both can read +the embryo list; both can mutate it. The orchestrator subscribes; the +operator clicks. + +Visual semantics matter: + +* Coarse-only embryo → outlined ring + number. *Provisional.* +* SPIM-fine-calibrated → filled disc + number. *Committed.* + +Calibration state is then visible at a glance across the slide — the +operator can scan and see "embryo 3 still needs alignment" without +opening anything. + +The pick-up / drop interaction (Phase 5) deliberately rejects +click-to-add: the Map is a schematic, not a satellite view. Adding a +new sighting without a visual reference is guessing. New embryos go +through the bottom-camera marking canvas. The Map is for **editing what +already exists**. + +### Future arc + +* **Annotations beyond position**: operator marks "this is the control", + "this one is dead, skip", "I think this is in 2-cell stage". These + become first-class scientific observations through additional + `OPERATOR_*` events. +* **Satellite tile**: render the live bottom-camera frame as an overlay + on the Map at the current stage XY, scaled by um_per_pixel. Inside + that tile, click-to-add becomes meaningful (you can see what you're + picking). Outside, the Map stays schematic. + +--- + +## 8. Revolutionary Trajectories + +Some of these are reasonable extensions; some are genuinely new. + +### 8.1 Plans-as-goals, not scripts + +Operator specifies "characterise gut development for these four +embryos." Orchestrator translates this into a continuously adapted +imaging plan that changes based on what perception sees mid-run. The +plan isn't a fixed script handed to Bluesky — it's a negotiation the +orchestrator keeps in flight, with the world model as the substrate +for adaptation. + +Requires: tier-1 + tier-2 world model, decent perception loop, a way +to express goals as predicates over the world model. + +### 8.2 Compounding cross-session learning + +`agent/learnings/` already exists. Today it's barely used. With replay ++ shadow, an architecture that proposes priors ("embryos at 3-fold +typically need slower piezo") becomes **A/B testable across sessions**. +Improvement gets *measurable*, which is the unlock — most "smart +microscopy" today is shallow because it has no measurement loop. + +The right framing: each session is a **trial**, the orchestrator is the +**experimenter**, the world model is what carries learning between +trials. + +### 8.3 Collaborative world model + +The Map (operator edits embryos) is the first instance. Extend +everywhere: + +* Operator annotates morphology on the Map → orchestrator updates + hypothesis space. +* Operator marks a focus failure → orchestrator marks the calibration + region as untrustworthy. +* Operator confirms a perception → orchestrator increases confidence in + the perception predicate for similar inputs. + +The point is making the operator's tacit knowledge **first-class data** +that the system can reason about, not just record. + +### 8.4 Reverse-mode microscopy + +"I want to know X — plan the imaging that answers X." The orchestrator +translates scientific goals into imaging plans. This is the +plans-as-goals idea taken to its conclusion: the operator describes +intent in scientific terms, the orchestrator owns the imaging strategy. + +Tractable only once 8.1 and the goal language are built. + +### 8.5 Continuous shadow / always-on critic + +Run the production orchestrator + a shadow candidate continuously, and +log all decision divergences. Over weeks, the divergence log becomes a +**dataset of disagreements**. Each disagreement is either: + +* Production was right, candidate was wrong → candidate needs a fix. +* Candidate was right, production was wrong → consider promotion or + investigate why production picked differently. +* Both were defensible → annotate the case. + +Free with the eval substrate (Phase 6); the only addition is a +divergence collator. + +--- + +## 9. Concretely Built Today (`paradigm/closed-loop` branch) + +| # | Commit | What | +| --- | --- | --- | +| 1 | `3e410581` | Schema split: `position_coarse` / `position_fine` / derived `stage_position`. | +| 2 | `617e54c9` | `ExperimentState.notify_embryos_changed()` observer → `EMBRYOS_UPDATE` broadcast. | +| 3 | `144d9fc9` | Map render layer — lavender rings (coarse) / discs (fine) / numbers. | +| 4 | `4fbb9edf` | `detect_embryos` flows through web Marking canvas; `auth.py` + `require_control`. | +| 5 | `8f6553e1` | Map pick-up / drop / Delete to edit embryos in place (control-gated PUT/DELETE). | +| 6 | `808fe813` | Side-fix: re-enable XY joystick at device-layer boot. | +| 7 | `f7a13d69` | Side-fix: image-anchored crosshair + scroll-to-zoom in camera panel. | +| 8 | `d69cc219` | `gently/eval/`: event capture / replay / shadow / decision log scaffolding. | +| 9 | `75d7c9db` | Production decision capture wired through `ConversationManager.call_claude`. | +| 10 | `0a97563e` | `OPERATOR_*` event vocabulary + `ReactiveCandidate` (first real shadow). | + +### Per-session disk shape now + +`D:\Gently3\sessions\{id}\` + +* `events.jsonl` — captured event bus, telemetry-filtered. +* `decisions.jsonl` — every Claude turn (success + error). +* `interaction_log.jsonl` — pre-existing chat-shaped interactions. +* `timeline.jsonl` — pre-existing session timeline. +* Plus everything from the legacy FileStore layout. + +### Eval CLI + +`python scripts/replay_session.py {session_id_prefix} [--histogram] [--candidate {name}] [--real-time] [--time-scale N]` + +--- + +## 10. What is *Not* Done Yet + +These are the natural follow-ups; sketched as future-self breadcrumbs. + +### Near-term (days) + +* **Tier-1 world snapshot** as a system-prompt section the brain sees + on every wake. Build the snapshot from `agent.experiment` plus the + last cached `DEVICE_STATE_UPDATE`. ~30 lines of formatted prose, every + wake. +* **Tier-2 events digest** — hand-written formatter that reads the + bus's recent meaningful events (or the captured jsonl tail) and + produces a one-paragraph "since last response" note. +* **Snapshot ingest into the brain's prompt** — `_update_system_prompt` + already takes a `context_summary`; route tier-1 + tier-2 through it. + +### Medium-term (weeks) + +* **Wake-router** — the code-level scheduler from §3. Currently the + brain only wakes on user message. Add: event-driven wake (subscribe + to wake-worthy events), periodic-tick wake (heartbeat), debounce / + coalesce buffer. +* **More operator events** — `OPERATOR_ANNOTATED_EMBRYO` ("this is the + control", "skip, looks dead"), `OPERATOR_STARTED_TIMELAPSE`, + `OPERATOR_INTERRUPTED_PLAN`, `OPERATOR_TOGGLED_CAMERA`. Whatever the + Map / web UI lets the operator do should publish a typed event. +* **SPIM-fine alignment workflow** — populate `position_fine`. Tool + + per-embryo state transition. Triggers `EMBRYOS_UPDATE` and a new + `FINE_ALIGNMENT_COMPLETED` event the orchestrator can react to. +* **Continuous-shadow harness** — extend `ShadowRunner` to run a + candidate alongside production in the live agent process (not just + during replay). Collect divergences into a per-session + `divergences.jsonl`. + +### Longer arc (months) + +* **A goal expression language** — predicates over the world model that + let the operator say "image until 4-fold" or "follow the cell + divisions in embryo 3 at high resolution." This is the substrate for + §8.1 (plans-as-goals). +* **LLM-driven candidates** — once the rule-based `ReactiveCandidate` + proves the substrate, add Claude-driven candidates (Haiku for cheap, + Opus for thinking). Use the snapshot+digest as their input. +* **Cross-session learning loop** — wire the `learnings/` store into + the world model as priors. Add a learning-write surface (a tool the + orchestrator can call when it notices a pattern). Use shadow A/B to + validate that learnings improve decisions. +* **Goal-driven evaluation** — once goals exist, "did the experiment + achieve its goal" becomes a measurable end-to-end success rate. The + ultimate metric is this, not turn-level decision diffs. + +--- + +## 11. Principles That Surface Throughout + +A few recurring design priors worth naming: + +1. **Distill, don't dump.** Structured summaries beat raw logs in + prompts. Hand-written formatters beat LLM summarisers for + structured data. LLMs for prose, code for structure. +2. **Pull beats push when uncertain.** Default to tools the agent + queries on demand, not data shoved into every prompt. Push only + what's universally relevant (the world snapshot). +3. **Same shape for production and shadow.** If production writes a + Decision with these fields, shadow candidates write Decisions with + the same fields. Diff is then trivial. +4. **Events carry intent; state carries position.** `EMBRYOS_UPDATE` + is state (the embryo list now). `OPERATOR_EDITED_EMBRYO` is intent + (a human just did this). Both exist; they answer different + questions. +5. **The brain doesn't move hardware.** All hardware action goes + through tools that go through the device layer that goes through + ophyd that goes through MMCore. Shadow candidates are constructively + prevented from acting. Layers are not negotiable. +6. **No SaveCardSettings.** Firmware persistent state silently inherits + between sessions; if it ever gets out of sync with code it's a + debugging nightmare. Apply firmware config every boot, code wins. +7. **Localhost is the diSPIM box. Remote is view-only by default.** + Auth surface stays tiny and explicit. Token upgrade is the seam, + not user accounts. + +--- + +## 12. Open Questions (Worth Revisiting Later) + +* **Continuous vs episodic shadows.** Continuous always-on shadow + captures divergence over time but multiplies cost (multiple LLM + candidates running). Episodic shadow at decision moments is cheaper + but misses timing-sensitive cases. Hybrid? +* **Is the conversation history the right substrate at all?** With + decision moments hours apart, prior chat may be more distracting + than useful. Maybe the brain shouldn't see chat history beyond N + hours; the world model + events digest are the durable memory and + chat is just for the active dialogue. +* **How much should the operator know about the orchestrator's plan?** + Today the operator drives by asking. With autonomous mode, the + orchestrator runs experiments largely on its own. Should there be a + permanent "what is the orchestrator thinking right now" surface + visible on the Map? An always-on intent display? +* **Failure semantics.** If a candidate would have made a different + decision than production, and production's decision led to a bad + outcome, the candidate "wins." How do we score? Define "bad outcome" + rigorously enough that it can be measured? + +These are not blockers. They are notes for the next iteration of this +document, after a few weeks of running on the substrate built here. diff --git a/docs/EVAL.md b/docs/EVAL.md new file mode 100644 index 00000000..73f35b69 --- /dev/null +++ b/docs/EVAL.md @@ -0,0 +1,187 @@ + + +> **Status:** design + intended usage for the `gently/eval/` capture/replay substrate and the +> proposed offline replay harness for testing agentic orchestrator patterns. Grounded in the +> code as of the 0.22 epoch; the harness itself is a work-in-progress (see the incremental plan). + +# Testing agentic orchestrator patterns offline (replay harness) + +## Goal + +We want to iterate on the agent's design — its realtime reasoning and the wake-router that +turns developmental events into autonomous turns — **without booking a live microscope run**. +Concretely: take a recorded session, simulate the microscope conditions from its on-disk +artifacts (captured events, recorded volumes, recorded perception traces), drive the *real* +`WakeRouter -> run_wake_turn -> Claude` loop offline, and observe/diff what the agent decides. +This lets us tune wake triggers, coalescing/throttling, prompt construction, and tool policy on +a laptop, replayed at a controllable clock (e.g. 10x), instead of waiting hours for embryos to +develop on a live rig. + +## What's already in place (reuse) + +A real, tested replay/eval substrate shipped in the 0.22 epoch (`gently/eval/`), plus production +capture wiring. None of this is hypothetical — it's on disk and runnable today: + +- **`gently/eval/event_capture.py`** — `EventCapture` wildcard-subscribes the bus and appends every + `Event.to_dict()` to `{session_dir}/events.jsonl`. Auto-wired into **every** live session by + `gently/app/agent.py` `_init_event_capture()` (line ~506, called at agent init). Skips only + `_NO_HISTORY_TYPES` (`DEVICE_STATE_UPDATE`, `BOTTOM_CAMERA_FRAME`, `LOG_RECORD`); `DETECTOR_EVALUATED` + and lifecycle events are **not** skipped. +- **`gently/eval/event_replay.py`** — `EventReplay(path).replay(target_bus, real_time=, time_scale=, on_event=)` + republishes each event via `target.publish_event(ev)`, **preserving the original `Event.timestamp`** + (not re-stamping `now()`). `real_time=True` sleeps `(ev.timestamp - prev)/time_scale` between events, + so cadence is reproducible. `event_types()` gives a pre-flight histogram. +- **`gently/eval/shadow.py` + `candidates.py`** — `ShadowRunner` + `OrchestratorCandidate` host + sandboxed rule-based candidates (e.g. `ReactiveCandidate`) that may *only* write a `DecisionLog`. +- **`gently/eval/decision_log.py`** — `Decision`/`DecisionLog` + `prompt_hash()` (sha256[:16] over + system prompt + messages) for apples-to-apples A/B diffing. +- **`scripts/replay_session.py`** — working CLI: resolves a session by id-prefix via + `FileStore.list_sessions`, prints `--histogram`, or replays `events.jsonl` into a **fresh** `EventBus()` + with an optional `NoOpCandidate`. +- **Recorded perception inputs/outputs** persist via `FileStore` (`gently/core/file_store.py`): + `embryos/{id}/volumes/t{NNNN}.tif` + `.meta.yaml`, `projections/t{NNNN}.jpg`, + `predictions.jsonl`, and `traces/t{NNNN}.json` (verbatim `predicted_stage`/`reasoning`/`raw_response`/`stability`). + Verified on disk: session `68e7dc33` has 9 embryos, 56 predictions on embryo_001, volume + `t0001.tif` shape `[50,512,2048]` uint16. +- **`timeline.jsonl`** (durable, predates eval) carries 256 `detection/evaluated` records on `68e7dc33` + with exactly the fields `WakeRouter._is_wake_worthy` reads (`embryo_id`, `timepoint`, + `detector_name`, `stage`, `reasoning`) — a fallback event source for pre-eval-scaffold sessions. + +### The one central wiring gap + +The real agent subscribes its `WakeRouter` to the **global singleton** bus +(`gently/app/agent.py:126` `self._event_bus = get_event_bus()`; WakeRouter built with that same bus). +But `scripts/replay_session.py:124` replays into a **fresh** `EventBus()` that the agent never sees. +**So today's replay reaches shadow candidates but never the real WakeRouter/agent.** Bridging this — +either `set_event_bus(replay_bus)` before constructing the agent, or replaying into `get_event_bus()` +directly — is the core seam to build. + +## Approaches, compared + +### (A) Event-stream replay into the agent's bus — *recommended first* +Publish recorded `DETECTOR_EVALUATED` + critical events (`HATCHING_DETECTED`, `EMBRYO_TERMINATED`, +`ERROR_OCCURRED`, …) onto the bus the agent's `WakeRouter` is subscribed to, on a controllable clock. + +- **Reuses:** `EventReplay`, `EventCapture` output, the entire real `WakeRouter` (`_is_wake_worthy` + filter at wake_router.py:129, coalesce `COALESCE_WINDOW=20s`, throttle `MIN_WAKE_INTERVAL=120s`, + `_flush -> agent.run_wake_turn`). +- **Fidelity:** Exercises the *real* wake path end-to-end: filtering, transition gate, coalescing, + throttling, prompt build, and a real Claude turn (`run_wake_turn -> handle_message_stream`, gated on + `agent.mode=='run'`). Highest leverage for the least new code. +- **Effort:** Medium. Needs (1) the bus bridge above; (2) a running asyncio loop so the async dispatch + + `loop.call_later` coalesce timers fire (`EventReplay.replay` is a blocking `time.sleep` loop — run it + in a thread or port it to `await asyncio.sleep`, and call `bus.set_event_loop(loop)`); (3) a stub + client so any tools the woken agent calls don't hit hardware (autonomous mode already refuses + irreversible tools via `_autonomous_active`). +- **Can't catch:** Anything depending on *fresh* perception of new pixels — the wake note embeds + `build_perception_snapshot(agent.perceiver, ...)`, which reads **live** Perceiver state, so this + approach needs (B) to make that snapshot reflect the replayed timepoint rather than empty state. +- **Blocker today:** No recorded session yet contains `DETECTOR_EVALUATED` (verified: all 20 captured + `events.jsonl` hold only `STATUS_CHANGED`/`EMBRYO_DETECTED`/`EMBRYOS_UPDATE`). Either capture one fresh + perception-driven session, or synthesize `DETECTOR_EVALUATED` events from `traces/`+`timeline.jsonl`. + +### (B) Perceiver stub — feed recorded traces +Replace `agent.perceiver`/`orchestrator.perceiver` with a duck-typed stub whose `__call__(...)` returns +`.stage`/`.reasoning` from `traces/t{NNNN}.json`, and whose `get_session(embryo_id)` returns an object +with `.stability`/`.summary()` matching what `build_perception_snapshot` reads +(`current_stage`/`stability`/`temporal`/`stage_sequence`). + +- **Reuses:** All downstream code in `_run_perception` (DETECTOR_EVALUATED emit, trace write, + `store_prediction`, `_check_interval_rules`) is pure local code; the Perceiver is the *only* external + VLM dependency. `perceiver` is already an optional ctor arg (timelapse.py:71). +- **Fidelity:** Reproduces recorded perception verbatim — no VLM spend, deterministic. Makes (A)'s wake + snapshot reflect the replayed timepoint. +- **Effort:** Low-medium (one stub class). +- **Can't catch:** Perception on *new* conditions — it only echoes what the recorded run already saw. + Also the stub interface was inferred from call sites (`templates.py` `build_perception_snapshot`, + `timelapse.py` `_run_perception`), not from `gently_perception` source — **verify against the + installed package** before relying on it. + +### (C) Full offline re-feed through the timelapse loop +Inject a fake `microscope_client` whose `acquire_volume(...)` returns +`{'success': True, 'volume': }` keyed by `(embryo_id, timepoint)`, +plus `move_to_position`, etc. `_has_microscope()` (`return self.client is not None`) then gates the +orchestrator *on*, driving the entire per-timepoint loop (acquire -> callback -> `_run_perception`). + +- **Reuses:** The whole `TimelapseOrchestrator`; `client` is a single ctor arg accessed only via named + async methods. +- **Fidelity:** Highest — exercises scheduling, acquisition callbacks, perception, event emission, and + wakes as one system. +- **Effort:** High. The orchestrator schedules entirely off `datetime.now()`/`asyncio.sleep` + (`_pick_next_due`, `_reschedule`, the acquire loop) and ignores `Event.timestamp` — a faithful + time-scaled run needs an **injectable clock** threaded through both the orchestrator and the + WakeRouter's wall-clock timers. There is also no helper to join recorded TIFFs to the event stream. +- **Can't catch:** Same perception-novelty limit as (B); plus device-state/camera-frame telemetry is + absent from `events.jsonl` and must be sourced from disk or synthesized. + +### (D) Shadow mode — score candidates / replay captured turns +Keep the existing `ShadowRunner` path: replay captured events into a bus with rule-based candidates +attached, diff `decisions.jsonl` (production) vs `replay-decisions-*.jsonl` (candidate) via `prompt_hash`. + +- **Reuses:** Fully built already (`scripts/replay_session.py --candidate`). +- **Fidelity:** Tests *alternative* (non-LLM) orchestrator architectures, not the production Claude agent. +- **Effort:** None (exists). +- **Can't catch:** The real agent's reasoning. Also: production only writes `Decision`s for **user turns** + (`conversation.py:343` hardcodes `trigger=DecisionTrigger.USER_MESSAGE`); wake turns via + `call_claude_stream` aren't logged as decisions, so there's currently no production wake-decision row to + diff against. + +## Honest fidelity limits + +- **Recorded perception ≠ new perception.** Approaches B/C echo `traces/`; they cannot evaluate the + Perceiver on conditions the original run didn't encounter. Genuinely testing perception requires a live + (or freshly captured) run. +- **LLM nondeterminism.** `run_wake_turn` makes a real Claude call; the same replayed input can yield + different tool calls run-to-run. `prompt_hash` isolates *input* identity but not *output* determinism — + diffs are about distributions/policy, not exact equality. +- **Clock vs coalesce/throttle.** `WakeRouter` uses real wall-clock `loop.call_later(COALESCE_WINDOW=20s)` + and `loop.time()`-based `MIN_WAKE_INTERVAL=120s` — these are **not** scaled by `time_scale`. A fast + replay collapses bursts into one wake; a high `time_scale` shrinks inter-event sleeps below the fixed + 20s window, again collapsing wakes. These tunables (currently module-level constants in + `wake_router.py:33-35`) must be parameterized/injectable for faithful timed replay. +- **Wall-clock reads break replay.** `TimelapseOrchestrator` (`timelapse.py`) drives scheduling off + `datetime.now()`/`asyncio.sleep` and never consults `Event.timestamp`; perception stamps + `timestamp=datetime.now()`. Any *new* events the woken agent emits use `publish()` (fresh `now()`), + intermixing replayed-historical and live-now timestamps on the same bus — a consistency hazard for + downstream diffing. +- **Telemetry gaps.** `EventCapture` skips `DEVICE_STATE_UPDATE`/`BOTTOM_CAMERA_FRAME`/`LOG_RECORD`, so a + replay can't reconstruct live device readouts or frames from `events.jsonl` (re-capture with + `EventCapture(path, skip=set())` or synthesize). +- **Data availability (verified).** No single recorded session yet combines a full timelapse with + non-trivial capture: `68e7dc33` has 9 embryos + volumes/traces but **no** `events.jsonl`; the newest + sessions have `events.jsonl` but 0 embryos and empty `decisions.jsonl`. All 20 captured `events.jsonl` + contain only setup-phase events — **zero** `DETECTOR_EVALUATED`. + +## Concrete incremental plan + +**Step 0 — Generate one good input stream (unblocks everything).** Either (a) run a single fresh +perception-driven session (live or with a stub client) after the eval-scaffold commit so +`events.jsonl` + non-empty `decisions.jsonl` coexist with volumes/traces; or (b) write a tiny +`synthesize_events.py` that emits `DETECTOR_EVALUATED` events from `68e7dc33`'s `traces/`+`timeline.jsonl` +into a synthetic `events.jsonl`. Validate with `python scripts/replay_session.py --histogram`. + +**Step 1 (smallest useful) — Bus bridge + offline driver skeleton.** New script +`scripts/replay_into_agent.py`: construct a `GentlyAgent` with a stub microscope client, call +`set_event_bus(replay_bus)` (or replay into `get_event_bus()`), set `agent.mode='run'` and +`wake_router.set_mode('ask')`, run an asyncio loop, `bus.set_event_loop(loop)`, and run +`EventReplay(...).replay(bus, real_time=True, time_scale=N)` in a thread. First milestone: a recorded +`DETECTOR_EVALUATED` actually fires `_on_event -> _flush -> run_wake_turn`. Reuses `EventReplay`, +`WakeRouter`, `run_wake_turn` unchanged. + +**Step 2 — Perceiver stub (B).** Add a `RecordedPerceiver` reading `traces/t{NNNN}.json`, injected via +`agent.perceiver`. Verify its `summary()`/result shape against the installed `gently_perception`. Now the +wake prompt's `build_perception_snapshot` reflects the replayed timepoint. + +**Step 3 — Injectable clock + parameterized tunables.** Thread a clock/`now()` provider through +`TimelapseOrchestrator` and make `COALESCE_WINDOW`/`MIN_WAKE_INTERVAL` injectable on `WakeRouter` so a +time-scaled replay reproduces the live wake set. Also scale or virtualize the loop timers. + +**Step 4 — Capture wake decisions + fix trigger labels.** Add `DecisionLog` capture to the +`call_claude_stream` (wake) path and emit `DecisionTrigger.EVENT` for wake turns (today `conversation.py` +only writes `USER_MESSAGE` decisions from `call_claude`). This makes replayed autonomous turns diffable. + +**Step 5 — Optional full re-feed (C).** Add a `RecordedMicroscopeClient` whose `acquire_volume` loads +`volumes/t{NNNN}.tif`, gating the orchestrator on for end-to-end loop testing. + +**Step 6 — Write `docs/EVAL.md`** (referenced as TODO in `gently/eval/__init__.py`) documenting the +replay workflow and fidelity tiers. diff --git a/docs/HEURISTICS-AUDIT.md b/docs/HEURISTICS-AUDIT.md new file mode 100644 index 00000000..249b27cc --- /dev/null +++ b/docs/HEURISTICS-AUDIT.md @@ -0,0 +1,112 @@ +# Heuristics audit — where to use the model (as a typed-output function) instead + +Codebase sweep (5 parallel scanners + synthesis) for heuristics that **fake +judgment** an LLM would do better — in the spirit of the genotype→channel +refactor (drop the lookup table, let the model infer, keep a typed provenance +record + confirm-when-unsure). The flip side — logic that **must stay +deterministic** (safety, math, calibration, transport) — is listed at the end so +we don't mistakenly LLM-ify it. + +The unifying move for every candidate: **LLM with a typed structured-output +schema + provenance + a confirm/UNCERTAIN escape**, never free-text-then-parse. + +## Model candidates (ranked) + +### High value + +1. **Hatching / time-to-stage prediction** — `organisms/celegans/developmental_tracker.py` + *(the closest twin of genotype→channel; medium effort)* + Three hardcoded 20 °C lookup tables (`STAGE_TIMING_20C`, `TIME_TO_HATCHING`, + `TIMING_VARIABILITY`) plus magic `{HIGH:1.0, MEDIUM:1.5, LOW:2.0}` uncertainty + fudge factors. Structurally **can't use the rig's actual temperature** (we run + a TEC), the strain, or the embryo's observed progression rate. Let the model + produce a calibrated, explained interval; **keep the literature table as a + deterministic sanity bracket** and flag when the estimate falls outside it. + → `{ predicted_minutes_to_hatching, low, high, basis, assumptions{temperature_c,strain,used_observed_rate}, confidence, reasoning }` + +2. **Citation → PubMed query** — `harness/plan_mode/tools/research.py` (`_search_pmid`) + A regex that only handles "Surname et al YEAR …" + six hand-rolled query- + relaxation strategies + a stopword/word-position ladder that drops load-bearing + nouns. The model parses the sloppy citation and proposes relaxed queries; **code + keeps the deterministic esearch call and never fabricates a PMID.** + → `{ author_last, year, journal, topic_keywords[], organism, pubmed_query, alt_queries[], confidence }` + +3. **Lab-history retrieval** — `harness/plan_mode/tools/lab_context.py`, `harness/memory/interface.py` + Semantic recall faked by substring-OR over query tokens (matches "we"/"before", + misses every paraphrase). Feed the model the candidate records and have it + **rank/select from provided ids only** (no fabrication). Read-only, no + acquisition risk. + → `{ matches:[{kind,id,summary,relevance,why_relevant}], answer }` + +4. **Stage-label parse via 22-entry synonym dict** — `developmental_tracker.py` (`_parse_stage_name`) + *(small effort, pure robustness win)* The Vision call already classifies; the + brittleness is a plain-text `STAGE:/CONFIDENCE:` block scraped line-by-line, with + off-vocabulary phrasings silently collapsing to `UNKNOWN` (which kills the + downstream hatching prediction). Constrained-enum structured output deletes the + parser + synonym table. + → `{ stage: enum(...), confidence: enum(high|medium|low), is_transitional, reasoning }` + +### Medium value (mostly small — fix the output contract, not the judgment) + +5. **Calibration Vision calls** — `hardware/dispim/claude_client.py` + Four Vision calls return positional free text recovered by `'yes' in first_line` + / `re.search(r'\d+')` / first-valid-letter, with silent defaults (so "no, this is + not yes…" reads as *yes*). Typed output deletes the parse + silent-default layer. + +6. **ML architecture ranking** — `ml/architectures.py` (`get_suitable_architectures`) + Hard feasibility gates (VRAM / dataset) are correct **and stay**; the `+2/+1/+1` + point-score ranking that follows discards the per-arch prose. Let the model rank + the *pre-filtered feasible set* (ids constrained to that set). + +7. **Training label normalization** — `ml/data_loader.py` (`build_labels_from_store`) + Class space built by exact-string identity over free-text human annotations — + "1.5-fold" and "1.5 fold" become different classes. Model normalizes to the + canonical staging vocabulary, flags novel/ambiguous ones. + +### Lower value + +8. **"Plan has a control?"** — `plan_mode/tools/validation.py` — substring scan of a + 6-word keyword set; a scientific judgment over the whole plan. Non-blocking + warning → safe for the model. +9. **CGC HTML scraping** — `research.py` (`_cgc_search`) — positional multi-group + regex over fetched HTML; structured extraction the model does better (HTTP GET + stays code; **mark strain names low-confidence to avoid sending someone to order + a hallucinated strain**). + +### Cross-cutting batch (small each): typed output for the detector/verifier cluster +`harness/detection/verifier.py`, `app/detectors/hatching.py`, +`app/detectors/dopaminergic_signal.py`, `hardware/dispim/sam_detection.py` — all +already make the right model call but reconstruct the verdict via +`startswith`/regex-JSON-scraping with silent defaults. A batch move to native +structured output **strictly reduces parse-induced false negatives** without +touching the deterministic vote-tally/consensus/enum-dispatch downstream. + +**Reference implementations already in the repo (imitate, don't change):** +`dopaminergic_signal`'s perceiver→classifier rubric (typed enums, UNCERTAIN +escape, conservative-on-tie) and onboarding's `_extract_with_llm` (typed +extraction, degrade-to-verbatim fallback). + +## Keep deterministic (do NOT LLM-ify) +Safety, math, calibration, and transport — where a hallucinated value is unsafe +or breaks reproducibility: +- Laser-power safety limits + wavelength→MM-property map (`hardware/dispim/devices/optical.py`) +- SPIM trigger-timing arithmetic, piezo–galvo calibration, MM framing (`dispim/config.py`) +- Calibration prior EMA + R²≥0.75 slope-lock gate (`dispim/calibration.py`) +- SwitchBot GATT byte commands / status decoding (`hardware/switchbot.py`) +- Temperature setpoint bound [0,99.9] °C + stabilization I/O (`hardware/temperature.py`) +- Autofocus signal-processing, curve fitting, adaptive-sweep stop rules (`analysis/core.py`, `analysis/focus.py`) +- Classical-CV ROI detection + pixel→stage coordinate transforms (`detection.py`, `sam_detection.py` geometry) +- Timelapse rule dispatch + `confirm_timepoints` debounce + monotonic power ramp (`app/orchestration/timelapse.py`) +- Volume→b64 dark/flat calibration + fixed brightness scaling (`dopaminergic_signal._volume_to_b64` — deliberately non-adaptive) +- Wake-router debounce/throttle/stage-transition gate (`app/wake_router.py`) +- Plan hardware limits, detector-preset membership, dependency-cycle DFS, stage-order normalization (`plan_mode/tools/validation.py`) +- Ensemble vote tally + 0.70 quorum / unanimity consensus (`detection/verifier.py`) +- ML metric/aggregation math: confusion matrix, F1, federated averaging (`ml/evaluation.py`, `federated.py`) +- Core imaging geometry (max-projection, crop bounds, Euler rotations) + UI event reduction/routing/security (`core/imaging.py`, `ui/web/*`) +- Device-state SSE watchdog/staleness timers (`app/device_state_monitor.py`) +- Reference-type dispatch (PMID/DOI/URL by canonical syntax), `os.path.isfile` checks (`research.py`) + +## Note +`gap_assessment.conversation_weight` (the 0.25/0.1/0.05 readiness scalar) is now +largely **vestigial** — it only returns 'heavy' (lab onboarding) or 'none' — so +it's not worth an API call. Left off the candidate list. diff --git a/docs/HIERARCHICAL_CONTEXTS_DESIGN.md b/docs/HIERARCHICAL_CONTEXTS_DESIGN.md index e0903b08..3a11479b 100644 --- a/docs/HIERARCHICAL_CONTEXTS_DESIGN.md +++ b/docs/HIERARCHICAL_CONTEXTS_DESIGN.md @@ -106,27 +106,27 @@ class ContextScope: scope_id: str parent_id: Optional[str] scope_type: Literal["root", "focus", "timelapse", "embryo_perception", "ad_hoc"] - purpose: str # one-line task description - instructions: str # full brief, like a prompt to a colleague + purpose: str # one-line task description + instructions: str # full brief, like a prompt to a colleague conversation_history: List[Dict] allowed_tools: Set[str] - model: str # "opus" | "sonnet" | "haiku" + model: str # "opus" | "sonnet" | "haiku" status: Literal["active", "completed", "yielded", "cancelled", "failed"] - summary: Optional[str] # produced by Summarizer at end - key_findings: Dict[str, Any] # structured return value - children: List[str] # child scope_ids + summary: Optional[str] # produced by Summarizer at end + key_findings: Dict[str, Any] # structured return value + children: List[str] # child scope_ids created_at: datetime completed_at: Optional[datetime] - persist: bool # write to SQLite if True + persist: bool # write to SQLite if True # Runtime - inbox: asyncio.Queue # incoming messages from other scopes - wakeup: asyncio.Event # signaled when there is work + inbox: asyncio.Queue # incoming messages from other scopes + wakeup: asyncio.Event # signaled when there is work cancel_token: asyncio.Event cancel_reason: Optional[str] pending_queries: Dict[str, asyncio.Future] - peer_events: List[str] # event types this scope subscribes to - parent_snapshot_cache: Dict # parent's last known summary + findings + peer_events: List[str] # event types this scope subscribes to + parent_snapshot_cache: Dict # parent's last known summary + findings ``` A `ScopeManager` (lives on `MicroscopyCopilot`) owns the scope tree, dispatches the inner agent loop, and exposes the orchestrator-facing tools. @@ -198,22 +198,26 @@ async def run_scope(scope: ContextScope) -> SummaryResult: while not scope.inbox.empty(): items.append(scope.inbox.get_nowait()) if not items and not scope.cancel_token.is_set(): - continue # nothing to do, back to sleep, no API call + continue # nothing to do, back to sleep, no API call # CHECK CANCELLATION — give child one final turn to summarize if scope.cancel_token.is_set() and scope.status != "cancelling": scope.status = "cancelling" - scope.conversation_history.append({ - "role": "user", - "content": f"[CANCELLATION] {scope.cancel_reason}. " - f"Emit a final summary using `complete` and stop.", - }) + scope.conversation_history.append( + { + "role": "user", + "content": f"[CANCELLATION] {scope.cancel_reason}. " + f"Emit a final summary using `complete` and stop.", + } + ) # APPEND injected messages to history - scope.conversation_history.append({ - "role": "user", - "content": format_inbox_items(items), - }) + scope.conversation_history.append( + { + "role": "user", + "content": format_inbox_items(items), + } + ) # ONE LLM TURN response = await call_claude( @@ -222,23 +226,32 @@ async def run_scope(scope: ContextScope) -> SummaryResult: messages=scope.conversation_history, tools=tool_registry.filter(scope.allowed_tools), ) - scope.conversation_history.append({"role":"assistant","content":response}) + scope.conversation_history.append({"role": "assistant", "content": response}) # HANDLE TOOL CALLS if response.stop_reason == "tool_use": tool_results = [] for tool_call in response.tool_uses: match tool_call.name: - case "delegate": result = await spawn_and_run(scope, **tool_call.input) - case "yield_checkpoint": result = await emit_checkpoint(scope, **tool_call.input) - case "escalate": result = await escalate_to_root(scope, **tool_call.input) - case "query_parent": result = await ask_parent(scope, **tool_call.input, timeout=30) - case "read_parent_state": result = scope.parent_snapshot_cache - case "respond_to_query": result = await deliver_answer(**tool_call.input) - case "complete": scope.status = "completed"; scope.key_findings = tool_call.input - case _: result = await execute_tool(tool_call, scope) + case "delegate": + result = await spawn_and_run(scope, **tool_call.input) + case "yield_checkpoint": + result = await emit_checkpoint(scope, **tool_call.input) + case "escalate": + result = await escalate_to_root(scope, **tool_call.input) + case "query_parent": + result = await ask_parent(scope, **tool_call.input, timeout=30) + case "read_parent_state": + result = scope.parent_snapshot_cache + case "respond_to_query": + result = await deliver_answer(**tool_call.input) + case "complete": + scope.status = "completed" + scope.key_findings = tool_call.input + case _: + result = await execute_tool(tool_call, scope) tool_results.append(result) - scope.conversation_history.append({"role":"user","content":tool_results}) + scope.conversation_history.append({"role": "user", "content": tool_results}) if response.stop_reason == "end_turn": scope.status = "completed" diff --git a/docs/PERCEPTION_V2_IMPROVEMENT_STRATEGY.md b/docs/PERCEPTION_V2_IMPROVEMENT_STRATEGY.md index 2844265f..b6b6ada7 100644 --- a/docs/PERCEPTION_V2_IMPROVEMENT_STRATEGY.md +++ b/docs/PERCEPTION_V2_IMPROVEMENT_STRATEGY.md @@ -77,7 +77,7 @@ MISSING: hatching/ folder - NO HATCHING EXAMPLES! ```python def at_least_stage(self, stage: str) -> bool: """Check if embryo has reached at least the given stage.""" - order = ['early', 'bean', 'comma', '1.5fold', '2fold', '3fold', 'hatching', 'hatched'] + order = ["early", "bean", "comma", "1.5fold", "2fold", "3fold", "hatching", "hatched"] # ... simple linear comparison ``` @@ -155,28 +155,38 @@ Create a single source of truth: from enum import Enum from typing import List, Dict, Tuple + class DevelopmentalStage(str, Enum): """Unified C. elegans developmental stages.""" - EARLY = "early" # Gastrulation, ~100+ cells, oval shape - BEAN = "bean" # Early morphogenesis, slight asymmetry - COMMA = "comma" # Clear C-shape, head/tail distinguishable - FOLD_1_5 = "1.5fold" # Elongation, ~1.5x eggshell - FOLD_2 = "2fold" # Further elongation, folding back - FOLD_3 = "3fold" # Tight coil, maximum compaction (pretzel) - HATCHING = "hatching" # Active emergence, shell breach visible - HATCHED = "hatched" # Fully emerged L1 larva + + EARLY = "early" # Gastrulation, ~100+ cells, oval shape + BEAN = "bean" # Early morphogenesis, slight asymmetry + COMMA = "comma" # Clear C-shape, head/tail distinguishable + FOLD_1_5 = "1.5fold" # Elongation, ~1.5x eggshell + FOLD_2 = "2fold" # Further elongation, folding back + FOLD_3 = "3fold" # Tight coil, maximum compaction (pretzel) + HATCHING = "hatching" # Active emergence, shell breach visible + HATCHED = "hatched" # Fully emerged L1 larva @classmethod - def ordered_list(cls) -> List['DevelopmentalStage']: - return [cls.EARLY, cls.BEAN, cls.COMMA, cls.FOLD_1_5, - cls.FOLD_2, cls.FOLD_3, cls.HATCHING, cls.HATCHED] + def ordered_list(cls) -> List["DevelopmentalStage"]: + return [ + cls.EARLY, + cls.BEAN, + cls.COMMA, + cls.FOLD_1_5, + cls.FOLD_2, + cls.FOLD_3, + cls.HATCHING, + cls.HATCHED, + ] @classmethod - def get_order(cls, stage: 'DevelopmentalStage') -> int: + def get_order(cls, stage: "DevelopmentalStage") -> int: return cls.ordered_list().index(stage) @classmethod - def is_terminal(cls, stage: 'DevelopmentalStage') -> bool: + def is_terminal(cls, stage: "DevelopmentalStage") -> bool: return stage == cls.HATCHED @@ -339,15 +349,18 @@ from dataclasses import dataclass from typing import List, Optional, Tuple from .stages import DevelopmentalStage + @dataclass class TransitionState: """Represents the current transition state.""" + current_stage: DevelopmentalStage confidence: float is_transitioning: bool next_stage: Optional[DevelopmentalStage] transition_progress: float # 0.0 to 1.0 + class TransitionDetector: """ Detects and tracks stage transitions with temporal smoothing. @@ -481,12 +494,14 @@ from dataclasses import dataclass from typing import Dict from .stages import DevelopmentalStage + @dataclass class CalibratedConfidence: """Calibrated confidence with interpretable thresholds.""" - raw_score: float # 0.0 to 1.0 from VLM - calibrated_score: float # Adjusted based on stage difficulty - interpretation: str # "high", "medium", "low" + + raw_score: float # 0.0 to 1.0 from VLM + calibrated_score: float # Adjusted based on stage difficulty + interpretation: str # "high", "medium", "low" @property def is_reliable(self) -> bool: @@ -496,14 +511,14 @@ class CalibratedConfidence: # Stage-specific confidence calibration # Some stages are harder to classify, adjust thresholds accordingly STAGE_DIFFICULTY = { - DevelopmentalStage.EARLY: 0.0, # Easy - distinct morphology - DevelopmentalStage.BEAN: 0.3, # Hard - brief, subtle - DevelopmentalStage.COMMA: 0.2, # Medium - can overlap with bean - DevelopmentalStage.FOLD_1_5: 0.2, # Medium - DevelopmentalStage.FOLD_2: 0.2, # Medium - DevelopmentalStage.FOLD_3: 0.1, # Easy - distinct pretzel + DevelopmentalStage.EARLY: 0.0, # Easy - distinct morphology + DevelopmentalStage.BEAN: 0.3, # Hard - brief, subtle + DevelopmentalStage.COMMA: 0.2, # Medium - can overlap with bean + DevelopmentalStage.FOLD_1_5: 0.2, # Medium + DevelopmentalStage.FOLD_2: 0.2, # Medium + DevelopmentalStage.FOLD_3: 0.1, # Easy - distinct pretzel DevelopmentalStage.HATCHING: 0.25, # Medium-hard - brief window - DevelopmentalStage.HATCHED: 0.0, # Easy - distinct + DevelopmentalStage.HATCHED: 0.0, # Easy - distinct } @@ -587,7 +602,7 @@ Return JSON: class HatchingDetector: """Specialized detector for hatching stages.""" - def __init__(self, engine: 'PerceptionEngine'): + def __init__(self, engine: "PerceptionEngine"): self.engine = engine self.consecutive_hatching = 0 self.consecutive_hatched = 0 @@ -604,16 +619,16 @@ class HatchingDetector: ) # Track consecutive detections for confirmation - if result.get('classification') == 'hatching': + if result.get("classification") == "hatching": self.consecutive_hatching += 1 self.consecutive_hatched = 0 - elif result.get('classification') == 'hatched': + elif result.get("classification") == "hatched": self.consecutive_hatched += 1 # Require 2+ consecutive "hatched" to confirm # (hatching can look like hatched momentarily) if self.consecutive_hatched < 2: - result['classification'] = 'hatching' - result['note'] = 'Awaiting confirmation of hatched status' + result["classification"] = "hatching" + result["note"] = "Awaiting confirmation of hatched status" else: self.consecutive_hatching = 0 self.consecutive_hatched = 0 diff --git a/docs/PLAN_MODE_DESIGN.md b/docs/PLAN_MODE_DESIGN.md index ddf02c05..9e95e34d 100644 --- a/docs/PLAN_MODE_DESIGN.md +++ b/docs/PLAN_MODE_DESIGN.md @@ -79,8 +79,8 @@ The agent gets a `self.mode` attribute: ```python class AgentMode(str, Enum): - EXECUTION = "execution" # Current behavior - PLAN = "plan" # Experimental design mode + EXECUTION = "execution" # Current behavior + PLAN = "plan" # Experimental design mode ``` The main message loop (`_call_claude_stream()`) selects prompt and tools based on mode: @@ -323,23 +323,24 @@ The core tracking unit. Every task in the plan — imaging or not — is a PlanI @dataclass class PlanItem: """A single item in an experimental plan.""" + id: str - campaign_id: str # Which campaign/phase - type: str # imaging, bench, genetics, analysis, decision_point - title: str # "Pilot — rab-3p::GFP visibility test" - description: Optional[str] = None # Detailed notes, protocols, what to watch for - status: str = "planned" # planned → in_progress → completed | skipped | blocked + campaign_id: str # Which campaign/phase + type: str # imaging, bench, genetics, analysis, decision_point + title: str # "Pilot — rab-3p::GFP visibility test" + description: Optional[str] = None # Detailed notes, protocols, what to watch for + status: str = "planned" # planned → in_progress → completed | skipped | blocked depends_on: List[str] = field(default_factory=list) # PlanItem IDs - outcome: Optional[str] = None # What happened (filled after completion) + outcome: Optional[str] = None # What happened (filled after completion) # Specifications (type-dependent) - imaging_spec: Optional[ImagingSpec] = None # if type == "imaging" - bench_spec: Optional[BenchSpec] = None # if type in (bench, genetics, analysis) + imaging_spec: Optional[ImagingSpec] = None # if type == "imaging" + bench_spec: Optional[BenchSpec] = None # if type in (bench, genetics, analysis) # Linking - planned_session_id: Optional[str] = None # → PlannedSession (for imaging items) - session_id: Optional[str] = None # → Actual session (once executed) - inherit_from: Optional[str] = None # PlanItem ID to inherit spec from + planned_session_id: Optional[str] = None # → PlannedSession (for imaging items) + session_id: Optional[str] = None # → Actual session (once executed) + inherit_from: Optional[str] = None # PlanItem ID to inherit spec from # Ordering phase_order: int = 0 @@ -358,34 +359,34 @@ class ImagingSpec: """Complete specification for a planned imaging session.""" # ── Sample ────────────────────────────────────────── - strain: Optional[str] = None # "OH904" - genotype: Optional[str] = None # "otIs355[rab-3p::2xNLS::TagRFP]" - reporter: Optional[str] = None # "rab-3p::GFP (pan-neuronal)" - sample_prep: Optional[str] = None # "Standard egg prep, poly-lysine pads" - temperature_c: Optional[float] = None # 20.0 - num_embryos: Optional[int] = None # 4 + strain: Optional[str] = None # "OH904" + genotype: Optional[str] = None # "otIs355[rab-3p::2xNLS::TagRFP]" + reporter: Optional[str] = None # "rab-3p::GFP (pan-neuronal)" + sample_prep: Optional[str] = None # "Standard egg prep, poly-lysine pads" + temperature_c: Optional[float] = None # 20.0 + num_embryos: Optional[int] = None # 4 # ── Acquisition ───────────────────────────────────── - num_slices: Optional[int] = None # 80 - exposure_ms: Optional[float] = None # 10.0 - laser_wavelength_nm: Optional[int] = None # 488 - laser_power_pct: Optional[float] = None # 10.0 - galvo_amplitude: Optional[float] = None # 8.0 + num_slices: Optional[int] = None # 80 + exposure_ms: Optional[float] = None # 10.0 + laser_wavelength_nm: Optional[int] = None # 488 + laser_power_pct: Optional[float] = None # 10.0 + galvo_amplitude: Optional[float] = None # 8.0 piezo_amplitude_um: Optional[float] = None # 50.0 # ── Timing ────────────────────────────────────────── - interval_s: Optional[int] = None # 180 + interval_s: Optional[int] = None # 180 adaptive_intervals: Optional[Dict[str, int]] = None # e.g. {"early_to_comma": 300, "comma_to_2fold": 60, "after_2fold": 180} # ── Developmental Window ──────────────────────────── - target_window: Optional[str] = None # "comma → pretzel" - start_stage: Optional[str] = None # "comma" - stop_condition: Optional[str] = None # "pretzel" + target_window: Optional[str] = None # "comma → pretzel" + start_stage: Optional[str] = None # "comma" + stop_condition: Optional[str] = None # "pretzel" estimated_duration_h: Optional[float] = None # 4.0 # ── Detection ─────────────────────────────────────── - detectors: Optional[List[str]] = None # ["comma", "pretzel"] + detectors: Optional[List[str]] = None # ["comma", "pretzel"] pre_terminal_speedup: Optional[bool] = None # True # ── Validation ────────────────────────────────────── @@ -403,12 +404,13 @@ Specification for non-imaging tasks (bench work, genetics, analysis). @dataclass class BenchSpec: """Specification for bench/genetics/analysis tasks.""" - protocol: Optional[str] = None # "Standard chemotaxis assay" - reagents: Optional[List[str]] = None # ["anti-UNC-33", "secondary 568"] - strains: Optional[List[str]] = None # ["OH904", "N2"] - target_genotype: Optional[str] = None # "unc-6(ev400); otIs355" - estimated_days: Optional[int] = None # 14 - success_criteria: Optional[str] = None # "Homozygous GFP+ line established" + + protocol: Optional[str] = None # "Standard chemotaxis assay" + reagents: Optional[List[str]] = None # ["anti-UNC-33", "secondary 568"] + strains: Optional[List[str]] = None # ["OH904", "N2"] + target_genotype: Optional[str] = None # "unc-6(ev400); otIs355" + estimated_days: Optional[int] = None # 14 + success_criteria: Optional[str] = None # "Homozygous GFP+ line established" notes: Optional[str] = None ``` diff --git a/docs/README.md b/docs/README.md index 598ebf6a..a7e1878b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -112,6 +112,7 @@ Create `gently/mmcore_wrapper.py` using the template in the Java-MMCore Interfac ```python from gently.mmcore_wrapper import GentlyProperties, GentlyDevices, DeviceKeys, PropertyKeys + def volume_scan_plan(core, num_slices=100, slice_step_um=0.5): # Initialize devices = GentlyDevices(core) diff --git a/docs/TOOLS.md b/docs/TOOLS.md index 6d9e020c..feafbfbe 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -13,7 +13,7 @@ Source: `gently/agent/tools/` (run mode) and `gently/agent/plan_mode/tools/` (pl |------|-------------| | `acquire_volume` | Acquire a single 3D lightsheet volume for a specific embryo with calibration data | | `capture_lightsheet` | Capture a single 2D lightsheet fluorescence image at specified piezo/galvo position | -| `batch_lightsheet` | Capture lightsheet images from ALL embryos and display as a stack | +| `batch_lightsheet` | Capture lightsheet images from ALL embryos and show them in the web UI viewer | ### Analysis (`analysis_tools.py`) @@ -29,22 +29,13 @@ Source: `gently/agent/tools/` (run mode) and `gently/agent/plan_mode/tools/` (pl | `calibrate_embryo` | Run full piezo-galvo calibration for a specific embryo using Claude vision | | `calibrate_all_embryos` | Run piezo-galvo calibration for all detected embryos sequentially | -### Data (`data_tools.py`) - -| Tool | Description | -|------|-------------| -| `list_runs` | List recent Bluesky runs from Databroker | -| `get_run_data` | Get data from a specific Bluesky run | -| `get_run_image` | Get an image from a Bluesky run for analysis | -| `search_runs` | Search Databroker runs by metadata criteria | - ### Detection (`detection_tools.py`) | Tool | Description | |------|-------------| | `detect_embryos` | Automatically detect embryos using brightness detection and SAM segmentation | | `manual_mark_embryos` | Open interactive window to manually mark embryos by clicking | -| `edit_embryos` | Open napari editor to add/remove/move embryo positions | +| `edit_embryos` | Add/remove/move embryo positions in the web map view | | `show_detected_embryos` | Capture fresh image and display all tracked embryos with labeled bounding boxes | ### Detectors (`detector_tools.py`) @@ -145,7 +136,7 @@ Source: `gently/agent/tools/` (run mode) and `gently/agent/plan_mode/tools/` (pl | Tool | Description | |------|-------------| | `view_image` | Capture and display current bottom camera widefield image | -| `view_volume` | Open a volume in napari for 3D visualization | +| `view_volume` | Open a volume in the in-browser 3D viewer | | `list_volumes` | List available volumes for an embryo or all embryos | --- diff --git a/docs/architecture/hardware-profile-template.md b/docs/architecture/hardware-profile-template.md new file mode 100644 index 00000000..bee826df --- /dev/null +++ b/docs/architecture/hardware-profile-template.md @@ -0,0 +1,97 @@ +# Hardware Profile Template + +Use this template when adding or documenting a new hardware profile. + +## Profile Identity + +- Profile name: +- Primary sample type: +- Primary acquisition modality: +- Device-layer entry point: +- Configuration file(s): + +## Standard Operations + +| Operation | Supported | Implementation | Notes | +| --- | --- | --- | --- | +| `sample_overview` | yes/no | | | +| `detect_samples` | yes/no | | | +| `move_to_sample` | yes/no | | | +| `focus_scan` | yes/no | | | +| `position_calibration` | yes/no | | | +| `acquire_snapshot` | yes/no | | | +| `acquire_volume` | yes/no | | | +| `set_illumination` | yes/no | | | +| `read_device_state` | yes/no | | | + +## Coordinate Frames + +List every coordinate frame exposed to the agent or UI. + +| Frame | Units | Origin | Axes | Conversion owner | +| --- | --- | --- | --- | --- | +| overview pixels | px | | | | +| stage | | | | | +| acquisition volume | voxels | | | | + +Required notes: + +- Which frame is stored in `position_coarse`. +- Which frame is stored in `position_fine`. +- Whether any axis is inverted relative to the overview image. +- Which code owns pixel-to-stage transforms. + +## Safety Limits + +| Device/axis | Lower | Upper | Enforcement layer | How verified | +| --- | --- | --- | --- | --- | +| | | | device `set()` | | +| | | | firmware | | + +Every motion source should have a lowest-layer safety limit. If joystick, +manual controls, or vendor UI can bypass Python checks, document the firmware or +operator procedure that closes that path. + +## State Reporting + +| State field | Source | Poll/callback | Frequency | Read-only | +| --- | --- | --- | --- | --- | +| stage position | | | | yes | +| focus actuator | | | | yes | +| illumination state | | | | yes | +| temperature | | | | yes | + +State reporting endpoints must not mutate hardware. + +## Data Products + +| Product | Format | Storage owner | Metadata required | +| --- | --- | --- | --- | +| overview image | | | sample id, position, pixel size | +| snapshot | | | sample id, channel, exposure | +| volume | | | sample id, voxel size, channel, exposure | + +Large arrays should be written by the device layer or persisted once, then +referenced by uid/path. Avoid repeated JSON/base64 transfer for production data. + +## Sample Metrics Populated + +Check the metrics this profile updates: + +- [ ] `position_coarse` +- [ ] `position_fine` +- [ ] `position_history` +- [ ] `exposure_count` +- [ ] `total_exposure_ms` +- [ ] `focus_history` +- [ ] `signal_intensity_history` +- [ ] `perception_runs` + +## Test Plan + +- [ ] Offline import smoke test +- [ ] Mock device unit tests +- [ ] Hardware availability diagnostic +- [ ] Safety limit rejection test +- [ ] Acquisition dry-run or simulated run +- [ ] Live acquisition test with explicit operator opt-in diff --git a/docs/architecture/sample-hardware-domains.md b/docs/architecture/sample-hardware-domains.md new file mode 100644 index 00000000..8b8ab4b6 --- /dev/null +++ b/docs/architecture/sample-hardware-domains.md @@ -0,0 +1,104 @@ +# Sample and Hardware Domain Model + +Gently's current production profile is a diSPIM imaging C. elegans embryos, but +the control model should be described in terms that also fit other microscopes +and sample types. This document defines that vocabulary. + +## Design Rule + +Application code should talk about samples, observations, acquisitions, and +calibration. Hardware modules translate those concepts to device-specific +plans, device names, and timing details. + +The current `Embryo` state is therefore an implementation of a more general +sample state. It should remain useful for C. elegans while keeping fields and +operations that other sample models can reuse. + +## Concept Mapping + +| Gently concept | Current diSPIM implementation | Other modality examples | +| --- | --- | --- | +| Sample | C. elegans embryo | well, organoid, cell colony, tissue region | +| Sample overview | Bottom-camera widefield image | low-mag confocal tile, plate overview, brightfield montage | +| Sample detection | Embryo detection/marking | well detection, cell segmentation, ROI picking | +| Position calibration | Pixel-to-stage and SPIM alignment | tile registration, well-to-stage map, objective-specific alignment | +| Focus scan | Piezo/galvo focus sweep | Z-stack focus curve, autofocus objective sweep | +| 3D acquisition | Lightsheet volume | confocal Z-stack, widefield deconvolution stack | +| Timepoint | One scheduled sample observation | one image, stack, or multimodal acquisition at a sample/time | +| Perception | VLM/classifier stage reasoning | phenotype call, quality control, event detection | + +## Standard Operation Names + +Hardware profiles should expose conceptually named operations even when their +device implementation is modality-specific. + +| Operation name | Meaning | diSPIM backing operation | +| --- | --- | --- | +| `sample_overview` | Capture a field that can locate samples | bottom camera capture | +| `detect_samples` | Produce sample candidates/ROIs | embryo detector or manual marking | +| `move_to_sample` | Move the instrument to a sample's resolved position | XY stage move | +| `focus_scan` | Search focus around a sample or plane | piezo/galvo focus sweep | +| `position_calibration` | Refine sample position/calibration state | center/verify and SPIM calibration | +| `acquire_volume` | Acquire a 3D observation | lightsheet volume scan | +| `acquire_snapshot` | Acquire a 2D observation | lightsheet snap or overview image | +| `set_illumination` | Configure light source state/power | laser, LED, room light controls | +| `read_device_state` | Report physical state without mutation | device-layer status endpoints | + +Names in tool descriptions, plan metadata, logs, and docs should prefer these +general concepts. The hardware package may still contain files named for the +real devices (`bottom_camera`, `galvo`, `piezo`) because those are implementation +details inside the diSPIM profile. + +## Layering + +1. `gently.core` defines storage, events, coordinates, and image utilities. +2. `gently.harness` defines reusable agent, session, tool, prompt, and planning + mechanics. +3. `gently.organisms` defines biological semantics, such as C. elegans stages. +4. `gently.hardware` defines device profiles and maps standard operations to + concrete devices/plans. +5. `gently.app` composes one organism and one hardware profile into the + microscopy agent. + +Domain state should move upward as structured sample records. Raw device +details should stay in the hardware profile unless the UI is explicitly showing +hardware diagnostics. + +## Plan Naming Convention + +Use `*_plan` for Bluesky/device-layer plans and include the conceptual operation +in the metadata when possible: + +```python +_md = { + "plan_name": "acquire_volume", + "operation": "acquire_volume", + "hardware_profile": "dispim", +} +``` + +Recommended naming: + +| Preferred | Avoid for new public API | Reason | +| --- | --- | --- | +| `sample_overview_plan` | `bottom_camera_plan` | overview generalizes beyond diSPIM | +| `focus_scan_plan` | `piezo_sweep_plan` | focus is the user-facing concept | +| `position_calibration_plan` | `center_embryo_plan` | calibration can apply to many samples | +| `acquire_volume_plan` | `lightsheet_only_plan` | volume acquisition is modality-neutral | + +Existing diSPIM-specific names do not need churn. New public tools, docs, and +metadata should use the conceptual names and point to the diSPIM implementation. + +## Extension Checklist + +A new hardware profile should document: + +- Which device or plan provides `sample_overview`. +- How sample coordinates map to stage coordinates. +- Which acquisition operations are supported: snapshot, volume, timelapse, + multichannel, burst. +- Which state readings are available without moving hardware. +- Which safety limits are enforced in hardware/device classes. +- Which sample tracking metrics are populated by the profile. + +Use `docs/architecture/hardware-profile-template.md` as the starting point. diff --git a/docs/architecture/sample-tracking-metrics.md b/docs/architecture/sample-tracking-metrics.md new file mode 100644 index 00000000..2421d44b --- /dev/null +++ b/docs/architecture/sample-tracking-metrics.md @@ -0,0 +1,122 @@ +# Sample Tracking Metrics + +Sample tracking metrics describe what happened to a sample over time. They are +kept separate from organism-specific interpretation so they can support embryos, +wells, cells, organoids, or tissue regions. + +## Metric Categories + +| Category | Purpose | Examples | +| --- | --- | --- | +| Position | Where the sample was observed | coarse XY, fine XY, position history | +| Exposure | How much light/acquisition burden it received | exposure count, total exposure ms | +| Focus | Whether optical focus is stable | focus history, drift rate | +| Signal | Quantitative image measurements | channel intensity, photobleaching curve | +| Perception | Semantic interpretations | developmental stage, health state, event flags | +| Provenance | How the record was produced | detector id, model version, confidence | + +## Universal Fields + +These fields are broadly useful across sample types and should remain stable +where possible. + +| Field | Type | Description | +| --- | --- | --- | +| `sample_id` | string | Stable id within a session. Current C. elegans code uses `embryo_id`. | +| `sample_uid` | string/null | Optional globally meaningful id. Current code uses `uid`/`embryo_uid`. | +| `role` | string | Experimental role such as test, calibration, control, unassigned. | +| `position_coarse` | object/null | Overview/manual position, usually XY stage coordinates. | +| `position_fine` | object/null | Refined acquisition position for the primary objective/modality. | +| `has_fine_position` | bool | True when fine position should override coarse for acquisition. | +| `position_history` | list | Optional time series of positions and sources. | +| `exposure_count` | integer | Number of acquisitions or exposure events. | +| `total_exposure_ms` | number | Integrated illumination/exposure time. | +| `last_imaged` | ISO datetime/null | Last successful observation time. | +| `focus_history` | list | Focus measurements keyed by position, modality, and score. | +| `signal_intensity_history` | object/list | Per-channel measurements over time. | +| `perception_runs` | list | Links to semantic classifications and reasoning traces. | + +## Domain-Specific Fields + +Domain fields are valuable, but should be clearly scoped to an organism/sample +plugin. + +| Field | Current meaning | Scope | +| --- | --- | --- | +| `developmental_stage` | C. elegans morphology stage | C. elegans organism profile | +| `hatching_status` | C. elegans hatch state | C. elegans organism profile | +| `morphology_history` | Stage/shape observations | organism-specific | +| `custom_classifications` | User-defined labels | experiment-specific | + +## Position Semantics + +Gently distinguishes coarse and fine positions: + +- `position_coarse` comes from an overview image, manual marking, or low-mag + sample map. +- `position_fine` comes from a refined alignment step for the acquisition + modality. +- `stage_position` is a compatibility/read convenience: fine if present, + otherwise coarse. + +Persistence and import/export code should preserve both positions. Updating the +coarse position should clear or invalidate fine position unless the update is +known not to affect fine alignment. + +## Exposure Semantics + +Exposure tracking should describe burden on the sample, not simply image count. + +Minimum record: + +```yaml +sample_id: embryo_1 +timepoint: 12 +modality: lightsheet_volume +frames: 50 +exposure_ms: 10.0 +channels: + 488nm: + laser_power_pct: 5.0 + total_ms: 500.0 +created_at: 2026-05-30T12:00:00 +``` + +For multimodal acquisitions, store one record per channel/modality or include a +structured `channels` map. Do not collapse illumination wavelength and camera +exposure into a single ambiguous number. + +## Focus Semantics + +Focus measurements should include: + +- acquisition modality or objective, +- stage position/context, +- focus actuator position, +- score algorithm, +- score value, +- timestamp, +- source: hardware autofocus, FFT, VLM, manual. + +This allows drift estimates such as micrometers/hour without tying the schema to +one microscope. + +## Perception Provenance + +Every semantic decision should be traceable: + +- model or detector id, +- input image/volume uid, +- prompt or detector config hash when applicable, +- confidence or score, +- reasoning trace path when available, +- timestamp and triggering event. + +This is the bridge between sample tracking and the evaluation/trajectory +debugging systems. + +## Compatibility Notes + +Current `EmbryoState` fields map directly onto this schema. Future sample types +can either implement their own typed state object or reuse a generic +`SampleState` shape, provided API responses keep the universal fields above. diff --git a/docs/asidispim_camera_triggering.md b/docs/asidispim_camera_triggering.md index bfcf30bd..7d12ab91 100644 --- a/docs/asidispim_camera_triggering.md +++ b/docs/asidispim_camera_triggering.md @@ -129,6 +129,7 @@ for i in range(num_slices): img = core.popNextImage() try: import rpyc + img = rpyc.classic.obtain(img) except: pass @@ -252,19 +253,20 @@ core.setProperty(galvo_device, "SingleAxisYMode", "3 - Enabled with axes synced" ```python # TRIGGER_SOURCE property "INTERNAL" # For live/snap mode + "EXTERNAL" # For hardware-triggered acquisition # SENSOR_MODE property -"AREA" # Standard split readout (top/bottom simultaneous) +"AREA" # Standard split readout (top/bottom simultaneous) "PROGRESSIVE" # Rolling shutter for light sheet (slower but better for SPIM) # TRIGGER_ACTIVE property -"EDGE" # Edge trigger - single pulse starts exposure -"LEVEL" # Level trigger - TTL high duration = exposure time -"SYNCREADOUT" # Overlap mode - synchronous readout +"EDGE" # Edge trigger - single pulse starts exposure +"LEVEL" # Level trigger - TTL high duration = exposure time +"SYNCREADOUT" # Overlap mode - synchronous readout # TriggerPolarity property -"POSITIVE" # Trigger on rising edge (required) +"POSITIVE" # Trigger on rising edge (required) ``` **Configuration Sequence (from Cameras.java:231-260):** @@ -299,9 +301,10 @@ core.setExposure(camera_name, 10.0) # milliseconds ```python # Triggermode property -"Internal" # For live/snap mode -"External" # Edge trigger mode -"External Exp. Ctrl." # Level trigger mode +"Internal" # For live/snap mode + +"External" # Edge trigger mode +"External Exp. Ctrl." # Level trigger mode # PixelRate property "slow scan" # Slower pixel readout, higher quality @@ -327,17 +330,18 @@ core.setProperty(camera_name, "Triggermode", "External Exp. Ctrl.") ```python # TriggerMode property "Internal (Recommended for fast acquisitions)" # Live mode -"External" # Edge trigger -"External Exposure" # Level trigger + +"External" # Edge trigger +"External Exposure" # Level trigger # Overlap property -"On" # Overlap mode enabled +"On" # Overlap mode enabled "Off" # Standard mode # LightScanPlus-SensorReadoutMode property "Centre Out Simultaneous" # Standard split readout -"Bottom Up Sequential" # Rolling shutter for light sheet -"Bottom Up Simultaneous" # Split readout from bottom +"Bottom Up Sequential" # Rolling shutter for light sheet +"Bottom Up Simultaneous" # Split readout from bottom ``` **Configuration Sequence (from Cameras.java:284-331):** @@ -362,10 +366,11 @@ core.setProperty(camera_name, "Overlap", "Off") ```python # TriggerMode property "Internal Trigger" # Live mode -"Edge Trigger" # Hardware triggered + +"Edge Trigger" # Hardware triggered # ClearMode property -"Never" # No clearing between frames +"Never" # No clearing between frames "Pre-Exposure" # Clear before each exposure "Pre-Sequence" # Clear once before sequence ``` @@ -389,43 +394,43 @@ All properties are set on the **Micro-mirror (Galvo) card device**: galvo_device = "Scanner:AB:33" # Your device name # SPIM State Machine Control -"SPIMState" # Values: "Idle", "Armed", "Running" -"SPIMNumSlices" # Number of slices per side (e.g., 100) -"SPIMNumSides" # 1 = single side, 2 = dual view -"SPIMFirstSide" # "A" or "B" - which side starts +"SPIMState" # Values: "Idle", "Armed", "Running" +"SPIMNumSlices" # Number of slices per side (e.g., 100) +"SPIMNumSides" # 1 = single side, 2 = dual view +"SPIMFirstSide" # "A" or "B" - which side starts # SPIM Timing Properties (ALL IN MILLISECONDS, 0.25ms resolution) -"SPIMDelayBeforeScan(ms)" # Delay before scan mirror starts -"SPIMScanDuration(ms)" # Scan mirror sweep duration -"SPIMDelayBeforeLaser(ms)" # Delay before laser trigger -"SPIMLaserDuration(ms)" # Laser TTL pulse width -"SPIMDelayBeforeCamera(ms)" # Delay before camera trigger -"SPIMCameraDuration(ms)" # Camera TTL pulse width ← CRITICAL! +"SPIMDelayBeforeScan(ms)" # Delay before scan mirror starts +"SPIMScanDuration(ms)" # Scan mirror sweep duration +"SPIMDelayBeforeLaser(ms)" # Delay before laser trigger +"SPIMLaserDuration(ms)" # Laser TTL pulse width +"SPIMDelayBeforeCamera(ms)" # Delay before camera trigger +"SPIMCameraDuration(ms)" # Camera TTL pulse width ← CRITICAL! # SPIM Multi-Acquisition Properties -"SPIMNumRepeats" # Volumes per trigger (for hardware timepoints) -"SPIMDelayBeforeRepeat(ms)" # Delay between volumes -"SPIMDelayBeforeSide(ms)" # Delay between sides (for dual view) -"SPIMNumScansPerSlice" # Usually 1 -"SPIMNumSlicesPerPiezo" # For multichannel slice-by-slice -"SPIMAlternateDirectionsEnable" # "Yes" or "No" -"SPIMInterleaveSidesEnable" # For interleaved stage scan -"SPIMPiezoHomeDisable" # For stage scan mode +"SPIMNumRepeats" # Volumes per trigger (for hardware timepoints) +"SPIMDelayBeforeRepeat(ms)" # Delay between volumes +"SPIMDelayBeforeSide(ms)" # Delay between sides (for dual view) +"SPIMNumScansPerSlice" # Usually 1 +"SPIMNumSlicesPerPiezo" # For multichannel slice-by-slice +"SPIMAlternateDirectionsEnable" # "Yes" or "No" +"SPIMInterleaveSidesEnable" # For interleaved stage scan +"SPIMPiezoHomeDisable" # For stage scan mode # Scan Mirror Properties -"SingleAxisXAmplitude(deg)" # Light sheet width (X-axis) -"SingleAxisXOffset(deg)" # Light sheet position offset -"SingleAxisXPattern" # "0 - Ramp", "1 - Triangle" -"SingleAxisXMode" # "0 - Disabled", "1 - Enabled", "3 - Enabled with axes synced" +"SingleAxisXAmplitude(deg)" # Light sheet width (X-axis) +"SingleAxisXOffset(deg)" # Light sheet position offset +"SingleAxisXPattern" # "0 - Ramp", "1 - Triangle" +"SingleAxisXMode" # "0 - Disabled", "1 - Enabled", "3 - Enabled with axes synced" -"SingleAxisYAmplitude(deg)" # Slice stepping amplitude (Y-axis) -"SingleAxisYOffset(deg)" # Slice position offset -"SingleAxisYPattern" # "0 - Ramp", "1 - Triangle" -"SingleAxisYMode" # "0 - Disabled", "3 - Enabled with axes synced" +"SingleAxisYAmplitude(deg)" # Slice stepping amplitude (Y-axis) +"SingleAxisYOffset(deg)" # Slice position offset +"SingleAxisYPattern" # "0 - Ramp", "1 - Triangle" +"SingleAxisYMode" # "0 - Disabled", "3 - Enabled with axes synced" # Critical Output Configuration -"LaserOutputMode" # "shutter + side" ← MUST BE SET FOR TRIGGERS! -"BeamEnabled" # "Yes" or "No" - disable during SPIM acquisition +"LaserOutputMode" # "shutter + side" ← MUST BE SET FOR TRIGGERS! +"BeamEnabled" # "Yes" or "No" - disable during SPIM acquisition ``` ### Piezo (Z-drive) Properties @@ -435,12 +440,12 @@ galvo_device = "Scanner:AB:33" # Your device name piezo_device = "Piezo:A:37" # Your device name # Piezo Sweep Properties -"SPIMState" # "Idle", "Armed" ← Must arm before galvo trigger -"SPIMNumSlices" # Number of Z positions -"SA_AMPLITUDE" # Sweep amplitude in micrometers -"SA_OFFSET" # Center position in micrometers -"SA_PATTERN" # "0 - Ramp", "1 - Triangle" -"SA_MODE_Z" # "0 - Disabled", "1 - Enabled", "3 - Enabled with axes synced" +"SPIMState" # "Idle", "Armed" ← Must arm before galvo trigger +"SPIMNumSlices" # Number of Z positions +"SA_AMPLITUDE" # Sweep amplitude in micrometers +"SA_OFFSET" # Center position in micrometers +"SA_PATTERN" # "0 - Ramp", "1 - Triangle" +"SA_MODE_Z" # "0 - Disabled", "1 - Enabled", "3 - Enabled with axes synced" ``` ### PLogic Card Properties (Optional) @@ -450,9 +455,9 @@ piezo_device = "Piezo:A:37" # Your device name plogic_device = "PLogic:E:36" # Your device name # PLogic Control -"PLogicMode" # "Disp. Seq. positions" -"PLogicOutputChannel" # "6,7" for lasers on BNC6 & BNC7 -"PoLogicPreset" # "3 - cell 1 high" during acquisition +"PLogicMode" # "Disp. Seq. positions" +"PLogicOutputChannel" # "6,7" for lasers on BNC6 & BNC7 +"PoLogicPreset" # "3 - cell 1 high" during acquisition # Note: PLogic adds 0.25ms delay to all TTL outputs ``` @@ -510,9 +515,14 @@ def configure_camera_for_hardware_trigger(core, camera_name, camera_mode="EDGE", ### Phase 2: Timing Calculation ```python -def calculate_spim_timing(camera_exposure_ms, camera_reset_ms, camera_readout_ms, - scan_laser_buffer_ms=0.25, scan_filter_freq_khz=0.2, - has_plogic=False): +def calculate_spim_timing( + camera_exposure_ms, + camera_reset_ms, + camera_readout_ms, + scan_laser_buffer_ms=0.25, + scan_filter_freq_khz=0.2, + has_plogic=False, +): """ Calculate SPIM timing parameters following ASI diSPIM plugin logic. @@ -527,6 +537,7 @@ def calculate_spim_timing(camera_exposure_ms, camera_reset_ms, camera_readout_ms Returns: Dictionary of timing parameters """ + # Round to 0.25ms (Tiger controller resolution) def round_quarter_ms(val): return round(val * 4) / 4.0 @@ -547,14 +558,16 @@ def calculate_spim_timing(camera_exposure_ms, camera_reset_ms, camera_readout_ms scan_delay_filter -= 0.25 # PLogic adds 0.25ms delay timing = { - 'scanDelay': global_exposure_delay_max - scan_laser_buffer_ms - scan_delay_filter, - 'scanPeriod': scan_duration, - 'laserDelay': global_exposure_delay_max, - 'laserDuration': laser_duration, - 'cameraDelay': camera_readout_max, - 'cameraDuration': 1.0, # Short pulse for EDGE mode - 'cameraExposure': camera_exposure_ms + 0.1, # Add safety margin - 'sliceDuration': max(scan_duration, laser_duration, camera_readout_max + camera_exposure_ms) + "scanDelay": global_exposure_delay_max - scan_laser_buffer_ms - scan_delay_filter, + "scanPeriod": scan_duration, + "laserDelay": global_exposure_delay_max, + "laserDuration": laser_duration, + "cameraDelay": camera_readout_max, + "cameraDuration": 1.0, # Short pulse for EDGE mode + "cameraExposure": camera_exposure_ms + 0.1, # Add safety margin + "sliceDuration": max( + scan_duration, laser_duration, camera_readout_max + camera_exposure_ms + ), } # Round all values to 0.25ms @@ -567,9 +580,9 @@ def calculate_spim_timing(camera_exposure_ms, camera_reset_ms, camera_readout_ms ### Phase 3: Tiger Controller Configuration ```python -def configure_tiger_controller_for_spim(core, galvo_device, piezo_device, - num_slices=100, num_sides=1, first_side_a=True, - timing=None): +def configure_tiger_controller_for_spim( + core, galvo_device, piezo_device, num_slices=100, num_sides=1, first_side_a=True, timing=None +): """ Configure Tiger controller SPIM state machine. @@ -626,12 +639,12 @@ def configure_tiger_controller_for_spim(core, galvo_device, piezo_device, # ⚠️ CRITICAL: Set ALL timing properties explicitly if timing: - core.setProperty(galvo_device, "SPIMDelayBeforeScan(ms)", timing['scanDelay']) - core.setProperty(galvo_device, "SPIMScanDuration(ms)", timing['scanPeriod']) - core.setProperty(galvo_device, "SPIMDelayBeforeLaser(ms)", timing['laserDelay']) - core.setProperty(galvo_device, "SPIMLaserDuration(ms)", timing['laserDuration']) - core.setProperty(galvo_device, "SPIMDelayBeforeCamera(ms)", timing['cameraDelay']) - core.setProperty(galvo_device, "SPIMCameraDuration(ms)", timing['cameraDuration']) + core.setProperty(galvo_device, "SPIMDelayBeforeScan(ms)", timing["scanDelay"]) + core.setProperty(galvo_device, "SPIMScanDuration(ms)", timing["scanPeriod"]) + core.setProperty(galvo_device, "SPIMDelayBeforeLaser(ms)", timing["laserDelay"]) + core.setProperty(galvo_device, "SPIMLaserDuration(ms)", timing["laserDuration"]) + core.setProperty(galvo_device, "SPIMDelayBeforeCamera(ms)", timing["cameraDelay"]) + core.setProperty(galvo_device, "SPIMCameraDuration(ms)", timing["cameraDuration"]) else: # Use safe defaults core.setProperty(galvo_device, "SPIMDelayBeforeScan(ms)", 0.0) @@ -749,7 +762,7 @@ def wait_for_images(core, camera_name, num_expected, timeout_sec=30.0): img = core.popNextImage() img = rpyc.classic.obtain(img) # For rpyc remote objects images.append(img) - print(f" Image {i+1}/{count}: shape={img.shape}, range=[{img.min()}, {img.max()}]") + print(f" Image {i + 1}/{count}: shape={img.shape}, range=[{img.min()}, {img.max()}]") return images ``` @@ -757,8 +770,9 @@ def wait_for_images(core, camera_name, num_expected, timeout_sec=30.0): ### Complete Acquisition Function ```python -def acquire_spim_volume(core, camera_name, galvo_device, piezo_device, - num_slices=100, camera_exposure_ms=5.0): +def acquire_spim_volume( + core, camera_name, galvo_device, piezo_device, num_slices=100, camera_exposure_ms=5.0 +): """ Complete hardware-triggered SPIM volume acquisition. @@ -775,16 +789,16 @@ def acquire_spim_volume(core, camera_name, galvo_device, piezo_device, """ try: # Phase 1: Configure camera - configure_camera_for_hardware_trigger(core, camera_name, - camera_mode="EDGE", - exposure_ms=camera_exposure_ms) + configure_camera_for_hardware_trigger( + core, camera_name, camera_mode="EDGE", exposure_ms=camera_exposure_ms + ) # Phase 2: Calculate timing timing = calculate_spim_timing( camera_exposure_ms=camera_exposure_ms, - camera_reset_ms=3.0, # Hamamatsu Flash4 typical - camera_readout_ms=10.0, # Depends on ROI and scan mode - has_plogic=True + camera_reset_ms=3.0, # Hamamatsu Flash4 typical + camera_readout_ms=10.0, # Depends on ROI and scan mode + has_plogic=True, ) print("\nCalculated timing:") @@ -792,11 +806,15 @@ def acquire_spim_volume(core, camera_name, galvo_device, piezo_device, print(f" {key}: {val} ms") # Phase 3: Configure Tiger controller - configure_tiger_controller_for_spim(core, galvo_device, piezo_device, - num_slices=num_slices, - num_sides=1, - first_side_a=True, - timing=timing) + configure_tiger_controller_for_spim( + core, + galvo_device, + piezo_device, + num_slices=num_slices, + num_sides=1, + first_side_a=True, + timing=timing, + ) # Phase 4: Start camera sequence start_camera_sequence(core, camera_name, num_slices) @@ -805,7 +823,7 @@ def acquire_spim_volume(core, camera_name, galvo_device, piezo_device, trigger_spim_acquisition(core, galvo_device) # Phase 6: Wait for images - expected_time = num_slices * timing['sliceDuration'] / 1000.0 + expected_time = num_slices * timing["sliceDuration"] / 1000.0 timeout = expected_time * 2 + 10.0 images = wait_for_images(core, camera_name, num_slices, timeout) @@ -1143,7 +1161,7 @@ print(f" SPIMDelayBeforeCamera(ms): {core.getProperty(galvo_device, 'SPIMDelayB print(f" SPIMCameraDuration(ms): {core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')}") # CRITICAL CHECK -camera_duration = float(core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')) +camera_duration = float(core.getProperty(galvo_device, "SPIMCameraDuration(ms)")) if camera_duration <= 0: print("\n⚠️ WARNING: SPIMCameraDuration is 0 - NO TRIGGERS WILL BE GENERATED!") print(" You must set this property to > 0 (typically 1.0 ms)") @@ -1187,7 +1205,7 @@ print(f" SA_AMPLITUDE: {core.getProperty(piezo_device, 'SA_AMPLITUDE')} µm") print(f" SA_OFFSET: {core.getProperty(piezo_device, 'SA_OFFSET')} µm") # Piezo should be Armed before galvo is set to Running -piezo_state = core.getProperty(piezo_device, 'SPIMState') +piezo_state = core.getProperty(piezo_device, "SPIMState") assert piezo_state == "Armed", f"Piezo should be Armed, not {piezo_state}" ``` @@ -1369,7 +1387,7 @@ piezo_device = "Piezo:A:37" num_slices = 100 camera_exposure_ms = 5.0 # Light exposure time -camera_reset_ms = 3.0 # Hamamatsu Flash4 typical +camera_reset_ms = 3.0 # Hamamatsu Flash4 typical camera_readout_ms = 10.0 # Depends on ROI print("=" * 70) @@ -1412,6 +1430,7 @@ try: def ceil_quarter_ms(val): import math + return math.ceil(val * 4) / 4.0 camera_readout_max = ceil_quarter_ms(camera_readout_ms) @@ -1496,9 +1515,11 @@ try: core.setProperty(galvo_device, "SPIMDelayBeforeRepeat(ms)", 0.0) # Verify timing properties are set - print(f" SPIMCameraDuration(ms): {core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')} ← MUST BE > 0!") + print( + f" SPIMCameraDuration(ms): {core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')} ← MUST BE > 0!" + ) - camera_duration_check = float(core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')) + camera_duration_check = float(core.getProperty(galvo_device, "SPIMCameraDuration(ms)")) if camera_duration_check <= 0: raise Exception("SPIMCameraDuration is 0 - triggers will not be generated!") @@ -1547,13 +1568,14 @@ try: # Step 9: Retrieve images count = core.getRemainingImageCount() - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") if count >= num_slices: print(f"✓ SUCCESS! Acquired {count} images") - print(f"{'='*70}") + print(f"{'=' * 70}") import rpyc + images = [] print(f"\nRetrieving {count} images...") for i in range(count): @@ -1561,26 +1583,29 @@ try: img = rpyc.classic.obtain(img) images.append(img) if i < 5 or i >= count - 5: # Print first and last 5 - print(f" Image {i+1}: shape={img.shape}, range=[{img.min()}, {img.max()}]") + print(f" Image {i + 1}: shape={img.shape}, range=[{img.min()}, {img.max()}]") volume = np.array(images) print(f"\nVolume shape: {volume.shape}") # Save as TIFF from PIL import Image + img_list = [Image.fromarray(img.astype(np.uint16)) for img in images] - img_list[0].save('spim_volume_fixed.tif', save_all=True, append_images=img_list[1:]) + img_list[0].save("spim_volume_fixed.tif", save_all=True, append_images=img_list[1:]) print(f"Saved to: spim_volume_fixed.tif") else: print(f"✗ FAILED - Got {count}/{num_slices} images") - print(f"{'='*70}") + print(f"{'=' * 70}") # Diagnostic output print("\nDiagnostics:") print(f" SPIMState: {core.getProperty(galvo_device, 'SPIMState')}") print(f" LaserOutputMode: {core.getProperty(galvo_device, 'LaserOutputMode')}") - print(f" SPIMCameraDuration(ms): {core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')}") + print( + f" SPIMCameraDuration(ms): {core.getProperty(galvo_device, 'SPIMCameraDuration(ms)')}" + ) print(f" Camera trigger: {core.getProperty(camera_name, 'TRIGGER SOURCE')}") print("\nPossible issues:") print(" - Check physical BNC cable connection to camera") @@ -1589,9 +1614,9 @@ try: finally: # Cleanup - print(f"\n{'='*70}") + print(f"\n{'=' * 70}") print("CLEANUP") - print(f"{'='*70}") + print(f"{'=' * 70}") try: if core.isSequenceRunning(camera_name): diff --git a/docs/asidispim_piezo_scanning_technical_report.md b/docs/asidispim_piezo_scanning_technical_report.md index 2570b495..89260f6b 100644 --- a/docs/asidispim_piezo_scanning_technical_report.md +++ b/docs/asidispim_piezo_scanning_technical_report.md @@ -500,6 +500,7 @@ def configure_piezo_for_scan(piezo_device, start_um, end_um, num_slices): core.waitForDevice(piezo_device) time.sleep(0.5) # Allow settling + # CALIBRATION (Lines 99-115) def piezo_to_galvo_y(piezo_pos_um, slope=100.306, offset=4.102): """Convert piezo position to galvo Y-axis angle.""" @@ -755,15 +756,15 @@ def configure_piezo_for_scan(piezo_device, start_um, end_um, num_slices): print(f" ✓ Piezo configured and armed for SPIM scanning") return { - 'start_um': start_um, - 'end_um': end_um, - 'center_um': piezo_center, - 'amplitude_um': piezo_amplitude, - 'step_size_um': step_size, - 'galvo_y_start': galvo_y_start, - 'galvo_y_end': galvo_y_end, - 'galvo_y_center': galvo_y_center, - 'galvo_y_amplitude': galvo_y_amplitude, + "start_um": start_um, + "end_um": end_um, + "center_um": piezo_center, + "amplitude_um": piezo_amplitude, + "step_size_um": step_size, + "galvo_y_start": galvo_y_start, + "galvo_y_end": galvo_y_end, + "galvo_y_center": galvo_y_center, + "galvo_y_amplitude": galvo_y_amplitude, } ``` @@ -773,16 +774,15 @@ Update the main acquisition function to use the calculated galvo values: ```python # Step 5: Configure piezo -piezo_config = configure_piezo_for_scan(PIEZO_DEVICE, piezo_start_um, - piezo_end_um, num_slices) +piezo_config = configure_piezo_for_scan(PIEZO_DEVICE, piezo_start_um, piezo_end_um, num_slices) # Step 6: Configure Tiger controller with synchronized galvo Y-axis configure_tiger_controller( GALVO_DEVICE, num_slices, timing, - galvo_y_amp=piezo_config['galvo_y_amplitude'], - galvo_y_offset=piezo_config['galvo_y_center'] + galvo_y_amp=piezo_config["galvo_y_amplitude"], + galvo_y_offset=piezo_config["galvo_y_center"], ) ``` diff --git a/docs/full-stack-microscopy.md b/docs/full-stack-microscopy.md new file mode 100644 index 00000000..0188968a --- /dev/null +++ b/docs/full-stack-microscopy.md @@ -0,0 +1,55 @@ +# Full Stack Microscopy + +Gently should be documented as a microscopy system, not only as an agent or a +device controller. The useful view for a biologist or instrument developer is +the full path from experimental intent to stored evidence. + +## Stack Map + +| Stack | What it owns | Gently surface | +| --- | --- | --- | +| Experimental intent | Scientific question, hypothesis, controls, success criteria | plan mode, campaigns, plan items | +| Sample preparation | Organism, strain, treatment, mounting, perturbation | sample records, sample-tracking metrics | +| Hardware integration | Microscope devices, safety limits, device state, calibration | hardware profiles, device layer, profile templates | +| Acquisition | Snapshots, volumes, timepoints, illumination, temperature | acquisition tools, Bluesky plans, session metadata | +| Perception | Detection, classification, quality control, event recognition | perception traces, predictions, reasoning records | +| Closed-loop decisions | When to continue, stop, adapt, or ask the operator | agent tools, event logs, decision logs | +| Data and provenance | Raw data, derived data, logs, plans, exports | FileStore/GentlyStore, session directories, debug bundles | +| Operator experience | Setup, monitoring, intervention, recovery | web UI, chat, settings, docs/tutorials | + +## Documentation Shape + +Generated docs should include three kinds of material: + +- Tutorials: task-focused paths such as "run without hardware", "add a hardware + profile", and "start a safe timelapse". +- Concepts: the full-stack map, sample/hardware domain boundaries, and data + provenance expectations. +- References: API surfaces, command-line flags, storage layouts, hardware + profile checklists, and test markers. + +## Hardware as One Stack + +Hardware is a core stack, but it should not dominate the documentation model. +The device layer matters because it connects intent to physical state safely: +limits, calibration, timing, illumination, and temperature all shape what +scientific claims can be made from the data. + +Hardware docs should therefore connect each device profile to: + +- the sample state it can observe or change +- the safety boundaries it enforces +- the metadata it records +- the simulator or live-hardware tests that cover it +- the operator workflow for setup and recovery + +## Tutorial Roadmap + +Priority tutorials: + +- run Gently offline and create a plan +- connect a local diSPIM device layer +- add a new hardware profile +- add a new organism/sample type +- inspect a stored session and export a debug bundle +- write a hardware contract test and an opt-in live hardware test diff --git a/docs/guides/build-a-plugin.md b/docs/guides/build-a-plugin.md index ce3ca4bd..d6c11872 100644 --- a/docs/guides/build-a-plugin.md +++ b/docs/guides/build-a-plugin.md @@ -25,14 +25,14 @@ Three protocols define the plugin contracts. All are in `gently/harness/protocol ```python @runtime_checkable class OrganismProtocol(Protocol): - ORGANISM_NAME: str # e.g. "drosophila" - ORGANISM_DISPLAY_NAME: str # e.g. "Drosophila melanogaster" - SAMPLE_TERM: str # e.g. "embryo", "cell", "organoid" - SAMPLE_TERM_PLURAL: str # e.g. "embryos" - STAGES: list # Developmental stages (ordered) - TERMINAL_STAGES: set # e.g. {"hatched"} - BIOLOGY_KNOWLEDGE: str # Markdown for LLM context - PERCEPTION_SYSTEM_PROMPT: str # VLM classification prompt + ORGANISM_NAME: str # e.g. "drosophila" + ORGANISM_DISPLAY_NAME: str # e.g. "Drosophila melanogaster" + SAMPLE_TERM: str # e.g. "embryo", "cell", "organoid" + SAMPLE_TERM_PLURAL: str # e.g. "embryos" + STAGES: list # Developmental stages (ordered) + TERMINAL_STAGES: set # e.g. {"hatched"} + BIOLOGY_KNOWLEDGE: str # Markdown for LLM context + PERCEPTION_SYSTEM_PROMPT: str # VLM classification prompt ``` ### HardwareProtocol @@ -40,10 +40,10 @@ class OrganismProtocol(Protocol): ```python @runtime_checkable class HardwareProtocol(Protocol): - HARDWARE_NAME: str # e.g. "twophoton" - HARDWARE_DISPLAY_NAME: str # e.g. "Two-Photon Microscope" - HARDWARE_DESCRIPTION: str # Markdown capabilities text - CAPABILITIES: set # e.g. {"xy_stage", "z_stack", "fluorescence"} + HARDWARE_NAME: str # e.g. "twophoton" + HARDWARE_DISPLAY_NAME: str # e.g. "Two-Photon Microscope" + HARDWARE_DESCRIPTION: str # Markdown capabilities text + CAPABILITIES: set # e.g. {"xy_stage", "z_stack", "fluorescence"} ``` Standard capability names: `xy_stage`, `z_control`, `volume`, `snap`, `z_stack`, `dual_view`, `autofocus`, `detection`, `fluorescence`, `transmitted`. @@ -80,6 +80,7 @@ def create_device_layer(config: dict): """Create the hardware control server. Returns a server with .run(port=N).""" ... + def create_client(http_url: str): """Create an HTTP client for the device layer. Returns a client with .connect().""" ... @@ -105,8 +106,10 @@ gently/organisms/drosophila/ # gently/organisms/drosophila/stages.py from enum import Enum + class DevelopmentalStage(str, Enum): """Drosophila embryo developmental stages.""" + SYNCYTIAL = "syncytial" CELLULARIZATION = "cellularization" GASTRULATION = "gastrulation" @@ -119,6 +122,7 @@ class DevelopmentalStage(str, Enum): ARRESTED = "arrested" NO_OBJECT = "no_object" + # Ordered list for the perception engine STAGES = list(DevelopmentalStage) @@ -298,14 +302,16 @@ CAPABILITIES = { def create_device_layer(config: dict): """Create the 2P device layer server.""" from .device_layer import TwoPhotonDeviceLayer + return TwoPhotonDeviceLayer( - config_path=config.get('config_path', 'config/config.yml'), + config_path=config.get("config_path", "config/config.yml"), ) def create_client(http_url: str): """Create an HTTP client for the 2P device layer.""" from .client import TwoPhotonClient + return TwoPhotonClient(http_url=http_url) ``` @@ -318,24 +324,26 @@ Each hardware type has its own calibration model. For 2P, it's simpler than diSP from dataclasses import dataclass from typing import Optional + @dataclass class TwoPhotonCalibration: """Z-axis calibration for a two-photon microscope.""" - z_top: float = 0.0 # Top of sample (µm) - z_bottom: float = 100.0 # Bottom of sample (µm) - optimal_z: float = 50.0 # Best focal plane (µm) + + z_top: float = 0.0 # Top of sample (µm) + z_bottom: float = 100.0 # Bottom of sample (µm) + optimal_z: float = 50.0 # Best focal plane (µm) optimal_power: float = 10.0 # Laser power (mW) def to_dict(self) -> dict: return { - 'z_top': self.z_top, - 'z_bottom': self.z_bottom, - 'optimal_z': self.optimal_z, - 'optimal_power': self.optimal_power, + "z_top": self.z_top, + "z_bottom": self.z_bottom, + "optimal_z": self.optimal_z, + "optimal_power": self.optimal_power, } @classmethod - def from_dict(cls, data: dict) -> 'TwoPhotonCalibration': + def from_dict(cls, data: dict) -> "TwoPhotonCalibration": return cls(**{k: data[k] for k in cls.__dataclass_fields__ if k in data}) ``` @@ -370,16 +378,14 @@ Tools are registered with the `@tool` decorator from `gently/harness/tools/regis ```python from gently.harness.tools.registry import tool, ToolCategory, ToolExample + @tool( name="measure_wing_disc", description="Measure the size of a wing imaginal disc in the current image", category=ToolCategory.ANALYSIS, requires_microscope=False, examples=[ - ToolExample( - "Measure the wing disc in embryo 3", - {"embryo_id": "embryo_3"} - ), + ToolExample("Measure the wing disc in embryo 3", {"embryo_id": "embryo_3"}), ], ) async def measure_wing_disc( @@ -404,13 +410,12 @@ Hardware-specific tools (acquisition, calibration, focus) should live alongside ```python @tool(name="acquire_zstack", requires_microscope=True) -async def acquire_zstack(embryo_id: str, num_planes: int = 50, - z_step_um: float = 1.0, context: dict = None) -> str: +async def acquire_zstack( + embryo_id: str, num_planes: int = 50, z_step_um: float = 1.0, context: dict = None +) -> str: client = context.get("client") # This tool only works with a 2P client - result = await client.acquire_zstack( - num_planes=num_planes, z_step_um=z_step_um - ) + result = await client.acquire_zstack(num_planes=num_planes, z_step_um=z_step_um) ... ``` @@ -452,13 +457,13 @@ The harness provides a generic `FocusDataPoint` for tracking focus measurements: ```python @dataclass class FocusDataPoint: - z: float # Primary focus axis (µm) + z: float # Primary focus axis (µm) secondary_axis: float # Second axis (galvo for diSPIM, 0.0 for single-axis) - score: float # Focus quality - r_squared: float # Fit quality (0-1) + score: float # Focus quality + r_squared: float # Fit quality (0-1) timestamp: datetime - method: str # 'calibration', 'fine_focus', 'manual' - algorithm: str # 'fft_bandpass', 'gradient', etc. + method: str # 'calibration', 'fine_focus', 'manual' + algorithm: str # 'fft_bandpass', 'gradient', etc. ``` For single-axis systems (2P, confocal), set `secondary_axis=0.0`. The harness tracks focus history per-embryo and provides drift analysis and interpolation. diff --git a/docs/guides/capabilities.md b/docs/guides/capabilities.md index d3b097c7..846d662c 100644 --- a/docs/guides/capabilities.md +++ b/docs/guides/capabilities.md @@ -133,7 +133,6 @@ This design means experimental AI code — perception systems, coding agents, no | **Analysis** | analyze_volume, classify_embryo_stage | No | | **Experiment** | get_experiment_summary, query_embryo_status | No | | **Session** | list_sessions, import_embryos_from_session | No | -| **Data** | list_runs, get_run_data, search_runs | No | | **Planning** | create_campaign, propose_plan, search_literature | No | | **Research** | search_literature, read_paper, search_strains | No | diff --git a/docs/guides/try-offline.md b/docs/guides/try-offline.md index 50ca6e65..e4341291 100644 --- a/docs/guides/try-offline.md +++ b/docs/guides/try-offline.md @@ -4,29 +4,38 @@ Get the agent running in 10 minutes — no microscope needed. ## Prerequisites -- **Python 3.11+** -- **Node.js 18+** (for the terminal UI) +- **Python 3.10+** - An **Anthropic API key** (`ANTHROPIC_API_KEY` environment variable) +Gently is web-first — the agent runs in your browser, so there's no terminal UI to build (no Node.js needed for the app). + ## Install ```bash git clone https://github.com/pskeshu/gently.git cd gently -pip install -r requirements.txt +``` -# Build the TUI (one-time) -cd gently/tui -npm install -npm run build -cd ../.. +Create an environment and install — **either path works**: + +```bash +# venv + pip +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -e . +``` + +```bash +# or uv (https://docs.astral.sh/uv/) +uv venv +uv pip install -e . ``` ## Launch ```bash -export ANTHROPIC_API_KEY=sk-ant-... -python launch_gently.py --offline +export ANTHROPIC_API_KEY=sk-ant-... # Windows: set ANTHROPIC_API_KEY=sk-ant-... +python launch_gently.py --offline # uv (no activate): uv run python launch_gently.py --offline ``` The `--offline` flag skips the hardware connection. The full agent launches — conversation, perception, plan mode, memory — just without microscope control. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..08b8b6a6 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,31 @@ +# Gently Documentation + +Gently is a full-stack microscopy system: it joins sample preparation, +instrument control, perception, planning, storage, and operator workflows into +one inspectable loop. + +This documentation is organized around that full stack rather than around a +single API layer. + +## Start Here + +- [Full Stack Microscopy](full-stack-microscopy.md): the system-integration map. +- [Try Without Hardware](guides/try-offline.md): run Gently offline. +- [Hardware Setup](guides/hardware-setup.md): connect a microscope/device layer. +- [Build a Plugin](guides/build-a-plugin.md): add organisms or hardware profiles. + +## Architecture References + +- [Sample and Hardware Domains](architecture/sample-hardware-domains.md) +- [Sample Tracking Metrics](architecture/sample-tracking-metrics.md) +- [Hardware Profile Template](architecture/hardware-profile-template.md) + +## Build Locally + +If MkDocs is installed, run: + +```shell +mkdocs serve +``` + +The docs are plain Markdown, so they also remain readable directly in GitHub. diff --git a/docs/java-mmcore-interface-pattern.md b/docs/java-mmcore-interface-pattern.md index 120baffc..5c29f88d 100644 --- a/docs/java-mmcore-interface-pattern.md +++ b/docs/java-mmcore-interface-pattern.md @@ -581,6 +581,7 @@ logger = logging.getLogger(__name__) # Property Keys Enum # ============================================================================ + class PropertyKeys(Enum): """ Enum of all device adapter property names used in Gently. @@ -666,6 +667,7 @@ class PropertyKeys(Enum): # Property Values Enum # ============================================================================ + class PropertyValues(Enum): """ Enum of common property values. @@ -703,6 +705,7 @@ class PropertyValues(Enum): # Device Keys Enum # ============================================================================ + class DeviceKeys(Enum): """ Enum of device roles in the Gently system. @@ -735,6 +738,7 @@ class DeviceKeys(Enum): # Gently Devices Class # ============================================================================ + class GentlyDevices: """ Manages the mapping between device roles (DeviceKeys) and MMCore device names. @@ -838,6 +842,7 @@ class GentlyDevices: # Gently Properties Class # ============================================================================ + class GentlyProperties: """ Type-safe wrapper for MMCore property access. @@ -879,7 +884,7 @@ class GentlyProperties: device_key: DeviceKeys, property_key: PropertyKeys, value: Union[str, int, float, PropertyValues], - ignore_error: bool = False + ignore_error: bool = False, ): """ Set a device property via MMCore. @@ -915,7 +920,7 @@ class GentlyProperties: if not should_set: try: current_value = self.core.getProperty(mm_device, prop_name) - should_set = (str(value) != str(current_value)) + should_set = str(value) != str(current_value) except Exception: # If we can't read current value, set it anyway should_set = True @@ -937,7 +942,7 @@ class GentlyProperties: device_keys: List[DeviceKeys], property_key: PropertyKeys, value: Union[str, int, float, PropertyValues], - ignore_error: bool = False + ignore_error: bool = False, ): """ Set a property on multiple devices. @@ -951,11 +956,7 @@ class GentlyProperties: for device_key in device_keys: self.set_property(device_key, property_key, value, ignore_error) - def get_property_string( - self, - device_key: DeviceKeys, - property_key: PropertyKeys - ) -> str: + def get_property_string(self, device_key: DeviceKeys, property_key: PropertyKeys) -> str: """ Get a property value as a string. @@ -978,11 +979,7 @@ class GentlyProperties: logger.warning(f"Error getting {device_key}.{property_key}: {e}") return "" - def get_property_int( - self, - device_key: DeviceKeys, - property_key: PropertyKeys - ) -> int: + def get_property_int(self, device_key: DeviceKeys, property_key: PropertyKeys) -> int: """ Get a property value as an integer. @@ -1002,11 +999,7 @@ class GentlyProperties: logger.warning(f"Error parsing int from {device_key}.{property_key}: {e}") return 0 - def get_property_float( - self, - device_key: DeviceKeys, - property_key: PropertyKeys - ) -> float: + def get_property_float(self, device_key: DeviceKeys, property_key: PropertyKeys) -> float: """ Get a property value as a float. @@ -1041,8 +1034,11 @@ Example: Configure ASI diSPIM controller for volume acquisition using MMCore wra """ from gently.mmcore_wrapper import ( - GentlyDevices, GentlyProperties, - DeviceKeys, PropertyKeys, PropertyValues + GentlyDevices, + GentlyProperties, + DeviceKeys, + PropertyKeys, + PropertyValues, ) @@ -1093,10 +1089,7 @@ def configure_spim_volume_scan( # Step 1: Disable beam during configuration # ======================================================================== props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.BEAM_ENABLED, - PropertyValues.NO, - ignore_error=True + DeviceKeys.GALVO_A, PropertyKeys.BEAM_ENABLED, PropertyValues.NO, ignore_error=True ) # ======================================================================== @@ -1104,82 +1097,38 @@ def configure_spim_volume_scan( # ======================================================================== # Number of slices - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_NUM_SLICES, - num_slices - ) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_NUM_SLICES, num_slices) # Scan amplitude (determines slice coverage) - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SA_AMPLITUDE_Y_DEG, - galvo_amplitude_deg - ) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SA_AMPLITUDE_Y_DEG, galvo_amplitude_deg) # Timing - scan - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_DELAY_SCAN, - delay_before_scan - ) - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_DURATION_SCAN, - scan_duration_ms - ) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DELAY_SCAN, delay_before_scan) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DURATION_SCAN, scan_duration_ms) # Timing - camera - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_DELAY_CAMERA, - delay_before_camera - ) - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_DURATION_CAMERA, - camera_duration - ) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DELAY_CAMERA, delay_before_camera) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DURATION_CAMERA, camera_duration) # Timing - laser - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_DELAY_LASER, - delay_before_laser - ) - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_DURATION_LASER, - laser_duration - ) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DELAY_LASER, delay_before_laser) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DURATION_LASER, laser_duration) # ======================================================================== # Step 3: Configure piezo SPIM properties # ======================================================================== # Piezo amplitude (total range of travel) - props.set_property( - DeviceKeys.PIEZO_A, - PropertyKeys.SA_AMPLITUDE, - piezo_amplitude_um - ) + props.set_property(DeviceKeys.PIEZO_A, PropertyKeys.SA_AMPLITUDE, piezo_amplitude_um) # Number of slices (shared with galvo) - props.set_property( - DeviceKeys.PIEZO_A, - PropertyKeys.SPIM_NUM_SLICES_PER_PIEZO, - num_slices - ) + props.set_property(DeviceKeys.PIEZO_A, PropertyKeys.SPIM_NUM_SLICES_PER_PIEZO, num_slices) # ======================================================================== # Step 4: Arm SPIM state machine # ======================================================================== - props.set_property( - DeviceKeys.GALVO_A, - PropertyKeys.SPIM_STATE, - PropertyValues.SPIM_ARMED - ) + props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, PropertyValues.SPIM_ARMED) print(f"✓ Configured SPIM volume scan:") print(f" - Slices: {num_slices}") @@ -1214,17 +1163,14 @@ def read_spim_status(core): print(f" - Amplitude: {amplitude}°") print(f" - State: {state}") - return { - 'num_slices': num_slices, - 'amplitude': amplitude, - 'state': state - } + return {"num_slices": num_slices, "amplitude": amplitude, "state": state} # ============================================================================ # Integration with existing Gently device classes # ============================================================================ + def integrate_with_existing_devices(): """ Example of how to integrate the wrapper with existing Gently device classes. @@ -1252,12 +1198,18 @@ def integrate_with_existing_devices(): def configure_spim(self, num_slices, amplitude_deg, scan_duration_ms): """Configure SPIM parameters using type-safe wrapper.""" self.props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_NUM_SLICES, num_slices) - self.props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SA_AMPLITUDE_Y_DEG, amplitude_deg) - self.props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DURATION_SCAN, scan_duration_ms) + self.props.set_property( + DeviceKeys.GALVO_A, PropertyKeys.SA_AMPLITUDE_Y_DEG, amplitude_deg + ) + self.props.set_property( + DeviceKeys.GALVO_A, PropertyKeys.SPIM_DURATION_SCAN, scan_duration_ms + ) def arm_spim(self): """Arm SPIM state machine.""" - self.props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, PropertyValues.SPIM_ARMED) + self.props.set_property( + DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, PropertyValues.SPIM_ARMED + ) def get_spim_state(self) -> str: """Get current SPIM state.""" @@ -1279,7 +1231,7 @@ if __name__ == "__main__": slice_step_um=0.5, galvo_amplitude_deg=5.0, scan_duration_ms=10.0, - camera_exposure_ms=8.0 + camera_exposure_ms=8.0, ) # Read status diff --git a/docs/lightsheet-creation-explained.md b/docs/lightsheet-creation-explained.md index ff17bf6c..72241694 100644 --- a/docs/lightsheet-creation-explained.md +++ b/docs/lightsheet-creation-explained.md @@ -200,8 +200,11 @@ props_.setPropValue(galvoDevice, Properties.Keys.SPIM_STATE, ```python from gently.mmcore_wrapper import ( - GentlyProperties, GentlyDevices, - DeviceKeys, PropertyKeys, PropertyValues + GentlyProperties, + GentlyDevices, + DeviceKeys, + PropertyKeys, + PropertyValues, ) # Initialize @@ -224,11 +227,9 @@ props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DELAY_CAMERA, 0.5) props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_DURATION_CAMERA, 9.0) # Arm and trigger -props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, - PropertyValues.SPIM_ARMED) +props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, PropertyValues.SPIM_ARMED) # ... then trigger via TTL or: -props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, - PropertyValues.SPIM_RUNNING) +props.set_property(DeviceKeys.GALVO_A, PropertyKeys.SPIM_STATE, PropertyValues.SPIM_RUNNING) ``` --- diff --git a/docs/product-ideation/BACKLOG.md b/docs/product-ideation/BACKLOG.md new file mode 100644 index 00000000..106a1c2f --- /dev/null +++ b/docs/product-ideation/BACKLOG.md @@ -0,0 +1,302 @@ +# Gently — Product-Ideation Backlog + +45 ranked ideas from the audit-grounded ideation engine. Kinds per `FRAMEWORK.md`; generated by fan-out, deduped + ranked (impact × 1/effort, biased to missing-affordance + cross-feature-link + loop-closure). + +## Top bets + +- **IDEA-01 — Persist ground-truth stage corrections** — The flagship deep-impact win: the persistence layer (set/get_ground_truth) already exists, yet human corrections dead-end in localStorage. Persists a new entity and single-handedly unblocks accuracy, dataset-readiness (IDEA-25), few-shot examples, and model comparison (IDEA-38) — the whole perception feedback flywheel. +- **IDEA-04 — Clickable notebook chips (+ reverse links)** — Best leverage/effort in the app: the embryo/strain/session foreign keys are already in the note payload and rendered as dead text; the work is rendering a chip that navigates. Turns the notebook from a reading room into a navigable web and the nav graph from a star into a mesh. +- **IDEA-05 — Learnings/watchpoints inside the Operate step** — Canonical loop-closer the brief calls out: Operate is completely severed from the agent's memory (operate.js has zero context/notebook refs). Bringing durable insight to the moment of hardware commitment is the highest-value cross-feature link and only a filtered read of existing stores. +- **IDEA-03 — New campaign / New plan controls in the workspace** — High/low ratio and passes agent-arbitrage: originating a research program is a core loop trapped behind an incidental logo-click, and the wizard launcher already exists — pure wiring for a gap that affects every returning user. +- **IDEA-02 — Notebook add-note composer** — The single most obvious verb on a lab notebook is absent while the store already models human-authored notes; without it the 'shared lab notebook' framing is one-directional. Grounds a missing affordance in the entity that most needs cross-links. +- **IDEA-10 — Device-offline banner + toast on failed actions** — The richest untapped vein the happy-path core lenses were blind to: 7 stories 502 silently. The online/offline signal already exists in the status store and just needs wiring to the action surfaces — a low-effort fix to a trust-breaking silent failure. + +## Clusters + +- **Close the perception feedback loop (correction → model):** IDEA-01, IDEA-36, IDEA-14, IDEA-38, IDEA-29, IDEA-37, IDEA-25 +- **Make the notebook a two-way memory:** IDEA-02, IDEA-04, IDEA-07, IDEA-18, IDEA-40, IDEA-41 +- **Create & edit from the workspace, not the agent:** IDEA-03, IDEA-20, IDEA-45, IDEA-23, IDEA-26, IDEA-22 +- **Bring memory & judgement to the moment of decision:** IDEA-05, IDEA-06, IDEA-28, IDEA-32, IDEA-39 +- **Trust & safety on live hardware:** IDEA-09, IDEA-10, IDEA-12, IDEA-16, IDEA-35, IDEA-31 +- **Get results out (provenance / export):** IDEA-08, IDEA-42, IDEA-17, IDEA-44 +- **Navigation, attention & findability:** IDEA-11, IDEA-13, IDEA-24, IDEA-27, IDEA-30, IDEA-33 +- **Perturbation → response science:** IDEA-15, IDEA-42, IDEA-06 +- **Unattended & multi-instance operation:** IDEA-21, IDEA-16, IDEA-43, IDEA-09 +- **Subtraction, roles & polish:** IDEA-34, IDEA-19, IDEA-33 + +## Full backlog + +| ID | Kind | Imp | Eff | Idea | Sketch | +|---|---|:--:|:--:|---|---| +| IDEA-01 | missing-affordance | high | med | Persist ground-truth stage corrections (replace localStorage Agree/Disagree) | Replace the binary I-Agree/I-Disagree (embryos.js markAgreement → localStorage 'gently-detection-agreements', a dead end) with a stage picker that POSTs to a new /api/embryos/{id}/ground_truth backed by the already-implemented FileStore.set_ground_truth(stage,timepoint,annotator). Show a persisted 'corrected → {stage} by {user}' badge on the card and Board sparkline. Lights up three waiting consumers: accuracy, data-assessment.annotated_embryos, and perception few-shot examples. | +| IDEA-02 | missing-affordance | high | med | Notebook add-note composer (human authoring), with 'note this' from any surface | A '+ Add note' composer (kind observation/finding/question, free text, auto-linked strain/embryo/session/thread chips from context, author=human) POSTing to a new /api/notebook/notes. notebook.js is read-only ('authoring arrives in a later increment'); notebook.py exposes only GET notes/threads + POST ask. Seed a lightweight 'note this' variant on the detection card / session row that pre-fills the entity links. Optional agentic 'draft-with-Gently' pass structures rough text into a linked note. | +| IDEA-03 | missing-affordance | high | low | New campaign / New plan create controls in the Plans workspace | A labelled '+ New campaign' in the navigator (minimal title/goal/organism form) and '+ New plan' launching the same wizard the Home hero fires (AgentChat /wizard hook), plus '+ Add item' in Doc view. Creation is reachable today only via the header-logo→landing reset or agent chat (create_campaign tool); campaigns.py has no POST create though FileContextStore.create_campaign exists. | +| IDEA-04 | cross-feature-link | high | low | Make notebook note chips clickable deep-links (+ reverse 'notes about this') | Notes already carry links[]/strains/embryos/sessions/threads (rendered as inert ). Make 🧬strain → cross-filter Embryos/notebook to that strain, ◌embryo → switchTab('embryos')+select that reasoning rail, session → that session, timepoint → Gallery frame. Reciprocally add an 'N notes' link on the embryo card that filters the notebook to it (the endpoint already accepts an embryo filter). Also carry the note id when jumping Home→Notebook. | +| IDEA-05 | cross-feature-link | high | med | Surface relevant learnings / watchpoints / questions inside the Operate step and tactic cards | A compact 'what we know' rail in the Run chooser and on the active tactic card pulling learnings/observations + active watchpoints scoped to the selected embryos/strain (matched by their existing strain/embryo basis), so the operator sees 'ky123 ~80% lethal → over-sample n=12' before committing a burst interval. operate.js today has zero references to context/notebook/learnings. | +| IDEA-06 | cross-feature-link | high | med | Bidirectional embryo/prediction ↔ tactic link (+ rationale→learning) | On an embryo's stage strip add a 'governed by: ' chip (role + cadence) linking into Operations at that card; make the tactic card's scope list click through to each embryo's Board/Vitals; link the card's RATIONALE row to the notebook finding justifying its cadence. experiment-overview.js already computes the embryo↔tactic mapping for the roster lens but never renders it as a link. | +| IDEA-07 | cross-feature-link | high | med | Link notebook note ↔ plan item ('informs' / 'notes that shaped this') | On a note whose basis references a plan item, an 'Informs → open plan item' affordance; reciprocally a 'Notes that shaped this' strip on the plan-item inspector sourced from /api/notebook/notes?thread=/embryo=. Turns the reasoning ('over-sample the Robo-loss arm to n=12') into a link to the plan item it should drive. | +| IDEA-08 | provenance / interop / export | high | med | Export/download across the data-heavy surfaces (+ reproducible per-embryo bundle) | Add download where the payload lives: raw t{NNNN}.tif / projection jpg in the lightbox+Gallery, 'Export predictions (CSV) / trace (JSON)' on Embryos, 'Export timeline' on Logs, temp-trace CSV on Devices, transcript on the chat panel — reusing the Blob-download idiom settings.js already uses. Top tier: a per-embryo/session 'Export bundle' zipping volumes+projections+predictions.jsonl+traces+ground_truth.yaml+manifest+stage-over-time CSV (file_store already groups these per embryo). | +| IDEA-09 | collaboration / presence / control-ownership | high | med | Header control/presence chip (who's driving + sign-in + request/hand-off) | The /ws/agent server already broadcasts control_status {holder,holder_label,you_have_control} to every client but only agent-chat.js consumes it (as a banner inside a closed panel). Add a persistent header chip: green 'You're driving' vs amber 'Watching — {holder_label} is driving · Take control', a 'driving' ring on the holder's presence avatar, and a discoverable 'Sign in to control' when logged out. On AGENT_CONTROL, disable/annotate hardware buttons instead of letting them 403 into the misleading 'Log in' toast. | +| IDEA-10 | error-recovery / failure-path | high | med | Device-offline banner + toast on failed device actions + disable controls | When /api/devices/* returns 502/503, show a persistent amber strip ('Microscope offline — controls disabled; last seen 3m ago'), subscribe acquire/stage/run controls to the existing microscopeConnected status-store value to disable them, and route failures through the toast control-auth.js already uses on 403 (instead of console.error only). Optional agentic 'explain-and-recover': read the device_layer log + live status, name the likely cause, and offer one recovery action wired to the relevant control. | +| IDEA-11 | cross-feature-link | med | low | Deep-link context-surface rows to their embryo + badge watched embryos | context-surface.js wires a whole-row click that currently points everything at switchTab('notebook'). When the item carries an embryo ref, deep-link to that embryo instead (switchTab('embryos')+select), add an embryo chip to the row, and put an 'eye: watched' / 'expected: comma by Tue' marker on that embryo's Board/worklist row. | +| IDEA-12 | safety / reversibility | high | low | Proportionate guards on laser-on and temperature setpoint changes | Laser-on and a setpoint change (e.g. +4°C step to 32°C on live embryos) fire with no confirmation, while cheap UI-only actions ('Remove embryo', 'Stop run') DO confirm — guards are inverted. Add a hold-to-arm on the laser and a preview/confirm on the setpoint ('raises stage temp to 32.0°C over ~N min · affects 3 live embryos · Confirm'). | +| IDEA-13 | step-reduction | med | low | Fix Home/badge misdirected links (recent image, recent plan, session badge) | Home recent-image tiles render bare with no handler → wire to the existing lightbox / Gallery pre-filtered to that embryo+timepoint (the tile knows the coords). Recent-plan rows carry only data-go-tab='plans' with no id → pass the campaign id and have the navigator focus it. The session-id badge navigates to the landing → point it at the Sessions tab focused on the current session; add an 'Open' on recent-session rows. | +| IDEA-14 | trust / provenance / feedback-integrity | high | med | Overlay the agent's expectation on the stage strip + auto-score outcomes | Draw each embryo's active expectation ('will reach comma by T, uncertainty X') as a target marker/band on its Vitals stage strip so the forecast becomes falsifiable in the same view the human reads for stage-over-time. When expected_time passes, the agent auto-evaluates the belief against actual predictions and pre-fills the resolution (hit/miss + observed stage as evidence); aggregate into a small 'forecast accuracy N/M' calibration readout, and log a learning/observation on a miss. | +| IDEA-15 | flywheel (producer→consumer) | high | med | Overlay setpoint-change markers on the stage strip (and stage transitions on the temp graph) | temperature.jsonl and predictions.jsonl are both produced per session but live on separate tabs. On the Vitals stage strip, draw vertical markers where the setpoint changed (from the temp log + temp-change tactic); on the Devices temp graph, drop stage-transition ticks from predictions. They share the session timeline, so the join is a timestamp align. | +| IDEA-16 | hidden-state / make-visible | high | med | Surface the session lock (live-here / live-on-peer / stale) + guard Resume | FileStore writes session.lock={pid,hostname} while active and unlinks on release, but no route exposes it and no JS reads it. Add lock info to /api/sessions and render a per-session chip: 'Live on this machine' / 'Live on {hostname} (peer)' / 'Stale lock — process gone' (test pid liveness + hostname match). Gate Resume with a warning when actively locked elsewhere so two instances don't drive one session. | +| IDEA-17 | consistency / cross-surface parity | high | med | Show + author session ↔ campaign/plan-item membership from the Session side | The session↔plan-item edge is authored only from the plan-item end (campaigns.js '+ link session'). Every Session surface shows no campaign membership and offers no link. Show the linked campaign/item as a chip on the session-review header and Sessions list, and add 'Link to plan item' there reusing the same picker inverted. | +| IDEA-18 | missing-affordance | high | low | Human-raise a question + resolve from the Notebook + save an Ask answer as a note | Three parts on the same entity: (a) an 'Open a question' control so a human can post into the shared uncertainty queue (the Question entity explicitly allows human-posed; context.py exposes only resolve today); (b) mirror the context-surface's inline Answer/Resolve control onto the inert notebook question card; (c) beneath a notebook Ask answer, 'Save as finding' (persists the grounded answer citing its returned note_ids as basis) and 'Open as question'. | +| IDEA-19 | hidden-state / make-visible | med | low | Show and toggle an embryo's role (subject/reference) on the Embryos tab | Roles (subject='test' / reference='calibration') are picked and shown only in the fleeting Operate run-chooser role chips, yet they govern cadence and detector. Add a role chip (with its cadence) to each embryo row/card on the Embryos tab and allow the same toggle there, so it's clear why one embryo is imaged every 2 min and another every 20. | +| IDEA-20 | missing-affordance | high | high | Structural plan editing: add/remove/reorder items + draw dependency edges | In Graph, add a node (new item), delete/reorder in Doc, and drag between nodes to author a blocks/blocked-by edge (backed by add_plan_item_dependency). Today campaigns.js only PATCHes spec fields; item structure and the dependency edges that drive get_unblocked_plan_items are agent-only. | +| IDEA-21 | unattended / temporal | high | med | Notify-me on watchpoints, stalls, and run completion (out-of-app + ETA) | The run chooser already has stop conditions and Settings has in-page amber/red thresholds, but all in-page. Wire watchpoints + pending agent questions to a browser/push/email channel so a scientist away for the ~14h window is told when hatching is detected, a run stalls, or the agent needs an answer; add a run ETA ('finishes ~03:40'). | +| IDEA-22 | consistency / cross-surface parity | med | med | Save-to-library + Run-again from the Operations tactic card | Operate consumes the library (GET /api/tactic_library) and runs tactics, but nothing writes a composed/running tactic back INTO the library — the reuse loop is one-way. Add 'Save to library' and 'Run again' to the expanded Overview tactic card so a tactic the agent just composed becomes reusable. | +| IDEA-23 | cross-feature-link | med | med | In-UI plan-item picker in Operate's 'Continue a plan' mode | loadPlanItems() renders only static text and Start ships a natural-language prompt asking the agent to guess 'the right plan item'. Instead list unblocked plan items (get_unblocked_plan_items) as selectable cards like the library picker already does, and attach the chosen item on Start. | +| IDEA-24 | cross-feature-link | med | med | App-wide attention surface (agent questions/watchpoints on every tab) | #context-surface is mounted inside #home-content, so watchpoints/questions/expectations are visible and resolvable only on Home. Mirror the blocking ask-stage pattern (already dual-rendered app-wide): put a compact unresolved-count badge in the always-visible rail that expands the context lens, or render the surface app-wide. | +| IDEA-25 | flywheel (producer→consumer) | high | high | Surface ML dataset-readiness and turn coverage gaps into planned imaging | The agent already writes save_data_assessment (annotated_embryos, stage_distribution, coverage_gaps, quality_notes) and ML pipeline/run records, with ZERO UI consumers. Add a 'Dataset readiness' card rendering stage_distribution + coverage_gaps, and give each gap a '→ Plan imaging to fill' button calling create_plan_item / create_planned_session pre-filled with the under-sampled strain+stage. | +| IDEA-26 | cross-feature-link | med | med | Schedule + start a planned session (planning→operating loop, with back-link) | A 'Schedule session' form in the plan-item inspector (title/date/duration/source session to inherit params) POSTing to a new create endpoint, then a 'Start this run' action that opens Operate pre-seeded with those params and stamps the resulting Session with a back-reference ('fulfils: in '). Only a GET route exists for planned-sessions today. | +| IDEA-27 | navigation / findability | med | med | Global finder (Cmd-K) + entity-mention linkifier everywhere | A command-palette overlay: type 'CX3198' or 'aiy-pol' and jump straight there (Sessions currently hides 39 sessions behind a filter with no search). Plus promote the timepoint-only linkifyTimepoints into a general entity-linker so '🧬CX3198', '◌emb_0007', '#nerve-ring-pioneers', 'Moyle et al. 2021' become links wherever they appear as text. | +| IDEA-28 | agentic (augmented-LLM in the loop) | high | med | Recommend-a-tactic at the Run chooser instead of a blind hand-off | The chooser knows roles + embryos but the pick is blind and 'Hand to agent' is a bare prompt. Add a 'Recommend' affordance that, given the marked embryos' roles + latest stage predictions + the active plan item, returns a ranked tactic + cadence with a one-line rationale and a diff vs the library default, pre-selecting a run mode; Start executes it. | +| IDEA-29 | flywheel (producer→consumer) | med | low | Render agent embryo-understanding (needs_attention/health) as badges + auto-watchpoints | update_embryo_understanding persists current_stage, health_assessment, needs_attention + attention_reason, is_hatched, notes per embryo, with ZERO UI consumers. Surface needs_attention as a red badge + attention_reason tooltip on cards, show health/notes, auto-promote needs_attention embryos into the 'Watching' surface, and use is_hatched to retire an embryo from the worklist. | +| IDEA-30 | missing-affordance | med | low | Watch-this-embryo (create a watchpoint from an embryo) | A 'Watch' button on an embryo card/row creating a watchpoint (embryo + condition + priority) shown in the always-on 'Watching' surface. Watchpoints are agent-created today; the UI can only resolve them. | +| IDEA-31 | hidden-state / make-visible | med | med | Timelapse liveness + next-acquisition ETA on the always-on strip | The v2-strip renders only 'N embryos · Connected' and the template hardcodes a static 'LIVE' label that can lie. Feed it from timelapse.yaml + the per-session events already used for the temp graph to show 'Timelapse running · t23/120 · next in 1:40' or 'Idle' from any tab; run monitoring lives only inside Operate today. | +| IDEA-32 | agentic (augmented-LLM in the loop) | high | med | Pre-commit plan review: Gently self-critiques the run against the notebook | Fill the incomplete US-05 review/commit step with an agentic pass: before commit, check the designed items against existing learnings/notes and flag conflicts inline (it already knows 'ky123 ~80% lethal → over-sample to n=12', 'cross-check Moyle et al. 2021 before the burst interval'). Each flag is an Accept-suggestion / Ignore card; Commit stays gated until reviewed. | +| IDEA-33 | onboarding & expert-mode | med | low | Empty-state deep-links + expert accelerants (repeat-last-run, remember choices) | Novice half: every empty state names + deep-links the single next action ('No embryos yet' → the Operate/mark step; 'No temperature data' → set a setpoint) instead of dead-ending. Expert half: stop re-showing the landing on every reload, add a one-tap 'Repeat last run' that reapplies the previous run's roles+mode+cadence+stop for this campaign, and 'all subject / alternate' bulk role toggles. | +| IDEA-34 | remove-don't-add (subtraction) | med | low | Remove / fix dead & misleading controls | (a) Until wired to ground truth (IDEA-01), remove the localStorage-only Agree/Disagree rather than fake persisted feedback. (b) Sessions shows '39 empty sessions hidden' yet still counts '39 sessions' — auto-prune/fold empties. (c) The shortcuts modal lists tabs that no longer exist (Embryos=1, System=2, Live View=3…) vs the real nav — correct or delete. | +| IDEA-35 | hidden-state / make-visible | med | low | Ambient 'Gently is acting' indicator during autonomous turns | When the docked panel is closed the agent's live activity is invisible (the 'working…' row lives inside the panel; the toggle dot only reflects connection). During a wake/autonomous turn (busySource==='wake') the agent can move the stage or acquire. Drive a pulsing 'Gently is acting…' state on the header toggle + strip from the existing agentBusy/busySource. | +| IDEA-36 | agentic (augmented-LLM in the loop) | med | med | Batch-triage low-confidence predictions into a human confirm queue | Gently scans predictions.jsonl for low-confidence or is_transitional timepoints, re-examines each trace, and presents a compact review stack ('12 uncertain calls — 5 min') where each item shows the projection, the agent's second-opinion stage, and Confirm / Correct. Confirmed/corrected items write ground_truth.yaml in bulk. | +| IDEA-37 | cross-feature-link | med | low | Promote a per-timepoint VLM follow-up into ground truth / a notebook note | There's already a live per-timepoint VLM chat (/api/perception/chat/{s}/{e}/{tp}) whose output is buried in traces/t{NNNN}_chat.jsonl. Add two end-of-conversation actions the agent offers on a conclusion: 'This settles it → set stage' (writes ground_truth) and 'Save to notebook' (drafts an observation pre-linked to this embryo/session/timepoint). | +| IDEA-38 | flywheel (producer→consumer) | med | med | Perception-run selector to compare model versions against ground truth | create_perception_run records name/model_name/method/config per run and every prediction carries run_id; get_predictions already accepts a run_id filter but no UI exposes it. Add a run chip/selector above the stage charts to overlay run A (model v1) vs run B (model v2) on the same timepoints and — once ground truth exists — show each run's accuracy; feed the winner into ml_pipeline.best_run_id. | +| IDEA-39 | agentic (augmented-LLM in the loop) | high | high | Watchpoint-fired agentic triage → propose + one-tap apply a tactic | A fired watchpoint just sits with a manual Resolve today. Make firing trigger a triage: the agent pulls the embryo's recent predictions + temperature trace + governing tactic, then raises an Ask on #ask-stage with a concrete recommendation ('SubB2 approaching hatch in ~40 min — switch to recovery-monitor at 2-min cadence?') plus one-tap Apply, so resolving the watchpoint and running the tactic become one action. | +| IDEA-40 | trust / provenance / feedback-integrity | med | med | Trace a finding to its supporting observations (learning basis drill-down) | A learning carries a 'basis' field; render it as expandable links to the observations/notes that support it, so a FINDING can be drilled into its OBSERVATION evidence rather than reading as an unsourced assertion. Findings and Observations are two independent filter tabs of one list today. | +| IDEA-41 | cross-feature-link | med | med | Contextual 'Ask about this embryo/session' (scoped notebook Ask) | The notebook Ask box (POST /api/notebook/ask) already accepts thread/strain scoping via select_notes. Add an 'Ask about this embryo' box on the embryo detail rail and session-review header that calls notebook/ask pre-scoped to that embryo's notes/trace. | +| IDEA-42 | consistency / cross-surface parity | med | low | Temperature graph + note-from-excursion on Session Review and Operations | Temperature history is session-scoped (/api/temperature/{id}/history) and temperature-graph.js already renders it, but Session Review shows only Embryos/Detections/Conversation. Add a Temperature panel to review reusing the component. Plus a 'Note this →' on the temp graph / live burst that pre-fills a notebook observation with timestamp + from/to setpoint + embryo scope. | +| IDEA-43 | hidden-state / cross-feature-link | low | med | Mesh peers + campaign-sharing surface + 'claimed by {peer}' badges on plan items | campaigns.py implements share/unshare/join/claim/status and peer discovery raises agent asks, but the only client mesh UI is a read-only config block. Turn it into a participants panel with online/offline dots, add a 'Share/Join' action on a campaign, and render the claimed_by/claimed_by_hostname the campaign tree already serializes as a claim badge on item rows (disable local start on items claimed by another instance). | +| IDEA-44 | consistency / cross-surface parity | low | low | Copy-id affordance on embryo identifiers | The header already has copySessionId (clipboard + copied feedback). Embryo uids are the token you paste into chat or a note but have no copy affordance. Add a small copy icon next to the embryo uid, mirroring the session button. | +| IDEA-45 | missing-affordance | med | med | Inline-edit campaign metadata (target / status / description) | Make the campaign header fields editable in the inspector the same way plan-item spec fields already PATCH. Today only plan-item spec is editable; the parent campaign's own metadata is agent-only. | + +## Detail + +### IDEA-01 — Persist ground-truth stage corrections (replace localStorage Agree/Disagree) `missing-affordance` · impact high / effort med +Replace the binary I-Agree/I-Disagree (embryos.js markAgreement → localStorage 'gently-detection-agreements', a dead end) with a stage picker that POSTs to a new /api/embryos/{id}/ground_truth backed by the already-implemented FileStore.set_ground_truth(stage,timepoint,annotator). Show a persisted 'corrected → {stage} by {user}' badge on the card and Board sparkline. Lights up three waiting consumers: accuracy, data-assessment.annotated_embryos, and perception few-shot examples. +- **Why:** Flagship: DEEP (persists a new entity + closes the human-correction→perception flywheel), the persistence layer already exists (set/get_ground_truth) so effort is bounded, and today feedback is silently thrown to the browser. US-31 gap; canonical missing-affordance + loop-closure. +- **Surfaces:** Embryos > Default (detection cards), Embryos > Default ('Show VLM reasoning'), Embryos > Board · **Entities:** Ground truth, Prediction/Stage, Trace, Embryo + +### IDEA-02 — Notebook add-note composer (human authoring), with 'note this' from any surface `missing-affordance` · impact high / effort med +A '+ Add note' composer (kind observation/finding/question, free text, auto-linked strain/embryo/session/thread chips from context, author=human) POSTing to a new /api/notebook/notes. notebook.js is read-only ('authoring arrives in a later increment'); notebook.py exposes only GET notes/threads + POST ask. Seed a lightweight 'note this' variant on the detection card / session row that pre-fills the entity links. Optional agentic 'draft-with-Gently' pass structures rough text into a linked note. +- **Why:** The single most obvious verb on a notebook is absent while the store models human-authored notes; passes agent-arbitrage (a discoverable in-context capture beats dictating to chat). The note's power is its cross-links, which the composer auto-fills. +- **Surfaces:** Notebook tab header, Embryos > Default (detection cards), Gallery lightbox, Sessions tab · **Entities:** Notebook note, Observation, Question, Embryo, Session + +### IDEA-03 — New campaign / New plan create controls in the Plans workspace `missing-affordance` · impact high / effort low +A labelled '+ New campaign' in the navigator (minimal title/goal/organism form) and '+ New plan' launching the same wizard the Home hero fires (AgentChat /wizard hook), plus '+ Add item' in Doc view. Creation is reachable today only via the header-logo→landing reset or agent chat (create_campaign tool); campaigns.py has no POST create though FileContextStore.create_campaign exists. +- **Why:** High/low ratio and passes agent-arbitrage: a core loop (originate a research program) trapped behind an incidental logo-click is gold, and the launcher already exists — pure wiring. US-06/US-35. +- **Surfaces:** Plans tab (campaign navigator), Plans > Doc ('+ Add item'), Home hero · **Entities:** Campaign, Operation Plan, Plan item + +### IDEA-04 — Make notebook note chips clickable deep-links (+ reverse 'notes about this') `cross-feature-link` · impact high / effort low +Notes already carry links[]/strains/embryos/sessions/threads (rendered as inert ). Make 🧬strain → cross-filter Embryos/notebook to that strain, ◌embryo → switchTab('embryos')+select that reasoning rail, session → that session, timepoint → Gallery frame. Reciprocally add an 'N notes' link on the embryo card that filters the notebook to it (the endpoint already accepts an embryo filter). Also carry the note id when jumping Home→Notebook. +- **Why:** Best leverage/effort in the app: the join is a foreign key that already exists in the payload; the work is rendering a chip that navigates. Dangling-edge generator; the nav graph currently has zero entity-to-entity edges. +- **Surfaces:** Notebook tab (note chips), Embryos tab, Sessions tab, Gallery, Home 'From the notebook' · **Entities:** Notebook note, Embryo, Session, Strain, Volume/Image + +### IDEA-05 — Surface relevant learnings / watchpoints / questions inside the Operate step and tactic cards `cross-feature-link` · impact high / effort med +A compact 'what we know' rail in the Run chooser and on the active tactic card pulling learnings/observations + active watchpoints scoped to the selected embryos/strain (matched by their existing strain/embryo basis), so the operator sees 'ky123 ~80% lethal → over-sample n=12' before committing a burst interval. operate.js today has zero references to context/notebook/learnings. +- **Why:** Canonical loop-closer: Operate is completely severed from the agent's memory; bringing durable insight to the moment of decision is the highest-value cross-feature link and a filtered read of stores that already exist. +- **Surfaces:** Operate > Run chooser, Operations > Overview (tactic cards), Operate worklist · **Entities:** Learning, Watchpoint, Question, Tactic, Embryo + +### IDEA-06 — Bidirectional embryo/prediction ↔ tactic link (+ rationale→learning) `cross-feature-link` · impact high / effort med +On an embryo's stage strip add a 'governed by: ' chip (role + cadence) linking into Operations at that card; make the tactic card's scope list click through to each embryo's Board/Vitals; link the card's RATIONALE row to the notebook finding justifying its cadence. experiment-overview.js already computes the embryo↔tactic mapping for the roster lens but never renders it as a link. +- **Why:** Trust infrastructure: lets a scientist answer 'why is this embryo imaged this way / which embryos does this burst touch?' — the core of adaptive-timelapse trust. Provenance + missing-edge; mapping already computed. +- **Surfaces:** Embryos > Board/Vitals (stage sparkline), Operations > Overview (tactic cards), Operate worklist · **Entities:** Prediction/Stage, Tactic, Role, Learning, Embryo + +### IDEA-07 — Link notebook note ↔ plan item ('informs' / 'notes that shaped this') `cross-feature-link` · impact high / effort med +On a note whose basis references a plan item, an 'Informs → open plan item' affordance; reciprocally a 'Notes that shaped this' strip on the plan-item inspector sourced from /api/notebook/notes?thread=/embryo=. Turns the reasoning ('over-sample the Robo-loss arm to n=12') into a link to the plan item it should drive. +- **Why:** Closes the 'why is this plan item here' gap (US-33/US-36); the note model already stores basis links, so it is render + one reverse query. Pairs with IDEA-02. +- **Surfaces:** Notebook tab (note cards), Plans > Doc (item rows), plan-item inspector · **Entities:** Notebook note, Plan item, Campaign, Learning + +### IDEA-08 — Export/download across the data-heavy surfaces (+ reproducible per-embryo bundle) `provenance / interop / export` · impact high / effort med +Add download where the payload lives: raw t{NNNN}.tif / projection jpg in the lightbox+Gallery, 'Export predictions (CSV) / trace (JSON)' on Embryos, 'Export timeline' on Logs, temp-trace CSV on Devices, transcript on the chat panel — reusing the Blob-download idiom settings.js already uses. Top tier: a per-embryo/session 'Export bundle' zipping volumes+projections+predictions.jsonl+traces+ground_truth.yaml+manifest+stage-over-time CSV (file_store already groups these per embryo). +- **Why:** App-boundary blind spot: results cannot leave the app (only prefs JSON + plan markdown today). Reproducibility/sharing is a core scientific need; files already exist on disk. Noise-collapse folds ~6 per-surface export requests into one systemic idea. US-32. +- **Surfaces:** Gallery + lightbox, Embryos tab, Logs tab, Devices temp graph, Agent chat panel · **Entities:** Volume/Image, Projection, Prediction/Stage, Trace, Event/log, Temperature sample/graph + +### IDEA-09 — Header control/presence chip (who's driving + sign-in + request/hand-off) `collaboration / presence / control-ownership` · impact high / effort med +The /ws/agent server already broadcasts control_status {holder,holder_label,you_have_control} to every client but only agent-chat.js consumes it (as a banner inside a closed panel). Add a persistent header chip: green 'You're driving' vs amber 'Watching — {holder_label} is driving · Take control', a 'driving' ring on the holder's presence avatar, and a discoverable 'Sign in to control' when logged out. On AGENT_CONTROL, disable/annotate hardware buttons instead of letting them 403 into the misleading 'Log in' toast. +- **Why:** The single-driver lock is real and multi-user and the holder identity is already pushed everywhere, yet the only ambient signal is a reactive 403 that misattributes another operator's lock to 'not logged in'. Collaboration + hidden-state; merges US-43/US-44. +- **Surfaces:** header presence-container / session badge, Devices, Operate, /login · **Entities:** Auth / control, Agent chat / turn, Device state, Mesh / peer instance + +### IDEA-10 — Device-offline banner + toast on failed device actions + disable controls `error-recovery / failure-path` · impact high / effort med +When /api/devices/* returns 502/503, show a persistent amber strip ('Microscope offline — controls disabled; last seen 3m ago'), subscribe acquire/stage/run controls to the existing microscopeConnected status-store value to disable them, and route failures through the toast control-auth.js already uses on 403 (instead of console.error only). Optional agentic 'explain-and-recover': read the device_layer log + live status, name the likely cause, and offer one recovery action wired to the relevant control. +- **Why:** The richest untapped vein: US-09/10/11/12/14/18/26 all 502 silently (toast_visible=False). The online/offline signal already exists in the status store but is never wired to the action surfaces. Happy-path bias made every core lens blind to it. +- **Surfaces:** Devices tab, Operate rail, acquire buttons (Snap Volume / Burst / stage-move) · **Entities:** Device state, Volume/Image, Session + +### IDEA-11 — Deep-link context-surface rows to their embryo + badge watched embryos `cross-feature-link` · impact med / effort low +context-surface.js wires a whole-row click that currently points everything at switchTab('notebook'). When the item carries an embryo ref, deep-link to that embryo instead (switchTab('embryos')+select), add an embryo chip to the row, and put an 'eye: watched' / 'expected: comma by Tue' marker on that embryo's Board/worklist row. +- **Why:** Near-free (redirect an existing handler) and fixes a wrong-destination dead-end: watchpoints exist to pull attention to a specific embryo but send you to a generic notebook. Great ratio, structural. +- **Surfaces:** context-surface 'Watching'/'Expectations'/'Open questions', Embryos > Board/Vitals, Operate worklist · **Entities:** Watchpoint, Expectation, Question, Embryo + +### IDEA-12 — Proportionate guards on laser-on and temperature setpoint changes `safety / reversibility` · impact high / effort low +Laser-on and a setpoint change (e.g. +4°C step to 32°C on live embryos) fire with no confirmation, while cheap UI-only actions ('Remove embryo', 'Stop run') DO confirm — guards are inverted. Add a hold-to-arm on the laser and a preview/confirm on the setpoint ('raises stage temp to 32.0°C over ~N min · affects 3 live embryos · Confirm'). +- **Why:** Microscopy acts on live specimens; the physically-irreversible actions are the only unguarded ones and no undo exists anywhere. High/low ratio; a whole risk axis the core lenses can't see. US-16/US-26. +- **Surfaces:** Devices Manual laser toggle, Devices header temp setpoint, Operations tactic (temp-change) · **Entities:** Setpoint (temperature), Device state, Embryo + +### IDEA-13 — Fix Home/badge misdirected links (recent image, recent plan, session badge) `step-reduction` · impact med / effort low +Home recent-image tiles render bare with no handler → wire to the existing lightbox / Gallery pre-filtered to that embryo+timepoint (the tile knows the coords). Recent-plan rows carry only data-go-tab='plans' with no id → pass the campaign id and have the navigator focus it. The session-id badge navigates to the landing → point it at the Sessions tab focused on the current session; add an 'Open' on recent-session rows. +- **Why:** Three misdirected-link frictions with all data already in hand; the click currently loses the entity identity or dumps you at the start screen. Cheap navigation-fidelity cluster. +- **Surfaces:** Home > Recent Images/Plans/Sessions, header session-id badge · **Entities:** Volume/Image, Campaign, Session, Embryo + +### IDEA-14 — Overlay the agent's expectation on the stage strip + auto-score outcomes `trust / provenance / feedback-integrity` · impact high / effort med +Draw each embryo's active expectation ('will reach comma by T, uncertainty X') as a target marker/band on its Vitals stage strip so the forecast becomes falsifiable in the same view the human reads for stage-over-time. When expected_time passes, the agent auto-evaluates the belief against actual predictions and pre-fills the resolution (hit/miss + observed stage as evidence); aggregate into a small 'forecast accuracy N/M' calibration readout, and log a learning/observation on a miss. +- **Why:** Expectations are the agent's forward beliefs but 'confirmed' is currently a hand-click with no evidence — a dead-end resolution. Links Expectation→Prediction (data already stored) into a trust/calibration loop. US-29. +- **Surfaces:** Embryos > Vitals (stage strip), context-surface 'Expectations', Notebook > Findings · **Entities:** Expectation, Prediction/Stage, Learning, Embryo + +### IDEA-15 — Overlay setpoint-change markers on the stage strip (and stage transitions on the temp graph) `flywheel (producer→consumer)` · impact high / effort med +temperature.jsonl and predictions.jsonl are both produced per session but live on separate tabs. On the Vitals stage strip, draw vertical markers where the setpoint changed (from the temp log + temp-change tactic); on the Devices temp graph, drop stage-transition ticks from predictions. They share the session timeline, so the join is a timestamp align. +- **Why:** Whether a thermal perturbation shifts development is the microscope's whole scientific point, and both halves already exist and are already charted — just never on the same axis. Two live producers never joined; the cause→effect readout the instrument is built to show is invisible. +- **Surfaces:** Embryos > Vitals (stage strip), Devices temperature graph, Operations tactic (temp-change) · **Entities:** Temperature sample/graph, Prediction/Stage, Setpoint (temperature), Tactic + +### IDEA-16 — Surface the session lock (live-here / live-on-peer / stale) + guard Resume `hidden-state / make-visible` · impact high / effort med +FileStore writes session.lock={pid,hostname} while active and unlinks on release, but no route exposes it and no JS reads it. Add lock info to /api/sessions and render a per-session chip: 'Live on this machine' / 'Live on {hostname} (peer)' / 'Stale lock — process gone' (test pid liveness + hostname match). Gate Resume with a warning when actively locked elsewhere so two instances don't drive one session. +- **Why:** The lock already holds exactly the identity needed (pid+hostname) but is invisible; a crashed run leaves a stale lock with no signal, and with mesh a peer can legitimately hold it. Hidden-state + safety against collisions. +- **Surfaces:** Sessions tab, Home > Recent Sessions, header session badge, landing > Resume · **Entities:** Session, Mesh / peer instance + +### IDEA-17 — Show + author session ↔ campaign/plan-item membership from the Session side `consistency / cross-surface parity` · impact high / effort med +The session↔plan-item edge is authored only from the plan-item end (campaigns.js '+ link session'). Every Session surface shows no campaign membership and offers no link. Show the linked campaign/item as a chip on the session-review header and Sessions list, and add 'Link to plan item' there reusing the same picker inverted. +- **Why:** One relationship, affordance on only one end: a session opened in review can't even tell you which experiment it belongs to. Parity defect; gives Sessions provenance (US-42). +- **Surfaces:** Sessions (session-review header), Home > Recent Sessions, header session badge · **Entities:** Session, Plan item, Campaign + +### IDEA-18 — Human-raise a question + resolve from the Notebook + save an Ask answer as a note `missing-affordance` · impact high / effort low +Three parts on the same entity: (a) an 'Open a question' control so a human can post into the shared uncertainty queue (the Question entity explicitly allows human-posed; context.py exposes only resolve today); (b) mirror the context-surface's inline Answer/Resolve control onto the inert notebook question card; (c) beneath a notebook Ask answer, 'Save as finding' (persists the grounded answer citing its returned note_ids as basis) and 'Open as question'. +- **Why:** Consistency + capability-orphan: the resolve endpoint already exists and the Ask already returns basis note_ids — the grounding is right there and discarded. Turns one-off Q&A into accumulating memory at near-zero cost. US-37. +- **Surfaces:** Notebook > Questions filter, Notebook Ask box, context-surface 'Open questions' · **Entities:** Question, Learning, Notebook note, Campaign + +### IDEA-19 — Show and toggle an embryo's role (subject/reference) on the Embryos tab `hidden-state / make-visible` · impact med / effort low +Roles (subject='test' / reference='calibration') are picked and shown only in the fleeting Operate run-chooser role chips, yet they govern cadence and detector. Add a role chip (with its cadence) to each embryo row/card on the Embryos tab and allow the same toggle there, so it's clear why one embryo is imaged every 2 min and another every 20. +- **Why:** role_class drives how Operations foregrounds/images an embryo but is invisible on the very surface dedicated to embryos. Cheap parity/hidden-state win. +- **Surfaces:** Embryos > Default/Board/Vitals, Operate worklist · **Entities:** Role, Embryo, Tactic, Cadence + +### IDEA-20 — Structural plan editing: add/remove/reorder items + draw dependency edges `missing-affordance` · impact high / effort high +In Graph, add a node (new item), delete/reorder in Doc, and drag between nodes to author a blocks/blocked-by edge (backed by add_plan_item_dependency). Today campaigns.js only PATCHes spec fields; item structure and the dependency edges that drive get_unblocked_plan_items are agent-only. +- **Why:** Deep (creates entities + edges) and structural, but the highest-effort item in the top tier — the whole create/delete column of the campaign hierarchy is agent-only. US-07. +- **Surfaces:** Plans > Graph, Plans > Doc ('→ blocks:'), Plans > Board · **Entities:** Plan item, Plan item dependency, Campaign + +### IDEA-21 — Notify-me on watchpoints, stalls, and run completion (out-of-app + ETA) `unattended / temporal` · impact high / effort med +The run chooser already has stop conditions and Settings has in-page amber/red thresholds, but all in-page. Wire watchpoints + pending agent questions to a browser/push/email channel so a scientist away for the ~14h window is told when hatching is detected, a run stalls, or the agent needs an answer; add a run ETA ('finishes ~03:40'). +- **Why:** A developmental timelapse outlives attention; the codebase uses no Notification API today. A whole temporal axis the time-agnostic core lenses miss. +- **Surfaces:** Operate run chooser (stop conditions), Operations, Settings > Alerts · **Entities:** Watchpoint, Question, Session, Operation Plan + +### IDEA-22 — Save-to-library + Run-again from the Operations tactic card `consistency / cross-surface parity` · impact med / effort med +Operate consumes the library (GET /api/tactic_library) and runs tactics, but nothing writes a composed/running tactic back INTO the library — the reuse loop is one-way. Add 'Save to library' and 'Run again' to the expanded Overview tactic card so a tactic the agent just composed becomes reusable. +- **Why:** 'run a tactic → keep it for next time' is broken at the human-authoring step; the apply half exists, only the save half is conceptual. US-17. +- **Surfaces:** Operations > Overview (tactic cards), Operate > Run chooser ('From library') · **Entities:** Tactic, Tactic library, Operation Plan + +### IDEA-23 — In-UI plan-item picker in Operate's 'Continue a plan' mode `cross-feature-link` · impact med / effort med +loadPlanItems() renders only static text and Start ships a natural-language prompt asking the agent to guess 'the right plan item'. Instead list unblocked plan items (get_unblocked_plan_items) as selectable cards like the library picker already does, and attach the chosen item on Start. +- **Why:** The chooser lets you click a saved tactic but 'Continue a plan' forces a chat hand-off and hopes the agent guesses; the ImagingSpec plan items already exist. Closes planning→operating binding. +- **Surfaces:** Operate > Run chooser ('Continue a plan'), Plans tab · **Entities:** Plan item, Session, Operation Plan, Campaign + +### IDEA-24 — App-wide attention surface (agent questions/watchpoints on every tab) `cross-feature-link` · impact med / effort med +#context-surface is mounted inside #home-content, so watchpoints/questions/expectations are visible and resolvable only on Home. Mirror the blocking ask-stage pattern (already dual-rendered app-wide): put a compact unresolved-count badge in the always-visible rail that expands the context lens, or render the surface app-wide. +- **Why:** During a live run the operator is on Operations/Embryos/Devices but the non-blocking items needing a human decision are stranded on Home. The blocking ASK is already app-wide; the uncertainty queue isn't. +- **Surfaces:** context-surface 'Agent's view', all workspace tabs, left rail / status area · **Entities:** Question, Watchpoint, Expectation, Agent chat / turn + +### IDEA-25 — Surface ML dataset-readiness and turn coverage gaps into planned imaging `flywheel (producer→consumer)` · impact high / effort high +The agent already writes save_data_assessment (annotated_embryos, stage_distribution, coverage_gaps, quality_notes) and ML pipeline/run records, with ZERO UI consumers. Add a 'Dataset readiness' card rendering stage_distribution + coverage_gaps, and give each gap a '→ Plan imaging to fill' button calling create_plan_item / create_planned_session pre-filled with the under-sampled strain+stage. +- **Why:** An entire storage domain (agent/ml/*) is a pure dead-end producer; the coverage-gap signal that should steer the NEXT session is discarded. The core research-program flywheel (acquire→assess→acquire the gaps) with the consumer half missing. Depends on IDEA-01 for real ground truth. +- **Surfaces:** Plans tab (side panel), Notebook > Findings, Embryos tab · **Entities:** ML data assessment, Plan item, Planned session, Ground truth + +### IDEA-26 — Schedule + start a planned session (planning→operating loop, with back-link) `cross-feature-link` · impact med / effort med +A 'Schedule session' form in the plan-item inspector (title/date/duration/source session to inherit params) POSTing to a new create endpoint, then a 'Start this run' action that opens Operate pre-seeded with those params and stamps the resulting Session with a back-reference ('fulfils: in '). Only a GET route exists for planned-sessions today. +- **Why:** A Planned session is defined as the thing that becomes a Session when started, yet there's no create/start and no actual↔planned back-link — schedule and execution are disconnected. US-14/US-42. +- **Surfaces:** Plans tab (planned-sessions), plan-item inspector, Sessions tab · **Entities:** Planned session, Session, Plan item, Campaign + +### IDEA-27 — Global finder (Cmd-K) + entity-mention linkifier everywhere `navigation / findability` · impact med / effort med +A command-palette overlay: type 'CX3198' or 'aiy-pol' and jump straight there (Sessions currently hides 39 sessions behind a filter with no search). Plus promote the timepoint-only linkifyTimepoints into a general entity-linker so '🧬CX3198', '◌emb_0007', '#nerve-ring-pioneers', 'Moyle et al. 2021' become links wherever they appear as text. +- **Why:** No search exists anywhere; linkification exists for exactly one entity type. Serves both wayfinding and the expert accelerant the uniform core lenses miss. +- **Surfaces:** global shell/header, Notebook cards, Embryos VLM reasoning, Operations tactic cards · **Entities:** Session, Embryo, Campaign, Tactic, Notebook note, Strain + +### IDEA-28 — Recommend-a-tactic at the Run chooser instead of a blind hand-off `agentic (augmented-LLM in the loop)` · impact high / effort med +The chooser knows roles + embryos but the pick is blind and 'Hand to agent' is a bare prompt. Add a 'Recommend' affordance that, given the marked embryos' roles + latest stage predictions + the active plan item, returns a ranked tactic + cadence with a one-line rationale and a diff vs the library default, pre-selecting a run mode; Start executes it. +- **Why:** Deepens the single highest-stakes moment (committing hardware time) by linking Prediction + Role + Plan item → Tactic, entities adjacent in the model but unlinked in the UI. Agent narrows, human commits. +- **Surfaces:** Operate > Run chooser, Operations > Overview, Operate worklist · **Entities:** Tactic, Role, Embryo, Prediction/Stage, Plan item + +### IDEA-29 — Render agent embryo-understanding (needs_attention/health) as badges + auto-watchpoints `flywheel (producer→consumer)` · impact med / effort low +update_embryo_understanding persists current_stage, health_assessment, needs_attention + attention_reason, is_hatched, notes per embryo, with ZERO UI consumers. Surface needs_attention as a red badge + attention_reason tooltip on cards, show health/notes, auto-promote needs_attention embryos into the 'Watching' surface, and use is_hatched to retire an embryo from the worklist. +- **Why:** The agent's richest per-embryo judgement is written every cycle and read only back into its own prompt; the human never sees 'this embryo needs attention because X'. Orphaned producer feeding an existing surface — cheap. +- **Surfaces:** Embryos > Board/Vitals, context-surface 'Watching', Operate worklist · **Entities:** Embryo understanding, Watchpoint, Embryo + +### IDEA-30 — Watch-this-embryo (create a watchpoint from an embryo) `missing-affordance` · impact med / effort low +A 'Watch' button on an embryo card/row creating a watchpoint (embryo + condition + priority) shown in the always-on 'Watching' surface. Watchpoints are agent-created today; the UI can only resolve them. +- **Why:** Both a missing affordance and a cross-feature link (Embryo cards ↔ Watching surface) that lets human attention join the agent's. Low cost; pairs with IDEA-11/IDEA-29. +- **Surfaces:** Embryos > Board/Vitals/Default, context-surface 'Watching' · **Entities:** Watchpoint, Embryo, Expectation + +### IDEA-31 — Timelapse liveness + next-acquisition ETA on the always-on strip `hidden-state / make-visible` · impact med / effort med +The v2-strip renders only 'N embryos · Connected' and the template hardcodes a static 'LIVE' label that can lie. Feed it from timelapse.yaml + the per-session events already used for the temp graph to show 'Timelapse running · t23/120 · next in 1:40' or 'Idle' from any tab; run monitoring lives only inside Operate today. +- **Why:** Whether a run is actually acquiring, how far along, and when the next frame fires are only reachable by navigating into Operate — and the 'LIVE' chip is static. Hidden-state + temporal. +- **Surfaces:** v2-strip, header, Operations Overview · **Entities:** Session, Operation Plan, Tactic, Volume/Image + +### IDEA-32 — Pre-commit plan review: Gently self-critiques the run against the notebook `agentic (augmented-LLM in the loop)` · impact high / effort med +Fill the incomplete US-05 review/commit step with an agentic pass: before commit, check the designed items against existing learnings/notes and flag conflicts inline (it already knows 'ky123 ~80% lethal → over-sample to n=12', 'cross-check Moyle et al. 2021 before the burst interval'). Each flag is an Accept-suggestion / Ignore card; Commit stays gated until reviewed. +- **Why:** The notebook already surfaces these plan-relevant cautions but disconnected from the plan being authored; an agent review is the natural place to enforce Learning→Plan-item links and plugs the concrete US-05 gap. +- **Surfaces:** plan wizard 'THE PLAN' / commit, Plans > Doc/Decide, Home > Recent Plans · **Entities:** Operation Plan, Plan item, Learning, Notebook note, ImagingSpec + +### IDEA-33 — Empty-state deep-links + expert accelerants (repeat-last-run, remember choices) `onboarding & expert-mode` · impact med / effort low +Novice half: every empty state names + deep-links the single next action ('No embryos yet' → the Operate/mark step; 'No temperature data' → set a setpoint) instead of dead-ending. Expert half: stop re-showing the landing on every reload, add a one-tap 'Repeat last run' that reapplies the previous run's roles+mode+cadence+stop for this campaign, and 'all subject / alternate' bulk role toggles. +- **Why:** Noise-collapse folds ~15 'nicer empty state' items into one systemic idea (deep-link to the seeding action). Adds the expert accelerant + frequency-weighted repeat-run for the highest-frequency loop (mark→run, a 6-decision funnel). US-13. +- **Surfaces:** all headless tabs (Embryos/Sessions/Devices temp/Gallery), landing, Operate run chooser · **Entities:** Session, Embryo, Tactic, Role, Setpoint (temperature) + +### IDEA-34 — Remove / fix dead & misleading controls `remove-don't-add (subtraction)` · impact med / effort low +(a) Until wired to ground truth (IDEA-01), remove the localStorage-only Agree/Disagree rather than fake persisted feedback. (b) Sessions shows '39 empty sessions hidden' yet still counts '39 sessions' — auto-prune/fold empties. (c) The shortcuts modal lists tabs that no longer exist (Embryos=1, System=2, Live View=3…) vs the real nav — correct or delete. +- **Why:** Subtraction axis the additive core lenses structurally can't propose: each item is worse-than-absent (misleads). Cheap. +- **Surfaces:** Embryos detection cards, Sessions list, keyboard-shortcuts modal · **Entities:** Session, Prediction/Stage + +### IDEA-35 — Ambient 'Gently is acting' indicator during autonomous turns `hidden-state / make-visible` · impact med / effort low +When the docked panel is closed the agent's live activity is invisible (the 'working…' row lives inside the panel; the toggle dot only reflects connection). During a wake/autonomous turn (busySource==='wake') the agent can move the stage or acquire. Drive a pulsing 'Gently is acting…' state on the header toggle + strip from the existing agentBusy/busySource. +- **Why:** The busy/wake state is already tracked; only the closed-panel ambient surface is missing. Autonomous hardware motion with no standing signal is a trust/safety gap. Low cost. +- **Surfaces:** header agent-chat toggle, v2-strip, context-surface · **Entities:** Agent chat / turn, Operation Plan, Device state + +### IDEA-36 — Batch-triage low-confidence predictions into a human confirm queue `agentic (augmented-LLM in the loop)` · impact med / effort med +Gently scans predictions.jsonl for low-confidence or is_transitional timepoints, re-examines each trace, and presents a compact review stack ('12 uncertain calls — 5 min') where each item shows the projection, the agent's second-opinion stage, and Confirm / Correct. Confirmed/corrected items write ground_truth.yaml in bulk. +- **Why:** Turns hours of scrubbing into a short confirm pass — agent narrows, human decides — feeding the same ground-truth store as IDEA-01 at scale. Depends on IDEA-01. +- **Surfaces:** Embryos > Board/Vitals/Default · **Entities:** Prediction/Stage, Ground truth, Trace, Embryo + +### IDEA-37 — Promote a per-timepoint VLM follow-up into ground truth / a notebook note `cross-feature-link` · impact med / effort low +There's already a live per-timepoint VLM chat (/api/perception/chat/{s}/{e}/{tp}) whose output is buried in traces/t{NNNN}_chat.jsonl. Add two end-of-conversation actions the agent offers on a conclusion: 'This settles it → set stage' (writes ground_truth) and 'Save to notebook' (drafts an observation pre-linked to this embryo/session/timepoint). +- **Why:** An agentic surface already exists but dead-ends — the reasoning a biologist extracts by chatting never becomes ground truth or shared memory. Cheap, reuses shipped chat infra. +- **Surfaces:** Embryos > Default follow-up chat ('Ask a follow-up about this timepoint'), Notebook tab · **Entities:** Trace, Prediction/Stage, Ground truth, Notebook note, Embryo + +### IDEA-38 — Perception-run selector to compare model versions against ground truth `flywheel (producer→consumer)` · impact med / effort med +create_perception_run records name/model_name/method/config per run and every prediction carries run_id; get_predictions already accepts a run_id filter but no UI exposes it. Add a run chip/selector above the stage charts to overlay run A (model v1) vs run B (model v2) on the same timepoints and — once ground truth exists — show each run's accuracy; feed the winner into ml_pipeline.best_run_id. +- **Why:** Read path is 90% built (run_id filter) and simply never surfaced — a cheap way to close the model-eval loop. Depends on IDEA-01 for accuracy. +- **Surfaces:** Embryos > Board (sparkline), Embryos > Vitals (stage strip) · **Entities:** Perception run, Prediction/Stage, Ground truth, Trace + +### IDEA-39 — Watchpoint-fired agentic triage → propose + one-tap apply a tactic `agentic (augmented-LLM in the loop)` · impact high / effort high +A fired watchpoint just sits with a manual Resolve today. Make firing trigger a triage: the agent pulls the embryo's recent predictions + temperature trace + governing tactic, then raises an Ask on #ask-stage with a concrete recommendation ('SubB2 approaching hatch in ~40 min — switch to recovery-monitor at 2-min cadence?') plus one-tap Apply, so resolving the watchpoint and running the tactic become one action. +- **Why:** Bridges Watchpoint→Tactic→Ask, three entities with no edge today; canonical 'agentic triage of an alert' turning passive attention into an approve-only decision. High effort keeps it below the cheap structural wins. +- **Surfaces:** context-surface 'Watching'/'Expectations', Operations tactic spine, #ask-stage · **Entities:** Watchpoint, Expectation, Tactic, Ask (agent → human), Embryo + +### IDEA-40 — Trace a finding to its supporting observations (learning basis drill-down) `trust / provenance / feedback-integrity` · impact med / effort med +A learning carries a 'basis' field; render it as expandable links to the observations/notes that support it, so a FINDING can be drilled into its OBSERVATION evidence rather than reading as an unsourced assertion. Findings and Observations are two independent filter tabs of one list today. +- **Why:** Makes the agent's conclusions auditable (trust) and reuses data already stored on the learning. US-33/US-36. +- **Surfaces:** Notebook > Findings, Notebook > Observations · **Entities:** Learning, Observation, Notebook note + +### IDEA-41 — Contextual 'Ask about this embryo/session' (scoped notebook Ask) `cross-feature-link` · impact med / effort med +The notebook Ask box (POST /api/notebook/ask) already accepts thread/strain scoping via select_notes. Add an 'Ask about this embryo' box on the embryo detail rail and session-review header that calls notebook/ask pre-scoped to that embryo's notes/trace. +- **Why:** The ask affordance is siloed in the notebook though its API already accepts scoping; the surfaces with the richest local context can't invoke it. +- **Surfaces:** Embryos > Default (embryo detail rail), Session Review header, Notebook Ask box · **Entities:** Embryo, Notebook note, Trace, Session + +### IDEA-42 — Temperature graph + note-from-excursion on Session Review and Operations `consistency / cross-surface parity` · impact med / effort low +Temperature history is session-scoped (/api/temperature/{id}/history) and temperature-graph.js already renders it, but Session Review shows only Embryos/Detections/Conversation. Add a Temperature panel to review reusing the component. Plus a 'Note this →' on the temp graph / live burst that pre-fills a notebook observation with timestamp + from/to setpoint + embryo scope. +- **Why:** Same entity, same endpoint, component already exists; review omits the one place a completed session's thermal record is most worth seeing. Note-from-excursion composes with IDEA-02. +- **Surfaces:** Session Review, Devices temperature graph, Operations (temp-change burst) · **Entities:** Temperature sample/graph, Session, Setpoint (temperature), Observation + +### IDEA-43 — Mesh peers + campaign-sharing surface + 'claimed by {peer}' badges on plan items `hidden-state / cross-feature-link` · impact low / effort med +campaigns.py implements share/unshare/join/claim/status and peer discovery raises agent asks, but the only client mesh UI is a read-only config block. Turn it into a participants panel with online/offline dots, add a 'Share/Join' action on a campaign, and render the claimed_by/claimed_by_hostname the campaign tree already serializes as a claim badge on item rows (disable local start on items claimed by another instance). +- **Why:** Full collaboration backend with no front-end; in a shared campaign the claim badge is the only signal a peer is already executing an item, so instances can silently collide. Lower impact for the single-rig case. US-43. +- **Surfaces:** Settings > mesh block, Plans (campaign navigator + item rows), new Peers view · **Entities:** Mesh / peer instance, Campaign, Plan item, Session + +### IDEA-44 — Copy-id affordance on embryo identifiers `consistency / cross-surface parity` · impact low / effort low +The header already has copySessionId (clipboard + copied feedback). Embryo uids are the token you paste into chat or a note but have no copy affordance. Add a small copy icon next to the embryo uid, mirroring the session button. +- **Why:** Identical need, affordance exists only for the session id. Trivial parity win; kept for breadth, ranked last. +- **Surfaces:** Embryos cards / Operate worklist ('◌ uid'), Notebook chips · **Entities:** Embryo, Session + +### IDEA-45 — Inline-edit campaign metadata (target / status / description) `missing-affordance` · impact med / effort med +Make the campaign header fields editable in the inspector the same way plan-item spec fields already PATCH. Today only plan-item spec is editable; the parent campaign's own metadata is agent-only. +- **Why:** Plan item has an edit path but the parent Campaign doesn't — a researcher can't correct a target or flip status without the agent, though the inline-edit pattern already exists one level down. Pairs with IDEA-03. +- **Surfaces:** Plans tab (campaign inspector), Home > Recent Plans · **Entities:** Campaign, Plan item diff --git a/docs/product-ideation/CONSULT.md b/docs/product-ideation/CONSULT.md new file mode 100644 index 00000000..ca351739 --- /dev/null +++ b/docs/product-ideation/CONSULT.md @@ -0,0 +1,74 @@ +# Gently — Product-Ideation: Second Consult (Fable) + +A depth+critique pass over the ideation engine + 45-idea backlog (`BACKLOG.md`). Where the +generation run gave breadth, this reframes the backlog into a sequenced program, deepens the +flagships, and — critically — audits the engine as a *decision instrument*. + +## Thesis + +gently already lets a bench biologist DRIVE THE HARDWARE but only WATCH THE MIND: the agent holds a monopoly on authoring the shared record (ground truth, notes, learnings, watchpoints, the plan graph) and the human's sole write-path in is to dictate to chat — so the backlog is not "make it participatory" in general, it is one precise three-move program (AUTHOR the record, CONNECT it, TRUST the instrument), and it must be run keystone-first because two writes — ground-truth authoring and note authoring — are hard preconditions that gate the entire flywheel; meanwhile the engine that produced the backlog is a coverage instrument that can structurally only say "add" and scores impact by self-report, so its ranking must be corrected before it is trusted. + +## The three-move program (sequencing, not a flat 45) + +- AUTHOR THE RECORD (sequence 1) — break the agent's monopoly on writing the knowledge layer. Holds both keystones: ground-truth authoring (IDEA-01) and the note composer (IDEA-02), hard preconditions for the flywheel and notebook-link clusters, both sitting on persistence that already exists (set_ground_truth, a human-authored note model, create_campaign) so effort is bounded. First release: the two keystones + New-campaign/plan (IDEA-03) + human-raise-question (IDEA-18) + watch-this-embryo (IDEA-30); structural plan editing (IDEA-20) is the deep tail. FEEL: gently stops being a dashboard you watch the agent operate and becomes YOUR notebook the agent also writes in — a correction takes one click and MEANS something instead of vanishing into localStorage; the human moves from petitioner to co-author. + +- CONNECT THE RECORD (sequence 2) — turn separate tabs sharing a header into one traversable investigation. Extreme leverage because the foreign keys already exist as inert text: IDEA-04/06/11 are the cheapest, highest-ratio wins in the app (render a chip that navigates + reverse 'N notes' back-links). Then the flywheel joins (expectation-on-strip IDEA-14, setpoint-on-strip IDEA-15, dataset-readiness IDEA-25), then finder/linkifier (IDEA-27) and export (IDEA-08) as the app boundary. Most of these point at entities the human can only author once theme 1 ships. FEEL: an investigation becomes a thread you pull — 'did the thermal bump shift development?' is one glance, not a tab-hop reconstruction; the agent's conclusions become drillable to their evidence. + +- TRUST THE INSTRUMENT (cross-cutting P0, pulled to the front) — a cheap honesty slice jumps the whole queue: offline banner (IDEA-10), proportionate laser/temperature guards (IDEA-12, today INVERTED — laser-on and a +4C setpoint on live embryos fire unconfirmed while UI-only Remove/Stop DO confirm), and deleting the lying controls (IDEA-34: fake Agree/Disagree, static 'LIVE', shortcuts to dead tabs). Subtraction-and-honesty before addition — you must stop the observatory actively misleading before inviting writes into it. The deeper tail (presence chip IDEA-09, session lock IDEA-16, liveness/ETA IDEA-31, notify-me IDEA-21, watchpoint-fired triage IDEA-39) lands once authorship and traversal exist to escalate into. FEEL: the biologist can leave a 14-hour timelapse overnight and be called only when a human is needed. + +- SEQUENCING RULE the flat-45 ranking obscures: KEYSTONE-FIRST, not impact/effort-first. Two writes (ground-truth, notes) gate whole clusters, so do them before the cheap wiring even though the impact/effort formula ranks the wiring higher. A cheap honesty/subtraction slice cross-cuts to the very front. Then connect. Then the deep structural + agentic-triage tail. + +## Deepened flagships — shallow vs deep + +### IDEA-02 +SHALLOW: a '+ Add note' modal (free text + kind dropdown) POSTing author=human. Closes the US-36 authoring gap but produces an ISLAND note — no entity links, so it never enters the mesh IDEA-04 traverses, never surfaces on embryo/session/plan, and loses to 'I'll just tell Gently in chat' (agent-arbitrage red flag). It moves a verb into the UI without earning depth. DEEP: depth is not the textarea — it is the LINKS the note is born carrying and the consumers that read them back. Three properties a plain box can't have: (1) PROVENANCE-CARRYING IN-CONTEXT CAPTURE — authored FROM a surface ('Note this ->' on a detection card, session header, temp-graph excursion, VLM reasoning panel) so embryo/session/strain/timepoint/thread links auto-populate from where you stood; every note is a fully-wired node, not storage. (2) FIRST-CLASS PRODUCER — the note shows in IDEA-05's 'what we know' rail, IDEA-32 pre-commit review, and back-links on the plan item via IDEA-07. (3) LIFECYCLE — proposed->confirmed, and a human finding promotable into a Learning with basis, so notes accrete into memory instead of piling up as free text. DESIGN: two entry paths — a notebook-header inline composer (kind toggle Observation/Finding/Question, textarea, live EDITABLE link-chip row via the shared entity pickers) and contextual 'Note this ->' seeds; states collapsed->composing->(optional draft-with-Gently)->saved (animates into the list + Home 'From the notebook')->offline (draft preserved in localStorage so nothing is lost). AGENT STEP: 'Draft with Gently' — genuinely helps, never forced (type+auto-link works alone): human dumps rough text ('aiy looks arrested, maybe the ky123 arm'), agent types it (picks kind, tightens wording, resolves loose entity MENTIONS to real ids across 30+ embryos — the resolution a human can't do by hand — renders chips) and checks it against existing learnings, annotating corroborates/CONFLICTS ('agrees with finding ky123 ~80% lethal' / 'contradicts observation X'). Agent narrows and structures; human commits. + +### IDEA-01 +SHALLOW: swap the localStorage Agree/Disagree for a stage-picker dropdown POSTing one corrected stage to set_ground_truth + a 'corrected -> {stage}' badge. That is the SAME dead-end one layer deeper — nobody reads it back so the human sees no effect, and per-frame correction on a 120-frame strip is punishing. DEEP: ground truth is deep only if it visibly SEEDS a flywheel at the moment of authoring and is set in the natural unit. Four properties: (1) DOWNSTREAM NUMBERS MOVE the instant you correct — an 'agrees M/N' accuracy readout, the IDEA-25 annotated_embryos count, and the few-shot example pool all update in view, so the human sees judgement do work rather than vanish; (2) correction over a RANGE via drag-select on the stage strip ('stayed at comma t40-t58'), not per-frame, matching how development reads; (3) PROVENANCE — annotator + timestamp + the projection/trace it was judged from, so a second annotator's disagreement is shown, not silently overwritten; (4) it is the shared SUBSTRATE for IDEA-38 (per-run accuracy vs GT) and IDEA-36 (batch triage) — the one write that unlocks the whole perception-eval cluster. DESIGN: click a point or drag a range on the Vitals stage strip -> a stage-picker popover over the canonical vocabulary, PRE-FILLED with the agent's predicted stage so the common case is one 'confirm as ground truth' click; corrected points render as a filled diamond vs the model's hollow dot so agreement/divergence reads at a glance; conflicting annotators both shown + flagged. Also removes the misleading localStorage Agree/Disagree per IDEA-34. AGENT STEP: 'Confirm the obvious ones' (IDEA-36 batch triage as the acceleration path; per-point correction always remains) — agent pre-screens the strip into high-agreement stretches vs transitional/low-confidence frames, bulk-confirms the confident run as GT (one range write), and presents ONLY the uncertain frames — projection + second-opinion stage + Confirm/Correct — to the human. Collapses the 90% a human would rubber-stamp onto the ambiguous calls where human ground truth is worth most. + +### IDEA-04 +SHALLOW: make clickable so a strain chip fires switchTab+filter — one-way breadcrumbs that dump you into a filtered tab with no reciprocal 'notes about this' and no way back; the graph stays a star with a few one-way spokes. DEEP: make the notebook a bidirectional MESH (the framework's explicit G_nav goal — today zero entity-to-entity edges). (1) REVERSE LINKS everywhere the entity lives — an 'N notes ->' affordance on embryo card, session header, plan item, strain (the /notes endpoint already accepts the filter) so the edge is traversable from both ends; (2) DEEP-LINKS CARRY SELECTION STATE, not just a tab switch — embryo -> Embryos with that reasoning rail already selected, timepoint -> Gallery at that exact frame; (3) a persistent CLEARABLE 'filtered by emb_0007 x' context pill so you never lose your place; (4) note-id carry on Home->Notebook. Every edge is a foreign key already in the payload — highest leverage/effort win in the app. AGENT STEP: deliberately NONE — a pure structural/link win; an LLM here is exactly the agent-redundant bolt-on the framework hard-rejects. + +### IDEA-05 +SHALLOW: a side panel dumping every learning via unfiltered GET — undifferentiated noise at the moment of hardware commitment, not tied to the marked embryos, changes no decision. DEEP: scoped, decision-shaped, actionable memory delivered as a LEVER at commitment. (1) SCOPE-MATCH learnings/watchpoints/questions to the marked embryos + strain(s) + active plan item on the fields those records already carry, so 'ky123 ~80% lethal -> over-sample n=12' appears precisely because ky123 is in the set; (2) DECISION-SHAPED — each item states its implication for THIS run, not raw prose; (3) ACTIONABLE INLINE — 'apply to cadence/role/stop-condition', resolve/carry a watchpoint without leaving Operate; (4) TWO-WAY — the executed tactic links back to the learning that shaped it. Renders in the Run chooser + live tactic card; a learning that contradicts the composed tactic is an amber row; an unresolved control-holder question on these embryos soft-gates Start. AGENT STEP: pre-commit conflict check (IDEA-32 at the Operate moment) — matching free-text memory to a numeric tactic spec is fuzzy judgement a filter can't do; agent flags concrete conflicts ('your 5-min burst on ky123 conflicts with the phototoxicity finding') as Accept/Ignore cards, gating Start. CAVEAT (feeds distrust): the real effort is the UNBUILT relevance-matcher — this is not 'only a filtered read' — and the rail is EMPTY at cold-start, a new campaign's first Operate step, exactly when guidance matters most. + +### IDEA-03 +SHALLOW: a '+ New campaign' form POSTing to a create route, then dropping you into an empty navigator — a row-inserter that does not cross the real gap (intent -> a populated plan). DEEP: make creation the ENTRANCE to the guided-plan loop, not a row-inserter. (1) '+ New campaign' seeds the container AND offers 'design the first plan with Gently', firing the same /wizard hook the Home hero uses so you land in US-04 with the new campaign already attached; (2) inline '+ New plan' / '+ Add item' for the manual path; (3) TEMPLATES — clone an existing campaign's plan skeleton so a returning researcher isn't starting blank. Pairs with IDEA-45 (edit campaign metadata) so the container is correctable without the agent. AGENT STEP: the wizard hand-off IS the agent step and already exists — the deep contribution is wiring create -> wizard so a newly-originated campaign flows straight into agentic plan design (US-04) instead of dead-ending at an empty container. No new LLM surface is invented; an existing one is connected to the creation moment. + +### IDEA-10 +SHALLOW: a toast on 502/503 — whack-a-mole across 7 silent failures while the acquire/run buttons still look enabled and invite re-clicking the same dead action. DEEP: wire ONE online/offline signal SYSTEMICALLY to PREVENT the failure, not narrate it. (1) subscribe every hardware-acting control (Snap, Burst, stage-move, Start run, laser, setpoint) to the existing microscopeConnected status-store value -> disabled + 'offline' annotation, so the dead action can't be triggered; (2) ONE persistent amber banner ('Microscope offline — controls disabled . last seen 3m ago') instead of N toasts; (3) route residual/edge failures through the toast idiom control-auth.js already uses on 403; (4) a recovery path. Collapses the 7 per-story silent failures (US-09/10/11/12/14/18/26) into one state machine. AGENT STEP: 'Explain and recover' — on persistent offline the agent reads the device_layer log + last live status, names the likely cause, and offers ONE wired recovery action (restart device layer / retry enumerate). DISTRUST NOTE: build this as a CORRECTNESS FIX, not a flagship value bet — its rated impact is a harness artifact; see the distrust field. + +## Guardrails — the engine as a decision instrument (its failure modes) + +- ADD A REAL SUBTRACTION GENERATOR, not another lens. The two deriving engines (capability-orphan, dangling-edge) can only emit 'add X', and every empty matrix cell is defined as a candidate missing-affordance, so a blank can never mean 'this shouldn't exist'. Read G_nav for low-degree/redundant surfaces and the inventory for always-empty entities and emit merge/delete candidates by CONSTRUCTION the way orphan/dangling do; enforce a fixed ~15% subtraction quota that must survive into the ranked list; reframe every empty cell two-way ('fill this OR should this entity/surface exist?'). Today 44/45 ideas add surface area; subtraction produced exactly one (IDEA-34, ranked 34th). + +- SEPARATE COVERAGE FROM VALUE. A checkmark means 'an affordance exists', not 'anyone values it' — a fully-checked matrix would declare victory over a bloated app of dead affordances — and the impact/reach numbers driving the rank are LLM self-reported scores dressed as evidence, the exact anti-pattern in your own 'no self-reported confidence' rule. Forbid a checkmark from counting as value; derive value from a signal already on disk (timeline.jsonl / interaction_log.jsonl usage traces, or your own frequency ranking of the 36 stories); require every 'impact: high' to cite an observed friction in STATUS, not an asserted number. + +- GATE EVERY CROSS-FEATURE-LINK WITH A JTBD SENTENCE: 'a researcher doing task T, at moment M, needs to reach B from A because '. Auto-reject any link justified circularly ('so they can follow the link') or by a graph property alone. A structural gap is not user value — some FKs (e.g. basis links) exist for backend reasons no biologist will ever traverse; IDEA-44 copy-id is the honest tell that the generator yield is dominated by structurally-real, value-thin candidates. + +- RUN TWO TRACKS, NOT ONE RANKING: a sequencing track (impact/effort, cheap wins) AND an absolute-impact track (the 3 changes that alter the product's identity regardless of cost). Cap how many sub-1-day wiring items may occupy 'Top bets'. As-is, impact x 1/effort promotes micro-wiring (IDEA-04/11/13/03) to the very top and starves the identity-changing bets (IDEA-20 structural plan editing, IDEA-25 dataset flywheel, IDEA-39 triage) purely by cost — it tells you to do 15 tiny link-fixes before the one feature that redefines the product. + +- ADD A CLUSTER-DISSOLUTION PASS: for every cluster of 4+ ideas ask 'what single structural change makes this whole cluster unnecessary?' and score that re-architecture against the SUM of the cluster's effort+impact. Ideas are emitted as diffs against a FIXED star-of-tabs G_nav, so no generator can propose an entity-centric shell (select an embryo -> its predictions, ground truth, watchpoints, tactic, notes travel with it) that would dissolve ~8 link-items (IDEA-04/06/11/19/29/30/41/44) at once. Without this pass the engine keeps paying interest instead of retiring the debt. + +- TAG EVERY IDEA'S EVIDENCE AS PRODUCTION-CONDITION vs AUDIT-ARTIFACT. The audit ran with the device layer offline throughout (9 blocked stories, every acquire/stage/temp route 502) and the stores empty — both are harness artifacts, not steady state — yet high-impact ratings are being minted from them (IDEA-10's whole case is 'US-09/10/11/12/14/18/26 all 502'). Down-weight any 'high' resting on the offline device or empty dev data, and require re-validation against a live-rig + populated-store run before it can enter Top bets. + +## Distrust + +DISTRUST IDEA-10 (device-offline banner) in its top-6 slot — it is the clearest false positive because its entire impact case is a MEASUREMENT ARTIFACT of the audit, not a product signal. The audit ran with the device layer offline the whole time, so every hardware route 502'd, and the engine then congratulates itself for discovering that '7 stories 502 silently' is 'the richest untapped vein the happy-path core lenses were blind to' — but that vein is the test harness leaking into the results: on a working rig those same 7 stories succeed. The banner is unambiguously correct DEFENSIVE UX (silent 502s are always bad), so ship the cheap disable+banner as a correctness fix — but how often a scientist actually hits an offline device in normal operation is unknown and probably low, and was never measured because the device was never online to measure against. It is not wrong, it is badly OVER-RANKED into a flagship slot it earns only under the harness's artificial failure state, displacing a genuinely value-generating item. RUNNER-UP suspect: IDEA-05 (learnings-in-Operate), whose effort is understated — it is sold as 'only a filtered read of existing stores' but the real work is the unbuilt RELEVANCE-matching function that decides which of N free-text learnings apply to these embryos right now — and whose value COLLAPSES at cold-start: a brand-new campaign's first Operate step, exactly when guidance matters most, has an empty memory store and therefore an empty rail. + +## The paradigm upgrade to build next + +Build the CLOSE-THE-LOOP upgrade: promote the story audit from a sibling report into a FOURTH living graph, G_story, and make every backlog idea carry a FALSIFIABLE PREDICTION against the crawler's already-machine-readable per-story signals (STATUS emits toast_visible, resume, export, revealed, side, items, start). This turns the backlog from an open-loop wishlist into a self-verifying acceptance harness. MECHANISM — (1) PREDICTION CONTRACT: a new `predicts` field per idea binds to granular signals, not the coarse status label: IDEA-01 -> {US-31: ground_truth_control 0->>=1}, IDEA-03 -> {US-06 new_plan_control 0->1, US-35 create_campaign_control 0->1}, IDEA-10 -> {US-09 toast_visible False->True} (verifiable HEADLESS precisely because the device is offline — the exact failure it targets), IDEA-08 -> {US-32 export_control 0->>=1}. This forces the engine to distinguish UI-gap ideas (headless-verifiable) from rig/agent-blocked stories (9 of 36) that a UI change alone can't flip — a discipline the impact/effort guesses hide. (2) REVERSE GENERATION: any idea mapping to NO story (IDEA-09 presence chip, IDEA-15 setpoint overlay, IDEA-21 notify-me) must EMIT a new acceptance story with expected signals; idea<->story becomes a covering bijection, not a loose reference. (3) VERIFIER: after shipping, re-run run_stories.py -> new status.json; a diff scores each idea VERIFIED / REGRESSED / PENDING, replacing manual 'is IDEA-X done?' and catching regressions when a later change re-breaks US-31. (4) COVERAGE MATRIX (stories x ideas) exposes three blind spots as empty cells: gap-stories with no idea (generation miss), ideas with no story (scope creep / unverifiable), and the critical-path idea that unblocks the most stalled stories — a grounded reach metric ('flips N observed signals') that replaces hand-asserted high/med/low. WHY HIGHEST LEVERAGE: it closes the engine's OWN currently-open loop (generate->rank->ship->nothing feeds back); it is MECHANICAL (a diff over graphs, matching the G_nav/G_data/G_verb design philosophy and reusing run_stories.py + Playwright + the granular observation fields that already exist, so cost is low); and it binds to reality (the audit already records the exact booleans/counts a prediction needs, so predictions are falsifiable today with no new instrumentation). It also becomes the INSTRUMENT that adjudicates the one product-level paradigm move no generator can propose — entity-centric shell vs ~15 hand-wired links — because the coverage matrix can score which flips more stalled stories. FOLD IN the self-updating-backlog kernel (re-derive + diff on each re-run) as the DELIVERY mechanism, not a standalone build. REJECT persona-session simulation as the first build: it injects a fuzzy LLM oracle into an engine whose whole strength is mechanical derivation, duplicates the existing frequency x friction lens, and violates your own 'don't ask a model to self-rate confidence' principle — a possible later demand-side complement, not the supply-side loop-closer. + +## Recommended path + +1. SHIP the honesty/subtraction slice + the two keystones as the first release. Subtract before you add: delete the lying controls (IDEA-34 — fake localStorage Agree/Disagree, static 'LIVE' chip, shortcuts to dead tabs), fix the INVERTED guards (IDEA-12 — laser-on and +4C setpoint must confirm, cheap UI-only Remove/Stop need not), and wire the offline banner + control-disable (IDEA-10, as a correctness fix, not a flagship). Then land the two authorship keystones that gate every flywheel — ground-truth RANGE authoring on the stage strip (IDEA-01) and the context-capturing note composer (IDEA-02). You must stop the surface from misleading before you invite writes into it. + +2. CLOSE THE ENGINE'S LOOP before mining more ideas. Build G_story: a `predicts` contract per idea against the crawler's granular signals, reverse-generated acceptance stories for the idea-only axes (presence, offline-recovery, notify-me, export), and a verifier that re-runs run_stories.py and diffs VERIFIED/REGRESSED/PENDING. Re-rank the backlog on 'flips N observed signals' instead of self-asserted impact — this demotes IDEA-10 out of the flagship slot and lets the coverage matrix name the true critical-path unblocker. Also add the subtraction generator and the two-track ranking here so the instrument stops only ever saying 'add'. + +3. RUN THE 'CONNECT' THEME AGAINST A MEASURED DECISION, on a live rig. With the loop closed, score the entity-centric shell (one structural change that dissolves the IDEA-04/06/11/19/29/30/41 cluster) against the SUM of that cluster — build the shell if it wins the coverage-matrix count, else ship the cheap dead-FK chip renders + reverse 'N notes' back-links (IDEA-04) and Operate<->memory (IDEA-05). Re-validate every remaining top bet against a live device + populated stores before crowning it, so no harness artifact is mistaken for product signal. + +--- +*Second consult by Claude Fable 5. Enhances `FRAMEWORK.md` / `BACKLOG.md`; some claims +(esp. the inverted safety guards, and any 'high' impact resting on the device-offline audit) +need verification against a live rig + populated stores before acting.* \ No newline at end of file diff --git a/docs/product-ideation/ENTITIES.md b/docs/product-ideation/ENTITIES.md new file mode 100644 index 00000000..dbc4c57f --- /dev/null +++ b/docs/product-ideation/ENTITIES.md @@ -0,0 +1,251 @@ +# Gently — Entity Inventory + Matrices + +30 domain entities the UI exposes, with where they surface and which operations exist today. + +## Entity × Operation matrix + +✓ = affordance exists in the UI · (blank) = candidate missing-affordance. + +| Entity | view | create | edit | delete | link | export | +|---|:--:|:--:|:--:|:--:|:--:|:--:| +| Session | ✓ | | | | ✓ | | +| Embryo | ✓ | ✓ | ✓ | ✓ | ✓ | | +| Volume/Image | ✓ | ✓ | | | | | +| Projection | ✓ | | | | | | +| Prediction/Stage | ✓ | | | | | | +| Trace | ✓ | | | | | | +| Ground truth | | | | | | | +| Campaign | ✓ | | | | ✓ | ✓ | +| Plan item | ✓ | | ✓ | | ✓ | | +| Plan item dependency | ✓ | | | | | | +| Planned session | ✓ | | | | | | +| Operation Plan | ✓ | | | | | | +| Tactic | ✓ | | | | | | +| Tactic library | ✓ | | | | | | +| Notebook note | ✓ | | | | ✓ | | +| Learning | ✓ | | | | | | +| Observation | ✓ | | | | | | +| Question | ✓ | | | | ✓ | | +| Watchpoint | ✓ | | | | | | +| Expectation | ✓ | | | | | | +| Role | ✓ | | | | ✓ | | +| Setpoint (temperature) | ✓ | | ✓ | | | | +| Temperature sample/graph | ✓ | | | | | | +| Device state | ✓ | | | | | | +| Agent chat / turn | ✓ | ✓ | | | | | +| Ask (agent → human) | ✓ | | | | | | +| Event / log | ✓ | | | | | | +| Config / dashboard prefs | ✓ | | ✓ | | | ✓ | +| Mesh / peer instance | ✓ | | | | | | +| Auth / control | ✓ | | | | | | + +## Entity × Entity — related but NOT linked in the UI + +Pairs worth connecting (value med/high) with no traversable UI path today — the cross-feature backlog. + +| A | ↔ | B | value | +|---|:--:|---|:--:| +| Session | ↔ | Notebook note | high | +| Embryo | ↔ | Ground truth | high | +| Embryo | ↔ | Watchpoint | high | +| Embryo | ↔ | Expectation | high | +| Embryo | ↔ | Notebook note | high | +| Embryo | ↔ | Tactic | high | +| Prediction/Stage | ↔ | Ground truth | high | +| Prediction/Stage | ↔ | Expectation | high | +| Prediction/Stage | ↔ | Tactic | high | +| Campaign | ↔ | Learning | high | +| Campaign | ↔ | Notebook note | high | +| Plan item | ↔ | Notebook note | high | +| Operation Plan | ↔ | Learning | high | +| Tactic | ↔ | Learning | high | +| Session | ↔ | Campaign | med | +| Session | ↔ | Planned session | med | +| Embryo | ↔ | Question | med | +| Campaign | ↔ | Tactic | med | +| Campaign | ↔ | Mesh / peer instance | med | +| Plan item | ↔ | Tactic | med | +| Tactic | ↔ | Role | med | +| Learning | ↔ | Observation | med | + +## Inventory + +### Session +A single run of the microscope (folder under sessions/, session.yaml + lock + timeline). The unit that owns embryos, volumes, events, and temperature. Created by starting a run; carries an intent (planned vs actual). +- **Surfaced in:** Home > Recent Sessions, Sessions tab, header session-id badge/link (#session-id-link), landing > Resume, Logs tab (its events) +- **Ops today:** view, list, act(resume via POST /api/sessions/{id}/resume), act(copy id / touch), link(to campaign, only via a plan-item inspector) +- **Relates to:** Embryo, Volume/Image, Operation Plan, Campaign, Plan item, Session intent, Event/log, Temperature sample, Notebook note + +### Embryo +A marked/tracked C. elegans embryo with position, calibration, uid; the subject of perception. Detected on the bottom cam (SAM) then confirmed into the single worklist ('THE PLAN'). +- **Surfaced in:** Embryos tab (Default/Board/Film/Vitals), Operate view worklist (#op-board 'THE PLAN'), header embryo count, Notebook note chips, Gallery filter 'All embryos' +- **Ops today:** view, list, create(detect via POST /api/devices/detect_embryos + confirm /api/devices/embryos/confirm), edit(position PUT /api/embryos/{id}/position), delete(DELETE /api/embryos/{id}), act(mark, center /api/devices/stage/move), link(assign role POST /api/embryos/roles) +- **Relates to:** Session, Volume/Image, Prediction/Stage, Trace, Role, Embryo understanding, Ground truth, Watchpoint, Notebook note + +### Volume/Image +An acquired 3D stack (volumes/t{NNNN}.tif + meta) plus its 2D projection (jpg) and standalone snapshots. The raw imaging payload. +- **Surfaced in:** Gallery tab (type filters Volume/Projection/Snapshot), Devices > 3D view, Home > Recent Images, timepoint player / lightbox, Calibration > Gallery +- **Ops today:** view, create(acquire POST /api/devices/acquire/volume, /acquire/burst), act(3D render /api/volumes3d, slice, raw /api/volume-raw), list(/api/volumes,/api/snapshots) +- **Relates to:** Embryo, Session, Projection, Prediction/Stage, Trace + +### Projection +2D max/summary projection derived from a volume, the default thumbnail shown for a timepoint. +- **Surfaced in:** Gallery tab, Embryos > Default ('View All Projections'), projection-viewer, Sessions (session projection) +- **Ops today:** view, act(view-all / paginate via projection-viewer) +- **Relates to:** Volume/Image, Embryo, Prediction/Stage + +### Prediction/Stage +Per-timepoint developmental-stage call for an embryo (predicted_stage, confidence, is_transitional) appended to predictions.jsonl. The core perception output. +- **Surfaced in:** Embryos > Board (stage sparkline column), Embryos > Vitals (stage strip chart over timepoints), Embryos > Default (detection cards) +- **Ops today:** view, act(agree/disagree — saved only to localStorage, not persisted) +- **Relates to:** Embryo, Trace, Ground truth, Expectation, Volume/Image + +### Trace +The complete perception record for a timepoint (traces/t{NNNN}.json) — classifier/perceiver VLM reasoning and observed features behind a prediction. +- **Surfaced in:** Embryos > Default ('Show VLM reasoning', raw trace JSON) +- **Ops today:** view +- **Relates to:** Prediction/Stage, Embryo, Volume/Image + +### Ground truth +Human-annotated correct stage for an embryo/timepoint (ground_truth.yaml; FileStore.set_ground_truth/get_ground_truth). Exists in storage but has NO authoring UI — the only feedback is binary Agree/Disagree to localStorage. +- **Surfaced in:** (no dedicated surface — backend store only; would live on Embryos detection cards) +- **Ops today:** — +- **Relates to:** Prediction/Stage, Embryo + +### Campaign +A long-running research program with a description, target, status and hierarchy; the container for phases/plan items and linked sessions. Created only by the agent (create_campaign tool), never from the UI. +- **Surfaced in:** Plans tab (campaign navigator + canvas + inspector; Doc/Graph/Board/Decide/Matrix/Timeline), Home > Recent Plans, landing plan-wizard ('Continue …' / 'Start new campaign') +- **Ops today:** view, list(/api/campaigns), act(tree /tree, document /document, versions /versions), link(session via item inspector /items/{id}/sessions) +- **Relates to:** Plan item, Session, Planned session, Tactic, Learning, Plan snapshot/version + +### Plan item +A single unit of work inside a campaign (imaging/bench/genetics/analysis/decision_point) with title, status, dependencies, references, and an ImagingSpec. Structural edits (add/remove/reorder/deps) are agent-only. +- **Surfaced in:** Plans tab > Doc (item rows, '→ blocks:'), Plans > Graph/Board/Matrix/Timeline/Decide, item inspector +- **Ops today:** view, edit(inline field edits in inspector), link(session link/delink), act(resolve imaging spec) +- **Relates to:** Campaign, Session, Planned session, ImagingSpec, Plan item dependency + +### Plan item dependency +Blocks / blocked-by edges between plan items driving readiness (get_unblocked_plan_items). +- **Surfaced in:** Plans > Doc ('→ blocks: …'), Plans > Graph, Plans > Decide +- **Ops today:** view +- **Relates to:** Plan item, Campaign + +### Planned session +A scheduled future session (title, date/time, estimated duration, acquisition params inherited from a source session) that becomes an actual Session when started. +- **Surfaced in:** Plans tab (campaign planned-sessions /api/campaigns/{id}/planned-sessions), plan-item inspector +- **Ops today:** view, list +- **Relates to:** Campaign, Plan item, Session, ImagingSpec + +### Operation Plan +The live per-session tactic spine (operation_plans/{session}.yaml) — the ordered set of tactics with states (done/in-use/queued) that the agent is executing right now. +- **Surfaced in:** Operations tab > Overview (title + tactic spine) +- **Ops today:** view +- **Relates to:** Session, Tactic, Embryo, Setpoint (temperature) + +### Tactic +An imaging behavior/protocol card (monitor, transmission burst, temp-change burst, recovery monitor …) with rationale/scope/cadence; the executable step of an Operation Plan and the reusable unit saved to the library. +- **Surfaced in:** Operations > Overview (expandable tactic cards, US-17), Operate > Run chooser ('From library', 'Continue a plan', 'Hand to agent'), tactic library +- **Ops today:** view, act(expand card), act(run POST /api/operate/run-tactic), act(apply saved tactic /api/tactic_library) +- **Relates to:** Operation Plan, Embryo (scope), Role, Setpoint (temperature), Campaign + +### Tactic library +Saved reusable tactics (agent/ml store) that can be instantiated into a run. +- **Surfaced in:** Operate > Run chooser ('From library — a saved tactic') +- **Ops today:** view, list(/api/tactic_library), act(apply into run) +- **Relates to:** Tactic, Operation Plan + +### Notebook note +The unified shared-lab-notebook entry: kind (observation/finding/question), author (human/agent), status (proposed/confirmed/open), with strain/embryo/session/thread/basis links. Read-only in the UI — there is NO 'add note' control. +- **Surfaced in:** Notebook tab (kind filters, thread rail, Ask box), Home / Agent chat 'AGENT'S VIEW > From the notebook', context-surface 'Agent's view' +- **Ops today:** view, list(/api/notebook/notes), filter(kind/author/status/strain/embryo/thread), act(ask /api/notebook/ask), link(threads /api/notebook/threads) +- **Relates to:** Learning, Observation, Question, Embryo, Session, Campaign, Plan item + +### Learning +A durable agent insight (content, confidence, basis) accumulated in memory (learnings/*.yaml). Surfaces in the notebook as FINDING-kind notes. +- **Surfaced in:** Notebook tab > Findings filter +- **Ops today:** view +- **Relates to:** Observation, Notebook note, Embryo, Campaign + +### Observation +A recorded observation (stage_transition/anomaly/session_summary/milestone) with significance and gently_refs. Surfaces as OBSERVATION-kind notes. +- **Surfaced in:** Notebook tab > Observations filter +- **Ops today:** view +- **Relates to:** Notebook note, Embryo, Session, Learning + +### Question +An open question capturing agent uncertainty (or a human-posed one). Resolvable only by the control holder. +- **Surfaced in:** Notebook tab > Questions filter, context-surface 'Open questions' +- **Ops today:** view, act(resolve/answer POST /api/context/questions/{id}/resolve — control), link(thread) +- **Relates to:** Notebook note, Embryo, Watchpoint, Expectation + +### Watchpoint +An active attention target (embryo + condition, e.g. 'approaching hatching') with priority. No dedicated tab — lives only in the always-on agent's-view surface. +- **Surfaced in:** context-surface 'Watching' +- **Ops today:** view, act(resolve POST /api/context/watchpoints/{id}/resolve — control) +- **Relates to:** Embryo, Question, Expectation + +### Expectation +An agent belief about the future ('will reach comma stage' by expected_time, with uncertainty/basis) — the agent's forward prediction, distinct from a per-timepoint stage Prediction. +- **Surfaced in:** context-surface 'Expectations' +- **Ops today:** view, act(confirm POST /api/context/expectations/{id}/resolve — control) +- **Relates to:** Embryo, Prediction/Stage, Watchpoint + +### Role +An embryo role from the static registry (unassigned/test/calibration/lineaging/subject/reference…) with role_class (subject vs reference), default cadence, and detector; governs how Operations foregrounds and images an embryo. +- **Surfaced in:** Operate > Run chooser ('ROLES (marked → subject)' chips), roles registry /api/roles +- **Ops today:** view, list, act(assign toggle subject/reference) +- **Relates to:** Embryo, Tactic, Cadence + +### Setpoint (temperature) +A temperature target for the stage/thermalizer (ACUITYnano). Set from Devices, changed inside a temp-change tactic, and configured (serial/MQTT) in Settings. +- **Surfaced in:** Devices header temp control (input + Set; hidden until controller online), Operations tactic card ('→ 32.0 °C', 'setpoint change'), Settings > Hardware > Thermalizer +- **Ops today:** view, act(set POST /api/devices/temperature/set — control), edit(config /api/devices/temperature/config), act(test connection /config/test) +- **Relates to:** Temperature sample/graph, Session, Tactic, Device state + +### Temperature sample/graph +Live water/setpoint temperature trace (temperature log per session; TEMPERATURE_UPDATE events + history backfill). +- **Surfaced in:** Devices tab temperature graph, Operations tactic 'STAGE TEMP' readout +- **Ops today:** view, act(history /api/temperature/{session}/history) +- **Relates to:** Setpoint (temperature), Session + +### Device state +Live hardware state and controls — XY stage, bottom cam, SPIM/lightsheet, laser, LED/room-light, F-drive, piezo/galvo. The 'scope' surface reachable with no plan. +- **Surfaced in:** Devices tab (Operate/Map/Details/3D/Manual), landing 'Take a quick look' +- **Ops today:** view(/api/device-status), act(stage move, camera start, laser off, led/room-light set, live params, F-drive/bottom-Z nudge) +- **Relates to:** Embryo, Volume/Image, Setpoint (temperature), Session + +### Agent chat / turn +The docked conversation with the agent — the delegation and steering channel; also the only place campaigns/plans get created and the notebook is briefed. +- **Surfaced in:** docked Agent chat panel (#agent-chat), 'Talk to Gently' rail button, landing 'or just tell me what you need' +- **Ops today:** view, act(send), act(stop/interrupt turn), act(queue message while busy) +- **Relates to:** Ask, Session, Campaign, Notebook note, Operation Plan + +### Ask (agent → human) +A pending question/choice the agent raises mid-turn, rendered prominently on the main stage as well as in the transcript; answered by the control holder. +- **Surfaced in:** #ask-stage (main stage), Agent chat transcript +- **Ops today:** view, act(answer choice — control) +- **Relates to:** Agent chat / turn, Question + +### Event / log +Session event stream and timeline (timeline.jsonl / interaction_log). The audit trail of what happened. +- **Surfaced in:** Logs tab (Log/Timeline/Summary views), footer counters ('N events') +- **Ops today:** view, list(/api/events), act(clear) +- **Relates to:** Session, Embryo, Volume/Image + +### Config / dashboard prefs +Effective server config (read-only), per-browser dashboard view prefs, alert thresholds, and restart-required advanced tunables. +- **Surfaced in:** Settings page (Views/Alerts/Ambient/Board/Filmstrip/Vitals/Default/Effective config/Advanced) +- **Ops today:** view, edit(dashboard-defaults PUT, settings-overrides PUT, advanced save), export(prefs JSON), import(prefs JSON), act(save as rig defaults / reset) +- **Relates to:** Embryo (view rendering), Setpoint (temperature), Mesh / peer + +### Mesh / peer instance +Other gently instances on the network (peer discovery, campaign sharing/participants server-side). Has NO interactive UI — only a read-only block in Settings and Advanced thresholds. +- **Surfaced in:** Settings > Effective config ('mesh' block, read-only), Settings > Advanced ('Mesh network' thresholds) +- **Ops today:** view +- **Relates to:** Campaign, Config / dashboard prefs + +### Auth / control +The control-vs-view-only model: signing in grants control of the microscope; logged-out users watch read-only. No discoverable sign-in in the workspace — surfaced reactively via a 403 control-toast or the /login URL. +- **Surfaced in:** /login page, control-auth toast (control-auth.js) +- **Ops today:** act(login POST /api/auth/login), act(logout), act(continue view-only), view(/api/auth/me) +- **Relates to:** Session, Setpoint (temperature), Question, Watchpoint, Expectation, Agent chat / turn diff --git a/docs/product-ideation/ENTITY-EVOLUTION.md b/docs/product-ideation/ENTITY-EVOLUTION.md new file mode 100644 index 00000000..1da99be7 --- /dev/null +++ b/docs/product-ideation/ENTITY-EVOLUTION.md @@ -0,0 +1,139 @@ +# Gently — Entity Evolution (Fable assessment) + +Fable's forward-looking read of how gently's domain model should evolve to serve the goal — +agentic smart microscopy + integrated ELN + user/session/project management — extensible across +organisms / modalities / tactics, while staying grounded in today's diSPIM + C. elegans reality. +Companion to `ENTITIES.md` (the current model). + +## Verdict + +The evolution gently needs is not a new taxonomy of entities — it is (1) closing the empty scientific middle of the graph and (2) generalizing the two seams that are still welded to code, while explicitly refusing to generalize the seam that has only one instance. Grounding matters: the organism axis is already the right shape — gently.organisms is a plugin package (get_organism/load_organism, per-organism STAGES/STOP_CONDITIONS/SAMPLE_TERM/detector presets/perception prompt/timing) selected by config — so 'stage-vocab → DevelopmentalModel' is ~80% done as a code plugin and must not be re-invented as a parallel YAML entity. Accounts already exist (AccountStore, viewer/operator/admin). Note already carries strains/embryos/sessions/threads/basis/links/artifacts, subsuming the generic thread/attachment/citation proposals. So the real shape is: add a thin ELN scientific spine (Strain-record → Experiment → Hypothesis → Result) that turns today's data-collection loop into a claim-testing loop, and perform ONE structural generalization now — Setpoint → typed Actuator — because that machinery already exists end-to-end for temperature and generalizing the type unlocks every other environmental axis plus reactive closed-loop control for near-free. Everything else (Specimen rename, Modality/Device capability layer, Channel, Team/Permission/Booking, Ontology, Inventory) is a seam to shape but not build until a second organism, modality, fluorescence assay, or tenant physically arrives. + +## Minimal high-leverage new entities + +### Strain / Specimen line +Promote the bare 'strain' string to a record: genotype, fluorescent markers/reporters, background, source/lab, physical stock (thaw/freeze dates), and a reference to its Organism plugin. Stored under agent/strains/{id}.yaml. ImagingSpec.strain/genotype/reporter, BenchSpec.strains[], Note.strains[], and ground_truth all resolve to it (with string back-compat). +- **Why:** It is the single most-referenced thing in the codebase that isn't an entity — strain/genotype/reporter are duplicated bare strings across ImagingSpec, BenchSpec, and Note. It is simultaneously the top ELN win (attributable, queryable genotype/stock provenance) AND the organism-generalization seam: a Strain belongs to an Organism (already a plugin), so swapping organism leaves the entire downstream graph unchanged. Keystone — unblocks the most with the least. + +### Experiment +The controlled-comparison unit that sits between Campaign (program) and Session (run): named arms/conditions (e.g. 32C vs control), replicates, defined controls, the Strain(s) under test, and a link to the Hypothesis it tests. Groups the Sessions that constitute its arms/replicates. Stored under agent/experiments/{id}.yaml, parented to a Campaign. +- **Why:** The empty middle of the model. Campaign is a long-running goal, SessionIntent/PlannedSession is one run, Project is work-decomposition — nothing expresses 'these 6 sessions are 3 replicates of two arms of one comparison.' It makes n, controls, and arms first-class instead of implicit in a folder of sessions, and it is the object Result and Hypothesis attach to. Reuses the existing campaign-hierarchy plumbing. + +### Hypothesis +A falsifiable claim with status (proposed/supported/refuted/inconclusive) and the predictions it entails, bound to the Experiment(s) testing it. Consolidates the diffuse triad of Expectation (forward belief), Question (uncertainty), and Learning (accumulated insight) around a single testable statement. +- **Why:** The agent already forms Expectations (which carry expected_time and an auto-resolvable status) and logs Learnings, but nothing states the claim they bear on or records whether it survived. Adding it turns the loop from data-collection into claim-testing with almost no new machinery — Expectation auto-resolution + existing Ground truth become the scoring mechanism. Small entity, largest conceptual leverage. + +### Actuator (generalized environmental control) +A typed environmental/perturbation actuator {actuator_type, unit, setpoint, ramp, sample_stream_ref, safety_bounds, device_ref}. Temperature becomes actuator_type='temperature'; the same shape covers CO2, O2, humidity, illumination, drug/microfluidic dose. Generalizes today's temperature-only Setpoint + temperature_sampler + temp-change tactic. +- **Why:** Temperature is currently the ONLY environmental variable and is welded end-to-end (Setpoint, temperature_sampler, the temp-change tactic, ACUITYnano driver, event_bus.target_setpoint_c). It is the highest-leverage generalization of an entity that already works end-to-end: generalizing the type makes the whole setpoint/sample/graph/perturbation-tactic stack work for any axis for free, and turns the actuator into the bindable channel that reactive closed-loop control requires. + +### Result / Measurement +A quantitative or derived finding distinct from a prose Learning — a value/table/derived dataset (hatch rate per arm, stage-transition timing, dose-response curve, accuracy vs ground truth) with its method, inputs, and provenance. Stored under agent/results/, attached to an Experiment/Hypothesis. +- **Why:** Learning/Observation capture prose insight; nothing captures a measured number with its derivation. It is the object a figure, a stats test, or a paper reads, and it is what makes the Hypothesis-scoring and perturbation-response loops persist a comparable, exportable artifact instead of two charts on separate tabs. The consumable output the ELN half of the mission needs. + +## Top combinations (capability-level) + +- **Hypothesis → Expectation → outcome falsification engine** — Makes the scientific method a first-class computational object. Hypothesis + Result plug into machinery that already exists — Expectation carries expected_time and an auto-resolvable status, Ground truth is already set/get in FileStore, Predictions are run_id-tagged. When expected_time passes, outcomes auto-score against Predictions/Ground truth, confirm or refute the Hypothesis, and persist a scored Result with a calibrated Learning. The agent proposes, the instrument tests, the record self-scores — replacing hand-confirmed dead-end Expectations. +- **Experiment as controlled comparison (arms × replicates × controls)** — Turns a folder of sessions into an analyzable design. Experiment + Strain + the Sessions it groups + Result make n, arms, and controls first-class, so cross-arm readouts (developmental rate vs temperature, time-to-hatch, lethality fraction) are computed across the experiment's sessions automatically rather than reconstructed by hand. This is the ELN scientific spine and the unit another lab or instance can reproduce. +- **Generalized perturbation-response measurement** — The actual science the thermal rig exists to produce, elevated from two disjoint charts to a defined, reusable measurement — and freed from temperature. Actuator (generalized Setpoint) + Prediction/Stage + Result join the perturbation trace and the developmental readout into a persisted dose-response Result, and because Actuator is typed, the same capability works for CO2/O2/illumination/dose the moment a driver exists, with zero new response-analysis code. +- **Reactive closed-loop control (watchpoint → actuator)** — Genuine reactive smart microscopy versus attention that merely notifies. A Watchpoint today fires and dead-ends at a manual Resolve; with Actuator as a bindable channel, a watchpoint condition binds to an actuator or tactic under a proportionate approve-only gate ('embryo approaching hatch → switch to recovery-monitor at 2-min'). Reuses the existing operator/approve posture as the trust boundary, and Result captures the response it triggered. +- **Strain-anchored provenance across organisms** — One record makes genotype/marker/stock provenance queryable and cross-session, and makes the organism axis real. Strain (with an Organism-plugin ref) links Experiment, Note, Ground truth, and EmbryoUnderstanding; because the staging vocabulary, detector presets, and perception prompt already come from the organism plugin, registering a new organism + its strains leaves the entire entity graph intact — the extend-to-other-organisms path becomes configuration, not code. + +## Extensibility architecture + +Three layers, and the tell of a correct generalization is that adding organism #2 or actuator #2 touches only layers 1-2, never model.py. + +LAYER 1 — DATA (config, not code): stage lists/ordering/terminal stages/expected durations, sample terms, detector presets and size priors, the actuator roster + safety bounds, per-device capability manifests, acquisition recipes, the tactic-kind registry, and controlled-vocabulary terms. These describe; they do not branch. + +LAYER 2 — PLUGIN (code behind a named contract): organism modules (perception prompts, transition/staging behavior — genuinely behavioral, not just data), device drivers, actuator drivers. New organism/device/actuator = a new plugin satisfying the contract, not new branches in core. + +LAYER 3 — CORE (rarely changes): the entity graph (Strain→Organism, Experiment→Sessions, Hypothesis→Expectation→Result, Actuator→Device), FileStore/FileContextStore, the perceive→decide→act loop. + +Applying this to the four named seams: + +STAGE-VOCAB → DEVELOPMENTALMODEL: already done, as a code plugin. gently.organisms exposes STAGES/STOP_CONDITIONS/SAMPLE_TERM/detector_presets/perception_prompt/TIME_TO_HATCHING per organism, selected by config. Do NOT re-invent it as a YAML entity — that would duplicate working code and strangle the genuinely behavioral parts (staging logic, prompts). Instead: (a) formalize the organism module's required exports as an explicit Protocol/interface with a schema-validation test, and (b) push the purely declarative bits (stage list, ordering, terminal stages, durations, sample term) into a data file the module loads, so a new organism ships mostly-data with only real behavior in code. That formalized contract IS the DevelopmentalModel. + +TEMPERATURE-SETPOINT → ACTUATOR: the one generalization to do now, because the machinery already exists once (Setpoint, temperature_sampler, temp-change tactic, ACUITYnano driver, event_bus.target_setpoint_c). Introduce a typed Actuator record; temperature becomes actuator_type='temperature'; the sampler/graph/perturbation-tactic become actuator-parameterized. Which actuators a rig has becomes a list in hardware.yaml, each pointing at a driver satisfying the actuator capability contract (layer 2). + +EMBRYO → SPECIMEN: do NOT do the physical rename now (it touches the embryos/ storage layout, FileStore API, EmbryoUnderstanding, roles, UI). The organism plugin's SAMPLE_TERM/SAMPLE_TERM_PLURAL already delivers ~90% of the human-facing benefit; route all 'embryo' display text through it. Add specimen_type as a field on EmbryoInfo/EmbryoUnderstanding defaulting to 'embryo' as the only hedge. Specimen geometry/size priors and the detector choice already live in the organism plugin (detection_defaults, detector_presets) — that is the Specimen profile. Perform the rename when a non-embryo subject actually arrives. + +DISPIM → MODALITY/DEVICE: shape the seam, do not build it. config/hardware.yaml + device_factory.py + bluesky literal kwargs (xy_stage=, volume_scanner=) weld everything to one modality. Two zero-behavior-change preparations only: (1) add a capabilities manifest to hardware.yaml documenting what each device advertises (moves-XY, scans-volume, emits-light@λ), and (2) lift the already-enumerated tactic kinds (standing_timelapse|reactive_monitor|scripted_protocol|exclusive_burst|oneshot|custom) into a TacticType registry with declared required-capabilities, so a new tactic ships as a validated registry entry. Hold the device_factory/capability-binding refactor until modality #2 is physically on the bench. + +## Sequencing + +1. Strain record first (keystone). Create agent/strains/{id}.yaml with genotype/markers/background/stock + an Organism-plugin ref; give ImagingSpec.genotype/reporter a home; migrate ImagingSpec.strain / BenchSpec.strains[] / Note.strains[] / ground_truth string refs to resolve to Strain ids with string back-compat. Unblocks ELN attribution and the organism seam simultaneously. +2. Experiment (the empty middle). Group Sessions into arms/replicates/controls, link to Strain(s), parent under Campaign. Reuse the existing campaign-hierarchy plumbing rather than a new store schema. Keep membership OPTIONAL — a Session need not belong to an Experiment. +3. Hypothesis + Result together. Bind Hypothesis to Experiment; wire the existing Expectation auto-resolution (expected_time already present) plus Ground truth to score it; persist scored outcomes as Result. Closes the falsification loop on mostly-existing machinery. +4. Actuator generalization. Refactor Setpoint → typed Actuator (temperature = first and only concrete instance); generalize temperature_sampler/graph/perturbation-tactic to actuator_type. The perturbation-response Result then comes nearly free. +5. Reactive closed-loop. Add Watchpoint→Actuator/Tactic binding under an approve-only gate, now that Actuator is a bindable channel and Result exists to measure the response. +6. Only on trigger, not on schedule: AcquisitionRecipe extraction from ImagingSpec (when modality #2 arrives); Model-version promotion of the existing ml/runs store (when a retrain is actually run against corrected ground truth); Channel/Fluorophore (when a fluorescence multichannel assay is real); Specimen rename + Modality/Device capability layer (when organism #2 / modality #2 hits the bench). + +## Risks — where NOT to generalize yet (YAGNI) + +- Modality/Device generalization now is premature. One rig, one modality; the device_factory.py + bluesky literal-kwarg refactor is high blast-radius for zero current payoff. Ship only a documentation-level capabilities manifest in hardware.yaml; defer the code refactor until modality #2 is physically present. +- Embryo→Specimen rename now is pure churn. It touches storage layout, FileStore API, EmbryoUnderstanding, roles, and UI while the organism plugin's SAMPLE_TERM already delivers the human-facing benefit. A defaulted specimen_type field is enough; do the rename only when a non-embryo subject actually exists. +- Do NOT re-invent the organism/DevelopmentalModel as a new entity. It already exists as the gently.organisms plugin. The explorations' 'Organism profile' / 'Staging-vocabulary' entities would duplicate working code and create a parallel YAML system fighting the code plugin. Formalize the existing contract instead. +- Hypothesis/Experiment/Result can over-formalize exploratory work. Much of gently use is watch-and-see, not claim-testing. Keep these as OPTIONAL overlays (a Session need not belong to an Experiment) and keep the low-friction Note taxonomy (observation/finding/question) as the default, or operators will route around the bureaucracy. +- The multi-tenant cluster (Team, Permission, Booking, Provenance/audit, Sign-off, Consumable/Inventory, Ontology/Tag, Figure) does not earn its complexity on a single rig with one operator. AccountStore + Note.links/threads/artifacts already cover the single-operator case. Building these now is speculative generality that will be wrong when the real cloud-venture multi-tenant requirements land — defer to that track. +- Don't model actuators you don't have. Ship the typed Actuator abstraction with temperature as the only concrete instance; the ramp/safety-bounds fields for CO2/O2/dose are untested fiction until a driver exists. Add each type when its driver arrives. +- Sequence the cheap authorship win before the expensive access graph. Accounts already exist; wire the account id into Note.author and ground-truth authorship first. Do NOT build Team×object×level Permission grants before that cheap wiring is even in place. +- AcquisitionRecipe and Model-version are real but not yet load-bearing. Extracting the recipe before a second modality, or promoting ml/runs to a versioned Model before a retrain is actually run, adds indirection with no consumer. Let the trigger pull them, not the roadmap. + +--- +## Appendix — raw explorations + +### ELN + user/session/project entities + +- **Strain / Line** (ELN, organism-ext, _now_) — First-class specimen line: genotype, allele(s), fluorescent markers/reporters, background, source/lab-of-origin, and links to its physical Strain-stocks (freezer vials with thaw/freeze dates). Stored as agent/strains/{id}.yaml. Today 'strain' is a bare string referenced by Notebook note, Embryo, and Ground truth but has no record of its own. +- **User / Account** (project-mgmt, ELN, agentic, _now_) — Persistent human identity: name, ORCID/email, and the account behind a login. Replaces the transient control 'holder_label' string with a durable actor that authored a note, corrected a stage, holds control, or signed off. Stored under agent/users/. +- **Experiment** (ELN, project-mgmt, _now_) — The controlled-comparison unit that sits between Campaign (program) and Session (one run): named conditions/arms (e.g. 32°C vs control), replicates, defined controls, the strain(s) under test, and the hypothesis it tests. Groups the Sessions that constitute its replicates/arms. Stored as agent/experiments/{id}.yaml. +- **Hypothesis** (ELN, agentic, _now_) — A falsifiable claim with status (proposed/supported/refuted/inconclusive), the predictions it entails, and the experiment(s) testing it. Promotes the diffuse triad of Expectation (forward belief), Question (uncertainty), and Learning (accumulated insight) into a single testable statement the whole loop is organized around. Stored under agent/hypotheses/. +- **Protocol / SOP** (ELN, tactic-ext, modality-ext, _now_) — A versioned, reusable procedure with steps, parameters, materials, and expected duration — for bench, genetics, mounting, AND imaging. Tactic is its imaging-only special case; a plan item of kind bench/genetics attaches a Protocol the way an imaging item attaches an ImagingSpec/Tactic. Stored under agent/protocols/{id}/ with version history. +- **Protocol-run** (ELN, tactic-ext, _next_) — An execution instance of a Protocol: which version ran, actual parameters, operator, timestamps, and recorded deviations from the SOP. Generalizes what Operation Plan is for imaging (the live tactic spine) to the whole wet-lab. Stored per session or under agent/protocol_runs/. +- **Tag / Ontology-term** (organism-ext, modality-ext, ELN, _next_) — A controlled-vocabulary term from a named ontology (developmental stages, phenotype/anatomy terms, gene names, imaging channels). Predictions, embryos, notes, and hypotheses reference terms instead of free strings; the stage set becomes data, not code. Stored as agent/ontologies/{name}.yaml. +- **Result / Measurement** (ELN, _next_) — A quantitative or derived finding distinct from a textual Learning: a value, table, or derived dataset (e.g. hatch rate per arm, stage-transition timing, accuracy vs ground truth) with its method, inputs, and provenance. Stored under agent/results/. The consumable, exportable output of an Experiment. +- **Citation / Reference** (ELN, _next_) — A literature reference (DOI/BibTeX-style) linking external work to internal reasoning. Notes already cite 'Moyle et al. 2021' as dead text; this makes it a resolvable object attachable to Hypothesis, Learning, Protocol, and Plan item. Backed by the lab's references/library.bib idiom. +- **Notification / Subscription** (project-mgmt, agentic, _next_) — A subscription rule (which events, which channel: browser/push/email) plus the delivery records it produces. The out-of-app counterpart to the in-app Ask/context surfaces. Backs IDEA-21. Stored under agent/notifications/. +- **Assignment / Task** (project-mgmt, _next_) — A human-ownership overlay on work: assignee (User), due date, and status, attached to a Plan item or Protocol-run. The lightest of the proposals — arguably an attribute set on Plan item rather than a standalone entity, but it's the piece that turns a plan into a shared to-do list. +- **Booking / Instrument reservation** (project-mgmt, _later_) — A calendar slot reserving the physical rig for a User/Team over a window, distinct from Planned session (scientific intent). A Planned session consumes a Booking when it runs. Prevents two people scheduling the scope at once. Stored under agent/bookings/. +- **Team / Lab** (project-mgmt, _later_) — An organizational grouping of Users and instruments that scopes ownership, sharing, and defaults (a PI's lab). Extends the existing Mesh/peer notion from 'other machines' to 'other people in one org'. Stored under agent/teams/. +- **Permission / Share grant** (project-mgmt, _later_) — An explicit access grant: (subject User/Team) × (object Campaign/Experiment/Session) × (level view/control/edit). campaigns.py already implements share/unshare/join/claim server-side; this models what that backend manipulates. Stored under agent/permissions/. +- **Sign-off / Review** (ELN, agentic, _later_) — A witnessing/approval record: a User approves (or rejects, with comment) a Result, Notebook finding, plan, or an agent-authored action. Doubles as the human-approves-agent gate for autonomous decisions. Stored as review records under agent/reviews/. +- **Provenance / Audit-record** (ELN, _later_) — An immutable, cross-entity change log: who mutated which entity, when, from/to values. Distinct from Event/log (operational session timeline) — this spans campaigns, notes, ground truth, protocols. Append-only under agent/audit/. +- **Attachment / File** (ELN, _later_) — A generic external file bound to any entity: a protocol PDF, a genotyping gel image, an external analysis CSV, a paper figure. Distinct from Volume/Image (instrument-acquired payload). Stored under agent/attachments/ or per-owner with a manifest. +- **Comment / Annotation-thread** (ELN, project-mgmt, _later_) — A generic threaded discussion attachable to ANY entity (an embryo, a volume, a plan item, a result) — human and agent turns, resolvable. Generalizes the Notebook note's thread mechanism, which is scoped to notes only. +- **Figure** (ELN, _later_) — A composed, publication-oriented visualization assembling volumes/projections/results with a caption and layout — the artifact that leaves the app for a paper. Distinct from Projection (auto thumbnail) and Result (the number). Stored under agent/figures/. +- **Consumable / Inventory** (project-mgmt, ELN, _later_) — Physical consumables stock (slides, agar/media plates, worm plates, coverslips) with counts, lot, and expiry — the non-strain materials a Protocol consumes. Strain-stock is the strain-specific case of this. Stored under agent/inventory/. + +### Extensibility abstraction entities + +- **Organism/Species-profile** (organism-ext, agentic, ELN, _now_) — A configurable profile (agent/organisms/{id}.yaml) holding species + strain conventions, specimen geometry/size priors, viability window, mounting/culture defaults, the detector to use for finding specimens, few-shot perception examples, and a pointer to its DevelopmentalModel. The keystone that makes 'the subject is a worm embryo' selectable data instead of code. +- **DevelopmentalModel/Staging-vocabulary** (organism-ext, agentic, ELN, _now_) — An ordered, named stage set with a transition graph, expected durations at a reference temperature, and terminal events (hatching), stored as data per organism. +- **Modality/Instrument-profile** (modality-ext, agentic, _now_) — A profile naming the imaging modality (dual-view light-sheet/diSPIM, confocal, widefield, brightfield) with its capabilities, coordinate conventions, objective set, and the device roster it requires. +- **Actuator/Environmental-control** (modality-ext, tactic-ext, _now_) — A generic typed environmental/perturbation actuator (temperature, CO2, O2, humidity, illumination, drug/microfluidic dose) with setpoint, unit, ramp profile, a live sample stream, and safety bounds. Generalizes Setpoint(temperature) + Temperature sample/graph. +- **AcquisitionRecipe** (modality-ext, tactic-ext, ELN, _now_) — A named, reusable acquisition parameter set — channels, z-range/step, exposure, laser/LED powers, views, cadence, stop conditions — decoupled from any one tactic or plan item. Generalizes ImagingSpec. +- **Specimen** (organism-ext, _next_) — The generalization of Embryo: a tracked subject with position, calibration, uid, an organism_profile ref, and a specimen_type (embryo/cell/organoid/tissue/whole-animal). Embryo becomes specimen_type='embryo'. +- **Device/Capability-driver** (modality-ext, agentic, _next_) — A capability descriptor per device (moves-XY, scans-volume, sets-temperature, emits-light@λ, captures-2D) decoupled from the concrete ophyd device name. +- **Channel/Fluorophore** (modality-ext, ELN, _next_) — A channel definition — fluorophore/label, excitation wavelength, emission filter, power, exposure — making multichannel acquisition data. Net-new; today implicit in the single transmission + one lightsheet path. +- **Phenotype/Feature-schema** (organism-ext, agentic, ELN, _next_) — A configurable schema of what perception should measure/label for a given organism+assay — stage, morphology, fluorescence intensity, count, motility, custom phenotype — beyond the single hardcoded 'stage' output. +- **TacticType-registry** (tactic-ext, agentic, _next_) — A registry of tactic archetypes (monitor, burst, perturbation, recovery, calibration…) with declared required-capabilities, parameters, and applicable roles — the tactic kinds as data rather than code. +- **CoordinateSpace/Calibration-profile** (modality-ext, organism-ext, _next_) — A named coordinate/calibration profile — pixel size, stage↔camera transform, view geometry, units — tied to a modality + objective, reusable across sessions. +- **Assay/Experiment-type** (project-mgmt, ELN, tactic-ext, _later_) — A typed experiment template (developmental time-lapse, dose-response, perturbation-recovery, phenotypic screen) bundling a default AcquisitionRecipe + staging model + phenotype schema + success criteria + expected TacticTypes. + +### Novel combinations + +- **Supervised adaptation loop (correction → model)** (data-loop, _next_) — Serves agentic-autonomy + data-loop rigor. Ground truth exists in FileStore (set/get_ground_truth) but there is no Model-version entity to be the CONSUMER — corrections dead-end. Introduce a versioned, deployable Model artifact so human/agent stage corrections + save_data_assessment become training signal that produces a new Model version, auto-scored on held-out ground truth via the existing run_id-tagged predictions, and promoted into ml_pipeline.best_run_id. Turns the perception stack from static into self-improving; this is the retraining/adaptation loop the whole ml/* store domain implies but never closes. +- **Organism-agnostic perception (schema-driven staging)** (abstraction/extensibility, _later_) — Serves organism-extensibility — the single biggest structural unlock. Today the stage vocabulary is hardcoded C. elegans embryo stages; Prediction/Stage and the perceiver's cues are implicitly bound to one organism. A first-class Phenotype/stage schema (ordered stages, allowed transitions, expected durations, morphological markers) that the perceiver reads at runtime decouples perception output from the organism, so the same pipeline stages zebrafish or Drosophila by swapping the active schema. Nothing else on the extend-to-other-organisms axis works until the stage set stops being a constant. +- **Transferable priors across organisms** (transfer/priors, _next_) — Serves organism-extensibility + transfer. Learnings today are free insights with a strain tag; scope them to an Organism profile (strain lineage, staging schema, baseline timings at temperature, markers). Priors like 'comma stage ~X min at 20 C' become queryable, comparable, and inheritable: when a new organism is registered the agent seeds it with related-organism priors flagged as unvalidated, and Expectations are generated from the profile's baseline rather than from scratch. Converts accumulated memory from per-run notes into a portable knowledge base. +- **Modality-specific protocol synthesis** (cross-modal, _later_) — Serves modality-extensibility + tactic-operation-extensibility. A Tactic (monitor burst, recovery monitor) is currently coupled to diSPIM acquisition implicitly. Introduce a Modality profile (channels, views, capabilities, constraints) and an Acquisition recipe (laser powers, z-range/step, exposure, dual-view geometry) so a Tactic becomes modality-parameterized: 'monitor at 2-min cadence' resolves to a concrete recipe per scope — diSPIM dual-view vs confocal z-stack vs widefield. The tactic library ports across microscopes instead of being one rig's macro set. +- **Portable experiment definition (run-anywhere bundle)** (reproducibility, _later_) — Serves reproducibility + modality-extensibility. Operation Plan is the live per-session tactic spine bound to one rig. Bundle it with its recipes, schema, roles and target organism into a self-contained, exportable/importable experiment definition that another gently instance (or another modality) can instantiate. Combined with the mesh backend this makes an experiment a shareable object — the difference between 'I ran this here' and 'run my experiment on your scope', the portability the extend-to-other-modalities goal requires. +- **Reproducible, citable ELN record** (reproducibility, _next_) — Serves ELN-rigor — the integrated-notebook half of the mission. A Notebook note/Learning today asserts a finding without an immutable chain to what produced it. A Provenance record binds each finding to its exact inputs (volumes, model version, ground truth, tactic, session, calibration) and packages the finding + its evidence + literature References into a signed, exportable, citable Result (RO-Crate style). This is what turns the human-browsable file store into an actual electronic lab notebook a paper can cite and a reviewer can reproduce. +- **Multi-user projects with attributed authorship** (multi-user/collab, _next_) — Serves user-session-project management. Auth/control is a single-driver lock with no User or Team entity, so campaigns have no owner and notes/corrections have no attributable author beyond 'human'. Introduce User + Team + project-level permissions so a Campaign is owned by a team, ground-truth corrections and notes carry an identified author, and control hand-off + the existing mesh claim model become real collaboration rather than one anonymous lock. Prerequisite for the platform being used by a lab instead of one operator. +- **Closed-loop reactive control (watchpoint → actuator)** (autonomy, _later_) — Serves agentic-autonomy. A Watchpoint fires and just waits for a manual Resolve; Setpoint/Device state are the only outputs and are not modeled as bindable control channels. Introduce an Actuator binding abstraction (temperature setpoint, cadence, laser, stage) so a watchpoint condition can bind to an actuator or tactic under a proportionate guard: 'embryo approaching hatch → switch to recovery-monitor at 2-min' executes autonomously (or as an approve-only Ask). This is genuine reactive smart-microscopy — perception-driven hardware response — versus attention that merely notifies. +- **Hypothesis → expectation → outcome falsification engine** (data-loop, _next_) — Serves ELN-rigor + agentic-autonomy. Campaign carries only a free-text target; there is no falsifiable Hypothesis entity. Add one that generates testable Expectations, whose outcomes auto-score against Predictions/Ground truth when expected_time passes, confirming or refuting the hypothesis and emitting a calibrated Learning with evidence. Makes the scientific method a first-class computational object — the agent proposes, the instrument tests, the record self-scores — instead of expectations being hand-confirmed dead ends. +- **Automated perturbation-response measurement** (data-loop, _next_) — Serves ELN-rigor + tactic-operation-extensibility. Temperature perturbation and stage prediction are both logged per session but never joined into a measured readout (IDEA-15 only overlays them visually). Introduce an Assay/Endpoint metric entity (e.g. 'developmental rate vs temperature', 'time-to-hatch', 'lethality fraction') so the instrument computes a dose-response curve across embryos automatically and persists it as a Result. This is the actual science the thermal rig exists to produce, elevated from two charts on separate tabs to a defined, reusable measurement. +- **Phototoxicity-aware acquisition (dose budget)** (autonomy, _next_) — Serves agentic-autonomy + modality-extensibility. Live imaging validity hinges on cumulative light dose, which the model does not track at all. A per-Embryo Dose ledger (accumulated from each recipe's laser power x exposure x frames) lets Tactics respect a dose ceiling, lets the agent trade temporal resolution against photodamage explicitly, and lets a Watchpoint fire on dose approaching the limit. Makes autonomous cadence decisions defensible rather than blind — a hard requirement before trusting the agent to over-sample. +- **Uncertainty-driven active-learning acquisition** (data-loop, _next_) — Serves agentic-autonomy + data-loop. Predictions carry confidence and is_transitional flags that today only render as sparkline color. Wire model uncertainty to acquisition: low-confidence/transitional calls auto-raise an oversampling tactic AND queue the timepoint into a human confirm stack whose answers write ground truth. The instrument then spends photons and human attention where the model is least sure — active learning closing on the same ground-truth store as the retraining loop. Unifies IDEA-25/IDEA-36 into a capability rather than two UI queues. +- **Full physical provenance lineage (bench → scope)** (reproducibility, _next_) — Serves ELN-rigor + organism-extensibility. The model starts at the imaged Embryo; everything upstream (strain source, mounting, agar pad, prep time, pre-imaging temperature history) is untracked, so cross-experiment comparison silently loses confounds. A Specimen/Sample-prep record links each Embryo back to its physical origin and prep conditions, giving the ELN a complete wet-lab-to-image chain and making the Organism profile's baseline timings attributable to real prep variance instead of noise. +- **Metrologically valid, comparable data (calibration in force)** (reproducibility, _next_) — Serves ELN-rigor + modality-extensibility. Calibration appears only as a gallery of images; there is no Calibration entity with a validity window. Stamp each Volume with the calibration in force (pixel size, PSF, stage/drift model), gate acquisition when calibration is stale, and make measurements physically comparable across sessions and rigs. Without this, quantitative endpoints and cross-modal/cross-instance results are not defensibly comparable — a precondition for the portable-experiment and perturbation-response capabilities above. +- **Perception-capability registry (know-what-we-can-stage)** (transfer/priors, _later_) — Serves organism- + modality-extensibility + transfer. A registry mapping (organism, modality, schema) → best available Model version makes the platform self-aware about what perception it can actually perform and where it must fall back to human labeling or transfer/fine-tune. Adding a new organism or scope becomes a registration + gap-analysis step ('no model for zebrafish on confocal → collect ground truth or transfer from the diSPIM model') rather than a silent competence hole. The connective tissue that makes the three extensibility axes a managed capability instead of ad hoc. diff --git a/docs/product-ideation/FRAMEWORK.md b/docs/product-ideation/FRAMEWORK.md new file mode 100644 index 00000000..b2956b7d --- /dev/null +++ b/docs/product-ideation/FRAMEWORK.md @@ -0,0 +1,51 @@ +# Gently — Product-Ideation Framework + +How we surface product ideas from the UX audit. Ideas come in KINDS; each lens is a +question swept across the entity inventory + the two matrices + the crawler graph + +the per-page screenshots. Built by a fan-out (7 generators + 4 method-improvers); the +improvers grew this from 7 core lenses to the set below. + +## Lenses + +### Core lenses + +- **missing-affordance** — For a verb the user obviously wants here (create/edit/delete/save), is there a control on this surface — or is it reachable only via the agent or an incidental path? +- **cross-feature-link** — Two entities co-exist and reference each other in the data model, but is there a traversable UI path between their surfaces? +- **hidden-state / make-visible** — Is there durable state (lock, control-holder, role, health, liveness) that the backend already knows but no surface shows? +- **step-reduction** — Does a common action cost more clicks/tab-hops than the data on hand requires, and can it collapse to one? +- **flywheel (producer→consumer)** — Is an artifact produced (ground truth, ML assessment, embryo-understanding, resolved expectation) that no consumer ever reads back — a dead-end producer? +- **agentic (augmented-LLM in the loop)** — Is there a tedious judgement/triage/authoring step where the agent should narrow and the human should decide? +- **consistency / cross-surface parity** — Does the same entity get an affordance on one surface and an inert copy on another, so the two drift? + +### Added by the method-improvers + +- **error-recovery / failure-path** — When this action fails or its dependency is offline/empty, does the surface name what broke and offer a way forward (retry, fallback, where-to-look)? +- **navigation / findability** — Can you jump from any entity MENTION to that entity in one click, and find a thing by name without knowing its tab (deep-link + Cmd-K)? +- **collaboration / presence / control-ownership** — If two humans, or a human and the agent, share this session, do they see each other, know who holds control, and can they hand it off? +- **remove-don't-add (subtraction)** — What here is noise, redundant, stale, or actively misleading and should be deleted or merged rather than augmented? +- **trust / provenance / feedback-integrity** — Can the user see WHY the agent/model did what it did, reach what it was based on, judge it, and does that judgment actually go somewhere? +- **safety / reversibility** — For irreversible or specimen-affecting actions (laser, temperature, delete, stop), is there a proportionate guard, preview, or undo? +- **unattended / temporal** — For operations that outlive attention (a ~14h timelapse), is there progress/ETA and a way to be told out-of-app when a human is needed? +- **provenance / interop / export (app boundary)** — Can data, results, and their provenance leave the app for analysis, citation, or reproduction — and come back in? +- **onboarding & expert-mode** — Does a first-timer get oriented from a cold/empty state, and does a returning power-user get accelerants (memory, defaults, bulk, shortcuts)? +- **loop-closure (JTBD spine)** — Does this close a loop on Plan→Operate→Acquire→Perceive→Learn→Decide, or force an app-exit / re-keying / agent round-trip mid-loop? (ranking lens) +- **capability-orphan (store-verb diff)** — For every mutating store method, is there a route AND a UI control that invokes it? Orphaned verbs are missing affordances, derived not guessed. (generator) +- **dangling-edge (data-model FK diff)** — Which entity already stores a foreign key to another entity that the UI renders as dead text? Highest leverage/effort links live here. (generator) +- **agent-arbitrage (filter)** — This is already doable via the agent — is the manual affordance materially better (faster/safer/in-context/discoverable/works-when-agent-busy), or a redundant reimplementation? +- **noise-collapse (meta-filter)** — Is this the Nth instance of a surface-pattern template — and what single SYSTEMIC idea does the whole cluster collapse into? +- **frequency × friction** — For the handful of things this persona does 10+ times a day, how many hops does each cost and what collapses it to one tap? (ranking weight) + +## Method notes + +METHOD IMPROVEMENTS. (1) Shift from a 7-kind additive taxonomy to a LENS LIBRARY of 22 questions run against every (surface × entity × graph-edge) triple, split by role: GENERATORS derive candidates by construction rather than inspiration — capability-orphan (every mutating store method → route → UI control; orphaned verbs = missing affordances, e.g. set_ground_truth, create_campaign, note-create) and dangling-edge (every foreign key in the data model rendered as dead text = a cross-feature link, e.g. Note.embryos/strains/basis, item.session_ids/depends_on, claimed_by). These read the CODE/data model, not the rendered screen, which is exactly where the deepest, lowest-effort ideas hide and where a screenshot-only audit is structurally blind. (2) RANKING lenses: loop-closure on the Plan→Operate→Acquire→Perceive→Learn→Decide spine, and frequency×friction (weight the mark→run loop run ~20×/day over once-per-project config). (3) FILTERS applied before scoring: agent-arbitrage (a manual affordance must be materially better than the agent path — faster/safer/in-context/discoverable/works-when-agent-busy — or it's dropped; 'the agent can do it in chat' is a RED FLAG masking a gap, not coverage) and noise-collapse (an Nth template instance collapses to one systemic idea — ~15 'nicer empty state' items became one 'empty states deep-link to their seeding action'). + +QUALITY RUBRIC. Score = (impact × reach × depth × trust) / effort, gated by code evidence, with a structural bias (+ for missing-affordance and cross-feature-link) applied per the brief. DEPTH axis is decisive: DEEP if it creates/persists a new entity or edge (ground truth, dependency, note↔plan link); COSMETIC if it only moves pixels — the two must never rank equal. HARD-REJECT before scoring: template spam, cosmetic-only polish, audit-echo (restating a US-## gap adds nothing over an idea with a concrete mechanism), and agent-redundant LLM bolt-ons (the rejected 'AI summary on Logs' / 'refresh button everywhere'). Effort-blind ranking is banned — a one-line render of an existing foreign key must outrank a huge-payoff/huge-cost item, which is why IDEA-04/IDEA-11 rank above IDEA-20/IDEA-25/IDEA-39. + +WHAT TO KEEP. The three graph artifacts as living inputs: G_nav (crawler graph.json — proves the app is a star of sibling tabs with zero entity-to-entity edges), G_data (entity/FK graph from the storage model), G_verb (store-method→route→handler capability graph). Emit ideas as mechanical diffs — cross-feature-link = a G_data edge whose endpoints both have surfaces but no G_nav path; missing-affordance = a G_verb verb that dead-ends before the UI; orphan-surface = a G_nav node only ever seen empty whose seeding action is unreachable. Dedup by systemic collapse (46 raw candidates → 45 ranked, with the largest merges being ground-truth ×11, add-note ×6, new-campaign/plan ×6, chip-deep-link ×5). Keep the added lenses that catch the core-lens blind spots the method-gap list named: failure branch, subtraction, second actor/control-ownership, temporal/unattended, app-boundary export, physical risk, and fine-grained navigability. + +## The two matrices (the mechanical core) + +- **Entity × Operation** (`ENTITIES.md`): view/create/edit/delete/link/export per entity — every empty cell is a candidate missing-affordance. Sharpened by *capability-orphan*: diff the store's mutating methods against routes+controls. +- **Entity × Entity linkage** (`ENTITIES.md`): related-but-unlinked pairs are cross-feature ideas. Sharpened by *dangling-edge*: a stored foreign key rendered as dead text is the highest-leverage link. + +_Ideas land in `BACKLOG.md` (+ `backlog.json`, queryable/appendable). Re-run the engine to refresh; append by hand as ideas arrive._ \ No newline at end of file diff --git a/docs/product-ideation/backlog.json b/docs/product-ideation/backlog.json new file mode 100644 index 00000000..bab22342 --- /dev/null +++ b/docs/product-ideation/backlog.json @@ -0,0 +1,1136 @@ +{ + "framework": [ + { + "lens": "missing-affordance", + "question": "For a verb the user obviously wants here (create/edit/delete/save), is there a control on this surface \u2014 or is it reachable only via the agent or an incidental path?", + "origin": "core" + }, + { + "lens": "cross-feature-link", + "question": "Two entities co-exist and reference each other in the data model, but is there a traversable UI path between their surfaces?", + "origin": "core" + }, + { + "lens": "hidden-state / make-visible", + "question": "Is there durable state (lock, control-holder, role, health, liveness) that the backend already knows but no surface shows?", + "origin": "core" + }, + { + "lens": "step-reduction", + "question": "Does a common action cost more clicks/tab-hops than the data on hand requires, and can it collapse to one?", + "origin": "core" + }, + { + "lens": "flywheel (producer\u2192consumer)", + "question": "Is an artifact produced (ground truth, ML assessment, embryo-understanding, resolved expectation) that no consumer ever reads back \u2014 a dead-end producer?", + "origin": "core" + }, + { + "lens": "agentic (augmented-LLM in the loop)", + "question": "Is there a tedious judgement/triage/authoring step where the agent should narrow and the human should decide?", + "origin": "core" + }, + { + "lens": "consistency / cross-surface parity", + "question": "Does the same entity get an affordance on one surface and an inert copy on another, so the two drift?", + "origin": "core" + }, + { + "lens": "error-recovery / failure-path", + "question": "When this action fails or its dependency is offline/empty, does the surface name what broke and offer a way forward (retry, fallback, where-to-look)?", + "origin": "added" + }, + { + "lens": "navigation / findability", + "question": "Can you jump from any entity MENTION to that entity in one click, and find a thing by name without knowing its tab (deep-link + Cmd-K)?", + "origin": "added" + }, + { + "lens": "collaboration / presence / control-ownership", + "question": "If two humans, or a human and the agent, share this session, do they see each other, know who holds control, and can they hand it off?", + "origin": "added" + }, + { + "lens": "remove-don't-add (subtraction)", + "question": "What here is noise, redundant, stale, or actively misleading and should be deleted or merged rather than augmented?", + "origin": "added" + }, + { + "lens": "trust / provenance / feedback-integrity", + "question": "Can the user see WHY the agent/model did what it did, reach what it was based on, judge it, and does that judgment actually go somewhere?", + "origin": "added" + }, + { + "lens": "safety / reversibility", + "question": "For irreversible or specimen-affecting actions (laser, temperature, delete, stop), is there a proportionate guard, preview, or undo?", + "origin": "added" + }, + { + "lens": "unattended / temporal", + "question": "For operations that outlive attention (a ~14h timelapse), is there progress/ETA and a way to be told out-of-app when a human is needed?", + "origin": "added" + }, + { + "lens": "provenance / interop / export (app boundary)", + "question": "Can data, results, and their provenance leave the app for analysis, citation, or reproduction \u2014 and come back in?", + "origin": "added" + }, + { + "lens": "onboarding & expert-mode", + "question": "Does a first-timer get oriented from a cold/empty state, and does a returning power-user get accelerants (memory, defaults, bulk, shortcuts)?", + "origin": "added" + }, + { + "lens": "loop-closure (JTBD spine)", + "question": "Does this close a loop on Plan\u2192Operate\u2192Acquire\u2192Perceive\u2192Learn\u2192Decide, or force an app-exit / re-keying / agent round-trip mid-loop? (ranking lens)", + "origin": "added" + }, + { + "lens": "capability-orphan (store-verb diff)", + "question": "For every mutating store method, is there a route AND a UI control that invokes it? Orphaned verbs are missing affordances, derived not guessed. (generator)", + "origin": "added" + }, + { + "lens": "dangling-edge (data-model FK diff)", + "question": "Which entity already stores a foreign key to another entity that the UI renders as dead text? Highest leverage/effort links live here. (generator)", + "origin": "added" + }, + { + "lens": "agent-arbitrage (filter)", + "question": "This is already doable via the agent \u2014 is the manual affordance materially better (faster/safer/in-context/discoverable/works-when-agent-busy), or a redundant reimplementation?", + "origin": "added" + }, + { + "lens": "noise-collapse (meta-filter)", + "question": "Is this the Nth instance of a surface-pattern template \u2014 and what single SYSTEMIC idea does the whole cluster collapse into?", + "origin": "added" + }, + { + "lens": "frequency \u00d7 friction", + "question": "For the handful of things this persona does 10+ times a day, how many hops does each cost and what collapses it to one tap? (ranking weight)", + "origin": "added" + } + ], + "backlog": [ + { + "id": "IDEA-01", + "kind": "missing-affordance", + "title": "Persist ground-truth stage corrections (replace localStorage Agree/Disagree)", + "surfaces": [ + "Embryos > Default (detection cards)", + "Embryos > Default ('Show VLM reasoning')", + "Embryos > Board" + ], + "entities": [ + "Ground truth", + "Prediction/Stage", + "Trace", + "Embryo" + ], + "sketch": "Replace the binary I-Agree/I-Disagree (embryos.js markAgreement \u2192 localStorage 'gently-detection-agreements', a dead end) with a stage picker that POSTs to a new /api/embryos/{id}/ground_truth backed by the already-implemented FileStore.set_ground_truth(stage,timepoint,annotator). Show a persisted 'corrected \u2192 {stage} by {user}' badge on the card and Board sparkline. Lights up three waiting consumers: accuracy, data-assessment.annotated_embryos, and perception few-shot examples.", + "impact": "high", + "effort": "med", + "why": "Flagship: DEEP (persists a new entity + closes the human-correction\u2192perception flywheel), the persistence layer already exists (set/get_ground_truth) so effort is bounded, and today feedback is silently thrown to the browser. US-31 gap; canonical missing-affordance + loop-closure." + }, + { + "id": "IDEA-02", + "kind": "missing-affordance", + "title": "Notebook add-note composer (human authoring), with 'note this' from any surface", + "surfaces": [ + "Notebook tab header", + "Embryos > Default (detection cards)", + "Gallery lightbox", + "Sessions tab" + ], + "entities": [ + "Notebook note", + "Observation", + "Question", + "Embryo", + "Session" + ], + "sketch": "A '+ Add note' composer (kind observation/finding/question, free text, auto-linked strain/embryo/session/thread chips from context, author=human) POSTing to a new /api/notebook/notes. notebook.js is read-only ('authoring arrives in a later increment'); notebook.py exposes only GET notes/threads + POST ask. Seed a lightweight 'note this' variant on the detection card / session row that pre-fills the entity links. Optional agentic 'draft-with-Gently' pass structures rough text into a linked note.", + "impact": "high", + "effort": "med", + "why": "The single most obvious verb on a notebook is absent while the store models human-authored notes; passes agent-arbitrage (a discoverable in-context capture beats dictating to chat). The note's power is its cross-links, which the composer auto-fills." + }, + { + "id": "IDEA-03", + "kind": "missing-affordance", + "title": "New campaign / New plan create controls in the Plans workspace", + "surfaces": [ + "Plans tab (campaign navigator)", + "Plans > Doc ('+ Add item')", + "Home hero" + ], + "entities": [ + "Campaign", + "Operation Plan", + "Plan item" + ], + "sketch": "A labelled '+ New campaign' in the navigator (minimal title/goal/organism form) and '+ New plan' launching the same wizard the Home hero fires (AgentChat /wizard hook), plus '+ Add item' in Doc view. Creation is reachable today only via the header-logo\u2192landing reset or agent chat (create_campaign tool); campaigns.py has no POST create though FileContextStore.create_campaign exists.", + "impact": "high", + "effort": "low", + "why": "High/low ratio and passes agent-arbitrage: a core loop (originate a research program) trapped behind an incidental logo-click is gold, and the launcher already exists \u2014 pure wiring. US-06/US-35." + }, + { + "id": "IDEA-04", + "kind": "cross-feature-link", + "title": "Make notebook note chips clickable deep-links (+ reverse 'notes about this')", + "surfaces": [ + "Notebook tab (note chips)", + "Embryos tab", + "Sessions tab", + "Gallery", + "Home 'From the notebook'" + ], + "entities": [ + "Notebook note", + "Embryo", + "Session", + "Strain", + "Volume/Image" + ], + "sketch": "Notes already carry links[]/strains/embryos/sessions/threads (rendered as inert ). Make \ud83e\uddecstrain \u2192 cross-filter Embryos/notebook to that strain, \u25ccembryo \u2192 switchTab('embryos')+select that reasoning rail, session \u2192 that session, timepoint \u2192 Gallery frame. Reciprocally add an 'N notes' link on the embryo card that filters the notebook to it (the endpoint already accepts an embryo filter). Also carry the note id when jumping Home\u2192Notebook.", + "impact": "high", + "effort": "low", + "why": "Best leverage/effort in the app: the join is a foreign key that already exists in the payload; the work is rendering a chip that navigates. Dangling-edge generator; the nav graph currently has zero entity-to-entity edges." + }, + { + "id": "IDEA-05", + "kind": "cross-feature-link", + "title": "Surface relevant learnings / watchpoints / questions inside the Operate step and tactic cards", + "surfaces": [ + "Operate > Run chooser", + "Operations > Overview (tactic cards)", + "Operate worklist" + ], + "entities": [ + "Learning", + "Watchpoint", + "Question", + "Tactic", + "Embryo" + ], + "sketch": "A compact 'what we know' rail in the Run chooser and on the active tactic card pulling learnings/observations + active watchpoints scoped to the selected embryos/strain (matched by their existing strain/embryo basis), so the operator sees 'ky123 ~80% lethal \u2192 over-sample n=12' before committing a burst interval. operate.js today has zero references to context/notebook/learnings.", + "impact": "high", + "effort": "med", + "why": "Canonical loop-closer: Operate is completely severed from the agent's memory; bringing durable insight to the moment of decision is the highest-value cross-feature link and a filtered read of stores that already exist." + }, + { + "id": "IDEA-06", + "kind": "cross-feature-link", + "title": "Bidirectional embryo/prediction \u2194 tactic link (+ rationale\u2192learning)", + "surfaces": [ + "Embryos > Board/Vitals (stage sparkline)", + "Operations > Overview (tactic cards)", + "Operate worklist" + ], + "entities": [ + "Prediction/Stage", + "Tactic", + "Role", + "Learning", + "Embryo" + ], + "sketch": "On an embryo's stage strip add a 'governed by: ' chip (role + cadence) linking into Operations at that card; make the tactic card's scope list click through to each embryo's Board/Vitals; link the card's RATIONALE row to the notebook finding justifying its cadence. experiment-overview.js already computes the embryo\u2194tactic mapping for the roster lens but never renders it as a link.", + "impact": "high", + "effort": "med", + "why": "Trust infrastructure: lets a scientist answer 'why is this embryo imaged this way / which embryos does this burst touch?' \u2014 the core of adaptive-timelapse trust. Provenance + missing-edge; mapping already computed." + }, + { + "id": "IDEA-07", + "kind": "cross-feature-link", + "title": "Link notebook note \u2194 plan item ('informs' / 'notes that shaped this')", + "surfaces": [ + "Notebook tab (note cards)", + "Plans > Doc (item rows)", + "plan-item inspector" + ], + "entities": [ + "Notebook note", + "Plan item", + "Campaign", + "Learning" + ], + "sketch": "On a note whose basis references a plan item, an 'Informs \u2192 open plan item' affordance; reciprocally a 'Notes that shaped this' strip on the plan-item inspector sourced from /api/notebook/notes?thread=/embryo=. Turns the reasoning ('over-sample the Robo-loss arm to n=12') into a link to the plan item it should drive.", + "impact": "high", + "effort": "med", + "why": "Closes the 'why is this plan item here' gap (US-33/US-36); the note model already stores basis links, so it is render + one reverse query. Pairs with IDEA-02." + }, + { + "id": "IDEA-08", + "kind": "provenance / interop / export", + "title": "Export/download across the data-heavy surfaces (+ reproducible per-embryo bundle)", + "surfaces": [ + "Gallery + lightbox", + "Embryos tab", + "Logs tab", + "Devices temp graph", + "Agent chat panel" + ], + "entities": [ + "Volume/Image", + "Projection", + "Prediction/Stage", + "Trace", + "Event/log", + "Temperature sample/graph" + ], + "sketch": "Add download where the payload lives: raw t{NNNN}.tif / projection jpg in the lightbox+Gallery, 'Export predictions (CSV) / trace (JSON)' on Embryos, 'Export timeline' on Logs, temp-trace CSV on Devices, transcript on the chat panel \u2014 reusing the Blob-download idiom settings.js already uses. Top tier: a per-embryo/session 'Export bundle' zipping volumes+projections+predictions.jsonl+traces+ground_truth.yaml+manifest+stage-over-time CSV (file_store already groups these per embryo).", + "impact": "high", + "effort": "med", + "why": "App-boundary blind spot: results cannot leave the app (only prefs JSON + plan markdown today). Reproducibility/sharing is a core scientific need; files already exist on disk. Noise-collapse folds ~6 per-surface export requests into one systemic idea. US-32." + }, + { + "id": "IDEA-09", + "kind": "collaboration / presence / control-ownership", + "title": "Header control/presence chip (who's driving + sign-in + request/hand-off)", + "surfaces": [ + "header presence-container / session badge", + "Devices", + "Operate", + "/login" + ], + "entities": [ + "Auth / control", + "Agent chat / turn", + "Device state", + "Mesh / peer instance" + ], + "sketch": "The /ws/agent server already broadcasts control_status {holder,holder_label,you_have_control} to every client but only agent-chat.js consumes it (as a banner inside a closed panel). Add a persistent header chip: green 'You're driving' vs amber 'Watching \u2014 {holder_label} is driving \u00b7 Take control', a 'driving' ring on the holder's presence avatar, and a discoverable 'Sign in to control' when logged out. On AGENT_CONTROL, disable/annotate hardware buttons instead of letting them 403 into the misleading 'Log in' toast.", + "impact": "high", + "effort": "med", + "why": "The single-driver lock is real and multi-user and the holder identity is already pushed everywhere, yet the only ambient signal is a reactive 403 that misattributes another operator's lock to 'not logged in'. Collaboration + hidden-state; merges US-43/US-44." + }, + { + "id": "IDEA-10", + "kind": "error-recovery / failure-path", + "title": "Device-offline banner + toast on failed device actions + disable controls", + "surfaces": [ + "Devices tab", + "Operate rail", + "acquire buttons (Snap Volume / Burst / stage-move)" + ], + "entities": [ + "Device state", + "Volume/Image", + "Session" + ], + "sketch": "When /api/devices/* returns 502/503, show a persistent amber strip ('Microscope offline \u2014 controls disabled; last seen 3m ago'), subscribe acquire/stage/run controls to the existing microscopeConnected status-store value to disable them, and route failures through the toast control-auth.js already uses on 403 (instead of console.error only). Optional agentic 'explain-and-recover': read the device_layer log + live status, name the likely cause, and offer one recovery action wired to the relevant control.", + "impact": "high", + "effort": "med", + "why": "The richest untapped vein: US-09/10/11/12/14/18/26 all 502 silently (toast_visible=False). The online/offline signal already exists in the status store but is never wired to the action surfaces. Happy-path bias made every core lens blind to it." + }, + { + "id": "IDEA-11", + "kind": "cross-feature-link", + "title": "Deep-link context-surface rows to their embryo + badge watched embryos", + "surfaces": [ + "context-surface 'Watching'/'Expectations'/'Open questions'", + "Embryos > Board/Vitals", + "Operate worklist" + ], + "entities": [ + "Watchpoint", + "Expectation", + "Question", + "Embryo" + ], + "sketch": "context-surface.js wires a whole-row click that currently points everything at switchTab('notebook'). When the item carries an embryo ref, deep-link to that embryo instead (switchTab('embryos')+select), add an embryo chip to the row, and put an 'eye: watched' / 'expected: comma by Tue' marker on that embryo's Board/worklist row.", + "impact": "med", + "effort": "low", + "why": "Near-free (redirect an existing handler) and fixes a wrong-destination dead-end: watchpoints exist to pull attention to a specific embryo but send you to a generic notebook. Great ratio, structural." + }, + { + "id": "IDEA-12", + "kind": "safety / reversibility", + "title": "Proportionate guards on laser-on and temperature setpoint changes", + "surfaces": [ + "Devices Manual laser toggle", + "Devices header temp setpoint", + "Operations tactic (temp-change)" + ], + "entities": [ + "Setpoint (temperature)", + "Device state", + "Embryo" + ], + "sketch": "Laser-on and a setpoint change (e.g. +4\u00b0C step to 32\u00b0C on live embryos) fire with no confirmation, while cheap UI-only actions ('Remove embryo', 'Stop run') DO confirm \u2014 guards are inverted. Add a hold-to-arm on the laser and a preview/confirm on the setpoint ('raises stage temp to 32.0\u00b0C over ~N min \u00b7 affects 3 live embryos \u00b7 Confirm').", + "impact": "high", + "effort": "low", + "why": "Microscopy acts on live specimens; the physically-irreversible actions are the only unguarded ones and no undo exists anywhere. High/low ratio; a whole risk axis the core lenses can't see. US-16/US-26." + }, + { + "id": "IDEA-13", + "kind": "step-reduction", + "title": "Fix Home/badge misdirected links (recent image, recent plan, session badge)", + "surfaces": [ + "Home > Recent Images/Plans/Sessions", + "header session-id badge" + ], + "entities": [ + "Volume/Image", + "Campaign", + "Session", + "Embryo" + ], + "sketch": "Home recent-image tiles render bare with no handler \u2192 wire to the existing lightbox / Gallery pre-filtered to that embryo+timepoint (the tile knows the coords). Recent-plan rows carry only data-go-tab='plans' with no id \u2192 pass the campaign id and have the navigator focus it. The session-id badge navigates to the landing \u2192 point it at the Sessions tab focused on the current session; add an 'Open' on recent-session rows.", + "impact": "med", + "effort": "low", + "why": "Three misdirected-link frictions with all data already in hand; the click currently loses the entity identity or dumps you at the start screen. Cheap navigation-fidelity cluster." + }, + { + "id": "IDEA-14", + "kind": "trust / provenance / feedback-integrity", + "title": "Overlay the agent's expectation on the stage strip + auto-score outcomes", + "surfaces": [ + "Embryos > Vitals (stage strip)", + "context-surface 'Expectations'", + "Notebook > Findings" + ], + "entities": [ + "Expectation", + "Prediction/Stage", + "Learning", + "Embryo" + ], + "sketch": "Draw each embryo's active expectation ('will reach comma by T, uncertainty X') as a target marker/band on its Vitals stage strip so the forecast becomes falsifiable in the same view the human reads for stage-over-time. When expected_time passes, the agent auto-evaluates the belief against actual predictions and pre-fills the resolution (hit/miss + observed stage as evidence); aggregate into a small 'forecast accuracy N/M' calibration readout, and log a learning/observation on a miss.", + "impact": "high", + "effort": "med", + "why": "Expectations are the agent's forward beliefs but 'confirmed' is currently a hand-click with no evidence \u2014 a dead-end resolution. Links Expectation\u2192Prediction (data already stored) into a trust/calibration loop. US-29." + }, + { + "id": "IDEA-15", + "kind": "flywheel (producer\u2192consumer)", + "title": "Overlay setpoint-change markers on the stage strip (and stage transitions on the temp graph)", + "surfaces": [ + "Embryos > Vitals (stage strip)", + "Devices temperature graph", + "Operations tactic (temp-change)" + ], + "entities": [ + "Temperature sample/graph", + "Prediction/Stage", + "Setpoint (temperature)", + "Tactic" + ], + "sketch": "temperature.jsonl and predictions.jsonl are both produced per session but live on separate tabs. On the Vitals stage strip, draw vertical markers where the setpoint changed (from the temp log + temp-change tactic); on the Devices temp graph, drop stage-transition ticks from predictions. They share the session timeline, so the join is a timestamp align.", + "impact": "high", + "effort": "med", + "why": "Whether a thermal perturbation shifts development is the microscope's whole scientific point, and both halves already exist and are already charted \u2014 just never on the same axis. Two live producers never joined; the cause\u2192effect readout the instrument is built to show is invisible." + }, + { + "id": "IDEA-16", + "kind": "hidden-state / make-visible", + "title": "Surface the session lock (live-here / live-on-peer / stale) + guard Resume", + "surfaces": [ + "Sessions tab", + "Home > Recent Sessions", + "header session badge", + "landing > Resume" + ], + "entities": [ + "Session", + "Mesh / peer instance" + ], + "sketch": "FileStore writes session.lock={pid,hostname} while active and unlinks on release, but no route exposes it and no JS reads it. Add lock info to /api/sessions and render a per-session chip: 'Live on this machine' / 'Live on {hostname} (peer)' / 'Stale lock \u2014 process gone' (test pid liveness + hostname match). Gate Resume with a warning when actively locked elsewhere so two instances don't drive one session.", + "impact": "high", + "effort": "med", + "why": "The lock already holds exactly the identity needed (pid+hostname) but is invisible; a crashed run leaves a stale lock with no signal, and with mesh a peer can legitimately hold it. Hidden-state + safety against collisions." + }, + { + "id": "IDEA-17", + "kind": "consistency / cross-surface parity", + "title": "Show + author session \u2194 campaign/plan-item membership from the Session side", + "surfaces": [ + "Sessions (session-review header)", + "Home > Recent Sessions", + "header session badge" + ], + "entities": [ + "Session", + "Plan item", + "Campaign" + ], + "sketch": "The session\u2194plan-item edge is authored only from the plan-item end (campaigns.js '+ link session'). Every Session surface shows no campaign membership and offers no link. Show the linked campaign/item as a chip on the session-review header and Sessions list, and add 'Link to plan item' there reusing the same picker inverted.", + "impact": "high", + "effort": "med", + "why": "One relationship, affordance on only one end: a session opened in review can't even tell you which experiment it belongs to. Parity defect; gives Sessions provenance (US-42)." + }, + { + "id": "IDEA-18", + "kind": "missing-affordance", + "title": "Human-raise a question + resolve from the Notebook + save an Ask answer as a note", + "surfaces": [ + "Notebook > Questions filter", + "Notebook Ask box", + "context-surface 'Open questions'" + ], + "entities": [ + "Question", + "Learning", + "Notebook note", + "Campaign" + ], + "sketch": "Three parts on the same entity: (a) an 'Open a question' control so a human can post into the shared uncertainty queue (the Question entity explicitly allows human-posed; context.py exposes only resolve today); (b) mirror the context-surface's inline Answer/Resolve control onto the inert notebook question card; (c) beneath a notebook Ask answer, 'Save as finding' (persists the grounded answer citing its returned note_ids as basis) and 'Open as question'.", + "impact": "high", + "effort": "low", + "why": "Consistency + capability-orphan: the resolve endpoint already exists and the Ask already returns basis note_ids \u2014 the grounding is right there and discarded. Turns one-off Q&A into accumulating memory at near-zero cost. US-37." + }, + { + "id": "IDEA-19", + "kind": "hidden-state / make-visible", + "title": "Show and toggle an embryo's role (subject/reference) on the Embryos tab", + "surfaces": [ + "Embryos > Default/Board/Vitals", + "Operate worklist" + ], + "entities": [ + "Role", + "Embryo", + "Tactic", + "Cadence" + ], + "sketch": "Roles (subject='test' / reference='calibration') are picked and shown only in the fleeting Operate run-chooser role chips, yet they govern cadence and detector. Add a role chip (with its cadence) to each embryo row/card on the Embryos tab and allow the same toggle there, so it's clear why one embryo is imaged every 2 min and another every 20.", + "impact": "med", + "effort": "low", + "why": "role_class drives how Operations foregrounds/images an embryo but is invisible on the very surface dedicated to embryos. Cheap parity/hidden-state win." + }, + { + "id": "IDEA-20", + "kind": "missing-affordance", + "title": "Structural plan editing: add/remove/reorder items + draw dependency edges", + "surfaces": [ + "Plans > Graph", + "Plans > Doc ('\u2192 blocks:')", + "Plans > Board" + ], + "entities": [ + "Plan item", + "Plan item dependency", + "Campaign" + ], + "sketch": "In Graph, add a node (new item), delete/reorder in Doc, and drag between nodes to author a blocks/blocked-by edge (backed by add_plan_item_dependency). Today campaigns.js only PATCHes spec fields; item structure and the dependency edges that drive get_unblocked_plan_items are agent-only.", + "impact": "high", + "effort": "high", + "why": "Deep (creates entities + edges) and structural, but the highest-effort item in the top tier \u2014 the whole create/delete column of the campaign hierarchy is agent-only. US-07." + }, + { + "id": "IDEA-21", + "kind": "unattended / temporal", + "title": "Notify-me on watchpoints, stalls, and run completion (out-of-app + ETA)", + "surfaces": [ + "Operate run chooser (stop conditions)", + "Operations", + "Settings > Alerts" + ], + "entities": [ + "Watchpoint", + "Question", + "Session", + "Operation Plan" + ], + "sketch": "The run chooser already has stop conditions and Settings has in-page amber/red thresholds, but all in-page. Wire watchpoints + pending agent questions to a browser/push/email channel so a scientist away for the ~14h window is told when hatching is detected, a run stalls, or the agent needs an answer; add a run ETA ('finishes ~03:40').", + "impact": "high", + "effort": "med", + "why": "A developmental timelapse outlives attention; the codebase uses no Notification API today. A whole temporal axis the time-agnostic core lenses miss." + }, + { + "id": "IDEA-22", + "kind": "consistency / cross-surface parity", + "title": "Save-to-library + Run-again from the Operations tactic card", + "surfaces": [ + "Operations > Overview (tactic cards)", + "Operate > Run chooser ('From library')" + ], + "entities": [ + "Tactic", + "Tactic library", + "Operation Plan" + ], + "sketch": "Operate consumes the library (GET /api/tactic_library) and runs tactics, but nothing writes a composed/running tactic back INTO the library \u2014 the reuse loop is one-way. Add 'Save to library' and 'Run again' to the expanded Overview tactic card so a tactic the agent just composed becomes reusable.", + "impact": "med", + "effort": "med", + "why": "'run a tactic \u2192 keep it for next time' is broken at the human-authoring step; the apply half exists, only the save half is conceptual. US-17." + }, + { + "id": "IDEA-23", + "kind": "cross-feature-link", + "title": "In-UI plan-item picker in Operate's 'Continue a plan' mode", + "surfaces": [ + "Operate > Run chooser ('Continue a plan')", + "Plans tab" + ], + "entities": [ + "Plan item", + "Session", + "Operation Plan", + "Campaign" + ], + "sketch": "loadPlanItems() renders only static text and Start ships a natural-language prompt asking the agent to guess 'the right plan item'. Instead list unblocked plan items (get_unblocked_plan_items) as selectable cards like the library picker already does, and attach the chosen item on Start.", + "impact": "med", + "effort": "med", + "why": "The chooser lets you click a saved tactic but 'Continue a plan' forces a chat hand-off and hopes the agent guesses; the ImagingSpec plan items already exist. Closes planning\u2192operating binding." + }, + { + "id": "IDEA-24", + "kind": "cross-feature-link", + "title": "App-wide attention surface (agent questions/watchpoints on every tab)", + "surfaces": [ + "context-surface 'Agent's view'", + "all workspace tabs", + "left rail / status area" + ], + "entities": [ + "Question", + "Watchpoint", + "Expectation", + "Agent chat / turn" + ], + "sketch": "#context-surface is mounted inside #home-content, so watchpoints/questions/expectations are visible and resolvable only on Home. Mirror the blocking ask-stage pattern (already dual-rendered app-wide): put a compact unresolved-count badge in the always-visible rail that expands the context lens, or render the surface app-wide.", + "impact": "med", + "effort": "med", + "why": "During a live run the operator is on Operations/Embryos/Devices but the non-blocking items needing a human decision are stranded on Home. The blocking ASK is already app-wide; the uncertainty queue isn't." + }, + { + "id": "IDEA-25", + "kind": "flywheel (producer\u2192consumer)", + "title": "Surface ML dataset-readiness and turn coverage gaps into planned imaging", + "surfaces": [ + "Plans tab (side panel)", + "Notebook > Findings", + "Embryos tab" + ], + "entities": [ + "ML data assessment", + "Plan item", + "Planned session", + "Ground truth" + ], + "sketch": "The agent already writes save_data_assessment (annotated_embryos, stage_distribution, coverage_gaps, quality_notes) and ML pipeline/run records, with ZERO UI consumers. Add a 'Dataset readiness' card rendering stage_distribution + coverage_gaps, and give each gap a '\u2192 Plan imaging to fill' button calling create_plan_item / create_planned_session pre-filled with the under-sampled strain+stage.", + "impact": "high", + "effort": "high", + "why": "An entire storage domain (agent/ml/*) is a pure dead-end producer; the coverage-gap signal that should steer the NEXT session is discarded. The core research-program flywheel (acquire\u2192assess\u2192acquire the gaps) with the consumer half missing. Depends on IDEA-01 for real ground truth." + }, + { + "id": "IDEA-26", + "kind": "cross-feature-link", + "title": "Schedule + start a planned session (planning\u2192operating loop, with back-link)", + "surfaces": [ + "Plans tab (planned-sessions)", + "plan-item inspector", + "Sessions tab" + ], + "entities": [ + "Planned session", + "Session", + "Plan item", + "Campaign" + ], + "sketch": "A 'Schedule session' form in the plan-item inspector (title/date/duration/source session to inherit params) POSTing to a new create endpoint, then a 'Start this run' action that opens Operate pre-seeded with those params and stamps the resulting Session with a back-reference ('fulfils: in '). Only a GET route exists for planned-sessions today.", + "impact": "med", + "effort": "med", + "why": "A Planned session is defined as the thing that becomes a Session when started, yet there's no create/start and no actual\u2194planned back-link \u2014 schedule and execution are disconnected. US-14/US-42." + }, + { + "id": "IDEA-27", + "kind": "navigation / findability", + "title": "Global finder (Cmd-K) + entity-mention linkifier everywhere", + "surfaces": [ + "global shell/header", + "Notebook cards", + "Embryos VLM reasoning", + "Operations tactic cards" + ], + "entities": [ + "Session", + "Embryo", + "Campaign", + "Tactic", + "Notebook note", + "Strain" + ], + "sketch": "A command-palette overlay: type 'CX3198' or 'aiy-pol' and jump straight there (Sessions currently hides 39 sessions behind a filter with no search). Plus promote the timepoint-only linkifyTimepoints into a general entity-linker so '\ud83e\uddecCX3198', '\u25ccemb_0007', '#nerve-ring-pioneers', 'Moyle et al. 2021' become links wherever they appear as text.", + "impact": "med", + "effort": "med", + "why": "No search exists anywhere; linkification exists for exactly one entity type. Serves both wayfinding and the expert accelerant the uniform core lenses miss." + }, + { + "id": "IDEA-28", + "kind": "agentic (augmented-LLM in the loop)", + "title": "Recommend-a-tactic at the Run chooser instead of a blind hand-off", + "surfaces": [ + "Operate > Run chooser", + "Operations > Overview", + "Operate worklist" + ], + "entities": [ + "Tactic", + "Role", + "Embryo", + "Prediction/Stage", + "Plan item" + ], + "sketch": "The chooser knows roles + embryos but the pick is blind and 'Hand to agent' is a bare prompt. Add a 'Recommend' affordance that, given the marked embryos' roles + latest stage predictions + the active plan item, returns a ranked tactic + cadence with a one-line rationale and a diff vs the library default, pre-selecting a run mode; Start executes it.", + "impact": "high", + "effort": "med", + "why": "Deepens the single highest-stakes moment (committing hardware time) by linking Prediction + Role + Plan item \u2192 Tactic, entities adjacent in the model but unlinked in the UI. Agent narrows, human commits." + }, + { + "id": "IDEA-29", + "kind": "flywheel (producer\u2192consumer)", + "title": "Render agent embryo-understanding (needs_attention/health) as badges + auto-watchpoints", + "surfaces": [ + "Embryos > Board/Vitals", + "context-surface 'Watching'", + "Operate worklist" + ], + "entities": [ + "Embryo understanding", + "Watchpoint", + "Embryo" + ], + "sketch": "update_embryo_understanding persists current_stage, health_assessment, needs_attention + attention_reason, is_hatched, notes per embryo, with ZERO UI consumers. Surface needs_attention as a red badge + attention_reason tooltip on cards, show health/notes, auto-promote needs_attention embryos into the 'Watching' surface, and use is_hatched to retire an embryo from the worklist.", + "impact": "med", + "effort": "low", + "why": "The agent's richest per-embryo judgement is written every cycle and read only back into its own prompt; the human never sees 'this embryo needs attention because X'. Orphaned producer feeding an existing surface \u2014 cheap." + }, + { + "id": "IDEA-30", + "kind": "missing-affordance", + "title": "Watch-this-embryo (create a watchpoint from an embryo)", + "surfaces": [ + "Embryos > Board/Vitals/Default", + "context-surface 'Watching'" + ], + "entities": [ + "Watchpoint", + "Embryo", + "Expectation" + ], + "sketch": "A 'Watch' button on an embryo card/row creating a watchpoint (embryo + condition + priority) shown in the always-on 'Watching' surface. Watchpoints are agent-created today; the UI can only resolve them.", + "impact": "med", + "effort": "low", + "why": "Both a missing affordance and a cross-feature link (Embryo cards \u2194 Watching surface) that lets human attention join the agent's. Low cost; pairs with IDEA-11/IDEA-29." + }, + { + "id": "IDEA-31", + "kind": "hidden-state / make-visible", + "title": "Timelapse liveness + next-acquisition ETA on the always-on strip", + "surfaces": [ + "v2-strip", + "header", + "Operations Overview" + ], + "entities": [ + "Session", + "Operation Plan", + "Tactic", + "Volume/Image" + ], + "sketch": "The v2-strip renders only 'N embryos \u00b7 Connected' and the template hardcodes a static 'LIVE' label that can lie. Feed it from timelapse.yaml + the per-session events already used for the temp graph to show 'Timelapse running \u00b7 t23/120 \u00b7 next in 1:40' or 'Idle' from any tab; run monitoring lives only inside Operate today.", + "impact": "med", + "effort": "med", + "why": "Whether a run is actually acquiring, how far along, and when the next frame fires are only reachable by navigating into Operate \u2014 and the 'LIVE' chip is static. Hidden-state + temporal." + }, + { + "id": "IDEA-32", + "kind": "agentic (augmented-LLM in the loop)", + "title": "Pre-commit plan review: Gently self-critiques the run against the notebook", + "surfaces": [ + "plan wizard 'THE PLAN' / commit", + "Plans > Doc/Decide", + "Home > Recent Plans" + ], + "entities": [ + "Operation Plan", + "Plan item", + "Learning", + "Notebook note", + "ImagingSpec" + ], + "sketch": "Fill the incomplete US-05 review/commit step with an agentic pass: before commit, check the designed items against existing learnings/notes and flag conflicts inline (it already knows 'ky123 ~80% lethal \u2192 over-sample to n=12', 'cross-check Moyle et al. 2021 before the burst interval'). Each flag is an Accept-suggestion / Ignore card; Commit stays gated until reviewed.", + "impact": "high", + "effort": "med", + "why": "The notebook already surfaces these plan-relevant cautions but disconnected from the plan being authored; an agent review is the natural place to enforce Learning\u2192Plan-item links and plugs the concrete US-05 gap." + }, + { + "id": "IDEA-33", + "kind": "onboarding & expert-mode", + "title": "Empty-state deep-links + expert accelerants (repeat-last-run, remember choices)", + "surfaces": [ + "all headless tabs (Embryos/Sessions/Devices temp/Gallery)", + "landing", + "Operate run chooser" + ], + "entities": [ + "Session", + "Embryo", + "Tactic", + "Role", + "Setpoint (temperature)" + ], + "sketch": "Novice half: every empty state names + deep-links the single next action ('No embryos yet' \u2192 the Operate/mark step; 'No temperature data' \u2192 set a setpoint) instead of dead-ending. Expert half: stop re-showing the landing on every reload, add a one-tap 'Repeat last run' that reapplies the previous run's roles+mode+cadence+stop for this campaign, and 'all subject / alternate' bulk role toggles.", + "impact": "med", + "effort": "low", + "why": "Noise-collapse folds ~15 'nicer empty state' items into one systemic idea (deep-link to the seeding action). Adds the expert accelerant + frequency-weighted repeat-run for the highest-frequency loop (mark\u2192run, a 6-decision funnel). US-13." + }, + { + "id": "IDEA-34", + "kind": "remove-don't-add (subtraction)", + "title": "Remove / fix dead & misleading controls", + "surfaces": [ + "Embryos detection cards", + "Sessions list", + "keyboard-shortcuts modal" + ], + "entities": [ + "Session", + "Prediction/Stage" + ], + "sketch": "(a) Until wired to ground truth (IDEA-01), remove the localStorage-only Agree/Disagree rather than fake persisted feedback. (b) Sessions shows '39 empty sessions hidden' yet still counts '39 sessions' \u2014 auto-prune/fold empties. (c) The shortcuts modal lists tabs that no longer exist (Embryos=1, System=2, Live View=3\u2026) vs the real nav \u2014 correct or delete.", + "impact": "med", + "effort": "low", + "why": "Subtraction axis the additive core lenses structurally can't propose: each item is worse-than-absent (misleads). Cheap." + }, + { + "id": "IDEA-35", + "kind": "hidden-state / make-visible", + "title": "Ambient 'Gently is acting' indicator during autonomous turns", + "surfaces": [ + "header agent-chat toggle", + "v2-strip", + "context-surface" + ], + "entities": [ + "Agent chat / turn", + "Operation Plan", + "Device state" + ], + "sketch": "When the docked panel is closed the agent's live activity is invisible (the 'working\u2026' row lives inside the panel; the toggle dot only reflects connection). During a wake/autonomous turn (busySource==='wake') the agent can move the stage or acquire. Drive a pulsing 'Gently is acting\u2026' state on the header toggle + strip from the existing agentBusy/busySource.", + "impact": "med", + "effort": "low", + "why": "The busy/wake state is already tracked; only the closed-panel ambient surface is missing. Autonomous hardware motion with no standing signal is a trust/safety gap. Low cost." + }, + { + "id": "IDEA-36", + "kind": "agentic (augmented-LLM in the loop)", + "title": "Batch-triage low-confidence predictions into a human confirm queue", + "surfaces": [ + "Embryos > Board/Vitals/Default" + ], + "entities": [ + "Prediction/Stage", + "Ground truth", + "Trace", + "Embryo" + ], + "sketch": "Gently scans predictions.jsonl for low-confidence or is_transitional timepoints, re-examines each trace, and presents a compact review stack ('12 uncertain calls \u2014 5 min') where each item shows the projection, the agent's second-opinion stage, and Confirm / Correct. Confirmed/corrected items write ground_truth.yaml in bulk.", + "impact": "med", + "effort": "med", + "why": "Turns hours of scrubbing into a short confirm pass \u2014 agent narrows, human decides \u2014 feeding the same ground-truth store as IDEA-01 at scale. Depends on IDEA-01." + }, + { + "id": "IDEA-37", + "kind": "cross-feature-link", + "title": "Promote a per-timepoint VLM follow-up into ground truth / a notebook note", + "surfaces": [ + "Embryos > Default follow-up chat ('Ask a follow-up about this timepoint')", + "Notebook tab" + ], + "entities": [ + "Trace", + "Prediction/Stage", + "Ground truth", + "Notebook note", + "Embryo" + ], + "sketch": "There's already a live per-timepoint VLM chat (/api/perception/chat/{s}/{e}/{tp}) whose output is buried in traces/t{NNNN}_chat.jsonl. Add two end-of-conversation actions the agent offers on a conclusion: 'This settles it \u2192 set stage' (writes ground_truth) and 'Save to notebook' (drafts an observation pre-linked to this embryo/session/timepoint).", + "impact": "med", + "effort": "low", + "why": "An agentic surface already exists but dead-ends \u2014 the reasoning a biologist extracts by chatting never becomes ground truth or shared memory. Cheap, reuses shipped chat infra." + }, + { + "id": "IDEA-38", + "kind": "flywheel (producer\u2192consumer)", + "title": "Perception-run selector to compare model versions against ground truth", + "surfaces": [ + "Embryos > Board (sparkline)", + "Embryos > Vitals (stage strip)" + ], + "entities": [ + "Perception run", + "Prediction/Stage", + "Ground truth", + "Trace" + ], + "sketch": "create_perception_run records name/model_name/method/config per run and every prediction carries run_id; get_predictions already accepts a run_id filter but no UI exposes it. Add a run chip/selector above the stage charts to overlay run A (model v1) vs run B (model v2) on the same timepoints and \u2014 once ground truth exists \u2014 show each run's accuracy; feed the winner into ml_pipeline.best_run_id.", + "impact": "med", + "effort": "med", + "why": "Read path is 90% built (run_id filter) and simply never surfaced \u2014 a cheap way to close the model-eval loop. Depends on IDEA-01 for accuracy." + }, + { + "id": "IDEA-39", + "kind": "agentic (augmented-LLM in the loop)", + "title": "Watchpoint-fired agentic triage \u2192 propose + one-tap apply a tactic", + "surfaces": [ + "context-surface 'Watching'/'Expectations'", + "Operations tactic spine", + "#ask-stage" + ], + "entities": [ + "Watchpoint", + "Expectation", + "Tactic", + "Ask (agent \u2192 human)", + "Embryo" + ], + "sketch": "A fired watchpoint just sits with a manual Resolve today. Make firing trigger a triage: the agent pulls the embryo's recent predictions + temperature trace + governing tactic, then raises an Ask on #ask-stage with a concrete recommendation ('SubB2 approaching hatch in ~40 min \u2014 switch to recovery-monitor at 2-min cadence?') plus one-tap Apply, so resolving the watchpoint and running the tactic become one action.", + "impact": "high", + "effort": "high", + "why": "Bridges Watchpoint\u2192Tactic\u2192Ask, three entities with no edge today; canonical 'agentic triage of an alert' turning passive attention into an approve-only decision. High effort keeps it below the cheap structural wins." + }, + { + "id": "IDEA-40", + "kind": "trust / provenance / feedback-integrity", + "title": "Trace a finding to its supporting observations (learning basis drill-down)", + "surfaces": [ + "Notebook > Findings", + "Notebook > Observations" + ], + "entities": [ + "Learning", + "Observation", + "Notebook note" + ], + "sketch": "A learning carries a 'basis' field; render it as expandable links to the observations/notes that support it, so a FINDING can be drilled into its OBSERVATION evidence rather than reading as an unsourced assertion. Findings and Observations are two independent filter tabs of one list today.", + "impact": "med", + "effort": "med", + "why": "Makes the agent's conclusions auditable (trust) and reuses data already stored on the learning. US-33/US-36." + }, + { + "id": "IDEA-41", + "kind": "cross-feature-link", + "title": "Contextual 'Ask about this embryo/session' (scoped notebook Ask)", + "surfaces": [ + "Embryos > Default (embryo detail rail)", + "Session Review header", + "Notebook Ask box" + ], + "entities": [ + "Embryo", + "Notebook note", + "Trace", + "Session" + ], + "sketch": "The notebook Ask box (POST /api/notebook/ask) already accepts thread/strain scoping via select_notes. Add an 'Ask about this embryo' box on the embryo detail rail and session-review header that calls notebook/ask pre-scoped to that embryo's notes/trace.", + "impact": "med", + "effort": "med", + "why": "The ask affordance is siloed in the notebook though its API already accepts scoping; the surfaces with the richest local context can't invoke it." + }, + { + "id": "IDEA-42", + "kind": "consistency / cross-surface parity", + "title": "Temperature graph + note-from-excursion on Session Review and Operations", + "surfaces": [ + "Session Review", + "Devices temperature graph", + "Operations (temp-change burst)" + ], + "entities": [ + "Temperature sample/graph", + "Session", + "Setpoint (temperature)", + "Observation" + ], + "sketch": "Temperature history is session-scoped (/api/temperature/{id}/history) and temperature-graph.js already renders it, but Session Review shows only Embryos/Detections/Conversation. Add a Temperature panel to review reusing the component. Plus a 'Note this \u2192' on the temp graph / live burst that pre-fills a notebook observation with timestamp + from/to setpoint + embryo scope.", + "impact": "med", + "effort": "low", + "why": "Same entity, same endpoint, component already exists; review omits the one place a completed session's thermal record is most worth seeing. Note-from-excursion composes with IDEA-02." + }, + { + "id": "IDEA-43", + "kind": "hidden-state / cross-feature-link", + "title": "Mesh peers + campaign-sharing surface + 'claimed by {peer}' badges on plan items", + "surfaces": [ + "Settings > mesh block", + "Plans (campaign navigator + item rows)", + "new Peers view" + ], + "entities": [ + "Mesh / peer instance", + "Campaign", + "Plan item", + "Session" + ], + "sketch": "campaigns.py implements share/unshare/join/claim/status and peer discovery raises agent asks, but the only client mesh UI is a read-only config block. Turn it into a participants panel with online/offline dots, add a 'Share/Join' action on a campaign, and render the claimed_by/claimed_by_hostname the campaign tree already serializes as a claim badge on item rows (disable local start on items claimed by another instance).", + "impact": "low", + "effort": "med", + "why": "Full collaboration backend with no front-end; in a shared campaign the claim badge is the only signal a peer is already executing an item, so instances can silently collide. Lower impact for the single-rig case. US-43." + }, + { + "id": "IDEA-44", + "kind": "consistency / cross-surface parity", + "title": "Copy-id affordance on embryo identifiers", + "surfaces": [ + "Embryos cards / Operate worklist ('\u25cc uid')", + "Notebook chips" + ], + "entities": [ + "Embryo", + "Session" + ], + "sketch": "The header already has copySessionId (clipboard + copied feedback). Embryo uids are the token you paste into chat or a note but have no copy affordance. Add a small copy icon next to the embryo uid, mirroring the session button.", + "impact": "low", + "effort": "low", + "why": "Identical need, affordance exists only for the session id. Trivial parity win; kept for breadth, ranked last." + }, + { + "id": "IDEA-45", + "kind": "missing-affordance", + "title": "Inline-edit campaign metadata (target / status / description)", + "surfaces": [ + "Plans tab (campaign inspector)", + "Home > Recent Plans" + ], + "entities": [ + "Campaign", + "Plan item" + ], + "sketch": "Make the campaign header fields editable in the inspector the same way plan-item spec fields already PATCH. Today only plan-item spec is editable; the parent campaign's own metadata is agent-only.", + "impact": "med", + "effort": "med", + "why": "Plan item has an edit path but the parent Campaign doesn't \u2014 a researcher can't correct a target or flip status without the agent, though the inline-edit pattern already exists one level down. Pairs with IDEA-03." + } + ], + "top_bets": [ + { + "title": "IDEA-01 \u2014 Persist ground-truth stage corrections", + "why": "The flagship deep-impact win: the persistence layer (set/get_ground_truth) already exists, yet human corrections dead-end in localStorage. Persists a new entity and single-handedly unblocks accuracy, dataset-readiness (IDEA-25), few-shot examples, and model comparison (IDEA-38) \u2014 the whole perception feedback flywheel." + }, + { + "title": "IDEA-04 \u2014 Clickable notebook chips (+ reverse links)", + "why": "Best leverage/effort in the app: the embryo/strain/session foreign keys are already in the note payload and rendered as dead text; the work is rendering a chip that navigates. Turns the notebook from a reading room into a navigable web and the nav graph from a star into a mesh." + }, + { + "title": "IDEA-05 \u2014 Learnings/watchpoints inside the Operate step", + "why": "Canonical loop-closer the brief calls out: Operate is completely severed from the agent's memory (operate.js has zero context/notebook refs). Bringing durable insight to the moment of hardware commitment is the highest-value cross-feature link and only a filtered read of existing stores." + }, + { + "title": "IDEA-03 \u2014 New campaign / New plan controls in the workspace", + "why": "High/low ratio and passes agent-arbitrage: originating a research program is a core loop trapped behind an incidental logo-click, and the wizard launcher already exists \u2014 pure wiring for a gap that affects every returning user." + }, + { + "title": "IDEA-02 \u2014 Notebook add-note composer", + "why": "The single most obvious verb on a lab notebook is absent while the store already models human-authored notes; without it the 'shared lab notebook' framing is one-directional. Grounds a missing affordance in the entity that most needs cross-links." + }, + { + "title": "IDEA-10 \u2014 Device-offline banner + toast on failed actions", + "why": "The richest untapped vein the happy-path core lenses were blind to: 7 stories 502 silently. The online/offline signal already exists in the status store and just needs wiring to the action surfaces \u2014 a low-effort fix to a trust-breaking silent failure." + } + ], + "clusters": [ + { + "theme": "Close the perception feedback loop (correction \u2192 model)", + "ideas": [ + "IDEA-01", + "IDEA-36", + "IDEA-14", + "IDEA-38", + "IDEA-29", + "IDEA-37", + "IDEA-25" + ] + }, + { + "theme": "Make the notebook a two-way memory", + "ideas": [ + "IDEA-02", + "IDEA-04", + "IDEA-07", + "IDEA-18", + "IDEA-40", + "IDEA-41" + ] + }, + { + "theme": "Create & edit from the workspace, not the agent", + "ideas": [ + "IDEA-03", + "IDEA-20", + "IDEA-45", + "IDEA-23", + "IDEA-26", + "IDEA-22" + ] + }, + { + "theme": "Bring memory & judgement to the moment of decision", + "ideas": [ + "IDEA-05", + "IDEA-06", + "IDEA-28", + "IDEA-32", + "IDEA-39" + ] + }, + { + "theme": "Trust & safety on live hardware", + "ideas": [ + "IDEA-09", + "IDEA-10", + "IDEA-12", + "IDEA-16", + "IDEA-35", + "IDEA-31" + ] + }, + { + "theme": "Get results out (provenance / export)", + "ideas": [ + "IDEA-08", + "IDEA-42", + "IDEA-17", + "IDEA-44" + ] + }, + { + "theme": "Navigation, attention & findability", + "ideas": [ + "IDEA-11", + "IDEA-13", + "IDEA-24", + "IDEA-27", + "IDEA-30", + "IDEA-33" + ] + }, + { + "theme": "Perturbation \u2192 response science", + "ideas": [ + "IDEA-15", + "IDEA-42", + "IDEA-06" + ] + }, + { + "theme": "Unattended & multi-instance operation", + "ideas": [ + "IDEA-21", + "IDEA-16", + "IDEA-43", + "IDEA-09" + ] + }, + { + "theme": "Subtraction, roles & polish", + "ideas": [ + "IDEA-34", + "IDEA-19", + "IDEA-33" + ] + } + ], + "method_notes": "METHOD IMPROVEMENTS. (1) Shift from a 7-kind additive taxonomy to a LENS LIBRARY of 22 questions run against every (surface \u00d7 entity \u00d7 graph-edge) triple, split by role: GENERATORS derive candidates by construction rather than inspiration \u2014 capability-orphan (every mutating store method \u2192 route \u2192 UI control; orphaned verbs = missing affordances, e.g. set_ground_truth, create_campaign, note-create) and dangling-edge (every foreign key in the data model rendered as dead text = a cross-feature link, e.g. Note.embryos/strains/basis, item.session_ids/depends_on, claimed_by). These read the CODE/data model, not the rendered screen, which is exactly where the deepest, lowest-effort ideas hide and where a screenshot-only audit is structurally blind. (2) RANKING lenses: loop-closure on the Plan\u2192Operate\u2192Acquire\u2192Perceive\u2192Learn\u2192Decide spine, and frequency\u00d7friction (weight the mark\u2192run loop run ~20\u00d7/day over once-per-project config). (3) FILTERS applied before scoring: agent-arbitrage (a manual affordance must be materially better than the agent path \u2014 faster/safer/in-context/discoverable/works-when-agent-busy \u2014 or it's dropped; 'the agent can do it in chat' is a RED FLAG masking a gap, not coverage) and noise-collapse (an Nth template instance collapses to one systemic idea \u2014 ~15 'nicer empty state' items became one 'empty states deep-link to their seeding action').\n\nQUALITY RUBRIC. Score = (impact \u00d7 reach \u00d7 depth \u00d7 trust) / effort, gated by code evidence, with a structural bias (+ for missing-affordance and cross-feature-link) applied per the brief. DEPTH axis is decisive: DEEP if it creates/persists a new entity or edge (ground truth, dependency, note\u2194plan link); COSMETIC if it only moves pixels \u2014 the two must never rank equal. HARD-REJECT before scoring: template spam, cosmetic-only polish, audit-echo (restating a US-## gap adds nothing over an idea with a concrete mechanism), and agent-redundant LLM bolt-ons (the rejected 'AI summary on Logs' / 'refresh button everywhere'). Effort-blind ranking is banned \u2014 a one-line render of an existing foreign key must outrank a huge-payoff/huge-cost item, which is why IDEA-04/IDEA-11 rank above IDEA-20/IDEA-25/IDEA-39.\n\nWHAT TO KEEP. The three graph artifacts as living inputs: G_nav (crawler graph.json \u2014 proves the app is a star of sibling tabs with zero entity-to-entity edges), G_data (entity/FK graph from the storage model), G_verb (store-method\u2192route\u2192handler capability graph). Emit ideas as mechanical diffs \u2014 cross-feature-link = a G_data edge whose endpoints both have surfaces but no G_nav path; missing-affordance = a G_verb verb that dead-ends before the UI; orphan-surface = a G_nav node only ever seen empty whose seeding action is unreachable. Dedup by systemic collapse (46 raw candidates \u2192 45 ranked, with the largest merges being ground-truth \u00d711, add-note \u00d76, new-campaign/plan \u00d76, chip-deep-link \u00d75). Keep the added lenses that catch the core-lens blind spots the method-gap list named: failure branch, subtraction, second actor/control-ownership, temporal/unattended, app-boundary export, physical risk, and fine-grained navigability." +} \ No newline at end of file diff --git a/docs/superpowers/PR-PLAN.md b/docs/superpowers/PR-PLAN.md new file mode 100644 index 00000000..042c5e5c --- /dev/null +++ b/docs/superpowers/PR-PLAN.md @@ -0,0 +1,32 @@ +# Wrap-up PR plan — the temperature-experiment + Operations goal + +**Principle (user-stated):** capture the breadth of work as **separate PRs**, each a distinct unit of +work with its own identity/ownership and its own tests, **additive on top of PR #58** +(`integration/ux2-all`, the UX-v2 stack), stacked so they **compose into the final product** in order. +Each PR diffs cleanly against its parent in the stack; integrate the chain when rig-verified. + +## The stack (base = PR #58 `integration/ux2-all`) + +| # | Branch | Unit of work | Parent | Tests / review | +|---|--------|--------------|--------|----------------| +| 1 | `feature/temperature-interface` (A) | Temperature persistence (`append/read_temperature_sample`) + sampler service + SVG temperature graph w/ setpoint line | #58 | per-task + whole-branch review; graph Chrome-audited | +| 2 | `feature/manual-mode-live-view` (B1) | Manual-mode imaging: lightsheet brightfield live view (sequence acquisition), illumination control (LED/laser presets), galvo/piezo scan params + **4 acquire-safety fixes** (C1/I1/I2/I3) | A | per-task + whole-branch (opus) review | +| 3 | `feature/temp-change-tactic` (C) | Automated temp-change burst protocol (`wait_for_temperature_lock` + driver), burst-acquisition wiring, protocol events + agent tool | B1 | per-task + whole-branch review + fixes | +| 4 | `feature/operations-tab` (D) | **Operations: the agent-authored Operation Plan** — typed declare tool, store, route, execution-linkage (tactic_id + updater), plan-item seeding, operation-spine renderer + live binding | C | 10 tasks + whole-branch (opus) review + 6 fixes; 105 tests; Chrome-audited | +| 5 | `feature/tactics-library` (G) | Save / list / apply reusable typed tactics (on D's substrate), mirroring plan-templates; apply→Operation Plan | D | 3 tasks + whole-branch review + fixes; ~64 tests | +| 6 | `feature/embryo-roles-observability` (D2) | Per-embryo **strain** field + roles-as-**use** (lineaging + subject/reference) + multi-embryo Operations roster lens (role + strain) | G | 4 tasks + whole-branch review + fix; 54 tests; Chrome-audited | +| 7 | `feature/session-plan-linking` (F) | Session↔**plans** link/delink: multi-plan model (`unlink_plan_item_session` + reverse-query), link/delink endpoints, Plans-tab controls + session Linked-plans panel | D2 | 4 tasks + whole-branch review + fix; ~70 tests; both surfaces Chrome-audited | +| 8 | `feature/manual-mode-dual-camera` (B2) | Dual-camera config + laser-preset browser + timelapse config form (extends B1 manual mode) | F | TBD (SDD) | + +Notes: +- Each branch is the natural unit of "distinct enough to own a PR." Sub-parts (e.g. B1's safety fixes) + stay inside their branch — granular enough to capture the work, not so granular it's noise. +- A/B1/C are **kept-as-is pending rig verification** (not merged); D/G/D2 build on top. The stack is + intact but unmerged — PRs can open stacked and merge the chain once verified on the rig. +- "Easy to put together into a final product" = the linear stack already composes; the integration + point is #58 → `development`. + +## At wrap-up +1. Verify each branch's tests pass (the SDD ledgers + whole-branch reviews are the evidence trail). +2. Open the stacked PRs in order (each targets its parent branch), each description capturing its unit. +3. Rebase/integrate the chain onto #58 when rig-verified; #58 → `development` as the final integration. diff --git a/docs/superpowers/mockups/2026-07-02-launcher-gate.html b/docs/superpowers/mockups/2026-07-02-launcher-gate.html new file mode 100644 index 00000000..18879dde --- /dev/null +++ b/docs/superpowers/mockups/2026-07-02-launcher-gate.html @@ -0,0 +1,82 @@ + + + + + +Launch Gently + + + +
+
Gently
+

What do you need today?

+ +
+
+ +
+
+
Microscope hardware
+
Start the device layer & connect to the rig · :60610
+
+ +
+ +
+
+ +
+
+
AI agent
+
Chat, perception & planning — uses your API key
+
+ +
+ + +
Advanced options · remembers your choice
+
+ + diff --git a/docs/superpowers/notes/2026-06-28-lightsheet-fps-measurement.md b/docs/superpowers/notes/2026-06-28-lightsheet-fps-measurement.md new file mode 100644 index 00000000..d1a4abd9 --- /dev/null +++ b/docs/superpowers/notes/2026-06-28-lightsheet-fps-measurement.md @@ -0,0 +1,57 @@ +# Lightsheet live-view FPS measurement + transport decision (B1 Task 7) + +Status: **deferred to the rig** — the numbers below must be filled in on the microscope +(the streamer needs the real `pymmcore` core, SPIM camera, and rpyc transport; it cannot run +on the Linux dev box). This file is the measurement protocol + the decision gate. + +## What to measure + +With the Manual view open and lightsheet live running, record three rates: + +| Metric | Where to read it | +|---|---| +| **device grab rate** (frames peeked/encoded /s) | device-layer log / instrument `_lightsheet_streamer` | +| **delivered rate** (frames broadcast /s) | device-layer `_broadcast_lightsheet` | +| **browser paint rate** | the Manual-view FPS readout (`computeLightsheetFps`) | + +Record at two resolution/quality settings: +- default **512 px / JPEG q70** (`_ls_target_max_dim=512`, `_ls_jpeg_quality=70`) +- reduced **384 px / q60** + +Note the exposure used (the peek floor is `max(exposure, 1/30 s)`). + +## Target + +**≥ ~15 fps usable for focus** (stretch 25–30), end-to-end latency < ~150 ms. + +## Diagnosis rule (the gate) + +- **device grab < target** → limiter is exposure / readout / rpyc, **not** transport. Tune + exposure, the 512 px size, and JPEG quality. A binary transport path will NOT help — stop here. +- **device grab ≥ target but browser paint < target** → transport is the bottleneck → build the + **binary WebSocket path** (below). + +## Conditional escalation — binary WebSocket path + +Only if the diagnosis points to transport. The current path is base64-JPEG-in-JSON over SSE → +EventBus → `ConnectionManager.broadcast` (`json.dumps` + `send_text`) — `connection_manager.py` +has **no `send_bytes` path** (confirmed). Escalation, on the **agent→browser hop** (where cost +multiplies per client): + +- push raw JPEG bytes via `websocket.send_bytes(prefix + jpeg)` (a 1-byte type tag identifies a + lightsheet frame), bypassing **base64 (+33%)**, the **per-client `json.dumps`**, and the + **EventBus fan-out**; +- browser `onmessage` (binary) → `createImageBitmap(new Blob([buf]))` → `ctx.drawImage`; +- the device→agent SSE stays as-is (single consumer = the monitor, so its base64 cost is paid + once, not per browser). + +Re-measure after building; record the before/after numbers here. + +## Results (fill in on the rig) + +| setting | device grab fps | delivered fps | browser paint fps | exposure | notes | +|---|---|---|---|---|---| +| 512px/q70 | _TBD_ | _TBD_ | _TBD_ | _TBD_ | | +| 384px/q60 | _TBD_ | _TBD_ | _TBD_ | _TBD_ | | + +Decision: _TBD (transport bottleneck? build binary path Y/N)_ diff --git a/docs/superpowers/plans/2026-06-16-notebook-foundation.md b/docs/superpowers/plans/2026-06-16-notebook-foundation.md new file mode 100644 index 00000000..4cb777e9 --- /dev/null +++ b/docs/superpowers/plans/2026-06-16-notebook-foundation.md @@ -0,0 +1,661 @@ +# Notebook Foundation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build the unit-testable foundation of the shared lab notebook — the unified `Note` model and a file-backed `NotebookStore` (write / read / scope-query / rebuildable reverse-indexes / link & supersede) — with no UI, API, or agent wiring. + +**Architecture:** A new self-contained module `gently/harness/memory/notebook.py`. One `Note` dataclass (three kinds: observation/finding/question) with author, status, confidence, scope facets (strains/embryos/sessions/threads), typed links, basis, and artifact pointers — orthogonal fields, not subtypes. `NotebookStore` persists one YAML per note under `notebook/notes/{id}_{slug}.yaml` (atomic write, mirroring `FileContextStore`), maintains rebuildable reverse-indexes by strain/embryo/thread, and answers scope+kind+status queries. This is Increment 1's keystone from the design doc (`docs/superpowers/specs/2026-06-16-shared-lab-notebook-design.md`). + +**Tech Stack:** Python 3.11, dataclasses, `str`-Enums, PyYAML, pytest (fixtures in `tests/conftest.py`, flat `tests/` layout). + +**Follow-on plans (NOT in scope here):** producer wiring (`apply_updates` → notebook), `/api/notebook` + Notebook tab (UI), retrieval/embeddings + brainstorm. Each ships on top of this foundation. + +--- + +### Task 1: The `Note` model + +**Files:** +- Create: `gently/harness/memory/notebook.py` +- Test: `tests/test_notebook_store.py` + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_notebook_store.py +"""Tests for the shared lab notebook: Note model + NotebookStore.""" + +from datetime import datetime + +from gently.harness.memory.model import Confidence +from gently.harness.memory.notebook import ( + Author, + Note, + NoteKind, + NoteStatus, + note_from_dict, + note_to_dict, +) + + +class TestNoteModel: + def test_round_trip_minimal(self): + n = Note(id="abc123", kind=NoteKind.OBSERVATION, body="dim rings at 10 ms") + d = note_to_dict(n) + assert d["kind"] == "observation" + assert d["author"] == "agent" # default + assert d["status"] == "confirmed" # default + back = note_from_dict(d) + assert back == n + + def test_round_trip_full(self): + n = Note( + id="def456", + kind=NoteKind.FINDING, + body="temperature shifts timing ~12 min/degC", + author=Author.AGENT, + title="Temp shifts timing", + status=NoteStatus.PROPOSED, + confidence=Confidence.MEDIUM, + strains=["N2", "OH904"], + embryos=["emb_0007"], + sessions=["20260615_1432_x"], + threads=["q_division_temp"], + basis=["obs_1", "obs_2"], + links=[{"rel": "supports", "to": "q_division_temp"}], + artifacts=[{"kind": "projection", "session": "s1", "embryo": "emb_0007", "t": 42}], + created_at=datetime(2026, 6, 16, 11, 0, 0), + updated_at=datetime(2026, 6, 16, 11, 0, 0), + ) + back = note_from_dict(note_to_dict(n)) + assert back == n + assert note_to_dict(n)["confidence"] == "medium" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestNoteModel -v` +Expected: FAIL — `ModuleNotFoundError: No module named 'gently.harness.memory.notebook'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# gently/harness/memory/notebook.py +""" +The shared lab notebook — unified memory entry (Note) and file-backed store. + +One Note kind taxonomy (observation / finding / question); everything else +(author, status, confidence, scope, links) is an orthogonal field. See +docs/superpowers/specs/2026-06-16-shared-lab-notebook-design.md. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from datetime import datetime +from enum import Enum +from typing import Any + +from .model import Confidence + + +class NoteKind(str, Enum): + OBSERVATION = "observation" # immutable record of what was seen/done/read/noted + FINDING = "finding" # revisable, supersedable believed claim + QUESTION = "question" # open inquiry; large ones are the thread spine + + +class Author(str, Enum): + HUMAN = "human" + AGENT = "agent" + PERCEPTION = "perception" + + +class NoteStatus(str, Enum): + OPEN = "open" # questions not yet answered + PROPOSED = "proposed" # agent-drafted finding awaiting human confirm + CONFIRMED = "confirmed" # accepted observation/finding (default) + ANSWERED = "answered" # question resolved + SUPERSEDED = "superseded" # replaced by a newer note + + +@dataclass +class Note: + id: str + kind: NoteKind + body: str + author: Author = Author.AGENT + title: str | None = None + status: NoteStatus = NoteStatus.CONFIRMED + confidence: Confidence | None = None + strains: list[str] = field(default_factory=list) + embryos: list[str] = field(default_factory=list) + sessions: list[str] = field(default_factory=list) + threads: list[str] = field(default_factory=list) + basis: list[str] = field(default_factory=list) # note ids this rests on + links: list[dict] = field(default_factory=list) # [{"rel": ..., "to": ...}] + artifacts: list[dict] = field(default_factory=list) # FileStore pointers + superseded_by: str | None = None + created_at: datetime = field(default_factory=datetime.now) + updated_at: datetime = field(default_factory=datetime.now) + + +def note_to_dict(n: Note) -> dict[str, Any]: + return { + "id": n.id, + "kind": n.kind.value, + "body": n.body, + "author": n.author.value, + "title": n.title, + "status": n.status.value, + "confidence": n.confidence.value if n.confidence else None, + "strains": list(n.strains), + "embryos": list(n.embryos), + "sessions": list(n.sessions), + "threads": list(n.threads), + "basis": list(n.basis), + "links": list(n.links), + "artifacts": list(n.artifacts), + "superseded_by": n.superseded_by, + "created_at": n.created_at.isoformat(), + "updated_at": n.updated_at.isoformat(), + } + + +def note_from_dict(d: dict[str, Any]) -> Note: + conf = d.get("confidence") + return Note( + id=d["id"], + kind=NoteKind(d["kind"]), + body=d.get("body", ""), + author=Author(d.get("author", "agent")), + title=d.get("title"), + status=NoteStatus(d.get("status", "confirmed")), + confidence=Confidence(conf) if conf else None, + strains=list(d.get("strains") or []), + embryos=list(d.get("embryos") or []), + sessions=list(d.get("sessions") or []), + threads=list(d.get("threads") or []), + basis=list(d.get("basis") or []), + links=list(d.get("links") or []), + artifacts=list(d.get("artifacts") or []), + superseded_by=d.get("superseded_by"), + created_at=datetime.fromisoformat(d["created_at"]), + updated_at=datetime.fromisoformat(d["updated_at"]), + ) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestNoteModel -v` +Expected: PASS (2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook.py tests/test_notebook_store.py +git commit -m "feat(notebook): unified Note model + dict serialization" +``` + +--- + +### Task 2: `NotebookStore` — write & read a note + +**Files:** +- Modify: `gently/harness/memory/notebook.py` (append `NotebookStore`) +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +from gently.harness.memory.notebook import NotebookStore + + +class TestNotebookStoreReadWrite: + def test_write_assigns_id_and_persists(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + n = Note(id="", kind=NoteKind.OBSERVATION, body="bean stage at t40") + note_id = store.write_note(n) + assert note_id # non-empty id assigned + files = list((tmp_path / "notebook" / "notes").glob("*.yaml")) + assert len(files) == 1 + assert files[0].name.startswith(note_id + "_") + + def test_get_note_round_trip(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + n = Note(id="", kind=NoteKind.FINDING, body="x", strains=["N2"], threads=["t1"]) + note_id = store.write_note(n) + got = store.get_note(note_id) + assert got is not None + assert got.id == note_id + assert got.kind == NoteKind.FINDING + assert got.strains == ["N2"] + + def test_get_missing_returns_none(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + assert store.get_note("nope") is None +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookStoreReadWrite -v` +Expected: FAIL — `ImportError: cannot import name 'NotebookStore'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# append to gently/harness/memory/notebook.py +import copy +import os +import re +import uuid +from pathlib import Path + +import yaml + + +class NotebookStore: + """File-backed store for notebook Notes. One YAML per note under notes/; + flat pool, rebuildable reverse-indexes (added in Task 3).""" + + def __init__(self, notebook_dir: Path): + self.root = Path(notebook_dir) + self.notes_dir = self.root / "notes" + self.index_dir = self.root / "index" + self.notes_dir.mkdir(parents=True, exist_ok=True) + self.index_dir.mkdir(parents=True, exist_ok=True) + + # ---- helpers (mirror FileContextStore conventions) ---- + @staticmethod + def _gen_id() -> str: + return str(uuid.uuid4())[:8] + + @staticmethod + def _slugify(text: str) -> str: + slug = re.sub(r"[^a-z0-9]+", "-", (text or "").lower()).strip("-") + return slug[:30] + + def _write_yaml(self, path: Path, data: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_suffix(".tmp") + with open(tmp, "w", encoding="utf-8") as fh: + yaml.safe_dump(data, fh, default_flow_style=False, allow_unicode=True, sort_keys=False) + os.replace(str(tmp), str(path)) + + def _read_yaml(self, path: Path) -> dict | None: + try: + with open(path, encoding="utf-8") as fh: + return yaml.safe_load(fh) + except OSError: + return None + + def _note_path(self, note_id: str) -> Path | None: + return next(self.notes_dir.glob(f"{note_id}_*.yaml"), None) + + # ---- read/write ---- + def write_note(self, note: Note) -> str: + if not note.id: + note.id = self._gen_id() + note.updated_at = datetime.now() + slug = self._slugify(note.title or note.body or note.kind.value) + # remove any stale file for this id (slug may have changed) + old = self._note_path(note.id) + if old is not None: + old.unlink() + self._write_yaml(self.notes_dir / f"{note.id}_{slug}.yaml", note_to_dict(note)) + return note.id + + def get_note(self, note_id: str) -> Note | None: + path = self._note_path(note_id) + if path is None: + return None + data = self._read_yaml(path) + return note_from_dict(data) if data else None +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookStoreReadWrite -v` +Expected: PASS (3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook.py tests/test_notebook_store.py +git commit -m "feat(notebook): NotebookStore write_note/get_note with atomic YAML" +``` + +--- + +### Task 3: Reverse-indexes (by strain / embryo / thread) + rebuild + +**Files:** +- Modify: `gently/harness/memory/notebook.py` (`NotebookStore`) +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +class TestNotebookIndex: + def test_index_updated_on_write(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + a = store.write_note(Note(id="", kind=NoteKind.OBSERVATION, body="a", strains=["N2"])) + b = store.write_note( + Note(id="", kind=NoteKind.OBSERVATION, body="b", strains=["N2", "OH904"]) + ) + assert set(store.ids_for_strain("N2")) == {a, b} + assert store.ids_for_strain("OH904") == [b] + assert store.ids_for_strain("missing") == [] + + def test_index_by_embryo_and_thread(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + a = store.write_note( + Note(id="", kind=NoteKind.FINDING, body="a", embryos=["e1"], threads=["t1"]) + ) + assert store.ids_for_embryo("e1") == [a] + assert store.ids_for_thread("t1") == [a] + + def test_rebuild_index_from_disk(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + a = store.write_note(Note(id="", kind=NoteKind.OBSERVATION, body="a", strains=["N2"])) + # a fresh store over the same dir must rebuild the index by scanning notes/ + store2 = NotebookStore(tmp_path / "notebook") + assert store2.ids_for_strain("N2") == [a] +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookIndex -v` +Expected: FAIL — `AttributeError: 'NotebookStore' object has no attribute 'ids_for_strain'` + +- [ ] **Step 3: Write minimal implementation** + +Modify `NotebookStore.__init__` to add index state + rebuild, extend `write_note` to update the index, and add the index methods. Replace the existing `__init__` and `write_note` with these versions and add the new methods: + +```python +# --- replace __init__ --- +def __init__(self, notebook_dir: Path): + self.root = Path(notebook_dir) + self.notes_dir = self.root / "notes" + self.index_dir = self.root / "index" + self.notes_dir.mkdir(parents=True, exist_ok=True) + self.index_dir.mkdir(parents=True, exist_ok=True) + # reverse indexes: facet -> {value: [note_id, ...]} + self._index: dict[str, dict[str, list[str]]] = {"strain": {}, "embryo": {}, "thread": {}} + self.rebuild_index() + + +# --- add: facet extraction + index maintenance --- +_FACETS = {"strain": "strains", "embryo": "embryos", "thread": "threads"} + + +def _index_note(self, note: Note) -> None: + for facet, attr in self._FACETS.items(): + for value in getattr(note, attr): + bucket = self._index[facet].setdefault(value, []) + if note.id not in bucket: + bucket.append(note.id) + + +def rebuild_index(self) -> None: + """Rebuild reverse-indexes by scanning notes/ (the notes are authoritative; + indexes are disposable caches).""" + self._index = {"strain": {}, "embryo": {}, "thread": {}} + for f in sorted(self.notes_dir.glob("*.yaml")): + data = self._read_yaml(f) + if data: + self._index_note(note_from_dict(data)) + + +def ids_for_strain(self, strain: str) -> list[str]: + return list(self._index["strain"].get(strain, [])) + + +def ids_for_embryo(self, embryo: str) -> list[str]: + return list(self._index["embryo"].get(embryo, [])) + + +def ids_for_thread(self, thread: str) -> list[str]: + return list(self._index["thread"].get(thread, [])) +``` + +Then add an index-update at the end of `write_note` (just before `return note.id`): + +```python + self._index_note(note) + return note.id +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookIndex -v` +Expected: PASS (3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook.py tests/test_notebook_store.py +git commit -m "feat(notebook): rebuildable reverse-indexes by strain/embryo/thread" +``` + +--- + +### Task 4: `query_notes` — filter by kind / author / status / scope + +**Files:** +- Modify: `gently/harness/memory/notebook.py` (`NotebookStore`) +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +class TestNotebookQuery: + def _seed(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + store.write_note(Note(id="o1", kind=NoteKind.OBSERVATION, body="o", strains=["N2"])) + store.write_note( + Note( + id="f1", + kind=NoteKind.FINDING, + body="f", + status=NoteStatus.PROPOSED, + strains=["N2"], + threads=["t1"], + ) + ) + store.write_note( + Note(id="q1", kind=NoteKind.QUESTION, body="q", status=NoteStatus.OPEN, threads=["t1"]) + ) + return store + + def test_query_by_kind(self, tmp_path): + store = self._seed(tmp_path) + ids = {n.id for n in store.query_notes(kind=NoteKind.FINDING)} + assert ids == {"f1"} + + def test_query_by_thread_scope(self, tmp_path): + store = self._seed(tmp_path) + ids = {n.id for n in store.query_notes(thread="t1")} + assert ids == {"f1", "q1"} + + def test_query_by_thread_and_kind(self, tmp_path): + store = self._seed(tmp_path) + ids = {n.id for n in store.query_notes(thread="t1", kind=NoteKind.QUESTION)} + assert ids == {"q1"} + + def test_query_by_status(self, tmp_path): + store = self._seed(tmp_path) + ids = {n.id for n in store.query_notes(status=NoteStatus.OPEN)} + assert ids == {"q1"} + + def test_query_all_sorted_newest_first(self, tmp_path): + store = self._seed(tmp_path) + notes = store.query_notes() + assert len(notes) == 3 + ts = [n.created_at for n in notes] + assert ts == sorted(ts, reverse=True) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookQuery -v` +Expected: FAIL — `AttributeError: 'NotebookStore' object has no attribute 'query_notes'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# add to NotebookStore + def query_notes( + self, + *, + kind: NoteKind | None = None, + author: Author | None = None, + status: NoteStatus | None = None, + strain: str | None = None, + embryo: str | None = None, + thread: str | None = None, + ) -> list[Note]: + """Structural query: narrow by scope via the indexes, then filter by + kind/author/status. Returned newest-first. (No semantic ranking here — + that's a later increment.)""" + # 1. candidate ids — intersect any scope facets given, else all notes + scope_sets: list[set[str]] = [] + if strain is not None: + scope_sets.append(set(self.ids_for_strain(strain))) + if embryo is not None: + scope_sets.append(set(self.ids_for_embryo(embryo))) + if thread is not None: + scope_sets.append(set(self.ids_for_thread(thread))) + if scope_sets: + candidate_ids: set[str] | None = set.intersection(*scope_sets) + else: + candidate_ids = None # means "all" + + # 2. load + filter + results: list[Note] = [] + if candidate_ids is not None: + notes = [n for n in (self.get_note(i) for i in candidate_ids) if n] + else: + notes = [ + note_from_dict(d) + for d in (self._read_yaml(f) for f in self.notes_dir.glob("*.yaml")) + if d + ] + for n in notes: + if kind is not None and n.kind != kind: + continue + if author is not None and n.author != author: + continue + if status is not None and n.status != status: + continue + results.append(n) + results.sort(key=lambda n: n.created_at, reverse=True) + return results +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookQuery -v` +Expected: PASS (5 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook.py tests/test_notebook_store.py +git commit -m "feat(notebook): query_notes by kind/author/status/scope" +``` + +--- + +### Task 5: `link_notes` and `supersede_note` + +**Files:** +- Modify: `gently/harness/memory/notebook.py` (`NotebookStore`) +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +class TestNotebookLinkSupersede: + def test_link_notes_adds_typed_edge(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + a = store.write_note(Note(id="", kind=NoteKind.FINDING, body="a")) + b = store.write_note(Note(id="", kind=NoteKind.QUESTION, body="b")) + store.link_notes(a, "supports", b) + got = store.get_note(a) + assert {"rel": "supports", "to": b} in got.links + + def test_supersede_marks_old_and_points_new(self, tmp_path): + store = NotebookStore(tmp_path / "notebook") + old = store.write_note(Note(id="", kind=NoteKind.FINDING, body="old claim")) + new = store.write_note(Note(id="", kind=NoteKind.FINDING, body="better claim")) + store.supersede_note(old, new) + old_n = store.get_note(old) + new_n = store.get_note(new) + assert old_n.status == NoteStatus.SUPERSEDED + assert old_n.superseded_by == new + assert {"rel": "refines", "to": old} in new_n.links +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestNotebookLinkSupersede -v` +Expected: FAIL — `AttributeError: 'NotebookStore' object has no attribute 'link_notes'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# add to NotebookStore + def link_notes(self, from_id: str, rel: str, to_id: str) -> None: + """Add a typed edge from one note to another (append-only).""" + note = self.get_note(from_id) + if note is None: + raise KeyError(from_id) + edge = {"rel": rel, "to": to_id} + if edge not in note.links: + note.links.append(edge) + self.write_note(note) + + def supersede_note(self, old_id: str, new_id: str) -> None: + """Mark old as superseded (kept, never deleted) and link the new note + back to it as a refinement — the chain is the intellectual history.""" + old = self.get_note(old_id) + if old is None: + raise KeyError(old_id) + old.status = NoteStatus.SUPERSEDED + old.superseded_by = new_id + self.write_note(old) + self.link_notes(new_id, "refines", old_id) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py -v` +Expected: PASS (all tests across all classes pass) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook.py tests/test_notebook_store.py +git commit -m "feat(notebook): link_notes + supersede_note (append-only history)" +``` + +--- + +## Self-Review + +**Spec coverage (against the design doc §2 data model):** +- Three kinds (Observation/Finding/Question) → Task 1 `NoteKind`. ✓ +- Orthogonal fields (author/status/confidence/scope/links/artifacts) → Task 1 `Note`. ✓ +- Flat note pool + rebuildable reverse-indexes (strain/embryo/thread) → Tasks 2-3. ✓ +- "By question + by strain + links coexist over flat YAML, no DB" → Tasks 3-4 (indexes + scope-intersect query). ✓ +- Append-only / supersede-never-overwrite → Task 5 `supersede_note`. ✓ +- Typed links graph → Tasks 1 (`links`) + 5 (`link_notes`). ✓ +- *Deferred to follow-on plans (correctly out of scope):* inquiry-thread object, working-memory split, producer wiring, API/tab, embeddings/retrieval, consolidation. Noted in header. + +**Placeholder scan:** No TBD/TODO; every code step shows complete code; commands have expected output. ✓ + +**Type consistency:** `Note`, `NoteKind`, `Author`, `NoteStatus`, `note_to_dict`/`note_from_dict`, and `NotebookStore.{write_note,get_note,rebuild_index,ids_for_strain,ids_for_embryo,ids_for_thread,query_notes,link_notes,supersede_note}` are named identically across all tasks and tests. `Confidence` is imported from `.model` (confirmed to exist). ✓ diff --git a/docs/superpowers/plans/2026-06-16-notebook-live-edge.md b/docs/superpowers/plans/2026-06-16-notebook-live-edge.md new file mode 100644 index 00000000..9265d769 --- /dev/null +++ b/docs/superpowers/plans/2026-06-16-notebook-live-edge.md @@ -0,0 +1,188 @@ +# Notebook Live Edge Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Make the Home "Agent's view" panel surface recent notebook activity — the ambient "live edge" of the notebook (design §3, two-faced presentation), complementing the Notebook tab (the reading room). + +**Architecture:** Add an optional `limit` to the notes read API (TDD). Extend `context-surface.js` to also fetch recent notes and render a "From the notebook" section whose rows click through to the Notebook tab. Reuse the existing `cx-dot` colors (amber/blue/green) for kinds — no new CSS. + +**Tech Stack:** FastAPI, pytest + TestClient (venv), vanilla JS. + +**Out of scope:** retrieval/"Ask the notebook" (next increment); proactive surfacing. + +--- + +### Task 1: `limit` param on `GET /api/notebook/notes` + +**Files:** +- Modify: `gently/ui/web/routes/notebook.py` +- Test: `tests/test_notebook_api.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_api.py +class TestLimit: + def test_limit_returns_newest(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + data = client.get("/api/notebook/notes?limit=1").json() + assert len(data["notes"]) == 1 # newest-first, capped +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_notebook_api.py::TestLimit -v` +Expected: FAIL — returns 2 notes, not 1. + +- [ ] **Step 3: Write minimal implementation** + +In `gently/ui/web/routes/notebook.py`, add a `limit` param to `list_notes` and slice. Replace the `list_notes` signature and the final return: + +```python + @router.get("/api/notebook/notes") + async def list_notes( + kind: str | None = None, + author: str | None = None, + status: str | None = None, + strain: str | None = None, + embryo: str | None = None, + thread: str | None = None, + limit: int | None = None, + ): + nb = _nb() + if nb is None: + return {"available": False, "notes": []} + notes = nb.query_notes( + kind=_coerce(NoteKind, kind), + author=_coerce(Author, author), + status=_coerce(NoteStatus, status), + strain=strain, + embryo=embryo, + thread=thread, + ) + if limit is not None and limit >= 0: + notes = notes[:limit] + return {"available": True, "notes": [note_to_dict(n) for n in notes]} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_notebook_api.py -q` +Expected: all pass + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/routes/notebook.py tests/test_notebook_api.py +git commit -m "feat(notebook): limit param on GET /api/notebook/notes" +``` + +--- + +### Task 2: "From the notebook" section in the Agent's-view panel + +**Files:** +- Modify: `gently/ui/web/static/js/context-surface.js` + +Verification is browser-based (chrome-devtools), not pytest. + +- [ ] **Step 1: Extend `fetchAndRender` to also pull recent notes** + +Replace `fetchAndRender` with a version that fetches both endpoints and passes notes to `render`: + +```javascript + async function fetchAndRender() { + if (!el || loading) return; + loading = true; + try { + const [ctx, nb] = await Promise.all([ + fetch('/api/context').then(r => r.json()).catch(() => ({})), + fetch('/api/notebook/notes?limit=5').then(r => r.json()).catch(() => ({})), + ]); + render(ctx || {}, (nb && nb.notes) || []); + } catch (e) { /* keep last render */ } + finally { loading = false; } + } +``` + +- [ ] **Step 2: Render the notebook section** + +Replace `render(data)` with `render(data, notes)`. Add the notebook section and include notes in the empty-state check. Replace the whole `render` function body: + +```javascript + function render(data, notes) { + if (!el) return; + notes = notes || []; + const hc = hasControl(); + const questions = data.questions || [], watchpoints = data.watchpoints || [], expectations = data.expectations || []; + el.classList.remove('hidden'); + if (!questions.length && !watchpoints.length && !expectations.length && !notes.length) { + el.innerHTML = '
Agent’s view
' + + '
Nothing yet — the agent’s notes, expectations, and open questions appear here as it works.
'; + return; + } + + const qHtml = questions.map(it => ` +
+ + ${esc(it.content)} + ${hc ? '' : ''} + ${hc ? '' : ''} +
`).join(''); + const wHtml = watchpoints.map(it => ` +
+ + ${esc(it.target)}${it.condition ? ' — ' + esc(it.condition) : ''} + ${hc ? '' : ''} +
`).join(''); + const eHtml = expectations.map(it => ` +
+ + ${esc(it.target)}${it.prediction ? ': ' + esc(it.prediction) : ''} + ${hc ? '' : ''} +
`).join(''); + // kind → existing cx-dot color: observation=blue, finding=green, question=amber + const dotFor = (k) => k === 'finding' ? 'cx-e' : (k === 'question' ? 'cx-q' : 'cx-w'); + const nHtml = notes.map(n => ` +
+ + ${esc(n.title || n.body)} +
`).join(''); + + el.innerHTML = '
Agent’s view
' + + section('Open questions', qHtml) + section('Watching', wHtml) + + section('Expectations', eHtml) + section('From the notebook', nHtml); + wire(); + } +``` + +- [ ] **Step 3: Make notebook rows click through to the Notebook tab** + +In `wire()`, after the existing `.cx-item` loop, add a handler for note rows (append inside `wire`, before its closing brace): + +```javascript + el.querySelectorAll('.cx-note').forEach(row => { + row.style.cursor = 'pointer'; + row.addEventListener('click', () => { + if (typeof switchTab === 'function') switchTab('notebook'); + }); + }); +``` + +- [ ] **Step 4: Verify in the browser** + +Restart the server, open `http://localhost:8080`, confirm the Home "Agent's view" panel shows a "From the notebook" section with recent notes, and clicking a row switches to the Notebook tab. Use chrome-devtools (navigate, evaluate_script to click, take_screenshot, list_console_messages → zero errors). + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/static/js/context-surface.js +git commit -m "feat(notebook): Agent's-view live edge — recent notes section -> Notebook tab" +``` + +--- + +## Self-Review +**Spec coverage:** design §3 two-faced presentation — the ambient live edge now surfaces recent notebook notes on Home, clicking through to the tab. ✓ Reuses `/api/notebook/notes` (+ new `limit`) and existing `cx-dot` colors. ✓ +**Placeholder scan:** none. ✓ +**Type consistency:** `render(data, notes)`, `dotFor`, `limit` param, `switchTab('notebook')` all consistent; kinds map to existing cx classes. ✓ diff --git a/docs/superpowers/plans/2026-06-16-notebook-producer-bridge.md b/docs/superpowers/plans/2026-06-16-notebook-producer-bridge.md new file mode 100644 index 00000000..27974225 --- /dev/null +++ b/docs/superpowers/plans/2026-06-16-notebook-producer-bridge.md @@ -0,0 +1,261 @@ +# Notebook Producer Bridge Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Make the existing agent-memory write path actually populate the shared notebook — when `FileContextStore.apply_updates()` records observations and learnings, mirror them into the `NotebookStore` as Notes. + +**Architecture:** Pure converters (`observation_to_note`, `learning_to_note`) in `notebook.py`; a lazy `FileContextStore.notebook` property rooted at `agent_dir/notebook`; a guarded mirror step at the end of `apply_updates`. Builds on the foundation plan (`2026-06-16-notebook-foundation.md`). Backend-only, no UI/agent-loop changes. Transitional dual-write (legacy YAML + notebook) — legacy silos retire in a later increment. + +**Tech Stack:** Python 3.11, dataclasses, PyYAML, pytest (`file_context_store` fixture in `tests/conftest.py`). + +**Out of scope:** wiring the live loop to *call* `apply_updates` (separate increment); read API + Notebook tab; mapping expectations/watchpoints (they're working memory, not notebook entries — see design doc §2). + +--- + +### Task 1: Converters — Observation/Learning → Note + +**Files:** +- Modify: `gently/harness/memory/notebook.py` +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +from datetime import datetime as _dt + +from gently.harness.memory.model import Learning, Observation +from gently.harness.memory.notebook import learning_to_note, observation_to_note + + +class TestConverters: + def test_observation_to_note(self): + obs = Observation( + id="o1", + timestamp=_dt(2026, 6, 16, 9, 0, 0), + type="milestone", + content="nerve ring formed", + embryo_id="e1", + session_id="s1", + relates_to=["o0"], + gently_refs={"kind": "projection", "t": 42}, + ) + n = observation_to_note(obs) + assert n.id == "o1" + assert n.kind == NoteKind.OBSERVATION + assert n.body == "nerve ring formed" + assert n.author == Author.AGENT + assert n.embryos == ["e1"] + assert n.sessions == ["s1"] + assert {"rel": "relates_to", "to": "o0"} in n.links + assert n.artifacts == [{"kind": "projection", "t": 42}] + assert n.created_at == _dt(2026, 6, 16, 9, 0, 0) + + def test_learning_to_note(self): + lrn = Learning(id="l1", content="rings form by comma", confidence=Confidence.HIGH) + n = learning_to_note(lrn) + assert n.id == "l1" + assert n.kind == NoteKind.FINDING + assert n.body == "rings form by comma" + assert n.status == NoteStatus.PROPOSED # agent-drafted, awaits confirm + assert n.confidence == Confidence.HIGH +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestConverters -v` +Expected: FAIL — `ImportError: cannot import name 'observation_to_note'` + +- [ ] **Step 3: Write minimal implementation** + +Add the model imports and converters to `notebook.py`. Extend the existing model import line: + +```python +from .model import Confidence, Learning, Observation +``` + +Append at end of `notebook.py` (module-level functions, after `note_from_dict`): + +```python +def observation_to_note(obs: Observation) -> Note: + """Bridge a legacy Observation into a notebook Note (kind=observation).""" + return Note( + id=obs.id, + kind=NoteKind.OBSERVATION, + body=obs.content, + author=Author.AGENT, + embryos=[obs.embryo_id] if obs.embryo_id else [], + sessions=[obs.session_id] if obs.session_id else [], + links=[{"rel": "relates_to", "to": r} for r in (obs.relates_to or [])], + artifacts=[obs.gently_refs] if obs.gently_refs else [], + created_at=obs.timestamp, + updated_at=obs.timestamp, + ) + + +def learning_to_note(learning: Learning) -> Note: + """Bridge a legacy Learning into a notebook Note (kind=finding, proposed).""" + return Note( + id=learning.id, + kind=NoteKind.FINDING, + body=learning.content, + author=Author.AGENT, + status=NoteStatus.PROPOSED, + confidence=learning.confidence, + created_at=learning.created_at, + updated_at=learning.created_at, + ) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestConverters -v` +Expected: PASS (2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook.py tests/test_notebook_store.py +git commit -m "feat(notebook): Observation/Learning -> Note converters" +``` + +--- + +### Task 2: `FileContextStore.notebook` property + +**Files:** +- Modify: `gently/harness/memory/file_store.py` (add property near `apply_updates`) +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +class TestContextStoreNotebook: + def test_notebook_property_rooted_under_agent_dir(self, file_context_store): + nb = file_context_store.notebook + assert nb.root == file_context_store.agent_dir / "notebook" + + def test_notebook_property_is_cached(self, file_context_store): + assert file_context_store.notebook is file_context_store.notebook +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestContextStoreNotebook -v` +Expected: FAIL — `AttributeError: 'FileContextStore' object has no attribute 'notebook'` + +- [ ] **Step 3: Write minimal implementation** + +In `gently/harness/memory/file_store.py`, add this property immediately **before** `def apply_updates(self, updates: ContextUpdates):` (line ~2178): + +```python +@property +def notebook(self): + """The shared lab notebook, rooted at agent_dir/notebook (lazy).""" + nb = getattr(self, "_notebook", None) + if nb is None: + from .notebook import NotebookStore + + nb = NotebookStore(self.agent_dir / "notebook") + self._notebook = nb + return nb +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestContextStoreNotebook -v` +Expected: PASS (2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/file_store.py tests/test_notebook_store.py +git commit -m "feat(notebook): FileContextStore.notebook lazy property" +``` + +--- + +### Task 3: Mirror observations & learnings in `apply_updates` + +**Files:** +- Modify: `gently/harness/memory/file_store.py` (`apply_updates`) +- Test: `tests/test_notebook_store.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_store.py +class TestApplyUpdatesMirror: + def test_apply_updates_mirrors_observations_and_learnings(self, file_context_store): + from gently.harness.memory.model import ContextUpdates + + cs = file_context_store + obs = Observation( + id="o1", + timestamp=_dt(2026, 6, 16, 9, 0, 0), + type="milestone", + content="ring formed", + embryo_id="e1", + ) + lrn = Learning(id="l1", content="rings form by comma", confidence=Confidence.HIGH) + cs.apply_updates(ContextUpdates(new_observations=[obs], new_learnings=[lrn])) + + bodies = {n.body for n in cs.notebook.query_notes()} + assert "ring formed" in bodies + assert "rings form by comma" in bodies + assert cs.notebook.ids_for_embryo("e1") == ["o1"] + + def test_apply_updates_empty_is_noop_for_notebook(self, file_context_store): + from gently.harness.memory.model import ContextUpdates + + cs = file_context_store + cs.apply_updates(ContextUpdates()) + assert cs.notebook.query_notes() == [] +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_store.py::TestApplyUpdatesMirror -v` +Expected: FAIL — `assert "ring formed" in set()` (notebook not populated yet) + +- [ ] **Step 3: Write minimal implementation** + +In `gently/harness/memory/file_store.py`, at the END of `apply_updates` (after the `if updates.new_focus is not None:` block), append: + +```python + # Mirror new observations & learnings into the shared notebook + # (best-effort — a notebook failure never breaks the legacy write). + from .notebook import learning_to_note, observation_to_note + + try: + for obs in updates.new_observations: + self.notebook.write_note(observation_to_note(obs)) + for learning in updates.new_learnings: + self.notebook.write_note(learning_to_note(learning)) + except Exception: + logger.warning("notebook mirror failed", exc_info=True) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_store.py::TestApplyUpdatesMirror -v` +Expected: PASS (2 passed) + +- [ ] **Step 5: Run full notebook suite + commit** + +Run: `python -m pytest tests/test_notebook_store.py -q` +Expected: all pass + +```bash +git add gently/harness/memory/file_store.py tests/test_notebook_store.py +git commit -m "feat(notebook): apply_updates mirrors observations & learnings into notebook" +``` + +--- + +## Self-Review + +**Spec coverage:** Producer wiring (design doc increment 1b) — `apply_updates` now populates the notebook. ✓ Converters honor the model (Observation→observation note, Learning→finding/proposed). ✓ Working-memory types (expectation/watchpoint) intentionally not mirrored (design §2). ✓ +**Placeholder scan:** none; complete code + commands throughout. ✓ +**Type consistency:** `observation_to_note`/`learning_to_note`, `FileContextStore.notebook`, `NoteKind`/`Author`/`NoteStatus`/`Confidence` match the foundation module and `model.py` (`Observation`, `Learning`, `ContextUpdates` confirmed at `file_store.py:2178-2203`). ✓ diff --git a/docs/superpowers/plans/2026-06-16-notebook-read-api.md b/docs/superpowers/plans/2026-06-16-notebook-read-api.md new file mode 100644 index 00000000..9bd42e20 --- /dev/null +++ b/docs/superpowers/plans/2026-06-16-notebook-read-api.md @@ -0,0 +1,273 @@ +# Notebook Read API Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (or subagent-driven-development) to implement task-by-task. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Expose the shared notebook over HTTP so the frontend can read it — list/filter notes, fetch one note, and list inquiry-threads with counts. + +**Architecture:** A new route module `gently/ui/web/routes/notebook.py` exposing `create_router(server)` (the established pattern), reading `server.context_store.notebook` (the `NotebookStore` added in the producer-bridge plan) and serializing via `note_to_dict`. Registered in `routes/__init__.py`. Read-only; authoring/curation is a later increment. + +**Tech Stack:** FastAPI `APIRouter`, pytest + `fastapi.testclient.TestClient`, the `file_context_store` fixture (`tests/conftest.py`). + +**Out of scope:** the Notebook tab UI (next plan, browser-verified); the Agent's-View rewire; retrieval/embeddings. + +--- + +### Task 1: Route module — list & get notes + +**Files:** +- Create: `gently/ui/web/routes/notebook.py` +- Modify: `gently/ui/web/routes/__init__.py` +- Test: `tests/test_notebook_api.py` + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_notebook_api.py +"""Tests for the notebook read API.""" + +from fastapi import FastAPI +from fastapi.testclient import TestClient + +from gently.harness.memory.notebook import Note, NoteKind, NoteStatus + + +def _make_app(context_store): + from gently.ui.web.routes.notebook import create_router + + app = FastAPI() + + class _Server: + pass + + server = _Server() + server.context_store = context_store + app.include_router(create_router(server)) + return app + + +def _seed(cs): + nb = cs.notebook + nb.write_note(Note(id="o1", kind=NoteKind.OBSERVATION, body="ring formed", strains=["N2"])) + nb.write_note( + Note( + id="f1", + kind=NoteKind.FINDING, + body="rings by comma", + status=NoteStatus.PROPOSED, + strains=["N2"], + threads=["t1"], + ) + ) + return cs + + +class TestListNotes: + def test_no_store_available_false(self): + client = TestClient(_make_app(None)) + data = client.get("/api/notebook/notes").json() + assert data == {"available": False, "notes": []} + + def test_list_all(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + data = client.get("/api/notebook/notes").json() + assert data["available"] is True + assert {n["id"] for n in data["notes"]} == {"o1", "f1"} + + def test_filter_by_kind(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + data = client.get("/api/notebook/notes?kind=finding").json() + assert {n["id"] for n in data["notes"]} == {"f1"} + + def test_filter_by_strain(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + data = client.get("/api/notebook/notes?strain=N2").json() + assert {n["id"] for n in data["notes"]} == {"o1", "f1"} + + def test_invalid_kind_is_ignored(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + data = client.get("/api/notebook/notes?kind=bogus").json() + assert {n["id"] for n in data["notes"]} == {"o1", "f1"} + + +class TestGetNote: + def test_get_existing(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + data = client.get("/api/notebook/notes/o1").json() + assert data["id"] == "o1" + assert data["body"] == "ring formed" + + def test_get_missing_404(self, file_context_store): + client = TestClient(_make_app(_seed(file_context_store))) + resp = client.get("/api/notebook/notes/nope") + assert resp.status_code == 404 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_api.py -v` +Expected: FAIL — `ModuleNotFoundError: No module named 'gently.ui.web.routes.notebook'` + +- [ ] **Step 3: Write minimal implementation** + +Create `gently/ui/web/routes/notebook.py`: + +```python +"""Notebook (shared lab notebook) read routes. + +Exposes the notebook's Notes for the Notebook tab + Agent's-View live edge. +Read-only here; authoring/curation come in a later increment. +""" + +from fastapi import APIRouter, HTTPException + +from gently.harness.memory.notebook import Author, NoteKind, NoteStatus, note_to_dict + + +def _coerce(enum_cls, value): + """Parse a query-param string into an enum; invalid/None → None (no filter).""" + if value is None: + return None + try: + return enum_cls(value) + except ValueError: + return None + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _nb(): + cs = getattr(server, "context_store", None) + return cs.notebook if cs is not None else None + + @router.get("/api/notebook/notes") + async def list_notes( + kind: str | None = None, + author: str | None = None, + status: str | None = None, + strain: str | None = None, + embryo: str | None = None, + thread: str | None = None, + ): + nb = _nb() + if nb is None: + return {"available": False, "notes": []} + notes = nb.query_notes( + kind=_coerce(NoteKind, kind), + author=_coerce(Author, author), + status=_coerce(NoteStatus, status), + strain=strain, + embryo=embryo, + thread=thread, + ) + return {"available": True, "notes": [note_to_dict(n) for n in notes]} + + @router.get("/api/notebook/notes/{note_id}") + async def get_note(note_id: str): + nb = _nb() + if nb is None: + raise HTTPException(status_code=404, detail="notebook unavailable") + note = nb.get_note(note_id) + if note is None: + raise HTTPException(status_code=404, detail="note not found") + return note_to_dict(note) + + return router +``` + +Then register it in `gently/ui/web/routes/__init__.py`. Add the import after the `images` import line: + +```python +from .notebook import create_router as create_notebook_router +``` + +And add `create_notebook_router,` to the factory tuple in `register_all_routes` (after `create_context_router,`): + +```python + create_context_router, + create_notebook_router, + ): +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_api.py -v` +Expected: PASS (7 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/routes/notebook.py gently/ui/web/routes/__init__.py tests/test_notebook_api.py +git commit -m "feat(notebook): read API — GET /api/notebook/notes + /notes/{id}" +``` + +--- + +### Task 2: `GET /api/notebook/threads` + +**Files:** +- Modify: `gently/ui/web/routes/notebook.py` +- Test: `tests/test_notebook_api.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_api.py +class TestThreads: + def test_no_store(self): + client = TestClient(_make_app(None)) + assert client.get("/api/notebook/threads").json() == {"available": False, "threads": []} + + def test_thread_counts(self, file_context_store): + cs = file_context_store + nb = cs.notebook + nb.write_note(Note(id="a", kind=NoteKind.QUESTION, body="q", threads=["t1"])) + nb.write_note(Note(id="b", kind=NoteKind.FINDING, body="f", threads=["t1", "t2"])) + client = TestClient(_make_app(cs)) + data = client.get("/api/notebook/threads").json() + assert data["available"] is True + assert data["threads"] == [{"id": "t1", "count": 2}, {"id": "t2", "count": 1}] +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m pytest tests/test_notebook_api.py::TestThreads -v` +Expected: FAIL — 404 (route not defined) + +- [ ] **Step 3: Write minimal implementation** + +Add this endpoint inside `create_router`, just before `return router`: + +```python + @router.get("/api/notebook/threads") + async def list_threads(): + nb = _nb() + if nb is None: + return {"available": False, "threads": []} + counts: dict[str, int] = {} + for n in nb.query_notes(): + for t in n.threads: + counts[t] = counts.get(t, 0) + 1 + threads = [{"id": t, "count": c} for t, c in sorted(counts.items())] + return {"available": True, "threads": threads} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m pytest tests/test_notebook_api.py -q` +Expected: all pass + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/routes/notebook.py tests/test_notebook_api.py +git commit -m "feat(notebook): read API — GET /api/notebook/threads with counts" +``` + +--- + +## Self-Review + +**Spec coverage:** read surface API for the notebook (design increment 1c, backend half) — query notes by kind/author/status/scope, fetch one, list threads. ✓ Reuses `NotebookStore.query_notes` + `note_to_dict` from the foundation. ✓ Follows `create_router(server)` + `server.context_store` convention (`context.py`). ✓ +**Placeholder scan:** none — complete code + commands. ✓ +**Type consistency:** `create_router`, `note_to_dict`, `NoteKind`/`Author`/`NoteStatus`, `server.context_store.notebook`, `nb.query_notes`/`get_note` all match the foundation + producer-bridge modules. Registration matches the existing tuple in `routes/__init__.py`. ✓ diff --git a/docs/superpowers/plans/2026-06-17-notebook-ask.md b/docs/superpowers/plans/2026-06-17-notebook-ask.md new file mode 100644 index 00000000..03d62f06 --- /dev/null +++ b/docs/superpowers/plans/2026-06-17-notebook-ask.md @@ -0,0 +1,418 @@ +# "Ask the Notebook" Implementation Plan (Increment 2, backend) + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** Let the notebook be *reasoned with* — given a question (+ optional scope), retrieve relevant Notes, ask Claude to synthesize a **grounded, cited** answer, and return it as a validated structured object. + +**Architecture:** A new module `gently/harness/memory/notebook_ask.py`: structural retrieval (`select_notes`) + a forced-`tool_choice` synthesis call (`answer_question`) that reuses gently's conventions — `anthropic.AsyncAnthropic` (per `chat.py:198`), `settings.models.main` (Opus 4.8), structured output via a pinned tool (per `verifier.py`), and **no self-rated confidence** (per the lab rule). A `POST /api/notebook/ask` route wires retrieval → synthesis. The Claude client is injected so everything is unit-testable with a fake. + +**Tech Stack:** Python 3.11, `anthropic` SDK, FastAPI, pytest + TestClient (venv). + +**Out of scope (later plans):** embeddings/semantic recall (structural-only here); the "Ask" UI box on the Notebook tab; proactive surfacing. + +--- + +### Task 1: Structural retrieval — `select_notes` + +**Files:** +- Create: `gently/harness/memory/notebook_ask.py` +- Test: `tests/test_notebook_ask.py` + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_notebook_ask.py +"""Tests for 'Ask the notebook' — retrieval + grounded synthesis.""" + +from gently.harness.memory.notebook import Note, NoteKind +from gently.harness.memory.notebook_ask import select_notes + + +def _seed(cs): + nb = cs.notebook + nb.write_note( + Note(id="o1", kind=NoteKind.OBSERVATION, body="ring formed", strains=["N2"], threads=["t1"]) + ) + nb.write_note( + Note(id="f1", kind=NoteKind.FINDING, body="12 min/degC", strains=["N2"], threads=["t1"]) + ) + nb.write_note(Note(id="x1", kind=NoteKind.OBSERVATION, body="unrelated", strains=["OH904"])) + return nb + + +class TestSelectNotes: + def test_scope_by_thread(self, file_context_store): + nb = _seed(file_context_store) + ids = {n.id for n in select_notes(nb, thread="t1")} + assert ids == {"o1", "f1"} + + def test_scope_by_strain(self, file_context_store): + nb = _seed(file_context_store) + ids = {n.id for n in select_notes(nb, strain="OH904")} + assert ids == {"x1"} + + def test_no_scope_returns_recent_capped(self, file_context_store): + nb = _seed(file_context_store) + notes = select_notes(nb, limit=2) + assert len(notes) == 2 # newest-first, capped +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_notebook_ask.py::TestSelectNotes -v` +Expected: FAIL — `ModuleNotFoundError: No module named 'gently.harness.memory.notebook_ask'` + +- [ ] **Step 3: Write minimal implementation** + +```python +# gently/harness/memory/notebook_ask.py +"""Ask the notebook — retrieve relevant Notes and synthesize a grounded, +cited answer with Claude. Structural retrieval only (semantic recall is a +later increment). See docs/superpowers/specs/2026-06-16-shared-lab-notebook-design.md §4. +""" + +from __future__ import annotations + +from .notebook import Note, NotebookStore + + +def select_notes( + store: NotebookStore, + *, + thread: str | None = None, + strain: str | None = None, + limit: int = 12, +) -> list[Note]: + """Structural narrowing: scope by thread/strain when given, else recent. + Returns newest-first, capped at `limit`.""" + notes = store.query_notes(thread=thread, strain=strain) + return notes[:limit] +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_notebook_ask.py::TestSelectNotes -v` +Expected: PASS (3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook_ask.py tests/test_notebook_ask.py +git commit -m "feat(notebook): select_notes — structural retrieval for ask" +``` + +--- + +### Task 2: Grounded synthesis — `answer_question` (forced tool) + +**Files:** +- Modify: `gently/harness/memory/notebook_ask.py` +- Test: `tests/test_notebook_ask.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_ask.py +import asyncio + +from gently.harness.memory.notebook_ask import ASK_TOOL, answer_question, build_ask_messages + + +class _FakeBlock: + def __init__(self, inp): + self.type = "tool_use" + self.input = inp + + +class _FakeResp: + def __init__(self, inp): + self.content = [_FakeBlock(inp)] + self.stop_reason = "tool_use" + + +class _FakeMessages: + def __init__(self, captured, inp): + self._captured, self._inp = captured, inp + + async def create(self, **kwargs): + self._captured.update(kwargs) + return _FakeResp(self._inp) + + +class _FakeClient: + def __init__(self, inp): + self.captured = {} + self.messages = _FakeMessages(self.captured, inp) + + +class TestAnswerQuestion: + def test_build_messages_embeds_note_ids(self): + notes = [Note(id="o1", kind=NoteKind.OBSERVATION, body="ring formed")] + msgs = build_ask_messages("what formed?", notes) + text = msgs[0]["content"] + assert "o1" in text and "ring formed" in text and "what formed?" in text + + def test_answer_returns_structured_and_forces_tool(self): + canned = { + "answer": "A ring formed.", + "points": [{"text": "ring formed", "note_ids": ["o1"]}], + "suggested_next": [], + "coverage": "covered", + } + client = _FakeClient(canned) + notes = [Note(id="o1", kind=NoteKind.OBSERVATION, body="ring formed")] + out = asyncio.run(answer_question(client, "m", "what formed?", notes)) + assert out == canned + # tool_choice is pinned to the ask tool (forced structured output) + assert client.captured["tool_choice"] == {"type": "tool", "name": ASK_TOOL["name"]} + assert client.captured["model"] == "m" + + def test_answer_no_notes_short_circuits_without_api(self): + client = _FakeClient({"should": "not be used"}) + out = asyncio.run(answer_question(client, "m", "anything?", [])) + assert out["coverage"] == "not_in_notebook" + assert client.captured == {} # no API call when nothing to ground on +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_notebook_ask.py::TestAnswerQuestion -v` +Expected: FAIL — `ImportError: cannot import name 'ASK_TOOL'` + +- [ ] **Step 3: Write minimal implementation** + +Append to `gently/harness/memory/notebook_ask.py`: + +```python +# ASK_TOOL pins the structured output. No confidence field — we don't ask the +# model to self-rate (lab rule); "coverage" is a factual grounding classification. +ASK_TOOL = { + "name": "answer_from_notebook", + "description": "Return a grounded answer built ONLY from the provided notebook entries.", + "input_schema": { + "type": "object", + "properties": { + "answer": {"type": "string", "description": "Direct synthesis grounded in the notes."}, + "points": { + "type": "array", + "description": "Supporting points, each citing the note ids it rests on.", + "items": { + "type": "object", + "properties": { + "text": {"type": "string"}, + "note_ids": {"type": "array", "items": {"type": "string"}}, + }, + "required": ["text", "note_ids"], + }, + }, + "suggested_next": { + "type": "array", + "items": {"type": "string"}, + "description": "Concrete next experiments/moves if the question asks what to do; else empty.", + }, + "coverage": { + "type": "string", + "enum": ["covered", "partial", "not_in_notebook"], + "description": "How well the provided notes cover the question.", + }, + }, + "required": ["answer", "points", "coverage"], + }, +} + +_SYSTEM = ( + "You reason over a shared lab notebook. Answer ONLY from the notebook entries " + "provided — every claim must cite the note id(s) it rests on. If the notes do " + "not contain the answer, say so plainly and set coverage to 'not_in_notebook'. " + "Never invent facts not in the notes. Call the answer_from_notebook tool." +) + + +def _render_notes(notes: list[Note]) -> str: + lines = [] + for n in notes: + scope = [] + if n.strains: + scope.append("strains=" + ",".join(n.strains)) + if n.embryos: + scope.append("embryos=" + ",".join(n.embryos)) + tag = f" [{'; '.join(scope)}]" if scope else "" + lines.append(f"[{n.id}] ({n.kind.value}){tag} {n.body}") + return "\n".join(lines) + + +def build_ask_messages(question: str, notes: list[Note]) -> list[dict]: + body = ( + "Notebook entries:\n" + _render_notes(notes) + f"\n\nQuestion: {question}\n\n" + "Answer using only these entries, citing note ids." + ) + return [{"role": "user", "content": body}] + + +async def answer_question(client, model: str, question: str, notes: list[Note]) -> dict: + """Force the ask tool and return its validated input dict. Short-circuits + (no API call) when there are no notes to ground on.""" + if not notes: + return { + "answer": "The notebook doesn't cover this yet.", + "points": [], + "suggested_next": [], + "coverage": "not_in_notebook", + } + resp = await client.messages.create( + model=model, + max_tokens=2048, + system=_SYSTEM, + tools=[ASK_TOOL], + tool_choice={"type": "tool", "name": ASK_TOOL["name"]}, + messages=build_ask_messages(question, notes), + ) + for block in resp.content: + if getattr(block, "type", None) == "tool_use": + return block.input + return {"answer": "", "points": [], "suggested_next": [], "coverage": "not_in_notebook"} +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_notebook_ask.py -q` +Expected: all pass + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/notebook_ask.py tests/test_notebook_ask.py +git commit -m "feat(notebook): answer_question — forced-tool grounded synthesis" +``` + +--- + +### Task 3: `POST /api/notebook/ask` + +**Files:** +- Modify: `gently/ui/web/routes/notebook.py` +- Test: `tests/test_notebook_api.py` (append) + +- [ ] **Step 1: Write the failing test** + +```python +# append to tests/test_notebook_api.py +class _AskBlock: + def __init__(self, inp): + self.type = "tool_use" + self.input = inp + + +class _AskResp: + def __init__(self, inp): + self.content = [_AskBlock(inp)] + self.stop_reason = "tool_use" + + +class _AskMessages: + def __init__(self, inp): + self._inp = inp + + async def create(self, **kwargs): + return _AskResp(self._inp) + + +class _AskClient: + def __init__(self, inp): + self.messages = _AskMessages(inp) + + +def _make_app_with_client(context_store, client): + from gently.ui.web.routes.notebook import create_router + + app = FastAPI() + + class _Server: + pass + + server = _Server() + server.context_store = context_store + server.claude_async = client + app.include_router(create_router(server)) + return app + + +class TestAsk: + def test_ask_returns_grounded_answer(self, file_context_store): + cs = _seed(file_context_store) + canned = { + "answer": "A ring formed.", + "points": [{"text": "ring", "note_ids": ["o1"]}], + "suggested_next": [], + "coverage": "covered", + } + client = TestClient(_make_app_with_client(cs, _AskClient(canned))) + resp = client.post("/api/notebook/ask", json={"question": "what happened?"}) + assert resp.status_code == 200 + assert resp.json()["coverage"] == "covered" + + def test_ask_no_store(self): + client = TestClient(_make_app(None)) + resp = client.post("/api/notebook/ask", json={"question": "x"}) + assert resp.json() == {"available": False} +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_notebook_api.py::TestAsk -v` +Expected: FAIL — 404/405 (route not defined) + +- [ ] **Step 3: Write minimal implementation** + +In `gently/ui/web/routes/notebook.py`, add `Body` to the fastapi import and add the route inside `create_router`, before `return router`: + +```python + @router.post("/api/notebook/ask") + async def ask( + question: str = Body(..., embed=True), + thread: str | None = Body(None, embed=True), + strain: str | None = Body(None, embed=True), + ): + nb = _nb() + if nb is None: + return {"available": False} + from gently.harness.memory.notebook_ask import answer_question, select_notes + from gently.settings import settings + + notes = select_notes(nb, thread=thread, strain=strain) + client = getattr(server, "claude_async", None) + if client is None: + import anthropic + + client = anthropic.AsyncAnthropic() + result = await answer_question(client, settings.models.main, question, notes) + result["available"] = True + result["note_ids"] = [n.id for n in notes] + return result +``` + +Update the import line at the top of the file: + +```python +from fastapi import APIRouter, Body, HTTPException +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_notebook_api.py -q` +Expected: all pass + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/routes/notebook.py tests/test_notebook_api.py +git commit -m "feat(notebook): POST /api/notebook/ask — grounded notebook Q&A" +``` + +--- + +## Self-Review +**Spec coverage (design §4):** structural retrieval (`select_notes`) → grounded synthesis (`answer_question`, forced tool, cited, "not_in_notebook" valid) → endpoint. ✓ No self-rated confidence (`coverage` is grounding, not correctness-confidence). ✓ Reuses `settings.models.main`, `anthropic.AsyncAnthropic`, the `verifier.py` forced-tool pattern. ✓ Client injected → unit-testable without real API. ✓ +**Deferred (correct):** embeddings/semantic recall; the "Ask" UI; proactive surfacing. +**Placeholder scan:** none — complete code + commands. ✓ +**Type consistency:** `select_notes`, `answer_question`, `ASK_TOOL`, `build_ask_messages` named consistently across tasks/tests; route uses `server.claude_async` (tests inject) with a real `AsyncAnthropic` fallback. ✓ diff --git a/docs/superpowers/plans/2026-06-27-temperature-interface.md b/docs/superpowers/plans/2026-06-27-temperature-interface.md new file mode 100644 index 00000000..025e8236 --- /dev/null +++ b/docs/superpowers/plans/2026-06-27-temperature-interface.md @@ -0,0 +1,842 @@ +# Temperature Interface Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Persist live temperature into each imaging session and chart it, so bursts/volumes are correlatable to temperature and the rise/fall trajectory is visible during the temperature-strain experiments. + +**Architecture:** A `FileStore`-backed append-only `temperature.jsonl` per session (mirroring `predictions.jsonl`); a `TemperatureSampler(Service)` in the agent process — modeled on `DeviceStateMonitor` — that, while a session is active, polls the device layer at 1 Hz, appends each reading, holds the latest in memory, and publishes a `TEMPERATURE_UPDATE` event that the viz server already forwards to the browser; acquisition code stamps the latest reading into volume/burst metadata; a FastAPI history route backfills the graph; a hand-rolled SVG component renders water-temp + stepped-setpoint as a card on the Devices tab. + +**Tech Stack:** Python 3 + asyncio, FastAPI, the project's `EventBus`, `FileStore` (file-based YAML/JSONL), vanilla-JS + hand-rolled SVG frontend (no build step), pytest with `asyncio_mode = "auto"`. + +## Global Constraints + +- **No new dependency** — chart is hand-rolled SVG (`createElementNS`), matching `experiment-overview.js`. No charting library. +- **Session-scoped capture only** — sampler persists/emits **only while a session is active**; no always-on facility daemon. +- **Sample schema** (one JSONL line): `{"t": , "water_c": , "setpoint_c": , "state": }`. +- **Append pattern**: reuse `FileStore._append_jsonl` (append mode, `json.dumps(..., default=str)`, trailing `\n`). Meta written via existing `_write_yaml` / `yaml.safe_dump`. +- **Defaults (flippable):** 1 Hz sampling; stamp the **latest sample** (no fresh blocking read) at acquisition. +- **Robustness:** a failed poll = a gap, logged, loop continues; a sampler error never crashes the session. Empty state, **never mock data**. +- **Tests:** `pytest`; `async def test_*` needs no decorator (auto mode); use the `file_store` fixture (`tests/conftest.py:38-45`, `FileStore(tmp_path/...)`). The frontend has **no JS unit harness** — verify it by running the app + Chrome DevTools MCP. +- **Env:** production runs pip + `requirements*.txt` (no `uv`); we add no deps, so nothing to declare. + +--- + +### Task 1: Temperature log store (FileStore methods) + +**Files:** +- Modify: `gently/core/file_store.py` (add two methods on `FileStore`; reuse module-level `_append_jsonl` at `:197-201` and `_read_jsonl` at `:204-215`, and `_session_dir`/`_require_session_dir` at `:255-269`) +- Test: `tests/test_temperature_store.py` (new) + +**Interfaces:** +- Produces: + - `FileStore.append_temperature_sample(self, session_id: str, sample: dict) -> None` — appends one line to `sessions/{folder}/temperature.jsonl`. + - `FileStore.read_temperature_log(self, session_id: str, since: str | None = None) -> list[dict]` — returns samples; if `since` (an ISO-UTC string) is given, only samples with `r["t"] >= since` (lexicographic compare is valid for fixed-format UTC ISO). + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_temperature_store.py +def _new_session(file_store): + return file_store.create_session(name="temp-test") # returns session_id + + +def test_append_and_read_roundtrip(file_store): + sid = _new_session(file_store) + file_store.append_temperature_sample( + sid, + {"t": "2026-06-27T10:00:00+00:00", "water_c": 28.0, "setpoint_c": 32.0, "state": "heating"}, + ) + file_store.append_temperature_sample( + sid, + {"t": "2026-06-27T10:00:01+00:00", "water_c": 28.3, "setpoint_c": 32.0, "state": "heating"}, + ) + rows = file_store.read_temperature_log(sid) + assert [r["water_c"] for r in rows] == [28.0, 28.3] + + +def test_read_since_filters(file_store): + sid = _new_session(file_store) + for i, t in enumerate( + ["2026-06-27T10:00:00+00:00", "2026-06-27T10:00:01+00:00", "2026-06-27T10:00:02+00:00"] + ): + file_store.append_temperature_sample( + sid, {"t": t, "water_c": 28.0 + i, "setpoint_c": 32.0, "state": "heating"} + ) + rows = file_store.read_temperature_log(sid, since="2026-06-27T10:00:01+00:00") + assert [r["water_c"] for r in rows] == [29.0, 30.0] + + +def test_read_unknown_session_is_empty(file_store): + assert file_store.read_temperature_log("does-not-exist") == [] +``` + +> NOTE for implementer: confirm the exact session-creation API on `FileStore` (search for `def create_session`). If its signature differs, adjust `_new_session` accordingly — the rest of the test is unaffected. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_temperature_store.py -v` +Expected: FAIL — `AttributeError: 'FileStore' object has no attribute 'append_temperature_sample'` + +- [ ] **Step 3: Write minimal implementation** + +Add to the `FileStore` class body in `gently/core/file_store.py` (near the other per-session jsonl helpers like `add_prediction`): + +```python +def append_temperature_sample(self, session_id: str, sample: dict) -> None: + """Append one temperature reading to the session's temperature.jsonl.""" + sd = self._require_session_dir(session_id) + _append_jsonl(sd / "temperature.jsonl", sample) + + +def read_temperature_log(self, session_id: str, since: str | None = None) -> list[dict]: + """Return temperature samples for a session, optionally filtered to t >= since (ISO-UTC string).""" + sd = self._session_dir(session_id) + if sd is None: + return [] + rows = _read_jsonl(sd / "temperature.jsonl") + if since is not None: + rows = [r for r in rows if str(r.get("t", "")) >= since] + return rows +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_temperature_store.py -v` +Expected: PASS (3 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/core/file_store.py tests/test_temperature_store.py +git commit -m "feat(temperature): session-scoped temperature.jsonl store on FileStore" +``` + +--- + +### Task 2: `TEMPERATURE_UPDATE` event type + +**Files:** +- Modify: `gently/core/event_bus.py` (add enum member near `:86`; add to `_NO_HISTORY_TYPES` near `:187`) +- Test: `tests/test_temperature_event.py` (new) + +**Interfaces:** +- Produces: `EventType.TEMPERATURE_UPDATE` (a new enum member). High-volume → excluded from event history. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_temperature_event.py +from gently.core.event_bus import EventType, EventBus + + +def test_temperature_update_event_exists(): + assert EventType.TEMPERATURE_UPDATE.value == "TEMPERATURE_UPDATE" + + +def test_temperature_update_publishes_to_subscriber(): + bus = EventBus() + seen = [] + bus.subscribe(EventType.TEMPERATURE_UPDATE, lambda e: seen.append(e.data)) + bus.publish(event_type=EventType.TEMPERATURE_UPDATE, data={"x": 1}, source="t") + assert seen == [{"x": 1}] +``` + +> NOTE: confirm `EventBus.subscribe` signature/usage from an existing test (`tests/` has event-bus usage); adjust the subscribe call if the project's API differs (e.g. `subscribe(event_type, handler)` vs `subscribe(handler, event_type)`). + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_temperature_event.py -v` +Expected: FAIL — `AttributeError: TEMPERATURE_UPDATE` + +- [ ] **Step 3: Write minimal implementation** + +In `gently/core/event_bus.py`, add the member alongside the other `EventType` values, **matching the existing `auto()` style** (`DEVICE_STATE_UPDATE = auto()` at `:86`). The wire protocol serializes `event.event_type.name` (`Event.to_dict` at event_bus.py:232; server.py:377/419), so the browser receives the string `"TEMPERATURE_UPDATE"` regardless — which is what the frontend (Task 7) subscribes to. The test must assert `.name == "TEMPERATURE_UPDATE"` (NOT `.value`, which is an `auto()` int): + +```python + TEMPERATURE_UPDATE = auto() # high-volume telemetry from the temperature controller +``` + +And add it to the high-volume set so it is not retained in history (next to `DEVICE_STATE_UPDATE` in `_NO_HISTORY_TYPES` near `:187`): + +```python +(EventType.TEMPERATURE_UPDATE,) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_temperature_event.py -v` +Expected: PASS (2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/core/event_bus.py tests/test_temperature_event.py +git commit -m "feat(temperature): add TEMPERATURE_UPDATE event type" +``` + +--- + +### Task 3: TemperatureSampler service + +**Files:** +- Create: `gently/app/temperature_sampler.py` +- Test: `tests/test_temperature_sampler.py` (new) +- Reference (template, do not modify): `gently/app/device_state_monitor.py`, `gently/core/service.py:63-172` + +**Interfaces:** +- Consumes: `EventType.TEMPERATURE_UPDATE` (Task 2); `FileStore.append_temperature_sample` (Task 1); a microscope client exposing `async get_temperature() -> dict` returning `{"success": bool, "temperature_c": float, "setpoint_c": float, "state": str, ...}` (`gently/hardware/dispim/client.py:837`). +- Produces: + - `TemperatureSampler(Service)` with `__init__(self, microscope, store, session_id_getter, interval_sec=1.0)`. + - `async on_start(self)` / `async on_stop(self)` (background asyncio loop, like `DeviceStateMonitor`). + - `async _tick(self, bus) -> None` — one poll/append/emit cycle (the unit under test). + - attribute `self.latest: dict | None` — most recent sample (for the acquisition stamp). + - module function `temperature_stamp(latest: dict | None) -> dict | None` — `{"water_c","setpoint_c","state","sampled_at"}` or `None`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_temperature_sampler.py +import asyncio +from gently.core.event_bus import EventBus, EventType +from gently.app.temperature_sampler import TemperatureSampler, temperature_stamp + + +class FakeScope: + def __init__(self, resp): + self.resp = resp + self.calls = 0 + + async def get_temperature(self): + self.calls += 1 + if isinstance(self.resp, Exception): + raise self.resp + return self.resp + + +def _capture(bus): + seen = [] + bus.subscribe(EventType.TEMPERATURE_UPDATE, lambda e: seen.append(e.data)) + return seen + + +async def test_tick_appends_emits_and_sets_latest(file_store): + sid = file_store.create_session(name="s") + scope = FakeScope( + {"success": True, "temperature_c": 28.4, "setpoint_c": 32.0, "state": "heating"} + ) + bus = EventBus() + seen = _capture(bus) + s = TemperatureSampler(scope, file_store, lambda: sid) + await s._tick(bus) + rows = file_store.read_temperature_log(sid) + assert len(rows) == 1 and rows[0]["water_c"] == 28.4 + assert s.latest["water_c"] == 28.4 + assert seen and seen[0]["sample"]["water_c"] == 28.4 and seen[0]["session_id"] == sid + + +async def test_tick_no_active_session_is_noop(file_store): + scope = FakeScope({"success": True, "temperature_c": 1.0, "setpoint_c": 2.0, "state": "x"}) + bus = EventBus() + seen = _capture(bus) + s = TemperatureSampler(scope, file_store, lambda: None) + await s._tick(bus) + assert scope.calls == 0 and s.latest is None and seen == [] + + +async def test_tick_poll_failure_is_a_gap_not_a_crash(file_store): + sid = file_store.create_session(name="s") + scope = FakeScope(RuntimeError("device down")) + bus = EventBus() + s = TemperatureSampler(scope, file_store, lambda: sid) + # _run swallows; _tick raises — assert the loop-level guard swallows by calling _run-style guard: + try: + await s._tick(bus) + except RuntimeError: + pass # _tick may raise; the loop in _run catches it (see test below) + assert file_store.read_temperature_log(sid) == [] + + +def test_temperature_stamp_shapes(): + assert temperature_stamp(None) is None + assert temperature_stamp( + {"t": "2026-06-27T10:00:00+00:00", "water_c": 28.4, "setpoint_c": 32.0, "state": "heating"} + ) == { + "water_c": 28.4, + "setpoint_c": 32.0, + "state": "heating", + "sampled_at": "2026-06-27T10:00:00+00:00", + } +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_temperature_sampler.py -v` +Expected: FAIL — `ModuleNotFoundError: gently.app.temperature_sampler` + +- [ ] **Step 3: Write minimal implementation** + +```python +# gently/app/temperature_sampler.py +"""Session-scoped temperature sampler — polls the device layer, persists, emits. + +Modeled on gently/app/device_state_monitor.py. While a session is active it polls +the microscope's temperature at a fixed cadence, appends each reading to the +session's temperature.jsonl, holds the latest reading (for acquisition stamping), +and publishes TEMPERATURE_UPDATE for the live graph. A failed poll is a gap, not a +crash; with no active session the loop idles. +""" + +from __future__ import annotations + +import asyncio +import logging +from datetime import datetime, timezone + +from gently.core.service import Service +from gently.core.event_bus import get_event_bus, EventType + +logger = logging.getLogger(__name__) + + +def _now_iso() -> str: + return datetime.now(timezone.utc).isoformat() + + +def temperature_stamp(latest: dict | None) -> dict | None: + """Build a temperature meta block from a latest sample, or None if unavailable.""" + if not latest: + return None + return { + "water_c": latest.get("water_c"), + "setpoint_c": latest.get("setpoint_c"), + "state": latest.get("state"), + "sampled_at": latest.get("t"), + } + + +class TemperatureSampler(Service): + def __init__(self, microscope, store, session_id_getter, interval_sec: float = 1.0): + super().__init__(name="temperature-sampler", service_type="monitor") + self._microscope = microscope + self._store = store + self._session_id_getter = session_id_getter + self._interval = interval_sec + self._task: asyncio.Task | None = None + self.latest: dict | None = None + + async def on_start(self) -> None: + self._task = asyncio.create_task(self._run(), name="temperature-sampler-loop") + + async def on_stop(self) -> None: + if self._task is not None: + self._task.cancel() + try: + await self._task + except asyncio.CancelledError: + pass + self._task = None + + async def _run(self) -> None: + bus = get_event_bus() + while True: + try: + await self._tick(bus) + except asyncio.CancelledError: + raise + except Exception as exc: # a gap, never a crash + logger.warning("temperature sampler tick failed: %s", exc) + await asyncio.sleep(self._interval) + + async def _tick(self, bus) -> None: + session_id = self._session_id_getter() + if not session_id: + return + resp = await self._microscope.get_temperature() + if not resp or not resp.get("success", True): + return + water = resp.get("temperature_c") + if water is None: + return + sample = { + "t": _now_iso(), + "water_c": water, + "setpoint_c": resp.get("setpoint_c"), + "state": resp.get("state"), + } + self._store.append_temperature_sample(session_id, sample) + self.latest = sample + bus.publish( + event_type=EventType.TEMPERATURE_UPDATE, + data={"session_id": session_id, "sample": sample}, + source="temperature-sampler", + ) +``` + +> Note the failure test: `_tick` propagates the poll exception, and `_run`'s `except Exception` swallows it. The test asserts the gap (no rows). If you prefer, also add a direct `_run`-guard test that starts the loop briefly and asserts it does not raise — optional. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_temperature_sampler.py -v` +Expected: PASS (4 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/app/temperature_sampler.py tests/test_temperature_sampler.py +git commit -m "feat(temperature): TemperatureSampler service (poll/persist/emit @1Hz)" +``` + +--- + +### Task 4: Wire the sampler into the agent lifecycle + +**Files:** +- Modify: `gently/app/agent.py` — init attribute (near `:212`), construct+start (near the `DeviceStateMonitor` block `:818-827` inside `start_viz_server`), stop (near `:850-855`). +- Test: `tests/test_temperature_sampler_wiring.py` (new) + +**Interfaces:** +- Consumes: `TemperatureSampler` (Task 3); the agent's microscope client (`self.microscope`), its `FileStore`, and `self.session_id`. +- Produces: `agent.temperature_sampler: TemperatureSampler | None` (the live instance, read by the acquisition stamp in Task 6). + +> IMPLEMENTER: confirm the agent's FileStore attribute name before writing the construction line. Search `gently/app/agent.py` for the `FileStore` it uses (likely `self.store`). Use that exact attribute. If the agent reaches the store indirectly, pass whatever object exposes `append_temperature_sample`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_temperature_sampler_wiring.py +from gently.app.temperature_sampler import TemperatureSampler + + +def test_agent_initializes_temperature_sampler_attribute(): + # The attribute must exist (None until start_viz_server runs with a microscope). + import gently.app.agent as agent_mod + + src = agent_mod.__file__ + text = open(src, encoding="utf-8").read() + assert "temperature_sampler" in text + assert "TemperatureSampler(" in text +``` + +> This is a light wiring guard (a full agent boot is an integration concern). It fails until the wiring lines exist. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_temperature_sampler_wiring.py -v` +Expected: FAIL — assertion error ("temperature_sampler" not in source) + +- [ ] **Step 3: Write minimal implementation** + +Near `:212` (with the other monitor attrs, e.g. `self.device_state_monitor = None`): + +```python + self.temperature_sampler = None +``` + +Inside `start_viz_server`, right after the `DeviceStateMonitor` start block (`:818-827`): + +```python +if self.microscope is not None and self.temperature_sampler is None: + from .temperature_sampler import TemperatureSampler + + self.temperature_sampler = TemperatureSampler( + self.microscope, self.store, lambda: self.session_id + ) + await self.temperature_sampler.start() +``` + +In the symmetric shutdown path (`:850-855`, where `device_state_monitor.stop()` is awaited): + +```python + if self.temperature_sampler is not None: + await self.temperature_sampler.stop() + self.temperature_sampler = None +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_temperature_sampler_wiring.py -v` +Expected: PASS (1 passed) + +Then full suite sanity: `pytest -q` — Expected: no new failures. + +- [ ] **Step 5: Commit** + +```bash +git add gently/app/agent.py tests/test_temperature_sampler_wiring.py +git commit -m "feat(temperature): start/stop TemperatureSampler with the agent" +``` + +--- + +### Task 5: History API route + +**Files:** +- Create: `gently/ui/web/routes/temperature.py` +- Modify: `gently/ui/web/routes/__init__.py` (import + add to the factories tuple in `register_all_routes`) +- Test: `tests/test_temperature_route.py` (new) +- Reference (template): `gently/ui/web/routes/experiments.py:1-66`, test template `tests/test_data_catalog.py:69-114` + +**Interfaces:** +- Consumes: `server.gently_store` (a `FileStore`) with `list_sessions()`, `_session_dir(id)`, and `read_temperature_log(id, since=)` (Task 1). +- Produces: `GET /api/temperature/{session_id}/history?since=` → `{"session_id": str, "samples": list[dict]}`; `session_id="current"` resolves to newest session. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_temperature_route.py +from unittest.mock import MagicMock +from pathlib import Path +from fastapi import FastAPI +from fastapi.testclient import TestClient +from gently.ui.web.routes.temperature import create_router + + +def _server(samples, sessions=(("sess-1", True),)): + store = MagicMock() + store.list_sessions.return_value = [{"session_id": sid} for sid, _ in sessions] + store._session_dir.side_effect = lambda sid: ( + Path("/x") if any(sid == s for s, _ in sessions) else None + ) + store.read_temperature_log.return_value = samples + srv = MagicMock() + srv.gently_store = store + return srv, store + + +def _client(server): + app = FastAPI() + app.include_router(create_router(server)) + return TestClient(app) + + +def test_history_returns_samples(): + srv, store = _server( + [ + { + "t": "2026-06-27T10:00:00+00:00", + "water_c": 28.0, + "setpoint_c": 32.0, + "state": "heating", + } + ] + ) + r = _client(srv).get("/api/temperature/sess-1/history") + assert r.status_code == 200 + body = r.json() + assert body["session_id"] == "sess-1" + assert body["samples"][0]["water_c"] == 28.0 + + +def test_history_passes_since_through(): + srv, store = _server([]) + _client(srv).get("/api/temperature/sess-1/history?since=2026-06-27T10:00:01+00:00") + store.read_temperature_log.assert_called_with("sess-1", since="2026-06-27T10:00:01+00:00") + + +def test_history_current_resolves_newest(): + srv, store = _server([], sessions=(("newest", True),)) + r = _client(srv).get("/api/temperature/current/history") + assert r.status_code == 200 + assert r.json()["session_id"] == "newest" + + +def test_history_unknown_session_404(): + srv, store = _server([], sessions=(("sess-1", True),)) + # _session_dir returns None for unknown -> 404 + store._session_dir.side_effect = lambda sid: None + r = _client(srv).get("/api/temperature/ghost/history") + assert r.status_code == 404 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_temperature_route.py -v` +Expected: FAIL — `ModuleNotFoundError: gently.ui.web.routes.temperature` + +- [ ] **Step 3: Write minimal implementation** + +```python +# gently/ui/web/routes/temperature.py +"""Read-only temperature history for the live graph (backfill on mount/reload). + +Live updates ride the TEMPERATURE_UPDATE event channel; this route is backfill only. +Mirrors routes/experiments.py session resolution. +""" + +from fastapi import APIRouter, HTTPException + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _resolve_session(session_id: str): + store = getattr(server, "gently_store", None) + if store is None: + raise HTTPException(status_code=503, detail="FileStore not configured on viz server") + if session_id == "current": + sessions = store.list_sessions() + if not sessions: + raise HTTPException(status_code=404, detail="No sessions in store") + session_id = sessions[0].get("session_id") + if store._session_dir(session_id) is None: + raise HTTPException(status_code=404, detail=f"Session not found: {session_id}") + return session_id + + @router.get("/api/temperature/{session_id}/history") + async def get_history(session_id: str, since: str | None = None): + real_id = _resolve_session(session_id) + store = server.gently_store + samples = store.read_temperature_log(real_id, since=since) + return {"session_id": real_id, "samples": samples} + + return router +``` + +Register it in `gently/ui/web/routes/__init__.py` — add the import and append `create_router` to the factories iterated by `register_all_routes` (follow the existing pattern exactly; alias to avoid name clashes, e.g. `from .temperature import create_router as create_temperature_router`). + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_temperature_route.py -v` +Expected: PASS (4 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/routes/temperature.py gently/ui/web/routes/__init__.py tests/test_temperature_route.py +git commit -m "feat(temperature): GET /api/temperature/{session}/history backfill route" +``` + +--- + +### Task 6: Acquisition temperature stamp (burst + volume) + +**Files:** +- Modify: `gently/app/orchestration/exclusive.py` — `_persist_burst_to_disk` (per-frame `metadata` at `:387-407`, `burst.yaml` manifest dict at `:423-442`); the `BurstAcquisition`/`ExclusiveAcquisition` construction to receive a temperature provider. +- Modify: the volume acquisition call site that builds `metadata` for `FileStore.put_volume`/`register_volume` (locate the caller in `gently/app/orchestration/timelapse.py` / `gently/app/tools/acquisition_tools.py`). +- Test: `tests/test_temperature_stamp.py` (new) — covers the pure helper + the volume metadata channel. +- Consumes: `temperature_stamp` and `agent.temperature_sampler.latest` (Tasks 3–4). + +**Interfaces:** +- Produces: a `temperature` block under `metadata` for volumes (`meta["metadata"]["temperature"]`) and under both per-frame `metadata` and the `burst.yaml` manifest top-level for bursts. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_temperature_stamp.py +import numpy as np +from gently.app.temperature_sampler import temperature_stamp + + +def test_stamp_none_when_no_reading(): + assert temperature_stamp(None) is None + + +def test_volume_metadata_carries_temperature(file_store): + sid = file_store.create_session(name="s") + emb = file_store.create_embryo(sid, position={"x": 0, "y": 0, "z": 0}) # confirm signature + stamp = temperature_stamp( + {"t": "2026-06-27T10:00:00+00:00", "water_c": 28.4, "setpoint_c": 32.0, "state": "heating"} + ) + vol = np.zeros((2, 4, 4), dtype="uint16") + file_store.put_volume(sid, emb, timepoint=0, volume=vol, metadata={"temperature": stamp}) + meta = file_store.get_volume_meta(sid, emb, 0) # confirm accessor name + assert meta["metadata"]["temperature"]["water_c"] == 28.4 +``` + +> IMPLEMENTER: confirm `create_embryo` and the volume-meta accessor (`get_volume_meta` or read the `.meta.yaml` directly via `get_volume_path(...).with_suffix` — adjust to the real API). The assertion target — `metadata["temperature"]` round-tripping into `t0000.meta.yaml` — is the contract; `put_volume` already nests the passed `metadata`, so this passes once the helper exists and the accessor is right. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_temperature_stamp.py -v` +Expected: FAIL — initially on import/accessor; fix the accessor name from the real API, then it exercises the channel. + +- [ ] **Step 3: Write minimal implementation** + +**Volume path** — at the acquisition call site that calls `put_volume`/`register_volume`, fold the stamp into the existing `metadata` dict: + +```python +from gently.app.temperature_sampler import temperature_stamp + +# ... where `agent` (or self) holds the sampler and `metadata` is being built: +stamp = temperature_stamp(getattr(getattr(agent, "temperature_sampler", None), "latest", None)) +if stamp is not None: + metadata["temperature"] = stamp +``` + +**Burst path** — `gently/app/orchestration/exclusive.py`: +1. Add a constructor param to the acquisition class that persists bursts: `temperature_provider=None` (a zero-arg callable returning the latest sample dict, or `None`), stored as `self._temperature_provider`. +2. In `_persist_burst_to_disk`, compute once: + +```python +from gently.app.temperature_sampler import temperature_stamp + +_temp = temperature_stamp(self._temperature_provider() if self._temperature_provider else None) +``` + +3. Inject into the per-frame `metadata` dict (`:387-407`): add `"temperature": _temp` (only when not None — or always; `None` is acceptable YAML). +4. Inject into the `burst.yaml` manifest dict (`:423-442`): add a top-level `"temperature": _temp`. +5. Where this acquisition class is constructed (the orchestrator that owns bursts — `gently/app/orchestration/timelapse.py`), pass `temperature_provider=lambda: agent.temperature_sampler.latest if agent.temperature_sampler else None` (use the orchestrator's existing agent/store handle; confirm the attribute). + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_temperature_stamp.py -v` +Expected: PASS + +- [ ] **Step 5: Verify burst wiring by inspection + targeted run** + +The burst persist writes real TIFFs, so it is verified by inspection (the `_temp` block is added in both dict sites) plus, during end-to-end verification (after Task 7), trigger one burst against the mock device and confirm `burst.yaml` and a frame `.meta.yaml` contain a `temperature` block. Note in the commit if the volume call site could not be located and was deferred — do not silently skip it. + +- [ ] **Step 6: Commit** + +```bash +git add gently/app/orchestration/exclusive.py gently/app/orchestration/timelapse.py tests/test_temperature_stamp.py +git commit -m "feat(temperature): stamp latest reading into burst + volume metadata" +``` + +--- + +### Task 7: Frontend — temperature graph component + Devices card + +**Files:** +- Create: `gently/ui/web/static/js/temperature-graph.js` +- Modify: `gently/ui/web/templates/index.html` — add a chart container inside `#devices-content` (the existing small readout is at `:452-465`; mount the chart as a section under the Details/Map view). +- Modify: `gently/ui/web/static/js/devices.js` — initialize the chart in `init()` (`:1556`) and subscribe it to `TEMPERATURE_UPDATE` (next to the `DEVICE_STATE_UPDATE` subscription at `:1561`). +- Reference (SVG style): `gently/ui/web/static/js/experiment-overview.js`; (event API) `gently/ui/web/static/js/event-bus.js:9-48` (`ClientEventBus.on(type, handler)`). + +**No JS unit harness exists** — this task is verified by running the app + Chrome DevTools MCP (see Step 4), consistent with the repo and the "UI audit before done" practice. + +- [ ] **Step 1: Build the component** + +Create `temperature-graph.js` exposing a small object/module: + +```javascript +// gently/ui/web/static/js/temperature-graph.js +// Hand-rolled SVG line chart: water-temp trace + stepped setpoint line. +// No dependency. Backfills from /api/temperature/{session}/history, then appends +// from TEMPERATURE_UPDATE events. Calm empty state, never mock data. +const TemperatureGraph = (() => { + const SVGNS = "http://www.w3.org/2000/svg"; + const MAX_POINTS = 600; // rolling ~10 min @ 1 Hz + let _root = null, _samples = [], _session = "current"; + + function init(container, sessionId) { + _root = container; _session = sessionId || "current"; _samples = []; + backfill(); + ClientEventBus.on("TEMPERATURE_UPDATE", onEvent); + } + + async function backfill() { + try { + const r = await fetch(`/api/temperature/${_session}/history`); + if (!r.ok) { renderEmpty(); return; } + const body = await r.json(); + _session = body.session_id || _session; + _samples = (body.samples || []).slice(-MAX_POINTS); + render(); + } catch (e) { renderEmpty(); } + } + + function onEvent(data) { + if (!data || !data.sample) return; + _samples.push(data.sample); + if (_samples.length > MAX_POINTS) _samples.shift(); + render(); + } + + function renderEmpty() { + _root.innerHTML = '
No temperature data yet
'; + } + + function render() { + if (!_samples.length) { renderEmpty(); return; } + const W = _root.clientWidth || 480, H = 160, pad = 24; + const xs = _samples.map((_, i) => i); + const ws = _samples.map(s => s.water_c).filter(v => v != null); + const sps = _samples.map(s => s.setpoint_c).filter(v => v != null); + const lo = Math.min(...ws, ...sps) - 1, hi = Math.max(...ws, ...sps) + 1; + const sx = i => pad + (i / Math.max(1, xs.length - 1)) * (W - 2 * pad); + const sy = v => H - pad - ((v - lo) / Math.max(0.001, hi - lo)) * (H - 2 * pad); + + const svg = document.createElementNS(SVGNS, "svg"); + svg.setAttribute("viewBox", `0 0 ${W} ${H}`); svg.setAttribute("width", "100%"); + + const line = (pts, cls) => { + const p = document.createElementNS(SVGNS, "polyline"); + p.setAttribute("points", pts); p.setAttribute("class", cls); + p.setAttribute("fill", "none"); svg.appendChild(p); + }; + line(_samples.map((s, i) => s.water_c != null ? `${sx(i)},${sy(s.water_c)}` : "").filter(Boolean).join(" "), "temp-water"); + // Stepped setpoint: carry previous y until it changes. + let sp = []; _samples.forEach((s, i) => { if (s.setpoint_c != null) sp.push(`${sx(i)},${sy(s.setpoint_c)}`); }); + line(sp.join(" "), "temp-setpoint"); + + const last = _samples[_samples.length - 1]; + const readout = document.createElement("div"); + readout.className = "temp-graph-readout"; + readout.textContent = `${last.water_c?.toFixed?.(1) ?? "—"} °C → ${last.setpoint_c?.toFixed?.(1) ?? "—"} °C (${last.state ?? ""})`; + + _root.innerHTML = ""; _root.appendChild(readout); _root.appendChild(svg); + } + + function dispose() { ClientEventBus.off("TEMPERATURE_UPDATE", onEvent); } + return { init, dispose, _render: render, _samples: () => _samples }; +})(); +window.TemperatureGraph = TemperatureGraph; +``` + +Add minimal CSS (in the devices stylesheet) for `.temp-water` (stroke: water color), `.temp-setpoint` (dashed stroke), `.temp-graph-empty` (muted), matching existing palette. + +- [ ] **Step 2: Mount it** + +In `templates/index.html`, add inside `#devices-content` (under the map/details view): + +```html +
+``` + +Load the script (next to the other `static/js/*.js` includes for the devices tab). + +In `devices.js` `init()` (`:1556`), after existing setup: + +```javascript + const tg = document.getElementById('devices-temp-graph'); + if (tg && window.TemperatureGraph) TemperatureGraph.init(tg, 'current'); +``` + +(No extra subscription needed in `devices.js` — the component self-subscribes. Optionally also route the existing small readout off the new event.) + +- [ ] **Step 3: Add the script tag** + +Add `` in `index.html` alongside the other component scripts, **before** `devices.js` loads (so `window.TemperatureGraph` exists when `init()` runs). + +- [ ] **Step 4: Verify live in the app (Chrome DevTools MCP)** + +Use the `run` skill to launch the app with the mock temperature backend and an active session. Then with Chrome DevTools MCP: +- navigate to the Devices tab, take a snapshot/screenshot; +- confirm the empty state shows when no samples, then the water trace + stepped setpoint render and update live as the sampler emits; +- run the UI audit (alignment/spacing/overflow/contrast) per the "UI audit before done" practice and fix any flaws; +- trigger one burst and confirm (Task 6) that `burst.yaml` + a frame `.meta.yaml` carry a `temperature` block. + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/static/js/temperature-graph.js gently/ui/web/templates/index.html gently/ui/web/static/js/devices.js +git commit -m "feat(temperature): live SVG temperature graph on the Devices tab" +``` + +--- + +## Self-Review + +**Spec coverage:** +- Persistence / `temperature.jsonl` → Task 1. ✓ +- Sampler (poll @1 Hz, session-gated, latest-in-memory, SSE) → Tasks 2–4. ✓ +- Per-acquisition stamp (burst + volume) → Task 6. ✓ +- History API (`current` resolution, `since`) → Task 5. ✓ +- Reusable SVG graph on Devices card (water trace + stepped setpoint + readout, backfill + live, empty state) → Task 7. ✓ +- Error/empty handling (gap-not-crash, no-device idle, empty state) → Tasks 3 & 7. ✓ +- Out-of-scope (setpoint control, choreography, always-on) → not implemented, by design. ✓ + +**Open verification items folded into tasks (not placeholders):** session-creation API (Task 1/3/6), `EventBus.subscribe` shape (Task 2), the agent's FileStore attribute (Task 4), the volume-meta accessor + `create_embryo` signature (Task 6), the burst-acquisition construction site (Task 6). Each is an explicit "confirm from the real API" instruction with a concrete fallback, not a TODO. + +**Type consistency:** `temperature_stamp` returns `{water_c, setpoint_c, state, sampled_at}` everywhere; sample lines are `{t, water_c, setpoint_c, state}`; event payload is `{session_id, sample}`; history response is `{session_id, samples}`. Consistent across Tasks 1/3/5/6/7. diff --git a/docs/superpowers/plans/2026-06-28-manual-mode-live-view.md b/docs/superpowers/plans/2026-06-28-manual-mode-live-view.md new file mode 100644 index 00000000..abdd9fce --- /dev/null +++ b/docs/superpowers/plans/2026-06-28-manual-mode-live-view.md @@ -0,0 +1,845 @@ +# Manual Mode — SPIM Live View (B1) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A Manual view in the Devices tab with a continuous SPIM single-slice brightfield live view (galvo/piezo/exposure controlled live), brightfield illumination, temperature, and by-hand burst/volume triggers — the hand-driven surface for next week's temperature-strain experiments. + +**Architecture:** A device-layer lightsheet live streamer (MMCore continuous sequence acquisition + peek-latest, parked galvo/piezo), bridged to the browser by a `LightSheetStreamMonitor` (EventBus `LIGHTSHEET_FRAME`) over the existing base64-JPEG/SSE→WS transport; `require_control` FastAPI proxy routes; a Manual view that mirrors the bottom-camera panel. FPS is measured on the rig; a binary transport path is a conditional follow-up. + +**Tech Stack:** Python asyncio + aiohttp (device layer), FastAPI (viz proxy), `pymmcore.CMMCore`, the project EventBus, vanilla-JS + canvas/SVG frontend (no build step), pytest (`asyncio_mode=auto`). + +## Global Constraints + +- **No new dependency.** Reuse `_encode_frame_for_stream` (OpenCV already present), the bottom-camera streamer pattern, A's portable temperature graph. +- **Single SPIM camera** today: `self.devices.get("camera")` (`HamCam1`). No side-A/B selector in B1 (deferred to B2). +- **Live = continuous sequence acquisition**, never a snap loop: `core.startContinuousSequenceAcquisition(0)` → peek `core.getLastImage()` (NEVER `popNextImage` — don't drain) → `stopSequenceAcquisition()` on exit. The core handle is `self.system.core` in the device layer; unwrap rpyc frames with `_safe_obtain`. +- **Park** before/under live: `piezo.setPosition(z)`, `scanner.sa_offset_y.setPosition(deg)` (or `scanner.set_y_offset(deg)`), `scanner.set_spim_state("Idle")`, `piezo.set_spim_state("Idle")`. galvo/piezo updates apply live (no restart); exposure change → stop→`setExposure`→restart. +- **Brightfield safety:** laser forced off in manual live (`setConfig(laser_group, "ALL OFF")` / `set_laser_power(...,0)`). +- **Concurrency:** the streamer honors `self._state_pause_counter > 0` (heavy plan owns MMCore → back off / stop sequence). Only one live stream at a time. +- **Lightsheet stream resolution/quality:** its own config — default `_ls_target_max_dim = 512`, `_ls_jpeg_quality = 70` (higher than the bottom-camera 360/55 thumbnail, for focus). +- **Transport stays JSON/`send_text`** (no binary path exists); a binary hop is Task 8, conditional on the FPS measurement. +- **Auth:** browser-facing writes are FastAPI proxy routes guarded by `Depends(require_control)`; device-layer aiohttp routes have no auth, so the browser must go through the proxy. +- **Tests:** `pytest`; `file_store`-style fixtures; FastAPI `TestClient` + mock client for routes; a fake core for the streamer. Frontend has no JS unit harness → `node --check` + Chrome-MCP harness + UI audit. Much of B1 needs the real rig; off-rig we cover streamer (fake core), routes, client, frontend harness, and defer live/FPS verification. + +--- + +### Task 1: `LIGHTSHEET_FRAME` event type + frontend exclusion + +**Files:** +- Modify: `gently/core/event_bus.py` (enum near `:88`; `_NO_HISTORY_TYPES` near `:186-195`) +- Modify: `gently/ui/web/static/js/websocket.js` (exclusion guard `:104-107`) +- Test: `tests/test_lightsheet_event.py` + +**Interfaces:** +- Produces: `EventType.LIGHTSHEET_FRAME` (declared with `auto()`, matching `BOTTOM_CAMERA_FRAME`), in `_NO_HISTORY_TYPES`. Wire serialization uses `.name` (so the browser receives `"LIGHTSHEET_FRAME"`). + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_lightsheet_event.py +from gently.core.event_bus import EventType, EventBus, _NO_HISTORY_TYPES + + +def test_lightsheet_frame_event_exists(): + assert EventType.LIGHTSHEET_FRAME.name == "LIGHTSHEET_FRAME" + + +def test_lightsheet_frame_excluded_from_history(): + assert EventType.LIGHTSHEET_FRAME in _NO_HISTORY_TYPES + + +def test_lightsheet_frame_publishes_to_subscriber(): + bus = EventBus() + seen = [] + bus.subscribe(EventType.LIGHTSHEET_FRAME, lambda e: seen.append(e.data)) + bus.publish(event_type=EventType.LIGHTSHEET_FRAME, data={"jpeg_b64": "x"}, source="t") + assert seen == [{"jpeg_b64": "x"}] +``` + +> Confirm `EventBus.subscribe` signature / `_NO_HISTORY_TYPES` exportability against the real file; adapt the import/subscribe if needed, keep the assertions. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `pytest tests/test_lightsheet_event.py -v` +Expected: FAIL — `AttributeError: LIGHTSHEET_FRAME` + +- [ ] **Step 3: Write minimal implementation** + +In `gently/core/event_bus.py`, add the member next to `BOTTOM_CAMERA_FRAME` (`:88`): +```python + LIGHTSHEET_FRAME = auto() # Live JPEG frame from the SPIM lightsheet live stream +``` +Add to `_NO_HISTORY_TYPES` (next to `BOTTOM_CAMERA_FRAME`): +```python +(EventType.LIGHTSHEET_FRAME,) # high-volume live frames — keep out of history +``` +In `gently/ui/web/static/js/websocket.js`, extend the exclusion guard (`:104-107`) so the frame skips the Events table but still reaches `ClientEventBus.emit`: +```javascript + if (msg.event_type !== 'DEVICE_STATE_UPDATE' && + msg.event_type !== 'BOTTOM_CAMERA_FRAME' && + msg.event_type !== 'TEMPERATURE_UPDATE' && + msg.event_type !== 'LIGHTSHEET_FRAME') { +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `pytest tests/test_lightsheet_event.py -v` (3 passed) and `node --check gently/ui/web/static/js/websocket.js` (exit 0) + +- [ ] **Step 5: Commit** + +```bash +git add gently/core/event_bus.py gently/ui/web/static/js/websocket.js tests/test_lightsheet_event.py +git commit -m "feat(manual-mode): add LIGHTSHEET_FRAME event type + frontend exclusion" +``` + +--- + +### Task 2: Device-layer lightsheet live streamer (continuous sequence acquisition) + +**Files:** +- Modify: `gently/hardware/dispim/device_layer.py` — add config attrs (near `:160-169`), `_park_lightsheet_sync`, `_grab_lightsheet_frame_sync`, `_lightsheet_streamer`, `_broadcast_lightsheet`, `handle_lightsheet_stream`, `handle_lightsheet_params`; register two routes (near `:2806`). +- Test: `tests/test_lightsheet_streamer.py` +- Reference (mirror): `_bottom_camera_streamer`/`_broadcast_camera`/`handle_bottom_camera_stream`/`_encode_frame_for_stream` (same file); sequence-acq calls in `devices/acquisition.py:186-254`. + +**Interfaces:** +- Consumes: `self.system.core` (`pymmcore.CMMCore`), `self.devices["camera"]` / `["scanner"]` / `["piezo"]`, `self._state_pause_counter`, `self._encode_frame_for_stream` (reused verbatim), `_safe_obtain` (rpyc unwrap, imported as in `acquisition.py`). +- Produces: SSE `GET /api/lightsheet/stream`; `POST /api/lightsheet/live/params` `{galvo, piezo, exposure}`; in-process live param state `self._ls_params`. + +> **Implementer confirmations (cannot be verified off-rig — confirm against the real code, do not guess silently):** +> 1. The device-layer core handle (`self.system.core`) and that `pymmcore.CMMCore` exposes `startContinuousSequenceAcquisition(float)`, `getLastImage()`, `stopSequenceAcquisition()`, `isSequenceRunning()` (standard CMMCore API). If `getLastImage` is unavailable, use `getLastImageMD`/`getNBeforeLastImage`. +> 2. The scanner/piezo park calls: `self.devices["scanner"].sa_offset_y.setPosition(deg)` and `self.devices["piezo"].setPosition(z)`, and `set_spim_state("Idle")` on both (from `devices/scanner.py:271`, `devices/piezo.py:226`, recon §3). +> 3. Whether brightfield live needs the scanner/beam enabled or a static park — confirm against `../micro-manager/plugins/ASIdiSPIM/src/main/java/org/micromanager/asidispim/SetupPanel.java`. Default: static park, laser "ALL OFF". + +- [ ] **Step 1: Write the failing test (fake core + fake devices)** + +```python +# tests/test_lightsheet_streamer.py +import asyncio, numpy as np, pytest +from gently.hardware.dispim.device_layer import DeviceLayer # confirm class name/import + + +class FakeCore: + def __init__(self): + self.running = False + self.exposure = None + self.cam = None + self._frame = np.full((64, 64), 1000, dtype=np.uint16) + self.started = 0 + self.stopped = 0 + + def setCameraDevice(self, n): + self.cam = n + + def getCameraDevice(self): + return self.cam + + def setExposure(self, n, ms): + self.exposure = ms + + def startContinuousSequenceAcquisition(self, interval): + self.running = True + self.started += 1 + + def stopSequenceAcquisition(self): + self.running = False + self.stopped += 1 + + def isSequenceRunning(self): + return self.running + + def getLastImage(self): + return self._frame + + +class FakeAxis: + def __init__(self): + self.pos = None + + def setPosition(self, v): + self.pos = v + + +class FakeScanner: + def __init__(self): + self.sa_offset_y = FakeAxis() + self.name = "Scanner" + self.state = None + + def set_spim_state(self, s): + self.state = s + + +class FakePiezo(FakeAxis): + def __init__(self): + super().__init__() + self.name = "Piezo" + self.state = None + + def set_spim_state(self, s): + self.state = s + + +def _streamer(dl): + dl.system = type("S", (), {"core": FakeCore()})() + dl.devices = { + "camera": type("C", (), {"name": "HamCam1"})(), + "scanner": FakeScanner(), + "piezo": FakePiezo(), + } + return dl + + +async def test_grab_parks_and_peeks(monkeypatch): + dl = _streamer(DeviceLayer.__new__(DeviceLayer)) + dl._state_pause_counter = 0 + dl._ls_target_max_dim = 512 + dl._ls_jpeg_quality = 70 + dl._ls_params = {"galvo": 1.5, "piezo": 40.0, "exposure": 20.0} + dl._ls_seq_started = False + dl._ls_applied = {} + img = await asyncio.to_thread(dl._grab_lightsheet_frame_sync) + assert img is not None and img.shape == (64, 64) + assert dl.system.core.running is True # sequence started + assert dl.devices["piezo"].pos == 40.0 # piezo parked + assert dl.devices["scanner"].sa_offset_y.pos == 1.5 # galvo parked + + +async def test_exposure_change_restarts_sequence(): + dl = _streamer(DeviceLayer.__new__(DeviceLayer)) + dl._state_pause_counter = 0 + dl._ls_target_max_dim = 512 + dl._ls_jpeg_quality = 70 + dl._ls_params = {"galvo": 0.0, "piezo": 50.0, "exposure": 10.0} + dl._ls_seq_started = False + dl._ls_applied = {} + await asyncio.to_thread(dl._grab_lightsheet_frame_sync) + starts = dl.system.core.started + dl._ls_params["exposure"] = 30.0 # exposure change + await asyncio.to_thread(dl._grab_lightsheet_frame_sync) + assert dl.system.core.stopped >= 1 and dl.system.core.started == starts + 1 + assert dl.system.core.exposure == 30.0 +``` + +- [ ] **Step 2: Run to verify it fails** + +Run: `pytest tests/test_lightsheet_streamer.py -v` +Expected: FAIL — `AttributeError: _grab_lightsheet_frame_sync` (or import). + +- [ ] **Step 3: Implement the streamer** + +Add config attrs in `__init__` (near `:169`, after the `_cam_*` block): +```python +# Lightsheet (SPIM) live stream — continuous sequence acquisition. +self._ls_subscribers: list[asyncio.Queue] = [] +self._ls_task: asyncio.Task | None = None +self._ls_interval_sec: float = 0.0 # peek as fast as exposure allows +self._ls_target_max_dim: int = 512 +self._ls_jpeg_quality: int = 70 +self._ls_params: dict = {"galvo": 0.0, "piezo": 50.0, "exposure": 20.0} +self._ls_seq_started: bool = False +self._ls_applied: dict = {} # last-applied galvo/piezo/exposure +``` + +Add the grab/park/peek (mirrors the sequence-acq calls from `acquisition.py`; uses `self.system.core`): +```python +def _park_lightsheet_sync(self) -> None: + """Park scanner galvo + imaging piezo at the current live params (static sheet).""" + p = self._ls_params + scanner = self.devices.get("scanner") + piezo = self.devices.get("piezo") + if scanner is not None: + try: + scanner.set_spim_state("Idle") + except Exception: + pass + scanner.sa_offset_y.setPosition(float(p["galvo"])) + if piezo is not None: + try: + piezo.set_spim_state("Idle") + except Exception: + pass + piezo.setPosition(float(p["piezo"])) + + +def _ensure_lightsheet_sequence_sync(self) -> None: + """Start (or restart on exposure change) the continuous sequence on the SPIM camera.""" + core = self.system.core + cam = self.devices.get("camera") + if cam is None: + raise RuntimeError("No lightsheet camera configured") + p = self._ls_params + need_restart = not self._ls_seq_started or self._ls_applied.get("exposure") != p["exposure"] + if need_restart: + if core.isSequenceRunning(): + core.stopSequenceAcquisition() + if core.getCameraDevice() != cam.name: + core.setCameraDevice(cam.name) + core.setExposure(cam.name, float(p["exposure"])) + core.startContinuousSequenceAcquisition(self._ls_interval_sec * 1000.0) + self._ls_seq_started = True + self._ls_applied["exposure"] = p["exposure"] + + +def _grab_lightsheet_frame_sync(self): + """Park → ensure sequence running → peek the latest frame (never drain).""" + try: + self._park_lightsheet_sync() # galvo/piezo applied live + self._ensure_lightsheet_sequence_sync() # start / restart on exposure + from gently.hardware.dispim.devices.acquisition import _safe_obtain + + core = self.system.core + img = core.getLastImage() + try: + img = _safe_obtain(img) + except (ImportError, AttributeError): + pass + return np.asarray(img) + except Exception as exc: + logger.debug("Lightsheet grab failed: %s", exc) + return None + + +def _stop_lightsheet_sequence_sync(self) -> None: + try: + if self.system.core.isSequenceRunning(): + self.system.core.stopSequenceAcquisition() + except Exception: + logger.debug("stop lightsheet sequence failed", exc_info=True) + self._ls_seq_started = False + self._ls_applied = {} +``` + +Add the streamer loop + broadcast (mirror `_bottom_camera_streamer`/`_broadcast_camera`, reusing `_encode_frame_for_stream`): +```python +async def _lightsheet_streamer(self): + logger.info("Lightsheet streamer started") + try: + while self._ls_subscribers: + if self._state_pause_counter > 0: + # Heavy plan owns MMCore: release the sequence and back off. + if self._ls_seq_started: + await asyncio.to_thread(self._stop_lightsheet_sequence_sync) + await asyncio.sleep(0.1) + continue + tick = time.monotonic() + img = await asyncio.to_thread(self._grab_lightsheet_frame_sync) + payload = self._encode_frame_for_stream(img) if img is not None else None + if payload is not None: + await self._broadcast_lightsheet(payload) + elapsed = time.monotonic() - tick + # Pace to at least the exposure; peek-rate caps near the camera rate. + floor = max(self._ls_interval_sec, self._ls_params["exposure"] / 1000.0) + await asyncio.sleep(max(0.0, floor - elapsed)) + except asyncio.CancelledError: + raise + except Exception: + logger.exception("Lightsheet streamer crashed") + finally: + await asyncio.to_thread(self._stop_lightsheet_sequence_sync) + logger.info("Lightsheet streamer exiting") + + +async def _broadcast_lightsheet(self, payload): + if not self._ls_subscribers: + return + dead = [] + for q in self._ls_subscribers: + try: + q.put_nowait(payload) + except asyncio.QueueFull: + try: + _ = q.get_nowait() + q.put_nowait(payload) + except Exception: + dead.append(q) + for q in dead: + try: + self._ls_subscribers.remove(q) + except ValueError: + pass +``` + +> `_encode_frame_for_stream` uses `self._cam_target_max_dim`/`self._cam_jpeg_quality`. To get the 512/70 lightsheet settings without duplicating the encoder, add an optional override: change its signature to `_encode_frame_for_stream(self, img, max_dim=None, quality=None)` defaulting to the `_cam_*` values, and call it `self._encode_frame_for_stream(img, self._ls_target_max_dim, self._ls_jpeg_quality)` from the lightsheet loop. (One-line change to the encoder; bottom-camera behavior unchanged.) + +Add the SSE handler + params handler (mirror `handle_bottom_camera_stream`; the params handler updates `self._ls_params` — galvo/piezo apply on the next grab, exposure triggers the restart path): +```python +async def handle_lightsheet_stream(self, request): + response = web.StreamResponse( + status=200, + reason="OK", + headers={ + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive", + "X-Accel-Buffering": "no", + }, + ) + await response.prepare(request) + queue: asyncio.Queue = asyncio.Queue(maxsize=4) + self._ls_subscribers.append(queue) + if len(self._ls_subscribers) == 1 and (self._ls_task is None or self._ls_task.done()): + self._ls_task = asyncio.create_task(self._lightsheet_streamer(), name="lightsheet-streamer") + try: + await response.write(b": connected\n\n") + while True: + try: + payload = await asyncio.wait_for(queue.get(), timeout=10.0) + except asyncio.TimeoutError: + await response.write(b": keepalive\n\n") + continue + if payload is None: + break + await response.write(f"data: {json.dumps(payload)}\n\n".encode()) + except (asyncio.CancelledError, ConnectionResetError, ConnectionAbortedError): + pass + except Exception: + logger.exception("Lightsheet SSE writer failed") + finally: + try: + self._ls_subscribers.remove(queue) + except ValueError: + pass + return response + + +async def handle_lightsheet_params(self, request): + body = await request.json() + for k in ("galvo", "piezo", "exposure"): + if k in body and body[k] is not None: + self._ls_params[k] = float(body[k]) + return web.json_response({"params": self._ls_params}) +``` + +Register both routes near `:2806`: +```python + self._app.router.add_get("/api/lightsheet/stream", self.handle_lightsheet_stream) + self._app.router.add_post("/api/lightsheet/live/params", self.handle_lightsheet_params) +``` + +- [ ] **Step 4: Run to verify it passes** + +Run: `pytest tests/test_lightsheet_streamer.py -v` +Expected: PASS (2 passed). If `DeviceLayer.__new__` bypassing `__init__` leaves attrs unset, the test sets the needed ones explicitly (it does). + +- [ ] **Step 5: Commit** + +```bash +git add gently/hardware/dispim/device_layer.py tests/test_lightsheet_streamer.py +git commit -m "feat(manual-mode): device-layer lightsheet live streamer (continuous sequence acquisition)" +``` + +--- + +### Task 3: Client methods — `stream_lightsheet` + `set_lightsheet_live_params` + +**Files:** +- Modify: `gently/hardware/dispim/client.py` (add two methods on `DiSPIMMicroscope`, near `stream_bottom_camera` `:913`) +- Test: `tests/test_lightsheet_client.py` + +**Interfaces:** +- Produces: `async def stream_lightsheet(self, timeout=None)` (async generator over `GET /api/lightsheet/stream`, identical SSE parse to `stream_bottom_camera`); `async def set_lightsheet_live_params(self, galvo=None, piezo=None, exposure=None) -> dict` (`POST /api/lightsheet/live/params`). + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_lightsheet_client.py +import pytest +from gently.hardware.dispim.client import DiSPIMMicroscope + + +async def test_set_params_posts_body(monkeypatch): + m = DiSPIMMicroscope.__new__(DiSPIMMicroscope) + sent = {} + + async def fake_post(path, body): + sent["path"] = path + sent["body"] = body + return {"params": body} + + m._api_post = fake_post # confirm the real low-level POST helper name + res = await m.set_lightsheet_live_params(galvo=1.0, piezo=42.0, exposure=15.0) + assert sent["path"] == "/api/lightsheet/live/params" + assert sent["body"] == {"galvo": 1.0, "piezo": 42.0, "exposure": 15.0} + assert res == {"params": {"galvo": 1.0, "piezo": 42.0, "exposure": 15.0}} + + +def test_stream_lightsheet_is_async_generator(): + m = DiSPIMMicroscope.__new__(DiSPIMMicroscope) + import inspect + + assert inspect.isasyncgenfunction(m.stream_lightsheet) +``` + +> Confirm the real low-level POST helper (the recon shows `set_led` etc. POST via an internal helper — find whether it's `self._api_post(path, body)` or an inline `self._session.post`). Match it; if `set_lightsheet_live_params` should drop `None` keys, build the body from only the provided args (the test passes all three). + +- [ ] **Step 2: Run to verify it fails** + +Run: `pytest tests/test_lightsheet_client.py -v` → FAIL (no such methods). + +- [ ] **Step 3: Implement** + +```python +async def stream_lightsheet(self, timeout: float | None = None): + """Async generator yielding JPEG frames from the lightsheet live SSE stream. + + Mirrors :meth:`stream_bottom_camera`; subscriber-gated on the device layer. + """ + self._ensure_connected() + client_timeout = aiohttp.ClientTimeout(total=None, sock_read=timeout, sock_connect=10.0) + url = f"{self.http_url}/api/lightsheet/stream" + async with self._session.get(url, timeout=client_timeout) as resp: + resp.raise_for_status() + buf = b"" + async for chunk in resp.content.iter_any(): + if not chunk: + continue + buf += chunk + while b"\n\n" in buf: + event_block, buf = buf.split(b"\n\n", 1) + data_lines = [] + for line in event_block.splitlines(): + if not line or line.startswith(b":"): + continue + if line.startswith(b"data:"): + data_lines.append(line[5:].lstrip()) + if not data_lines: + continue + raw = b"\n".join(data_lines).decode("utf-8", errors="replace") + try: + import json as _json + + yield _json.loads(raw) + except Exception as exc: + logger.warning("Malformed lightsheet SSE payload skipped: %s", exc) + + +async def set_lightsheet_live_params(self, galvo=None, piezo=None, exposure=None) -> dict: + """POST live galvo/piezo/exposure to the device-layer lightsheet streamer.""" + body = {} + if galvo is not None: + body["galvo"] = float(galvo) + if piezo is not None: + body["piezo"] = float(piezo) + if exposure is not None: + body["exposure"] = float(exposure) + return await self._api_post("/api/lightsheet/live/params", body) +``` + +> If the real POST helper isn't `_api_post`, adapt this one call site (and the test) to the real helper. `stream_lightsheet` copies `stream_bottom_camera` verbatim except the URL. + +- [ ] **Step 4: Run to verify it passes** + +Run: `pytest tests/test_lightsheet_client.py -v` (2 passed) + +- [ ] **Step 5: Commit** + +```bash +git add gently/hardware/dispim/client.py tests/test_lightsheet_client.py +git commit -m "feat(manual-mode): client stream_lightsheet + set_lightsheet_live_params" +``` + +--- + +### Task 4: `LightSheetStreamMonitor` (agent Service) + agent wiring + +**Files:** +- Create: `gently/app/lightsheet_monitor.py` +- Modify: `gently/app/agent.py` (init attr; construct in `start_viz_server` near the bottom-camera monitor `:849-860`; stop in `stop_viz_server` near `:862-869`) +- Test: `tests/test_lightsheet_monitor.py` +- Reference (mirror verbatim, swapping names/event): `gently/app/bottom_camera_monitor.py` + +**Interfaces:** +- Consumes: `microscope.stream_lightsheet()` (Task 3); `EventType.LIGHTSHEET_FRAME` (Task 1). +- Produces: `LightSheetStreamMonitor(Service)` with `running` property, `on_start`/`on_stop`; publishes `LIGHTSHEET_FRAME`. `agent.lightsheet_monitor` (constructed, not started; started via proxy in Task 5). + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_lightsheet_monitor.py +import asyncio +from gently.core.event_bus import EventType, get_event_bus +from gently.app.lightsheet_monitor import LightSheetStreamMonitor + + +class FakeScope: + async def stream_lightsheet(self): + for i in range(3): + yield {"t": float(i), "jpeg_b64": f"f{i}"} + await asyncio.sleep(0) + + +async def test_monitor_publishes_frames(): + bus = get_event_bus() + seen = [] + bus.subscribe(EventType.LIGHTSHEET_FRAME, lambda e: seen.append(e.data)) + mon = LightSheetStreamMonitor(FakeScope(), reconnect_delay_sec=0.01) + await mon.start() + await asyncio.sleep(0.05) + await mon.stop() + assert any(d.get("jpeg_b64") == "f0" for d in seen) + assert mon.running is False +``` + +- [ ] **Step 2: Run to verify it fails** — `pytest tests/test_lightsheet_monitor.py -v` → FAIL (no module). + +- [ ] **Step 3: Implement** — copy `gently/app/bottom_camera_monitor.py` to `gently/app/lightsheet_monitor.py` and change: class → `LightSheetStreamMonitor`; `name="lightsheet-monitor"`; the `_run` loop calls `self.microscope.stream_lightsheet()` and publishes `EventType.LIGHTSHEET_FRAME` with `source="lightsheet-monitor"`. (Everything else — Service base, on_start/on_stop, reconnect loop, `_last_frame_ts`, `running` — is identical.) + +Agent wiring in `gently/app/agent.py`: add `self.lightsheet_monitor = None` next to `self.bottom_camera_monitor = None`; in `start_viz_server` (after the bottom-camera monitor construction `:849-860`): +```python +if self.microscope is not None and self.lightsheet_monitor is None: + try: + from .lightsheet_monitor import LightSheetStreamMonitor + + self.lightsheet_monitor = LightSheetStreamMonitor(self.microscope) + logger.info("Lightsheet monitor ready (not started)") + except Exception as e: + logger.warning(f"Failed to construct lightsheet monitor: {e}") + self.lightsheet_monitor = None +``` +In `stop_viz_server` (near `:862`): +```python + if self.lightsheet_monitor is not None: + try: + await self.lightsheet_monitor.stop() + except Exception: + logger.exception("Failed to stop lightsheet monitor") + self.lightsheet_monitor = None +``` + +- [ ] **Step 4: Run to verify it passes** — `pytest tests/test_lightsheet_monitor.py -v` (1 passed); `pytest -q` (no new failures). + +- [ ] **Step 5: Commit** + +```bash +git add gently/app/lightsheet_monitor.py gently/app/agent.py tests/test_lightsheet_monitor.py +git commit -m "feat(manual-mode): LightSheetStreamMonitor bridge + agent wiring" +``` + +--- + +### Task 5: Browser proxy routes (`require_control`) + +**Files:** +- Modify: `gently/ui/web/routes/data.py` (add routes in `create_router`, mirroring the bottom-camera + room-light routes) +- Test: `tests/test_lightsheet_routes.py` +- Reference: bottom-camera start/stop/status (`data.py:260-311`), room-light proxy (`data.py:335-355`), `_resolve_client` (`:313`), `require_control` (`:10`). + +**Interfaces:** +- Consumes: `agent.lightsheet_monitor` (Task 4); `client.set_lightsheet_live_params`, `set_led`, `set_laser_power`, `set_camera_led_mode`, `move_to_position`, `acquire_burst`, `acquire_volume`. +- Produces (all `Depends(require_control)` except GET status): `POST /api/devices/lightsheet/live/{start,stop}`, `GET /api/devices/lightsheet/live/status`, `POST /api/devices/lightsheet/live/params`, `POST /api/devices/led/set`, `POST /api/devices/laser/off`, `POST /api/devices/camera/led_mode`, `POST /api/devices/stage/move`, `POST /api/devices/acquire/{burst,volume}`. + +- [ ] **Step 1: Write the failing test (TestClient + mock client/monitor)** + +```python +# tests/test_lightsheet_routes.py +from unittest.mock import MagicMock, AsyncMock +from fastapi import FastAPI +from fastapi.testclient import TestClient +from gently.ui.web.routes.data import create_router +import gently.ui.web.auth as auth + + +def _app(client=None, monitor=None): + server = MagicMock() + server.agent_bridge.agent.client = client + server.agent_bridge.agent.lightsheet_monitor = monitor + app = FastAPI() + app.include_router(create_router(server)) + # legacy localhost = CONTROL; TestClient client.host is "testclient" → force CONTROL: + app.dependency_overrides[auth.require_control] = lambda: True + return TestClient(app) + + +def test_live_params_forwards(): + client = MagicMock() + client.set_lightsheet_live_params = AsyncMock(return_value={"params": {}}) + r = _app(client=client).post( + "/api/devices/lightsheet/live/params", json={"galvo": 1.0, "piezo": 40.0, "exposure": 20.0} + ) + assert r.status_code == 200 + client.set_lightsheet_live_params.assert_awaited_once_with(galvo=1.0, piezo=40.0, exposure=20.0) + + +def test_acquire_burst_forwards(): + client = MagicMock() + client.acquire_burst = AsyncMock(return_value={"success": True, "request_id": "b1"}) + r = _app(client=client).post( + "/api/devices/acquire/burst", + json={"frames": 60, "mode": "1hz", "num_slices": 1, "exposure_ms": 5.0}, + ) + assert r.status_code == 200 and r.json().get("request_id") == "b1" + + +def test_live_start_requires_monitor(): + r = _app(monitor=None).post("/api/devices/lightsheet/live/start") + assert r.status_code == 503 +``` + +> Confirm the `require_control` override mechanism: in legacy mode `TestClient` requests are not localhost, so override the dependency (as above) to isolate route logic. A separate test can assert the gate by NOT overriding and expecting 403. + +- [ ] **Step 2: Run to verify it fails** — `pytest tests/test_lightsheet_routes.py -v` → FAIL (routes 404). + +- [ ] **Step 3: Implement** — in `create_router`, add (mirroring the referenced routes). Live start/stop/status copy the bottom-camera versions verbatim, swapping `bottom_camera_monitor` → `lightsheet_monitor`. Then: +```python +@router.post("/api/devices/lightsheet/live/params", dependencies=[Depends(require_control)]) +async def lightsheet_live_params(payload: dict = Body(...)): # noqa: B008 + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + res = await client.set_lightsheet_live_params( + galvo=payload.get("galvo"), piezo=payload.get("piezo"), exposure=payload.get("exposure") + ) + except Exception as exc: + logger.exception("lightsheet live params failed") + raise HTTPException(status_code=502, detail=f"params failed: {exc}") from exc + return res + + +@router.post("/api/devices/led/set", dependencies=[Depends(require_control)]) +async def led_set(payload: dict = Body(...)): # noqa: B008 + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_led(str(payload.get("state", "Closed"))) + except Exception as exc: + raise HTTPException(status_code=502, detail=f"led failed: {exc}") from exc + + +@router.post("/api/devices/laser/off", dependencies=[Depends(require_control)]) +async def laser_off(): + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_laser_power(488, 0) # confirm signature; force off + except Exception as exc: + raise HTTPException(status_code=502, detail=f"laser off failed: {exc}") from exc + + +@router.post("/api/devices/camera/led_mode", dependencies=[Depends(require_control)]) +async def camera_led_mode(payload: dict = Body(...)): # noqa: B008 + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_camera_led_mode(bool(payload.get("use_led", False))) + except Exception as exc: + raise HTTPException(status_code=502, detail=f"camera led mode failed: {exc}") from exc + + +@router.post("/api/devices/stage/move", dependencies=[Depends(require_control)]) +async def stage_move(payload: dict = Body(...)): # noqa: B008 + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.move_to_position(float(payload["x"]), float(payload["y"])) + except KeyError: + raise HTTPException(status_code=400, detail="x and y required") + except Exception as exc: + raise HTTPException(status_code=502, detail=f"stage move failed: {exc}") from exc + + +@router.post("/api/devices/acquire/burst", dependencies=[Depends(require_control)]) +async def acquire_burst(payload: dict = Body(...)): # noqa: B008 + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.acquire_burst( + frames=int(payload.get("frames", 60)), + mode=str(payload.get("mode", "1hz")), + num_slices=int(payload.get("num_slices", 1)), + exposure_ms=float(payload.get("exposure_ms", 5.0)), + ) + except Exception as exc: + raise HTTPException(status_code=502, detail=f"burst failed: {exc}") from exc + + +@router.post("/api/devices/acquire/volume", dependencies=[Depends(require_control)]) +async def acquire_volume(payload: dict = Body(...)): # noqa: B008 + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.acquire_volume( + num_slices=int(payload.get("num_slices", 50)), + exposure_ms=float(payload.get("exposure_ms", 10.0)), + ) + except Exception as exc: + raise HTTPException(status_code=502, detail=f"volume failed: {exc}") from exc +``` +Add the live start/stop/status routes by copying `start_bottom_camera_stream`/`stop_bottom_camera_stream`/`get_bottom_camera_status` (`data.py:260-311`) under `/api/devices/lightsheet/live/...` with `getattr(agent, "lightsheet_monitor", None)`. + +- [ ] **Step 4: Run to verify it passes** — `pytest tests/test_lightsheet_routes.py -v` (3 passed); `pytest -q` (no new failures). + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/routes/data.py tests/test_lightsheet_routes.py +git commit -m "feat(manual-mode): require_control proxy routes for lightsheet live + illumination + acquire" +``` + +--- + +### Task 6: Manual view (frontend) + +**Files:** +- Modify: `gently/ui/web/templates/index.html` — add a `data-view="manual"` button to `#devices-view-switcher` (`:445-449`); add a `#devices-view-manual` container with the live canvas + control rail (model the camera panel `:569-608`); load A's `temperature-graph.js` if not already loaded on this page. +- Modify: `gently/ui/web/static/js/devices.js` — add `'manual'` to `VIEWS` (`:17`); a `handleLightsheetFrame` (mirror `handleCameraFrame` `:1111`); live toggle (mirror `toggleCameraStream`); galvo/piezo/exposure controls (debounced POST `live/params`); illumination toggles; acquire buttons; FPS readout; embed `TemperatureGraph.init`; a `'v'`-style key handled only if not deferred — leave existing keys, add nothing conflicting. +- Modify: the devices stylesheet for the manual panel (reuse `.devices-camera-*` styles where possible). + +**No JS unit harness** — verified by `node --check` + a Chrome-MCP harness (like A) + a UI audit. + +- [ ] **Step 1: Markup** — add to `#devices-view-switcher`: +```html + +``` +Add the view container after `#devices-view-optical3d` (`:716`-ish), with: a live `` (or ``) + placeholder + FPS/side overlay + Start/Stop toggle (`#devices-ls-toggle`); a right rail with exposure input, galvo slider (`#devices-ls-galvo`), piezo slider (`#devices-ls-piezo`), illumination toggles (LED `#devices-ls-led`, camera-LED-mode, room light, a static "Laser: OFF" indicator), a temperature setpoint + `
`, and Snap-volume / Burst buttons + a `#devices-ls-lastcap` card. Mirror the `.devices-camera-*` class structure for the image stage so the existing zoom/pan inline machinery can be reused or replicated. + +- [ ] **Step 2: JS — frame paint + controls** + +Add `'manual'` to `VIEWS`. Add a frame handler mirroring `handleCameraFrame` (separate DOM ids `_lsImg`/`_lsMeta`, its own FPS window) and subscribe `ClientEventBus.on('LIGHTSHEET_FRAME', handleLightsheetFrame)` in `setupCameraWiring` (or a new `setupManualWiring`). Live toggle hits `/api/devices/lightsheet/live/start|stop`. Galvo/piezo/exposure inputs: on `input`, **debounce ~120 ms**, then `fetch('/api/devices/lightsheet/live/params', {method:'POST', headers:{'Content-Type':'application/json'}, body: JSON.stringify({galvo, piezo, exposure})})`. Illumination toggles POST their routes. Acquire buttons POST `/api/devices/acquire/burst|volume` with the current params, show "acquiring…" then render the result ref in `#devices-ls-lastcap`. Embed A's graph: `if (window.TemperatureGraph) TemperatureGraph.init(document.getElementById('devices-ls-tempgraph'), 'current')`. FPS readout from the frame handler's window (reuse the `computeCameraFps` approach). + +- [ ] **Step 3: `node --check`** — `node --check gently/ui/web/static/js/devices.js` (exit 0). + +- [ ] **Step 4: Chrome-MCP harness verification** — build a standalone harness (like A's): copy `event-bus.js`, `temperature-graph.js`, the new manual JS, and `main.css`; stub `fetch` for `live/params` + status + acquire; feed simulated `LIGHTSHEET_FRAME` events (a moving synthetic gradient that shifts when galvo/piezo "params" change) to demonstrate the slide-and-see; screenshot; run the alignment/spacing/contrast UI audit and fix flaws. Live in-app + FPS verification is deferred to the rig. + +- [ ] **Step 5: Commit** + +```bash +git add gently/ui/web/templates/index.html gently/ui/web/static/js/devices.js gently/ui/web/static/css/main.css +git commit -m "feat(manual-mode): Manual view — SPIM live canvas, galvo/piezo/exposure, illumination, acquire, temp" +``` + +--- + +### Task 7: FPS measurement (rig) + conditional binary transport + +**Files:** +- Create: `docs/superpowers/notes/2026-06-28-lightsheet-fps-measurement.md` (record the numbers) +- (Conditional) Modify: `gently/ui/web/connection_manager.py` + `agent_ws`/`server.py` + `websocket.js` for a binary frame path. + +**This task is a measurement + a gate, not unconditional code.** + +- [ ] **Step 1:** On the rig, start lightsheet live and record from the Manual-view FPS readout + device logs: **device grab rate**, **delivered rate**, **browser paint rate**, at 512 px/q70 and at a reduced 384 px/q60. Write them into the notes file with the exposure used. +- [ ] **Step 2: Diagnose.** If device grab < ~15 fps → limiter is exposure/readout/rpyc, not transport — tune exposure/size/quality, stop here. If device grab ≥ target but browser paint < target → transport is the bottleneck → do Step 3. +- [ ] **Step 3 (conditional): binary WebSocket path.** Add a `send_bytes`-based frame channel: device→agent SSE stays; on the agent→browser hop, push the raw JPEG via `websocket.send_bytes(prefix + jpeg)` (small type byte), bypassing base64 + the per-client `json.dumps` + the EventBus fan-out; browser `onmessage` binary → `createImageBitmap(new Blob([buf]))` → `ctx.drawImage`. Re-measure and record. +- [ ] **Step 4: Commit** the notes (and any binary-path code, if built). + +```bash +git add docs/superpowers/notes/2026-06-28-lightsheet-fps-measurement.md +git commit -m "docs(manual-mode): lightsheet live FPS measurement + transport decision" +``` + +--- + +## Self-Review + +**Spec coverage:** §2.1 streamer → Task 2; §2.2 monitor → Task 4; §2.3 proxy routes → Task 5; §2.4 client methods → Task 3; §2.5 Manual view → Task 6; §2.6 concurrency (`_state_pause_counter` back-off) → Task 2 streamer loop; §2.7 brightfield laser-off → Tasks 2 & 5 (`laser/off`); §3 transport baseline + measurement + binary escalation → Tasks 1/4 (baseline) + Task 7 (measure/escalate); §4 error handling → Tasks 2/5 (try/except, 503/502); §5 testing → each task's tests + Task 6 harness; LIGHTSHEET_FRAME event → Task 1. Single-camera (no side A/B) reflected throughout. ✓ + +**Open confirmations (explicit, not placeholders):** pymmcore sequence-acq method availability + core handle (Task 2); scanner/piezo park method names (Task 2); the SetupPanel scanner/beam question (Task 2); the client low-level POST helper name (Task 3); the `require_control` test-override mechanism (Task 5). Each names a concrete fallback. + +**Type consistency:** frame payload `{t, shape, downsample, mime, jpeg_b64}` (reused encoder) across Tasks 2/3/4/6; live params `{galvo, piezo, exposure}` across Tasks 2/3/5/6; event `LIGHTSHEET_FRAME` across Tasks 1/4/6; `lightsheet_monitor` attr across Tasks 4/5. Consistent. diff --git a/docs/superpowers/plans/2026-06-28-operations-tab.md b/docs/superpowers/plans/2026-06-28-operations-tab.md new file mode 100644 index 00000000..46625492 --- /dev/null +++ b/docs/superpowers/plans/2026-06-28-operations-tab.md @@ -0,0 +1,76 @@ +# Operations — agent-authored Operation Plan (D v3) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development. Steps use `- [ ]`. + +**Goal:** The agent emits a typed Operation Plan (its tactics, planned/active/done); Operations renders it ⊕ live telemetry. Built backend-first (the user-directed, low-rework part), then the renderer. + +**Architecture:** A typed `OperationPlan` in `FileContextStore` (new domain), written by a forced-tool agent call, served on a route, rendered by a data-driven operation-spine generalized for tactic kinds, with live-telemetry binding + scenario test mode. + +**Tech Stack:** Python (FileContextStore YAML, the `@tool`/forced-`tool_choice` pattern, EventBus `CONTEXT_UPDATED`), FastAPI route, vanilla-JS + SVG renderer, pytest. + +## Global Constraints +- Source of truth = the agent's typed plan (NOT a backend reconstruction). Live telemetry binds onto declared tactics; the agent declares tactic identity, the system supplies live values. +- Plan schema (see spec §1): `{session_id,title,goal,tactics:[{id,name,kind,state,scope,rationale,structure,live_bind,relations}],updated_at,updated_reason}`. `kind∈{standing_timelapse,reactive_monitor,scripted_protocol,exclusive_burst,oneshot,custom}`, `state∈{planned,active,done}`. No round-robin. +- Forced typed output mirrors `gently/harness/memory/notebook_ask.py` (ASK_TOOL + `tool_choice={'type':'tool',...}` → validated `block.input`) OR the `@tool` auto-schema (`harness/tools/registry.py`). +- Store mirrors `session_intents`/`active/` domains; fire the existing `CONTEXT_UPDATED`. +- Route mirrors `gently/ui/web/routes/context.py` (`/api/context`). +- Renderer = the validated operation-spine (harness `scratchpad/opsdesign/harness.html`) with the audit fixes (queued reads cocked + "next" marker; active phase/status is the headline; flatten — no card-in-card; whole-left-column amber + colored left edge per state; copy "queued"; mono values). Generalize for the tactic kinds (standing→per-embryo cadence strip; reactive→watch/reaction/status; scripted→phases). +- Frontend: no JS harness → `node --check` + Chrome-MCP audit across scenario fixtures. +- Git hygiene: stage only your files by explicit path; never `git add -A`. + +--- + +### Task 1: OperationPlan model + FileContextStore domain +**Files:** Create/extend `gently/harness/memory/model.py` (an `OperationPlan`/`Tactic` dataclass or a documented dict schema); Modify `gently/harness/memory/file_store.py` (`set_operation_plan(session_id, plan)` / `get_operation_plan(session_id)`, YAML under `agent/operation_plans/{session_id}.yaml`, fire `CONTEXT_UPDATED`). Test: `tests/test_operation_plan_store.py`. +- [ ] Confirm the real `FileContextStore` domain pattern from `set_session_intent`/`create_session_intent` (~file_store.py:758) + the `_notify_*`/`CONTEXT_UPDATED` emit. Mirror it. +- [ ] TDD: set→get round-trip preserves the tactics list + states; `CONTEXT_UPDATED` fired on set. `pytest tests/test_operation_plan_store.py -v`; `pytest -q` clean. Commit `feat(operations): OperationPlan store domain in FileContextStore`. + +### Task 2: `declare_operation_plan` agent tool (forced typed output) +**Files:** Create `gently/app/tools/operation_plan_tools.py` (+ register in `tools/__init__`). Test: `tests/test_operation_plan_tool.py`. +- [ ] Confirm the `@tool` decorator + the forced-tool pattern (registry.py auto-schema, or a literal schema like notebook_ask.ASK_TOOL). The tool accepts the typed plan (tactics list) and writes it via `store.set_operation_plan`. Resolve the store from context (mirror existing memory tools). +- [ ] TDD: calling the tool with a plan persists it (get returns it) + returns a confirmation; missing store → error. `pytest tests/test_operation_plan_tool.py -v`; `pytest -q` clean. Commit `feat(operations): declare_operation_plan typed agent tool`. + +### Task 3: Route `GET /api/operation_plan/{session_id}` +**Files:** Create `gently/ui/web/routes/operation_plan.py` (+ register in `routes/__init__.py`). Test: `tests/test_operation_plan_route.py`. +- [ ] Mirror `routes/context.py` / `routes/temperature.py` `_resolve_session` + `server.context_store`/`gently_store` resolution. Return the stored plan; 404/empty handled; `session=current` resolves newest. +- [ ] TDD (TestClient + mock store): returns the plan; empty when none. `pytest tests/test_operation_plan_route.py -v`; `pytest -q` clean. Commit `feat(operations): GET /api/operation_plan/{session} route`. + +### Task 4: The operation-spine renderer + scenario library (frontend) +**Files:** Create `gently/ui/web/static/js/operations-scenarios.js` (plan fixtures: temp_strain, expression_onset, hatching_detect, transmission_survey, decided_plan, async_multi, idle); Rewrite the Overview path in `gently/ui/web/static/js/experiment-overview.js` to the data-driven operation-spine (generalized per tactic kind, audit fixes); port CSS to `experiment.css`. Reference: `scratchpad/opsdesign/harness.html`. +- [ ] Port the renderer (renderOperation/renderTactic/renderReadout/renderPhase) + add per-kind rendering: standing→per-embryo cadence strip; reactive→watch/reaction/status; scripted→phase stepper; exclusive/oneshot→compact. Audit fixes baked in. `?scenario=` dev mode loads a fixture. +- [ ] `node --check` both JS; build a Chrome-MCP harness at `scratchpad/opsv3/` (real repo files) for the controller to audit across fixtures. Commit `feat(operations): operation-spine renderer + plan scenario library (data-driven)`. + +### Task 5: Live binding + refresh +**Files:** Modify `experiment-overview.js` (fetch `/api/operation_plan`, bind live telemetry from `/strategy`/get_status onto tactics by `live_bind`, subscribe `CONTEXT_UPDATED` + tactic events → debounced refetch). +- [ ] Bind temperature/current-burst/cadence/signal onto declared tactics' readouts/progress; live refresh on plan-change + telemetry events (debounced). `node --check`; harness check. Commit `feat(operations): live telemetry binding + event-driven refresh`. + +## Self-Review +- Store→Task1; tool→Task2; route→Task3; renderer+scenarios→Task4 (audit fixes); live binding→Task5. ✓ +- Open confirmations: FileContextStore domain pattern (T1), the forced-tool/@tool pattern (T2), the route store-resolution (T3), the Overview render seam + harness renderer (T4), the live telemetry sources (T5). +- Type consistency: the plan/tactic schema is identical across store (T1), tool (T2), route (T3), fixtures+renderer (T4), binding (T5). + +--- +## Execution-linkage tasks (added after recon — close the planning→execution loop) + +### Task 6: `transition_tactic` store helper +**Files:** Modify `gently/harness/memory/file_store.py` (add `transition_tactic(session_id, tactic_id, state=None, **bind)` next to `set/get_operation_plan` ~:818 — read the plan, find the tactic by `id`, set `state` and merge `bind` into its `live`/`structure`, write back, fire `CONTEXT_UPDATED`; no-op if plan/tactic absent). Test: `tests/test_transition_tactic.py`. +- [ ] TDD: declare a plan, transition a tactic planned→active with a `request_id` bind → get shows the new state + bound value; unknown tactic_id → no-op (no crash). Commit `feat(operations): transition_tactic store helper`. + +### Task 7: tactic_id threading + start-edge marking in execution tools +**Files:** Modify `gently/app/tools/timelapse_tools.py` (`enable_monitoring_mode`, `queue_burst`, stop/pause), `gently/app/tools/temperature_protocol_tools.py`, and the burst/protocol event payloads (`exclusive.py` BURST_*, `temperature_protocol.py` TEMP_PROTOCOL_*) to carry an optional `tactic_id`. On execute, the tool calls `cs.transition_tactic(session, tactic_id, 'active')`; on stop/pause → 'done'/'paused'. Test: `tests/test_tool_tactic_linkage.py`. +- [ ] Add optional `tactic_id` param; thread into event `data`; flip the plan tactic active on execute (guard: only if a plan + tactic_id exist). TDD with a fake context store capturing transitions. Commit `feat(operations): link execution tools to plan tactics via tactic_id`. + +### Task 8: `OperationPlanUpdater` service (completion edges via the bus) +**Files:** Create `gently/app/operation_plan_updater.py` (a `Service` modeled on `gently/app/temperature_sampler.py` / `TimelineManager`); wire in `gently/app/agent.py` beside the temperature sampler (~:838-849). Test: `tests/test_operation_plan_updater.py`. +- [ ] Subscribe `BURST_COMPLETE`, `TEMP_PROTOCOL_COMPLETED`, `EMBRYO_CADENCE_CHANGED`, `TRIGGER_FIRED`; on each, resolve the tactic (by `tactic_id` in payload, else by kind+embryo) and `cs.transition_tactic(session, tactic_id, 'done', **bind)` binding live values (request_id/mp4_path/setpoint/cadence). Mirror the sampler's start/stop lifecycle + session_id getter. TDD against a fake bus + store. Commit `feat(operations): OperationPlanUpdater — execution events transition plan tactics`. + +--- +## Plan-item ↔ operation linkage tasks (added — tactics planned at plan time) + +### Task 9: PlanItem/ImagingSpec tactical outline +**Files:** Modify `gently/harness/memory/model.py` (`ImagingSpec` ~:200 / `PlanItem` ~:265 — add optional `tactics: list[dict]` outline field: each entry a lightweight tactic `{kind, name, target?, scope?, structure?}`). Ensure the plan-mode planning tools (`gently/harness/plan_mode/tools/planning.py` create_plan_item/update_plan_item) accept/persist it. Test: extend the plan-item model/store tests. +- [ ] Confirm the real ImagingSpec/PlanItem dataclass + how planning tools set the spec. Add the optional `tactics` outline (default empty), persisted in the campaign plan YAML. TDD: a plan item round-trips its tactics outline. Commit `feat(operations): plan-item tactical outline (plan tactics with the imaging spec)`. + +### Task 10: Operation Plan goal/plan_item linkage + seeding +**Files:** Modify `gently/app/tools/operation_plan_tools.py` (or a small seeding helper / the agent): resolve the current session's `plan_item_id`/`campaign_id`/goal from the `session_intent` (`get_current_session_intent` ~file_store.py:788) → set them on the Operation Plan top-level; when a session linked to a plan item with a `tactics` outline begins (or on first declare), SEED the Operation Plan's `planned` tactics from the outline. Test: `tests/test_operation_plan_seeding.py`. +- [ ] Confirm the session_intent→plan_item linkage accessors. On declare/seed, populate `plan_item_id`/`campaign_id`/`goal` from the linked plan item and seed `planned` tactics from its outline (idempotent — don't clobber tactics already active/done). TDD with a fake store: a session linked to a plan item with an outline produces a seeded Operation Plan. Commit `feat(operations): seed Operation Plan goal + planned tactics from the linked plan item`. diff --git a/docs/superpowers/plans/2026-06-28-tactics-library.md b/docs/superpowers/plans/2026-06-28-tactics-library.md new file mode 100644 index 00000000..a845bfa8 --- /dev/null +++ b/docs/superpowers/plans/2026-06-28-tactics-library.md @@ -0,0 +1,35 @@ +# Tactics Library (G) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development. Steps use `- [ ]`. + +**Goal:** Save / list / apply typed tactics (D's tactic objects) as reusable templates — a near-copy of the plan-template pattern. + +**Architecture:** A tactic-library domain in `FileContextStore` (save/list/get/apply), agent tools, and a read route — mirroring `save_plan_template`/`apply_plan_template`. + +## Global Constraints +- Mirror the plan-template pattern: `_plans.py:551-667` (`save_plan_template`/`list_plan_templates`/`get_plan_template`/`apply_plan_template`) + `plan_mode/tools/templates.py`. +- A saved tactic is a TEMPLATE (planned form): `{id,name,slug,kind,structure,scope_hint?,description,params?,created_at,created_by}` — NO live state. `apply_tactic` returns a fresh planned tactic (new run id, state="planned", no `live`). +- Store domain `agent/tactic_library/{id}_{slug}.yaml`; fire `CONTEXT_UPDATED`. +- Git hygiene: stage only your files by explicit path; never `git add -A`. + +--- + +### Task 1: Tactic-library store domain +**Files:** Modify `gently/harness/memory/file_store.py` (or `_plans.py` if templates live there) — add `save_tactic(tactic, name=None) -> str`, `list_tactics() -> list[dict]`, `get_tactic(id_or_name) -> dict | None`, `apply_tactic(id_or_name) -> dict | None`. Test: `tests/test_tactic_library_store.py`. +- [ ] Confirm the exact plan-template implementation (`_plans.py:551-667`) — the id/slug generation, the YAML write helper, the `_notify`/`CONTEXT_UPDATED` emit — and mirror it for tactics at `agent/tactic_library/`. `apply_tactic` returns a deep copy with a fresh `id` (e.g. `tac_<8hex>`), `state="planned"`, and `live`/run-state stripped. +- [ ] TDD: save a tactic → list shows it → get returns it → apply returns a fresh planned tactic (new id, no live, state=planned); get/apply unknown → None. `pytest tests/test_tactic_library_store.py -v`; `pytest -q` clean. Commit `feat(tactics-library): tactic-library store domain (save/list/get/apply)`. + +### Task 2: Agent tools +**Files:** Create `gently/app/tools/tactic_library_tools.py` (+ register in `tools/__init__`). Test: `tests/test_tactic_library_tools.py`. +- [ ] Mirror `plan_mode/tools/templates.py` (the `@tool` usage + store resolution from context). Tools: `save_tactic(name, tactic, description="")` → `store.save_tactic`; `list_tactics()` → `store.list_tactics`; `apply_tactic(id_or_name)` → `store.apply_tactic` then append the planned tactic to the current Operation Plan (`get_operation_plan(session)` → append → `set_operation_plan`; create a minimal plan if none). Resolve the context store + session from the agent (as `declare_operation_plan` does). Register the module. +- [ ] TDD (fake context store + session): save persists; apply adds a planned tactic to the current Operation Plan; list returns the library; missing store → error. `pytest tests/test_tactic_library_tools.py -v`; `pytest -q` clean. Commit `feat(tactics-library): save/list/apply_tactic agent tools`. + +### Task 3: Route `GET /api/tactic_library` +**Files:** Create `gently/ui/web/routes/tactic_library.py` (+ register in `routes/__init__.py`). Test: `tests/test_tactic_library_route.py`. +- [ ] Mirror `routes/operation_plan.py` — resolve `server.context_store`, return `{tactics: store.list_tactics()}`; empty list when none. Register the router. +- [ ] TDD (TestClient + mock store): returns the library; empty when none. `pytest tests/test_tactic_library_route.py -v`; `pytest -q` clean. Commit `feat(tactics-library): GET /api/tactic_library route`. + +## Self-Review +- Store→Task1; tools→Task2; route→Task3. ✓ +- Open confirmations: the plan-template implementation to mirror (T1), the `@tool` + store/session resolution (T2), the route store handle (T3). +- Type consistency: the saved-tactic dict shape is identical across store (T1), tools (T2), route (T3); `apply_tactic`'s fresh planned tactic matches D's tactic schema. diff --git a/docs/superpowers/plans/2026-06-28-temp-change-burst-tactic.md b/docs/superpowers/plans/2026-06-28-temp-change-burst-tactic.md new file mode 100644 index 00000000..4d8b0bf4 --- /dev/null +++ b/docs/superpowers/plans/2026-06-28-temp-change-burst-tactic.md @@ -0,0 +1,368 @@ +# Temp-Change Burst Tactic (C) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (`- [ ]`) syntax. + +**Goal:** A scripted temperature-change burst protocol — brightfield bursts before a setpoint change, during the ramp (until lock), and after — launchable as an agent tool, observable on the Experiment tab. + +**Architecture:** A thin async `TimelapseOrchestrator.run_temp_change_burst_protocol` driver composing existing `BurstAcquisition` (extended to force lasers off), `set_temperature`/`get_temperature`, and brightfield primitives; new timeline EventTypes so the tactic + setpoint changes render; an agent tool that launches the driver via `asyncio.create_task`. + +**Tech Stack:** Python asyncio, the gently EventBus + TimelineManager, pytest (`asyncio_mode=auto`). + +## Global Constraints +- C composes A (temperature stamp/persistence) + B1 (`set_laser_config("ALL OFF")`). No new deps. +- Brightfield every burst: `laser_config="ALL OFF"`; lasers never left on, even on error/cancel. +- Lock contract: poll `client.get_temperature()["state"]` until `"LOCKED" in state` (the device reports `'[ SYSTEM LOCKED ]'`). +- Bursts are temperature-stamped automatically (A) and emit `BURST_START/COMPLETE` (render for free). +- New EventTypes use `auto()` (wire serializes `.name`). +- Tests: fakes for client + burst; `asyncio_mode=auto` (no decorator). Rig-deferred: real ramp timing. +- Git hygiene: stage only your files by explicit path; never `git add -A` (pre-existing untracked screenshots/mockups + uv.lock are not yours). + +--- + +### Task 1: Timeline EventTypes for the tactic + +**Files:** Modify `gently/core/event_bus.py` (3 new `auto()` members near the other domain events); Modify `gently/harness/session/timeline.py` (add 3 entries to the EventType→subtype map, near the `BURST_*` entries ~line 213-225). Test: `tests/test_temp_protocol_events.py`. + +**Interfaces:** Produces `EventType.TEMPERATURE_SETPOINT_CHANGED`, `EventType.TEMP_PROTOCOL_STARTED`, `EventType.TEMP_PROTOCOL_COMPLETED`, each mapped to a timeline subtype (`setpoint_changed`, `temp_protocol_started`, `temp_protocol_completed`). + +- [ ] **Step 1: failing test** +```python +# tests/test_temp_protocol_events.py +from gently.core.event_bus import EventType + + +def test_new_event_types_exist(): + for n in ("TEMPERATURE_SETPOINT_CHANGED", "TEMP_PROTOCOL_STARTED", "TEMP_PROTOCOL_COMPLETED"): + assert getattr(EventType, n).name == n + + +def test_timeline_maps_the_subtypes(): + from gently.harness.session import timeline as tl + + src = open(tl.__file__, encoding="utf-8").read() + for sub in ("temp_protocol_started", "temp_protocol_completed", "setpoint_changed"): + assert sub in src +``` +- [ ] **Step 2: run, expect FAIL** — `pytest tests/test_temp_protocol_events.py -v` +- [ ] **Step 3: implement** — in `event_bus.py`, alongside the burst events: +```python +TEMPERATURE_SETPOINT_CHANGED = auto() # discrete setpoint change (timeline) +TEMP_PROTOCOL_STARTED = auto() # temp-change burst protocol began +TEMP_PROTOCOL_COMPLETED = auto() # protocol ended +``` +In `timeline.py`'s map (mirror the `EventType.BURST_START: {...}` entries), add: +```python + EventType.TEMPERATURE_SETPOINT_CHANGED: {"category": "temperature", "event_subtype": "setpoint_changed"}, + EventType.TEMP_PROTOCOL_STARTED: {"category": "tactic", "event_subtype": "temp_protocol_started"}, + EventType.TEMP_PROTOCOL_COMPLETED: {"category": "tactic", "event_subtype": "temp_protocol_completed"}, +``` +> Confirm the real map structure (keys/value shape) from the existing `BURST_START` entry and match it exactly. +- [ ] **Step 4: run, expect PASS**; `pytest -q` no new failures. +- [ ] **Step 5: commit** — `git add gently/core/event_bus.py gently/harness/session/timeline.py tests/test_temp_protocol_events.py && git commit -m "feat(tactic): timeline event types for temp-change burst protocol"` + +--- + +### Task 2: Brightfield burst — thread `laser_config` + +**Files:** Modify `gently/app/orchestration/exclusive.py` (`BurstAcquisition.__init__` ~line 95 add param; its `client.acquire_burst(...)` call ~line 159-168 pass it); Modify `gently/app/orchestration/timelapse.py` (`queue_burst` ~line 2058 add `laser_config=None`, pass to `BurstAcquisition`). Test: `tests/test_burst_laser_config.py`. + +**Interfaces:** Consumes `client.acquire_burst(..., laser_config=...)` (already accepts it, `client.py:642`). Produces `BurstAcquisition(..., laser_config=None)` and `queue_burst(..., laser_config=None)`. + +- [ ] **Step 1: failing test** +```python +# tests/test_burst_laser_config.py +import asyncio +from gently.app.orchestration.exclusive import BurstAcquisition + + +class FakeClient: + def __init__(self): + self.calls = [] + + async def acquire_burst(self, **kw): + self.calls.append(kw) + return {"success": True, "request_id": "b1", "frames": []} + + +async def test_burst_passes_laser_config(monkeypatch): + b = BurstAcquisition("emb1", frames=3, mode="1hz", num_slices=1, laser_config="ALL OFF") + assert b._laser_config == "ALL OFF" + # the run() path forwards _laser_config into client.acquire_burst kwargs: + # (unit-level: assert the attribute + that run threads it — see note) +``` +> NOTE: `BurstAcquisition.run` needs an orchestrator with a client + embryo lookup + persistence; a full run is heavy. Assert at minimum that `__init__` stores `_laser_config` and that the `acquire_burst` call site in `run` includes `laser_config=self._laser_config` (verify by reading; optionally add a focused test that monkeypatches the embryo/persistence to capture the `acquire_burst` kwargs). Keep the test honest — if a full run is impractical, assert the attribute and add an inline source check that `laser_config=self._laser_config` appears in the `acquire_burst(...)` call. +- [ ] **Step 2: run, expect FAIL** +- [ ] **Step 3: implement** — add `laser_config: str | None = None` to `__init__`, store `self._laser_config = laser_config`; in the `client.acquire_burst(...)` call add `laser_config=self._laser_config`. In `queue_burst`, add `laser_config: str | None = None` and pass `laser_config=laser_config` into the `BurstAcquisition(...)` construction. +- [ ] **Step 4: run, expect PASS**; `pytest -q` clean. +- [ ] **Step 5: commit** — `feat(tactic): thread laser_config through BurstAcquisition + queue_burst (brightfield bursts)` + +--- + +### Task 3: `wait_for_temperature_lock` helper + +**Files:** Create `gently/app/orchestration/temperature_protocol.py` (module for the helper + later the driver). Test: `tests/test_wait_for_lock.py`. + +**Interfaces:** Produces `async def wait_for_temperature_lock(client, timeout_s, poll_s=2.0) -> bool`. + +- [ ] **Step 1: failing test** +```python +# tests/test_wait_for_lock.py +from gently.app.orchestration.temperature_protocol import wait_for_temperature_lock + + +class FakeClient: + def __init__(self, states): + self.states = list(states) + self.calls = 0 + + async def get_temperature(self): + i = min(self.calls, len(self.states) - 1) + self.calls += 1 + return {"state": self.states[i]} + + +async def test_returns_true_when_locked(): + c = FakeClient(["[ IDLE ]", "[ HEATING ]", "[ SYSTEM LOCKED ]"]) + assert await wait_for_temperature_lock(c, timeout_s=5.0, poll_s=0.001) is True + + +async def test_returns_false_on_timeout(): + c = FakeClient(["[ HEATING ]"]) + assert await wait_for_temperature_lock(c, timeout_s=0.02, poll_s=0.001) is False +``` +- [ ] **Step 2: run, expect FAIL** +- [ ] **Step 3: implement** +```python +# gently/app/orchestration/temperature_protocol.py +import asyncio, logging + +logger = logging.getLogger(__name__) + + +async def wait_for_temperature_lock(client, timeout_s: float, poll_s: float = 2.0) -> bool: + """Poll the controller until it reports a locked state, or timeout. Substring 'LOCKED'.""" + loop = asyncio.get_event_loop() + t0 = loop.time() + while True: + try: + resp = await client.get_temperature() + except Exception as exc: + logger.warning("wait_for_temperature_lock poll failed: %s", exc) + resp = {} + if "LOCKED" in str(resp.get("state", "")): + return True + if loop.time() - t0 >= timeout_s: + return False + await asyncio.sleep(poll_s) +``` +- [ ] **Step 4: run, expect PASS** +- [ ] **Step 5: commit** — `feat(tactic): wait_for_temperature_lock poll helper` + +--- + +### Task 4: The protocol driver + +**Files:** Modify `gently/app/orchestration/temperature_protocol.py` (add the driver fn that takes the orchestrator). Test: `tests/test_temp_protocol_driver.py`. + +**Interfaces:** Produces `async def run_temp_change_burst_protocol(orchestrator, embryo_id, target_setpoint_c, *, frames=60, mode="1hz", num_slices=1, bursts_before=1, bursts_after=1, lock_timeout_s=600.0, poll_s=2.0, burst_runner=None) -> dict`. `burst_runner` is an injectable `async (BurstAcquisition)->None` for tests (defaults to `lambda b: b.run(orchestrator)`). + +- [ ] **Step 1: failing test** +```python +# tests/test_temp_protocol_driver.py +from gently.app.orchestration.temperature_protocol import run_temp_change_burst_protocol +from gently.core.event_bus import EventType + + +class FakeClient: + def __init__(self): + self.laser = None + self.led = None + self.setpoint = None + self._poll = 0 + + async def set_laser_config(self, c): + self.laser = c + + async def set_led(self, s): + self.led = s + + async def set_temperature(self, t): + self.setpoint = t + + async def get_temperature(self): + self._poll += 1 + return {"state": "[ SYSTEM LOCKED ]" if self._poll >= 2 else "[ HEATING ]"} + + +class FakeOrch: + def __init__(self, client): + self._client = client + self._temperature_provider = lambda: None + self.events = [] + + @property + def client(self): + return self._client + + def _emit_event(self, et, data): + self.events.append((et, data)) + + +async def test_phase_order_and_brightfield(monkeypatch): + client = FakeClient() + orch = FakeOrch(client) + bursts = [] + + async def runner(b): + bursts.append({"phase": getattr(b, "_phase", None), "laser": b._laser_config}) + + res = await run_temp_change_burst_protocol( + orch, + "emb1", + 25.0, + frames=3, + bursts_before=1, + bursts_after=1, + lock_timeout_s=5.0, + poll_s=0.001, + burst_runner=runner, + ) + assert client.laser == "ALL OFF" and client.led == "Open" + assert client.setpoint == 25.0 + assert all(b["laser"] == "ALL OFF" for b in bursts) # every burst brightfield + assert len(bursts) >= 3 # before + >=1 during + after + ets = [e[0] for e in orch.events] + assert EventType.TEMP_PROTOCOL_STARTED in ets + assert EventType.TEMPERATURE_SETPOINT_CHANGED in ets + assert EventType.TEMP_PROTOCOL_COMPLETED in ets + assert res["locked"] is True +``` +- [ ] **Step 2: run, expect FAIL** +- [ ] **Step 3: implement** — append to `temperature_protocol.py`: +```python +from gently.app.orchestration.exclusive import BurstAcquisition +from gently.core.event_bus import EventType + + +async def run_temp_change_burst_protocol( + orchestrator, + embryo_id, + target_setpoint_c, + *, + frames=60, + mode="1hz", + num_slices=1, + bursts_before=1, + bursts_after=1, + lock_timeout_s=600.0, + poll_s=2.0, + burst_runner=None, +): + client = orchestrator.client + if burst_runner is None: + + async def burst_runner(b): + await b.run(orchestrator) + + async def one_burst(phase): + b = BurstAcquisition( + embryo_id, + frames=frames, + mode=mode, + num_slices=num_slices, + temperature_provider=getattr(orchestrator, "_temperature_provider", None), + laser_config="ALL OFF", + ) + b._phase = phase + await burst_runner(b) + + locked = False + error = None + cancelled = False + try: + await client.set_laser_config("ALL OFF") + await client.set_led("Open") + orchestrator._emit_event( + EventType.TEMP_PROTOCOL_STARTED, + { + "embryo_id": embryo_id, + "target_setpoint_c": target_setpoint_c, + "frames": frames, + "bursts_before": bursts_before, + "bursts_after": bursts_after, + }, + ) + for _ in range(bursts_before): + await one_burst("before") + await client.set_temperature(target_setpoint_c) + orchestrator._emit_event( + EventType.TEMPERATURE_SETPOINT_CHANGED, + {"embryo_id": embryo_id, "to": target_setpoint_c}, + ) + loop = asyncio.get_event_loop() + t0 = loop.time() + while True: + await one_burst("during") + try: + st = str((await client.get_temperature()).get("state", "")) + except Exception: + st = "" + if "LOCKED" in st: + locked = True + break + if loop.time() - t0 >= lock_timeout_s: + break + for _ in range(bursts_after): + await one_burst("after") + except asyncio.CancelledError: + cancelled = True + raise + except Exception as exc: + error = str(exc) + logger.exception("temp-change burst protocol failed") + finally: + orchestrator._emit_event( + EventType.TEMP_PROTOCOL_COMPLETED, + {"embryo_id": embryo_id, "locked": locked, "cancelled": cancelled, "error": error}, + ) + return {"locked": locked, "cancelled": cancelled, "error": error} +``` +- [ ] **Step 4: run, expect PASS**; `pytest -q` clean. +- [ ] **Step 5: commit** — `feat(tactic): temp-change burst protocol driver (brightfield before/during/after)` + +--- + +### Task 5: Agent tool + +**Files:** Create `gently/app/tools/temperature_protocol_tools.py` (or add to an existing tools module — follow the `@tool` pattern). Test: `tests/test_temp_protocol_tool.py`. + +**Interfaces:** Produces a `run_temp_change_burst_protocol` agent tool that resolves orchestrator+client from `context`, launches the driver via `asyncio.create_task`, returns a started message; validates embryo/client presence. + +- [ ] **Step 1: failing test** — assert the tool, given a context with a fake orchestrator/client, creates a task and returns a "started" string; given no client, returns an error without creating a task. +> Confirm the real `@tool` decorator + context helpers (`ctx_get(context,"client")`, `require_agent`, `require_timelapse_orchestrator`) from an existing tool (e.g. `gently/app/tools/temperature_tools.py`); mirror them. Write the test to the real registration shape. +- [ ] **Step 2: run, expect FAIL** +- [ ] **Step 3: implement** — mirror an existing tool: resolve `orchestrator` + `client`, guard None (return error dict/string), `asyncio.create_task(run_temp_change_burst_protocol(orchestrator, embryo_id, target_setpoint_c, frames=frames, bursts_before=bursts_before, bursts_after=bursts_after))`, return `f"Temp-change burst protocol started for {embryo_id} → {target_setpoint_c} C"`. +- [ ] **Step 4: run, expect PASS** +- [ ] **Step 5: commit** — `feat(tactic): run_temp_change_burst_protocol agent tool` + +--- + +### Task 6: Observability in strategy_snapshot + +**Files:** Modify `gently/ui/web/strategy_snapshot.py` (`_replay_timeline` ~line 627; mirror the burst-phase handling at ~762/777). Test: `tests/test_temp_protocol_snapshot.py`. + +**Interfaces:** Consumes the timeline subtypes (`temp_protocol_started/completed`, `setpoint_changed`). Produces, in the snapshot: a `temp_protocol` band (open on started, close on completed) and a `setpoint_changes` list. + +- [ ] **Step 1: failing test** — feed `_replay_timeline` (or `build_strategy_snapshot` over a temp `timeline.jsonl`) a sequence: `temp_protocol_started`, `setpoint_changed(to=25)`, `burst_started/completed`, `temp_protocol_completed`; assert the snapshot exposes a temp_protocol span and a setpoint change of 25. +> Confirm the real `_replay_timeline` input/output shape from the existing burst handling; match the snapshot dict structure it already produces. +- [ ] **Step 2: run, expect FAIL** +- [ ] **Step 3: implement** — in `_replay_timeline`, handle the three subtypes: on `temp_protocol_started` open a band (record start t + params); on `temp_protocol_completed` close it; on `setpoint_changed` append `{t, to}` to a `setpoint_changes` list in the snapshot. Surface them in the returned snapshot dict next to the existing phases. +- [ ] **Step 4: run, expect PASS**; `pytest -q` clean. +- [ ] **Step 5: commit** — `feat(tactic): surface temp-protocol band + setpoint changes in strategy snapshot` + +--- + +## Self-Review +- §2.1 brightfield burst → Task 2; §2.2 wait-for-lock → Task 3; §2.3 driver → Task 4; §2.4 events → Tasks 1 & 6; §2.5 tool → Task 5. ✓ +- Open confirmations (explicit): the timeline map value-shape (Task 1), the burst `run` acquire_burst call site (Task 2), the `@tool` + context helpers (Task 5), the `_replay_timeline` shape (Task 6). Each names a fallback. +- Type consistency: `laser_config="ALL OFF"` everywhere; event names `TEMPERATURE_SETPOINT_CHANGED`/`TEMP_PROTOCOL_STARTED`/`TEMP_PROTOCOL_COMPLETED` across Tasks 1/4/6; driver returns `{locked,cancelled,error}`. diff --git a/docs/superpowers/plans/2026-06-29-embryo-roles-observability.md b/docs/superpowers/plans/2026-06-29-embryo-roles-observability.md new file mode 100644 index 00000000..cdbe76dc --- /dev/null +++ b/docs/superpowers/plans/2026-06-29-embryo-roles-observability.md @@ -0,0 +1,43 @@ +# Embryo roles/strain + multi-embryo Operations observability (D2) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development. Steps use `- [ ]`. + +**Goal:** Add a per-embryo `strain` field, refine roles to *use* (add lineaging + a subject/reference `class`), and build the Operations roster lens that reads existing roles + strain. Detector stays on role this pass (full detector→strain separation is tracked future work — spec §4). + +**Architecture:** Reuse gently's existing `role` (`roles.REGISTRY`, `EmbryoState.role`, `embryo.yaml`). Add `strain` alongside role; extend the registry; expose roles to the frontend; render a role+strain roster lens above the operation spine. + +## Global Constraints +- REUSE the existing role concept — do NOT invent a new taxonomy. Extend `gently/harness/roles.py:REGISTRY` (`EmbryoRole` dataclass) and read role via the existing accessors (`EmbryoState.role`, `EmbryoInfo["role"]`, `get_role`/`REGISTRY.get`, `/api/embryos/positions`). +- Strain is a FREE-FORM string per embryo (no registry this pass). Coexists with plan-level strain/genotype overrides. +- Roles render with their REAL `REGISTRY` `ui_color`/`ui_icon` (magenta test, cyan calibration, lineaging's own) — never invented colors. +- `class` ('subject'|'reference') is an attribute ON `EmbryoRole`, derived from role; test/unassigned→subject, calibration/lineaging→reference. +- Backward compatible: embryos without strain → None; plans without role-scope render as D today. +- Detector stays wired to role THIS PASS (spec §4 documents the full separation as future work — do not implement it here). +- Git hygiene: stage only your files by explicit path; never `git add -A`. + +--- + +### Task 1: Per-embryo `strain` field +**Files:** Modify `gently/core/store_types.py` (`EmbryoInfo` — add `strain: str | None`); `gently/harness/state.py` (`EmbryoState` — add `strain: str | None = None`); `gently/core/file_store.py` (`register_embryo` — accept `strain=None`, write/coalesce it in embryo.yaml like `role`); `gently/ui/web/routes/data.py` (`/api/embryos/positions` ~:689 — add `"strain": emb.get("strain")`). Test: `tests/test_embryo_strain.py`. +- [ ] Confirm how `role` is threaded through `register_embryo` (`file_store.py:507-580`, write at :576, coalesce at :564) and EmbryoState (`state.py:138`); mirror it for `strain`. Confirm the positions endpoint shape (`data.py:661-696`). +- [ ] TDD: register an embryo with `strain="pan-nuclear GFP"` → get_embryo returns it; update coalesces; absent → None; positions endpoint includes strain. `pytest tests/test_embryo_strain.py -v`; `pytest -q` clean. Commit `feat(d2): per-embryo strain field`. + +### Task 2: Roles refined — lineaging + subject/reference class +**Files:** Modify `gently/harness/roles.py` (add `class_: str = "subject"` — or `klass`/`role_class` to avoid the `class` keyword — to `EmbryoRole`; add a `lineaging` entry to `REGISTRY`; set `class_` on each role: test/unassigned→subject, calibration/lineaging→reference; give lineaging its own ui_color/ui_icon/default_cadence/detector kept None or nuclear like calibration). Test: `tests/test_roles_registry.py` (extend if exists). +- [ ] Add the `class_` field (default "subject") to `EmbryoRole`; set it on all REGISTRY entries; add `lineaging` (reference, distinct ui_color e.g. a teal/green, ui_icon, cadence). Keep `detector_name` as-is on each role (staged). Confirm nothing else constructs EmbryoRole positionally in a way the new field breaks. +- [ ] TDD: `REGISTRY["lineaging"].class_=="reference"`; `REGISTRY["test"].class_=="subject"`; `REGISTRY["calibration"].class_=="reference"`; `get_role("lineaging")` works; existing roles/fields unchanged. `pytest tests/test_roles_registry.py -v`; `pytest -q` clean. Commit `feat(d2): roles-as-use — add lineaging + subject/reference class`. + +### Task 3: `/api/roles` route + role-scoped tactic scope +**Files:** Create `gently/ui/web/routes/roles.py` (`GET /api/roles` → `{roles:[{name,description,class_,ui_color,ui_icon,default_cadence_seconds}]}` from `list_roles()`/REGISTRY; register in `routes/__init__.py`). Modify `gently/app/tools/operation_plan_tools.py` (allow `scope.mode=='role'` + `scope.role` in validation — accept a REGISTRY key); add a pure resolver `resolve_scope_embryos(scope, roster_or_embryos) -> list[str]` (in a small module or operation_plan_tools) mapping mode=role→embryo_ids by role. Test: `tests/test_roles_route.py`, `tests/test_role_scope.py`. +- [ ] `/api/roles` mirrors `routes/tactic_library.py` (simple list route, graceful). The resolver maps `{mode:'role',role:'test'}` against a list of embryos-with-roles → the matching ids; global→all, embryos→explicit. Validation accepts mode=role with a valid role key. +- [ ] TDD: route returns the registry incl. lineaging + class_; resolver resolves role→ids, global→all, embryos→explicit, unknown role→[]. `pytest tests/test_roles_route.py tests/test_role_scope.py -v`; `pytest -q` clean. Commit `feat(d2): /api/roles route + role-scoped tactic scope resolver`. + +### Task 4: Operations roster lens (frontend) +**Files:** Modify `gently/ui/web/static/js/experiment-overview.js` (add a roster lens above the operation spine: fetch `/api/embryos/positions` + `/api/roles`, group embryos by role `class_` (Subjects foregrounded, References compact) then by role, each row `id · role chip (REGISTRY ui_color/ui_icon) · strain · cadence-phase chip · current tactic (from the plan's role-scoped tactics) · state`; render tactic-node scope by role using the resolver/role labels); `gently/ui/web/static/css/experiment.css` (the `.ops-roster*` classes, using the role colors from the API, not hardcoded). Reference: the validated prototype `scratchpad/d2proto/index.html` (regrounded to real role colors). +- [ ] Build the roster lens reading the real endpoints + role metadata (colors from `/api/roles`, not invented); class split → role groups → strain; spine nodes show role-scope ("→ test · E01.."). Backward compat: no embryos/roles → omit the lens, spine renders as D. `node --check`; build/refresh the opsv3 (or d2) Chrome harness with the real files for the controller to audit. Commit `feat(d2): Operations roster lens — embryos by role + strain`. + +## Self-Review +- Strain→T1; roles/class→T2; roles route + role-scope→T3; roster lens→T4. ✓ +- Open confirmations: register_embryo/EmbryoState role threading (T1), EmbryoRole construction sites (T2), the route/resolver pattern (T3), the embryos+roles endpoints + plan cross-reference for current-tactic (T4). +- Type consistency: `strain` str|None across model/store/endpoint; `class_` on EmbryoRole + in /api/roles + read by the renderer; role keys consistent across REGISTRY, scope.role, resolver, renderer. +- Staged: detector stays on role (spec §4 future work referenced, not implemented). diff --git a/docs/superpowers/plans/2026-06-29-manual-mode-dual-camera.md b/docs/superpowers/plans/2026-06-29-manual-mode-dual-camera.md new file mode 100644 index 00000000..96e4c141 --- /dev/null +++ b/docs/superpowers/plans/2026-06-29-manual-mode-dual-camera.md @@ -0,0 +1,38 @@ +# Manual mode B2 — dual-camera + laser-preset browser + timelapse form Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development. Steps use `- [ ]`. + +**Goal:** Extend B1's single-camera manual mode with a laser-preset browser, dual-camera (side A/B) config, and a manual timelapse-config form — building the headless parts now (live dual-view + real acquisition are rig-deferred). + +**Architecture:** New `require_control` proxy routes wrapping existing client/device-layer + agent-tool paths; device_factory registers a second camera defensively; UI additions to `#devices-view-manual` / `devices.js`. + +## Global Constraints +- HEADLESS-buildable parts only; mark RIG-DEFERRED parts (live dual view, real acquisition/timelapse) as noted in the spec — don't fake hardware. +- Backward compatible: single-camera rigs must still start (defensive HamCam2 registration); the manual-view-entry laser-off safety (B1 I3) stays intact. +- Laser preset list already exists: `GET /api/devices/laser/configs` (data.py:526). Reuse it; add only the set proxy + UI. +- Proxy routes mirror the existing `routes/data.py` `require_control` pattern (e.g. `/api/devices/laser/off` :508). Device-layer/client calls mirror existing ones. +- UI extends `#devices-view-manual` (index.html ~:720-835) + `DevicesManager` in `devices.js`. +- Git hygiene: stage only your files by explicit path; never `git add -A`. + +--- + +### Task 1: Laser-preset browser +**Files:** Modify `gently/ui/web/routes/data.py` (add `POST /api/devices/laser/config` `require_control` → `client.set_laser_config(name)`, mirror `/laser/off` :508); `gently/ui/web/templates/index.html` (the Illumination group ~:800 — replace the static `#devices-ls-laser-status` indicator with a `` populated from `GET /api/devices/laser/configs`; on change POST the + chosen preset. Keep "ALL OFF" the safe default + the existing manual-view-entry laser-off safety (don't + remove the I3 guard — selecting a preset is an explicit user action). +- **Rig-deferred:** the actual laser firing (the preset just calls `setConfig` on the rig). + +## 2. Dual-camera config +- **Backend (headless):** register a second `DiSPIMCamera("HamCam2")` as `devices["camera_b"]` in + `device_factory.py` — DEFENSIVELY (only if the camera is in the core's loaded devices; skip + log + otherwise, so single-camera rigs still start). Add a `side` field ('A'|'B') to `_ls_params` + + `handle_lightsheet_params`; `_ensure_lightsheet_sequence_sync` picks `camera` vs `camera_b` by side and + restarts the sequence on side change (reuse the exposure-change restart path). New `GET /api/devices/cameras` + endpoint listing available camera roles (A always; B if registered). +- **UI (headless):** a "Side A / B" selector in the manual rail → carries `side` on the live/params POST. +- **Rig-deferred:** live DUAL view via the "Multi Camera" fusion device (live-only) + dual-side acquisition + (two parallel `startSequenceAcquisition` + tag demux). v1 = single live stream, switchable side. + +## 3. Timelapse config form +- **Backend (headless):** new `POST /api/devices/timelapse/start` (`require_control`) proxy wrapping the + agent path `start_adaptive_timelapse(embryo_ids, stop_condition, interval_seconds, condition_value, + monitoring_mode)` (validate params; resolve the orchestrator like the agent tool does). The volume + geometry (num_slices/exposure/galvo±/piezo±/laser_config/power) is captured in the form + passed through + / persisted to per-embryo calibration where applicable. +- **UI (headless):** a collapsible "Timelapse" panel in the manual rail gathering cadence/stop/embryos/ + monitoring_mode + the volume geometry, reading `GET /api/devices/scan_geometry` + `/api/devices/laser/configs` + for defaults. A "Start timelapse" submit → the new proxy. +- **Rig-deferred:** the actual timelapse run + galvo/piezo motion. + +## 4. Out of scope / deferred +- Live Multi-Camera dual view + dual-side acquisition demux (rig). +- Saving timelapse configs as reusable presets (could reuse the tactic-library/plan-template later). +- Per-line laser power UI beyond the preset (the clamps in `optical.py` still apply). + +## 5. Testing +- Laser-preset: the POST proxy (TestClient + mock client asserts `set_laser_config(name)`); `node --check` + + Chrome audit of the dropdown populated from a stubbed configs endpoint. +- Dual-camera: device_factory registers camera_b against a FAKE core that has HamCam2 (and skips when + absent); the `side` param threads into `_ls_params` + selects the camera; `/api/devices/cameras` lists + roles; `node --check` + Chrome audit of the side selector. +- Timelapse: the start proxy validates + calls the orchestrator path (mock); the form gathers + posts the + params; `node --check` + Chrome audit of the form. +- All three: backward compatible (single-camera rig unaffected; the laser-off safety intact). diff --git a/docs/superpowers/specs/2026-06-29-session-plan-linking-design.md b/docs/superpowers/specs/2026-06-29-session-plan-linking-design.md new file mode 100644 index 00000000..4b3be8f6 --- /dev/null +++ b/docs/superpowers/specs/2026-06-29-session-plan-linking-design.md @@ -0,0 +1,64 @@ +# Design: Session ↔ plans link/delink (sub-project F) + +Status: design 2026-06-29 (after recon + user steering). Branch `feature/session-plan-linking` (off D2). +Lets a session link to MULTIPLE plan items, with link/delink from BOTH the Plans tab and the session +view. "Repo/base plans" is DEFERRED (user has ideas — separate follow-on). + +## 0. What exists (recon) +- Session↔**campaign**: many-to-many (`SessionIntent.campaign_ids` list; `link/unlink_session_campaign`). +- Session↔**plan-item**: `PlanItem.session_ids` is a LIST (a session can already appear under multiple + items in storage) — but a session's *own* notion of "its plans" is a single `active_plan_item_id` + pointer, and `SessionIntent` stores no plan-item ids. +- `attach_session_to_plan` appends to `item.session_ids` (via `link_plan_item_session`) but overwrites + the single active pointer; `detach_session_from_plan` only clears the pointer (NOT a data delink). +- Plans tab (`campaigns.js:789-802`) shows a per-item read-only Sessions list ("No linked sessions"). +- NO link/delink endpoint or UI anywhere; NO `unlink_plan_item_session`; session endpoint returns no linkage. + +## 1. The model — source of truth = `PlanItem.session_ids` +A session's linked plan items = the reverse query over plan items whose `session_ids` includes the +session. No new field on SessionIntent (avoids dual source of truth). Multi-plan falls out naturally +(a session can be in many items' `session_ids`). The campaign edge stays on `SessionIntent.campaign_ids`. + +- **Link** session↔plan-item: `link_plan_item_session(item_id, session_id)` (exists, appends) + + `link_session_campaign(session_id, item.campaign_id)` (exists). +- **Delink**: NEW `unlink_plan_item_session(item_id, session_id)` — remove the session from + `item.session_ids` (+ clear the back-compat `session_id` if it pointed there); fire `_notify_plan_change`. + Campaign edge: leave it unless no other item of that campaign links the session (refinement — for v1, + delink only touches the plan-item edge; campaign delink stays the existing separate control). +- **Session's plans**: NEW `get_plan_items_for_session(session_id) -> list[PlanItem]` (reverse query + across `get_active_campaigns` → `get_plan_items` → filter `session_id in item.session_ids`). + +## 2. Endpoints (mirror `routes/campaigns.py`) +- `POST /api/campaigns/{cid}/items/{item_id}/sessions` body `{session_id}` → link (link_plan_item_session + + link_session_campaign). Returns the updated item sessions. +- `DELETE /api/campaigns/{cid}/items/{item_id}/sessions/{session_id}` → delink (unlink_plan_item_session). +- `GET /api/sessions/{id}/plans` → the session's linked plan items (id, title, campaign_id, status) via + `get_plan_items_for_session`. (A new sub-route; leaves the existing session payload untouched.) + +## 3. UI — both surfaces +### 3.1 Plans tab item-detail (`campaigns.js` ~:789-802) +The existing per-item Sessions list gains: a **[+ link session]** control (a picker of recent sessions +from `/api/sessions`) and a **[delink]** button per listed session. Calls the POST/DELETE endpoints, +re-renders the item detail. Empty state keeps "No linked sessions" + the link control. + +### 3.2 Session / Operations view — "Linked plans" panel +A panel (in the Operations/experiment view header or a session detail strip) listing the session's +linked plan items (from `/api/sessions/{id}/plans`): each row `plan item title · campaign · status · +[delink]`, plus **[+ link to a plan]** (a picker of plan items from the active campaigns). Symmetric +with 3.1 — link/delink from either side; both hit the same endpoints + refresh. + +## 4. Out of scope (deferred) +- **Repo/base plans** — the user has ideas; a separate follow-on (a repo plans library / seed). Noted, + not built here. +- Campaign-edge auto-cleanup on plan delink (v1 leaves the campaign link; refine later). +- Reworking `attach_session_to_plan`/`detach` agent tools beyond what's needed — the data-layer + delink (`unlink_plan_item_session`) is added; wiring a `detach` that calls it is a small optional add. + +## 5. Testing +- Data layer: `unlink_plan_item_session` removes the session (+ back-compat session_id); idempotent on + absent; `get_plan_items_for_session` reverse-query returns the right items across campaigns; multi-plan + (a session under 2 items) round-trips. +- Endpoints: POST links (item.session_ids gains it + campaign linked); DELETE delinks; GET returns the + session's plans; mirror `tests/test_*route*` with a mock store. +- UI: link/delink controls on both surfaces (node --check + Chrome audit of the Plans-tab item detail + + the session "Linked plans" panel); link from one side shows on the other after refresh. diff --git a/docs/superpowers/specs/2026-06-30-bottom-cam-operator-surface-design.md b/docs/superpowers/specs/2026-06-30-bottom-cam-operator-surface-design.md new file mode 100644 index 00000000..4228e84b --- /dev/null +++ b/docs/superpowers/specs/2026-06-30-bottom-cam-operator-surface-design.md @@ -0,0 +1,123 @@ +# Bottom-cam → SPIM Operator Surface ("Operate" view) — Design + +Date: 2026-06-30 +Status: Approved (proceed to implementation) +Branch: feature/temperature-operations-all + +## Purpose + +A single, professional, guided operator surface for the manual bottom-camera → +SPIM acquisition workflow, replacing the scattered current UX (bottom cam on the +Map view, SPIM controls on the Manual view, embryo marking on the Embryos page). +Mirrors how the operator physically works the rig: + +1. Focus the bottom objective on the embryos. +2. Mark **all** embryos in one pass (a single FOV holds them). +3. Per embryo: center → lower the SPIM head → focus the SPIM (LED on) → acquire. + +This is sub-project **A** (the spine). Two data flywheels hang off it and are +specified separately: **B** marking→localization labels (retire SAM), **C** +manual-focus→autofocus-validation. A exposes the hook points (confirm, focus +score) they will tap, but does not implement them. + +## Settled decisions + +- **Home:** a new `Operate` view in the device tab, alongside Map/Details/3D/Manual. + One purpose per view: Map stays a passive spatial monitor; Manual stays raw + knobs. The Detect/Center/enlarge controls already added to Map migrate to Operate; + Map keeps only read-only embryo dots. +- **Focus control:** software **nudge** buttons (± fixed steps), hard-fenced to + the axis limits (F-drive floor 30 µm). **No autofocus** (objective-crash risk). + Live focus-score readout to assist. +- **Marking:** batch — mark all embryos on one frozen full-res frame. **Positions + only, no roles.** Roles are a separate, later, optional concern. +- **SPIM focus step:** inline in Operate (lightsheet live + galvo/piezo/LED nudges), + not a handoff to Manual. +- **Single source of truth:** the canonical `experiment.embryos` list (already wired + via EMBRYOS_UPDATE / /api/embryos/current). Detect feeds it through a + human-confirm step, not a side list. +- **UI quality:** treat as a design pass (frontend-design), not a port of the + amateur marking canvas. + +## Architecture + +New device-tab view `operate` (devices.js view list becomes +`['operate','map','details','optical3d','manual']`), three vertical zones: + +1. **Survey** — enlarged bottom-cam live; bottom-Z focus nudge (fenced) + live + focus score; Detect (SAM → candidates) or click-to-mark on a frozen frame; + Confirm. +2. **Embryos** — the one canonical list, each row with a state chip and select. +3. **Acquire** (selected embryo) — Center → Lower SPIM head (F-drive, fenced) → + inline lightsheet live + galvo/piezo/LED nudges + focus score → Acquire volume. + +Per-embryo state machine (client-side, keyed by embryo id; persistence deferred): + +``` +marked ──Center──▶ centered ──Lower SPIM + focus──▶ focused ──Acquire──▶ imaged +``` + +### Component reuse + +| Need | Reuse | New | +|---|---|---| +| Bottom-cam live + enlarge | camera panel (built) | move into Operate | +| Mark-all on frozen frame | MarkingManager interaction logic | re-homed canvas, redesigned, positions-only | +| Embryo list + Center | SSOT list + stage/move (built) | per-embryo state chips | +| SPIM focus | Manual galvo/piezo/LED/lightsheet-live endpoints | inline placement | +| Acquire | /api/devices/acquire/volume | — | +| Bottom-Z + F-drive nudge | DiSPIMZstage / DiSPIMFDrive device classes | device-factory wiring, polling, fenced endpoints | +| Focus score | analysis/core.calculate_focus_score | inject into camera stream payloads | +| Register marks (agent-free) | experiment.add_embryo | register-on-confirm endpoint | + +## New backend endpoints (web routes proxy → device layer) + +- `GET /api/devices/stage/bottom_z` · `POST /api/devices/stage/bottom_z/nudge {delta}` + — read + fenced nudge of the bottom-camera focus Z (DiSPIMZstage). +- `GET /api/devices/spim/fdrive` · `POST /api/devices/spim/fdrive/nudge {delta}` + — read + fenced nudge of the SPIM-head F-drive (floor 30 µm; report distance-to-floor). +- `POST /api/devices/detect_embryos` (revised) — return SAM candidates + `{embryos:[{pixel_x,pixel_y,stage_x_um,stage_y_um,confidence}], stage_position}`; + no auto-register. +- `POST /api/devices/embryos/confirm {markers:[{pixel_x,pixel_y}], stage_position, + pixel_size_um, objective_mag}` — pixel→stage, register each into experiment.embryos + (role 'unassigned'), fire EMBRYOS_UPDATE. Agent-free. +- Focus score injected into existing bottom-cam + lightsheet SSE payloads + (`focus_score` field), computed server-side on the full frame. + +Device-layer additions: instantiate DiSPIMZstage in the device factory when +present; add bottom-Z and F-drive to the slow position poller; fenced nudge +handlers (clamp/reject out-of-range). + +## Safety / error handling + +- All Z moves fenced server-side; device classes hard-enforce limits; out-of-range + → 4xx, surfaced in UI. Nudges are bounded single steps — no autonomous/repeated + moves. +- Device layer down → 503; controls disabled with clear state. +- Frozen-frame capture failure → error toast, stay in Survey. Confirm with zero + markers → disabled. +- F-drive: never below floor; show distance-to-floor. + +## Testing + +- Backend (TDD): pixel→stage in register-on-confirm (reuse coordinate tests); + fenced Z endpoints reject out-of-range; focus-score payload shape. +- Frontend: launch with the gently_perception shim; Chrome MCP drives + detect→mark→confirm→EMBRYOS_UPDATE, per-embryo state transitions, out-of-range + nudge blocked; screenshots + UI audit against the professional bar. +- Rig-only: real Z moves, real SPIM focus, SAM on a live frame. + +## Out of scope (separate sub-projects) + +- B: persist (frame + pixel markers + roles) as localization labels; benchmark + classical/trained detectors vs SAM. +- C: poll missing Z axes for passive focus-trace logging + offline validator. + (A wires the Z axes for read/move, which C extends to logging.) + +## MVP boundary + +Ship the Operate view end-to-end with maximal reuse: Survey (live + fenced bottom-Z ++ Detect/mark-all + Confirm), the SSOT list with state chips + Center, Acquire zone +(F-drive nudge + inline SPIM focus controls + Acquire). Per-embryo state client-side. +Defer label/focus-trace persistence (B/C). diff --git a/docs/superpowers/specs/2026-06-30-operate-tactics-integration-design.md b/docs/superpowers/specs/2026-06-30-operate-tactics-integration-design.md new file mode 100644 index 00000000..0b950d34 --- /dev/null +++ b/docs/superpowers/specs/2026-06-30-operate-tactics-integration-design.md @@ -0,0 +1,130 @@ +# Operate → Tactics/Timelapse Integration ("Phase C: Run") — Design + +Date: 2026-06-30 +Status: Approved (build all 3 phases) +Branch: feature/temperature-operations-all +Source: Opus expert workflow (tactics + timelapse + agent/resolution + UI study → 3 candidates → synthesis) + +## Problem + +The Operate view dead-ends: `confirmMarks()` registers positions-only embryos +(role `unassigned`) into `experiment.embryos` (the SSOT), and `onEmbryosUpdate` +auto-dives into the manual per-embryo loop. There is no path from "embryos +marked" to the agentic timelapse or to a tactic/plan. + +Underneath, all four imaging surfaces (Operate manual loop, Manual-view timelapse +form, agent tools, Operations spine) already share **one engine** +(`TimelapseOrchestrator`) and **one language** (the per-session *Operation Plan* +of **tactics**). Operate is wired to none of them. `resolve_scope_embryos` +(role_scope.py) — the scope→embryos resolver — has zero callers; it was built for +exactly this hand-off. + +## Decisions (user) + +- **Build all three phases** (not just the first slice). +- **Adaptive timelapse default monitoring mode = `idle`** (pure fixed-cadence; + operator opts into expression/pre-terminal monitoring explicitly). +- **Live run is monitored in-Operate** (the rail flips to a compact read-only + run-spine), with a deep-link to the Operations tab. + +## Design — Phase C "Run" on the Operator Spine + +Stepper gains a third node: **① Focus → ② Mark → ③ Run.** After Confirm, instead +of auto-diving into the manual loop, the stepper advances to ③ Run and the right +rail shows a **Run chooser**. The **tactic** is the single unifying object: every +run mode emits exactly one tactic scoped to the marked set +(`scope.mode='embryos', embryo_ids=[marked]`). + +| Mode | Behavior | Tactic kind | +|---|---|---| +| **A — Manual one-by-one** | the existing Phase-B b1–b5 loop, now reached from the chooser | `oneshot` (cosmetic; keeps the spine coherent) | +| **B — Adaptive timelapse** | inline form: interval, stop condition, monitoring_mode (default **idle**); reuses `/api/devices/timelapse/start` with explicit `embryo_ids=[marked]` | `standing_timelapse` (+ optional `reactive_monitor` layered on) | +| **C1 — From library** | apply a saved tactic, scope re-pointed to the marked set | template's kind, via `apply_tactic` | +| **C2 — Continue a plan** | resume_plan candidate → `execute_plan_item(item_ref, embryo_ids=[marked])` + `seed_operation_plan_from_plan_item` | tactics seeded from `ImagingSpec.tactics` | +| **C3 — Hand to agent** | open AgentChat preloaded with the roster; agent authors + starts the plan | agent-authored via `declare_operation_plan` | + +**Running:** the rail flips to a compact read-only **run-spine** (reuse +`experiment-overview.js` `_renderOpsTactic`/`_renderOperationSpine`): state-colored +tactic cards + live readouts + **Pause/Stop** (`pause_timelapse`/`stop_timelapse`) ++ **Open in Operations** deep-link. Optional `set_autonomy('ask'|'auto')`. + +**Roles wrinkle (load-bearing):** marking stays positions-only, but +`expression_monitoring` rules scope to `role=='test'` (subject) — so role-scoped +monitoring matches zero just-marked embryos unless roles are set. The Run chooser +shows a **role chip strip** (all marked default to **Subject**, flip any to +**Reference**); choosing a non-manual mode applies roles via a new thin +`POST /api/embryos/roles`. Roles are assigned **at Run, not at marking** — +consistent with the "marking is positions-only" rule. + +## Keystone new component: the Tactic Executor + +`gently/app/orchestration/tactic_executor.py` — +`execute_tactic(session, tactic)`: `resolve_scope_embryos(scope, roster)` → +dispatch by `kind` to `orchestrator.start` / `enable_monitoring_mode` / +`queue_burst` (later `acquire_volume` / temp protocols), threading `tactic_id` → +`transition_tactic('active')` + merge live binds. This makes the tactics language +*executable* (not just descriptive) and is `resolve_scope_embryos`'s first caller. +It centralizes the `kind`→tool mapping the agent also uses (no duplication). + +## Integration points (verified code seams) + +- `operate.js` `onEmbryosUpdate` — stop auto-selecting embryo 1; on first confirm + advance stepper to ③ Run + render the chooser. +- `operate.js` `renderStep` single-driver — host Phase C chooser + live run-spine + as new render branches (`data-active='c0'` / running) without disturbing a1/b*. +- `POST /api/devices/embryos/confirm` (`data.py`) — unchanged (positions-only SSOT). +- `POST /api/devices/timelapse/start` (`data.py`) — Mode B reuses verbatim with + `embryo_ids=[marked]`; Phase 2 ADDS tactic seeding (closes the `data.py:~1004` + plan-auto-link TODO). `volume_geometry` stays NOT forwarded (RIG-DEFERRED). +- `TimelapseOrchestrator.start/enable_monitoring_mode/queue_burst` — the engine + the executor dispatches into; holds marked `EmbryoState` refs (zero copy). +- `resolve_scope_embryos` (role_scope.py) — Tactic Executor is its first caller. +- `start_adaptive_timelapse` (timelapse_tools.py) — add `tactic_id` for lifecycle + symmetry (the other start/stop tools already have it). +- `OperationPlanUpdater` — already maps BURST_COMPLETE→done, + EMBRYO_CADENCE_CHANGED/TRIGGER_FIRED→bind; drives the run-spine once tactics + carry `tactic_id`. +- resolution dispatch (`bridge._dispatch_resolution_pick`) + `execute_plan_item` + + `seed_operation_plan_from_plan_item` — Modes C2/C3 + session guard. +- `experiment-overview.js` `_renderOperationSpine`/`_renderOpsTactic` — reused for + the in-Operate run-spine. +- `AgentChat.togglePanel`/`runCommand` — Mode C3. + +## New backend + +1. **Tactic Executor** (`gently/app/orchestration/tactic_executor.py`) — the keystone (above) + unit tests. +2. **`POST /api/embryos/roles`** (thin) — reuse `assign_embryo_roles` internals → set `EmbryoState.role` + fire EMBRYOS_UPDATE. Default marked→subject mandatory. +3. **Tactic seeding on `/api/devices/timelapse/start`** — declare+seed `standing_timelapse` (+ `reactive_monitor`) and transition active (closes the TODO). Additive, minimal, idempotent. +4. **`tactic_id` on `start_adaptive_timelapse`** — lifecycle symmetry. +5. **Tactic structure schema extension** (`operation_plan_tools._validate_tactics`) — add `stop_condition`/`condition_value`/`monitoring_mode`/`interval` to `standing_timelapse`/`reactive_monitor` so a tactic is self-describing for the executor. +6. **Session guard** — ensure a live session/orchestrator before Phase C runs (orchestrator is None without one); reuse `should_enter_resolution`/bootstrap. + +## Phasing + +- **Phase 1 (cheap slice):** Operate Phase C scaffold (③ Run node, stop auto-dive, + Run chooser) + **Mode B** (reuses `/timelapse/start`, no new backend) + thin + `POST /api/embryos/roles` + role chip strip + in-Operate run-spine. Working + marking→adaptive-timelapse hand-off. +- **Phase 2 (tactics integrity):** tactic seeding on `/timelapse/start`; + `tactic_id` on `start_adaptive_timelapse`; tactic structure schema extension; + Mode A `oneshot`. +- **Phase 3 (keystone + breadth):** Tactic Executor (+ tests) powering Mode C1 + (library); Mode C2 (continue a plan); Mode C3 (hand to agent); `set_autonomy` + in the run-spine. + +## Testing + +- Backend (TDD): Tactic Executor (scope resolution + kind dispatch, mocked + orchestrator); roles route; schema validator extension. +- Frontend: shim + Chrome MCP — drive mark→Confirm→Run chooser→role assign→Mode B + start→run-spine; verify stepper/chooser/run-spine; UI audit. +- Adversarial code-review workflow over the full diff before merge. + +## Rig-only / honesty flags + +Real stage motion + acquisition stay RIG-DEFERRED (orchestrator calls +`client.acquire_volume` directly; "Bluesky" framing is aspirational). +`volume_geometry` not forwarded by `/timelapse/start`. The `oneshot` manual tactic +is cosmetic (no orchestrator mechanism backs it). Timelapse start needs a live +session/orchestrator — unavailable in the hardware-free shim, so Mode B's actual +start is rig/session-verified; the UI flow + tactic emission are shim-verifiable. diff --git a/docs/superpowers/specs/2026-07-01-settings-panel-thermalizer-config-design.md b/docs/superpowers/specs/2026-07-01-settings-panel-thermalizer-config-design.md new file mode 100644 index 00000000..8527455b --- /dev/null +++ b/docs/superpowers/specs/2026-07-01-settings-panel-thermalizer-config-design.md @@ -0,0 +1,60 @@ +# Settings Panel — editable ACUITYnano thermalizer + config visibility — Design + +Date: 2026-07-01 +Status: Approved (build all phases) +Branch: feature/temperature-operations-all (→ #72) +Source: Opus audit workflow + two implementation-reference passes. + +## Problem / audit + +The gently "Settings" panel is **100% client-side display preferences** — every control writes browser `localStorage` (`gently-dashboard-config` + `gently-theme`); `settings.js` makes **zero** backend calls. The **ACUITYnano thermalizer connection** (serial COM / MQTT-HiveMQ / mock) is in **no GUI** — it lives in `config/config.yml` `temperature:`, read once at device-layer boot; changing transport/port/creds = edit YAML + restart. Naming trap: the Vitals "Temperature model (20/25 °C)" radio is a *developmental-timing reference curve*, not the hardware setpoint. + +Two-process: viz (FastAPI :8080) proxies to the device layer (aiohttp :60610) via `DiSPIMClient`; the controller lives only in the device-layer process. + +## Decisions (user) + +- Build **all phases**. +- Apply mode: **try live hot-swap, fall back to restart-required**. +- **Mock-SIM: dev/debug only** — not selectable in the production panel. + +## Design + +New server-backed **"Hardware / Thermalizer"** section (separate from the localStorage panel; explicit "machine-wide, saved on the server" note). Fields grounded in `temperature.py`: +- Backend radio **Serial | MQTT (HiveMQ)** (Mock hidden unless a dev flag). Serial: `com_port` (required), `baud_rate` (115200). MQTT: `broker`, `port` (8883), `user`, `password` (write-only, `••••`, never echoed; blank = embedded HiveMQ SIM). Common: `stabilize_timeout` (600), `feedback_peltier`. +- **Test connection** (non-committing): build transient backend → `read()`/`get_system_state` → `close()` → report; never swaps the live device. +- **Apply**: live hot-swap — build the NEW controller first (`create_temperature_controller`), keep the old on failure, swap `self.devices["temperature"]`, `old.close()`. Guard: **409 if `self.RE.state != "idle"`** or a `set()` worker holds the controller lock (mid-ramp/mid-plan swap corrupts a live `bps.mv(temperature,…)`). If teardown/rebuild can't apply live, persist + "restart the device layer to apply" banner. +- **Persistence**: sidecar `config/config.local.yml` (`temperature:` block) merged over `config.yml` at device-layer boot — preserves `config.yml`'s comments; password written only when a new non-redacted value is submitted. + +**Effective-config viewer** (Phase 2, read-only, secrets redacted): ports/hosts, model IDs, storage base_path + derived dirs, mmconfig/mmdirectory, organism/hardware, switchbot name, coverslip, XY safety envelope (from the live device-state stream), mesh instance-id + cert fingerprint, timeouts, ML params, `ux_v2`. **Never expose** (redact/omit): `ANTHROPIC_API_KEY`, `GENTLY_CONTROL_TOKEN`, `mesh_key.pem`, MQTT creds. + +## Call chain (per new capability) +Browser `fetch('/api/devices/temperature/config…')` → FastAPI `routes/data.py` (`require_control` on mutations) → `_resolve_client().()` → `client.py` `_api_*` → device-layer `handle_*` → `self.devices["temperature"]`. + +## Phasing / changes + +**Phase 0 (visibility + test):** +- device-layer: `GET /api/temperature/config` (current `temperature` block, password redacted, + live backend + `read()` state); `POST /api/temperature/config/test` (transient backend probe). Register in `on_start` (~:3394). +- client: `get_temperature_config`, `test_temperature_config`. +- viz: `GET /api/devices/temperature/config` (read-only), `POST /api/devices/temperature/config/test` (`require_control`). +- UI: read-only Hardware/Thermalizer section + Test button (a separate `ThermalizerSettings` JS object, isolated from `SettingsManager`); relabel the Vitals "Temperature model" field. + +**Phase 1 (editable + live reconnect):** +- device-layer: `POST /api/temperature/config` (validate; 409 guard; build-new-before-swap; sidecar persist; return live state); boot-merge sidecar over `config.yml` `temperature` (after `yaml.safe_load` ~:227). +- client: `set_temperature_config`. viz: `POST /api/devices/temperature/config` (`require_control`). +- UI: editable Serial/MQTT form (Mock dev-only), Apply (no auto-save), applied-live vs restart-required banner. + +**Phase 2 (visibility + prefs):** +- viz `GET /api/config/effective` (read-only, redacted) + a read-only "Effective config" viewer in the panel. +- Server-side dashboard-pref **defaults** + reset/export/import (viz route storing rig defaults in a file; `settings.js` layers over localStorage). +- Restart-required editors for SAFE `settings.py` knobs (timeouts, mesh timing, ML, `ux_v2`, NCBI) via a `config/settings.local.yml`/env override read at launch, with an explicit "restart required" path (never mutate the frozen `settings` singleton live). If the launcher-override mechanism proves out-of-scope, ship the viewer + pref-defaults and defer the editors. + +## Safety +- Never echo the MQTT password (redact on GET; persist only on new value). Redact all secrets in the effective-config viewer. +- `require_control` on every config write/test proxy. +- 409 reconfigure guard while RE running / lock held; build-new-before-swap so a bad config never leaves the rig with no thermalizer. +- Keep the 0.0–99.9 °C clamp in both layers; GUI can't widen it. +- Sidecar persistence (not `config.yml` rewrite); restrict perms on any file holding the plaintext MQTT password. +- Restart-required for frozen `settings.py` values — write override + prompt restart, never live-mutate. + +## Rig-only / honesty +The vendor SDK (`acuitynano_precision_thermalizer_*`) isn't on PyPI and isn't installed in the hardware-free shim, so **serial/MQTT construction, Test, and live-swap are rig-verified**; the shim path exercises routes + validation + the mock backend + UI flow. Live hot-swap's clean teardown per transport needs on-rig confirmation (fallback = restart banner). diff --git a/docs/superpowers/specs/2026-07-02-unified-launcher-design.md b/docs/superpowers/specs/2026-07-02-unified-launcher-design.md new file mode 100644 index 00000000..3e24228c --- /dev/null +++ b/docs/superpowers/specs/2026-07-02-unified-launcher-design.md @@ -0,0 +1,136 @@ +# Unified Launcher — single entry point + device-layer process management — Design + +Date: 2026-07-02 +Status: **Initial design (RFC) — PARKED.** A starting point to build on, not a finished +design; direction + mockup done, implementation deferred to a future session. +Idea credit: **Magdalena** — the single unified launcher. +Branch: `feature/unified-launcher` (off `development`). + +## Problem + +Starting gently today is two commands in two terminals: `python start_device_layer.py` +(the hardware server) then `python launch_gently.py` (agent + web). gently can't +start or stop the device layer — it only connects to it as an HTTP client — so there's +no in‑app way to bring hardware up/down. Goal: make `launch_gently.py` the **single** +entry point, with a dead‑simple launch gate and the ability to start/stop the device +layer from gently. + +## How the two processes relate (today) + +- `start_device_layer.py` boots the hardware control server (aiohttp) on **port 60610** + (`DEVICE_PORT`) and runs the configured hardware module's `create_device_layer()`. + Independent OS process. +- `launch_gently.py` boots the agent + web/viz server and creates an HTTP client + (`QueueServerClient → http://127.0.0.1:60610`); `client.is_connected` drives the + "Device: ○ offline" banner. The two only talk over HTTP. + +## The launch gate — BARE BASIC (the key decision) + +The launch screen answers exactly two questions, nothing more: + +1. **Microscope hardware — on/off.** On → gently starts + connects the device layer. + Off → software‑only (analysis, planning, reviewing saved data). +2. **AI agent (API) — on/off.** On → chat, perception, planning (uses the API key). + Off → UI‑only. + +Then **Let's go →**. That's the whole screen. Everything else — organism, hardware +module, device port, SAM device, session resume — is a **default** (config / last‑used), +reachable behind a muted **"Advanced options"** disclosure and in **Settings**. It is +NOT on the gate: if you're not using hardware, you should never be asked which hardware. + +Visual reference (dark, sharp, sleek): `docs/superpowers/mockups/2026-07-02-launcher-gate.html` +— two toggle cards (`Microscope hardware`, `AI agent`) + `Let's go →` + `Advanced +options · remembers your choice`. + +## Process model — managed child subprocess + +A **`DeviceLayerSupervisor`** in the gently process spawns `start_device_layer.py` as a +**child** (`subprocess.Popen([sys.executable, "start_device_layer.py", "--port", …, +"--sam-device", …])`), holds its handle, monitors liveness, captures its log, and stops +it. Ownership is what makes start/stop‑from‑UI and no‑orphans work. +- API: `start(config)`, `stop(force=False)`, `status()`, log tail, `atexit`/signal cleanup. +- **External device layer still supported:** if one is already running on 60610, gently + connects and shows it as "external (not managed)" — it just won't stop what it didn't start. +- Rejected alternatives: independent processes + a `/shutdown` endpoint (weaker ownership, + more parts); in‑process device layer (a hardware crash would take down the UI). + +## Boot flow (defer‑init) + +`launch_gently` boots a minimal web server → shows the **launch gate** (prefilled from +last choice) → on **Let's go**, it initializes per the two toggles: start the agent (if +API on) and start the device layer via the supervisor (if hardware on) → then the landing +("Good afternoon…") → dashboard. Deferring heavy init until the gate is submitted is what +lets a web screen control boot‑level behavior (agent on/off). + +## Runtime control + +A **device‑layer panel in the Devices tab** mirrors the gate's hardware block at runtime: +live status (running / stopped / external / crashed), Start / Stop, and a tail of the +device‑layer log — so you can bring hardware up/down without restarting gently. + +## Stop safety (graceful + mid‑run guard) + +Stop → if a plan/acquisition is active, warn + require explicit confirm ("hardware is +active — stop anyway?") → SIGTERM (clean shutdown) → SIGKILL fallback (~5 s). Reuses the +409 + `"blocked"` pattern from the thermalizer work. Startup failure (hardware off) +surfaces the existing plain‑language `_render_startup_failure` diagnosis, not a traceback. + +## Persistence + +`config/launch.local.json` (gitignored) remembers the two toggles (+ any advanced values) +so the gate is prefilled every boot. + +## Non‑goals (YAGNI) + +Auto‑restart‑on‑crash loops; multiple / remote device layers (mesh already covers +cross‑machine); managing an externally‑started device layer's lifecycle; putting +organism/hardware/port/SAM/session on the gate (they're defaults + Advanced/Settings). + +## Phasing (when resumed) + +1. `DeviceLayerSupervisor` + the runtime Devices start/stop panel + the **hardware toggle** + on a minimal gate. Delivers "no more separate `start_device_layer.py`" immediately. +2. Defer‑init boot refactor + the **agent toggle** + persistence + "Advanced options". + +## Open questions to fold in (this is initial work) + +This is a starting point — the following must be worked through as it grows: + +- **Background startup.** After "Let's go", the device‑layer startup should run in the + **background (non‑blocking)** while gently proceeds to the dashboard — not block the gate + until the device connects. (To confirm + design the progress signalling.) +- **Usable during startup.** gently must stay **usable while the device layer boots** — + software features (planning, review, analysis) available immediately; hardware‑dependent + actions gated behind a "device starting…" state until connected. +- **On‑demand startup‑sequence screen.** There must be a screen to watch the device + **startup sequence** on demand — a device console (in the Devices panel) streaming the + `start_device_layer` boot log + per‑stage connection progress. +- **Shutdown from that same screen.** That console is also the **shutdown control** — start + the sequence, watch it, and stop the device from one place. +- **Guided shutdown housekeeping.** Shutdown is a **sequence, not just a kill** — e.g. it + can prompt the operator to **reset the F drive** (post‑shutdown drive/housekeeping) as a + step in the flow. + +## Future direction — Windows desktop app (Electron) + +Bigger fold‑in to weigh: package the unified launcher as a **desktop app for Windows**, +potentially **Electron**, so gently is a double‑click application instead of terminal +commands. Electron would *be* the launcher shell — it owns and spawns the Python backend +(`launch_gently`) and the device‑layer child, renders the existing web UI in a native +window (the launch gate is the first screen), and gets native process lifecycle +(children killed on quit), a tray/menu, and a real installer + auto‑update. + +Tradeoffs for later: bundling/shipping the Python environment (embeddable Python / +PyInstaller), app size, auto‑update, and keeping the **web UI the single source of truth** +(Electron stays a thin shell — no UI logic moves into it). A lighter native wrapper +(**pywebview** or **Tauri**) is the fallback if Electron's footprint is too heavy. This is +a packaging/architecture decision, not required for the launcher's first cut — but the +managed‑child process model here is exactly what an Electron shell would take over. + +## Decisions log (from brainstorming) + +- Launcher model: **web startup screen + runtime panel**. +- Boot flow: **show the gate every boot, remember last choices** (one click to proceed). +- Stop: **graceful SIGTERM + mid‑run guard + SIGKILL fallback**. +- Scope: **fold in gently's own options** — but as a **bare‑basic 2‑question gate** + (hardware, agent), with the rest behind Advanced/Settings. diff --git a/docs/superpowers/specs/2026-07-13-session-replay-design.md b/docs/superpowers/specs/2026-07-13-session-replay-design.md new file mode 100644 index 00000000..c689bdf7 --- /dev/null +++ b/docs/superpowers/specs/2026-07-13-session-replay-design.md @@ -0,0 +1,141 @@ +# Session replay + agent postmortem — design + +**Date:** 2026-07-13 · **Status:** approved design, pre-implementation +**Branch:** `feature/session-replay` · **Owner:** Keshu + +## Purpose + +Record every real interaction with the gently web UI — clicks, inputs, DOM changes — +so a session can be replayed post-hoc and, critically, **postmortemed by a Claude Code +agent**. Two consumers, in priority order: + +1. **Agent postmortem** — Claude Code walks a session's logs, correlates UI actions + with what the agent/microscope were doing at the same timestamps, and answers + questions like "what did the operator do before the crash" or "where do users + hesitate at the launch gate." +2. **Human replay** — a scrubber page that plays the session back visually. + +This extends the ui_crawler philosophy (agent-readable artifacts: text first, PNGs on +demand) from synthetic story runs to real sessions. Playwright traces cover what *we* +drive; this covers what *humans* do. + +## Decisions (settled) + +- **Engine:** self-rolled [rrweb](https://github.com/rrweb-io/rrweb) — no PostHog / + OpenReplay / SaaS. Vendored `rrweb.min.js`, no CDN (offline microscope PC). +- **Always on.** Recording starts with every page load. Kill switch: `GENTLY_REPLAY=0`. +- **Size is immaterial; quality of data is what matters.** No compression (saves + main-thread CPU), no retention cap in v1. Raw JSONL on disk. +- **Zero performance compromise.** The app always wins over the recording (see + *Performance posture*). +- **Detached from everything else.** Nothing in gently imports the recorder; removal + = delete one template include + one router file. + +## Architecture — three layers + +All artifacts live in the file store, per session: + +``` +{GENTLY_STORAGE_PATH}/sessions/{session}/ui-replay/ + actions.jsonl # layer 1 — semantic action log (agent-first) + rrweb-{tab}.jsonl # layer 2 — full rrweb event stream (human-first) + meta.yaml # tab ids, user agent, viewport, start/end times +``` + +### Layer 1 — semantic action log (primary for agents) + +One JSON line per meaningful event: + +```json +{"t": "2026-07-13T10:42:03.120Z", "tab": "a3f2", "action": "start-timelapse", + "target": "button#start-tl", "label": "Start", "route": "/dashboard", "params": {}} +``` + +Sources: delegated capture-phase click listener keyed on `data-action` attributes +(fall back to tag+id+text for unannotated controls), route changes, form submits. +Cost is near zero. This is what Claude reads end-to-end; timestamps join against +agent traces, predictions, device-layer logs, and timelapse state already on disk. + +### Layer 2 — rrweb event stream (full fidelity) + +Standard rrweb recording: initial DOM snapshot + mutations + sampled mouse/scroll + +inputs. Config: + +- `recordCanvas: false` — canvas is the one expensive capture, and gently's heavy + pixels don't need it: microscope frames are `` elements whose `src` URLs point + at the file store, which is permanent. Replay re-resolves them by reference. +- Mouse/scroll sampling at rrweb defaults (throttled). +- Batched flush: buffer in memory, flush every ~5 s via `requestIdleCallback` + + `fetch(keepalive)`; `sendBeacon` on `pagehide` so tab close loses ≤ one batch. +- One stream per browser tab (`rrweb-{tab}.jsonl`), tab id random per load. + +### Layer 3 — the bridge: on-demand frames for agents + +An agent cannot watch a replay; it reads text and inspects pixels on demand. +`tools/session_replay/render_frame.py --session S --t TIMESTAMP` replays the rrweb +stream headlessly (rrweb-player under Playwright — reusing the ui_crawler stack) and +screenshots the chosen instant to PNG. Claude's postmortem loop: read `actions.jsonl` +→ cross-reference backend logs → render only the moments that matter → look. + +A `gently-session-postmortem` skill (sibling of `gently-debugging`) documents the +walk: where artifacts live, how to join timestamps, when to render frames. + +## Server side + +One router file, e.g. `gently/ui/web/routes/replay.py`: + +- `POST /replay/ingest` — append a batch to the session's JSONL. Append-only writes, + no parsing, no validation beyond size caps. Negligible cost. +- `GET /replay/{session}` — static player page (vendored `rrweb-player`) that loads + the streams. Used post-hoc only. + +No imports from gently core beyond the file-store path helper. The recorder JS is one +`', '', html, flags=_re.DOTALL) - text = _re.sub(r']*>.*?', '', html, flags=_re.DOTALL) - text = _re.sub(r'<[^>]+>', ' ', text) - text = _re.sub(r'\s+', ' ', text).strip() + text = _re.sub(r"]*>.*?", "", html, flags=_re.DOTALL) + text = _re.sub(r"]*>.*?", "", html, flags=_re.DOTALL) + text = _re.sub(r"<[^>]+>", " ", text) + text = _re.sub(r"\s+", " ", text).strip() if len(text) > 15000: text = text[:15000] + "\n\n[... truncated ...]" @@ -899,7 +939,7 @@ async def _fetch_url_text(url: str) -> Optional[str]: return None -def _read_pdf_file(path: str) -> Optional[str]: +def _read_pdf_file(path: str) -> str | None: """Extract text from a local PDF file using pymupdf if available.""" import os @@ -908,6 +948,7 @@ def _read_pdf_file(path: str) -> Optional[str]: try: import fitz # pymupdf + doc = fitz.open(path) pages = [] for page in doc: @@ -920,15 +961,16 @@ def _read_pdf_file(path: str) -> Optional[str]: return text if text.strip() else None except ImportError: - logger.info("pymupdf not installed — cannot extract PDF text. " - "Install with: pip install pymupdf") + logger.info( + "pymupdf not installed — cannot extract PDF text. Install with: pip install pymupdf" + ) return None except Exception as e: logger.warning(f"PDF extraction failed for {path}: {e}") return None -async def _pubmed_abstract(pmid: str) -> Optional[Dict]: +async def _pubmed_abstract(pmid: str) -> dict | None: """Fetch article metadata + abstract from PubMed.""" import aiohttp @@ -948,6 +990,7 @@ async def _pubmed_abstract(pmid: str) -> Optional[Dict]: xml_text = await resp.text() import xml.etree.ElementTree as ET + root = ET.fromstring(xml_text) article = root.find(".//PubmedArticle") @@ -998,26 +1041,26 @@ async def _pubmed_abstract(pmid: str) -> Optional[Dict]: return None -async def _resolve_reference(reference: str) -> Dict: +async def _resolve_reference(reference: str) -> dict: """Parse a reference string and determine what kind of input it is. Returns a dict with keys: type, pmid, doi, url, path, query """ - import re import os + import re ref = reference.strip() result = {"type": "unknown", "raw": ref} # PMID - m = re.match(r'^(?:PMID[:\s]*)?(\d{6,9})$', ref, re.IGNORECASE) + m = re.match(r"^(?:PMID[:\s]*)?(\d{6,9})$", ref, re.IGNORECASE) if m: result["type"] = "pmid" result["pmid"] = m.group(1) return result # DOI - m = re.search(r'(10\.\d{4,}/[^\s]+)', ref) + m = re.search(r"(10\.\d{4,}/[^\s]+)", ref) if m: result["type"] = "doi" result["doi"] = m.group(1).rstrip(".,;)") @@ -1029,13 +1072,13 @@ async def _resolve_reference(reference: str) -> Dict: result["url"] = ref if ref.startswith("http") else "https://" + ref # Extract PMID from PubMed URLs - m = re.search(r'pubmed\.ncbi.*?/(\d{6,9})', ref) + m = re.search(r"pubmed\.ncbi.*?/(\d{6,9})", ref) if m: result["type"] = "pmid" result["pmid"] = m.group(1) # Extract PMCID from PMC URLs - m = re.search(r'/pmc/articles/(PMC\d+)', ref) + m = re.search(r"/pmc/articles/(PMC\d+)", ref) if m: result["type"] = "pmcid" result["pmcid"] = m.group(1) @@ -1054,7 +1097,7 @@ async def _resolve_reference(reference: str) -> Dict: return result -async def _search_pmid(query: str) -> Optional[str]: +async def _search_pmid(query: str) -> str | None: """Search PubMed for a citation string and return the best PMID. Tries multiple query strategies to handle imprecise citations like @@ -1066,7 +1109,7 @@ async def _search_pmid(query: str) -> Optional[str]: # Detect "Author et al YEAR topic" pattern m = _re.match( - r'^([A-Z][a-z]+)\s+(?:et\s+al\.?\s+)?(\d{4})?\s*(.*)?$', + r"^([A-Z][a-z]+)\s+(?:et\s+al\.?\s+)?(\d{4})?\s*(.*)?$", query.strip(), ) if m: @@ -1076,8 +1119,10 @@ async def _search_pmid(query: str) -> Optional[str]: # Fix common organism names in topic topic_fixed = _re.sub( - r'\bC\.?\s*elegans\b', '"Caenorhabditis elegans"', - topic, flags=_re.IGNORECASE, + r"\bC\.?\s*elegans\b", + '"Caenorhabditis elegans"', + topic, + flags=_re.IGNORECASE, ) # Strategy 1: author + organism MeSH + quoted topic (most specific) @@ -1087,15 +1132,11 @@ async def _search_pmid(query: str) -> Optional[str]: ) # Strategy 2: author + organism MeSH (no topic — topic may not be in title) - strategies.append( - f'{author}[author] AND "Caenorhabditis elegans"[Mesh]' - ) + strategies.append(f'{author}[author] AND "Caenorhabditis elegans"[Mesh]') # Strategy 3: author + year + topic (exact year, may be wrong) if year and topic_fixed: - strategies.append( - f'{author}[author] AND {year}[pdat] AND {topic_fixed}' - ) + strategies.append(f"{author}[author] AND {year}[pdat] AND {topic_fixed}") # Strategy 4: author + year only if year: @@ -1103,8 +1144,10 @@ async def _search_pmid(query: str) -> Optional[str]: # Strategy 5: original query with organism name fix fixed = _re.sub( - r'\bC\.?\s*elegans\b', '"Caenorhabditis elegans"', - query, flags=_re.IGNORECASE, + r"\bC\.?\s*elegans\b", + '"Caenorhabditis elegans"', + query, + flags=_re.IGNORECASE, ) if fixed != query: strategies.append(fixed) @@ -1122,6 +1165,7 @@ async def _search_pmid(query: str) -> Optional[str]: # Try each strategy import aiohttp + for attempt in unique: try: async with _http_session() as session: @@ -1145,7 +1189,7 @@ async def _search_pmid(query: str) -> Optional[str]: return None -async def _doi_to_pmid(doi: str) -> Optional[str]: +async def _doi_to_pmid(doi: str) -> str | None: """Resolve a DOI to a PMID via PubMed search.""" return await _search_pmid(f"{doi}[doi]") @@ -1177,7 +1221,7 @@ async def _doi_to_pmid(doi: str) -> Optional[str]: ) async def read_paper( reference: str, - context: Dict = None, + context: dict | None = None, ) -> str: """Read a scientific paper and return its content. @@ -1214,7 +1258,7 @@ async def read_paper( f"Try providing a PMID, DOI, or more specific citation." ) - if ref_type == "doi" and not pmid: + if ref_type == "doi" and not pmid and doi: status_lines.append(f"Resolving DOI: {doi}") pmid = await _doi_to_pmid(doi) if pmid: @@ -1225,15 +1269,11 @@ async def read_paper( # --- Step 2: Try local PDF first if file path --- - if ref_type == "file": + if ref_type == "file" and path: status_lines.append(f"Reading local PDF: {path}") text = _read_pdf_file(path) if text: - return ( - f"[Paper from local file: {path}]\n\n" - f"{text}\n\n---\n" - f"Source: local file" - ) + return f"[Paper from local file: {path}]\n\n{text}\n\n---\nSource: local file" else: return ( f"[Paper from local file: {path}]\n\n" @@ -1308,7 +1348,7 @@ async def read_paper( meta = await _pubmed_abstract(pmid) if meta: lines = [ - f"[Abstract only — full text not freely available]\n", + "[Abstract only — full text not freely available]\n", f"# {meta['title']}\n", f"**Authors:** {meta['authors']}", f"**Journal:** {meta['journal']} ({meta['year']})", @@ -1319,9 +1359,9 @@ async def read_paper( lines.append(f"\n## Abstract\n\n{meta['abstract']}") lines.append( - f"\n---\n" - f"*Full text not available through open access channels. " - f"If you have a PDF, provide the file path and I can read it.*" + "\n---\n" + "*Full text not available through open access channels. " + "If you have a PDF, provide the file path and I can read it.*" ) lines.append(f"\n*Resolution path: {' → '.join(status_lines)}*") diff --git a/gently/harness/plan_mode/tools/templates.py b/gently/harness/plan_mode/tools/templates.py index 006c4c95..03557477 100644 --- a/gently/harness/plan_mode/tools/templates.py +++ b/gently/harness/plan_mode/tools/templates.py @@ -5,9 +5,7 @@ dependencies) for re-use with different strains, temperatures, etc. """ -from typing import Dict, Optional - -from ...tools.registry import tool, ToolCategory, ToolExample +from ...tools.registry import ToolCategory, ToolExample, tool @tool( @@ -33,8 +31,8 @@ async def save_plan_template( campaign_id: str, name: str, - description: str = None, - context: Dict = None, + description: str | None = None, + context: dict | None = None, ) -> str: """Save a campaign as a reusable template.""" agent = context.get("agent") if context else None @@ -67,7 +65,7 @@ async def save_plan_template( category=ToolCategory.UTILITY, ) async def list_templates( - context: Dict = None, + context: dict | None = None, ) -> str: """List available plan templates.""" agent = context.get("agent") if context else None @@ -112,8 +110,8 @@ async def list_templates( ) async def apply_template( template_id: str, - overrides: Dict = None, - context: Dict = None, + overrides: dict | None = None, + context: dict | None = None, ) -> str: """Instantiate a template into a new campaign.""" agent = context.get("agent") if context else None diff --git a/gently/harness/plan_mode/tools/validation.py b/gently/harness/plan_mode/tools/validation.py index 8e09802a..031918ca 100644 --- a/gently/harness/plan_mode/tools/validation.py +++ b/gently/harness/plan_mode/tools/validation.py @@ -5,11 +5,9 @@ detector validity, missing controls, dependency cycles, and completeness. """ -import json import logging -from typing import Dict, List, Optional, Set, Tuple -from ...tools.registry import tool, ToolCategory, ToolExample +from ...tools.registry import ToolCategory, ToolExample, tool logger = logging.getLogger(__name__) @@ -22,7 +20,7 @@ "num_slices": (10, 200), "exposure_ms": (5.0, 100.0), "laser_power_pct": (0.0, 100.0), - "interval_s": (10, None), # minimum 10s, no hard max + "interval_s": (10, None), # minimum 10s, no hard max "piezo_amplitude_um": (None, 200.0), # max ±200 μm } @@ -40,9 +38,9 @@ # Temperature scaling factors (relative to 20°C) TEMP_SCALE = { - 15.0: 24.0 / 14.0, # ~1.71× slower + 15.0: 24.0 / 14.0, # ~1.71× slower 20.0: 1.0, - 25.0: 10.0 / 14.0, # ~0.71× faster + 25.0: 10.0 / 14.0, # ~0.71× faster } CONTROL_KEYWORDS = {"control", "wildtype", "n2", "wt", "wild-type", "wild type"} @@ -52,7 +50,8 @@ # Helpers # --------------------------------------------------------------------------- -def _get_temp_factor(temperature_c: Optional[float]) -> float: + +def _get_temp_factor(temperature_c: float | None) -> float: """Return scaling factor for developmental timing at given temperature.""" if temperature_c is None: return 1.0 @@ -68,19 +67,19 @@ def _get_temp_factor(temperature_c: Optional[float]) -> float: return TEMP_SCALE[20.0] + frac * (TEMP_SCALE[25.0] - TEMP_SCALE[20.0]) -def _check_dependency_cycles(items) -> List[str]: +def _check_dependency_cycles(items) -> list[str]: """DFS-based cycle detection on the dependency graph.""" # Build adjacency list: item_id -> list of dependency IDs - adj: Dict[str, List[str]] = {} - id_to_title: Dict[str, str] = {} + adj: dict[str, list[str]] = {} + id_to_title: dict[str, str] = {} for item in items: adj[item.id] = list(item.depends_on) id_to_title[item.id] = item.title WHITE, GRAY, BLACK = 0, 1, 2 - color: Dict[str, int] = {nid: WHITE for nid in adj} - cycles: List[str] = [] - path: List[str] = [] + color: dict[str, int] = {nid: WHITE for nid in adj} + cycles: list[str] = [] + path: list[str] = [] def dfs(node: str): if node not in color: @@ -110,13 +109,16 @@ def dfs(node: str): return cycles -def _stage_order(stage_name: str) -> Optional[int]: +def _stage_order(stage_name: str) -> int | None: """Get ordinal position of a stage, or None if unrecognised.""" from gently_perception.organism import CELEGANS + stages = CELEGANS.stages aliases = { - "3fold": "pretzel", "threefold": "pretzel", - "1.5-fold": "1.5fold", "2-fold": "2fold", + "3fold": "pretzel", + "threefold": "pretzel", + "1.5-fold": "1.5fold", + "2-fold": "2fold", } normed = stage_name.lower().replace("-", "").replace(" ", "") name = aliases.get(normed, normed) @@ -127,9 +129,10 @@ def _stage_order(stage_name: str) -> Optional[int]: return None -def _normalise_stage(name: str) -> Optional[str]: +def _normalise_stage(name: str) -> str | None: """Normalise a stage name to canonical form, or None.""" from gently_perception.organism import CELEGANS + stages = CELEGANS.stages low = name.lower().strip() for s in stages: @@ -143,6 +146,7 @@ def _normalise_stage(name: str) -> Optional[str]: # Tool # --------------------------------------------------------------------------- + @tool( name="validate_plan", description=( @@ -161,7 +165,7 @@ def _normalise_stage(name: str) -> Optional[str]: ) async def validate_plan( campaign_id: str, - context: Dict = None, + context: dict | None = None, ) -> str: """Validate a plan and return errors/warnings.""" agent = context.get("agent") if context else None @@ -178,14 +182,18 @@ async def validate_plan( if not items: return f"Campaign '{campaign.description}' has no plan items to validate." - errors: List[str] = [] - warnings: List[str] = [] + errors: list[str] = [] + warnings: list[str] = [] # Load detector presets for validation try: from gently.organisms import get_organism + org = get_organism() - presets_mod = __import__(f"gently.organisms.{org.ORGANISM_NAME}.detector_presets", fromlist=["get_detector_presets"]) + presets_mod = __import__( + f"gently.organisms.{org.ORGANISM_NAME}.detector_presets", + fromlist=["get_detector_presets"], + ) valid_detectors = set(presets_mod.get_detector_presets().keys()) except ImportError: valid_detectors = set() @@ -199,16 +207,31 @@ async def validate_plan( label = f"[{item.type.value}] '{item.title}'" # Check for control mentions - text_blob = " ".join(filter(None, [ - item.title, item.description, item.outcome, - ])).lower() + text_blob = " ".join( + filter( + None, + [ + item.title, + item.description, + item.outcome, + ], + ) + ).lower() if item.imaging_spec: - text_blob += " " + " ".join(filter(None, [ - item.imaging_spec.strain, - item.imaging_spec.genotype, - item.imaging_spec.reporter, - item.imaging_spec.success_criteria, - ])).lower() + text_blob += ( + " " + + " ".join( + filter( + None, + [ + item.imaging_spec.strain, + item.imaging_spec.genotype, + item.imaging_spec.reporter, + item.imaging_spec.success_criteria, + ], + ) + ).lower() + ) if any(kw in text_blob for kw in CONTROL_KEYWORDS): has_control = True @@ -221,13 +244,9 @@ async def validate_plan( if val is None: continue if lo is not None and val < lo: - errors.append( - f"{label}: {field_name}={val} below minimum {lo}" - ) + errors.append(f"{label}: {field_name}={val} below minimum {lo}") if hi is not None and val > hi: - errors.append( - f"{label}: {field_name}={val} exceeds maximum {hi}" - ) + errors.append(f"{label}: {field_name}={val} exceeds maximum {hi}") # Stage consistency if spec.start_stage and spec.stop_condition: diff --git a/gently/harness/prompts/manager.py b/gently/harness/prompts/manager.py index 23cb9291..1a104cd5 100644 --- a/gently/harness/prompts/manager.py +++ b/gently/harness/prompts/manager.py @@ -9,13 +9,14 @@ import json import logging from datetime import datetime -from typing import Dict, List, Optional +from typing import Any from gently.settings import settings -from .templates import build_system_prompt, build_context_message + from ..plan_mode.prompt import build_plan_prompt from ..resolution_mode.prompt import build_resolution_prompt from ..tools.registry import get_tool_registry +from .templates import build_system_prompt logger = logging.getLogger(__name__) @@ -33,21 +34,22 @@ def __init__(self, claude_client, model): self.model = model # Context summary caching - self._context_summary_cache: Optional[str] = None - self._context_summary_time: Optional[datetime] = None + self._context_summary_cache: str | None = None + self._context_summary_time: datetime | None = None self._context_summary_ttl: int = 300 # 5 minutes # Memory awareness caching - self._memory_awareness_cache: Optional[str] = None - self._memory_awareness_time: Optional[datetime] = None + self._memory_awareness_cache: str | None = None + self._memory_awareness_time: datetime | None = None self._memory_awareness_ttl: int = 600 # 10 minutes # Set by agent after construction self.context_store = None self.memory = None # AgentMemory instance - def update_system_prompt(self, experiment, client, mode: str, - context_summary: str = None) -> str: + def update_system_prompt( + self, experiment, client, mode: str, context_summary: str | None = None, perceiver=None + ) -> str: """ Rebuild system prompt with current experiment state and connection status. @@ -86,16 +88,19 @@ def update_system_prompt(self, experiment, client, mode: str, # Execution mode if client: connection_status = { - 'device_layer': client.is_connected, - 'sam_detection': client.has_sam, + "device_layer": client.is_connected, + "sam_detection": client.has_sam, } else: connection_status = None return build_system_prompt( - experiment, connection_status, context_summary, + experiment, + connection_status, + context_summary, memory_awareness=memory_awareness, microscope=client, + perceiver=perceiver, ) def get_tools_for_mode(self, mode: str, has_microscope: bool) -> list: @@ -142,17 +147,32 @@ def get_tools_for_mode(self, mode: str, has_microscope: bool) -> list: return [t for t in all_tools if t["name"] in resolution_tool_names] if mode == "plan": plan_tool_names = { - "create_campaign", "create_plan_item", "update_plan_item", - "link_plan_items", "propose_plan", "get_plan_status", + "create_campaign", + "create_plan_item", + "update_plan_item", + "link_plan_items", + "propose_plan", + "get_plan_status", "get_plan_item", - "move_plan_item", "delete_plan_item", "reorder_plan_items", - "update_phase", "delete_phase", + "move_plan_item", + "delete_plan_item", + "reorder_plan_items", + "update_phase", + "delete_phase", "export_plan", - "query_lab_history", "check_hardware_capability", - "search_literature", "search_strains", + "query_lab_history", + "check_hardware_capability", + "search_literature", + "search_strains", "validate_plan", - "batch_update_status", "batch_update_spec", - "save_plan_template", "list_templates", "apply_template", + "batch_update_status", + "batch_update_spec", + "snapshot_plan", + "list_plan_versions", + "restore_plan_version", + "save_plan_template", + "list_templates", + "apply_template", "ask_user_choice", } all_tools = registry.get_claude_schemas(has_microscope=False) @@ -165,14 +185,16 @@ def get_cached_memory_awareness(self) -> str: if not self.memory: return "" now = datetime.now() - if (self._memory_awareness_cache is None or - self._memory_awareness_time is None or - (now - self._memory_awareness_time).total_seconds() > self._memory_awareness_ttl): + if ( + self._memory_awareness_cache is None + or self._memory_awareness_time is None + or (now - self._memory_awareness_time).total_seconds() > self._memory_awareness_ttl + ): self._memory_awareness_cache = self.memory.get_awareness_summary() self._memory_awareness_time = now return self._memory_awareness_cache - def get_active_plan_summary(self) -> Optional[str]: + def get_active_plan_summary(self) -> str | None: """Get a summary of the active experimental plan, if any.""" if not self.context_store: return None @@ -190,13 +212,14 @@ def get_active_plan_summary(self) -> Optional[str]: f" ({status['completed']}/{status['total']} items done)" ) if status["next_actions"]: - lines.append(" Next: " + ", ".join( - a.title for a in status["next_actions"][:3] - )) + lines.append( + " Next: " + ", ".join(a.title for a in status["next_actions"][:3]) + ) if status["pending_decisions"]: - lines.append(" Decisions pending: " + ", ".join( - d.title for d in status["pending_decisions"] - )) + lines.append( + " Decisions pending: " + + ", ".join(d.title for d in status["pending_decisions"]) + ) return "\n".join(lines) if lines else None except Exception: return None @@ -219,37 +242,37 @@ def gather_context_data(self, experiment, timelapse_orch, timeline_mgr) -> dict: dict Context data including timelapse status, events, and detections """ - data = { - 'current_time': datetime.now().isoformat(), - 'timelapse_status': None, - 'recent_events': [], - 'recent_detections': [], - 'detection_reasoning': [], + data: dict[str, Any] = { + "current_time": datetime.now().isoformat(), + "timelapse_status": None, + "recent_events": [], + "recent_detections": [], + "detection_reasoning": [], } if timelapse_orch: try: status = timelapse_orch.get_status() - data['timelapse_status'] = { - 'state': status.status.value if status.status else 'unknown', - 'total_timepoints': status.total_timepoints or 0, - 'started_at': status.started_at.isoformat() if status.started_at else None, - 'embryo_count': len(status.embryos) if status.embryos else 0, + data["timelapse_status"] = { + "state": status.status.value if status.status else "unknown", + "total_timepoints": status.total_timepoints or 0, + "started_at": status.started_at.isoformat() if status.started_at else None, + "embryo_count": len(status.embryos) if status.embryos else 0, } except Exception as e: logger.debug(f"Could not get timelapse status: {e}") if timeline_mgr: try: - events = timeline_mgr.get_events(limit=20, session_id='current') - data['recent_events'] = [ + events = timeline_mgr.get_events(limit=20, session_id="current") + data["recent_events"] = [ { - 'type': e.event_subtype, - 'time': e.timestamp.isoformat(), - 'embryo': e.embryo_id, - 'detector': e.detector_name, - 'timepoint': e.timepoint, - 'confidence': e.confidence, + "type": e.event_subtype, + "time": e.timestamp.isoformat(), + "embryo": e.embryo_id, + "detector": e.detector_name, + "timepoint": e.timepoint, + "confidence": e.confidence, } for e in events ] @@ -258,32 +281,35 @@ def gather_context_data(self, experiment, timelapse_orch, timeline_mgr) -> dict: try: for embryo_id, embryo_state in experiment.embryos.items(): - if not hasattr(embryo_state, 'detection_results'): + if not hasattr(embryo_state, "detection_results"): continue for detector_name, results in embryo_state.detection_results.items(): recent_results = results[-3:] if len(results) > 3 else results for r in recent_results: - if r.get('detected'): - data['recent_detections'].append({ - 'detector': detector_name, - 'embryo': embryo_id, - 'timepoint': r.get('timepoint'), - 'confidence': r.get('confidence'), - }) - if r.get('reasoning'): - data['detection_reasoning'].append({ - 'detector': detector_name, - 'embryo': embryo_id, - 'timepoint': r.get('timepoint'), - 'reasoning': r.get('reasoning')[:500], - }) + if r.get("detected"): + data["recent_detections"].append( + { + "detector": detector_name, + "embryo": embryo_id, + "timepoint": r.get("timepoint"), + "confidence": r.get("confidence"), + } + ) + if r.get("reasoning"): + data["detection_reasoning"].append( + { + "detector": detector_name, + "embryo": embryo_id, + "timepoint": r.get("timepoint"), + "reasoning": r.get("reasoning")[:500], + } + ) except Exception as e: logger.debug(f"Could not get detection results: {e}") return data - async def generate_context_summary(self, experiment, timelapse_orch, - timeline_mgr) -> str: + async def generate_context_summary(self, experiment, timelapse_orch, timeline_mgr) -> str: """ Generate concise context summary using Haiku. @@ -300,22 +326,23 @@ async def generate_context_summary(self, experiment, timelapse_orch, """ raw_data = self.gather_context_data(experiment, timelapse_orch, timeline_mgr) - has_timelapse = raw_data['timelapse_status'] is not None - has_events = len(raw_data['recent_events']) > 0 - has_detections = len(raw_data['recent_detections']) > 0 + has_timelapse = raw_data["timelapse_status"] is not None + has_events = len(raw_data["recent_events"]) > 0 + has_detections = len(raw_data["recent_detections"]) > 0 if not (has_timelapse or has_events or has_detections): return "" - prompt = f"""Summarize the current microscopy session state in 2-3 sentences for another AI assistant. -Focus on: timelapse status (is it running, completed, or idle?), time since last activity, and notable detections. -Be factual and concise. + prompt = f"""Summarize the current microscopy session state in 2-3 sentences for +another AI assistant. Focus on: timelapse status (is it running, completed, or idle?), +time since last activity, and notable detections. Be factual and concise. Raw session data: {json.dumps(raw_data, indent=2, default=str)} Write a brief status summary. Examples: -- "Timelapse completed 10h ago with 233 timepoints. Hatching was detected at timepoints 175-193 with HIGH confidence." +- "Timelapse completed 10h ago with 233 timepoints. Hatching was detected at timepoints + 175-193 with HIGH confidence." - "Timelapse is currently running for embryo_1 at timepoint 45. No detections yet." - "No active timelapse. Last session had 50 timepoints, with comma stage detected at t=30." """ @@ -325,15 +352,14 @@ async def generate_context_summary(self, experiment, timelapse_orch, self.claude.messages.create, model=settings.models.fast, max_tokens=150, - messages=[{"role": "user", "content": prompt}] + messages=[{"role": "user", "content": prompt}], ) return response.content[0].text.strip() except Exception as e: logger.warning(f"Failed to generate context summary: {e}") return "" - async def get_cached_context_summary(self, experiment, timelapse_orch, - timeline_mgr) -> str: + async def get_cached_context_summary(self, experiment, timelapse_orch, timeline_mgr) -> str: """ Get context summary with caching (5-minute TTL). @@ -349,9 +375,11 @@ async def get_cached_context_summary(self, experiment, timelapse_orch, Cached or newly generated context summary """ now = datetime.now() - if (self._context_summary_cache is None or - self._context_summary_time is None or - (now - self._context_summary_time).total_seconds() > self._context_summary_ttl): + if ( + self._context_summary_cache is None + or self._context_summary_time is None + or (now - self._context_summary_time).total_seconds() > self._context_summary_ttl + ): self._context_summary_cache = await self.generate_context_summary( experiment, timelapse_orch, timeline_mgr ) @@ -382,6 +410,6 @@ def get_cached_system_prompt(self, system_prompt: str) -> list: { "type": "text", "text": system_prompt, - "cache_control": {"type": "ephemeral", "ttl": "1h"} + "cache_control": {"type": "ephemeral", "ttl": "1h"}, } ] diff --git a/gently/harness/prompts/templates.py b/gently/harness/prompts/templates.py index 55894c22..06b7cbad 100644 --- a/gently/harness/prompts/templates.py +++ b/gently/harness/prompts/templates.py @@ -2,17 +2,20 @@ System prompts and context builders for the Microscopy Agent """ -from typing import Dict, List -from ..state import ExperimentState -from gently.organisms import get_organism from gently.hardware import get_hardware +from gently.organisms import get_organism +from ..state import ExperimentState # Interactive choice guidance USER_INTERACTION_GUIDELINES = """ # Interactive User Choices — MANDATORY -CRITICAL RULE: Whenever you need to ask the user a question — whether it's a yes/no confirmation, a choice between options, or any question where the answer could be one of several discrete responses — you MUST use the `ask_user_choice` tool. NEVER present options as numbered text lists or bullet points. NEVER ask the user to type their choice as text when you could present selectable options instead. +CRITICAL RULE: Whenever you need to ask the user a question — whether it's a yes/no +confirmation, a choice between options, or any question where the answer could be one of +several discrete responses — you MUST use the `ask_user_choice` tool. NEVER present options +as numbered text lists or bullet points. NEVER ask the user to type their choice as text when +you could present selectable options instead. ## When to use ask_user_choice @@ -45,14 +48,21 @@ GOOD (always do this): Call the `ask_user_choice` tool. Example parameters: question: "What would you like to work on today?" - options: [{"id": "new", "label": "Start a new experiment"}, {"id": "resume", "label": "Resume a session"}] + options: [{"id": "new", "label": "Start a new experiment"}, + {"id": "resume", "label": "Resume a session"}] -The user interface renders these as an interactive picker with arrow-key navigation — much better UX than typing. -Do NOT write tool calls as XML tags or code blocks in your text — always invoke tools through the tool mechanism. +The user interface renders these as an interactive picker with arrow-key navigation — much +better UX than typing. +Do NOT write tool calls as XML tags or code blocks in your text — always invoke tools through +the tool mechanism. -IMPORTANT: This is not optional. ALWAYS use ask_user_choice when presenting choices or asking questions. The ONLY exception is when you need a completely free-form text response (like asking for a name or description). +IMPORTANT: This is not optional. ALWAYS use ask_user_choice when presenting choices or asking +questions. The ONLY exception is when you need a completely free-form text response (like +asking for a name or description). -Each option should be a specific, distinct choice. The picker automatically adds a free-text "Something else..." input at the bottom for custom responses, so your options can focus on the most likely concrete answers. +Each option should be a specific, distinct choice. The picker automatically adds a free-text +"Something else..." input at the bottom for custom responses, so your options can focus on +the most likely concrete answers. """ @@ -87,62 +97,32 @@ # CV Subagent capabilities CV_SUBAGENT = """ -# CV Subagent for Advanced Analysis - -For complex computer vision analysis, you have access to a specialized CV subagent via the `cv_analyze` tool. - -## IMPORTANT: Volume Required First! - -Before using cv_analyze or classify_embryo_stage, you MUST ensure the embryo has a volume acquired -in this session. If the user asks for cell counting, stage classification, or any analysis: - -1. Check if the embryo has been imaged (recent_images exists) -2. If NOT, acquire a volume first with `acquire_volume` -3. Then proceed with analysis - -Example workflow: -User: "Count the cells in embryo_3" -→ First: acquire_volume(embryo_id="embryo_3") # Get fresh data -→ Then: cv_analyze(intent="count cells", embryo_id="embryo_3") - -## When to use cv_analyze - -Use the CV subagent when you need: -- **Accurate stage classification** - It segments nuclei (Cellpose) and uses count + morphology for staging -- **Cell counting** - 3D segmentation gives precise nuclei counts, not visual estimates -- **Division tracking** - Tracks cells across timepoints, identifies division events -- **Morphology measurements** - Elongation ratio, circularity (important for comma/fold stages) -- **Anomaly detection** - Compares to expected developmental patterns - -## When NOT to use cv_analyze - -Don't use it for: -- Quick visual checks (use simple image viewing instead) -- Hatching detection (the hatching detector handles this) -- Basic "what stage is this?" if rough estimate is fine - -## How it works - -The CV subagent is itself an AI agent that: -1. Loads volume data from the data store -2. Segments with Cellpose/StarDist (nuclei count!) -3. Measures morphology (elongation for fold stages) -4. Adds scale bars and annotations -5. Uses Claude Vision with rich quantitative context - -This gives much more accurate results than just sending an image to vision. - -## Example usage - -User: "How many cells does embryo 1 have?" -→ First acquire_volume if needed, then cv_analyze with intent="count cells and nuclei" - -User: "What stage is embryo 2?" -→ If precision matters: acquire_volume then cv_analyze intent="classify developmental stage" -→ If quick check: view the image yourself - -User: "Track cell divisions over the last 5 timepoints" -→ cv_analyze with intent="track cell divisions" and timepoints=[t-4, t-3, t-2, t-1, t] +# Perception & Analysis + +You see and reason about embryo development through three channels: + +1. **Live perception (the perceiver).** During a timelapse a vision-language + perceiver classifies each acquired volume's developmental stage and tracks + each embryo's trajectory. Its current read is injected into your context + under "## Perception (live)" — stage, stability (how long it's held that + stage), time-in-stage, and a possible-arrest flag. Call + `get_recent_perceptions(embryo_id)` for the fuller picture: stage history, + trajectory, the arrest signal, and the perceiver's own reasoning. This is + your primary signal for "how is it developing?" and for deciding whether to + adapt acquisition. + +2. **On-demand vision (`analyze_volume`).** Ask Claude Vision a specific + question about an acquired volume (e.g. "is the reporter saturating?", + "describe the morphology"). Requires a volume in this session — acquire one + first with `acquire_volume` if none exists. + +3. **Stage tools.** `classify_embryo_stage` (a vision spot-check of the latest + image), `get_stage_history`, and `predict_hatching` — the latter two read the + live perceiver when available, so they work without a manual classify call. + +Prefer the live perception snapshot + `get_recent_perceptions` for routine +"what stage / is anything stuck" questions; reach for `analyze_volume` when you +need a specific visual judgement about a particular volume. """ @@ -222,7 +202,7 @@ | User describes... | Mode to install | |---|---| -| reporter expression, GFP/mCherry onset, "neurons lighting up", dopaminergic signal, anything where fluorescence turns on | `expression_monitoring` | +| reporter onset: GFP/mCherry, dopaminergic signal, neurons lighting up | `expression_monitoring` | | hatching timing, pre-hatch dynamics, "track until they hatch" | `pre_terminal_monitoring` | | plain imaging, exploratory, no specific signal target | none (idle) | @@ -242,6 +222,104 @@ """ +OPERATION_PLAN_GUIDANCE = """ +## Operation Plan — keep it current + +At experiment planning time, call `declare_operation_plan` with every tactic +you intend to run. Each tactic needs at minimum: `id` (short stable string), +`name`, `kind`, `state` (start as `"planned"`), `scope`, and `rationale`. +For richer display, populate a `live` object on the tactic: + +- `readouts` — list of `{label, value}` dicts for the instrument strip + (e.g. `{label: "cadence", value: "120 s"}`). +- `phases` — list of `{name, state, count, pips}` for scripted/phased tactics. +- Flat bound keys (`request_id`, `sustained_hz`, `setpoint`, `locked`, + `last_fired`, `new_phase`, …) are merged in by the updater as live telemetry + arrives; you can seed them at declaration time if the value is already known. + +### Allowed values — use these exact strings (renderer dispatches on them) + +**`kind`** ∈ one of: +| value | use when | +|---|---| +| `standing_timelapse` | continuous / periodic imaging running throughout | +| `reactive_monitor` | armed watcher that fires on a condition (signal, threshold) | +| `scripted_protocol` | fixed sequence of named phases (ramp, hold, recovery, …) | +| `exclusive_burst` | short high-cadence burst that blocks other acquisition | +| `oneshot` | single action (z-stack, snapshot, one-off step) | +| `custom` | anything that doesn't fit the above | + +**`state`** (tactic) ∈ `planned | active | done | paused` +Start every tactic as `"planned"`; advance to `"active"` when it begins, +`"done"` when it finishes, `"paused"` if suspended. + +**`scope`** — always an object with a `mode` key (never a bare list or string): +- `{"mode": "global"}` — applies to every embryo in the session +- `{"mode": "embryos", "embryo_ids": ["E01", "E02"]}` — specific embryo IDs +- `{"mode": "role", "role": "test"}` — all embryos carrying the named role + +**`live.phases[].state`** ∈ `todo | active | done` + +### Minimal tactic example + +```json +{ + "id": "t2", + "name": "Temperature ramp", + "kind": "scripted_protocol", + "state": "planned", + "scope": {"mode": "embryos", "embryo_ids": ["E01", "E02"]}, + "rationale": "25 → 16 °C step to trigger stress response", + "live": { + "readouts": [{"label": "setpoint", "value": "25 °C"}], + "phases": [ + {"name": "ramp", "state": "todo", "count": 0, "pips": []}, + {"name": "hold", "state": "todo", "count": 0, "pips": []} + ] + } +} +``` + +### Plan the whole roster — subjects AND references + +Experiments often require both subjects and references. Declare tactics for all +roster classes in the same `declare_operation_plan` call. + +| Roster class | Role value(s) | Planning note | +|---|---|---| +| Subjects | `test` | adaptive protocol + reactive tactics apply | +| References | `calibration`, `lineaging` | steady acquisition only; no adaptive protocol | + +**Reference tactic**: when the assay needs calibration or stage-clock embryos, declare +a `standing_timelapse` scoped to that role alongside the subject tactics: + +```json +{ + "id": "ref_acq", + "name": "Reference steady acquisition", + "kind": "standing_timelapse", + "state": "planned", + "scope": {"mode": "role", "role": "calibration"}, + "rationale": "Steady 5-min imaging of calibration embryos — stage timing + normalization" +} +``` + +**Surface role requirements** in the plan `goal` or a tactic `rationale`: state which +roles the run needs and roughly how many (e.g. *"needs ≥2 test subjects + ≥1 calibration +reference"*) so the operator knows what embryos to assign before the run begins. Never +plan only for `test` when the assay depends on reference embryos. + +Valid `scope.role` strings: `test` / `calibration` / `lineaging` / `unassigned`. + +Re-call `declare_operation_plan` (patch) whenever a tactic's state changes: +`"planned"` → `"active"` when you start it, `"active"` → `"done"` when it +finishes. This keeps the Operations view in the UI synchronized with reality. +Execution tools (`queue_burst`, `enable_monitoring_mode`, `stop_timelapse`, +`pause_timelapse`) also accept an optional `tactic_id` and flip the state +automatically — pass it when a tool maps cleanly to one tactic. +""" + + ADAPTIVE_TIMELAPSE = """ # Adaptive Timelapse System @@ -252,7 +330,8 @@ 1. **Non-blocking operation**: The timelapse runs independently - you can still chat with the user 2. **Per-embryo stop conditions**: Each embryo can stop at different times (e.g., when hatching) 3. **Dynamic intervals**: Adjust imaging frequency per-embryo during the experiment -4. **Detector integration**: Stop conditions triggered by visual detection (hatching, comma stage, etc.) +4. **Detector integration**: Stop conditions triggered by visual detection + (hatching, comma stage, etc.) ## Stop Conditions @@ -266,36 +345,129 @@ 1. User: "Run timelapse until all embryos hatch" 2. Agent: - - Enables hatching detector (enable_preset_detector) - - Starts timelapse with stop_condition="hatching" + - Starts the timelapse with stop_condition="hatching" (the stop condition + wires the detection; the perception loop classifies each acquired volume) + - Optionally installs a monitoring mode (enable_monitoring_mode) for + reactive cadence/power - Reports progress on request - Each embryo stops automatically when it hatches -## Available Preset Detectors +## Stage detection -- **hatching**: Detects eggshell breach and embryo emergence -- **comma**: Detects comma stage morphology -- **pretzel**: Detects 3-fold/pretzel stage -- **gastrulation**: Detects cell internalization -- **first_division**: Detects 1-cell to 2-cell transition +Developmental stage comes from the live perception loop (see "Perception & +Analysis"), surfaced in your context and via get_recent_perceptions. Stop +conditions can key on it — e.g. stop_condition="hatching" or "comma". ## Commands During Timelapse - Query status: get_timelapse_status - Stop one embryo: stop_timelapse_embryo -- Change interval: modify_timelapse_embryo +- Change interval (all embryos): modify_timelapse_interval +- Change one embryo's cadence: set_embryo_cadence +- Other per-embryo params: modify_timelapse_embryo / modify_parameters - Pause all: pause_timelapse - Resume: resume_timelapse - Stop all: stop_timelapse """ +AUTONOMY_AND_ADAPTATION = """ +# Adapting Acquisition — Gently + +Gentleness is the prime directive: every imaging action spends photodose on a +precious, living sample. Always prefer the *least* light that answers the +question. When you do adapt, you have direct, live knobs — each takes effect on +the embryo's next acquisition, no restart: + +- **Cadence**: `modify_timelapse_interval` (whole run) / `set_embryo_cadence` + (one embryo). Speed up only around events worth catching (e.g. approaching + hatching); slow back down when nothing is changing. +- **Dose levers**: `modify_parameters` — num_slices, exposure_ms, acquisition + mode (volume ↔ snap, snap is far gentler), and per-embryo 488 power (hard + clamped 2–6%). `set_photodose_budget` caps cumulative exposure and pauses an + embryo that exceeds it; `get_photodose_status` shows where each stands. +- **Events**: `add_stop_condition` (auto-stop on hatching/stage/duration), + `queue_burst` (one-shot high-rate capture of a transient), and per-embryo + pause / resume / stop. +- **Reactive modes**: `enable_monitoring_mode` installs perception-driven rules + that fire on their own (pre-hatching speedup, 488 rampdown on saturation, + burst on stable structure). + +Bias toward the gentlest sufficient action — snap over volume, fewer slices, +lower power, longer interval — unless an event genuinely needs the resolution. + +# Autonomy (OFF / ASK / AUTO) + +You may act between user messages, but only as far as the operator allows. The +mode is set with `set_autonomy` and is **OFF by default**: + +- **off** — act only when the user messages you. +- **ask** — on a notable event (a developmental stage transition, possible + arrest, hatching, an embryo terminating, or an error) you wake, briefly state + your PROPOSED change and why, then call `ask_user_choice` with + Approve / Modify / Skip and act ONLY on Approve. +- **auto** — you adapt on your own on those events. Still: prefer the gentlest + action, and a few irreversible tools (turning the laser on via + `set_laser_power`, `remove_embryo`, `stop_timelapse`) are hard-blocked from + autonomous use — ask the operator for those. + +When you wake autonomously, your turn and the trigger that woke you are shown to +the operator in the chat. Keep autonomous turns tight: assess, make the smallest +helpful change (or none), and explain it in a sentence or two. +""" + + +def build_perception_snapshot(perceiver, embryos) -> str: + """One compact line per embryo of live perception state for the system prompt. + + Reads straight from the perception sessions (current stage, stability, time in + stage, arrest signal, short trajectory). Every read here is synchronous and + side-effect-free — it never triggers a VLM call. Returns '' when there is + nothing to show, so callers can drop the section entirely. + """ + if not perceiver or not embryos: + return "" + lines = [] + for embryo_id in sorted(embryos): + try: + session = perceiver.get_session(embryo_id) + summary = session.summary() if session is not None else None + except Exception: + summary = None + if not summary or not summary.get("current_stage"): + lines.append(f"- {embryo_id}: no perception yet") + continue + parts = [ + f"stage={summary['current_stage']}", + f"stable={summary.get('stability', 0)}x", + ] + temporal = summary.get("temporal") # TemporalContext dataclass or None + if temporal is not None: + tmin = getattr(temporal, "time_in_stage_min", None) + exp = getattr(temporal, "expected_duration_min", None) + if tmin is not None: + seg = f"in_stage={tmin:.0f}min" + if exp: + seg += f"/{exp:.0f}" + parts.append(seg) + if getattr(temporal, "is_potentially_arrested", False): + parts.append("ARRESTED?") + seq = summary.get("stage_sequence") or [] + if len(seq) > 1: + parts.append("traj=" + "->".join(seq[-4:])) + lines.append(f"- {embryo_id}: " + " ".join(parts)) + if not lines: + return "" + return "## Perception (live)\n\n" + "\n".join(lines) + + def build_system_prompt( experiment_state: ExperimentState, - connection_status: dict = None, - context_summary: str = None, - memory_awareness: str = None, + connection_status: dict | None = None, + context_summary: str | None = None, + memory_awareness: str | None = None, microscope=None, + perceiver=None, ) -> str: """ Build complete system prompt for Claude @@ -314,14 +486,16 @@ def build_system_prompt( str Complete system prompt """ - embryo_summary = experiment_state.get_summary() if experiment_state.embryos else "No embryos loaded yet" + embryo_summary = ( + experiment_state.get_summary() if experiment_state.embryos else "No embryos loaded yet" + ) # Build connection status section if connection_status: - device_layer = "connected" if connection_status.get('device_layer') else "NOT CONNECTED" - sam = "available" if connection_status.get('sam_detection') else "not available" + device_layer = "connected" if connection_status.get("device_layer") else "NOT CONNECTED" + sam = "available" if connection_status.get("sam_detection") else "not available" - if not connection_status.get('device_layer'): + if not connection_status.get("device_layer"): connection_section = f"""# Hardware Connection Status ⚠️ **OFFLINE MODE** - Device layer is not connected. @@ -329,7 +503,8 @@ def build_system_prompt( - Device Layer: {device_layer} - SAM Detection: {sam} -**Important**: You cannot perform hardware operations (detect embryos, capture images, move stage, etc.) +**Important**: You cannot perform hardware operations (detect embryos, capture images, +move stage, etc.) without a connected device layer. If the user asks for hardware operations, inform them that the microscope is not connected and suggest they start the server or check the connection.""" else: @@ -357,6 +532,15 @@ def build_system_prompt( else: context_section = "" + # Live per-embryo perception snapshot (deterministic, read straight from the + # perception sessions — bypasses the AI context-summary cache so stage data is + # never stale). + perception_section = "" + if perceiver is not None and experiment_state.embryos: + snap = build_perception_snapshot(perceiver, experiment_state.embryos) + if snap: + perception_section = f"\n{snap}\n" + # Pull organism-specific content from the active organism module organism = get_organism() organism_display = organism.ORGANISM_DISPLAY_NAME @@ -364,13 +548,13 @@ def build_system_prompt( biology_knowledge = organism.BIOLOGY_KNOWLEDGE # Build stop conditions list from organism module - stop_condition_names = list(organism.STOP_CONDITIONS.keys()) - detector_names = list(organism.get_detector_presets().keys()) + list(organism.STOP_CONDITIONS.keys()) + list(organism.get_detector_presets().keys()) # Pull hardware description — prefer microscope (from device layer handshake), # fall back to the static hardware module hardware = get_hardware() - hardware_description = getattr(microscope, 'DESCRIPTION', '') or hardware.HARDWARE_DESCRIPTION + hardware_description = getattr(microscope, "DESCRIPTION", "") or hardware.HARDWARE_DESCRIPTION hardware_display = hardware.HARDWARE_DISPLAY_NAME return f"""You are Gently — an AI scientific collaborator running {hardware_display} @@ -397,6 +581,10 @@ def build_system_prompt( {REACTIVE_MONITORING_MODES} +{OPERATION_PLAN_GUIDANCE} + +{AUTONOMY_AND_ADAPTATION} + {USER_INTERACTION_GUIDELINES} {SESSION_MANAGEMENT} @@ -404,33 +592,46 @@ def build_system_prompt( # Current Experiment State {embryo_summary} +{perception_section} {context_section} # Tool Use Guidelines Answer the user's request using relevant tools. Before calling a tool, do some analysis: 1. Think about which of the provided tools is relevant to answer the user's request -2. Go through each required parameter and determine if the user has provided or given enough information to infer a value +2. Go through each required parameter and determine if the user has provided or given enough + information to infer a value 3. If all required parameters are present or can be reasonably inferred, PROCEED WITH THE TOOL CALL 4. If a required parameter is missing, ask the user to provide it 5. DO NOT ask for more information on optional parameters if not provided - use defaults IMPORTANT: When you need information (status, positions, etc.), CALL THE TOOL IMMEDIATELY. -Do NOT explain what you "would need to do" - just do it. Never say "I would need to query..." - just query it. +Do NOT explain what you "would need to do" - just do it. Never say "I would need to +query..." - just query it. # Behavior Guidelines -1. **Act, then explain**: Call tools first, then explain results. Don't describe what you would do - do it. -2. **Be scientifically accurate**: Base interpretations on actual developmental biology, not speculation +1. **Act, then explain**: Call tools first, then explain results. Don't describe what you + would do - do it. +2. **Be scientifically accurate**: Base interpretations on actual developmental biology, + not speculation 3. **Prioritize sample health**: Always minimize photobleaching and photodamage -4. **Respect embryo roles**: Every embryo line shows `[role=TEST]`, `[role=CALIBRATION]`, or `[role=UNASSIGNED]`. Calibrate / sweep / classify on CALIBRATION embryos; conserve photodose on TEST. Never suggest calibrating against a TEST embryo (see Embryo Roles section). +4. **Respect embryo roles**: Every embryo line shows `[role=TEST]`, `[role=CALIBRATION]`, + or `[role=UNASSIGNED]`. Calibrate / sweep / classify on CALIBRATION embryos; conserve + photodose on TEST. Never suggest calibrating against a TEST embryo (see Embryo Roles + section). 5. **Use proper terminology**: Refer to embryos by ID, nickname, or user label naturally 6. **Track temporal context**: Remember what you've seen in recent images when analyzing new data 6. **Generate safe plans**: Always validate parameters are within hardware limits 7. **Be conversational**: You're a scientific colleague, not a robot -8. **Stop after success**: When a tool returns a success message (starts with ✓), do NOT retry. Report success and wait for next request. -9. **Single tool = complete action**: Tools like capture_lightsheet, view_image, and acquire_volume are COMPLETE actions. Do NOT chain them unless explicitly asked. -10. **Use defaults**: If a tool has default parameters and the user doesn't specify values, use the defaults. -11. **ALWAYS use ask_user_choice**: When asking the user ANY question with selectable answers, MUST use the `ask_user_choice` tool. NEVER list options as text. This is the #1 UX rule. +8. **Stop after success**: When a tool returns a success message (starts with ✓), do NOT + retry. Report success and wait for next request. +9. **Single tool = complete action**: Tools like capture_lightsheet, view_image, and + acquire_volume are COMPLETE actions. Do NOT chain them unless explicitly asked. +10. **Use defaults**: If a tool has default parameters and the user doesn't specify values, + use the defaults. +11. **ALWAYS use ask_user_choice**: When asking the user ANY question with selectable + answers, MUST use the `ask_user_choice` tool. NEVER list options as text. This is the + #1 UX rule. # Embryo Naming @@ -446,7 +647,7 @@ def build_system_prompt( """ -def build_context_message(experiment_state: ExperimentState) -> Dict: +def build_context_message(experiment_state: ExperimentState) -> dict: """ Build context message with current experiment state @@ -464,5 +665,7 @@ def build_context_message(experiment_state: ExperimentState) -> Dict: """ return { "role": "user", - "content": f"[System update - current experiment state]\n\n{experiment_state.get_summary()}" + "content": ( + f"[System update - current experiment state]\n\n{experiment_state.get_summary()}" + ), } diff --git a/gently/harness/protocols.py b/gently/harness/protocols.py index 9b8b29e9..eb2e811a 100644 --- a/gently/harness/protocols.py +++ b/gently/harness/protocols.py @@ -10,7 +10,7 @@ from gently.harness.protocols import MicroscopeClientProtocol """ -from typing import Protocol, runtime_checkable, Dict, List, Set, Tuple, Optional +from typing import Protocol, runtime_checkable @runtime_checkable @@ -24,11 +24,11 @@ class OrganismProtocol(Protocol): ORGANISM_NAME: str ORGANISM_DISPLAY_NAME: str - SAMPLE_TERM: str # "embryo", "cell", "organoid" + SAMPLE_TERM: str # "embryo", "cell", "organoid" SAMPLE_TERM_PLURAL: str STAGES: list TERMINAL_STAGES: set - BIOLOGY_KNOWLEDGE: str # Markdown text for LLM context + BIOLOGY_KNOWLEDGE: str # Markdown text for LLM context PERCEPTION_SYSTEM_PROMPT: str @@ -55,10 +55,10 @@ class HardwareProtocol(Protocol): HARDWARE_NAME: str HARDWARE_DISPLAY_NAME: str - HARDWARE_DESCRIPTION: str # Markdown text for LLM context - CAPABILITIES: set # Set of capability strings + HARDWARE_DESCRIPTION: str # Markdown text for LLM context + CAPABILITIES: set # Set of capability strings # Backward-compat alias — the Microscope base class in harness/microscope.py # replaces this Protocol. Import from there for new code. -from .microscope import Microscope as MicroscopeClientProtocol # noqa: F401 +from .microscope import Microscope as MicroscopeClientProtocol # noqa: E402, F401 diff --git a/gently/harness/resolution_mode/prompt.py b/gently/harness/resolution_mode/prompt.py index 6b3d0e5e..ad4ae321 100644 --- a/gently/harness/resolution_mode/prompt.py +++ b/gently/harness/resolution_mode/prompt.py @@ -13,11 +13,8 @@ and call one of the resolution lifecycle tools to record it. """ -from typing import Optional - -from gently.organisms import get_organism from gently.hardware import get_hardware - +from gently.organisms import get_organism RESOLUTION_MODE_IDENTITY = """\ You're in **session resolution** — figure out what the researcher @@ -110,8 +107,8 @@ def build_resolution_prompt( - context_summary: Optional[str] = None, - memory_awareness: Optional[str] = None, + context_summary: str | None = None, + memory_awareness: str | None = None, ) -> str: """ Build the system prompt for resolution mode. diff --git a/gently/harness/roles.py b/gently/harness/roles.py index 8708f817..68bee551 100644 --- a/gently/harness/roles.py +++ b/gently/harness/roles.py @@ -15,12 +15,19 @@ - ``test``: the biological subject (precious sample). Custom ad-hoc detector. - ``calibration``: reference embryo used for staging/calibration. Absorbs more photodose. Standard perception pipeline. +- ``lineaging``: lineage-tracing reference — tracks nuclei/divisions. Often + a pan-nuclear strain, but the strain is separate from this use. - ``unassigned``: explicit "not yet classified" state. Treated like ``test`` for safety until the user resolves the assignment. + +Role classes +------------ +``role_class`` distinguishes how Operations foregrounds embryos: +- ``"subject"`` — the primary biological subjects of the experiment. +- ``"reference"`` — reference embryos (staging, calibration, lineaging). """ from dataclasses import dataclass -from typing import Dict, List, Optional @dataclass(frozen=True) @@ -29,10 +36,11 @@ class EmbryoRole: Frozen so role definitions are immutable references after registry build. """ + name: str description: str default_cadence_seconds: float = 300.0 - detector_name: Optional[str] = None + detector_name: str | None = None photodose_budget_multiplier: float = 1.0 ui_color: str = "#888888" ui_icon: str = "circle" @@ -43,10 +51,13 @@ class EmbryoRole: # drift back; once they're out of view they stay out, so they get # a short threshold. Test embryos can occasionally pop out and # back, so they get a longer one. - no_object_consecutive_terminal: Optional[int] = None + no_object_consecutive_terminal: int | None = None + # 'subject' | 'reference' — used by Operations to foreground subjects + # vs references in multi-embryo layouts and scheduling decisions. + role_class: str = "subject" -REGISTRY: Dict[str, EmbryoRole] = { +REGISTRY: dict[str, EmbryoRole] = { "unassigned": EmbryoRole( name="unassigned", description="No role assigned yet — treated like 'test' for safety.", @@ -56,6 +67,7 @@ class EmbryoRole: ui_color="#888888", ui_icon="circle", no_object_consecutive_terminal=None, + role_class="subject", # safe default: protect like a subject ), "test": EmbryoRole( name="test", @@ -71,6 +83,7 @@ class EmbryoRole: ui_color="#ff66cc", # magenta ui_icon="star", no_object_consecutive_terminal=5, # forgiving — they might drift back + role_class="subject", ), "calibration": EmbryoRole( name="calibration", @@ -85,6 +98,21 @@ class EmbryoRole: ui_color="#00cccc", # cyan ui_icon="diamond", no_object_consecutive_terminal=2, # they don't drift back; gone == gone + role_class="reference", + ), + "lineaging": EmbryoRole( + name="lineaging", + description=( + "Lineage-tracing reference — tracks nuclei/divisions; often a " + "pan-nuclear strain but the strain is separate from this use." + ), + default_cadence_seconds=300.0, + detector_name="perception", # nuclear pipeline, same as calibration + photodose_budget_multiplier=5.0, + ui_color="#33cc88", # teal-green — distinct from cyan (calibration) and magenta (test) + ui_icon="triangle", + no_object_consecutive_terminal=2, # reference embryos don't drift back + role_class="reference", ), } @@ -94,10 +122,7 @@ class EmbryoRole: def get_role(name: str) -> EmbryoRole: """Look up a role by name. Raises KeyError with helpful message.""" if name not in REGISTRY: - raise KeyError( - f"Unknown embryo role: {name!r}. " - f"Available: {sorted(REGISTRY.keys())}" - ) + raise KeyError(f"Unknown embryo role: {name!r}. Available: {sorted(REGISTRY.keys())}") return REGISTRY[name] @@ -105,6 +130,6 @@ def is_valid_role(name: str) -> bool: return name in REGISTRY -def list_roles() -> List[str]: +def list_roles() -> list[str]: """All registered role names, sorted.""" return sorted(REGISTRY.keys()) diff --git a/gently/harness/session/interaction_logger.py b/gently/harness/session/interaction_logger.py index 15c32ce1..73c82f8a 100644 --- a/gently/harness/session/interaction_logger.py +++ b/gently/harness/session/interaction_logger.py @@ -14,12 +14,12 @@ """ import json +import logging import subprocess -from dataclasses import dataclass, field, asdict +from dataclasses import asdict, dataclass, field from datetime import datetime from pathlib import Path -from typing import Any, Dict, List, Optional -import logging +from typing import Any logger = logging.getLogger(__name__) @@ -27,12 +27,13 @@ @dataclass class ToolCallRecord: """Record of a single tool call""" + tool_name: str - tool_input: Dict[str, Any] + tool_input: dict[str, Any] result: str duration_seconds: float is_error: bool = False - error_message: Optional[str] = None + error_message: str | None = None @dataclass @@ -43,6 +44,7 @@ class InteractionRecord: An interaction is one user message and the agent's response, including any tool calls made during that response. """ + # Unique ID for this interaction interaction_id: str @@ -51,44 +53,42 @@ class InteractionRecord: timestamp: datetime # System state snapshot at time of request - system_state: Dict[str, Any] = field(default_factory=dict) + system_state: dict[str, Any] = field(default_factory=dict) # What happened - tool_calls: List[ToolCallRecord] = field(default_factory=list) + tool_calls: list[ToolCallRecord] = field(default_factory=list) assistant_response: str = "" total_duration_seconds: float = 0.0 # Errors - error: Optional[str] = None - error_traceback: Optional[str] = None + error: str | None = None + error_traceback: str | None = None # Correction detection (filled in after next turn) was_corrected: bool = False - correction_prompt: Optional[str] = None - correction_indicators: List[str] = field(default_factory=list) + correction_prompt: str | None = None + correction_indicators: list[str] = field(default_factory=list) # Metadata session_id: str = "" codebase_version: str = "" model: str = "" - def to_dict(self) -> Dict: + def to_dict(self) -> dict: """Serialize to dictionary for JSON storage""" d = asdict(self) # Convert datetime to ISO format - d['timestamp'] = self.timestamp.isoformat() + d["timestamp"] = self.timestamp.isoformat() # Convert tool calls - d['tool_calls'] = [asdict(tc) for tc in self.tool_calls] + d["tool_calls"] = [asdict(tc) for tc in self.tool_calls] return d @classmethod - def from_dict(cls, d: Dict) -> 'InteractionRecord': + def from_dict(cls, d: dict) -> "InteractionRecord": """Deserialize from dictionary""" d = d.copy() - d['timestamp'] = datetime.fromisoformat(d['timestamp']) - d['tool_calls'] = [ - ToolCallRecord(**tc) for tc in d.get('tool_calls', []) - ] + d["timestamp"] = datetime.fromisoformat(d["timestamp"]) + d["tool_calls"] = [ToolCallRecord(**tc) for tc in d.get("tool_calls", [])] return cls(**d) @@ -152,7 +152,7 @@ def __init__( self.log_file = self.logs_dir / f"{session_id}.jsonl" # In-memory buffer of recent interactions (for correction detection) - self._recent_interactions: List[InteractionRecord] = [] + self._recent_interactions: list[InteractionRecord] = [] self._max_recent = 10 # Get codebase version (git commit) @@ -171,7 +171,7 @@ def _get_git_version(self) -> str: capture_output=True, text=True, cwd=str(self.storage_path.parent), - timeout=5 + timeout=5, ) if result.returncode == 0: return result.stdout.strip() @@ -182,7 +182,7 @@ def _get_git_version(self) -> str: def start_interaction( self, user_prompt: str, - system_state: Dict[str, Any], + system_state: dict[str, Any], ) -> InteractionRecord: """ Start recording a new interaction @@ -219,11 +219,11 @@ def record_tool_call( self, interaction: InteractionRecord, tool_name: str, - tool_input: Dict[str, Any], + tool_input: dict[str, Any], result: str, duration_seconds: float, is_error: bool = False, - error_message: Optional[str] = None, + error_message: str | None = None, ): """ Record a tool call within an interaction @@ -268,8 +268,8 @@ def complete_interaction( interaction: InteractionRecord, assistant_response: str, total_duration_seconds: float, - error: Optional[str] = None, - error_traceback: Optional[str] = None, + error: str | None = None, + error_traceback: str | None = None, ): """ Complete and save an interaction record @@ -342,17 +342,13 @@ def _detect_correction(self, current: InteractionRecord): f"(indicators: {indicators_found})" ) - def _save_interaction( - self, - interaction: InteractionRecord, - append: bool = True - ): + def _save_interaction(self, interaction: InteractionRecord, append: bool = True): """Save interaction to JSONL file""" try: if append: # Append to log file - with open(self.log_file, 'a', encoding='utf-8') as f: - f.write(json.dumps(interaction.to_dict()) + '\n') + with open(self.log_file, "a", encoding="utf-8") as f: + f.write(json.dumps(interaction.to_dict()) + "\n") else: # Need to update existing record - rewrite file # This is less efficient but corrections are rare @@ -368,7 +364,7 @@ def _rewrite_with_update(self, updated: InteractionRecord): # Read all interactions interactions = [] try: - with open(self.log_file, 'r', encoding='utf-8') as f: + with open(self.log_file, encoding="utf-8") as f: for line in f: if line.strip(): record = InteractionRecord.from_dict(json.loads(line)) @@ -382,32 +378,32 @@ def _rewrite_with_update(self, updated: InteractionRecord): # Rewrite file try: - with open(self.log_file, 'w', encoding='utf-8') as f: + with open(self.log_file, "w", encoding="utf-8") as f: for record in interactions: - f.write(json.dumps(record.to_dict()) + '\n') + f.write(json.dumps(record.to_dict()) + "\n") except Exception as e: logger.error(f"Failed to rewrite log file: {e}") - def _sanitize_state(self, state: Dict[str, Any]) -> Dict[str, Any]: + def _sanitize_state(self, state: dict[str, Any]) -> dict[str, Any]: """Remove large/sensitive data from state snapshot""" - sanitized = {} + sanitized: dict[str, Any] = {} # Keep summary info - if 'embryos' in state: - sanitized['embryo_count'] = len(state['embryos']) - sanitized['embryo_ids'] = list(state['embryos'].keys()) + if "embryos" in state: + sanitized["embryo_count"] = len(state["embryos"]) + sanitized["embryo_ids"] = list(state["embryos"].keys()) - if 'detectors' in state: - sanitized['detector_count'] = len(state['detectors']) + if "detectors" in state: + sanitized["detector_count"] = len(state["detectors"]) - if 'acquisition_status' in state: - sanitized['acquisition_status'] = state['acquisition_status'] + if "acquisition_status" in state: + sanitized["acquisition_status"] = state["acquisition_status"] return sanitized - def _sanitize_tool_input(self, tool_input: Dict[str, Any]) -> Dict[str, Any]: + def _sanitize_tool_input(self, tool_input: dict[str, Any]) -> dict[str, Any]: """Remove large/binary data from tool input""" - sanitized = {} + sanitized: dict[str, Any] = {} for key, value in tool_input.items(): if isinstance(value, (str, int, float, bool, type(None))): if isinstance(value, str) and len(value) > 500: @@ -425,14 +421,14 @@ def _sanitize_tool_input(self, tool_input: Dict[str, Any]) -> Dict[str, Any]: sanitized[key] = f"[{type(value).__name__}]" return sanitized - def get_session_stats(self) -> Dict[str, Any]: + def get_session_stats(self) -> dict[str, Any]: """Get statistics for current session""" if not self.log_file.exists(): return { - 'total_interactions': 0, - 'corrections': 0, - 'errors': 0, - 'tool_calls': 0, + "total_interactions": 0, + "corrections": 0, + "errors": 0, + "tool_calls": 0, } total = 0 @@ -441,35 +437,35 @@ def get_session_stats(self) -> Dict[str, Any]: tool_calls = 0 try: - with open(self.log_file, 'r', encoding='utf-8') as f: + with open(self.log_file, encoding="utf-8") as f: for line in f: if line.strip(): record = json.loads(line) total += 1 - if record.get('was_corrected'): + if record.get("was_corrected"): corrections += 1 - if record.get('error'): + if record.get("error"): errors += 1 - tool_calls += len(record.get('tool_calls', [])) + tool_calls += len(record.get("tool_calls", [])) except Exception: pass return { - 'total_interactions': total, - 'corrections': corrections, - 'errors': errors, - 'tool_calls': tool_calls, - 'correction_rate': corrections / total if total > 0 else 0, + "total_interactions": total, + "corrections": corrections, + "errors": errors, + "tool_calls": tool_calls, + "correction_rate": corrections / total if total > 0 else 0, } - def load_session_interactions(self) -> List[InteractionRecord]: + def load_session_interactions(self) -> list[InteractionRecord]: """Load all interactions from current session""" if not self.log_file.exists(): return [] interactions = [] try: - with open(self.log_file, 'r', encoding='utf-8') as f: + with open(self.log_file, encoding="utf-8") as f: for line in f: if line.strip(): record = InteractionRecord.from_dict(json.loads(line)) diff --git a/gently/harness/session/manager.py b/gently/harness/session/manager.py index 9985b324..ce89fa67 100644 --- a/gently/harness/session/manager.py +++ b/gently/harness/session/manager.py @@ -8,7 +8,6 @@ import json import logging import uuid -from typing import Dict, List, Optional logger = logging.getLogger(__name__) @@ -24,11 +23,11 @@ class SessionManager: def __init__(self, store, storage_path): self.store = store self.storage_path = storage_path - self._session_id: Optional[str] = None + self._session_id: str | None = None @property - def session_id(self) -> str: - """Get current session ID.""" + def session_id(self) -> str | None: + """Get current session ID (None before create_session()).""" return self._session_id def create_session(self): @@ -74,41 +73,45 @@ def _resume_session(self, session_id: str, experiment): conversation_history = [] if snapshot: - raw_history = snapshot.get('conversation_history', []) + raw_history = snapshot.get("conversation_history", []) conversation_history = self.sanitize_loaded_messages(raw_history) - experiment_data = snapshot.get('experiment_data', {}) - experiment.active_plan_item_id = experiment_data.get('active_plan_item_id') - embryo_states = experiment_data.get('embryos', {}) + experiment_data = snapshot.get("experiment_data", {}) + experiment.active_plan_item_id = experiment_data.get("active_plan_item_id") + embryo_states = experiment_data.get("embryos", {}) for embryo_id, embryo_data in embryo_states.items(): - pos = embryo_data.get('stage_position', {}) + pos = embryo_data.get("stage_position", {}) experiment.add_embryo( embryo_id=embryo_id, position=pos, - calibration=embryo_data.get('calibration', {}), - user_label=embryo_data.get('user_label'), - uid=embryo_data.get('uid'), + calibration=embryo_data.get("calibration", {}), + user_label=embryo_data.get("user_label"), + uid=embryo_data.get("uid"), ) embryo = experiment.embryos[embryo_id] - embryo.nickname = embryo_data.get('nickname') - embryo.interval_seconds = embryo_data.get('interval_seconds') - embryo.num_slices = embryo_data.get('num_slices', 50) - embryo.exposure_ms = embryo_data.get('exposure_ms', 10.0) - embryo.priority = embryo_data.get('priority', 'normal') - embryo.timepoints_acquired = embryo_data.get('timepoints_acquired', 0) - embryo.should_skip = embryo_data.get('should_skip', False) - embryo.skip_reason = embryo_data.get('skip_reason') - - # Also load embryos from store's embryo table + embryo.nickname = embryo_data.get("nickname") + embryo.interval_seconds = embryo_data.get("interval_seconds") + embryo.num_slices = embryo_data.get("num_slices", 50) + embryo.exposure_ms = embryo_data.get("exposure_ms", 10.0) + embryo.priority = embryo_data.get("priority", "normal") + embryo.timepoints_acquired = embryo_data.get("timepoints_acquired", 0) + embryo.should_skip = embryo_data.get("should_skip", False) + embryo.skip_reason = embryo_data.get("skip_reason") + + # Also load embryos from store's embryo table. FileStore returns + # position_coarse / position_fine (with legacy position_x / position_y + # backfilled into coarse on read), so both calibration stages survive + # the resume. store_embryos = self.store.list_embryos(session_id) for e in store_embryos: - eid = e['embryo_id'] + eid = e["embryo_id"] if eid not in experiment.embryos: experiment.add_embryo( embryo_id=eid, - position={'x': e.get('position_x'), 'y': e.get('position_y')}, - calibration=json.loads(e['calibration']) if e.get('calibration') else {}, + position=e.get("position_coarse") or {}, + position_fine=e.get("position_fine") or {}, + calibration=json.loads(e["calibration"]) if e.get("calibration") else {}, ) self.store.touch_session(session_id) @@ -137,11 +140,14 @@ def save_session(self, experiment, conversation_history, system_prompt) -> bool: if not self._session_id: return False try: - self.store.save_session_snapshot(self._session_id, { - 'conversation_history': self.serialize_messages(conversation_history), - 'experiment_data': experiment.to_dict(), - 'system_prompt': system_prompt, - }) + self.store.save_session_snapshot( + self._session_id, + { + "conversation_history": self.serialize_messages(conversation_history), + "experiment_data": experiment.to_dict(), + "system_prompt": system_prompt, + }, + ) self._sync_embryos_to_db(experiment) self.store.touch_session(self._session_id) return True @@ -154,11 +160,14 @@ def auto_save(self, experiment, conversation_history, system_prompt): if not self._session_id: return try: - self.store.save_session_snapshot(self._session_id, { - 'conversation_history': self.serialize_messages(conversation_history), - 'experiment_data': experiment.to_dict(), - 'system_prompt': system_prompt, - }) + self.store.save_session_snapshot( + self._session_id, + { + "conversation_history": self.serialize_messages(conversation_history), + "experiment_data": experiment.to_dict(), + "system_prompt": system_prompt, + }, + ) self._sync_embryos_to_db(experiment) self.store.touch_session(self._session_id) except Exception: @@ -169,15 +178,16 @@ def _sync_embryos_to_db(self, experiment): for embryo_id, embryo in experiment.embryos.items(): pos = embryo.stage_position or {} self.store.register_embryo( - self._session_id, embryo_id, - embryo_uid=getattr(embryo, 'uid', None), - nickname=getattr(embryo, 'user_label', None), - position_x=pos.get('x'), - position_y=pos.get('y'), + self._session_id, + embryo_id, + embryo_uid=getattr(embryo, "uid", None), + nickname=getattr(embryo, "user_label", None), + position_x=pos.get("x"), + position_y=pos.get("y"), calibration=embryo.calibration, ) - def list_sessions(self) -> List[Dict]: + def list_sessions(self) -> list[dict]: """ List available sessions from FileStore. @@ -188,8 +198,9 @@ def list_sessions(self) -> List[Dict]: """ return self.store.list_sessions() - def resume_session(self, session_id: str, experiment, conversation_mgr, - prompt_mgr_update_fn) -> bool: + def resume_session( + self, session_id: str, experiment, conversation_mgr, prompt_mgr_update_fn + ) -> bool: """ Resume a session (public interface for CLI). @@ -230,7 +241,7 @@ def resume_session(self, session_id: str, experiment, conversation_mgr, # ===== Message Serialization ===== @staticmethod - def sanitize_loaded_messages(messages: List[Dict]) -> List[Dict]: + def sanitize_loaded_messages(messages: list[dict]) -> list[dict]: """Fix conversation history loaded from JSON snapshots. Old snapshots may contain content blocks that were serialized @@ -240,28 +251,28 @@ def sanitize_loaded_messages(messages: List[Dict]) -> List[Dict]: """ clean = [] for msg in messages: - content = msg.get('content') + content = msg.get("content") if content is None: continue if isinstance(content, str): clean.append(msg) continue if isinstance(content, list): - valid_blocks = [] + valid_blocks: list[dict | str] = [] for block in content: if isinstance(block, dict): valid_blocks.append(block) elif isinstance(block, str): - if block.startswith(('TextBlock(', 'ToolUseBlock(')): + if block.startswith(("TextBlock(", "ToolUseBlock(")): continue valid_blocks.append(block) if valid_blocks: - clean.append({**msg, 'content': valid_blocks}) + clean.append({**msg, "content": valid_blocks}) continue return clean @staticmethod - def serialize_messages(messages: List[Dict]) -> List[Dict]: + def serialize_messages(messages: list[dict]) -> list[dict]: """Convert conversation history to JSON-safe plain dicts. Anthropic SDK returns content blocks as objects (TextBlock, @@ -269,30 +280,31 @@ def serialize_messages(messages: List[Dict]) -> List[Dict]: repr strings. This converts everything to plain dicts so the history round-trips cleanly through JSON. """ + def _block_to_dict(block): if isinstance(block, dict): return block if isinstance(block, str): return block - if hasattr(block, 'model_dump'): + if hasattr(block, "model_dump"): return block.model_dump() - if hasattr(block, 'to_dict'): + if hasattr(block, "to_dict"): return block.to_dict() - if hasattr(block, 'type'): - d = {'type': block.type} - if block.type == 'text' and hasattr(block, 'text'): - d['text'] = block.text - elif block.type == 'tool_use': - d['id'] = getattr(block, 'id', '') - d['name'] = getattr(block, 'name', '') - d['input'] = getattr(block, 'input', {}) + if hasattr(block, "type"): + d = {"type": block.type} + if block.type == "text" and hasattr(block, "text"): + d["text"] = block.text + elif block.type == "tool_use": + d["id"] = getattr(block, "id", "") + d["name"] = getattr(block, "name", "") + d["input"] = getattr(block, "input", {}) return d return str(block) serialized = [] for msg in messages: - content = msg.get('content') + content = msg.get("content") if isinstance(content, list): content = [_block_to_dict(b) for b in content] - serialized.append({**msg, 'content': content}) + serialized.append({**msg, "content": content}) return serialized diff --git a/gently/harness/session/timeline.py b/gently/harness/session/timeline.py index 5d7ea6f0..5bb7fc97 100644 --- a/gently/harness/session/timeline.py +++ b/gently/harness/session/timeline.py @@ -11,12 +11,12 @@ import json import logging import threading -import uuid from collections import deque -from dataclasses import dataclass, field, asdict +from collections.abc import Callable +from dataclasses import dataclass, field from datetime import datetime, timedelta from pathlib import Path -from typing import Any, Callable, Dict, List, Optional +from typing import Any from gently.core.event_bus import Event, EventType, get_event_bus @@ -55,83 +55,84 @@ class TimelineEvent: severity : str Severity level: info | success | warning | error """ + event_id: str event_type: str event_subtype: str timestamp: datetime source: str - session_id: Optional[str] = None # Session this event belongs to - embryo_id: Optional[str] = None - detector_name: Optional[str] = None - timepoint: Optional[int] = None - confidence: Optional[str] = None - data: Dict[str, Any] = field(default_factory=dict) + session_id: str | None = None # Session this event belongs to + embryo_id: str | None = None + detector_name: str | None = None + timepoint: int | None = None + confidence: str | None = None + data: dict[str, Any] = field(default_factory=dict) icon: str = ">" severity: str = "info" - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: """Serialize to dictionary""" return { - 'event_id': self.event_id, - 'event_type': self.event_type, - 'event_subtype': self.event_subtype, - 'timestamp': self.timestamp.isoformat(), - 'source': self.source, - 'session_id': self.session_id, - 'embryo_id': self.embryo_id, - 'detector_name': self.detector_name, - 'timepoint': self.timepoint, - 'confidence': self.confidence, - 'data': self.data, - 'icon': self.icon, - 'severity': self.severity, + "event_id": self.event_id, + "event_type": self.event_type, + "event_subtype": self.event_subtype, + "timestamp": self.timestamp.isoformat(), + "source": self.source, + "session_id": self.session_id, + "embryo_id": self.embryo_id, + "detector_name": self.detector_name, + "timepoint": self.timepoint, + "confidence": self.confidence, + "data": self.data, + "icon": self.icon, + "severity": self.severity, } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> 'TimelineEvent': + def from_dict(cls, d: dict[str, Any]) -> "TimelineEvent": """Deserialize from dictionary""" return cls( - event_id=d['event_id'], - event_type=d['event_type'], - event_subtype=d['event_subtype'], - timestamp=datetime.fromisoformat(d['timestamp']), - source=d.get('source', 'unknown'), - session_id=d.get('session_id'), - embryo_id=d.get('embryo_id'), - detector_name=d.get('detector_name'), - timepoint=d.get('timepoint'), - confidence=d.get('confidence'), - data=d.get('data', {}), - icon=d.get('icon', '>'), - severity=d.get('severity', 'info'), + event_id=d["event_id"], + event_type=d["event_type"], + event_subtype=d["event_subtype"], + timestamp=datetime.fromisoformat(d["timestamp"]), + source=d.get("source", "unknown"), + session_id=d.get("session_id"), + embryo_id=d.get("embryo_id"), + detector_name=d.get("detector_name"), + timepoint=d.get("timepoint"), + confidence=d.get("confidence"), + data=d.get("data", {}), + icon=d.get("icon", ">"), + severity=d.get("severity", "info"), ) @property def short_label(self) -> str: """Short label for timeline display (e.g., 'TL', 'DET')""" - if self.event_type == 'timelapse': - return 'TL' - elif self.event_type == 'detection': - return 'DET' + if self.event_type == "timelapse": + return "TL" + elif self.event_type == "detection": + return "DET" else: - return 'SYS' + return "SYS" @property def description(self) -> str: """Human-readable description of the event""" - if self.event_type == 'timelapse': - if self.event_subtype == 'started': - embryos = self.data.get('embryo_ids', []) + if self.event_type == "timelapse": + if self.event_subtype == "started": + embryos = self.data.get("embryo_ids", []) return f"Started timelapse with {len(embryos)} embryo(s)" - elif self.event_subtype == 'volume_acquired': + elif self.event_subtype == "volume_acquired": return f"{self.embryo_id} @ t={self.timepoint}" - elif self.event_subtype == 'completed': - total = self.data.get('total_timepoints', '?') + elif self.event_subtype == "completed": + total = self.data.get("total_timepoints", "?") return f"Completed ({total} timepoints)" - elif self.event_subtype == 'failed': + elif self.event_subtype == "failed": return f"Failed: {self.data.get('error', 'unknown error')}" - elif self.event_type == 'detection': - detected = self.data.get('detected', False) + elif self.event_type == "detection": + detected = self.data.get("detected", False) status = "Detected" if detected else "Not detected" conf = f" ({self.confidence})" if self.confidence else "" return f"{self.detector_name} on {self.embryo_id} - {status}{conf}" @@ -141,91 +142,109 @@ def description(self) -> str: # Mapping from EventBus EventType to TimelineEvent properties EVENT_MAPPING = { EventType.ACQUISITION_STARTED: { - 'event_type': 'timelapse', - 'event_subtype': 'started', - 'icon': '>', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "started", + "icon": ">", + "severity": "info", }, EventType.VOLUME_ACQUIRED: { - 'event_type': 'timelapse', - 'event_subtype': 'volume_acquired', - 'icon': '+', - 'severity': 'success', + "event_type": "timelapse", + "event_subtype": "volume_acquired", + "icon": "+", + "severity": "success", }, EventType.ACQUISITION_COMPLETED: { - 'event_type': 'timelapse', - 'event_subtype': 'completed', - 'icon': '+', - 'severity': 'success', + "event_type": "timelapse", + "event_subtype": "completed", + "icon": "+", + "severity": "success", }, EventType.ACQUISITION_STOPPED: { - 'event_type': 'timelapse', - 'event_subtype': 'stopped', - 'icon': '-', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "stopped", + "icon": "-", + "severity": "info", }, EventType.ACQUISITION_FAILED: { - 'event_type': 'timelapse', - 'event_subtype': 'failed', - 'icon': 'x', - 'severity': 'error', + "event_type": "timelapse", + "event_subtype": "failed", + "icon": "x", + "severity": "error", }, EventType.DETECTOR_EVALUATED: { - 'event_type': 'detection', - 'event_subtype': 'evaluated', - 'icon': '?', - 'severity': 'info', + "event_type": "detection", + "event_subtype": "evaluated", + "icon": "?", + "severity": "info", }, EventType.DETECTION_TRIGGERED: { - 'event_type': 'detection', - 'event_subtype': 'triggered', - 'icon': '!', - 'severity': 'success', + "event_type": "detection", + "event_subtype": "triggered", + "icon": "!", + "severity": "success", }, EventType.HATCHING_DETECTED: { - 'event_type': 'detection', - 'event_subtype': 'hatching', - 'icon': '+', - 'severity': 'success', + "event_type": "detection", + "event_subtype": "hatching", + "icon": "+", + "severity": "success", }, # Strategy / experiment view persistence — these were already emitted on # the EventBus but weren't being captured to timeline.jsonl, so the # swimlane view had no event history to replay. EventType.EMBRYO_CADENCE_CHANGED: { - 'event_type': 'timelapse', - 'event_subtype': 'cadence_changed', - 'icon': '~', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "cadence_changed", + "icon": "~", + "severity": "info", }, EventType.POWER_RAMP_STEP: { - 'event_type': 'timelapse', - 'event_subtype': 'power_changed', - 'icon': '*', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "power_changed", + "icon": "*", + "severity": "info", }, EventType.TRIGGER_FIRED: { - 'event_type': 'timelapse', - 'event_subtype': 'trigger_fired', - 'icon': '<>', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "trigger_fired", + "icon": "<>", + "severity": "info", }, EventType.BURST_QUEUED: { - 'event_type': 'timelapse', - 'event_subtype': 'burst_queued', - 'icon': 'q', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "burst_queued", + "icon": "q", + "severity": "info", }, EventType.BURST_START: { - 'event_type': 'timelapse', - 'event_subtype': 'burst_started', - 'icon': '^', - 'severity': 'info', + "event_type": "timelapse", + "event_subtype": "burst_started", + "icon": "^", + "severity": "info", }, EventType.BURST_COMPLETE: { - 'event_type': 'timelapse', - 'event_subtype': 'burst_completed', - 'icon': 'v', - 'severity': 'success', + "event_type": "timelapse", + "event_subtype": "burst_completed", + "icon": "v", + "severity": "success", + }, + EventType.TEMPERATURE_SETPOINT_CHANGED: { + "event_type": "temperature", + "event_subtype": "setpoint_changed", + "icon": "T", + "severity": "info", + }, + EventType.TEMP_PROTOCOL_STARTED: { + "event_type": "tactic", + "event_subtype": "temp_protocol_started", + "icon": "~", + "severity": "info", + }, + EventType.TEMP_PROTOCOL_COMPLETED: { + "event_type": "tactic", + "event_subtype": "temp_protocol_completed", + "icon": "+", + "severity": "success", }, } @@ -243,9 +262,9 @@ class TimelineManager: def __init__( self, - storage_path: Optional[Path] = None, + storage_path: Path | None = None, max_events: int = 1000, - session_id: Optional[str] = None, + session_id: str | None = None, ): """ Parameters @@ -262,7 +281,7 @@ def __init__( self._session_id = session_id self._events: deque[TimelineEvent] = deque(maxlen=max_events) self._lock = threading.RLock() - self._unsubscribers: List[Callable] = [] + self._unsubscribers: list[Callable] = [] self._started = False # Load existing events from storage @@ -274,7 +293,7 @@ def set_session_id(self, session_id: str) -> None: self._session_id = session_id @property - def storage_file(self) -> Optional[Path]: + def storage_file(self) -> Path | None: """Path to the timeline JSONL file""" if self._storage_path: return self._storage_path / "timeline.jsonl" @@ -317,18 +336,18 @@ def _on_event(self, event: Event) -> None: timeline_event = TimelineEvent( event_id=event.event_id, - event_type=mapping['event_type'], - event_subtype=mapping['event_subtype'], + event_type=mapping["event_type"], + event_subtype=mapping["event_subtype"], timestamp=event.timestamp, source=event.source, session_id=self._session_id, # Tag with current session - embryo_id=data.get('embryo_id'), - detector_name=data.get('detector_name'), - timepoint=data.get('timepoint'), - confidence=data.get('confidence'), + embryo_id=data.get("embryo_id"), + detector_name=data.get("detector_name"), + timepoint=data.get("timepoint"), + confidence=data.get("confidence"), data=data, - icon=mapping['icon'], - severity=mapping['severity'], + icon=mapping["icon"], + severity=mapping["severity"], ) self.add_event(timeline_event) @@ -352,13 +371,13 @@ def add_event(self, event: TimelineEvent) -> None: def get_events( self, - event_type: Optional[str] = None, - embryo_id: Optional[str] = None, - since: Optional[datetime] = None, - until: Optional[datetime] = None, - session_id: Optional[str] = "current", + event_type: str | None = None, + embryo_id: str | None = None, + since: datetime | None = None, + until: datetime | None = None, + session_id: str | None = "current", limit: int = 50, - ) -> List[TimelineEvent]: + ) -> list[TimelineEvent]: """ Get filtered events from timeline @@ -405,7 +424,7 @@ def get_events( # Return limited, oldest first (chronological) return events[-limit:] if len(events) > limit else events - def get_time_range(self) -> tuple[Optional[datetime], Optional[datetime]]: + def get_time_range(self) -> tuple[datetime | None, datetime | None]: """ Get the time range of events in the timeline @@ -421,7 +440,7 @@ def get_time_range(self) -> tuple[Optional[datetime], Optional[datetime]]: return events[0].timestamp, events[-1].timestamp - def clear_events(self, before: Optional[datetime] = None) -> int: + def clear_events(self, before: datetime | None = None) -> int: """ Clear events from timeline @@ -443,7 +462,7 @@ def clear_events(self, before: Optional[datetime] = None) -> int: old_count = len(self._events) self._events = deque( (e for e in self._events if e.timestamp >= before), - maxlen=self._max_events + maxlen=self._max_events, ) count = old_count - len(self._events) @@ -460,11 +479,11 @@ def _load_from_file(self) -> None: return try: - with open(self.storage_file, 'r', encoding='utf-8') as f: + with open(self.storage_file, encoding="utf-8") as f: for line in f: line = line.strip() # Only parse lines that look like JSON objects - if line and line.startswith('{'): + if line and line.startswith("{"): try: data = json.loads(line) event = TimelineEvent.from_dict(data) @@ -484,8 +503,9 @@ def _persist_event(self, event: TimelineEvent) -> None: # Ensure directory exists self._storage_path.mkdir(parents=True, exist_ok=True) - with open(self.storage_file, 'a', encoding='utf-8') as f: - f.write(json.dumps(event.to_dict()) + '\n') + assert self.storage_file is not None # implied by _storage_path guard above + with open(self.storage_file, "a", encoding="utf-8") as f: + f.write(json.dumps(event.to_dict()) + "\n") except Exception as e: logger.error(f"Error persisting timeline event: {e}") @@ -500,9 +520,10 @@ def _rewrite_storage(self) -> None: with self._lock: events = list(self._events) - with open(self.storage_file, 'w', encoding='utf-8') as f: + assert self.storage_file is not None # implied by _storage_path guard above + with open(self.storage_file, "w", encoding="utf-8") as f: for event in events: - f.write(json.dumps(event.to_dict()) + '\n') + f.write(json.dumps(event.to_dict()) + "\n") except Exception as e: logger.error(f"Error rewriting timeline storage: {e}") @@ -512,7 +533,7 @@ def __len__(self) -> int: return len(self._events) -def parse_time_delta(s: str) -> Optional[timedelta]: +def parse_time_delta(s: str) -> timedelta | None: """ Parse a time delta string like "1h", "30m", "2d" @@ -531,13 +552,13 @@ def parse_time_delta(s: str) -> Optional[timedelta]: return None try: - if s.endswith('m'): + if s.endswith("m"): return timedelta(minutes=int(s[:-1])) - elif s.endswith('h'): + elif s.endswith("h"): return timedelta(hours=int(s[:-1])) - elif s.endswith('d'): + elif s.endswith("d"): return timedelta(days=int(s[:-1])) - elif s.endswith('w'): + elif s.endswith("w"): return timedelta(weeks=int(s[:-1])) else: # Try parsing as minutes diff --git a/gently/harness/state.py b/gently/harness/state.py index a6113480..bbd38b15 100644 --- a/gently/harness/state.py +++ b/gently/harness/state.py @@ -25,11 +25,13 @@ fields are now on ``EmbryoState`` directly. """ +import logging import re +from collections.abc import Callable from dataclasses import dataclass, field from datetime import datetime -from typing import Any, Dict, List, Optional, Tuple -from pathlib import Path +from typing import Any + import numpy as np # Re-export CalibrationPrior from its hardware-specific home for backward compat. @@ -37,6 +39,8 @@ # modules will define their own calibration models. from gently.hardware.dispim.calibration import CalibrationPrior +logger = logging.getLogger(__name__) + @dataclass class FocusDataPoint: @@ -56,13 +60,14 @@ class FocusDataPoint: - z: primary focus axis (µm) — piezo for diSPIM, Z-motor for 2P/confocal - secondary_axis: optional second axis — galvo for diSPIM, unused (0.0) for single-axis systems """ - z: float # Primary focus position (µm) + + z: float # Primary focus position (µm) secondary_axis: float # Secondary axis position (galvo deg for diSPIM, 0.0 otherwise) - score: float # Focus quality score (algorithm-dependent) - r_squared: float # Gaussian fit quality (0-1), higher = more reliable - timestamp: datetime # When this measurement was made - method: str # 'calibration', 'fine_focus', 'manual' - algorithm: str = 'fft_bandpass' # Focus algorithm used + score: float # Focus quality score (algorithm-dependent) + r_squared: float # Gaussian fit quality (0-1), higher = more reliable + timestamp: datetime # When this measurement was made + method: str # 'calibration', 'fine_focus', 'manual' + algorithm: str = "fft_bandpass" # Focus algorithm used # Backward-compatible properties for code that uses the old field names @property @@ -73,38 +78,39 @@ def piezo(self) -> float: def galvo(self) -> float: return self.secondary_axis - def to_dict(self) -> Dict: + def to_dict(self) -> dict: """Serialize for JSON storage""" return { - 'z': self.z, - 'secondary_axis': self.secondary_axis, - 'score': self.score, - 'r_squared': self.r_squared, - 'timestamp': self.timestamp.isoformat(), - 'method': self.method, - 'algorithm': self.algorithm, + "z": self.z, + "secondary_axis": self.secondary_axis, + "score": self.score, + "r_squared": self.r_squared, + "timestamp": self.timestamp.isoformat(), + "method": self.method, + "algorithm": self.algorithm, # Backward-compatible keys for existing serialized data - 'galvo': self.secondary_axis, - 'piezo': self.z, + "galvo": self.secondary_axis, + "piezo": self.z, } @classmethod - def from_dict(cls, data: Dict) -> 'FocusDataPoint': + def from_dict(cls, data: dict) -> "FocusDataPoint": """Deserialize from JSON. Handles both old (galvo/piezo) and new (z/secondary_axis) keys.""" return cls( - z=data.get('z', data.get('piezo', 0.0)), - secondary_axis=data.get('secondary_axis', data.get('galvo', 0.0)), - score=data['score'], - r_squared=data['r_squared'], - timestamp=datetime.fromisoformat(data['timestamp']), - method=data['method'], - algorithm=data.get('algorithm', 'fft_bandpass'), + z=data.get("z", data.get("piezo", 0.0)), + secondary_axis=data.get("secondary_axis", data.get("galvo", 0.0)), + score=data["score"], + r_squared=data["r_squared"], + timestamp=datetime.fromisoformat(data["timestamp"]), + method=data["method"], + algorithm=data.get("algorithm", "fft_bandpass"), ) @dataclass class ImageRecord: """Record of a single acquired image/volume""" + embryo_id: str timepoint: int timestamp: datetime @@ -112,8 +118,8 @@ class ImageRecord: max_projection_b64: str # Base64-encoded JPEG for Claude Vision size_kb: float # UID-based data references (new data layer) - volume_uid: Optional[str] = None # UID for volume in DataStore - projection_uid: Optional[str] = None # UID for max projection in DataStore + volume_uid: str | None = None # UID for volume in DataStore + projection_uid: str | None = None # UID for max projection in DataStore @dataclass @@ -122,22 +128,30 @@ class EmbryoState: # Identity id: str # "embryo_1" - uid: Optional[str] = None # Global unique identifier for cross-session tracking - nickname: Optional[str] = None # Agent-assigned: "the fast one" - user_label: Optional[str] = None # User-provided: "control_1" + uid: str | None = None # Global unique identifier for cross-session tracking + nickname: str | None = None # Agent-assigned: "the fast one" + user_label: str | None = None # User-provided: "control_1" # Role key into gently.harness.roles.REGISTRY. Drives cadence policy, # detector selection, photodose budget, UI presentation. Default "test" # is the safe choice — accidental Calibration→Test only over-protects; # accidental Test→Calibration would burn extra dose on the precious sample. role: str = "test" - - # Position - stage_position: Dict[str, float] = field(default_factory=dict) # {'x': 1234.5, 'y': 5678.9} - calibration: Dict = field(default_factory=dict) # Galvo/piezo parameters + # Free-form biological sample descriptor (orthogonal to role). Examples: + # "pan-nuclear GFP", "H2B-mCherry", "wild-type". None = unspecified. + strain: str | None = None + + # Position — two-stage: coarse (bottom-camera detection or manual map + # placement, always present once an embryo exists) and fine (populated + # later by SPIM-objective alignment). Resolved value is exposed by the + # `stage_position` property so downstream motion/perception can stay + # agnostic about which stage we're in. + position_coarse: dict[str, float] = field(default_factory=dict) # {'x': ..., 'y': ...} + position_fine: dict[str, float] = field(default_factory=dict) # empty until SPIM head alignment + calibration: dict = field(default_factory=dict) # Galvo/piezo parameters detection_confidence: float = 0.0 # SAM/detection confidence score (0-1) # Acquisition Parameters (current) - interval_seconds: Optional[float] = None # Per-embryo interval; None = use timelapse default + interval_seconds: float | None = None # Per-embryo interval; None = use timelapse default num_slices: int = 50 exposure_ms: float = 10.0 priority: str = "normal" # high/normal/low @@ -145,13 +159,13 @@ class EmbryoState: # Per-embryo 488 laser power %. None = use device-layer default (no # change at acquire time). Float values are hard-limited at the device # layer by DiSPIMLightSource.POWER_LIMITS_PCT[488] (default 2-6%). - laser_power_488_pct: Optional[float] = None + laser_power_488_pct: float | None = None # Status - last_imaged: Optional[datetime] = None + last_imaged: datetime | None = None timepoints_acquired: int = 0 should_skip: bool = False - skip_reason: Optional[str] = None + skip_reason: str | None = None # Timelapse runtime state (consolidated from former EmbryoAcquisitionState). # Populated/used by TimelapseOrchestrator while this embryo is part of an @@ -161,12 +175,12 @@ class EmbryoState: # is a Phase 9 concern. stop_condition: Any = None is_complete: bool = False - completion_reason: Optional[str] = None + completion_reason: str | None = None error_count: int = 0 - last_error: Optional[str] = None - detection_triggered_at: Optional[int] = None - detection_type: Optional[str] = None - no_object_since_timepoint: Optional[int] = None + last_error: str | None = None + detection_triggered_at: int | None = None + detection_type: str | None = None + no_object_since_timepoint: int | None = None # Count of consecutive "no_object" detections. Reset to 0 whenever # the embryo is detected again. When this crosses the role's # ``no_object_consecutive_terminal`` threshold, the orchestrator @@ -184,58 +198,65 @@ class EmbryoState: # - paused: skip in the due loop (over-budget, manually paused, or # idle during another embryo's burst) cadence_phase: str = "normal" - next_due_at: Optional[datetime] = None + next_due_at: datetime | None = None # Light exposure tracking (for phototoxicity monitoring) exposure_count: int = 0 # Number of imaging events (snaps + volumes) total_exposure_ms: float = 0.0 # Cumulative laser-on time in milliseconds # Analysis Results (cached) - hatching_status: Dict = field(default_factory=dict) + hatching_status: dict = field(default_factory=dict) # {hatched: bool, confidence: str, timepoint: int} - morphology_history: List[Dict] = field(default_factory=list) + morphology_history: list[dict] = field(default_factory=list) # [{timepoint, size, shape, activity_score}] - fluorescence_history: List[Dict] = field(default_factory=list) + fluorescence_history: list[dict] = field(default_factory=list) # [{timepoint, mean_intensity, photobleaching_estimate}] - custom_classifications: Dict = field(default_factory=dict) + custom_classifications: dict = field(default_factory=dict) # User-defined: {"first_cleavage": {detected: bool, timepoint: 42}} # Verification round tracking (for consecutive confirmation) pending_verification: bool = False # True when detection fired, awaiting verification consecutive_detection_count: int = 0 # Must reach 5 consecutive verified detections to stop - last_detection_round: Optional[int] = None # Round when detection was last verified + last_detection_round: int | None = None # Round when detection was last verified # Detection results from detector system - detection_results: Dict[str, List[Dict]] = field(default_factory=dict) + detection_results: dict[str, list[dict]] = field(default_factory=dict) # detector_name -> list of detection results # e.g., {"comma_stage": [{"timepoint": 120, "detected": False, "confidence": "HIGH"}, ...]} # CV Subagent analysis results (populated from CV_RESULT_READY events) - cv_analyses: Dict[str, List[Dict]] = field(default_factory=dict) + cv_analyses: dict[str, list[dict]] = field(default_factory=dict) # result_type -> list of results by timepoint # e.g., {"nuclei_count": [{"timepoint": 5, "num_nuclei": 66, ...}]} # Quick-access fields for latest CV results (for /embryos display) - latest_nuclei_count: Optional[int] = None - latest_developmental_stage: Optional[str] = None - latest_elongation_ratio: Optional[float] = None + latest_nuclei_count: int | None = None + latest_developmental_stage: str | None = None + latest_elongation_ratio: float | None = None # Images (recent for context) - recent_images: List[ImageRecord] = field(default_factory=list) + recent_images: list[ImageRecord] = field(default_factory=list) # Keep last 10 for temporal context in Claude Vision calls # Focus history - accumulated piezo-galvo measurements over time - focus_history: List[FocusDataPoint] = field(default_factory=list) + focus_history: list[FocusDataPoint] = field(default_factory=list) # Each focus operation adds a datapoint, building a focus map for this embryo - def add_focus_datapoint(self, z: float = None, secondary_axis: float = 0.0, - score: float = 0.0, r_squared: float = 0.0, - method: str = 'manual', algorithm: str = 'fft_bandpass', - # Backward-compatible kwargs - galvo: float = None, piezo: float = None): + def add_focus_datapoint( + self, + z: float | None = None, + secondary_axis: float = 0.0, + score: float = 0.0, + r_squared: float = 0.0, + method: str = "manual", + algorithm: str = "fft_bandpass", + # Backward-compatible kwargs + galvo: float | None = None, + piezo: float | None = None, + ): """ Record a focus measurement for this embryo. @@ -268,19 +289,24 @@ def add_focus_datapoint(self, z: float = None, secondary_axis: float = 0.0, if galvo is not None: secondary_axis = galvo - self.focus_history.append(FocusDataPoint( - z=z, - secondary_axis=secondary_axis, - score=score, - r_squared=r_squared, - timestamp=datetime.now(), - method=method, - algorithm=algorithm, - )) - - def get_focus_at_secondary(self, secondary_position: float, - max_age_hours: Optional[float] = None, - min_r_squared: float = 0.5) -> Optional[float]: + self.focus_history.append( + FocusDataPoint( + z=z, + secondary_axis=secondary_axis, + score=score, + r_squared=r_squared, + timestamp=datetime.now(), + method=method, + algorithm=algorithm, + ) + ) + + def get_focus_at_secondary( + self, + secondary_position: float, + max_age_hours: float | None = None, + min_r_squared: float = 0.5, + ) -> float | None: """ Get the best Z position for a given secondary axis position. @@ -326,35 +352,38 @@ def get_focus_at_secondary(self, secondary_position: float, axis_distance = abs(fp.secondary_axis - secondary_position) age_hours = (now - fp.timestamp).total_seconds() / 3600 - candidates.append({ - 'z': fp.z, - 'axis_distance': axis_distance, - 'age_hours': age_hours, - 'r_squared': fp.r_squared, - }) + candidates.append( + { + "z": fp.z, + "axis_distance": axis_distance, + "age_hours": age_hours, + "r_squared": fp.r_squared, + } + ) if not candidates: return None # If we have exact matches, use the most recent - exact_matches = [c for c in candidates if c['axis_distance'] < 0.01] + exact_matches = [c for c in candidates if c["axis_distance"] < 0.01] if exact_matches: # Sort by recency, return most recent - exact_matches.sort(key=lambda x: x['age_hours']) - return exact_matches[0]['z'] + exact_matches.sort(key=lambda x: x["age_hours"]) + return exact_matches[0]["z"] # Otherwise, interpolate from nearby measurements # Sort by axis distance - candidates.sort(key=lambda x: x['axis_distance']) - return candidates[0]['z'] # Return closest match + candidates.sort(key=lambda x: x["axis_distance"]) + return candidates[0]["z"] # Return closest match # Backward-compatible alias - def get_focus_at_galvo(self, galvo_position: float, **kwargs) -> Optional[float]: + def get_focus_at_galvo(self, galvo_position: float, **kwargs) -> float | None: """Backward-compatible alias for get_focus_at_secondary.""" return self.get_focus_at_secondary(galvo_position, **kwargs) - def get_z_axis_fit(self, max_age_hours: Optional[float] = None, - min_r_squared: float = 0.5) -> Optional[Tuple[float, float]]: + def get_z_axis_fit( + self, max_age_hours: float | None = None, min_r_squared: float = 0.5 + ) -> tuple[float, float] | None: """ Fit a linear relationship between Z and secondary axis from accumulated data. @@ -396,24 +425,27 @@ def get_z_axis_fit(self, max_age_hours: Optional[float] = None, return None # Linear fit: z = slope * secondary_axis + intercept - secondary = np.array(secondary) - zs = np.array(zs) + secondary_arr = np.array(secondary) + zs_arr = np.array(zs) # Use polyfit for linear regression try: - coeffs = np.polyfit(secondary, zs, 1) + coeffs = np.polyfit(secondary_arr, zs_arr, 1) return (float(coeffs[0]), float(coeffs[1])) # slope, intercept except Exception: return None # Backward-compatible alias - def get_piezo_galvo_fit(self, **kwargs) -> Optional[Tuple[float, float]]: + def get_piezo_galvo_fit(self, **kwargs) -> tuple[float, float] | None: """Backward-compatible alias for get_z_axis_fit.""" return self.get_z_axis_fit(**kwargs) - def get_focus_drift_rate(self, secondary_position: float = 0.0, - galvo_position: float = None, - min_measurements: int = 3) -> Optional[float]: + def get_focus_drift_rate( + self, + secondary_position: float = 0.0, + galvo_position: float | None = None, + min_measurements: int = 3, + ) -> float | None: """ Calculate how fast focus is drifting (µm/hour) at a given secondary axis position. @@ -434,8 +466,11 @@ def get_focus_drift_rate(self, secondary_position: float = 0.0, if galvo_position is not None: secondary_position = galvo_position # Get measurements at similar secondary axis position - relevant = [fp for fp in self.focus_history - if abs(fp.secondary_axis - secondary_position) < 0.1 and fp.r_squared >= 0.5] + relevant = [ + fp + for fp in self.focus_history + if abs(fp.secondary_axis - secondary_position) < 0.1 and fp.r_squared >= 0.5 + ] if len(relevant) < min_measurements: return None @@ -462,9 +497,12 @@ def get_focus_drift_rate(self, secondary_position: float = 0.0, except Exception: return None - def needs_refocus(self, max_age_minutes: float = 60, - secondary_position: float = 0.0, - galvo_position: float = None) -> bool: + def needs_refocus( + self, + max_age_minutes: float = 60, + secondary_position: float = 0.0, + galvo_position: float | None = None, + ) -> bool: """ Determine if this embryo needs focus re-measurement. @@ -515,7 +553,8 @@ def get_focus_summary(self) -> str: lines = [ f"Focus history: {n_points} measurements over {span_hours:.1f} hours", - f"Latest: z={last.z:.2f}µm @ secondary={last.secondary_axis:.2f} (R²={last.r_squared:.3f})", + f"Latest: z={last.z:.2f}µm @ secondary={last.secondary_axis:.2f}" + f" (R²={last.r_squared:.3f})", ] if drift is not None: @@ -528,7 +567,7 @@ def get_focus_summary(self) -> str: return "\n".join(lines) - def add_detection_result(self, detector_name: str, result: Dict): + def add_detection_result(self, detector_name: str, result: dict): """ Add detection result from detector system @@ -544,7 +583,7 @@ def add_detection_result(self, detector_name: str, result: Dict): self.detection_results[detector_name].append(result) - def get_latest_detection(self, detector_name: str) -> Optional[Dict]: + def get_latest_detection(self, detector_name: str) -> dict | None: """Get most recent detection result for a detector""" if detector_name not in self.detection_results: return None @@ -573,15 +612,15 @@ def was_detected(self, detector_name: str, require_verified: bool = False) -> bo return False for result in self.detection_results[detector_name]: - if result.get('detected', False): + if result.get("detected", False): if require_verified: - if result.get('verified', False): + if result.get("verified", False): return True else: return True return False - def mark_detection_verified(self, detector_name: str, timepoint: Optional[int] = None) -> bool: + def mark_detection_verified(self, detector_name: str, timepoint: int | None = None) -> bool: """ Mark a detection result as verified by the challenger system. @@ -608,19 +647,19 @@ def mark_detection_verified(self, detector_name: str, timepoint: Optional[int] = if timepoint is not None: # Find by timepoint for result in results: - if result.get('timepoint') == timepoint and result.get('detected', False): - result['verified'] = True + if result.get("timepoint") == timepoint and result.get("detected", False): + result["verified"] = True return True else: # Mark the most recent detected result for result in reversed(results): - if result.get('detected', False): - result['verified'] = True + if result.get("detected", False): + result["verified"] = True return True return False - def add_cv_result(self, result_type: str, result: Dict): + def add_cv_result(self, result_type: str, result: dict): """ Add CV analysis result from CV subagent. @@ -635,8 +674,8 @@ def add_cv_result(self, result_type: str, result: Dict): self.cv_analyses[result_type] = [] # Add timestamp if not present - if 'timestamp' not in result: - result['timestamp'] = datetime.now().isoformat() + if "timestamp" not in result: + result["timestamp"] = datetime.now().isoformat() self.cv_analyses[result_type].append(result) @@ -648,11 +687,7 @@ def add_cv_result(self, result_type: str, result: Dict): elif result_type == "elongation" and "elongation_ratio" in result: self.latest_elongation_ratio = result["elongation_ratio"] - def get_cv_result( - self, - result_type: str, - timepoint: Optional[int] = None - ) -> Optional[Dict]: + def get_cv_result(self, result_type: str, timepoint: int | None = None) -> dict | None: """ Get CV analysis result, optionally filtered by timepoint. @@ -683,7 +718,7 @@ def get_cv_result( # Return most recent return results[-1] - def get_cv_summary(self) -> Dict: + def get_cv_summary(self) -> dict: """ Get summary of CV analysis results for display. @@ -697,27 +732,27 @@ def get_cv_summary(self) -> Dict: "developmental_stage": self.latest_developmental_stage, "elongation_ratio": self.latest_elongation_ratio, "analyses_count": { - result_type: len(results) - for result_type, results in self.cv_analyses.items() + result_type: len(results) for result_type, results in self.cv_analyses.items() }, } - def update_from_analysis(self, analysis_result: Dict): + def update_from_analysis(self, analysis_result: dict): """Update state with new analysis""" - if 'hatching' in analysis_result: - self.hatching_status = analysis_result['hatching'] + if "hatching" in analysis_result: + self.hatching_status = analysis_result["hatching"] - if 'morphology' in analysis_result: - self.morphology_history.append({ - 'timepoint': self.timepoints_acquired, - **analysis_result['morphology'] - }) + if "morphology" in analysis_result: + self.morphology_history.append( + {"timepoint": self.timepoints_acquired, **analysis_result["morphology"]} + ) - if 'fluorescence' in analysis_result: - self.fluorescence_history.append({ - 'timepoint': self.timepoints_acquired, - **analysis_result['fluorescence'] - }) + if "fluorescence" in analysis_result: + self.fluorescence_history.append( + { + "timepoint": self.timepoints_acquired, + **analysis_result["fluorescence"], + } + ) def to_summary(self) -> str: """Format for Claude system prompt""" @@ -741,7 +776,7 @@ def to_summary(self) -> str: status_parts.append("not yet imaged") # Status - if self.hatching_status.get('hatched'): + if self.hatching_status.get("hatched"): status_parts.append(f"hatched at t{self.hatching_status['timepoint']:04d}") elif self.should_skip: status_parts.append(f"skipped ({self.skip_reason})") @@ -757,7 +792,12 @@ def to_summary(self) -> str: return " | ".join(status_parts) - def record_exposure(self, exposure_ms: float, num_frames: int = 1, timestamp: Optional[datetime] = None): + def record_exposure( + self, + exposure_ms: float, + num_frames: int = 1, + timestamp: datetime | None = None, + ): """ Record light exposure for phototoxicity tracking. @@ -789,39 +829,69 @@ def get_exposure_summary(self) -> str: return f"{self.exposure_count} exposures, {time_str} total" - def to_dict(self) -> Dict: + @property + def stage_position(self) -> dict[str, float]: + """Resolved XY position — fine if SPIM-aligned, else coarse. + + Coarse comes from the bottom-camera detection / manual map placement. + Fine comes from the SPIM-objective alignment workflow (not built yet). + Callers that just want "where is this embryo" read this; callers that + care about calibration state read position_coarse / position_fine + directly. + """ + return self.position_fine if self.position_fine else self.position_coarse + + @stage_position.setter + def stage_position(self, value: dict[str, float]) -> None: + """Back-compat setter — writes to coarse. + + Legacy callers that assigned `embryo.stage_position = {...}` were + writing a bottom-camera / manual position; that's the coarse stage. + New code should set position_coarse or position_fine explicitly. + """ + self.position_coarse = value or {} + + @property + def has_fine_position(self) -> bool: + """True once SPIM-objective alignment has refined the coarse position.""" + return bool(self.position_fine) + + def to_dict(self) -> dict: """Serialize for API responses""" return { - 'id': self.id, - 'uid': self.uid, - 'nickname': self.nickname, - 'user_label': self.user_label, - 'role': self.role, - 'stage_position': self.stage_position, - 'calibration': self.calibration, - 'detection_confidence': self.detection_confidence, - 'interval_seconds': self.interval_seconds, - 'num_slices': self.num_slices, - 'exposure_ms': self.exposure_ms, - 'priority': self.priority, - 'acquisition_mode': self.acquisition_mode, - 'laser_power_488_pct': self.laser_power_488_pct, - 'last_imaged': self.last_imaged.isoformat() if self.last_imaged else None, - 'timepoints_acquired': self.timepoints_acquired, - 'should_skip': self.should_skip, - 'skip_reason': self.skip_reason, - 'exposure_count': self.exposure_count, - 'total_exposure_ms': self.total_exposure_ms, - 'hatching_status': self.hatching_status, - 'pending_verification': self.pending_verification, - 'consecutive_detection_count': self.consecutive_detection_count, - 'last_detection_round': self.last_detection_round, - 'recent_analyses': { - 'morphology': self.morphology_history[-5:] if self.morphology_history else [], - 'fluorescence': self.fluorescence_history[-5:] if self.fluorescence_history else [], - 'custom': self.custom_classifications, + "id": self.id, + "uid": self.uid, + "nickname": self.nickname, + "user_label": self.user_label, + "role": self.role, + "stage_position": self.stage_position, + "position_coarse": self.position_coarse, + "position_fine": self.position_fine, + "has_fine_position": self.has_fine_position, + "calibration": self.calibration, + "detection_confidence": self.detection_confidence, + "interval_seconds": self.interval_seconds, + "num_slices": self.num_slices, + "exposure_ms": self.exposure_ms, + "priority": self.priority, + "acquisition_mode": self.acquisition_mode, + "laser_power_488_pct": self.laser_power_488_pct, + "last_imaged": self.last_imaged.isoformat() if self.last_imaged else None, + "timepoints_acquired": self.timepoints_acquired, + "should_skip": self.should_skip, + "skip_reason": self.skip_reason, + "exposure_count": self.exposure_count, + "total_exposure_ms": self.total_exposure_ms, + "hatching_status": self.hatching_status, + "pending_verification": self.pending_verification, + "consecutive_detection_count": self.consecutive_detection_count, + "last_detection_round": self.last_detection_round, + "recent_analyses": { + "morphology": self.morphology_history[-5:] if self.morphology_history else [], + "fluorescence": self.fluorescence_history[-5:] if self.fluorescence_history else [], + "custom": self.custom_classifications, }, - 'focus_history': [fp.to_dict() for fp in self.focus_history], + "focus_history": [fp.to_dict() for fp in self.focus_history], } @@ -829,38 +899,71 @@ class ExperimentState: """Global experiment state""" def __init__(self): - self.embryos: Dict[str, EmbryoState] = {} - self.start_time: Optional[datetime] = None + self.embryos: dict[str, EmbryoState] = {} + self.start_time: datetime | None = None self.acquisition_status: str = "idle" # idle/running/paused/completed - self.current_plan_name: Optional[str] = None - self.plan_history: List[Dict] = [] - self.metadata: Dict = {} + self.current_plan_name: str | None = None + self.plan_history: list[dict] = [] + self.metadata: dict = {} # Active plan item — set during plan context resolution at startup. # When set, the agent's system prompt includes the full ImagingSpec # so it knows what it's here to do without being told. - self.active_plan_item_id: Optional[str] = None + self.active_plan_item_id: str | None = None # Session-level calibration prior for cross-embryo learning # Updated after each successful calibration, used to initialize subsequent embryos self.calibration_prior: CalibrationPrior = CalibrationPrior() - def add_embryo(self, embryo_id: str, position: Dict = None, - calibration: Dict = None, user_label: Optional[str] = None, - confidence: float = 0.0, uid: Optional[str] = None, - role: str = "test"): + # Observer hook — agent wires this at startup to publish EMBRYOS_UPDATE + # over the event bus. Kept as a plain callback so this module stays + # bus-agnostic. + self.on_embryos_changed: Callable[[], None] | None = None + + def notify_embryos_changed(self) -> None: + """Fire the on_embryos_changed observer if one is wired. + + Call this after any mutation the agent can't intercept through + add_embryo / remove_embryo (e.g. a direct write to + embryo.position_coarse). UI hooks must not raise — failures here are + swallowed so state mutations stay durable. + """ + cb = self.on_embryos_changed + if cb is None: + return + try: + cb() + except Exception: + logger.exception("ExperimentState.on_embryos_changed callback failed") + + def add_embryo( + self, + embryo_id: str, + position: dict | None = None, + calibration: dict | None = None, + user_label: str | None = None, + confidence: float = 0.0, + uid: str | None = None, + role: str = "test", + position_fine: dict | None = None, + ): """Register new embryo. ``role`` must be a key in :data:`gently.harness.roles.REGISTRY` (e.g. ``"test"``, ``"calibration"``, ``"unassigned"``). Unknown roles raise KeyError. + `position` is the coarse XY (bottom-camera detection or manual map + placement). `position_fine` is reserved for the future SPIM-objective + alignment workflow and defaults to empty. + Emits an ``EMBRYO_DETECTED`` event so listeners (e.g. the viz server's TimelapseStateTracker, which feeds the device map) learn about marked embryos immediately — not just after the first acquisition. """ from gently.harness.roles import get_role + get_role(role) # raises KeyError if unknown # Auto-start experiment when first embryo is added @@ -871,17 +974,20 @@ def add_embryo(self, embryo_id: str, position: Dict = None, self.embryos[embryo_id] = EmbryoState( id=embryo_id, uid=uid, - stage_position=pos, + position_coarse=position or {}, + position_fine=position_fine or {}, calibration=calibration or {}, user_label=user_label, detection_confidence=confidence, role=role, ) + self.notify_embryos_changed() # Fire the registration event. Late-bound import keeps this module # decoupled from the event bus until first use. try: from gently.core import EventType, get_event_bus + get_event_bus().publish( event_type=EventType.EMBRYO_DETECTED, data={ @@ -903,6 +1009,7 @@ def remove_embryo(self, embryo_id: str) -> bool: """Remove embryo from experiment (e.g., false detection)""" if embryo_id in self.embryos: del self.embryos[embryo_id] + self.notify_embryos_changed() return True return False @@ -910,8 +1017,9 @@ def assign_nickname(self, embryo_id: str, nickname: str): """Agent assigns intuitive name""" if embryo_id in self.embryos: self.embryos[embryo_id].nickname = nickname + self.notify_embryos_changed() - def get_embryo_by_any_name(self, name: str) -> Optional[EmbryoState]: + def get_embryo_by_any_name(self, name: str) -> EmbryoState | None: """Get embryo by ID, nickname, or user label""" # Direct ID match if name in self.embryos: @@ -923,7 +1031,7 @@ def get_embryo_by_any_name(self, name: str) -> Optional[EmbryoState]: return embryo # Try extracting number from name like "embryo 3" -> "embryo_3" - match = re.search(r'(\d+)', name) + match = re.search(r"(\d+)", name) if match: num = int(match.group(1)) # Try simple format first (embryo_3) @@ -951,7 +1059,7 @@ def get_summary(self) -> str: f"Duration: {hours}h {minutes}m", f"Embryos: {len(self.embryos)}", "", - "Per-embryo status:" + "Per-embryo status:", ] for embryo in sorted(self.embryos.values(), key=lambda e: e.id): @@ -965,15 +1073,15 @@ def get_summary(self) -> str: return "\n".join(lines) - def to_dict(self) -> Dict: + def to_dict(self) -> dict: """Serialize for API responses""" return { - 'start_time': self.start_time.isoformat() if self.start_time else None, - 'acquisition_status': self.acquisition_status, - 'current_plan_name': self.current_plan_name, - 'active_plan_item_id': self.active_plan_item_id, - 'embryo_count': len(self.embryos), - 'embryos': {eid: e.to_dict() for eid, e in self.embryos.items()}, - 'metadata': self.metadata, - 'calibration_prior': self.calibration_prior.to_dict(), + "start_time": self.start_time.isoformat() if self.start_time else None, + "acquisition_status": self.acquisition_status, + "current_plan_name": self.current_plan_name, + "active_plan_item_id": self.active_plan_item_id, + "embryo_count": len(self.embryos), + "embryos": {eid: e.to_dict() for eid, e in self.embryos.items()}, + "metadata": self.metadata, + "calibration_prior": self.calibration_prior.to_dict(), } diff --git a/gently/harness/tools/helpers.py b/gently/harness/tools/helpers.py index 3d6b03b8..b4c58f39 100644 --- a/gently/harness/tools/helpers.py +++ b/gently/harness/tools/helpers.py @@ -5,17 +5,39 @@ used across multiple tools to reduce code duplication. """ -from typing import Any, Dict, List, Optional, Tuple from datetime import datetime +from typing import Any -def require_agent(context: Dict) -> Tuple[Optional[Any], Optional[str]]: +def ctx_get(context: dict | None, key: str) -> Any: + """ + Look up a key in a (possibly missing) tool execution context + + Parameters + ---------- + context : dict | None + Tool execution context + key : str + Key to look up + + Returns + ------- + Any + The value for ``key``, or ``None`` if ``context`` is ``None`` or + the key is absent. + """ + if context is None: + return None + return context.get(key) + + +def require_agent(context: dict | None) -> tuple[Any, str | None]: """ Extract agent from context or return error message Parameters ---------- - context : dict + context : dict | None Tool execution context Returns @@ -23,13 +45,13 @@ def require_agent(context: Dict) -> Tuple[Optional[Any], Optional[str]]: tuple (agent, None) if found, (None, error_message) if not """ - agent = context.get('agent') + agent = ctx_get(context, "agent") if not agent: return None, "Error: No agent context" return agent, None -def get_embryo_or_error(agent, embryo_id: str) -> Tuple[Optional[Any], Optional[str]]: +def get_embryo_or_error(agent, embryo_id: str) -> tuple[Any, str | None]: """ Get embryo by any name or return error message @@ -51,13 +73,13 @@ def get_embryo_or_error(agent, embryo_id: str) -> Tuple[Optional[Any], Optional[ return embryo, None -def require_microscope(context: Dict) -> Tuple[Optional[Any], Optional[str]]: +def require_microscope(context: dict | None) -> tuple[Any, str | None]: """ Get microscope client from context or return error message Parameters ---------- - context : dict + context : dict | None Tool execution context Returns @@ -65,13 +87,13 @@ def require_microscope(context: Dict) -> Tuple[Optional[Any], Optional[str]]: tuple (client, None) if connected, (None, error_message) if not """ - client = context.get('client') + client = ctx_get(context, "client") if not client: return None, "Not connected to microscope. Use connect_microscope first." return client, None -def require_interaction_logger(agent) -> Tuple[Optional[Any], Optional[str]]: +def require_interaction_logger(agent) -> tuple[Any, str | None]: """ Get interaction logger or return error message @@ -85,12 +107,12 @@ def require_interaction_logger(agent) -> Tuple[Optional[Any], Optional[str]]: tuple (logger, None) if available, (None, error_message) if not """ - if not hasattr(agent, 'interaction_logger') or not agent.interaction_logger: + if not hasattr(agent, "interaction_logger") or not agent.interaction_logger: return None, "Interaction logging not enabled." return agent.interaction_logger, None -def require_developmental_tracker(agent) -> Tuple[Optional[Any], Optional[str]]: +def require_developmental_tracker(agent) -> tuple[Any, str | None]: """ Get developmental tracker or return error message @@ -104,12 +126,15 @@ def require_developmental_tracker(agent) -> Tuple[Optional[Any], Optional[str]]: tuple (tracker, None) if available, (None, error_message) if not """ - if not hasattr(agent, 'developmental_tracker') or not agent.developmental_tracker: - return None, "No stage classifications recorded yet. Use classify_embryo_stage first." + if not hasattr(agent, "developmental_tracker") or not agent.developmental_tracker: + return ( + None, + "No stage classifications recorded yet. Use classify_embryo_stage first.", + ) return agent.developmental_tracker, None -def require_timelapse_orchestrator(agent) -> Tuple[Optional[Any], Optional[str]]: +def require_timelapse_orchestrator(agent) -> tuple[Any, str | None]: """ Get timelapse orchestrator or return error message @@ -123,12 +148,12 @@ def require_timelapse_orchestrator(agent) -> Tuple[Optional[Any], Optional[str]] tuple (orchestrator, None) if available, (None, error_message) if not """ - if not hasattr(agent, 'timelapse_orchestrator') or agent.timelapse_orchestrator is None: + if not hasattr(agent, "timelapse_orchestrator") or agent.timelapse_orchestrator is None: return None, "Timelapse orchestrator not initialized." return agent.timelapse_orchestrator, None -def require_databroker(agent) -> Tuple[Optional[Any], Optional[str]]: +def require_databroker(agent) -> tuple[Any, str | None]: """ Get databroker connection or return error message @@ -142,7 +167,7 @@ def require_databroker(agent) -> Tuple[Optional[Any], Optional[str]]: tuple (databroker, None) if available, (None, error_message) if not """ - if not hasattr(agent, 'databroker') or agent.databroker is None: + if not hasattr(agent, "databroker") or agent.databroker is None: return None, "No databroker connection. Data persistence not available." return agent.databroker, None @@ -184,13 +209,13 @@ def format_duration(seconds: float) -> str: def build_snapshot_metadata( - stage_position: Tuple[float, float], - image_shape: Tuple[int, ...], + stage_position: tuple[float, float], + image_shape: tuple[int, ...], experiment=None, pixel_size_um: float = 6.5, objective_mag: float = 10.0, - safety_limits: Optional[Dict] = None, -) -> Dict: + safety_limits: dict | None = None, +) -> dict: """Build metadata dict for a bottom camera snapshot. Captures everything needed to reconstruct embryo positions @@ -226,7 +251,7 @@ def build_snapshot_metadata( # gently/hardware/dispim/devices/stage.py::DiSPIMXYStage.__init__. safety_limits = {"x": (2000.0, 4000.0), "y": (-1000.0, 1000.0)} - meta: Dict[str, Any] = { + meta: dict[str, Any] = { "stage_x": stage_position[0], "stage_y": stage_position[1], "image_width": w, @@ -242,15 +267,17 @@ def build_snapshot_metadata( } if experiment and experiment.embryos: - embryos: List[Dict] = [] + embryos: list[dict] = [] for eid, emb in experiment.embryos.items(): pos = emb.stage_position or {} - embryos.append({ - "embryo_id": eid, - "stage_x": pos.get("x"), - "stage_y": pos.get("y"), - "nickname": getattr(emb, "nickname", None), - }) + embryos.append( + { + "embryo_id": eid, + "stage_x": pos.get("x"), + "stage_y": pos.get("y"), + "nickname": getattr(emb, "nickname", None), + } + ) meta["embryos"] = embryos return meta diff --git a/gently/harness/tools/registry.py b/gently/harness/tools/registry.py index 106bd7a2..040487aa 100644 --- a/gently/harness/tools/registry.py +++ b/gently/harness/tools/registry.py @@ -13,49 +13,56 @@ import functools import inspect import logging +import time +from collections.abc import Callable from dataclasses import dataclass, field from enum import Enum, auto from typing import ( - Any, Callable, Dict, List, Optional, Type, Union, - get_type_hints, get_origin, get_args + Any, + Union, + get_args, + get_origin, + get_type_hints, ) -import time logger = logging.getLogger(__name__) class ToolCategory(Enum): """Categories for organizing tools""" - ACQUISITION = auto() # Volume/image acquisition - MOVEMENT = auto() # Stage movement, positioning - CALIBRATION = auto() # Calibration procedures - ANALYSIS = auto() # Image/volume analysis - DETECTION = auto() # Detector management - EXPERIMENT = auto() # Experiment state management - EMBRYO = auto() # Embryo-specific operations - HARDWARE = auto() # Direct hardware control - DATA = auto() # Data/Databroker operations - UTILITY = auto() # Utility functions - ML = auto() # Machine learning training - TRANSFER = auto() # Bulk data transfer + + ACQUISITION = auto() # Volume/image acquisition + MOVEMENT = auto() # Stage movement, positioning + CALIBRATION = auto() # Calibration procedures + ANALYSIS = auto() # Image/volume analysis + DETECTION = auto() # Detector management + EXPERIMENT = auto() # Experiment state management + EMBRYO = auto() # Embryo-specific operations + HARDWARE = auto() # Direct hardware control + DATA = auto() # Data/Databroker operations + UTILITY = auto() # Utility functions + ML = auto() # Machine learning training + TRANSFER = auto() # Bulk data transfer @dataclass class ToolParameter: """Definition of a tool parameter""" + name: str type: str # JSON schema type description: str required: bool = True default: Any = None - enum: Optional[List[str]] = None + enum: list[str] | None = None @dataclass class ToolExample: """Example of when to use a tool""" + user_query: str - tool_input: Dict = field(default_factory=dict) + tool_input: dict = field(default_factory=dict) @dataclass @@ -69,23 +76,24 @@ class ToolDefinition: - Documentation - Filtering/discovery """ + name: str description: str handler: Callable - parameters: List[ToolParameter] = field(default_factory=list) + parameters: list[ToolParameter] = field(default_factory=list) category: ToolCategory = ToolCategory.UTILITY requires_microscope: bool = False is_async: bool = False - tags: List[str] = field(default_factory=list) - examples: List[ToolExample] = field(default_factory=list) + tags: list[str] = field(default_factory=list) + examples: list[ToolExample] = field(default_factory=list) - def to_claude_schema(self) -> Dict: + def to_claude_schema(self) -> dict: """Generate Claude API tool schema with examples embedded in description""" properties = {} required = [] for param in self.parameters: - prop = { + prop: dict[str, Any] = { "type": param.type, "description": param.description, } @@ -114,7 +122,7 @@ def to_claude_schema(self) -> Dict: "type": "object", "properties": properties, "required": required, - } + }, } @@ -156,29 +164,78 @@ def _python_type_to_json_schema(python_type) -> str: return "string" -def _extract_parameters_from_function(func: Callable) -> List[ToolParameter]: +def _unwrap_optional(tp: Any) -> Any: + """Reduce ``float | None`` / ``Optional[int]`` to the underlying scalar type. + + Returns the single non-None member of a union, or the type unchanged for a + plain annotation. Returns None when there's no unambiguous scalar (so callers + skip coercion). + """ + args = get_args(tp) + if args: + non_none = [a for a in args if a is not type(None)] + return non_none[0] if len(non_none) == 1 else None + return tp + + +def _coerce_kwargs(handler: Callable, kwargs: dict) -> dict: + """Best-effort coercion of string tool args to their annotated scalar types. + + Tool inputs arrive as JSON from the model (and sometimes as UI form strings), + so a param annotated ``float``/``int`` can show up as e.g. ``"120"``. Without + this, a downstream numeric comparison raises + ``'<' not supported between instances of 'str' and 'int'``. Coercion is + conservative: only string values whose annotation resolves to int/float/bool + are touched; anything that fails to parse is left as-is for the tool to report. + """ + try: + hints = get_type_hints(handler) + except Exception: + return kwargs + for name, value in list(kwargs.items()): + if name == "context" or not isinstance(value, str): + continue + target = _unwrap_optional(hints.get(name)) + if target in (int, float): + s = value.strip() + if not s: + continue + try: + kwargs[name] = target(s) + except (ValueError, TypeError): + pass + elif target is bool: + low = value.strip().lower() + if low in ("true", "1", "yes", "on"): + kwargs[name] = True + elif low in ("false", "0", "no", "off"): + kwargs[name] = False + return kwargs + + +def _extract_parameters_from_function(func: Callable) -> list[ToolParameter]: """Extract parameter definitions from function signature and type hints""" sig = inspect.signature(func) - hints = get_type_hints(func) if hasattr(func, '__annotations__') else {} + hints = get_type_hints(func) if hasattr(func, "__annotations__") else {} doc = inspect.getdoc(func) or "" # Parse docstring for parameter descriptions param_docs = {} in_params = False current_param = None - for line in doc.split('\n'): + for line in doc.split("\n"): line = line.strip() - if line.lower().startswith('parameters'): + if line.lower().startswith("parameters"): in_params = True continue if in_params: - if line.startswith('---'): + if line.startswith("---"): continue - if line.lower().startswith('returns'): + if line.lower().startswith("returns"): in_params = False continue - if ' : ' in line: - parts = line.split(' : ') + if " : " in line: + parts = line.split(" : ") current_param = parts[0].strip() param_docs[current_param] = "" elif current_param and line: @@ -187,7 +244,7 @@ def _extract_parameters_from_function(func: Callable) -> List[ToolParameter]: parameters = [] for param_name, param in sig.parameters.items(): # Skip 'self', 'tool_input' (legacy pattern), and 'context' (injected at runtime) - if param_name in ('self', 'tool_input', 'context'): + if param_name in ("self", "tool_input", "context"): continue python_type = hints.get(param_name, str) @@ -200,13 +257,15 @@ def _extract_parameters_from_function(func: Callable) -> List[ToolParameter]: # Get description from docstring description = param_docs.get(param_name, f"The {param_name} parameter").strip() - parameters.append(ToolParameter( - name=param_name, - type=json_type, - description=description, - required=required, - default=default, - )) + parameters.append( + ToolParameter( + name=param_name, + type=json_type, + description=description, + required=required, + default=default, + ) + ) return parameters @@ -223,8 +282,8 @@ class ToolRegistry: """ def __init__(self): - self._tools: Dict[str, ToolDefinition] = {} - self._context: Dict[str, Any] = {} # Shared context (agent, client, etc.) + self._tools: dict[str, ToolDefinition] = {} + self._context: dict[str, Any] = {} # Shared context (agent, client, etc.) def set_context(self, key: str, value: Any): """Set shared context available to all tools""" @@ -236,13 +295,13 @@ def get_context(self, key: str) -> Any: def register( self, - name: Optional[str] = None, - description: Optional[str] = None, + name: str | None = None, + description: str | None = None, category: ToolCategory = ToolCategory.UTILITY, requires_microscope: bool = False, - tags: Optional[List[str]] = None, - parameters: Optional[List[ToolParameter]] = None, - examples: Optional[List[ToolExample]] = None, + tags: list[str] | None = None, + parameters: list[ToolParameter] | None = None, + examples: list[ToolExample] | None = None, ) -> Callable: """ Decorator to register a function as a tool @@ -277,9 +336,10 @@ async def acquire_volume(embryo_id: str, num_slices: int = 50) -> str: examples : list of ToolExample, optional Usage examples showing when to call this tool """ + def decorator(func: Callable) -> Callable: tool_name = name or func.__name__ - tool_desc = description or (inspect.getdoc(func) or "").split('\n')[0] + tool_desc = description or (inspect.getdoc(func) or "").split("\n")[0] # Extract or use provided parameters tool_params = parameters or _extract_parameters_from_function(func) @@ -311,11 +371,11 @@ async def wrapper(*args, **kwargs): def register_function( self, func: Callable, - name: Optional[str] = None, - description: Optional[str] = None, + name: str | None = None, + description: str | None = None, category: ToolCategory = ToolCategory.UTILITY, requires_microscope: bool = False, - tags: Optional[List[str]] = None, + tags: list[str] | None = None, ): """ Register an existing function as a tool (non-decorator form) @@ -336,7 +396,7 @@ def register_function( Additional tags """ tool_name = name or func.__name__ - tool_desc = description or (inspect.getdoc(func) or "").split('\n')[0] + tool_desc = description or (inspect.getdoc(func) or "").split("\n")[0] tool_params = _extract_parameters_from_function(func) tool_def = ToolDefinition( @@ -360,23 +420,23 @@ def unregister(self, name: str) -> bool: return True return False - def get(self, name: str) -> Optional[ToolDefinition]: + def get(self, name: str) -> ToolDefinition | None: """Get tool definition by name""" return self._tools.get(name) - def list_all(self) -> List[ToolDefinition]: + def list_all(self) -> list[ToolDefinition]: """List all registered tools""" return list(self._tools.values()) - def list_by_category(self, category: ToolCategory) -> List[ToolDefinition]: + def list_by_category(self, category: ToolCategory) -> list[ToolDefinition]: """List tools in a category""" return [t for t in self._tools.values() if t.category == category] - def list_by_tag(self, tag: str) -> List[ToolDefinition]: + def list_by_tag(self, tag: str) -> list[ToolDefinition]: """List tools with a specific tag""" return [t for t in self._tools.values() if tag in t.tags] - def list_available(self, has_microscope: bool = False) -> List[ToolDefinition]: + def list_available(self, has_microscope: bool = False) -> list[ToolDefinition]: """List tools available given current context""" tools = [] for tool in self._tools.values(): @@ -385,14 +445,11 @@ def list_available(self, has_microscope: bool = False) -> List[ToolDefinition]: tools.append(tool) return tools - def get_claude_schemas(self, has_microscope: bool = False) -> List[Dict]: + def get_claude_schemas(self, has_microscope: bool = False) -> list[dict]: """Get Claude API tool schemas for available tools""" - return [ - tool.to_claude_schema() - for tool in self.list_available(has_microscope) - ] + return [tool.to_claude_schema() for tool in self.list_available(has_microscope)] - async def execute(self, tool_name: str, tool_input: Dict, context: Dict = None) -> str: + async def execute(self, tool_name: str, tool_input: dict, context: dict | None = None) -> str: """ Execute a tool by name @@ -420,14 +477,30 @@ async def execute(self, tool_name: str, tool_input: Dict, context: Dict = None) # 3. Fall back to stored registry context if context is not None: exec_context = context - elif 'context' in tool_input and tool_input['context'] is not None: - exec_context = tool_input['context'] + elif "context" in tool_input and tool_input["context"] is not None: + exec_context = tool_input["context"] else: exec_context = self._context + # Hybrid-autonomy backstop: during an autonomous (wake) turn, a small set + # of irreversible tools (laser-on, embryo termination, stopping the run) + # must NEVER execute without a human — even if the model tries to call + # them directly. The agent sets these flags around its autonomous turns; + # user-driven turns are unaffected. The blocked set is supplied by the + # agent so this layer stays free of app-specific tool names. + _agent = exec_context.get("agent") if isinstance(exec_context, dict) else None + if _agent is not None and getattr(_agent, "_autonomous_active", False): + blocked = getattr(_agent, "_autonomous_blocked_tools", None) or () + if tool_name in blocked: + logger.info("Autonomy backstop blocked '%s' (irreversible)", tool_name) + return ( + f"'{tool_name}' is an irreversible action and cannot run " + f"autonomously. Ask the operator to confirm it." + ) + # Check microscope requirement if tool.requires_microscope: - client = exec_context.get('client') + client = exec_context.get("client") if client is None: return "Error: Not connected to microscope server. Start the server and reconnect." @@ -437,10 +510,15 @@ async def execute(self, tool_name: str, tool_input: Dict, context: Dict = None) # Prepare arguments kwargs = dict(tool_input) + # Coerce string args to their annotated scalar types. JSON/UI inputs + # can deliver e.g. new_interval_seconds="120", which would otherwise + # crash on a numeric comparison inside the tool. + kwargs = _coerce_kwargs(tool.handler, kwargs) + # Inject context if handler expects it (but don't overwrite if already provided) sig = inspect.signature(tool.handler) - if 'context' in sig.parameters and 'context' not in kwargs: - kwargs['context'] = exec_context + if "context" in sig.parameters and "context" not in kwargs: + kwargs["context"] = exec_context # Execute handler if tool.is_async: @@ -455,6 +533,7 @@ async def execute(self, tool_name: str, tool_input: Dict, context: Dict = None) except Exception as e: import traceback + logger.error(f"Tool {tool_name} failed: {e}") return f"Error executing {tool_name}: {str(e)}\n{traceback.format_exc()}" @@ -466,7 +545,7 @@ def __len__(self) -> int: # Global registry instance -_global_registry: Optional[ToolRegistry] = None +_global_registry: ToolRegistry | None = None def get_tool_registry() -> ToolRegistry: @@ -485,12 +564,12 @@ def set_tool_registry(registry: ToolRegistry): # Convenience decorator using global registry def tool( - name: Optional[str] = None, - description: Optional[str] = None, + name: str | None = None, + description: str | None = None, category: ToolCategory = ToolCategory.UTILITY, requires_microscope: bool = False, - tags: Optional[List[str]] = None, - examples: Optional[List[ToolExample]] = None, + tags: list[str] | None = None, + examples: list[ToolExample] | None = None, ) -> Callable: """ Decorator to register a tool with the global registry diff --git a/gently/log_config.py b/gently/log_config.py index 4db29ad1..9adf93f1 100644 --- a/gently/log_config.py +++ b/gently/log_config.py @@ -9,6 +9,7 @@ GENTLY_LOG_FORMAT — console format string GENTLY_LOG_DATEFMT — timestamp format (default: %H:%M:%S) """ + import logging import os import sys @@ -19,8 +20,8 @@ def configure_logging( - level: str = None, - log_file: str = None, + level: str | None = None, + log_file: str | None = None, ): """Configure root logger for the Gently system. @@ -32,7 +33,7 @@ def configure_logging( # the standard streams to UTF-8 with replacement so logging never raises. for _stream in (sys.stdout, sys.stderr): try: - _stream.reconfigure(encoding="utf-8", errors="replace") + _stream.reconfigure(encoding="utf-8", errors="replace") # type: ignore[union-attr] except (AttributeError, OSError): pass @@ -54,11 +55,29 @@ def configure_logging( lgr.addHandler(console) # Suppress noisy third-party loggers on console - for name in ("uvicorn", "uvicorn.error", "uvicorn.access", - "httpx", "httpcore", "anthropic", "aiohttp", - "aiohttp.access", "bluesky", "bluesky.RE.state"): + for name in ( + "uvicorn", + "uvicorn.error", + "uvicorn.access", + "httpx", + "httpcore", + "anthropic", + "aiohttp", + "aiohttp.access", + "bluesky", + "bluesky.RE.state", + ): logging.getLogger(name).setLevel(logging.WARNING) + # The `websockets` library logs a full "data transfer failed" traceback at + # ERROR level every time a client disconnects ungracefully (e.g. a browser + # tab sleeping or dropping — Windows raises WinError 121, "semaphore timeout"). + # These are routine, not faults, and flood the console hundreds of lines deep, + # burying real errors. Suppress below CRITICAL so a genuinely fatal WS fault + # still surfaces. WARNING is not enough here because the noise is ERROR-level. + for name in ("websockets", "websockets.server", "websockets.client"): + logging.getLogger(name).setLevel(logging.CRITICAL) + # File handler — always INFO+ regardless of console level if log_file: file_fmt = os.environ.get("GENTLY_LOG_FILE_FORMAT", _DEFAULT_FILE_FORMAT) diff --git a/gently/mesh/audit.py b/gently/mesh/audit.py index 06e76729..e4f06746 100644 --- a/gently/mesh/audit.py +++ b/gently/mesh/audit.py @@ -53,7 +53,7 @@ def _count_lines(self): """Count existing lines for rotation tracking.""" if self._log_file.exists(): try: - with open(self._log_file, "r") as f: + with open(self._log_file) as f: self._line_count = sum(1 for _ in f) except OSError: self._line_count = 0 @@ -88,7 +88,7 @@ def log( def _rotate(self): """Keep last KEEP_LINES, discard the rest.""" try: - with open(self._log_file, "r") as f: + with open(self._log_file) as f: lines = f.readlines() keep = lines[-KEEP_LINES:] with open(self._log_file, "w") as f: diff --git a/gently/mesh/capability_provider.py b/gently/mesh/capability_provider.py index 2453dce9..d82f691c 100644 --- a/gently/mesh/capability_provider.py +++ b/gently/mesh/capability_provider.py @@ -8,18 +8,19 @@ import logging import os import platform -from typing import Any, Dict, List, Optional +from typing import Any from .models import DatasetAdvertisement, GpuInfo, PeerRole logger = logging.getLogger(__name__) -def _detect_gpus() -> List[GpuInfo]: +def _detect_gpus() -> list[GpuInfo]: """Detect available NVIDIA GPUs via torch.cuda (pynvml fallback).""" gpus = [] try: import torch + if torch.cuda.is_available(): for i in range(torch.cuda.device_count()): props = torch.cuda.get_device_properties(i) @@ -28,23 +29,26 @@ def _detect_gpus() -> List[GpuInfo]: mem_used_gb = 0.0 try: import pynvml + pynvml.nvmlInit() handle = pynvml.nvmlDeviceGetHandleByIndex(i) util = pynvml.nvmlDeviceGetUtilizationRates(handle) util_pct = float(util.gpu) mem_info = pynvml.nvmlDeviceGetMemoryInfo(handle) - mem_used_gb = mem_info.used / (1024 ** 3) + mem_used_gb = mem_info.used / (1024**3) except Exception: pass - gpus.append(GpuInfo( - device_index=i, - name=props.name, - vram_gb=round(props.total_mem / (1024 ** 3), 1), - compute_capability=f"{props.major}.{props.minor}", - utilization_pct=util_pct, - memory_used_gb=round(mem_used_gb, 2), - )) + gpus.append( + GpuInfo( + device_index=i, + name=props.name, + vram_gb=round(props.total_mem / (1024**3), 1), + compute_capability=f"{props.major}.{props.minor}", + utilization_pct=util_pct, + memory_used_gb=round(mem_used_gb, 2), + ) + ) except ImportError: pass except Exception as e: @@ -52,14 +56,16 @@ def _detect_gpus() -> List[GpuInfo]: return gpus -def _get_system_info() -> Dict[str, Any]: +def _get_system_info() -> dict[str, Any]: """Get CPU and RAM info.""" cpu_cores = os.cpu_count() or 0 ram_gb = 0.0 try: if platform.system() == "Windows": import ctypes - kernel32 = ctypes.windll.kernel32 + + # ctypes.windll exists only on Windows; this branch is platform-guarded. + kernel32 = ctypes.windll.kernel32 # type: ignore[attr-defined] mem_status = ctypes.c_ulonglong() kernel32.GetPhysicallyInstalledSystemMemory(ctypes.byref(mem_status)) ram_gb = round(mem_status.value / (1024 * 1024), 1) @@ -92,12 +98,12 @@ def __init__( self, gently_store=None, device_layer=None, - static_caps: Optional[Dict[str, Any]] = None, + static_caps: dict[str, Any] | None = None, ): self._store = gently_store self._device_layer = device_layer self._static = static_caps or {} - self._gpus: List[GpuInfo] = [] + self._gpus: list[GpuInfo] = [] self._system_info = _get_system_info() # Initial GPU detection (cached, refreshed on demand) self._gpus = _detect_gpus() @@ -106,7 +112,7 @@ def refresh_gpus(self): """Re-detect GPUs (call periodically for live utilization).""" self._gpus = _detect_gpus() - def _compute_roles(self) -> List[str]: + def _compute_roles(self) -> list[str]: """Determine dynamic roles based on current state.""" roles = [] # Microscope controller if device is connected and responding @@ -131,7 +137,7 @@ def _compute_roles(self) -> List[str]: roles.append(PeerRole.PLANNER.value) return roles - def _get_datasets(self) -> List[DatasetAdvertisement]: + def _get_datasets(self) -> list[DatasetAdvertisement]: """Query FileStore for dataset advertisements.""" if self._store is None: return [] @@ -164,15 +170,17 @@ def _get_datasets(self) -> List[DatasetAdvertisement]: except Exception: pass - datasets.append(DatasetAdvertisement( - session_id=sid, - session_name=sname, - embryo_count=embryo_count, - volume_count=vol_count, - has_ground_truth=gt_count > 0, - ground_truth_count=gt_count, - stages_covered=sorted(stages), - )) + datasets.append( + DatasetAdvertisement( + session_id=sid, + session_name=sname, + embryo_count=embryo_count, + volume_count=vol_count, + has_ground_truth=gt_count > 0, + ground_truth_count=gt_count, + stages_covered=sorted(stages), + ) + ) except Exception as e: logger.debug(f"Dataset advertisement failed: {e}") return datasets @@ -186,7 +194,7 @@ def _is_microscope_connected(self) -> bool: pass return False - def __call__(self) -> Dict[str, Any]: + def __call__(self) -> dict[str, Any]: """Build the full capability dict. Called on each heartbeat.""" datasets = self._get_datasets() roles = self._compute_roles() @@ -199,10 +207,12 @@ def __call__(self) -> Dict[str, Any]: storage_total_gb = 0.0 try: import shutil + from ..settings import settings + usage = shutil.disk_usage(str(settings.storage.base_path)) - storage_free_gb = round(usage.free / (1024 ** 3), 1) - storage_total_gb = round(usage.total / (1024 ** 3), 1) + storage_free_gb = round(usage.free / (1024**3), 1) + storage_total_gb = round(usage.total / (1024**3), 1) except Exception: pass diff --git a/gently/mesh/discovery.py b/gently/mesh/discovery.py index 29fecd4a..ca6394ab 100644 --- a/gently/mesh/discovery.py +++ b/gently/mesh/discovery.py @@ -15,7 +15,7 @@ import logging import socket import time -from typing import Callable, Optional +from collections.abc import Callable from ..settings import settings @@ -74,9 +74,12 @@ def __init__( self._pairing_manager = pairing_manager self._audit_log = audit_log self._known_ids: set = set() - self.transport: Optional[asyncio.DatagramTransport] = None + self.transport: asyncio.DatagramTransport | None = None - def connection_made(self, transport: asyncio.DatagramTransport): + def connection_made(self, transport: asyncio.BaseTransport) -> None: + # DatagramProtocol always receives a DatagramTransport here; narrow for + # the typed attribute (and matches the asyncio.BaseProtocol signature). + assert isinstance(transport, asyncio.DatagramTransport) self.transport = transport def datagram_received(self, data: bytes, addr: tuple): @@ -101,9 +104,12 @@ def datagram_received(self, data: bytes, addr: tuple): logger.debug(f"Mesh: rejected stale packet from {peer_id[:8]} (ts={ts})") if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.REPLAY_REJECTED, outcome="deny", - peer_id=peer_id, ip=sender_ip, + AuditEvent.REPLAY_REJECTED, + outcome="deny", + peer_id=peer_id, + ip=sender_ip, detail=f"ts_delta={abs(time.time() - ts):.1f}s", ) return @@ -120,9 +126,12 @@ def datagram_received(self, data: bytes, addr: tuple): logger.debug(f"Mesh: bad signature from {peer_id[:8]}") if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.SIG_INVALID, outcome="deny", - peer_id=peer_id, ip=sender_ip, + AuditEvent.SIG_INVALID, + outcome="deny", + peer_id=peer_id, + ip=sender_ip, ) if msg_type == "nudge": @@ -138,7 +147,7 @@ def datagram_received(self, data: bytes, addr: tuple): def error_received(self, exc: Exception): logger.debug(f"Mesh UDP error: {exc}") - def connection_lost(self, exc: Optional[Exception]): + def connection_lost(self, exc: Exception | None): pass def forget_peer(self, instance_id: str): @@ -169,9 +178,9 @@ def __init__( self._pairing_manager = pairing_manager self._audit_log = audit_log - self._protocol: Optional[_MeshProtocol] = None - self._transport: Optional[asyncio.DatagramTransport] = None - self._broadcast_task: Optional[asyncio.Task] = None + self._protocol: _MeshProtocol | None = None + self._transport: asyncio.DatagramTransport | None = None + self._broadcast_task: asyncio.Task | None = None self._running = False # Callbacks — set by MeshService before start() @@ -213,8 +222,7 @@ async def start(self): self._running = True self._broadcast_task = asyncio.create_task(self._broadcast_loop()) logger.info( - f"Mesh discovery started on port {self.mesh_port} " - f"(instance={self.instance_id[:8]})" + f"Mesh discovery started on port {self.mesh_port} (instance={self.instance_id[:8]})" ) async def stop(self): @@ -259,9 +267,7 @@ def send_nudge(self): packet = json.dumps(payload).encode("utf-8") try: - self._transport.sendto( - packet, ("255.255.255.255", self.mesh_port) - ) + self._transport.sendto(packet, ("255.255.255.255", self.mesh_port)) except OSError as e: logger.debug(f"Mesh nudge broadcast failed: {e}") @@ -286,9 +292,7 @@ async def _broadcast_loop(self): try: if self._transport: - self._transport.sendto( - heartbeat, ("255.255.255.255", self.mesh_port) - ) + self._transport.sendto(heartbeat, ("255.255.255.255", self.mesh_port)) except OSError as e: logger.debug(f"Mesh broadcast failed: {e}") diff --git a/gently/mesh/mesh_service.py b/gently/mesh/mesh_service.py index edac242c..96814908 100644 --- a/gently/mesh/mesh_service.py +++ b/gently/mesh/mesh_service.py @@ -11,12 +11,13 @@ import asyncio import logging import time +from collections.abc import Callable from pathlib import Path -from typing import Callable, Dict, List, Optional from gently.core.event_bus import EventType from gently.core.service import Service +from ..settings import settings from .discovery import MeshDiscovery from .models import PeerCapability, PeerInfo, PeerStatus from .peer_client import PeerClient @@ -24,8 +25,6 @@ logger = logging.getLogger(__name__) -from ..settings import settings - REAPER_INTERVAL = settings.mesh.reaper_interval_s STATUS_REFRESH_INTERVAL = settings.mesh.status_refresh_s @@ -59,7 +58,7 @@ def __init__( mesh_port: int = settings.network.mesh_port, pairing_manager=None, audit_log=None, - config_dir: Optional[Path] = None, + config_dir: Path | None = None, ): import socket as _socket @@ -78,12 +77,12 @@ def __init__( self._audit_log = audit_log self._hostname = _socket.gethostname() - self._peers: Dict[str, PeerInfo] = {} - self._discovery: Optional[MeshDiscovery] = None - self._peer_client: Optional[PeerClient] = None - self._reaper_task: Optional[asyncio.Task] = None - self._refresh_task: Optional[asyncio.Task] = None - self._cleanup_task: Optional[asyncio.Task] = None + self._peers: dict[str, PeerInfo] = {} + self._discovery: MeshDiscovery | None = None + self._peer_client: PeerClient | None = None + self._reaper_task: asyncio.Task | None = None + self._refresh_task: asyncio.Task | None = None + self._cleanup_task: asyncio.Task | None = None # Persistent verse map if config_dir is None: @@ -121,7 +120,8 @@ async def on_start(self): # When our own status changes, broadcast a nudge to all peers self._status_unsub = self._event_bus.subscribe( - EventType.STATUS_CHANGED, self._on_local_status_changed, + EventType.STATUS_CHANGED, + self._on_local_status_changed, ) async def on_stop(self): @@ -156,7 +156,8 @@ def _on_peer_discovered(self, data: dict, sender_ip: str, verified: bool = False # Check if this peer is already trusted trusted = ( self._pairing_manager.is_trusted(peer_id) - if self._pairing_manager else True # no manager = trust all (backward compat) + if self._pairing_manager + else True # no manager = trust all (backward compat) ) # Determine TLS status — trusted peers with a cert fingerprint use TLS @@ -187,24 +188,28 @@ def _on_peer_discovered(self, data: dict, sender_ip: str, verified: bool = False if was_offline: # Previously offline peer returned self._verse_map.on_peer_returned(peer_id) - self._emit_event(EventType.MESH_PEER_RETURNED, { - "instance_id": peer_id, - "hostname": peer.hostname, - "ip_address": sender_ip, - "is_trusted": trusted, - }) - logger.info( - f"Mesh: peer returned {peer.hostname} ({peer_id[:8]}) at {sender_ip}" + self._emit_event( + EventType.MESH_PEER_RETURNED, + { + "instance_id": peer_id, + "hostname": peer.hostname, + "ip_address": sender_ip, + "is_trusted": trusted, + }, ) + logger.info(f"Mesh: peer returned {peer.hostname} ({peer_id[:8]}) at {sender_ip}") else: - self._emit_event(EventType.MESH_PEER_DISCOVERED, { - "instance_id": peer_id, - "hostname": peer.hostname, - "ip_address": sender_ip, - "is_trusted": trusted, - "udp_verified": verified, - "tls_enabled": tls_enabled, - }) + self._emit_event( + EventType.MESH_PEER_DISCOVERED, + { + "instance_id": peer_id, + "hostname": peer.hostname, + "ip_address": sender_ip, + "is_trusted": trusted, + "udp_verified": verified, + "tls_enabled": tls_enabled, + }, + ) logger.info( f"Mesh: discovered peer {peer.hostname} ({peer_id[:8]}) at {sender_ip} " f"[trusted={trusted}, udp_verified={verified}, tls={tls_enabled}]" @@ -212,7 +217,7 @@ def _on_peer_discovered(self, data: dict, sender_ip: str, verified: bool = False # Only fetch status from trusted peers if trusted: - asyncio.ensure_future(self._fetch_and_update_peer(peer)) + self._schedule_status_fetch(peer) def _on_peer_heartbeat(self, instance_id: str, sender_ip: str, verified: bool = False): """Called on subsequent heartbeats from a known peer.""" @@ -228,7 +233,7 @@ def _on_nudge_received(self, peer_id: str, sender_ip: str): if peer: peer.last_seen = time.time() peer.ip_address = sender_ip - asyncio.ensure_future(self._fetch_and_update_peer(peer)) + self._schedule_status_fetch(peer) logger.debug(f"Mesh: nudge from {peer.hostname} ({peer_id[:8]}), refetching") def _on_local_status_changed(self, event): @@ -257,20 +262,28 @@ async def _reaper_loop(self): self._verse_map.on_peer_offline(pid) if self._discovery: self._discovery.forget_peer(pid) - self._emit_event(EventType.MESH_PEER_OFFLINE, { - "instance_id": pid, - "hostname": peer.hostname, - }) - logger.info(f"Mesh: peer offline {peer.hostname} ({pid[:8]}) — kept in verse map") + self._emit_event( + EventType.MESH_PEER_OFFLINE, + { + "instance_id": pid, + "hostname": peer.hostname, + }, + ) + logger.info( + f"Mesh: peer offline {peer.hostname} ({pid[:8]}) — kept in verse map" + ) else: # Untrusted peer: fully remove self._peers.pop(pid, None) if self._discovery: self._discovery.forget_peer(pid) - self._emit_event(EventType.MESH_PEER_LOST, { - "instance_id": pid, - "hostname": peer.hostname, - }) + self._emit_event( + EventType.MESH_PEER_LOST, + { + "instance_id": pid, + "hostname": peer.hostname, + }, + ) logger.info(f"Mesh: lost peer {peer.hostname} ({pid[:8]})") async def _refresh_loop(self): @@ -306,10 +319,27 @@ async def _fetch_and_update_peer(self, peer: PeerInfo): # Update verse map with latest capabilities self._verse_map.on_peer_updated(peer) - self._emit_event(EventType.MESH_PEER_UPDATED, { - "instance_id": peer.instance_id, - "hostname": peer.hostname, - }) + self._emit_event( + EventType.MESH_PEER_UPDATED, + { + "instance_id": peer.instance_id, + "hostname": peer.hostname, + }, + ) + + def _schedule_status_fetch(self, peer: PeerInfo) -> None: + """Schedule a best-effort peer status fetch when the service is running.""" + if not self._peer_client: + return + try: + loop = asyncio.get_running_loop() + except RuntimeError: + logger.debug( + "Mesh: skipping status fetch for %s because no event loop is running", + peer.instance_id[:8], + ) + return + loop.create_task(self._fetch_and_update_peer(peer)) # ------------------------------------------------------------------ # Pairing integration @@ -336,26 +366,26 @@ def mark_peer_trusted(self, instance_id: str): if cert_fp: peer.tls_enabled = True # Kick off an immediate status fetch now that we trust them - asyncio.ensure_future(self._fetch_and_update_peer(peer)) + self._schedule_status_fetch(peer) logger.info(f"Mesh: peer {peer.hostname} ({instance_id[:8]}) now trusted") # ------------------------------------------------------------------ # Public query API # ------------------------------------------------------------------ - def get_peers(self) -> List[PeerInfo]: + def get_peers(self) -> list[PeerInfo]: """Return all live (non-dead) peers.""" return [p for p in self._peers.values() if not p.is_dead] - def get_all_peers(self) -> List[PeerInfo]: + def get_all_peers(self) -> list[PeerInfo]: """Return all tracked peers including stale/dead ones.""" return list(self._peers.values()) - def get_peer(self, instance_id: str) -> Optional[PeerInfo]: + def get_peer(self, instance_id: str) -> PeerInfo | None: """Get a specific peer by instance_id.""" return self._peers.get(instance_id) - def find_peers_with(self, capability: str) -> List[PeerInfo]: + def find_peers_with(self, capability: str) -> list[PeerInfo]: """ Find live peers that have a given capability flag. @@ -386,11 +416,11 @@ def get_local_info(self) -> dict: } @property - def peer_client(self) -> Optional[PeerClient]: + def peer_client(self) -> PeerClient | None: """Expose the peer client for direct campaign operations.""" return self._peer_client - def find_peer_by_hostname(self, hostname: str) -> Optional[PeerInfo]: + def find_peer_by_hostname(self, hostname: str) -> PeerInfo | None: """Find a live peer by hostname (case-insensitive).""" hostname_lower = hostname.lower() for p in self.get_peers(): diff --git a/gently/mesh/models.py b/gently/mesh/models.py index 8c59877f..721525d0 100644 --- a/gently/mesh/models.py +++ b/gently/mesh/models.py @@ -14,13 +14,14 @@ import time from dataclasses import dataclass, field from enum import Enum -from typing import Any, Dict, List, Optional +from typing import Any from ..settings import settings class PeerRole(str, Enum): """Dynamic roles a gently node can fill.""" + MICROSCOPE_CONTROLLER = "microscope_controller" ML_TRAINER = "ml_trainer" DATA_SERVER = "data_server" @@ -30,6 +31,7 @@ class PeerRole(str, Enum): @dataclass class GpuInfo: """Details about a single GPU device.""" + device_index: int = 0 name: str = "" vram_gb: float = 0.0 @@ -37,7 +39,7 @@ class GpuInfo: utilization_pct: float = 0.0 memory_used_gb: float = 0.0 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "device_index": self.device_index, "name": self.name, @@ -48,7 +50,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "GpuInfo": + def from_dict(cls, d: dict[str, Any]) -> "GpuInfo": return cls( device_index=d.get("device_index", 0), name=d.get("name", ""), @@ -62,16 +64,17 @@ def from_dict(cls, d: Dict[str, Any]) -> "GpuInfo": @dataclass class DatasetAdvertisement: """Advertises what data a node has available for training.""" + session_id: str = "" session_name: str = "" embryo_count: int = 0 volume_count: int = 0 has_ground_truth: bool = False ground_truth_count: int = 0 - stages_covered: List[str] = field(default_factory=list) + stages_covered: list[str] = field(default_factory=list) total_size_gb: float = 0.0 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "session_id": self.session_id, "session_name": self.session_name, @@ -84,7 +87,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "DatasetAdvertisement": + def from_dict(cls, d: dict[str, Any]) -> "DatasetAdvertisement": return cls( session_id=d.get("session_id", ""), session_name=d.get("session_name", ""), @@ -107,19 +110,19 @@ class PeerCapability: gpu_name: str = "" gpu_vram_gb: float = 0.0 storage_free_gb: float = 0.0 - tool_categories: List[str] = field(default_factory=list) + tool_categories: list[str] = field(default_factory=list) organism: str = "" hardware_profile: str = "" # Enhanced capability fields (backward-compatible — old peers get defaults) - gpus: List[GpuInfo] = field(default_factory=list) - roles: List[str] = field(default_factory=list) - datasets: List[DatasetAdvertisement] = field(default_factory=list) + gpus: list[GpuInfo] = field(default_factory=list) + roles: list[str] = field(default_factory=list) + datasets: list[DatasetAdvertisement] = field(default_factory=list) microscope_connected: bool = False cpu_cores: int = 0 ram_gb: float = 0.0 storage_total_gb: float = 0.0 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "has_microscope": self.has_microscope, "has_sam": self.has_sam, @@ -140,7 +143,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "PeerCapability": + def from_dict(cls, d: dict[str, Any]) -> "PeerCapability": return cls( has_microscope=d.get("has_microscope", False), has_sam=d.get("has_sam", False), @@ -174,7 +177,7 @@ class PeerStatus: active_plan: str = "" version: str = "" - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "session_id": self.session_id, "acquisition_status": self.acquisition_status, @@ -187,7 +190,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "PeerStatus": + def from_dict(cls, d: dict[str, Any]) -> "PeerStatus": return cls( session_id=d.get("session_id", ""), acquisition_status=d.get("acquisition_status", "idle"), @@ -232,7 +235,7 @@ def is_dead(self) -> bool: """True if no heartbeat beyond the dead threshold.""" return (time.time() - self.last_seen) > settings.mesh.dead_threshold_s - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "instance_id": self.instance_id, "hostname": self.hostname, @@ -252,7 +255,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "PeerInfo": + def from_dict(cls, d: dict[str, Any]) -> "PeerInfo": return cls( instance_id=d.get("instance_id", ""), hostname=d.get("hostname", ""), @@ -289,15 +292,15 @@ class PersistedPeer: # Persistence fields online: bool = True last_online: float = field(default_factory=time.time) - roles: List[str] = field(default_factory=list) - datasets: List[DatasetAdvertisement] = field(default_factory=list) + roles: list[str] = field(default_factory=list) + datasets: list[DatasetAdvertisement] = field(default_factory=list) @property def base_url(self) -> str: scheme = "https" if self.tls_enabled else "http" return f"{scheme}://{self.ip_address}:{self.viz_port}" - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "instance_id": self.instance_id, "hostname": self.hostname, @@ -316,7 +319,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "PersistedPeer": + def from_dict(cls, d: dict[str, Any]) -> "PersistedPeer": return cls( instance_id=d.get("instance_id", ""), hostname=d.get("hostname", ""), diff --git a/gently/mesh/pairing.py b/gently/mesh/pairing.py index 32e939d5..e8eda9d9 100644 --- a/gently/mesh/pairing.py +++ b/gently/mesh/pairing.py @@ -24,7 +24,6 @@ import uuid from dataclasses import dataclass, field from pathlib import Path -from typing import Dict, List, Optional, Tuple logger = logging.getLogger(__name__) @@ -71,7 +70,7 @@ class TrustedPeer: paired_at: str = "" # ISO timestamp cert_fingerprint: str = "" # SHA256 of peer's TLS cert (DER) udp_signing_key: str = "" # hex-encoded key for UDP HMAC verification - scopes: List[str] = field(default_factory=lambda: list(ALL_SCOPES)) + scopes: list[str] = field(default_factory=lambda: list(ALL_SCOPES)) class PairingManager: @@ -94,8 +93,8 @@ def __init__(self, instance_id: str, hostname: str, config_dir: Path, audit_log= self._config_dir = config_dir self._audit_log = audit_log - self._sessions: Dict[str, PairingSession] = {} - self._trusted: Dict[str, TrustedPeer] = {} # keyed by instance_id + self._sessions: dict[str, PairingSession] = {} + self._trusted: dict[str, TrustedPeer] = {} # keyed by instance_id self._trust_file = config_dir / "mesh_trusted_peers.json" # Phase 2: TLS cert fingerprint (set by launch_gently after cert gen) @@ -109,7 +108,7 @@ def __init__(self, instance_id: str, hostname: str, config_dir: Path, audit_log= ).hexdigest() # Phase 2: rate limiting state - self._pair_attempts: Dict[str, List[float]] = {} # IP -> timestamps + self._pair_attempts: dict[str, list[float]] = {} # IP -> timestamps self._load_trusted() @@ -163,7 +162,7 @@ def is_trusted(self, instance_id: str) -> bool: """Check if a peer is trusted.""" return instance_id in self._trusted - def get_token_for_peer(self, instance_id: str) -> Optional[str]: + def get_token_for_peer(self, instance_id: str) -> str | None: """Get the current daily auth token for a trusted peer.""" tp = self._trusted.get(instance_id) if tp is None: @@ -171,7 +170,7 @@ def get_token_for_peer(self, instance_id: str) -> Optional[str]: epoch_day = self._current_epoch_day() return self._derive_daily_token(tp.base_token, epoch_day) - def verify_token(self, token: str) -> Optional[str]: + def verify_token(self, token: str) -> str | None: """ Check if a token matches any trusted peer (timing-safe). @@ -186,26 +185,26 @@ def verify_token(self, token: str) -> Optional[str]: return tp.instance_id return None - def get_all_trusted(self) -> List[TrustedPeer]: + def get_all_trusted(self) -> list[TrustedPeer]: """Return all trusted peers.""" return list(self._trusted.values()) - def get_udp_key_for_peer(self, instance_id: str) -> Optional[str]: + def get_udp_key_for_peer(self, instance_id: str) -> str | None: """Get the UDP signing key for a trusted peer.""" tp = self._trusted.get(instance_id) return tp.udp_signing_key if tp else None - def get_cert_fingerprint_for_peer(self, instance_id: str) -> Optional[str]: + def get_cert_fingerprint_for_peer(self, instance_id: str) -> str | None: """Get the TLS cert fingerprint for a trusted peer.""" tp = self._trusted.get(instance_id) return tp.cert_fingerprint if tp else None - def get_scopes_for_peer(self, instance_id: str) -> List[str]: + def get_scopes_for_peer(self, instance_id: str) -> list[str]: """Get the permission scopes for a trusted peer.""" tp = self._trusted.get(instance_id) return list(tp.scopes) if tp else [] - def set_scopes(self, identifier: str, scopes: List[str]) -> bool: + def set_scopes(self, identifier: str, scopes: list[str]) -> bool: """ Set permission scopes for a peer (by instance_id, prefix, or hostname). @@ -269,8 +268,10 @@ def unpair(self, identifier: str) -> bool: self._save_trusted() if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.PEER_UNPAIRED, outcome="info", + AuditEvent.PEER_UNPAIRED, + outcome="info", peer_id=removed_id, ) return True @@ -281,7 +282,7 @@ def unpair(self, identifier: str) -> bool: # Rate limiting # ------------------------------------------------------------------ - def check_rate_limit(self, ip: str) -> Tuple[bool, float]: + def check_rate_limit(self, ip: str) -> tuple[bool, float]: """ Check if a pairing attempt from this IP is allowed. @@ -298,9 +299,12 @@ def check_rate_limit(self, ip: str) -> Tuple[bool, float]: retry_after = RATE_LIMIT_WINDOW - (now - attempts[0]) if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.RATE_LIMITED, outcome="deny", - ip=ip, detail=f"max_attempts={RATE_LIMIT_MAX}", + AuditEvent.RATE_LIMITED, + outcome="deny", + ip=ip, + detail=f"max_attempts={RATE_LIMIT_MAX}", ) return False, max(retry_after, 1.0) @@ -312,9 +316,12 @@ def check_rate_limit(self, ip: str) -> Tuple[bool, float]: if elapsed < backoff: if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.RATE_LIMITED, outcome="deny", - ip=ip, detail=f"backoff={backoff:.1f}s", + AuditEvent.RATE_LIMITED, + outcome="deny", + ip=ip, + detail=f"backoff={backoff:.1f}s", ) return False, backoff - elapsed @@ -405,7 +412,7 @@ def handle_pair_request( # Confirmation # ------------------------------------------------------------------ - def confirm_pairing(self, pairing_id: str, confirmer_id: str) -> Optional[PairingSession]: + def confirm_pairing(self, pairing_id: str, confirmer_id: str) -> PairingSession | None: """ Mark one side as confirmed. @@ -428,36 +435,39 @@ def confirm_pairing(self, pairing_id: str, confirmer_id: str) -> Optional[Pairin return session - def reject_pairing(self, pairing_id: str) -> Optional[PairingSession]: + def reject_pairing(self, pairing_id: str) -> PairingSession | None: """Reject a pending pairing session.""" session = self._sessions.get(pairing_id) if session and session.status == "pending": session.status = "rejected" if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.PAIR_REJECTED, outcome="deny", + AuditEvent.PAIR_REJECTED, + outcome="deny", peer_id=session.initiator_id, ) return session - def get_session(self, pairing_id: str) -> Optional[PairingSession]: + def get_session(self, pairing_id: str) -> PairingSession | None: """Get a pairing session by ID.""" return self._sessions.get(pairing_id) - def get_pending_sessions(self) -> List[PairingSession]: + def get_pending_sessions(self) -> list[PairingSession]: """Get all pending pairing sessions (for /pair accept).""" return [ - s for s in self._sessions.values() - if s.status == "pending" - and s.responder_id == self.instance_id + s + for s in self._sessions.values() + if s.status == "pending" and s.responder_id == self.instance_id ] def cleanup_expired(self): """Remove expired pending sessions.""" now = time.time() expired = [ - pid for pid, s in self._sessions.items() + pid + for pid, s in self._sessions.items() if s.status == "pending" and (now - s.created_at) > PAIRING_EXPIRY ] for pid in expired: @@ -497,9 +507,12 @@ def _finalize_pairing(self, session: PairingSession): logger.info(f"Paired with {peer_hostname} ({peer_id[:8]})") if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.PAIR_COMPLETED, outcome="info", - peer_id=peer_id, detail=f"hostname={peer_hostname}", + AuditEvent.PAIR_COMPLETED, + outcome="info", + peer_id=peer_id, + detail=f"hostname={peer_hostname}", ) def _load_trusted(self): diff --git a/gently/mesh/peer_client.py b/gently/mesh/peer_client.py index c0b1be5e..b7fd81a7 100644 --- a/gently/mesh/peer_client.py +++ b/gently/mesh/peer_client.py @@ -10,12 +10,12 @@ import asyncio import logging import ssl -from typing import Any, Dict, List, Optional +from typing import Any import aiohttp -from .models import PeerInfo from ..settings import settings +from .models import PeerInfo logger = logging.getLogger(__name__) @@ -24,12 +24,12 @@ class PeerClient: """Fetches full status from a peer's viz server over HTTP.""" def __init__(self, pairing_manager=None, audit_log=None): - self._session: Optional[aiohttp.ClientSession] = None + self._session: aiohttp.ClientSession | None = None self._pairing_manager = pairing_manager self._audit_log = audit_log self._pinning_verified: set = set() # track first-success per peer - async def _ensure_session(self): + async def _ensure_session(self) -> aiohttp.ClientSession: if self._session is None or self._session.closed: timeout = aiohttp.ClientTimeout(total=settings.mesh.fetch_timeout_s) # Use permissive SSL context — we verify by cert fingerprint, not CA chain @@ -39,8 +39,9 @@ async def _ensure_session(self): connector = aiohttp.TCPConnector(ssl=ssl_ctx) self._session = aiohttp.ClientSession(timeout=timeout, connector=connector) self._pinning_verified.clear() + return self._session - def _auth_headers(self, peer: PeerInfo) -> Dict[str, str]: + def _auth_headers(self, peer: PeerInfo) -> dict[str, str]: """Build auth headers for a trusted peer.""" if self._pairing_manager is None: return {} @@ -59,16 +60,13 @@ def _ssl_for_peer(self, peer: PeerInfo): """ if self._pairing_manager is None: return False - fingerprint = self._pairing_manager.get_cert_fingerprint_for_peer( - peer.instance_id - ) + fingerprint = self._pairing_manager.get_cert_fingerprint_for_peer(peer.instance_id) if fingerprint: try: return aiohttp.Fingerprint(bytes.fromhex(fingerprint)) except (ValueError, TypeError): logger.warning( - f"Invalid cert fingerprint for {peer.instance_id[:8]}, " - "falling back to unpinned" + f"Invalid cert fingerprint for {peer.instance_id[:8]}, falling back to unpinned" ) return False @@ -78,9 +76,12 @@ def _log_pinning_success(self, peer: PeerInfo, ssl_fp): self._pinning_verified.add(peer.instance_id) if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.CERT_PIN_OK, outcome="allow", - peer_id=peer.instance_id, ip=peer.ip_address, + AuditEvent.CERT_PIN_OK, + outcome="allow", + peer_id=peer.instance_id, + ip=peer.ip_address, ) def _log_pinning_failure(self, peer: PeerInfo, error): @@ -88,30 +89,31 @@ def _log_pinning_failure(self, peer: PeerInfo, error): logger.warning(f"CERT PINNING FAILED for {peer.instance_id[:8]}: {error}") if self._audit_log: from .audit import AuditEvent + self._audit_log.log( - AuditEvent.CERT_PIN_FAIL, outcome="deny", - peer_id=peer.instance_id, ip=peer.ip_address, + AuditEvent.CERT_PIN_FAIL, + outcome="deny", + peer_id=peer.instance_id, + ip=peer.ip_address, detail=str(error), ) - async def fetch_peer_info(self, peer: PeerInfo) -> Optional[Dict[str, Any]]: + async def fetch_peer_info(self, peer: PeerInfo) -> dict[str, Any] | None: """ GET /api/mesh/status from a peer. Returns the parsed JSON dict on success, or None on failure. """ - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/mesh/status" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return await resp.json() - logger.debug( - f"Peer {peer.instance_id[:8]} returned HTTP {resp.status}" - ) + logger.debug(f"Peer {peer.instance_id[:8]} returned HTTP {resp.status}") except aiohttp.ServerFingerprintMismatch as e: self._log_pinning_failure(peer, e) except (aiohttp.ClientError, asyncio.TimeoutError, OSError) as e: @@ -123,14 +125,14 @@ async def fetch_peer_info(self, peer: PeerInfo) -> Optional[Dict[str, Any]]: # Campaign coordination methods # ------------------------------------------------------------------ - async def fetch_peer_campaigns(self, peer: PeerInfo) -> Optional[List]: + async def fetch_peer_campaigns(self, peer: PeerInfo) -> list | None: """GET /api/campaigns from a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/campaigns" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) data = await resp.json() @@ -141,14 +143,14 @@ async def fetch_peer_campaigns(self, peer: PeerInfo) -> Optional[List]: logger.debug(f"Failed to fetch campaigns from {peer.instance_id[:8]}: {e}") return None - async def fetch_campaign_export(self, peer: PeerInfo, campaign_id: str) -> Optional[Dict]: + async def fetch_campaign_export(self, peer: PeerInfo, campaign_id: str) -> dict | None: """GET /api/campaigns/{id}/export from a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/campaigns/{campaign_id}/export" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return await resp.json() @@ -159,18 +161,27 @@ async def fetch_campaign_export(self, peer: PeerInfo, campaign_id: str) -> Optio return None async def join_campaign( - self, peer: PeerInfo, campaign_id: str, instance_id: str, hostname: str, + self, + peer: PeerInfo, + campaign_id: str, + instance_id: str, + hostname: str, ) -> bool: """POST /api/campaigns/{id}/join on a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/campaigns/{campaign_id}/join" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.post(url, json={ - "instance_id": instance_id, - "hostname": hostname, - }, headers=headers, ssl=ssl_fp) as resp: + async with session.post( + url, + json={ + "instance_id": instance_id, + "hostname": hostname, + }, + headers=headers, + ssl=ssl_fp, + ) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return resp.status == 200 @@ -189,15 +200,20 @@ async def claim_item( hostname: str, ) -> bool: """POST /api/campaigns/{id}/items/{item_id}/claim on a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/campaigns/{campaign_id}/items/{item_id}/claim" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.post(url, json={ - "instance_id": instance_id, - "hostname": hostname, - }, headers=headers, ssl=ssl_fp) as resp: + async with session.post( + url, + json={ + "instance_id": instance_id, + "hostname": hostname, + }, + headers=headers, + ssl=ssl_fp, + ) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return resp.status == 200 @@ -208,15 +224,18 @@ async def claim_item( return False async def unclaim_item( - self, peer: PeerInfo, campaign_id: str, item_id: str, + self, + peer: PeerInfo, + campaign_id: str, + item_id: str, ) -> bool: """POST /api/campaigns/{id}/items/{item_id}/unclaim on a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/campaigns/{campaign_id}/items/{item_id}/unclaim" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.post(url, headers=headers, ssl=ssl_fp) as resp: + async with session.post(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return resp.status == 200 @@ -232,18 +251,18 @@ async def update_item_status( campaign_id: str, item_id: str, status: str, - outcome: Optional[str] = None, + outcome: str | None = None, ) -> bool: """POST /api/campaigns/{id}/items/{item_id}/status on a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/campaigns/{campaign_id}/items/{item_id}/status" - body: Dict[str, Any] = {"status": status} + body: dict[str, Any] = {"status": status} if outcome is not None: body["outcome"] = outcome headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.post(url, json=body, headers=headers, ssl=ssl_fp) as resp: + async with session.post(url, json=body, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return resp.status == 200 @@ -258,9 +277,14 @@ async def update_item_status( # ------------------------------------------------------------------ async def send_pair_request( - self, peer: PeerInfo, initiator_id: str, hostname: str, nonce: str, - cert_fingerprint: str = "", udp_sign_key: str = "", - ) -> Optional[Dict]: + self, + peer: PeerInfo, + initiator_id: str, + hostname: str, + nonce: str, + cert_fingerprint: str = "", + udp_sign_key: str = "", + ) -> dict | None: """POST /api/mesh/pair — initiate pairing with a peer. Returns response dict on success, or {"_error": "..."} on failure. @@ -278,18 +302,19 @@ async def send_pair_request( base = f"{peer.ip_address}:{peer.viz_port}" return {"_error": f"Could not reach {base} via HTTPS or HTTP"} - async def _pairing_request(self, peer: PeerInfo, method: str, path: str, - json_body: Optional[Dict] = None) -> Optional[Dict]: + async def _pairing_request( + self, peer: PeerInfo, method: str, path: str, json_body: dict | None = None + ) -> dict | None: """Make an HTTP request trying HTTPS first, then HTTP (for pre-pairing).""" - await self._ensure_session() + session = await self._ensure_session() base = f"{peer.ip_address}:{peer.viz_port}" for scheme in ("https", "http"): url = f"{scheme}://{base}{path}" try: if method == "GET": - req = self._session.get(url) + req = session.get(url) else: - req = self._session.post(url, json=json_body or {}) + req = session.post(url, json=json_body or {}) async with req as resp: if resp.status == 200: return await resp.json() @@ -297,18 +322,25 @@ async def _pairing_request(self, peer: PeerInfo, method: str, path: str, continue return None - async def poll_pair_status(self, peer: PeerInfo, pairing_id: str) -> Optional[Dict]: + async def poll_pair_status(self, peer: PeerInfo, pairing_id: str) -> dict | None: """GET /api/mesh/pair/{id}/status — poll pairing status.""" return await self._pairing_request( - peer, "GET", f"/api/mesh/pair/{pairing_id}/status", + peer, + "GET", + f"/api/mesh/pair/{pairing_id}/status", ) async def confirm_pair_remote( - self, peer: PeerInfo, pairing_id: str, confirmer_id: str, + self, + peer: PeerInfo, + pairing_id: str, + confirmer_id: str, ) -> bool: """POST /api/mesh/pair/{id}/confirm — confirm pairing on remote side.""" resp = await self._pairing_request( - peer, "POST", f"/api/mesh/pair/{pairing_id}/confirm", + peer, + "POST", + f"/api/mesh/pair/{pairing_id}/confirm", json_body={"confirmer_id": confirmer_id}, ) return resp is not None @@ -317,14 +349,14 @@ async def confirm_pair_remote( # Data catalog methods (Phase 2 — requires "data" scope) # ------------------------------------------------------------------ - async def fetch_peer_sessions(self, peer: PeerInfo) -> Optional[List]: + async def fetch_peer_sessions(self, peer: PeerInfo) -> list | None: """GET /api/data/sessions from a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/data/sessions" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) data = await resp.json() @@ -335,14 +367,14 @@ async def fetch_peer_sessions(self, peer: PeerInfo) -> Optional[List]: logger.debug(f"Failed to fetch sessions from {peer.instance_id[:8]}: {e}") return None - async def fetch_peer_session_detail(self, peer: PeerInfo, session_id: str) -> Optional[Dict]: + async def fetch_peer_session_detail(self, peer: PeerInfo, session_id: str) -> dict | None: """GET /api/data/sessions/{id} from a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/data/sessions/{session_id}" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return await resp.json() @@ -352,14 +384,14 @@ async def fetch_peer_session_detail(self, peer: PeerInfo, session_id: str) -> Op logger.debug(f"Failed to fetch session detail from {peer.instance_id[:8]}: {e}") return None - async def fetch_peer_coverage(self, peer: PeerInfo) -> Optional[Dict]: + async def fetch_peer_coverage(self, peer: PeerInfo) -> dict | None: """GET /api/data/coverage from a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/data/coverage" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return await resp.json() @@ -369,14 +401,14 @@ async def fetch_peer_coverage(self, peer: PeerInfo) -> Optional[Dict]: logger.debug(f"Failed to fetch coverage from {peer.instance_id[:8]}: {e}") return None - async def fetch_peer_stage_distribution(self, peer: PeerInfo) -> Optional[Dict]: + async def fetch_peer_stage_distribution(self, peer: PeerInfo) -> dict | None: """GET /api/data/stages from a peer.""" - await self._ensure_session() + session = await self._ensure_session() url = f"{peer.base_url}/api/data/stages" headers = self._auth_headers(peer) ssl_fp = self._ssl_for_peer(peer) try: - async with self._session.get(url, headers=headers, ssl=ssl_fp) as resp: + async with session.get(url, headers=headers, ssl=ssl_fp) as resp: if resp.status == 200: self._log_pinning_success(peer, ssl_fp) return await resp.json() diff --git a/gently/mesh/routes.py b/gently/mesh/routes.py index 1308340e..2f0edd38 100644 --- a/gently/mesh/routes.py +++ b/gently/mesh/routes.py @@ -51,15 +51,21 @@ async def require_mesh_auth(request: Request): if required_scope not in scopes: if audit_log: audit_log.log( - AuditEvent.SCOPE_DENIED, outcome="deny", - peer_id=peer_id, ip=host, + AuditEvent.SCOPE_DENIED, + outcome="deny", + peer_id=peer_id, + ip=host, detail=f"scope={required_scope} path={request.url.path}", ) if viz_server.event_bus is not None: viz_server.event_bus.publish( EventType.MESH_SCOPE_DENIED, - {"peer_id": peer_id, "scope": required_scope, - "ip": host, "path": str(request.url.path)}, + { + "peer_id": peer_id, + "scope": required_scope, + "ip": host, + "path": str(request.url.path), + }, source="mesh", ) raise HTTPException( @@ -68,16 +74,20 @@ async def require_mesh_auth(request: Request): ) if audit_log: audit_log.log( - AuditEvent.AUTH_SUCCESS, outcome="allow", - peer_id=peer_id, ip=host, + AuditEvent.AUTH_SUCCESS, + outcome="allow", + peer_id=peer_id, + ip=host, ) return # Auth failed if audit_log: audit_log.log( - AuditEvent.AUTH_FAILURE, outcome="deny", - ip=host, detail=f"path={request.url.path}", + AuditEvent.AUTH_FAILURE, + outcome="deny", + ip=host, + detail=f"path={request.url.path}", ) if viz_server.event_bus is not None: viz_server.event_bus.publish( @@ -106,13 +116,15 @@ async def mesh_status(): shared_list = [] for c in shared: status = cs.get_plan_status(c.id) - shared_list.append({ - "id": c.id, - "shorthand": c.shorthand, - "description": c.description, - "item_count": status["total"], - "completed_count": status["completed"], - }) + shared_list.append( + { + "id": c.id, + "shorthand": c.shorthand, + "description": c.description, + "item_count": status["total"], + "completed_count": status["completed"], + } + ) info["shared_campaigns"] = shared_list except Exception: pass @@ -123,12 +135,17 @@ async def mesh_status(): async def mesh_peers(): """List all discovered peers.""" peers = mesh_service.get_peers() - return JSONResponse({ - "peers": [p.to_dict() for p in peers], - "count": len(peers), - }) + return JSONResponse( + { + "peers": [p.to_dict() for p in peers], + "count": len(peers), + } + ) - @router.get("/api/mesh/peers/{instance_id}", dependencies=[Depends(_make_auth_dep("status"))]) + @router.get( + "/api/mesh/peers/{instance_id}", + dependencies=[Depends(_make_auth_dep("status"))], + ) async def mesh_peer_detail(instance_id: str): """Get specific peer details.""" peer = mesh_service.get_peer(instance_id) @@ -144,11 +161,13 @@ async def mesh_topology(): """Full mesh view: self + all peers.""" local = mesh_service.get_local_info() peers = mesh_service.get_all_peers() - return JSONResponse({ - "self": local, - "peers": [p.to_dict() for p in peers], - "total_nodes": 1 + len(peers), - }) + return JSONResponse( + { + "self": local, + "peers": [p.to_dict() for p in peers], + "total_nodes": 1 + len(peers), + } + ) # ------------------------------------------------------------------ # Pairing endpoints (no auth — these bootstrap trust) @@ -181,15 +200,19 @@ async def pair_request(request: Request): raise HTTPException(status_code=400, detail="initiator_id and nonce required") session = pairing_mgr.handle_pair_request( - initiator_id, hostname, nonce, + initiator_id, + hostname, + nonce, initiator_cert_fingerprint=initiator_cert_fp, initiator_udp_sign_key=initiator_udp_key, ) if audit_log: audit_log.log( - AuditEvent.PAIR_REQUESTED, outcome="info", - peer_id=initiator_id, ip=client_ip, + AuditEvent.PAIR_REQUESTED, + outcome="info", + peer_id=initiator_id, + ip=client_ip, detail=f"hostname={hostname}", ) @@ -205,15 +228,17 @@ async def pair_request(request: Request): source="mesh", ) - return JSONResponse({ - "nonce": session.nonce_responder, - "pairing_id": session.pairing_id, - "status": session.status, - "responder_id": mesh_service.instance_id, - "responder_hostname": mesh_service._hostname, - "cert_fingerprint": pairing_mgr.cert_fingerprint, - "udp_sign_key": pairing_mgr.udp_sign_key, - }) + return JSONResponse( + { + "nonce": session.nonce_responder, + "pairing_id": session.pairing_id, + "status": session.status, + "responder_id": mesh_service.instance_id, + "responder_hostname": mesh_service._hostname, + "cert_fingerprint": pairing_mgr.cert_fingerprint, + "udp_sign_key": pairing_mgr.udp_sign_key, + } + ) @router.get("/api/mesh/pair/{pairing_id}/status") async def pair_status(pairing_id: str): @@ -225,12 +250,14 @@ async def pair_status(pairing_id: str): if session is None: raise HTTPException(status_code=404, detail="Pairing session not found") - return JSONResponse({ - "pairing_id": session.pairing_id, - "status": session.status, - "confirmed_by_initiator": session.confirmed_by_initiator, - "confirmed_by_responder": session.confirmed_by_responder, - }) + return JSONResponse( + { + "pairing_id": session.pairing_id, + "status": session.status, + "confirmed_by_initiator": session.confirmed_by_initiator, + "confirmed_by_responder": session.confirmed_by_responder, + } + ) @router.post("/api/mesh/pair/{pairing_id}/confirm") async def pair_confirm(pairing_id: str, request: Request): @@ -267,10 +294,12 @@ async def pair_confirm(pairing_id: str, request: Request): source="mesh", ) - return JSONResponse({ - "pairing_id": session.pairing_id, - "status": session.status, - }) + return JSONResponse( + { + "pairing_id": session.pairing_id, + "status": session.status, + } + ) @router.post("/api/mesh/pair/{pairing_id}/reject") async def pair_reject(pairing_id: str): @@ -282,10 +311,12 @@ async def pair_reject(pairing_id: str): if session is None: raise HTTPException(status_code=404, detail="Pairing session not found") - return JSONResponse({ - "pairing_id": session.pairing_id, - "status": session.status, - }) + return JSONResponse( + { + "pairing_id": session.pairing_id, + "status": session.status, + } + ) # ------------------------------------------------------------------ # Verse map routes (scope: status) @@ -296,12 +327,14 @@ async def verse_map(): """Full persistent topology — includes offline peers.""" vm = mesh_service.verse_map peers = vm.get_all_peers() - return JSONResponse({ - "peers": [p.to_dict() for p in peers], - "online_count": len(vm.get_online_peers()), - "offline_count": len(vm.get_offline_peers()), - "total_count": len(peers), - }) + return JSONResponse( + { + "peers": [p.to_dict() for p in peers], + "online_count": len(vm.get_online_peers()), + "offline_count": len(vm.get_offline_peers()), + "total_count": len(peers), + } + ) @router.get( "/api/mesh/verse-map/resources/{capability}", @@ -311,11 +344,13 @@ async def verse_map_resources(capability: str): """Find peers matching a capability (route-finding).""" vm = mesh_service.verse_map peers = vm.find_resource(capability) - return JSONResponse({ - "capability": capability, - "peers": [p.to_dict() for p in peers], - "count": len(peers), - }) + return JSONResponse( + { + "capability": capability, + "peers": [p.to_dict() for p in peers], + "count": len(peers), + } + ) # ------------------------------------------------------------------ # Data catalog routes (scope: data) @@ -334,25 +369,32 @@ async def data_sessions(): sid = s.session_id if hasattr(s, "session_id") else s.get("session_id", "") name = s.name if hasattr(s, "name") else s.get("name", "") created = s.created_at if hasattr(s, "created_at") else s.get("created_at", "") - last_active = s.last_active if hasattr(s, "last_active") else s.get("last_active", "") + last_active = ( + s.last_active if hasattr(s, "last_active") else s.get("last_active", "") + ) embryos = store.list_embryos(sid) vol_count = 0 for e in embryos: eid = e.embryo_id if hasattr(e, "embryo_id") else e.get("embryo_id", "") vol_count += len(store.list_volumes(sid, eid)) - result.append({ - "session_id": sid, - "name": name, - "embryo_count": len(embryos), - "volume_count": vol_count, - "created_at": created, - "last_active": last_active, - }) + result.append( + { + "session_id": sid, + "name": name, + "embryo_count": len(embryos), + "volume_count": vol_count, + "created_at": created, + "last_active": last_active, + } + ) return JSONResponse({"sessions": result, "count": len(result)}) except Exception as e: return JSONResponse({"error": str(e)}, status_code=500) - @router.get("/api/data/sessions/{session_id}", dependencies=[Depends(_make_auth_dep("data"))]) + @router.get( + "/api/data/sessions/{session_id}", + dependencies=[Depends(_make_auth_dep("data"))], + ) async def data_session_detail(session_id: str): """Detailed session info with embryo list.""" store = getattr(viz_server, "gently_store", None) @@ -376,26 +418,29 @@ async def data_session_detail(session_id: str): try: gts = store.get_ground_truth(session_id, eid) has_gt = len(gts) > 0 - stages = list({ - (gt.stage if hasattr(gt, "stage") else gt.get("stage", "")) - for gt in gts - }) + stages = list( + {(gt.stage if hasattr(gt, "stage") else gt.get("stage", "")) for gt in gts} + ) except Exception: pass - embryo_list.append({ - "embryo_id": eid, - "nickname": nickname, - "volume_count": vol_count, - "has_ground_truth": has_gt, - "stages_annotated": stages, - }) + embryo_list.append( + { + "embryo_id": eid, + "nickname": nickname, + "volume_count": vol_count, + "has_ground_truth": has_gt, + "stages_annotated": stages, + } + ) sname = session.name if hasattr(session, "name") else session.get("name", "") - return JSONResponse({ - "session_id": session_id, - "name": sname, - "embryos": embryo_list, - "total_volumes": total_vols, - }) + return JSONResponse( + { + "session_id": session_id, + "name": sname, + "embryos": embryo_list, + "total_volumes": total_vols, + } + ) except HTTPException: raise except Exception as e: @@ -406,10 +451,16 @@ async def data_coverage(): """Annotation coverage summary across all sessions.""" store = getattr(viz_server, "gently_store", None) if store is None: - return JSONResponse({ - "total_embryos": 0, "annotated_embryos": 0, - "coverage_pct": 0.0, "stage_counts": {}, "imbalance_ratio": 0.0, "gaps": [], - }) + return JSONResponse( + { + "total_embryos": 0, + "annotated_embryos": 0, + "coverage_pct": 0.0, + "stage_counts": {}, + "imbalance_ratio": 0.0, + "gaps": [], + } + ) try: sessions = store.list_sessions() total_embryos = 0 @@ -437,14 +488,16 @@ async def data_coverage(): # Find stages with notably low counts avg = sum(counts) / len(counts) if counts else 0 gaps = [s for s, c in stage_counts.items() if c < avg * 0.5] - return JSONResponse({ - "total_embryos": total_embryos, - "annotated_embryos": annotated_embryos, - "coverage_pct": round(coverage_pct, 1), - "stage_counts": stage_counts, - "imbalance_ratio": round(imbalance_ratio, 2), - "gaps": gaps, - }) + return JSONResponse( + { + "total_embryos": total_embryos, + "annotated_embryos": annotated_embryos, + "coverage_pct": round(coverage_pct, 1), + "stage_counts": stage_counts, + "imbalance_ratio": round(imbalance_ratio, 2), + "gaps": gaps, + } + ) except Exception as e: return JSONResponse({"error": str(e)}, status_code=500) @@ -475,10 +528,12 @@ async def data_stages(): pass if session_dist: by_session[sid] = session_dist - return JSONResponse({ - "stage_distribution": total_dist, - "by_session": by_session, - }) + return JSONResponse( + { + "stage_distribution": total_dist, + "by_session": by_session, + } + ) except Exception as e: return JSONResponse({"error": str(e)}, status_code=500) diff --git a/gently/mesh/tls.py b/gently/mesh/tls.py index 9d1ec01c..412cc485 100644 --- a/gently/mesh/tls.py +++ b/gently/mesh/tls.py @@ -12,7 +12,6 @@ import logging import ssl from pathlib import Path -from typing import Optional, Tuple logger = logging.getLogger(__name__) @@ -21,7 +20,7 @@ CERT_DAYS = 3650 # ~10 years -def ensure_tls_cert(config_dir: Path) -> Tuple[Optional[Path], Optional[Path]]: +def ensure_tls_cert(config_dir: Path) -> tuple[Path | None, Path | None]: """ Ensure a TLS cert/key pair exists in config_dir. @@ -51,9 +50,11 @@ def ensure_tls_cert(config_dir: Path) -> Tuple[Optional[Path], Optional[Path]]: private_key = ec.generate_private_key(ec.SECP256R1()) now = datetime.datetime.now(datetime.timezone.utc) - subject = issuer = x509.Name([ - x509.NameAttribute(NameOID.COMMON_NAME, "gently-mesh"), - ]) + subject = issuer = x509.Name( + [ + x509.NameAttribute(NameOID.COMMON_NAME, "gently-mesh"), + ] + ) cert = ( x509.CertificateBuilder() @@ -64,20 +65,24 @@ def ensure_tls_cert(config_dir: Path) -> Tuple[Optional[Path], Optional[Path]]: .not_valid_before(now) .not_valid_after(now + datetime.timedelta(days=CERT_DAYS)) .add_extension( - x509.SubjectAlternativeName([ - x509.IPAddress(ipaddress.IPv4Address("0.0.0.0")), - ]), + x509.SubjectAlternativeName( + [ + x509.IPAddress(ipaddress.IPv4Address("0.0.0.0")), + ] + ), critical=False, ) .sign(private_key, hashes.SHA256()) ) # Write PEM files - key_path.write_bytes(private_key.private_bytes( - encoding=serialization.Encoding.PEM, - format=serialization.PrivateFormat.PKCS8, - encryption_algorithm=serialization.NoEncryption(), - )) + key_path.write_bytes( + private_key.private_bytes( + encoding=serialization.Encoding.PEM, + format=serialization.PrivateFormat.PKCS8, + encryption_algorithm=serialization.NoEncryption(), + ) + ) cert_path.write_bytes(cert.public_bytes(serialization.Encoding.PEM)) logger.info(f"Generated TLS cert: {cert_path}") @@ -85,8 +90,7 @@ def ensure_tls_cert(config_dir: Path) -> Tuple[Optional[Path], Optional[Path]]: except ImportError: logger.warning( - "cryptography package not installed — TLS disabled " - "(pip install cryptography)" + "cryptography package not installed — TLS disabled (pip install cryptography)" ) return None, None except Exception as e: @@ -114,7 +118,8 @@ def get_cert_fingerprint(cert_path: Path) -> str: def build_server_ssl_context( - cert_path: Path, key_path: Path, + cert_path: Path, + key_path: Path, ) -> ssl.SSLContext: """Build an SSL context for the uvicorn/FastAPI server.""" ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) diff --git a/gently/mesh/transfer/__init__.py b/gently/mesh/transfer/__init__.py index d7ec6818..cf8df0fc 100644 --- a/gently/mesh/transfer/__init__.py +++ b/gently/mesh/transfer/__init__.py @@ -4,8 +4,14 @@ Resumable, authenticated transfers for datasets and model weights. """ -from .models import TransferFile, TransferJob, TransferManifest, TransferStatus, TransferType from .client import TransferClient +from .models import ( + TransferFile, + TransferJob, + TransferManifest, + TransferStatus, + TransferType, +) from .server import TransferService from .tracker import TransferTracker diff --git a/gently/mesh/transfer/client.py b/gently/mesh/transfer/client.py index bdcd3507..066aea08 100644 --- a/gently/mesh/transfer/client.py +++ b/gently/mesh/transfer/client.py @@ -3,15 +3,12 @@ """ import asyncio -import hashlib import logging import time import uuid from pathlib import Path -from typing import List, Optional from ...core.event_bus import EventType, get_event_bus -from ...settings import settings from .models import TransferJob, TransferStatus, TransferType from .protocol import send_file @@ -42,7 +39,7 @@ async def send_dataset( peer_ip: str, peer_port: int, peer_instance_id: str, - file_paths: List[Path], + file_paths: list[Path], session_id: str = "", ) -> TransferJob: """Send dataset files to a peer. @@ -87,7 +84,10 @@ async def send_dataset( reader, writer = await asyncio.open_connection(peer_ip, peer_port) try: success, sha256 = await send_file( - writer, file_path, job.id, auth_token=token, + writer, + file_path, + job.id, + auth_token=token, ) if success: job.bytes_transferred += file_path.stat().st_size diff --git a/gently/mesh/transfer/models.py b/gently/mesh/transfer/models.py index c1534eae..a932d69e 100644 --- a/gently/mesh/transfer/models.py +++ b/gently/mesh/transfer/models.py @@ -2,14 +2,14 @@ Transfer data models. """ -import time from dataclasses import dataclass, field from enum import Enum -from typing import Any, Dict, List, Optional +from typing import Any class TransferType(str, Enum): """Type of data being transferred.""" + DATASET = "dataset" MODEL_WEIGHTS = "model_weights" SESSION = "session" @@ -17,6 +17,7 @@ class TransferType(str, Enum): class TransferStatus(str, Enum): """Transfer state machine.""" + PENDING = "pending" TRANSFERRING = "transferring" PAUSED = "paused" @@ -28,12 +29,13 @@ class TransferStatus(str, Enum): @dataclass class TransferFile: """A single file in a transfer manifest.""" + relative_path: str = "" total_size: int = 0 sha256: str = "" transferred: int = 0 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "relative_path": self.relative_path, "total_size": self.total_size, @@ -42,7 +44,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "TransferFile": + def from_dict(cls, d: dict[str, Any]) -> "TransferFile": return cls( relative_path=d.get("relative_path", ""), total_size=d.get("total_size", 0), @@ -54,11 +56,12 @@ def from_dict(cls, d: Dict[str, Any]) -> "TransferFile": @dataclass class TransferManifest: """List of files to transfer.""" - files: List[TransferFile] = field(default_factory=list) + + files: list[TransferFile] = field(default_factory=list) total_size: int = 0 file_count: int = 0 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "files": [f.to_dict() for f in self.files], "total_size": self.total_size, @@ -66,7 +69,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "TransferManifest": + def from_dict(cls, d: dict[str, Any]) -> "TransferManifest": files = [TransferFile.from_dict(f) for f in d.get("files", [])] return cls( files=files, @@ -78,6 +81,7 @@ def from_dict(cls, d: Dict[str, Any]) -> "TransferManifest": @dataclass class TransferJob: """State of a single transfer (send or receive).""" + id: str = "" transfer_type: str = TransferType.DATASET.value status: str = TransferStatus.PENDING.value @@ -103,7 +107,7 @@ def progress_pct(self) -> float: return 0.0 return (self.bytes_transferred / self.total_bytes) * 100 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "id": self.id, "transfer_type": self.transfer_type, @@ -125,7 +129,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "TransferJob": + def from_dict(cls, d: dict[str, Any]) -> "TransferJob": return cls( id=d.get("id", ""), transfer_type=d.get("transfer_type", TransferType.DATASET.value), diff --git a/gently/mesh/transfer/protocol.py b/gently/mesh/transfer/protocol.py index 89af0258..df72c837 100644 --- a/gently/mesh/transfer/protocol.py +++ b/gently/mesh/transfer/protocol.py @@ -22,7 +22,6 @@ import logging import struct from pathlib import Path -from typing import Optional, Tuple from ...settings import settings @@ -39,7 +38,7 @@ async def send_file( auth_token: str = "", offset: int = 0, chunk_size: int = 0, -) -> Tuple[bool, str]: +) -> tuple[bool, str]: """Send a file over a TCP connection. Parameters @@ -108,7 +107,7 @@ async def send_file( async def receive_file( reader: asyncio.StreamReader, dest_dir: Path, -) -> Tuple[Optional[dict], Optional[Path], str]: +) -> tuple[dict | None, Path | None, str]: """Receive a file over a TCP connection. Parameters diff --git a/gently/mesh/transfer/server.py b/gently/mesh/transfer/server.py index ff2cfb2b..9ecf0fb9 100644 --- a/gently/mesh/transfer/server.py +++ b/gently/mesh/transfer/server.py @@ -4,15 +4,12 @@ import asyncio import logging -import time import uuid from pathlib import Path -from typing import Optional from ...core.event_bus import EventType, get_event_bus from ...core.service import Service from ...settings import settings -from .models import TransferJob, TransferStatus from .protocol import receive_file logger = logging.getLogger(__name__) @@ -46,7 +43,7 @@ def __init__( self._dest_dir = dest_dir self._pairing_manager = pairing_manager self._port = port - self._server: Optional[asyncio.AbstractServer] = None + self._server: asyncio.AbstractServer | None = None self._active_transfers: dict = {} async def on_start(self): diff --git a/gently/mesh/transfer/tracker.py b/gently/mesh/transfer/tracker.py index 43f52cb2..be896b5f 100644 --- a/gently/mesh/transfer/tracker.py +++ b/gently/mesh/transfer/tracker.py @@ -5,7 +5,6 @@ import json import logging from pathlib import Path -from typing import Dict, List, Optional from .models import TransferJob, TransferStatus @@ -24,7 +23,7 @@ class TransferTracker: def __init__(self, config_dir: Path): self._config_dir = config_dir self._state_file = config_dir / "mesh_transfers.json" - self._jobs: Dict[str, TransferJob] = {} + self._jobs: dict[str, TransferJob] = {} self._load() def _load(self): @@ -65,20 +64,21 @@ def update_job(self, job_id: str, **kwargs): setattr(job, k, v) self._save() - def get_job(self, job_id: str) -> Optional[TransferJob]: + def get_job(self, job_id: str) -> TransferJob | None: """Get a transfer job by ID.""" return self._jobs.get(job_id) - def list_jobs(self, status: Optional[str] = None) -> List[TransferJob]: + def list_jobs(self, status: str | None = None) -> list[TransferJob]: """List all jobs, optionally filtered by status.""" if status: return [j for j in self._jobs.values() if j.status == status] return list(self._jobs.values()) - def get_resumable(self) -> List[TransferJob]: + def get_resumable(self) -> list[TransferJob]: """Get transfers that were interrupted and can be resumed.""" return [ - j for j in self._jobs.values() + j + for j in self._jobs.values() if j.status == TransferStatus.TRANSFERRING.value and j.bytes_transferred > 0 and j.bytes_transferred < j.total_bytes @@ -87,9 +87,11 @@ def get_resumable(self) -> List[TransferJob]: def cleanup_completed(self, max_age_hours: float = 24.0): """Remove old completed/failed transfers.""" import time + cutoff = time.time() - (max_age_hours * 3600) to_remove = [ - jid for jid, j in self._jobs.items() + jid + for jid, j in self._jobs.items() if j.status in (TransferStatus.COMPLETED.value, TransferStatus.FAILED.value) and j.completed_at > 0 and j.completed_at < cutoff diff --git a/gently/mesh/verse_map.py b/gently/mesh/verse_map.py index 7309f753..7fd36934 100644 --- a/gently/mesh/verse_map.py +++ b/gently/mesh/verse_map.py @@ -9,13 +9,9 @@ import logging import time from pathlib import Path -from typing import Dict, List, Optional from .models import ( - DatasetAdvertisement, - PeerCapability, PeerInfo, - PeerStatus, PersistedPeer, ) @@ -28,7 +24,7 @@ class VerseMap: def __init__(self, config_dir: Path): self._config_dir = config_dir self._map_file = config_dir / "mesh_verse_map.json" - self._peers: Dict[str, PersistedPeer] = {} + self._peers: dict[str, PersistedPeer] = {} self._load() # ------------------------------------------------------------------ @@ -109,19 +105,19 @@ def on_peer_returned(self, instance_id: str): # Queries # ------------------------------------------------------------------ - def get_all_peers(self) -> List[PersistedPeer]: + def get_all_peers(self) -> list[PersistedPeer]: """All peers, online and offline.""" return list(self._peers.values()) - def get_online_peers(self) -> List[PersistedPeer]: + def get_online_peers(self) -> list[PersistedPeer]: """Only online peers.""" return [p for p in self._peers.values() if p.online] - def get_offline_peers(self) -> List[PersistedPeer]: + def get_offline_peers(self) -> list[PersistedPeer]: """Only offline peers.""" return [p for p in self._peers.values() if not p.online] - def get_peer(self, instance_id: str) -> Optional[PersistedPeer]: + def get_peer(self, instance_id: str) -> PersistedPeer | None: """Get a specific peer by instance_id.""" return self._peers.get(instance_id) @@ -138,27 +134,25 @@ def was_online(self, instance_id: str) -> bool: # Route-finding: sorted online-first, then by last_seen recency # ------------------------------------------------------------------ - def _sorted_peers(self, peers: List[PersistedPeer]) -> List[PersistedPeer]: + def _sorted_peers(self, peers: list[PersistedPeer]) -> list[PersistedPeer]: """Sort peers: online first, then by last_seen descending.""" return sorted(peers, key=lambda p: (not p.online, -p.last_seen)) - def find_gpu_peers(self) -> List[PersistedPeer]: + def find_gpu_peers(self) -> list[PersistedPeer]: """Find peers with GPU capability, best candidates first.""" - results = [ - p for p in self._peers.values() - if p.capabilities.has_gpu or p.capabilities.gpus - ] + results = [p for p in self._peers.values() if p.capabilities.has_gpu or p.capabilities.gpus] return self._sorted_peers(results) - def find_microscope_peers(self) -> List[PersistedPeer]: + def find_microscope_peers(self) -> list[PersistedPeer]: """Find peers with microscope capability.""" results = [ - p for p in self._peers.values() + p + for p in self._peers.values() if p.capabilities.has_microscope or p.capabilities.microscope_connected ] return self._sorted_peers(results) - def find_data_peers(self, session_id: str = None) -> List[PersistedPeer]: + def find_data_peers(self, session_id: str | None = None) -> list[PersistedPeer]: """Find peers with data, optionally filtering by session.""" results = [] for p in self._peers.values(): @@ -171,7 +165,7 @@ def find_data_peers(self, session_id: str = None) -> List[PersistedPeer]: results.append(p) return self._sorted_peers(results) - def find_resource(self, capability: str) -> List[PersistedPeer]: + def find_resource(self, capability: str) -> list[PersistedPeer]: """Find peers matching a generic capability attribute. The capability string is checked against: diff --git a/gently/ml/__init__.py b/gently/ml/__init__.py index 9c817a33..7b4c2dc1 100644 --- a/gently/ml/__init__.py +++ b/gently/ml/__init__.py @@ -9,6 +9,7 @@ - Federated averaging for distributed training """ +from .architectures import ARCHITECTURE_REGISTRY, get_suitable_architectures from .models import ( DataSplit, MLPipeline, @@ -18,7 +19,6 @@ TrainingRun, TrainingStatus, ) -from .architectures import ARCHITECTURE_REGISTRY, get_suitable_architectures __all__ = [ "ARCHITECTURE_REGISTRY", diff --git a/gently/ml/_train_worker.py b/gently/ml/_train_worker.py index bc905a87..04448de2 100644 --- a/gently/ml/_train_worker.py +++ b/gently/ml/_train_worker.py @@ -43,7 +43,7 @@ def main(): sys.exit(1) try: - import torchvision.models as models + import torchvision.models as models # noqa: F401 except ImportError: _write_progress(progress_file, {"error": "torchvision not installed"}) sys.exit(1) @@ -56,7 +56,7 @@ def main(): sys.exit(1) # Build datasets - from gently.ml.data_loader import GentlyDataset, create_data_splits + from gently.ml.data_loader import create_data_splits architecture = model_config.get("architecture", "resnet18") num_classes = model_config.get("num_classes", 8) @@ -75,8 +75,11 @@ def main(): # Create datasets from labels train_data, val_data, test_data = create_data_splits( - labels_data, data_root, input_size, - train_ratio=0.7, val_ratio=0.15, + labels_data, + data_root, + input_size, + train_ratio=0.7, + val_ratio=0.15, ) train_loader = DataLoader(train_data, batch_size=batch_size, shuffle=True, num_workers=2) @@ -171,17 +174,20 @@ def main(): patience_counter += 1 # Write progress - _write_progress(progress_file, { - "epoch": epoch + 1, - "total_epochs": epochs, - "train_loss": round(train_loss, 4), - "train_accuracy": round(train_acc, 4), - "val_loss": round(val_loss, 4), - "val_accuracy": round(val_acc, 4), - "best_val_accuracy": round(best_val_acc, 4), - "lr": optimizer.param_groups[0]["lr"], - "timestamp": time.time(), - }) + _write_progress( + progress_file, + { + "epoch": epoch + 1, + "total_epochs": epochs, + "train_loss": round(train_loss, 4), + "train_accuracy": round(train_acc, 4), + "val_loss": round(val_loss, 4), + "val_accuracy": round(val_acc, 4), + "best_val_accuracy": round(best_val_acc, 4), + "lr": optimizer.param_groups[0]["lr"], + "timestamp": time.time(), + }, + ) # Early stopping if patience_counter >= early_stopping_patience: @@ -218,7 +224,12 @@ def _build_model(architecture, num_classes, pretrained, input_channels, dropout) if input_channels != 3: old_conv = model.conv1 model.conv1 = nn.Conv2d( - input_channels, 64, kernel_size=7, stride=2, padding=3, bias=False, + input_channels, + 64, + kernel_size=7, + stride=2, + padding=3, + bias=False, ) if pretrained and input_channels == 1: # Average RGB weights for grayscale @@ -241,7 +252,8 @@ def _build_model(architecture, num_classes, pretrained, input_channels, dropout) old_conv = model.features[0][0] out_channels = old_conv.out_channels model.features[0][0] = nn.Conv2d( - input_channels, out_channels, + input_channels, + out_channels, kernel_size=old_conv.kernel_size, stride=old_conv.stride, padding=old_conv.padding, @@ -260,7 +272,8 @@ def _build_model(architecture, num_classes, pretrained, input_channels, dropout) old_conv = model.features[0][0] out_channels = old_conv.out_channels model.features[0][0] = nn.Conv2d( - input_channels, out_channels, + input_channels, + out_channels, kernel_size=old_conv.kernel_size, stride=old_conv.stride, padding=old_conv.padding, @@ -278,7 +291,8 @@ def _build_model(architecture, num_classes, pretrained, input_channels, dropout) old_conv = model.features[0][0] out_channels = old_conv.out_channels model.features[0][0] = nn.Conv2d( - input_channels, out_channels, + input_channels, + out_channels, kernel_size=old_conv.kernel_size, stride=old_conv.stride, padding=old_conv.padding, diff --git a/gently/ml/architectures.py b/gently/ml/architectures.py index 41f4fe77..2c6f91a8 100644 --- a/gently/ml/architectures.py +++ b/gently/ml/architectures.py @@ -5,12 +5,12 @@ for a given task, dataset size, and hardware constraints. """ -from typing import Any, Dict, List +from typing import Any from .models import ModelArchitectureType # Architecture registry: metadata per architecture -ARCHITECTURE_REGISTRY: Dict[str, Dict[str, Any]] = { +ARCHITECTURE_REGISTRY: dict[str, dict[str, Any]] = { ModelArchitectureType.RESNET_18.value: { "name": "ResNet-18", "family": "resnet", @@ -147,7 +147,7 @@ def get_suitable_architectures( dataset_size: int, vram_gb: float, image_type: str = "microscopy", -) -> List[Dict[str, Any]]: +) -> list[dict[str, Any]]: """Filter architectures suitable for given constraints. Parameters diff --git a/gently/ml/data_loader.py b/gently/ml/data_loader.py index 082fae1e..f34f4f06 100644 --- a/gently/ml/data_loader.py +++ b/gently/ml/data_loader.py @@ -2,11 +2,10 @@ GentlyDataset — PyTorch Dataset loading projections + ground_truth from FileStore. """ -import json import logging import random from pathlib import Path -from typing import Any, Dict, List, Optional, Tuple +from typing import Any logger = logging.getLogger(__name__) @@ -14,11 +13,13 @@ import numpy as np import torch from torch.utils.data import Dataset + HAS_TORCH = True except ImportError: HAS_TORCH = False + # Stub for import-time safety - class Dataset: + class Dataset: # type: ignore[no-redef] pass @@ -37,7 +38,7 @@ class GentlyDataset(Dataset): def __init__( self, - samples: List[Tuple[str, int]], + samples: list[tuple[str, int]], input_size: int = 224, augment: bool = False, ): @@ -57,6 +58,7 @@ def __getitem__(self, idx): # Load image try: from PIL import Image + img = Image.open(img_path).convert("L") # grayscale img = img.resize((self.input_size, self.input_size)) img_np = np.array(img, dtype=np.float32) / 255.0 @@ -92,13 +94,13 @@ def _apply_augmentations(self, img: np.ndarray) -> np.ndarray: def create_data_splits( - labels_data: Dict[str, Any], + labels_data: dict[str, Any], data_root: Path, input_size: int = 224, train_ratio: float = 0.7, val_ratio: float = 0.15, random_seed: int = 42, -) -> Tuple: +) -> tuple: """Create train/val/test datasets from a labels file. Parameters @@ -133,7 +135,7 @@ def create_data_splits( all_samples.append((full_path, label)) # Stratified split - by_label = {} + by_label: dict[Any, list] = {} for path, label in all_samples: by_label.setdefault(label, []).append((path, label)) @@ -141,15 +143,15 @@ def create_data_splits( val_samples = [] test_samples = [] - for label, items in by_label.items(): + for _label, items in by_label.items(): random.shuffle(items) n = len(items) n_train = max(1, int(n * train_ratio)) n_val = max(1, int(n * val_ratio)) train_samples.extend(items[:n_train]) - val_samples.extend(items[n_train:n_train + n_val]) - test_samples.extend(items[n_train + n_val:]) + val_samples.extend(items[n_train : n_train + n_val]) + test_samples.extend(items[n_train + n_val :]) train_ds = GentlyDataset(train_samples, input_size=input_size, augment=True) val_ds = GentlyDataset(val_samples, input_size=input_size, augment=False) @@ -158,7 +160,7 @@ def create_data_splits( return train_ds, val_ds, test_ds -def build_labels_from_store(gently_store, session_ids: Optional[List[str]] = None) -> Dict: +def build_labels_from_store(gently_store, session_ids: list[str] | None = None) -> dict: """Build a labels dict from FileStore ground truth. Returns @@ -166,7 +168,7 @@ def build_labels_from_store(gently_store, session_ids: Optional[List[str]] = Non dict {"class_names": [...], "samples": [{"path": "...", "label": int}, ...]} """ - stage_to_idx = {} + stage_to_idx: dict[Any, int] = {} samples = [] sessions = gently_store.list_sessions() @@ -193,13 +195,15 @@ def build_labels_from_store(gently_store, session_ids: Optional[List[str]] = Non try: proj_path = gently_store.get_projection_path(sid, eid, start_tp) if proj_path: - samples.append({ - "path": str(proj_path), - "label": stage_to_idx[stage], - "session_id": sid, - "embryo_id": eid, - "stage": stage, - }) + samples.append( + { + "path": str(proj_path), + "label": stage_to_idx[stage], + "session_id": sid, + "embryo_id": eid, + "stage": stage, + } + ) except Exception: pass except Exception: diff --git a/gently/ml/evaluation.py b/gently/ml/evaluation.py index b03f35b5..deffce4d 100644 --- a/gently/ml/evaluation.py +++ b/gently/ml/evaluation.py @@ -6,7 +6,7 @@ import logging from dataclasses import dataclass, field from pathlib import Path -from typing import Any, Dict, List, Optional +from typing import Any logger = logging.getLogger(__name__) @@ -14,17 +14,18 @@ @dataclass class EvaluationReport: """Complete evaluation report for a trained model.""" + run_id: str = "" accuracy: float = 0.0 - per_stage_precision: Dict[str, float] = field(default_factory=dict) - per_stage_recall: Dict[str, float] = field(default_factory=dict) - per_stage_f1: Dict[str, float] = field(default_factory=dict) - confusion_matrix: List[List[int]] = field(default_factory=list) - class_names: List[str] = field(default_factory=list) + per_stage_precision: dict[str, float] = field(default_factory=dict) + per_stage_recall: dict[str, float] = field(default_factory=dict) + per_stage_f1: dict[str, float] = field(default_factory=dict) + confusion_matrix: list[list[int]] = field(default_factory=list) + class_names: list[str] = field(default_factory=list) total_samples: int = 0 correct: int = 0 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "run_id": self.run_id, "accuracy": self.accuracy, @@ -38,7 +39,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "EvaluationReport": + def from_dict(cls, d: dict[str, Any]) -> "EvaluationReport": return cls( run_id=d.get("run_id", ""), accuracy=d.get("accuracy", 0.0), @@ -66,7 +67,7 @@ def summary(self) -> str: def evaluate_model( model, data_loader, - class_names: List[str], + class_names: list[str], device=None, run_id: str = "", ) -> EvaluationReport: @@ -111,7 +112,7 @@ def evaluate_model( outputs = model(batch_x) _, predicted = outputs.max(1) - for true, pred in zip(batch_y.cpu().tolist(), predicted.cpu().tolist()): + for true, pred in zip(batch_y.cpu().tolist(), predicted.cpu().tolist(), strict=False): cm[true][pred] += 1 total += 1 if true == pred: diff --git a/gently/ml/federated.py b/gently/ml/federated.py index c114b18e..8e6104be 100644 --- a/gently/ml/federated.py +++ b/gently/ml/federated.py @@ -12,11 +12,9 @@ import asyncio import copy -import json import logging -import time from pathlib import Path -from typing import Any, Dict, List, Optional, Tuple +from typing import Any from ..core.event_bus import EventType, get_event_bus @@ -24,9 +22,9 @@ def federated_average( - state_dicts: List[Dict[str, Any]], - weights: List[float], -) -> Dict[str, Any]: + state_dicts: list[dict[str, Any]], + weights: list[float], +) -> dict[str, Any]: """Compute weighted average of model state dicts. Parameters @@ -50,7 +48,7 @@ def federated_average( try: import torch except ImportError: - raise ImportError("PyTorch required for federated averaging") + raise ImportError("PyTorch required for federated averaging") from None total_weight = sum(weights) if total_weight == 0: @@ -66,7 +64,7 @@ def federated_average( averaged[key] = torch.zeros_like(state_dicts[0][key], dtype=torch.float32) # Weighted sum - for sd, w in zip(state_dicts, norm_weights): + for sd, w in zip(state_dicts, norm_weights, strict=False): for key in averaged: averaged[key] += sd[key].float() * w @@ -98,14 +96,14 @@ def __init__(self, verse_map, transfer_client=None, peer_client=None): async def run_federated_training( self, pipeline_id: str, - worker_peers: List, + worker_peers: list, initial_weights_path: Path, local_epochs_per_round: int = 5, max_rounds: int = 20, convergence_threshold: float = 0.001, - training_config: Optional[Dict] = None, - model_config: Optional[Dict] = None, - ) -> Dict[str, Any]: + training_config: dict | None = None, + model_config: dict | None = None, + ) -> dict[str, Any]: """Run federated averaging across mesh peers. Parameters @@ -175,10 +173,12 @@ async def run_federated_training( # 3. Federated average state_dicts = [r["state_dict"] for r in worker_results if r.get("state_dict")] - dataset_sizes = [r.get("dataset_size", 1) for r in worker_results if r.get("state_dict")] + dataset_sizes = [ + r.get("dataset_size", 1) for r in worker_results if r.get("state_dict") + ] if state_dicts: - global_state = federated_average(state_dicts, dataset_sizes) + federated_average(state_dicts, dataset_sizes) else: logger.warning(f"Round {round_num}: no state dicts to average") continue @@ -234,13 +234,13 @@ async def run_federated_training( async def _train_workers( self, - workers: List, + workers: list, pipeline_id: str, round_num: int, local_epochs: int, - training_config: Optional[Dict], - model_config: Optional[Dict], - ) -> List[Dict]: + training_config: dict | None, + model_config: dict | None, + ) -> list[dict]: """Send training jobs to all workers and collect results. In production this uses PeerClient to POST /api/ml/train on each @@ -253,18 +253,20 @@ async def _train_workers( for worker in workers: tasks.append( self._train_single_worker( - worker, pipeline_id, round_num, local_epochs, - training_config, model_config, + worker, + pipeline_id, + round_num, + local_epochs, + training_config, + model_config, ) ) completed = await asyncio.gather(*tasks, return_exceptions=True) - for worker, result in zip(workers, completed): - if isinstance(result, Exception): - logger.warning( - f"Worker {worker.hostname} failed in round {round_num}: {result}" - ) + for worker, result in zip(workers, completed, strict=False): + if isinstance(result, BaseException): + logger.warning(f"Worker {worker.hostname} failed in round {round_num}: {result}") continue if result: results.append(result) @@ -277,9 +279,9 @@ async def _train_single_worker( pipeline_id: str, round_num: int, local_epochs: int, - training_config: Optional[Dict], - model_config: Optional[Dict], - ) -> Optional[Dict]: + training_config: dict | None, + model_config: dict | None, + ) -> dict | None: """Train on a single worker peer via HTTP API. Returns worker result dict with state_dict, val_accuracy, dataset_size. @@ -289,7 +291,8 @@ async def _train_single_worker( # Build a PeerInfo for the HTTP client from ..models import PeerInfo - peer = PeerInfo( + + PeerInfo( instance_id=worker.instance_id, hostname=worker.hostname, ip_address=worker.ip_address, diff --git a/gently/ml/models.py b/gently/ml/models.py index 0a5b5c00..dcf689ab 100644 --- a/gently/ml/models.py +++ b/gently/ml/models.py @@ -4,11 +4,12 @@ from dataclasses import dataclass, field from enum import Enum -from typing import Any, Dict, List, Optional +from typing import Any class TrainingStatus(str, Enum): """Status of an ML pipeline or training run.""" + PLANNED = "planned" DATA_PREP = "data_prep" TRAINING = "training" @@ -20,6 +21,7 @@ class TrainingStatus(str, Enum): class ModelArchitectureType(str, Enum): """Supported model architecture families.""" + RESNET_18 = "resnet18" RESNET_50 = "resnet50" EFFICIENTNET_B0 = "efficientnet_b0" @@ -33,6 +35,7 @@ class ModelArchitectureType(str, Enum): @dataclass class ModelConfig: """Configuration for a model architecture.""" + architecture: str = "resnet18" num_classes: int = 8 pretrained: bool = True @@ -41,7 +44,7 @@ class ModelConfig: dropout: float = 0.2 freeze_backbone_epochs: int = 5 # freeze backbone for N epochs, then unfreeze - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "architecture": self.architecture, "num_classes": self.num_classes, @@ -53,7 +56,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "ModelConfig": + def from_dict(cls, d: dict[str, Any]) -> "ModelConfig": return cls( architecture=d.get("architecture", "resnet18"), num_classes=d.get("num_classes", 8), @@ -68,6 +71,7 @@ def from_dict(cls, d: Dict[str, Any]) -> "ModelConfig": @dataclass class TrainingConfig: """Training hyperparameters.""" + batch_size: int = 32 epochs: int = 50 learning_rate: float = 1e-4 @@ -76,13 +80,15 @@ class TrainingConfig: warmup_epochs: int = 5 mixed_precision: bool = True # AMP on A5000 early_stopping_patience: int = 10 - augmentations: List[str] = field(default_factory=lambda: [ - "random_horizontal_flip", - "random_rotation", - "random_brightness", - ]) + augmentations: list[str] = field( + default_factory=lambda: [ + "random_horizontal_flip", + "random_rotation", + "random_brightness", + ] + ) - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "batch_size": self.batch_size, "epochs": self.epochs, @@ -96,7 +102,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "TrainingConfig": + def from_dict(cls, d: dict[str, Any]) -> "TrainingConfig": return cls( batch_size=d.get("batch_size", 32), epochs=d.get("epochs", 50), @@ -113,14 +119,15 @@ def from_dict(cls, d: Dict[str, Any]) -> "TrainingConfig": @dataclass class DataSplit: """Defines how data is split for training.""" + train_ratio: float = 0.7 val_ratio: float = 0.15 test_ratio: float = 0.15 stratify_by: str = "stage" # stratify splits by stage label - session_ids: List[str] = field(default_factory=list) + session_ids: list[str] = field(default_factory=list) random_seed: int = 42 - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "train_ratio": self.train_ratio, "val_ratio": self.val_ratio, @@ -131,7 +138,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "DataSplit": + def from_dict(cls, d: dict[str, Any]) -> "DataSplit": return cls( train_ratio=d.get("train_ratio", 0.7), val_ratio=d.get("val_ratio", 0.15), @@ -145,12 +152,13 @@ def from_dict(cls, d: Dict[str, Any]) -> "DataSplit": @dataclass class TrainingRun: """State of a single training run.""" + id: str = "" pipeline_id: str = "" status: str = TrainingStatus.PLANNED.value - model_config: Optional[ModelConfig] = None - training_config: Optional[TrainingConfig] = None - data_split: Optional[DataSplit] = None + model_config: ModelConfig | None = None + training_config: TrainingConfig | None = None + data_split: DataSplit | None = None current_epoch: int = 0 total_epochs: int = 0 train_loss: float = 0.0 @@ -164,7 +172,7 @@ class TrainingRun: completed_at: str = "" error_message: str = "" - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "id": self.id, "pipeline_id": self.pipeline_id, @@ -187,7 +195,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "TrainingRun": + def from_dict(cls, d: dict[str, Any]) -> "TrainingRun": mc = d.get("model_config") tc = d.get("training_config") ds = d.get("data_split") @@ -216,20 +224,21 @@ def from_dict(cls, d: Dict[str, Any]) -> "TrainingRun": @dataclass class MLPipeline: """Top-level pipeline that coordinates one ML task.""" + id: str = "" campaign_id: str = "" name: str = "" task: str = "embryo_stage_classification" status: str = TrainingStatus.PLANNED.value - model_config: Optional[ModelConfig] = None - data_split: Optional[DataSplit] = None - training_config: Optional[TrainingConfig] = None + model_config: ModelConfig | None = None + data_split: DataSplit | None = None + training_config: TrainingConfig | None = None best_run_id: str = "" best_accuracy: float = 0.0 created_at: str = "" updated_at: str = "" - def to_dict(self) -> Dict[str, Any]: + def to_dict(self) -> dict[str, Any]: return { "id": self.id, "campaign_id": self.campaign_id, @@ -246,7 +255,7 @@ def to_dict(self) -> Dict[str, Any]: } @classmethod - def from_dict(cls, d: Dict[str, Any]) -> "MLPipeline": + def from_dict(cls, d: dict[str, Any]) -> "MLPipeline": mc = d.get("model_config") ds = d.get("data_split") tc = d.get("training_config") diff --git a/gently/ml/trainer.py b/gently/ml/trainer.py index 67da88cb..2ef764f5 100644 --- a/gently/ml/trainer.py +++ b/gently/ml/trainer.py @@ -9,14 +9,12 @@ import asyncio import json import logging -import os import sys from datetime import datetime from pathlib import Path -from typing import Optional from ..core.event_bus import EventType, get_event_bus -from .models import ModelConfig, TrainingConfig, TrainingRun, TrainingStatus +from .models import TrainingRun, TrainingStatus logger = logging.getLogger(__name__) @@ -33,8 +31,8 @@ class LocalTrainer: def __init__(self, run_dir: Path): self._run_dir = run_dir self._run_dir.mkdir(parents=True, exist_ok=True) - self._process: Optional[asyncio.subprocess.Process] = None - self._monitor_task: Optional[asyncio.Task] = None + self._process: asyncio.subprocess.Process | None = None + self._monitor_task: asyncio.Task | None = None @property def progress_file(self) -> Path: @@ -95,16 +93,16 @@ async def start_training( # Launch subprocess train_script = Path(__file__).parent / "_train_worker.py" self._process = await asyncio.create_subprocess_exec( - sys.executable, str(train_script), str(config_file), + sys.executable, + str(train_script), + str(config_file), stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, cwd=str(self._run_dir), ) # Start progress monitor - self._monitor_task = asyncio.create_task( - self._monitor_progress(run.id, run.pipeline_id) - ) + self._monitor_task = asyncio.create_task(self._monitor_progress(run.id, run.pipeline_id)) logger.info(f"Training started: run={run.id}, pid={self._process.pid}") return run @@ -175,7 +173,7 @@ async def cancel(self): if self._monitor_task and not self._monitor_task.done(): self._monitor_task.cancel() - def get_latest_progress(self) -> Optional[dict]: + def get_latest_progress(self) -> dict | None: """Read the last line from progress.jsonl.""" if not self.progress_file.exists(): return None diff --git a/gently/organisms/__init__.py b/gently/organisms/__init__.py index d388ab60..1db2c317 100644 --- a/gently/organisms/__init__.py +++ b/gently/organisms/__init__.py @@ -14,12 +14,23 @@ import importlib import logging +import pkgutil from types import ModuleType -from typing import Optional logger = logging.getLogger(__name__) -_active_organism: Optional[ModuleType] = None +_active_organism: ModuleType | None = None + + +def available_organisms() -> list[str]: + """Names of the organism plugins shipped under gently.organisms.""" + import gently.organisms as _pkg + + return sorted( + m.name + for m in pkgutil.iter_modules(_pkg.__path__) + if m.ispkg and not m.name.startswith("_") + ) def load_organism(name: str) -> ModuleType: @@ -43,7 +54,19 @@ def load_organism(name: str) -> ModuleType: If the organism module cannot be found. """ global _active_organism - module = importlib.import_module(f"gently.organisms.{name}") + try: + module = importlib.import_module(f"gently.organisms.{name}") + except ModuleNotFoundError as e: + # Only treat a missing organism *package* as a config error; if a + # dependency *inside* the organism module is missing, re-raise so the + # real ImportError isn't masked. + if e.name in (f"gently.organisms.{name}", name): + avail = ", ".join(available_organisms()) or "(none found)" + raise ValueError( + f"Unknown organism '{name}'. Available: {avail}. " + f"Set 'organism:' in config/config.yml." + ) from e + raise _active_organism = module logger.info("Loaded organism module: %s (%s)", name, module.ORGANISM_DISPLAY_NAME) return module @@ -60,7 +83,6 @@ def get_organism() -> ModuleType: """ if _active_organism is None: raise RuntimeError( - "No organism loaded. Call load_organism() at startup, " - "or set 'organism' in config.yml." + "No organism loaded. Call load_organism() at startup, or set 'organism' in config.yml." ) return _active_organism diff --git a/gently/organisms/celegans/__init__.py b/gently/organisms/celegans/__init__.py index b8d353a6..afd4db01 100644 --- a/gently/organisms/celegans/__init__.py +++ b/gently/organisms/celegans/__init__.py @@ -10,21 +10,45 @@ from pathlib import Path +from .biology import BIOLOGY_KNOWLEDGE +from .detection_defaults import DETECTION_DEFAULTS +from .detector_presets import get_detector_presets +from .perception_prompt import PERCEPTION_SYSTEM_PROMPT from .stages import ( - DevelopmentalStage, - STAGES, STAGE_CRITERIA, + STAGES, TRANSITION_ZONES, - get_transition_zone, - get_adjacent_stages, - get_stage_description, + DevelopmentalStage, format_stage_criteria_for_prompt, + get_adjacent_stages, get_all_criteria_for_prompt, + get_stage_description, + get_transition_zone, ) -from .biology import BIOLOGY_KNOWLEDGE -from .detector_presets import get_detector_presets -from .detection_defaults import DETECTION_DEFAULTS -from .perception_prompt import PERCEPTION_SYSTEM_PROMPT + +__all__ = [ + "BIOLOGY_KNOWLEDGE", + "DETECTION_DEFAULTS", + "get_detector_presets", + "PERCEPTION_SYSTEM_PROMPT", + "STAGE_CRITERIA", + "STAGES", + "TRANSITION_ZONES", + "DevelopmentalStage", + "format_stage_criteria_for_prompt", + "get_adjacent_stages", + "get_all_criteria_for_prompt", + "get_stage_description", + "get_transition_zone", + "ORGANISM_NAME", + "ORGANISM_DISPLAY_NAME", + "SAMPLE_TERM", + "SAMPLE_TERM_PLURAL", + "TERMINAL_STAGES", + "STOP_CONDITIONS", + "PRE_TERMINAL_SPEEDUP_STAGE", + "EXAMPLES_PATH", +] # --- Organism identity --- ORGANISM_NAME = "celegans" diff --git a/gently/organisms/celegans/biology.py b/gently/organisms/celegans/biology.py index be2f44f1..6c47ae70 100644 --- a/gently/organisms/celegans/biology.py +++ b/gently/organisms/celegans/biology.py @@ -8,7 +8,8 @@ BIOLOGY_KNOWLEDGE = """ # C. elegans Embryonic Development -C. elegans embryogenesis is highly stereotyped and invariant, proceeding through well-defined stages: +C. elegans embryogenesis is highly stereotyped and invariant, proceeding through +well-defined stages: ## Key Developmental Stages diff --git a/gently/organisms/celegans/detector_presets.py b/gently/organisms/celegans/detector_presets.py index b2a0be4d..30d8d191 100644 --- a/gently/organisms/celegans/detector_presets.py +++ b/gently/organisms/celegans/detector_presets.py @@ -5,10 +5,8 @@ (hatching, comma stage, pretzel, gastrulation, first division). """ -from typing import Dict - -def get_detector_presets() -> Dict: +def get_detector_presets() -> dict: """ Get predefined detector presets for common C. elegans stages. @@ -18,10 +16,11 @@ def get_detector_presets() -> Dict: Preset detector configurations keyed by event name. """ return { - 'hatching': { - 'name': 'hatching', - 'description': 'Detects when C. elegans embryo hatches from eggshell', - 'prompt': """Analyze this C. elegans embryo image (diSPIM light sheet max projection) and determine if the embryo has HATCHED. + "hatching": { + "name": "hatching", + "description": "Detects when C. elegans embryo hatches from eggshell", + "prompt": """Analyze this C. elegans embryo image (diSPIM light sheet max +projection) and determine if the embryo has HATCHED. TRUE HATCHING looks like (must meet at least one): - Most or all of the worm body is OUTSIDE the eggshell boundary @@ -46,16 +45,16 @@ def get_detector_presets() -> Dict: DETECTED: [YES/NO] CONFIDENCE: [HIGH/MEDIUM/LOW] REASONING: [Brief explanation - specifically state if worm is INSIDE or OUTSIDE the shell]""", - 'use_temporal_context': True, - 'temporal_context_size': 10, - 'confidence_threshold': 'HIGH', - 'stop_timelapse': True, # Auto-stop when hatching detected + "use_temporal_context": True, + "temporal_context_size": 10, + "confidence_threshold": "HIGH", + "stop_timelapse": True, # Auto-stop when hatching detected }, - - 'comma': { - 'name': 'comma', - 'description': 'Detects comma stage (major morphogenesis)', - 'prompt': """Analyze this C. elegans embryo and determine if it has reached the COMMA STAGE. + "comma": { + "name": "comma", + "description": "Detects comma stage (major morphogenesis)", + "prompt": """Analyze this C. elegans embryo and determine if it has reached the +COMMA STAGE. Key characteristics of comma stage (~400 minutes, ~6.5 hours): - Distinct comma or bean shape (ventral curvature) @@ -70,15 +69,15 @@ def get_detector_presets() -> Dict: DETECTED: [YES/NO] CONFIDENCE: [HIGH/MEDIUM/LOW] REASONING: [Brief explanation]""", - 'use_temporal_context': True, - 'temporal_context_size': 5, - 'confidence_threshold': 'MEDIUM' + "use_temporal_context": True, + "temporal_context_size": 5, + "confidence_threshold": "MEDIUM", }, - - 'pretzel': { - 'name': 'pretzel', - 'description': 'Detects pretzel/3-fold stage (highly elongated)', - 'prompt': """Analyze this C. elegans embryo and determine if it has reached the PRETZEL/3-FOLD STAGE. + "pretzel": { + "name": "pretzel", + "description": "Detects pretzel/3-fold stage (highly elongated)", + "prompt": """Analyze this C. elegans embryo and determine if it has reached the +PRETZEL/3-FOLD STAGE. Key characteristics of 3-fold stage (~550 minutes, ~9 hours): - Highly elongated, approximately 3x the eggshell length @@ -93,15 +92,14 @@ def get_detector_presets() -> Dict: DETECTED: [YES/NO] CONFIDENCE: [HIGH/MEDIUM/LOW] REASONING: [Brief explanation]""", - 'use_temporal_context': True, - 'temporal_context_size': 5, - 'confidence_threshold': 'MEDIUM' + "use_temporal_context": True, + "temporal_context_size": 5, + "confidence_threshold": "MEDIUM", }, - - 'gastrulation': { - 'name': 'gastrulation', - 'description': 'Detects onset of gastrulation', - 'prompt': """Analyze this C. elegans embryo and determine if GASTRULATION has begun. + "gastrulation": { + "name": "gastrulation", + "description": "Detects onset of gastrulation", + "prompt": """Analyze this C. elegans embryo and determine if GASTRULATION has begun. Key characteristics of gastrulation (~210 minutes, ~3.5 hours): - Visible internalization of cells (especially E cells - gut precursors) @@ -115,15 +113,15 @@ def get_detector_presets() -> Dict: DETECTED: [YES/NO] CONFIDENCE: [HIGH/MEDIUM/LOW] REASONING: [Brief explanation]""", - 'use_temporal_context': True, - 'temporal_context_size': 5, - 'confidence_threshold': 'MEDIUM' + "use_temporal_context": True, + "temporal_context_size": 5, + "confidence_threshold": "MEDIUM", }, - - 'first_division': { - 'name': 'first_division', - 'description': 'Detects first cell division (1-cell to 2-cell)', - 'prompt': """Analyze this C. elegans embryo and determine if FIRST CELL DIVISION has occurred. + "first_division": { + "name": "first_division", + "description": "Detects first cell division (1-cell to 2-cell)", + "prompt": """Analyze this C. elegans embryo and determine if FIRST CELL DIVISION +has occurred. Key characteristics: - Transition from single large cell to two cells @@ -137,8 +135,8 @@ def get_detector_presets() -> Dict: DETECTED: [YES/NO] CONFIDENCE: [HIGH/MEDIUM/LOW] REASONING: [Brief explanation]""", - 'use_temporal_context': True, - 'temporal_context_size': 3, - 'confidence_threshold': 'HIGH' + "use_temporal_context": True, + "temporal_context_size": 3, + "confidence_threshold": "HIGH", }, } diff --git a/gently/organisms/celegans/developmental_tracker.py b/gently/organisms/celegans/developmental_tracker.py index bd10a01e..1897a2c8 100644 --- a/gently/organisms/celegans/developmental_tracker.py +++ b/gently/organisms/celegans/developmental_tracker.py @@ -11,10 +11,11 @@ import logging from dataclasses import dataclass, field from datetime import datetime, timedelta -from typing import Any, Dict, List, Optional, Tuple from enum import Enum +from typing import Any, cast import anthropic +from anthropic.types import MessageParam, TextBlock from ...settings import settings @@ -29,6 +30,7 @@ class DevelopmentalStage(str, Enum): distinguish. The perception enum (in stages.py) maps "early" to everything before comma. """ + ONE_CELL = "1-cell" TWO_CELL = "2-cell" FOUR_CELL = "4-cell" @@ -97,12 +99,13 @@ class DevelopmentalStage(str, Enum): @dataclass class HatchingPrediction: """Prediction of time to hatching with confidence interval""" + embryo_id: str current_stage: DevelopmentalStage predicted_minutes: int min_minutes: int # Lower bound (optimistic) max_minutes: int # Upper bound (conservative) - confidence: str # Based on stage classification confidence + confidence: str # Based on stage classification confidence timestamp: datetime = field(default_factory=datetime.now) @property @@ -110,20 +113,20 @@ def predicted_hours(self) -> float: return self.predicted_minutes / 60 @property - def range_hours(self) -> Tuple[float, float]: + def range_hours(self) -> tuple[float, float]: return (self.min_minutes / 60, self.max_minutes / 60) - def to_dict(self) -> Dict: + def to_dict(self) -> dict: return { - 'embryo_id': self.embryo_id, - 'current_stage': self.current_stage.value, - 'predicted_minutes': self.predicted_minutes, - 'predicted_hours': self.predicted_hours, - 'min_minutes': self.min_minutes, - 'max_minutes': self.max_minutes, - 'range_hours': self.range_hours, - 'confidence': self.confidence, - 'timestamp': self.timestamp.isoformat(), + "embryo_id": self.embryo_id, + "current_stage": self.current_stage.value, + "predicted_minutes": self.predicted_minutes, + "predicted_hours": self.predicted_hours, + "min_minutes": self.min_minutes, + "max_minutes": self.max_minutes, + "range_hours": self.range_hours, + "confidence": self.confidence, + "timestamp": self.timestamp.isoformat(), } def __str__(self) -> str: @@ -137,25 +140,27 @@ def __str__(self) -> str: @dataclass class StageClassification: """Result of a stage classification""" + stage: DevelopmentalStage confidence: str # HIGH, MEDIUM, LOW reasoning: str timestamp: datetime = field(default_factory=datetime.now) timepoint: int = 0 - predicted_minutes_to_hatching: Optional[int] = None + predicted_minutes_to_hatching: int | None = None - def to_dict(self) -> Dict: + def to_dict(self) -> dict: return { - 'stage': self.stage.value, - 'confidence': self.confidence, - 'reasoning': self.reasoning, - 'timestamp': self.timestamp.isoformat(), - 'timepoint': self.timepoint, - 'predicted_minutes_to_hatching': self.predicted_minutes_to_hatching, + "stage": self.stage.value, + "confidence": self.confidence, + "reasoning": self.reasoning, + "timestamp": self.timestamp.isoformat(), + "timepoint": self.timepoint, + "predicted_minutes_to_hatching": self.predicted_minutes_to_hatching, } -STAGE_CLASSIFICATION_PROMPT = """Analyze this C. elegans embryo image and determine its DEVELOPMENTAL STAGE. +STAGE_CLASSIFICATION_PROMPT = """Analyze this C. elegans embryo image and determine its +DEVELOPMENTAL STAGE. Stages in order (earliest to latest): - 1-cell: Single cell, spherical, no division @@ -194,7 +199,7 @@ class DevelopmentalTracker: def __init__( self, - claude_client: Optional[anthropic.Anthropic] = None, + claude_client: anthropic.Anthropic | None = None, model: str = settings.models.perception, ): """ @@ -209,14 +214,14 @@ def __init__( self.model = model # Stage history per embryo - self._stage_history: Dict[str, List[StageClassification]] = {} + self._stage_history: dict[str, list[StageClassification]] = {} def classify_stage( self, image_b64: str, embryo_id: str, timepoint: int = 0, - recent_images: Optional[List[Dict]] = None, + recent_images: list[dict] | None = None, ) -> StageClassification: """ Classify the developmental stage of an embryo @@ -238,56 +243,59 @@ def classify_stage( Classification result """ # Build content for Claude Vision - content = [] + content: list[dict[str, Any]] = [] # Add temporal context if available if recent_images and len(recent_images) > 1: - content.append({ - "type": "text", - "text": f"Recent images from {embryo_id} (for temporal context):" - }) - for img in recent_images[:-1]: # All but last - content.append({ + content.append( + { "type": "text", - "text": f"Timepoint {img.get('timepoint', '?')}" - }) - content.append({ - "type": "image", - "source": { - "type": "base64", - "media_type": "image/jpeg", - "data": img['b64_image'] + "text": f"Recent images from {embryo_id} (for temporal context):", + } + ) + for img in recent_images[:-1]: # All but last + content.append({"type": "text", "text": f"Timepoint {img.get('timepoint', '?')}"}) + content.append( + { + "type": "image", + "source": { + "type": "base64", + "media_type": "image/jpeg", + "data": img["b64_image"], + }, } - }) + ) # Add current image - content.append({ - "type": "text", - "text": f"CURRENT image (timepoint {timepoint}) - classify this one:" - }) - content.append({ - "type": "image", - "source": { - "type": "base64", - "media_type": "image/jpeg", - "data": image_b64 + content.append( + { + "type": "text", + "text": f"CURRENT image (timepoint {timepoint}) - classify this one:", } - }) + ) + content.append( + { + "type": "image", + "source": { + "type": "base64", + "media_type": "image/jpeg", + "data": image_b64, + }, + } + ) # Add prompt - content.append({ - "type": "text", - "text": STAGE_CLASSIFICATION_PROMPT - }) + content.append({"type": "text", "text": STAGE_CLASSIFICATION_PROMPT}) try: + messages: list[MessageParam] = [{"role": "user", "content": cast(Any, content)}] response = self.claude.messages.create( model=self.model, max_tokens=500, - messages=[{"role": "user", "content": content}] + messages=messages, ) - result = self._parse_classification(response.content[0].text) + result = self._parse_classification(cast(TextBlock, response.content[0]).text) result.timepoint = timepoint # Calculate time to hatching @@ -321,31 +329,31 @@ def _parse_classification(self, response_text: str) -> StageClassification: confidence = "LOW" reasoning = "" - lines = response_text.strip().split('\n') + lines = response_text.strip().split("\n") for line in lines: line = line.strip() - if line.startswith('STAGE:'): - stage_str = line.split(':', 1)[1].strip().lower() + if line.startswith("STAGE:"): + stage_str = line.split(":", 1)[1].strip().lower() # Map to enum stage = self._parse_stage_name(stage_str) - elif line.startswith('CONFIDENCE:'): - confidence = line.split(':', 1)[1].strip().upper() - elif line.startswith('REASONING:'): - reasoning = line.split(':', 1)[1].strip() + elif line.startswith("CONFIDENCE:"): + confidence = line.split(":", 1)[1].strip().upper() + elif line.startswith("REASONING:"): + reasoning = line.split(":", 1)[1].strip() # Capture multi-line reasoning if not reasoning: in_reasoning = False reasoning_lines = [] for line in lines: - if line.startswith('REASONING:'): + if line.startswith("REASONING:"): in_reasoning = True - reasoning_lines.append(line.split(':', 1)[1].strip()) + reasoning_lines.append(line.split(":", 1)[1].strip()) elif in_reasoning and line: reasoning_lines.append(line) if reasoning_lines: - reasoning = ' '.join(reasoning_lines) + reasoning = " ".join(reasoning_lines) return StageClassification( stage=stage, @@ -359,43 +367,43 @@ def _parse_stage_name(self, name: str) -> DevelopmentalStage: # Direct matches mappings = { - '1-cell': DevelopmentalStage.ONE_CELL, - 'one-cell': DevelopmentalStage.ONE_CELL, - '2-cell': DevelopmentalStage.TWO_CELL, - 'two-cell': DevelopmentalStage.TWO_CELL, - '4-cell': DevelopmentalStage.FOUR_CELL, - 'four-cell': DevelopmentalStage.FOUR_CELL, - '8-cell': DevelopmentalStage.EIGHT_CELL, - 'eight-cell': DevelopmentalStage.EIGHT_CELL, - 'gastrulation': DevelopmentalStage.GASTRULATION, - 'comma': DevelopmentalStage.COMMA, - '1.5-fold': DevelopmentalStage.ONE_POINT_FIVE_FOLD, - '1.5 fold': DevelopmentalStage.ONE_POINT_FIVE_FOLD, - '2-fold': DevelopmentalStage.TWO_FOLD, - '2 fold': DevelopmentalStage.TWO_FOLD, - 'pretzel': DevelopmentalStage.PRETZEL, - '3-fold': DevelopmentalStage.PRETZEL, - '3 fold': DevelopmentalStage.PRETZEL, - 'pre-hatching': DevelopmentalStage.PRE_HATCHING, - 'prehatching': DevelopmentalStage.PRE_HATCHING, - 'hatching': DevelopmentalStage.HATCHING, - 'hatched': DevelopmentalStage.HATCHED, - 'dead': DevelopmentalStage.DEAD, - 'unknown': DevelopmentalStage.UNKNOWN, + "1-cell": DevelopmentalStage.ONE_CELL, + "one-cell": DevelopmentalStage.ONE_CELL, + "2-cell": DevelopmentalStage.TWO_CELL, + "two-cell": DevelopmentalStage.TWO_CELL, + "4-cell": DevelopmentalStage.FOUR_CELL, + "four-cell": DevelopmentalStage.FOUR_CELL, + "8-cell": DevelopmentalStage.EIGHT_CELL, + "eight-cell": DevelopmentalStage.EIGHT_CELL, + "gastrulation": DevelopmentalStage.GASTRULATION, + "comma": DevelopmentalStage.COMMA, + "1.5-fold": DevelopmentalStage.ONE_POINT_FIVE_FOLD, + "1.5 fold": DevelopmentalStage.ONE_POINT_FIVE_FOLD, + "2-fold": DevelopmentalStage.TWO_FOLD, + "2 fold": DevelopmentalStage.TWO_FOLD, + "pretzel": DevelopmentalStage.PRETZEL, + "3-fold": DevelopmentalStage.PRETZEL, + "3 fold": DevelopmentalStage.PRETZEL, + "pre-hatching": DevelopmentalStage.PRE_HATCHING, + "prehatching": DevelopmentalStage.PRE_HATCHING, + "hatching": DevelopmentalStage.HATCHING, + "hatched": DevelopmentalStage.HATCHED, + "dead": DevelopmentalStage.DEAD, + "unknown": DevelopmentalStage.UNKNOWN, } return mappings.get(name, DevelopmentalStage.UNKNOWN) - def get_stage_history(self, embryo_id: str) -> List[StageClassification]: + def get_stage_history(self, embryo_id: str) -> list[StageClassification]: """Get stage classification history for an embryo""" return self._stage_history.get(embryo_id, []) - def get_current_stage(self, embryo_id: str) -> Optional[StageClassification]: + def get_current_stage(self, embryo_id: str) -> StageClassification | None: """Get the most recent stage classification""" history = self._stage_history.get(embryo_id, []) return history[-1] if history else None - def predict_time_to_hatching(self, embryo_id: str) -> Optional[timedelta]: + def predict_time_to_hatching(self, embryo_id: str) -> timedelta | None: """ Predict time to hatching based on current stage @@ -422,7 +430,7 @@ def predict_time_to_stage( self, embryo_id: str, target_stage: DevelopmentalStage, - ) -> Optional[timedelta]: + ) -> timedelta | None: """ Predict time until embryo reaches target stage @@ -457,7 +465,7 @@ def predict_time_to_stage( minutes = target_timing - current_timing return timedelta(minutes=minutes) - def get_progression_summary(self, embryo_id: str) -> Dict[str, Any]: + def get_progression_summary(self, embryo_id: str) -> dict[str, Any]: """ Get a summary of stage progression for an embryo @@ -475,28 +483,28 @@ def get_progression_summary(self, embryo_id: str) -> Dict[str, Any]: if not history: return { - 'embryo_id': embryo_id, - 'observations': 0, - 'current_stage': None, - 'stages_observed': [], - 'predicted_hatching': None, + "embryo_id": embryo_id, + "observations": 0, + "current_stage": None, + "stages_observed": [], + "predicted_hatching": None, } current = history[-1] stages_observed = list(set(h.stage.value for h in history)) return { - 'embryo_id': embryo_id, - 'observations': len(history), - 'current_stage': current.stage.value, - 'current_confidence': current.confidence, - 'stages_observed': stages_observed, - 'first_observation': history[0].timestamp.isoformat(), - 'last_observation': current.timestamp.isoformat(), - 'predicted_minutes_to_hatching': current.predicted_minutes_to_hatching, + "embryo_id": embryo_id, + "observations": len(history), + "current_stage": current.stage.value, + "current_confidence": current.confidence, + "stages_observed": stages_observed, + "first_observation": history[0].timestamp.isoformat(), + "last_observation": current.timestamp.isoformat(), + "predicted_minutes_to_hatching": current.predicted_minutes_to_hatching, } - def get_hatching_prediction(self, embryo_id: str) -> Optional[HatchingPrediction]: + def get_hatching_prediction(self, embryo_id: str) -> HatchingPrediction | None: """ Get detailed hatching prediction with confidence interval @@ -525,9 +533,9 @@ def get_hatching_prediction(self, embryo_id: str) -> Optional[HatchingPrediction # Adjust confidence interval based on classification confidence confidence_multiplier = { - 'HIGH': 1.0, - 'MEDIUM': 1.5, - 'LOW': 2.0, + "HIGH": 1.0, + "MEDIUM": 1.5, + "LOW": 2.0, }.get(current.confidence, 2.0) adjusted_variability = int(variability * confidence_multiplier) @@ -541,7 +549,7 @@ def get_hatching_prediction(self, embryo_id: str) -> Optional[HatchingPrediction confidence=current.confidence, ) - def get_all_predictions(self, embryo_ids: List[str]) -> Dict[str, HatchingPrediction]: + def get_all_predictions(self, embryo_ids: list[str]) -> dict[str, HatchingPrediction]: """ Get predictions for multiple embryos @@ -562,7 +570,7 @@ def get_all_predictions(self, embryo_ids: List[str]) -> Dict[str, HatchingPredic predictions[embryo_id] = pred return predictions - def estimate_development_rate(self, embryo_id: str) -> Optional[float]: + def estimate_development_rate(self, embryo_id: str) -> float | None: """ Estimate relative development rate compared to standard @@ -584,7 +592,9 @@ def estimate_development_rate(self, embryo_id: str) -> Optional[float]: return None # Need at least two different stages - stages_seen = [(h.stage, h.timestamp) for h in history if h.stage != DevelopmentalStage.UNKNOWN] + stages_seen = [ + (h.stage, h.timestamp) for h in history if h.stage != DevelopmentalStage.UNKNOWN + ] if len(stages_seen) < 2: return None diff --git a/gently/organisms/celegans/perception_prompt.py b/gently/organisms/celegans/perception_prompt.py index 2604a6c7..895fa3e6 100644 --- a/gently/organisms/celegans/perception_prompt.py +++ b/gently/organisms/celegans/perception_prompt.py @@ -5,7 +5,8 @@ Extracted from gently/agent/perception/engine.py. """ -PERCEPTION_SYSTEM_PROMPT = """You are an expert microscopy perception system analyzing C. elegans embryo development. +PERCEPTION_SYSTEM_PROMPT = """You are an expert microscopy perception system analyzing +C. elegans embryo development. IMPORTANT PRINCIPLES: 1. DESCRIBE FIRST: Always describe what you actually see BEFORE classifying @@ -23,27 +24,35 @@ - XY (top-left): Looking DOWN - Best for end asymmetry, ventral indentation, folding - YZ (top-right): Looking from SIDE - Best for body height/thickness -- XZ (bottom): Looking from FRONT - CRITICAL for early->bean transition (look for "peanut" or central constriction) +- XZ (bottom): Looking from FRONT - CRITICAL for early->bean transition (look for "peanut" + or central constriction) -**ALWAYS ANALYZE XZ VIEW**: The XZ view often shows bean-stage features (central constriction, "peanut" shape) BEFORE they're visible in XY. If XZ shows ANY central narrowing or figure-8 appearance, this suggests bean stage even if XY looks symmetric. +**ALWAYS ANALYZE XZ VIEW**: The XZ view often shows bean-stage features (central +constriction, "peanut" shape) BEFORE they're visible in XY. If XZ shows ANY central +narrowing or figure-8 appearance, this suggests bean stage even if XY looks symmetric. DEVELOPMENTAL STAGES: EARLY: Elongated oval (~2:1), SYMMETRIC ENDS, both edges CONVEX, NO central constriction in XZ -BEAN: Even SUBTLE end asymmetry OR central constriction/"peanut" shape in XZ view, edges still CONVEX -COMMA: One edge FLAT or curves INWARD (ventral indentation). XZ shows side-by-side lobes (horizontal figure-8) -1.5-FOLD: Body folding back. XZ shows STACKED horizontal layers (two parallel bands, one above the other) +BEAN: Even SUBTLE end asymmetry OR central constriction/"peanut" shape in XZ view, edges + still CONVEX +COMMA: One edge FLAT or curves INWARD (ventral indentation). XZ shows side-by-side lobes + (horizontal figure-8) +1.5-FOLD: Body folding back. XZ shows STACKED horizontal layers (two parallel bands, one + above the other) 2-FOLD: Body doubled back completely. XZ shows TWO DISTINCT HORIZONTAL LINES with dark gap between PRETZEL: Tightly coiled, 3+ body segments visible as multiple stacked layers HATCHED: Worm exited shell CRITICAL FOR EARLY vs BEAN vs COMMA: - EARLY: Both ends symmetric AND both edges convex AND no central constriction in XZ -- BEAN: ANY of these: subtle end tapering, central constriction in XZ, "peanut" shape - edges still convex +- BEAN: ANY of these: subtle end tapering, central constriction in XZ, "peanut" shape - + edges still convex - COMMA: One edge is flat or curves INWARD (not convex) CRITICAL FOR BEAN/COMMA vs FOLD STAGES (examine XZ view carefully): -The XZ view shows two masses in BOTH bean/comma AND fold stages - the key is their VERTICAL ARRANGEMENT: +The XZ view shows two masses in BOTH bean/comma AND fold stages - the key is their VERTICAL +ARRANGEMENT: BEAN/COMMA XZ: Two lobes at the SAME VERTICAL LEVEL - Lobes are side-by-side horizontally, spanning the same vertical range @@ -67,11 +76,14 @@ - Figure-8 or peanut appearance in any view Mark as TRANSITIONAL (early->bean) or BEAN with appropriate confidence. -SPECIAL: If the field of view is EMPTY (no embryo, no eggshell, only background/debris), return "no_object". +SPECIAL: If the field of view is EMPTY (no embryo, no eggshell, only background/debris), +return "no_object". Respond with JSON: { - "observed_features": {"shape": "...", "curvature": "...", "shell_status": "...", "emergence": "..."}, + "observed_features": { + "shape": "...", "curvature": "...", "shell_status": "...", "emergence": "..." + }, "contrastive_reasoning": {"why_not_previous_stage": "...", "why_not_next_stage": "..."}, "stage": "early|bean|comma|1.5fold|2fold|pretzel|hatching|hatched|arrested|no_object", "is_transitional": true/false, diff --git a/gently/organisms/celegans/stages.py b/gently/organisms/celegans/stages.py index 4ad16970..b75f8d34 100644 --- a/gently/organisms/celegans/stages.py +++ b/gently/organisms/celegans/stages.py @@ -5,7 +5,7 @@ """ from enum import Enum -from typing import List, Dict, Any +from typing import Any class DevelopmentalStage(str, Enum): @@ -21,27 +21,34 @@ class DevelopmentalStage(str, Enum): Special states: - "arrested" is not part of normal progression (dead/arrested embryo) """ - EARLY = "early" # Gastrulation through early morphogenesis, oval shape - BEAN = "bean" # Elongated oval, "bean-shaped", pre-comma curvature - COMMA = "comma" # Clear C-shape, head/tail distinguishable - FOLD_1_5 = "1.5fold" # Elongation, ~1.5x eggshell length - FOLD_2 = "2fold" # Body folded back twice, between 1.5fold and pretzel - PRETZEL = "pretzel" # Tight coil, 3+ body segments (formerly 3fold) - HATCHING = "hatching" # Active emergence, shell breach visible - HATCHED = "hatched" # Fully emerged L1 larva - ARRESTED = "arrested" # Dead or developmentally arrested embryo (special state) - NO_OBJECT = "no_object" # No embryo visible in field of view (special state) + + EARLY = "early" # Gastrulation through early morphogenesis, oval shape + BEAN = "bean" # Elongated oval, "bean-shaped", pre-comma curvature + COMMA = "comma" # Clear C-shape, head/tail distinguishable + FOLD_1_5 = "1.5fold" # Elongation, ~1.5x eggshell length + FOLD_2 = "2fold" # Body folded back twice, between 1.5fold and pretzel + PRETZEL = "pretzel" # Tight coil, 3+ body segments (formerly 3fold) + HATCHING = "hatching" # Active emergence, shell breach visible + HATCHED = "hatched" # Fully emerged L1 larva + ARRESTED = "arrested" # Dead or developmentally arrested embryo (special state) + NO_OBJECT = "no_object" # No embryo visible in field of view (special state) @classmethod - def ordered_list(cls) -> List["DevelopmentalStage"]: + def ordered_list(cls) -> list["DevelopmentalStage"]: """Return stages in developmental order.""" return [ - cls.EARLY, cls.BEAN, cls.COMMA, cls.FOLD_1_5, - cls.FOLD_2, cls.PRETZEL, cls.HATCHING, cls.HATCHED + cls.EARLY, + cls.BEAN, + cls.COMMA, + cls.FOLD_1_5, + cls.FOLD_2, + cls.PRETZEL, + cls.HATCHING, + cls.HATCHED, ] @classmethod - def ordered_values(cls) -> List[str]: + def ordered_values(cls) -> list[str]: """Return stage string values in developmental order.""" return [s.value for s in cls.ordered_list()] @@ -61,7 +68,7 @@ def is_valid(cls, stage: str) -> bool: return stage in cls.all_valid_values() @classmethod - def all_valid_values(cls) -> List[str]: + def all_valid_values(cls) -> list[str]: """Return all valid stage values including special states like 'arrested'.""" return cls.ordered_values() + ["arrested", "no_object"] @@ -97,7 +104,7 @@ def compare(cls, stage_a: str, stage_b: str) -> int: # Each stage has: # - features: what to look for (positive indicators) # - NOT_if: what rules out this stage (negative indicators) -STAGE_CRITERIA: Dict[str, Dict[str, Any]] = { +STAGE_CRITERIA: dict[str, dict[str, Any]] = { "early": { "features": [ "oval/elliptical shape", @@ -249,7 +256,7 @@ def compare(cls, stage_a: str, stage_b: str) -> int: # Transition zones between stages # Used for detecting transitional states and setting expectations for temporal analysis -TRANSITION_ZONES: Dict[str, Dict[str, Any]] = { +TRANSITION_ZONES: dict[str, dict[str, Any]] = { "early_to_bean": { "from_stage": "early", "to_stage": "bean", @@ -325,7 +332,7 @@ def compare(cls, stage_a: str, stage_b: str) -> int: } -def get_transition_zone(from_stage: str, to_stage: str) -> Dict[str, Any]: +def get_transition_zone(from_stage: str, to_stage: str) -> dict[str, Any]: """Get transition zone info between two stages.""" key = f"{from_stage}_to_{to_stage}" return TRANSITION_ZONES.get(key, {}) diff --git a/gently/settings.py b/gently/settings.py index 5a0bdb89..3ff7ac9a 100644 --- a/gently/settings.py +++ b/gently/settings.py @@ -4,6 +4,7 @@ All configurable values live here. Override via environment variables prefixed with GENTLY_ (e.g., GENTLY_VIZ_PORT=9090). """ + import os from dataclasses import dataclass, field from pathlib import Path @@ -26,9 +27,34 @@ def _env(key: str, default): return val +def _load_local_overrides(): + """Merge config/settings.local.yml (a flat map of GENTLY_* keys) into the + environment BEFORE settings are resolved. setdefault so a real env var still + wins over the file. This is how the Settings panel's restart-required editors + persist overrides — every entry point (viz, device layer, agent) picks them + up at import.""" + try: + import yaml + + path = Path(__file__).resolve().parents[1] / "config" / "settings.local.yml" + if not path.exists(): + return + data = yaml.safe_load(path.read_text()) or {} + if isinstance(data, dict): + for k, v in data.items(): + if v is not None: + os.environ.setdefault(str(k), str(v)) + except Exception: + pass + + +_load_local_overrides() + + @dataclass(frozen=True) class NetworkSettings: """Ports, hosts, and bind addresses.""" + viz_port: int = field(default_factory=lambda: _env("VIZ_PORT", 8080)) viz_host: str = field(default_factory=lambda: _env("VIZ_HOST", "0.0.0.0")) device_port: int = field(default_factory=lambda: _env("DEVICE_PORT", 60610)) @@ -40,7 +66,10 @@ class NetworkSettings: @dataclass(frozen=True) class MeshSettings: """Mesh networking parameters.""" - broadcast_interval_s: float = field(default_factory=lambda: _env("MESH_BROADCAST_INTERVAL", 5.0)) + + broadcast_interval_s: float = field( + default_factory=lambda: _env("MESH_BROADCAST_INTERVAL", 5.0) + ) replay_window_s: float = field(default_factory=lambda: _env("MESH_REPLAY_WINDOW", 30.0)) reaper_interval_s: float = field(default_factory=lambda: _env("MESH_REAPER_INTERVAL", 10.0)) status_refresh_s: float = field(default_factory=lambda: _env("MESH_STATUS_REFRESH", 30.0)) @@ -51,16 +80,44 @@ class MeshSettings: @dataclass(frozen=True) class ModelSettings: - """Claude model identifiers.""" - main: str = field(default_factory=lambda: _env("MODEL_MAIN", "claude-opus-4-6")) - perception: str = field(default_factory=lambda: _env("MODEL_PERCEPTION", "claude-opus-4-5-20251101")) - fast: str = field(default_factory=lambda: _env("MODEL_FAST", "claude-haiku-4-5-20251001")) - medium: str = field(default_factory=lambda: _env("MODEL_MEDIUM", "claude-sonnet-4-5-20250929")) + """Claude model identifiers — the single source of truth for every tier. + + Tiers are split by role; capability-first per the latest models: + - main: Opus 4.8 ($5/$25). Per-user-turn reasoning + tool + orchestration (plan mode) and the dopaminergic classifier + stage. (Fable 5 was tried here but declined benign planning + turns — stop_reason="refusal" — forcing a fallback on every + turn; set MODEL_MAIN=claude-fable-5 to retry it.) + - perception: Opus 4.8 (high-res vision, $5/$25). Highest-frequency tier + (per timepoint); Opus-tier vision for perception accuracy. + - medium: Opus 4.8. Onboarding / wizard summaries. + - fast: Sonnet 4.6 ($3/$15). The cheaper/faster tier — drives the + verifier's parallel ensemble (ensemble_size calls per + verification) and blank-image / summary checks. + + API note: Opus 4.8 rejects thinking budget_tokens and sampling params + (temperature/top_p/top_k) — adaptive thinking only, depth via effort. + Sonnet 4.6 supports adaptive thinking. No assistant prefills anywhere + (4.6+ family rejects them). + """ + + main: str = field(default_factory=lambda: _env("MODEL_MAIN", "claude-opus-4-8")) + perception: str = field(default_factory=lambda: _env("MODEL_PERCEPTION", "claude-opus-4-8")) + fast: str = field(default_factory=lambda: _env("MODEL_FAST", "claude-sonnet-4-6")) + medium: str = field(default_factory=lambda: _env("MODEL_MEDIUM", "claude-opus-4-8")) + # If the main tier declines a turn (stop_reason="refusal"), retry it on this + # model instead of surfacing the refusal. Inert while main is Opus 4.8 (the + # guard skips it when fallback == main); relevant if main is set to Fable 5. + # Empty disables the fallback. + refusal_fallback: str = field( + default_factory=lambda: _env("MODEL_REFUSAL_FALLBACK", "claude-opus-4-8") + ) @dataclass(frozen=True) class StorageSettings: """File paths for data storage.""" + base_path: Path = field(default_factory=lambda: _env("STORAGE_PATH", Path("D:/Gently3"))) @property @@ -75,8 +132,8 @@ def traces_dir(self) -> Path: @dataclass(frozen=True) class TimeoutSettings: """Timeout values in seconds.""" + plan_execution: int = field(default_factory=lambda: _env("TIMEOUT_PLAN", 300)) - rpc_call: int = field(default_factory=lambda: _env("TIMEOUT_RPC", 60)) volume_acquisition: int = field(default_factory=lambda: _env("TIMEOUT_VOLUME", 15)) api_call: int = field(default_factory=lambda: _env("TIMEOUT_API", 10)) @@ -84,6 +141,7 @@ class TimeoutSettings: @dataclass(frozen=True) class ApiSettings: """External API configuration.""" + ncbi_tool: str = field(default_factory=lambda: _env("NCBI_TOOL", "gently")) ncbi_email: str = field(default_factory=lambda: _env("NCBI_EMAIL", "pskeshu@gmail.com")) @@ -91,6 +149,7 @@ class ApiSettings: @dataclass(frozen=True) class MlSettings: """Machine learning training parameters.""" + model_cache_dir: Path = field(default_factory=lambda: _env("ML_MODEL_CACHE", Path("models"))) default_batch_size: int = field(default_factory=lambda: _env("ML_BATCH_SIZE", 32)) default_epochs: int = field(default_factory=lambda: _env("ML_EPOCHS", 50)) @@ -100,14 +159,54 @@ class MlSettings: @dataclass(frozen=True) class TransferSettings: """Bulk transfer protocol parameters.""" + transfer_port: int = field(default_factory=lambda: _env("TRANSFER_PORT", 19548)) chunk_size: int = field(default_factory=lambda: _env("TRANSFER_CHUNK_SIZE", 1048576)) # 1MB - max_concurrent_transfers: int = field(default_factory=lambda: _env("TRANSFER_MAX_CONCURRENT", 4)) + max_concurrent_transfers: int = field( + default_factory=lambda: _env("TRANSFER_MAX_CONCURRENT", 4) + ) + + +@dataclass(frozen=True) +class UISettings: + """Web UI feature flags.""" + + # New agent-first UX paradigm (welcome→shell unfold, dual-rendered agent + # asks, inference-first plan mode, shared-visibility surface). Now ON by + # default; the v1 dashboard remains available as a fallback via + # GENTLY_UX_V2=0 until the v1 markup is removed in a later cleanup step. + ux_v2: bool = field(default_factory=lambda: _env("UX_V2", True)) + + # Always-on session replay (rrweb capture + semantic action log) feeding + # post-hoc human replay and agent postmortems. GENTLY_REPLAY=0 is the kill + # switch: it drops the recorder include and 403s the ingest endpoint; the + # player stays available for existing recordings. + replay: bool = field(default_factory=lambda: _env("REPLAY", True)) + # Default recording fidelity — one size does not fit all. High-fidelity DOM + # churn (the live map re-rendering at poll rate) is invaluable for a specific + # visual debug but wasteful always-on. Levels (overridable per-load with + # ?replay=): + # full — record every DOM mutation (highest churn; deep visual debug) + # balanced — block the machine-driven high-churn regions (map, 3D, temp + # graph); keeps clicks + panels. The sane always-on default. + # actions — semantic action log only (clicks/nav), no rrweb DOM stream + replay_fidelity: str = field(default_factory=lambda: _env("REPLAY_FIDELITY", "balanced")) + # Cap one tab's rrweb stream. A dashboard tab left open re-renders the live + # map / telemetry at poll rate, so a single long-lived tab can balloon its + # recording to gigabytes. Past the cap, rrweb frames are dropped (a one-time + # marker is logged); the small semantic action log keeps recording. + replay_max_tab_mb: float = field(default_factory=lambda: _env("REPLAY_MAX_TAB_MB", 120.0)) + # Total on-disk budget for all rrweb recordings. Oldest recordings are pruned + # (once, lazily) to keep the footprint under this. + replay_total_budget_mb: float = field( + default_factory=lambda: _env("REPLAY_TOTAL_BUDGET_MB", 1024.0) + ) @dataclass(frozen=True) class Settings: """Top-level settings container.""" + network: NetworkSettings = field(default_factory=NetworkSettings) mesh: MeshSettings = field(default_factory=MeshSettings) models: ModelSettings = field(default_factory=ModelSettings) @@ -116,6 +215,7 @@ class Settings: api: ApiSettings = field(default_factory=ApiSettings) ml: MlSettings = field(default_factory=MlSettings) transfer: TransferSettings = field(default_factory=TransferSettings) + ui: UISettings = field(default_factory=UISettings) # Singleton — import this everywhere diff --git a/gently/ui/web/__init__.py b/gently/ui/web/__init__.py index 1ef5ab2d..ab5bcd57 100644 --- a/gently/ui/web/__init__.py +++ b/gently/ui/web/__init__.py @@ -13,20 +13,23 @@ from .embryo_marker import mark_embryos_web from .plots import ( - generate_focus_curve_plot, generate_calibration_summary_plot, generate_edge_detection_plot, + generate_focus_curve_plot, ) + # Lazy import for server (requires FastAPI) def get_visualization_server(): from .server import VisualizationServer, create_visualization_server + return VisualizationServer, create_visualization_server + __all__ = [ - 'mark_embryos_web', - 'get_visualization_server', - 'generate_focus_curve_plot', - 'generate_calibration_summary_plot', - 'generate_edge_detection_plot', + "mark_embryos_web", + "get_visualization_server", + "generate_focus_curve_plot", + "generate_calibration_summary_plot", + "generate_edge_detection_plot", ] diff --git a/gently/ui/web/accounts.py b/gently/ui/web/accounts.py new file mode 100644 index 00000000..3da1c1a0 --- /dev/null +++ b/gently/ui/web/accounts.py @@ -0,0 +1,189 @@ +"""Self-managed user accounts for the web UI. + +A small, dependency-free account store: users live in a YAML file under the +storage directory (NOT the repo), passwords are PBKDF2-hashed, and browser +sessions are stateless HMAC-signed cookies. This is the "self-managed +accounts" backend chosen for the LAN deployment; institute SSO can be layered +on later behind the same ``resolve_role`` surface in ``auth.py``. + +Roles +----- + viewer -- read-only. Sees everything (today's watching experience). + operator -- viewer + may take the microscope control lock and drive. + admin -- operator + may manage users. + +Layout (under /auth/) + users.yaml -- { users: { : {role, salt, hash, iterations, created_at} } } + secret.key -- random key used to sign session cookies (created on first run) +""" + +from __future__ import annotations + +import base64 +import hashlib +import hmac +import logging +import secrets +import time +from datetime import datetime +from pathlib import Path + +import yaml + +logger = logging.getLogger(__name__) + +ROLES = ("viewer", "operator", "admin") +CONTROL_ROLES = frozenset({"operator", "admin"}) +_PBKDF2_ITERATIONS = 200_000 +_SESSION_TTL_SECONDS = 7 * 24 * 3600 # 1 week + + +def _b64(raw: bytes) -> str: + return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=") + + +def _unb64(s: str) -> bytes: + pad = "=" * (-len(s) % 4) + return base64.urlsafe_b64decode(s + pad) + + +class AccountStore: + """File-backed user accounts + signed session tokens.""" + + def __init__(self, auth_dir: Path): + self.auth_dir = Path(auth_dir) + self.auth_dir.mkdir(parents=True, exist_ok=True) + self.users_path = self.auth_dir / "users.yaml" + self.secret_path = self.auth_dir / "secret.key" + self._users: dict = self._load_users() + self._secret: bytes = self._load_or_create_secret() + + # ── Persistence ─────────────────────────────────────────── + def _load_users(self) -> dict: + if not self.users_path.exists(): + return {} + try: + data = yaml.safe_load(self.users_path.read_text(encoding="utf-8")) or {} + return data.get("users", {}) or {} + except Exception as e: + logger.error("Failed to read users.yaml: %s", e) + return {} + + def _save_users(self) -> None: + tmp = self.users_path.with_suffix(".yaml.tmp") + tmp.write_text(yaml.safe_dump({"users": self._users}, sort_keys=True), encoding="utf-8") + tmp.replace(self.users_path) # atomic + + def _load_or_create_secret(self) -> bytes: + if self.secret_path.exists(): + return self.secret_path.read_bytes() + secret = secrets.token_bytes(32) + self.secret_path.write_bytes(secret) + try: + self.secret_path.chmod(0o600) + except OSError: + pass # best-effort on Windows + return secret + + # ── Users ───────────────────────────────────────────────── + def has_users(self) -> bool: + return bool(self._users) + + def list_users(self) -> list: + return [ + {"username": u, "role": r.get("role", "viewer")} for u, r in sorted(self._users.items()) + ] + + def get_role(self, username: str) -> str | None: + rec = self._users.get(username) + return rec.get("role") if rec else None + + def _hash(self, password: str, salt: bytes, iterations: int) -> bytes: + return hashlib.pbkdf2_hmac("sha256", password.encode("utf-8"), salt, iterations) + + def create_user(self, username: str, password: str, role: str = "viewer") -> None: + username = (username or "").strip() + if not username: + raise ValueError("username required") + if role not in ROLES: + raise ValueError(f"role must be one of {ROLES}") + salt = secrets.token_bytes(16) + self._users[username] = { + "role": role, + "salt": salt.hex(), + "hash": self._hash(password, salt, _PBKDF2_ITERATIONS).hex(), + "iterations": _PBKDF2_ITERATIONS, + "created_at": datetime.now().isoformat(timespec="seconds"), + } + self._save_users() + + def verify_password(self, username: str, password: str) -> str | None: + """Return the user's role if the password matches, else None.""" + rec = self._users.get((username or "").strip()) + if not rec: + return None + try: + salt = bytes.fromhex(rec["salt"]) + expected = bytes.fromhex(rec["hash"]) + iterations = int(rec.get("iterations", _PBKDF2_ITERATIONS)) + except (KeyError, ValueError): + return None + candidate = self._hash(password, salt, iterations) + if hmac.compare_digest(candidate, expected): + return rec.get("role", "viewer") + return None + + def bootstrap_admin_if_empty(self) -> tuple[str, str] | None: + """If no users exist, create an admin with a random password. + + Returns (username, password) so the launcher can print it once, or + None if users already exist. + """ + if self._users: + return None + password = secrets.token_urlsafe(12) + self.create_user("admin", password, role="admin") + logger.info("Bootstrapped default admin account") + return ("admin", password) + + # ── Sessions (stateless signed cookie) ──────────────────── + def issue_session(self, username: str, ttl: int = _SESSION_TTL_SECONDS) -> str: + expiry = int(time.time()) + ttl + payload = f"{username}|{expiry}".encode() + sig = hmac.new(self._secret, payload, hashlib.sha256).digest() + return f"{_b64(payload)}.{_b64(sig)}" + + def verify_session(self, token: str) -> str | None: + """Return the username for a valid, unexpired token, else None.""" + if not token or "." not in token: + return None + try: + payload_b64, sig_b64 = token.split(".", 1) + payload = _unb64(payload_b64) + sig = _unb64(sig_b64) + except Exception: + return None + expected = hmac.new(self._secret, payload, hashlib.sha256).digest() + if not hmac.compare_digest(sig, expected): + return None + try: + username, expiry_s = payload.decode("utf-8").rsplit("|", 1) + if int(expiry_s) < int(time.time()): + return None + except Exception: + return None + # The user may have been deleted since the token was issued. + return username if username in self._users else None + + +# ── Module-level singleton (set during server init) ─────────── +_store: AccountStore | None = None + + +def set_account_store(store: AccountStore | None) -> None: + global _store + _store = store + + +def get_account_store() -> AccountStore | None: + return _store diff --git a/gently/ui/web/auth.py b/gently/ui/web/auth.py new file mode 100644 index 00000000..f57ccb2d --- /dev/null +++ b/gently/ui/web/auth.py @@ -0,0 +1,127 @@ +"""Web-UI authorization roles. + +Two roles: + view -- read-only. GET endpoints, SSE / WebSocket event streams. + control -- can drive hardware (POST/PUT/DELETE). Localhost is always + control; remote callers must present a matching token in the + X-Gently-Token header (token read from GENTLY_CONTROL_TOKEN). + +Routes that move hardware or mutate persistent state declare a dependency: + + from gently.ui.web.auth import require_control + + @router.post("/api/devices/foo") + async def foo(_=Depends(require_control)): + ... + +Default-deny on control: if the token env var is unset, remote callers get +view-only access until the operator provisions a token. That matches the +"diSPIM computer alone gives control directions" intent while leaving room +for authenticated remote operators later. +""" + +from __future__ import annotations + +import logging +import os +from enum import Enum + +from fastapi import HTTPException, Request + +logger = logging.getLogger(__name__) + + +_LOOPBACK_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"}) + +# Header name used to upgrade a remote session to control role (legacy +# single-shared-token path, used only when no user accounts are configured). +_TOKEN_HEADER = "X-Gently-Token" +_TOKEN_ENV = "GENTLY_CONTROL_TOKEN" + +# Browser session cookie set by the login flow (see routes/auth_routes.py). +SESSION_COOKIE = "gently_session" + + +class Role(str, Enum): + VIEW = "view" + CONTROL = "control" + + +def current_username(request: Request) -> str | None: + """Return the authenticated username from the session cookie, or None. + + None when no account store is configured or the cookie is missing/invalid. + """ + from gently.ui.web.accounts import get_account_store + + store = get_account_store() + if store is None: + return None + token = request.cookies.get(SESSION_COOKIE) + return store.verify_session(token) if token else None + + +def _configured_token() -> str | None: + """Return the shared control token, or None if no token is provisioned. + + Read fresh each request so the operator can rotate the token without + restarting the web server. + """ + tok = os.environ.get(_TOKEN_ENV, "").strip() + return tok or None + + +def resolve_role(request: Request) -> Role: + """Determine the effective role for a request. + + Account mode (preferred): if user accounts are configured, identity comes + from the signed session cookie — operators/admins get control, everyone + else (including anonymous) gets view. + + Legacy mode (no accounts configured): localhost is always control (the + diSPIM box); remote callers need X-Gently-Token matching + GENTLY_CONTROL_TOKEN. This keeps existing single-operator rigs working + until an admin provisions accounts. + """ + from gently.ui.web.accounts import CONTROL_ROLES, get_account_store + + store = get_account_store() + if store is not None and store.has_users(): + username = current_username(request) + if username: + role = store.get_role(username) + return Role.CONTROL if role in CONTROL_ROLES else Role.VIEW + return Role.VIEW + + # Legacy mode — no accounts configured. + client = request.client + host = client.host if client else None + if host in _LOOPBACK_HOSTS: + return Role.CONTROL + + token = _configured_token() + if token is not None: + supplied = request.headers.get(_TOKEN_HEADER, "").strip() + if supplied and supplied == token: + return Role.CONTROL + + return Role.VIEW + + +def require_control(request: Request) -> Role: + """FastAPI dependency — 403 unless the caller has the control role. + + Logs the denied client host (without leaking the token) so the operator + can spot if a remote browser is trying to drive hardware. + """ + role = resolve_role(request) + if role is Role.CONTROL: + return role + host = request.client.host if request.client else "unknown" + logger.warning("control-route 403 for %s -> %s %s", host, request.method, request.url.path) + raise HTTPException( + status_code=403, + detail="control role required (this endpoint moves hardware or " + "mutates persistent state; localhost has it by default, " + "remote callers need X-Gently-Token)", + ) diff --git a/gently/ui/web/connection_manager.py b/gently/ui/web/connection_manager.py index 266e9cb2..1d1f2a23 100644 --- a/gently/ui/web/connection_manager.py +++ b/gently/ui/web/connection_manager.py @@ -9,7 +9,6 @@ import json import logging from datetime import datetime -from typing import Dict, Optional from .models import ClientInfo, ImageData @@ -18,6 +17,7 @@ # Optional imports try: from fastapi import WebSocket + FASTAPI_AVAILABLE = True except ImportError: FASTAPI_AVAILABLE = False @@ -28,13 +28,25 @@ class ConnectionManager: # Colors for avatar backgrounds (pleasant, distinct colors) AVATAR_COLORS = [ - '#4a9eff', '#ff6b6b', '#51cf66', '#ffd43b', '#cc5de8', - '#ff922b', '#20c997', '#748ffc', '#f06595', '#69db7c', - '#ffa94d', '#9775fa', '#38d9a9', '#e599f7', '#74c0fc' + "#4a9eff", + "#ff6b6b", + "#51cf66", + "#ffd43b", + "#cc5de8", + "#ff922b", + "#20c997", + "#748ffc", + "#f06595", + "#69db7c", + "#ffa94d", + "#9775fa", + "#38d9a9", + "#e599f7", + "#74c0fc", ] def __init__(self): - self.active_connections: Dict[WebSocket, ClientInfo] = {} + self.active_connections: dict[WebSocket, ClientInfo] = {} self._lock = asyncio.Lock() def _generate_color(self, client_id: str) -> str: @@ -42,12 +54,15 @@ def _generate_color(self, client_id: str) -> str: hash_val = sum(ord(c) for c in client_id) return self.AVATAR_COLORS[hash_val % len(self.AVATAR_COLORS)] - async def connect(self, websocket: WebSocket, client_id: str = None, name: str = None): + async def connect( + self, websocket: WebSocket, client_id: str | None = None, name: str | None = None + ): await websocket.accept() # Generate defaults if not provided if not client_id: import uuid + client_id = str(uuid.uuid4())[:8] if not name: name = f"Anonymous {client_id[:4]}" @@ -56,12 +71,14 @@ async def connect(self, websocket: WebSocket, client_id: str = None, name: str = client_id=client_id, name=name, color=self._generate_color(client_id), - connected_at=datetime.now().isoformat() + connected_at=datetime.now().isoformat(), ) async with self._lock: self.active_connections[websocket] = client_info - logger.info(f"WebSocket connected: {name} ({client_id}). Total: {len(self.active_connections)}") + logger.info( + f"WebSocket connected: {name} ({client_id}). Total: {len(self.active_connections)}" + ) # Broadcast updated presence to all clients await self.broadcast_presence() @@ -70,7 +87,9 @@ async def disconnect(self, websocket: WebSocket): async with self._lock: client_info = self.active_connections.pop(websocket, None) if client_info: - logger.info(f"WebSocket disconnected: {client_info.name}. Total: {len(self.active_connections)}") + logger.info( + f"WebSocket disconnected: {client_info.name}. Total: {len(self.active_connections)}" + ) else: logger.info(f"WebSocket disconnected. Total: {len(self.active_connections)}") @@ -86,11 +105,11 @@ async def update_client_name(self, websocket: WebSocket, name: str): client_id=old_info.client_id, name=name, color=old_info.color, - connected_at=old_info.connected_at + connected_at=old_info.connected_at, ) await self.broadcast_presence() - def get_client_info(self, websocket: WebSocket) -> Optional[ClientInfo]: + def get_client_info(self, websocket: WebSocket) -> ClientInfo | None: """Get client info for a websocket""" return self.active_connections.get(websocket) @@ -102,12 +121,12 @@ async def broadcast_presence(self): # Deduplicate by client_id (same user in multiple tabs = one avatar) async with self._lock: seen_clients = {} - for ws, info in self.active_connections.items(): + for _ws, info in self.active_connections.items(): # Keep the most recent entry for each client_id seen_clients[info.client_id] = { - 'client_id': info.client_id, - 'name': info.name, - 'color': info.color + "client_id": info.client_id, + "name": info.name, + "color": info.color, } clients_list = list(seen_clients.values()) @@ -117,14 +136,8 @@ async def broadcast_presence(self): try: personalized = [] for client in clients_list: - personalized.append({ - **client, - 'is_you': client['client_id'] == info.client_id - }) - await ws.send_json({ - 'type': 'presence', - 'clients': personalized - }) + personalized.append({**client, "is_you": client["client_id"] == info.client_id}) + await ws.send_json({"type": "presence", "clients": personalized}) except Exception: disconnected.append(ws) @@ -133,7 +146,7 @@ async def broadcast_presence(self): for ws in disconnected: self.active_connections.pop(ws, None) - async def broadcast(self, message: Dict): + async def broadcast(self, message: dict): """Broadcast message to all connected clients""" if not self.active_connections: return @@ -145,7 +158,12 @@ async def broadcast(self, message: Dict): try: await connection.send_text(message_json) except Exception as e: - logger.warning(f"Failed to send to websocket: {e}") + # Expected when a client disconnects/reloads mid-broadcast + # (send after websocket.close). The connection is dropped + # below, so this is debug-level, not a warning. + logger.debug( + "Dropping a websocket that errored on send (client likely gone): %s", e + ) disconnected.append(connection) # Remove disconnected clients @@ -154,18 +172,19 @@ async def broadcast(self, message: Dict): async def send_image(self, image_data: ImageData): """Send image data to all connected clients""" - await self.broadcast({ - 'type': 'image', - 'data': image_data.to_dict() - }) + await self.broadcast({"type": "image", "data": image_data.to_dict()}) - async def send_event(self, event_type: str, data: Dict, source: str = None, event_id: str = None): + async def send_event( + self, event_type: str, data: dict, source: str | None = None, event_id: str | None = None + ): """Send event notification to all clients""" - await self.broadcast({ - 'type': 'event', - 'event_type': event_type, - 'data': data, - 'source': source or 'unknown', - 'event_id': event_id or '', - 'timestamp': datetime.now().isoformat() - }) + await self.broadcast( + { + "type": "event", + "event_type": event_type, + "data": data, + "source": source or "unknown", + "event_id": event_id or "", + "timestamp": datetime.now().isoformat(), + } + ) diff --git a/gently/ui/web/embryo_marker.py b/gently/ui/web/embryo_marker.py index e84b26a4..d2002bc1 100644 --- a/gently/ui/web/embryo_marker.py +++ b/gently/ui/web/embryo_marker.py @@ -13,9 +13,7 @@ """ import logging -from typing import List, Dict, Tuple, Optional from pathlib import Path -from datetime import datetime import numpy as np @@ -25,13 +23,13 @@ async def mark_embryos_web( viz_server, image: np.ndarray, - initial_stage_position: Tuple[float, float], + initial_stage_position: tuple[float, float], pixel_size_um: float = 0.65, - timeout: Optional[float] = None, - save_image_path: Optional[Path] = None, - initial_markers: Optional[List[Dict]] = None, + timeout: float | None = None, + save_image_path: Path | None = None, + initial_markers: list[dict] | None = None, default_role: str = "test", -) -> List[Dict]: +) -> list[dict]: """ Interactive embryo marking via the web map view. @@ -86,52 +84,66 @@ async def mark_embryos_web( return embryos -def _save_marked_image(image: np.ndarray, embryos: List[Dict], output_path: Path): +def _save_marked_image(image: np.ndarray, embryos: list[dict], output_path: Path): """Save image with embryo markers drawn on it.""" - from PIL import Image as PILImage, ImageDraw, ImageFont + from PIL import Image as PILImage + from PIL import ImageDraw, ImageFont output_path = Path(output_path) if image.dtype != np.uint8: - img_normalized = ((image - image.min()) / - max(image.max() - image.min(), 1) * 255).astype(np.uint8) + img_normalized = ((image - image.min()) / max(image.max() - image.min(), 1) * 255).astype( + np.uint8 + ) else: img_normalized = image pil_image = PILImage.fromarray(img_normalized) - if pil_image.mode != 'RGB': - pil_image = pil_image.convert('RGB') + if pil_image.mode != "RGB": + pil_image = pil_image.convert("RGB") draw = ImageDraw.Draw(pil_image) for embryo in embryos: - pixel_x, pixel_y = embryo['pixel_position'] - embryo_num = embryo.get('embryo_number') or embryo.get('embryo_id') or '?' + pixel_x, pixel_y = embryo["pixel_position"] + embryo_num = embryo.get("embryo_number") or embryo.get("embryo_id") or "?" marker_size = 20 draw.line( [(pixel_x - marker_size, pixel_y), (pixel_x + marker_size, pixel_y)], - fill=(0, 255, 255), width=3 + fill=(0, 255, 255), + width=3, ) draw.line( [(pixel_x, pixel_y - marker_size), (pixel_x, pixel_y + marker_size)], - fill=(0, 255, 255), width=3 + fill=(0, 255, 255), + width=3, ) circle_radius = 40 draw.ellipse( - [pixel_x - circle_radius, pixel_y - circle_radius, - pixel_x + circle_radius, pixel_y + circle_radius], - outline=(0, 255, 255), width=2 + [ + pixel_x - circle_radius, + pixel_y - circle_radius, + pixel_x + circle_radius, + pixel_y + circle_radius, + ], + outline=(0, 255, 255), + width=2, ) + font: ImageFont.FreeTypeFont | ImageFont.ImageFont try: font = ImageFont.truetype("arial.ttf", 24) except Exception: font = ImageFont.load_default() - draw.text((pixel_x - 10, pixel_y + circle_radius + 5), - str(embryo_num), fill=(0, 255, 255), font=font) + draw.text( + (pixel_x - 10, pixel_y + circle_radius + 5), + str(embryo_num), + fill=(0, 255, 255), + font=font, + ) pil_image.save(output_path) logger.info("Saved marked image: %s", output_path) diff --git a/gently/ui/web/image_store.py b/gently/ui/web/image_store.py index d27b236c..09fcd49f 100644 --- a/gently/ui/web/image_store.py +++ b/gently/ui/web/image_store.py @@ -5,11 +5,13 @@ Organized storage for images by type and embryo. """ -from typing import Dict, List, Optional - from .models import ( - ImageData, EmbryoImageCache, Volume3DData, - CALIBRATION_TYPES, VOLUME_TYPES, ANALYSIS_TYPES, + ANALYSIS_TYPES, + CALIBRATION_TYPES, + VOLUME_TYPES, + EmbryoImageCache, + ImageData, + Volume3DData, ) @@ -17,11 +19,11 @@ class ImageStore: """Organized storage for images by type and embryo (unlimited)""" def __init__(self): - self._embryo_caches: Dict[str, EmbryoImageCache] = {} - self._global_images: List[ImageData] = [] # Images without embryo_id - self._calibration_images: List[ImageData] = [] # Global calibration - self._volume_images: List[ImageData] = [] # Global volumes - self._volumes_3d: Dict[str, Volume3DData] = {} # 3D volumes by UID + self._embryo_caches: dict[str, EmbryoImageCache] = {} + self._global_images: list[ImageData] = [] # Images without embryo_id + self._calibration_images: list[ImageData] = [] # Global calibration + self._volume_images: list[ImageData] = [] # Global volumes + self._volumes_3d: dict[str, Volume3DData] = {} # 3D volumes by UID def _get_embryo_cache(self, embryo_id: str) -> EmbryoImageCache: if embryo_id not in self._embryo_caches: @@ -30,7 +32,7 @@ def _get_embryo_cache(self, embryo_id: str) -> EmbryoImageCache: def add_image(self, image: ImageData): """Add image to appropriate storage based on type and embryo""" - embryo_id = image.metadata.get('embryo_id') + embryo_id = image.metadata.get("embryo_id") data_type = image.data_type if data_type in CALIBRATION_TYPES or data_type in ANALYSIS_TYPES: @@ -55,7 +57,7 @@ def add_image(self, image: ImageData): else: self._global_images.append(image) - def get_all_calibration(self, embryo_id: Optional[str] = None) -> List[ImageData]: + def get_all_calibration(self, embryo_id: str | None = None) -> list[ImageData]: """Get calibration images, optionally filtered by embryo""" if embryo_id: cache = self._embryo_caches.get(embryo_id) @@ -66,7 +68,7 @@ def get_all_calibration(self, embryo_id: Optional[str] = None) -> List[ImageData all_cal.extend(cache.calibration) return sorted(all_cal, key=lambda x: x.timestamp) - def get_all_volumes(self, embryo_id: Optional[str] = None) -> List[ImageData]: + def get_all_volumes(self, embryo_id: str | None = None) -> list[ImageData]: """Get volume images, optionally filtered by embryo""" if embryo_id: cache = self._embryo_caches.get(embryo_id) @@ -76,7 +78,7 @@ def get_all_volumes(self, embryo_id: Optional[str] = None) -> List[ImageData]: all_vol.extend(cache.volumes) return sorted(all_vol, key=lambda x: x.timestamp) - def get_all_snapshots(self, embryo_id: Optional[str] = None) -> List[ImageData]: + def get_all_snapshots(self, embryo_id: str | None = None) -> list[ImageData]: """Get snapshot images (including volume projections), optionally filtered by embryo""" if embryo_id: cache = self._embryo_caches.get(embryo_id) @@ -91,11 +93,11 @@ def get_all_snapshots(self, embryo_id: Optional[str] = None) -> List[ImageData]: all_snap.extend(cache.volumes) return sorted(all_snap, key=lambda x: x.timestamp) - def get_embryo_ids(self) -> List[str]: + def get_embryo_ids(self) -> list[str]: """Get list of all embryo IDs with images""" return list(self._embryo_caches.keys()) - def get_image_by_uid(self, uid: str) -> Optional[ImageData]: + def get_image_by_uid(self, uid: str) -> ImageData | None: """Find image by UID across all storage""" for img in self._global_images: if img.uid == uid: @@ -120,11 +122,11 @@ def add_volume_3d(self, volume_data: Volume3DData): oldest_uid = next(iter(self._volumes_3d)) del self._volumes_3d[oldest_uid] - def get_volume_3d(self, uid: str) -> Optional[Volume3DData]: + def get_volume_3d(self, uid: str) -> Volume3DData | None: """Get a 3D volume by UID""" return self._volumes_3d.get(uid) - def get_all_volumes_3d(self) -> List[Dict]: + def get_all_volumes_3d(self) -> list[dict]: """Get info for all 3D volumes (without heavy data)""" return [v.to_info_dict() for v in self._volumes_3d.values()] @@ -132,9 +134,9 @@ def get_sequence( self, embryo_id: str, start: int = 0, - end: Optional[int] = None, - data_type: Optional[str] = None - ) -> List[ImageData]: + end: int | None = None, + data_type: str | None = None, + ) -> list[ImageData]: """Get ordered sequence of images for an embryo within a timepoint range. Args: @@ -158,8 +160,8 @@ def get_sequence( all_images = [img for img in all_images if img.data_type == data_type] # Filter by timepoint range - def get_timepoint(img: ImageData) -> Optional[int]: - tp = img.metadata.get('timepoint') + def get_timepoint(img: ImageData) -> int | None: + tp = img.metadata.get("timepoint") if tp is not None: return int(tp) return None @@ -179,7 +181,7 @@ def get_timepoint(img: ImageData) -> Optional[int]: filtered.sort(key=lambda x: get_timepoint(x) or 0) return filtered - def get_stats(self) -> Dict: + def get_stats(self) -> dict: """Get storage statistics""" total_cal = len(self._calibration_images) total_vol = len(self._volume_images) @@ -191,10 +193,10 @@ def get_stats(self) -> Dict: total_snap += len(cache.snapshots) return { - 'embryo_count': len(self._embryo_caches), - 'calibration_count': total_cal, - 'volume_count': total_vol, - 'snapshot_count': total_snap, - 'volumes_3d_count': len(self._volumes_3d), - 'embryo_ids': list(self._embryo_caches.keys()), + "embryo_count": len(self._embryo_caches), + "calibration_count": total_cal, + "volume_count": total_vol, + "snapshot_count": total_snap, + "volumes_3d_count": len(self._volumes_3d), + "embryo_ids": list(self._embryo_caches.keys()), } diff --git a/gently/ui/web/launch_prefs.py b/gently/ui/web/launch_prefs.py new file mode 100644 index 00000000..d6acfee4 --- /dev/null +++ b/gently/ui/web/launch_prefs.py @@ -0,0 +1,130 @@ +"""Persistence for the launch gate's two toggles (+ advanced defaults). + +The launch gate answers exactly two questions — *microscope hardware on/off* and +*AI agent on/off* — and everything else (device port, SAM device) is a remembered +default behind "Advanced options". Those choices live in +``config/launch.local.json`` (gitignored) so the gate is prefilled every boot. + +See ``docs/superpowers/specs/2026-07-02-unified-launcher-design.md`` (RFC #78). +""" + +from __future__ import annotations + +import json +import logging +from pathlib import Path + +from gently.settings import settings + +logger = logging.getLogger(__name__) + +# Repo root: gently/ui/web/launch_prefs.py -> parents[3] +_CONFIG_DIR = Path(__file__).resolve().parents[3] / "config" +PREFS_PATH = _CONFIG_DIR / "launch.local.json" + +# Only these keys are read from / written to disk. Advanced values fall back to +# settings so a fresh install has sensible defaults without a prefs file. +_DEFAULTS: dict = { + "hardware": True, # start + connect the device layer + "agent": True, # enable chat / perception / planning (needs API key) + "port": settings.network.device_port, + # "auto" is resolved to cuda/cpu at load time by GPU auto-detection — a + # biologist should never have to choose (RFC #78). Only a scope wrangler + # pins a concrete value (in Settings). + "sam_device": "auto", +} +_ALLOWED_KEYS = set(_DEFAULTS) + +_sam_device_cache: str | None = None + + +def detect_sam_device() -> str: + """Auto-detect the SAM inference device: ``cuda`` if an NVIDIA GPU is present, + else ``cpu``. Cached — detection is cheap once torch is loaded.""" + global _sam_device_cache + if _sam_device_cache is not None: + return _sam_device_cache + dev = "cpu" + try: + import torch + + if torch.cuda.is_available(): + dev = "cuda" + except Exception: + # torch missing/failed — fall back to a plain nvidia-smi presence check. + import shutil + import subprocess + + if shutil.which("nvidia-smi"): + try: + subprocess.run(["nvidia-smi"], capture_output=True, timeout=4, check=True) + dev = "cuda" + except Exception: + pass + _sam_device_cache = dev + logger.info("SAM device auto-detected: %s", dev) + return dev + + +def stored_prefs() -> dict: + """The raw persisted prefs (unresolved), or {} — for the Settings UI which + needs to show 'auto' vs a pinned SAM device.""" + try: + if PREFS_PATH.exists(): + data = json.loads(PREFS_PATH.read_text(encoding="utf-8")) + if isinstance(data, dict): + return data + except (OSError, ValueError): + pass + return {} + + +def load_prefs() -> dict: + """Return the persisted launch choices merged over defaults. + + Never raises — a missing or corrupt file just yields the defaults, so the + gate always renders. + """ + prefs = dict(_DEFAULTS) + try: + if PREFS_PATH.exists(): + stored = json.loads(PREFS_PATH.read_text(encoding="utf-8")) + if isinstance(stored, dict): + prefs.update({k: stored[k] for k in _ALLOWED_KEYS if k in stored}) + except (OSError, ValueError) as e: + logger.warning("launch prefs unreadable (%s) — using defaults", e) + # Resolve the SAM device unless a concrete value was pinned in Settings. + if prefs.get("sam_device") in (None, "", "auto"): + prefs["sam_device"] = detect_sam_device() + return prefs + + +def save_prefs(prefs: dict) -> dict: + """Persist a (partial) set of launch choices; returns the merged result. + + Unknown keys are ignored; known keys are coerced to the expected types. + """ + merged = load_prefs() + for key in _ALLOWED_KEYS & set(prefs): + merged[key] = _coerce(key, prefs[key]) + try: + _CONFIG_DIR.mkdir(parents=True, exist_ok=True) + PREFS_PATH.write_text(json.dumps(merged, indent=2), encoding="utf-8") + except OSError as e: + logger.error("could not persist launch prefs: %s", e) + return merged + + +def _coerce(key: str, value): + """Coerce a JSON-supplied value to the type its default implies.""" + default = _DEFAULTS[key] + if isinstance(default, bool): + if isinstance(value, str): + return value.strip().lower() in ("1", "true", "yes", "on") + return bool(value) + if isinstance(default, int): + try: + return int(value) + except (TypeError, ValueError): + return default + return value diff --git a/gently/ui/web/models.py b/gently/ui/web/models.py index b61147fd..43c05498 100644 --- a/gently/ui/web/models.py +++ b/gently/ui/web/models.py @@ -5,38 +5,47 @@ Dataclasses and type constants used across the visualization package. """ -from dataclasses import dataclass, field, asdict -from typing import Any, Dict, List, Optional +from dataclasses import asdict, dataclass, field +from typing import Any import numpy as np - # Data types for routing to tabs CALIBRATION_TYPES = { - 'focus_sweep', 'focus_plot', 'edge_detection', 'calibration_summary', - 'focus_snap', 'focus_coarse', 'focus_curve', 'focus_assess' + "focus_sweep", + "focus_plot", + "edge_detection", + "calibration_summary", + "focus_snap", + "focus_coarse", + "focus_curve", + "focus_assess", } -VOLUME_TYPES = { - 'volume', 'volume_projection', 'z_stack', 'timelapse' -} +VOLUME_TYPES = {"volume", "volume_projection", "z_stack", "timelapse"} # CV/Analysis types - shown in a separate "Analysis" category within Calibration ANALYSIS_TYPES = { - 'segmentation', 'detection', 'classification', 'tracking', + "segmentation", + "detection", + "classification", + "tracking", # CV agent visualization types - 'roi_detection', 'cropped_roi', 'vision_prepared', 'timeline', 'cv_visualization' + "roi_detection", + "cropped_roi", + "vision_prepared", + "timeline", + "cv_visualization", } # 3D types that support Z-slider browsing -VOLUME_3D_TYPES = { - 'segmentation_3d' -} +VOLUME_3D_TYPES = {"segmentation_3d"} @dataclass class ClientInfo: """Information about a connected WebSocket client for presence tracking""" + client_id: str name: str color: str # Hex color for avatar background @@ -46,13 +55,14 @@ class ClientInfo: @dataclass class Volume3DData: """Container for 3D volume data with segmentation overlay""" + uid: str data_type: str timestamp: str volume: np.ndarray # Original volume (Z, H, W) - masks: np.ndarray # Segmentation masks (Z, H, W) + masks: np.ndarray # Segmentation masks (Z, H, W) colors: np.ndarray # Cell colors (num_labels, 3) - metadata: Dict[str, Any] = field(default_factory=dict) + metadata: dict[str, Any] = field(default_factory=dict) @property def num_slices(self) -> int: @@ -89,37 +99,39 @@ def get_slice_overlay(self, z: int, alpha: float = 0.4) -> np.ndarray: return rgb - def to_info_dict(self) -> Dict: + def to_info_dict(self) -> dict: """Return metadata without the heavy arrays""" return { - 'uid': self.uid, - 'data_type': self.data_type, - 'timestamp': self.timestamp, - 'shape': list(self.shape), - 'num_slices': self.num_slices, - 'num_cells': int(self.masks.max()), - 'metadata': self.metadata + "uid": self.uid, + "data_type": self.data_type, + "timestamp": self.timestamp, + "shape": list(self.shape), + "num_slices": self.num_slices, + "num_cells": int(self.masks.max()), + "metadata": self.metadata, } @dataclass class ImageData: """Container for image data sent to clients""" + uid: str data_type: str # 'volume', 'projection', 'snapshot', 'detection', 'focus_sweep', etc. timestamp: str - metadata: Dict[str, Any] = field(default_factory=dict) - base64_png: Optional[str] = None - shape: Optional[tuple] = None + metadata: dict[str, Any] = field(default_factory=dict) + base64_png: str | None = None + shape: tuple | None = None - def to_dict(self) -> Dict: + def to_dict(self) -> dict: return asdict(self) @dataclass class EmbryoImageCache: """Per-embryo image organization""" + embryo_id: str - volumes: List[ImageData] = field(default_factory=list) - calibration: List[ImageData] = field(default_factory=list) - snapshots: List[ImageData] = field(default_factory=list) + volumes: list[ImageData] = field(default_factory=list) + calibration: list[ImageData] = field(default_factory=list) + snapshots: list[ImageData] = field(default_factory=list) diff --git a/gently/ui/web/plots.py b/gently/ui/web/plots.py index 437f792d..a376740a 100644 --- a/gently/ui/web/plots.py +++ b/gently/ui/web/plots.py @@ -5,22 +5,24 @@ Uses matplotlib with Agg backend for thread safety. """ -import numpy as np -from typing import Optional, Tuple, List +from typing import cast import matplotlib -matplotlib.use('Agg') # Non-interactive backend for thread safety +import numpy as np + +matplotlib.use("Agg") # Non-interactive backend for thread safety import matplotlib.pyplot as plt +from matplotlib.backends.backend_agg import FigureCanvasAgg def generate_focus_curve_plot( positions: np.ndarray, scores: np.ndarray, best_position: float, - fit_params: Optional[np.ndarray] = None, + fit_params: np.ndarray | None = None, r_squared: float = 0.0, title: str = "Focus Curve", - figsize: Tuple[int, int] = (6, 4), + figsize: tuple[int, int] = (6, 4), dpi: int = 100, ) -> np.ndarray: """ @@ -53,24 +55,34 @@ def generate_focus_curve_plot( fig, ax = plt.subplots(figsize=figsize, dpi=dpi) # Data points - ax.scatter(positions, scores, c='#2196F3', s=50, zorder=3, label='Measurements') + ax.scatter(positions, scores, c="#2196F3", s=50, zorder=3, label="Measurements") # Gaussian fit curve if fit_params is not None and len(fit_params) >= 4: a, mu, sigma, c = fit_params[:4] x_fit = np.linspace(positions.min(), positions.max(), 200) - y_fit = a * np.exp(-((x_fit - mu) ** 2) / (2 * sigma ** 2)) + c - ax.plot(x_fit, y_fit, color='#F44336', linewidth=2, - label=f'Gaussian fit (R²={r_squared:.3f})') + y_fit = a * np.exp(-((x_fit - mu) ** 2) / (2 * sigma**2)) + c + ax.plot( + x_fit, + y_fit, + color="#F44336", + linewidth=2, + label=f"Gaussian fit (R²={r_squared:.3f})", + ) # Best position marker - ax.axvline(best_position, color='#4CAF50', linestyle='--', linewidth=2, - label=f'Best: {best_position:.2f} µm') + ax.axvline( + best_position, + color="#4CAF50", + linestyle="--", + linewidth=2, + label=f"Best: {best_position:.2f} µm", + ) - ax.set_xlabel('Piezo Position (µm)', fontsize=11) - ax.set_ylabel('Focus Score', fontsize=11) - ax.set_title(title, fontsize=12, fontweight='bold') - ax.legend(loc='upper right', framealpha=0.9) + ax.set_xlabel("Piezo Position (µm)", fontsize=11) + ax.set_ylabel("Focus Score", fontsize=11) + ax.set_title(title, fontsize=12, fontweight="bold") + ax.legend(loc="upper right", framealpha=0.9) ax.grid(True, alpha=0.3) # Tight layout @@ -78,7 +90,7 @@ def generate_focus_curve_plot( # Convert to numpy array fig.canvas.draw() - buf = np.asarray(fig.canvas.buffer_rgba()) + buf = np.asarray(cast(FigureCanvasAgg, fig.canvas).buffer_rgba()) plt.close(fig) return buf[:, :, :3].astype(np.uint8) @@ -94,7 +106,7 @@ def generate_calibration_summary_plot( offset: float, r_squared_top: float = 0.0, r_squared_bottom: float = 0.0, - figsize: Tuple[int, int] = (7, 5), + figsize: tuple[int, int] = (7, 5), dpi: int = 100, ) -> np.ndarray: """ @@ -135,49 +147,63 @@ def generate_calibration_summary_plot( # Calibration points galvos = [galvo_top, galvo_bottom] piezos = [piezo_top, piezo_bottom] - ax.scatter(galvos, piezos, c='#2196F3', s=100, zorder=3, - label='Calibration points') + ax.scatter(galvos, piezos, c="#2196F3", s=100, zorder=3, label="Calibration points") # Linear fit line margin = 0.05 galvo_range = np.linspace( min(galvo_top, galvo_bottom) - margin, max(galvo_top, galvo_bottom) + margin, - 100 + 100, ) piezo_fit = slope * galvo_range + offset - ax.plot(galvo_range, piezo_fit, color='#F44336', linewidth=2, - label=f'Linear fit: piezo = {slope:.1f}·galvo + {offset:.1f}') + ax.plot( + galvo_range, + piezo_fit, + color="#F44336", + linewidth=2, + label=f"Linear fit: piezo = {slope:.1f}·galvo + {offset:.1f}", + ) # Annotations - ax.annotate(f'Top\nR²={r_squared_top:.3f}', - (galvo_top, piezo_top), textcoords="offset points", - xytext=(10, 10), fontsize=9, color='#666') - ax.annotate(f'Bottom\nR²={r_squared_bottom:.3f}', - (galvo_bottom, piezo_bottom), textcoords="offset points", - xytext=(10, -20), fontsize=9, color='#666') - - ax.set_xlabel('Galvo Position (degrees)', fontsize=11) - ax.set_ylabel('Piezo Position (µm)', fontsize=11) - ax.set_title(f'{embryo_id} - Piezo-Galvo Calibration', fontsize=12, fontweight='bold') - ax.legend(loc='upper left', framealpha=0.9) + ax.annotate( + f"Top\nR²={r_squared_top:.3f}", + (galvo_top, piezo_top), + textcoords="offset points", + xytext=(10, 10), + fontsize=9, + color="#666", + ) + ax.annotate( + f"Bottom\nR²={r_squared_bottom:.3f}", + (galvo_bottom, piezo_bottom), + textcoords="offset points", + xytext=(10, -20), + fontsize=9, + color="#666", + ) + + ax.set_xlabel("Galvo Position (degrees)", fontsize=11) + ax.set_ylabel("Piezo Position (µm)", fontsize=11) + ax.set_title(f"{embryo_id} - Piezo-Galvo Calibration", fontsize=12, fontweight="bold") + ax.legend(loc="upper left", framealpha=0.9) ax.grid(True, alpha=0.3) fig.tight_layout() fig.canvas.draw() - buf = np.asarray(fig.canvas.buffer_rgba()) + buf = np.asarray(cast(FigureCanvasAgg, fig.canvas).buffer_rgba()) plt.close(fig) return buf[:, :, :3].astype(np.uint8) def generate_edge_detection_plot( - galvo_positions: List[float], - visibility: List[bool], - edge_top: Optional[float] = None, - edge_bottom: Optional[float] = None, + galvo_positions: list[float], + visibility: list[bool], + edge_top: float | None = None, + edge_bottom: float | None = None, embryo_id: str = "embryo", - figsize: Tuple[int, int] = (6, 4), + figsize: tuple[int, int] = (6, 4), dpi: int = 100, ) -> np.ndarray: """ @@ -211,35 +237,51 @@ def generate_edge_detection_plot( vis_numeric = [1 if v else 0 for v in visibility] # Plot visibility as step function - colors = ['#4CAF50' if v else '#F44336' for v in visibility] + colors = ["#4CAF50" if v else "#F44336" for v in visibility] ax.scatter(galvo_positions, vis_numeric, c=colors, s=80, zorder=3) # Draw step-like connecting lines for i in range(len(galvo_positions) - 1): - color = '#4CAF50' if visibility[i] else '#F44336' - ax.hlines(vis_numeric[i], galvo_positions[i], galvo_positions[i+1], - color=color, alpha=0.3, linewidth=2) + color = "#4CAF50" if visibility[i] else "#F44336" + ax.hlines( + vis_numeric[i], + galvo_positions[i], + galvo_positions[i + 1], + color=color, + alpha=0.3, + linewidth=2, + ) # Mark edges if provided if edge_top is not None: - ax.axvline(edge_top, color='#2196F3', linestyle='--', linewidth=2, - label=f'Top edge: {edge_top:.3f}°') + ax.axvline( + edge_top, + color="#2196F3", + linestyle="--", + linewidth=2, + label=f"Top edge: {edge_top:.3f}°", + ) if edge_bottom is not None: - ax.axvline(edge_bottom, color='#FF9800', linestyle='--', linewidth=2, - label=f'Bottom edge: {edge_bottom:.3f}°') - - ax.set_xlabel('Galvo Position (degrees)', fontsize=11) - ax.set_ylabel('Embryo Visible', fontsize=11) + ax.axvline( + edge_bottom, + color="#FF9800", + linestyle="--", + linewidth=2, + label=f"Bottom edge: {edge_bottom:.3f}°", + ) + + ax.set_xlabel("Galvo Position (degrees)", fontsize=11) + ax.set_ylabel("Embryo Visible", fontsize=11) ax.set_yticks([0, 1]) - ax.set_yticklabels(['No', 'Yes']) - ax.set_title(f'{embryo_id} - Edge Detection', fontsize=12, fontweight='bold') + ax.set_yticklabels(["No", "Yes"]) + ax.set_title(f"{embryo_id} - Edge Detection", fontsize=12, fontweight="bold") if edge_top is not None or edge_bottom is not None: - ax.legend(loc='best', framealpha=0.9) - ax.grid(True, alpha=0.3, axis='x') + ax.legend(loc="best", framealpha=0.9) + ax.grid(True, alpha=0.3, axis="x") fig.tight_layout() fig.canvas.draw() - buf = np.asarray(fig.canvas.buffer_rgba()) + buf = np.asarray(cast(FigureCanvasAgg, fig.canvas).buffer_rgba()) plt.close(fig) return buf[:, :, :3].astype(np.uint8) diff --git a/gently/ui/web/routes/__init__.py b/gently/ui/web/routes/__init__.py index 13f72b53..4f6a2e5d 100644 --- a/gently/ui/web/routes/__init__.py +++ b/gently/ui/web/routes/__init__.py @@ -6,31 +6,51 @@ a FastAPI ``APIRouter`` bound to the server instance. """ -from .pages import create_router as create_pages_router -from .sessions import create_router as create_sessions_router -from .images import create_router as create_images_router -from .volumes import create_router as create_volumes_router -from .data import create_router as create_data_router -from .websocket import create_router as create_websocket_router from .agent_ws import create_router as create_agent_ws_router +from .auth_routes import create_router as create_auth_router from .campaigns import create_router as create_campaigns_router from .chat import create_router as create_chat_router +from .context import create_router as create_context_router +from .data import create_router as create_data_router +from .device_layer import create_router as create_device_layer_router from .experiments import create_router as create_experiments_router +from .images import create_router as create_images_router +from .logs import create_router as create_logs_router +from .notebook import create_router as create_notebook_router +from .operation_plan import create_router as create_operation_plan_router +from .pages import create_router as create_pages_router +from .replay import create_router as create_replay_router +from .roles import create_router as create_roles_router +from .sessions import create_router as create_sessions_router +from .tactic_library import create_router as create_tactic_library_router +from .temperature import create_router as create_temperature_router +from .volumes import create_router as create_volumes_router +from .websocket import create_router as create_websocket_router def register_all_routes(server): """Register all route groups on the server's FastAPI app.""" for factory in ( create_pages_router, + create_auth_router, + create_device_layer_router, create_sessions_router, create_campaigns_router, create_experiments_router, create_images_router, + create_logs_router, create_volumes_router, create_data_router, create_websocket_router, create_agent_ws_router, create_chat_router, + create_context_router, + create_notebook_router, + create_temperature_router, + create_operation_plan_router, + create_roles_router, + create_tactic_library_router, + create_replay_router, ): router = factory(server) server.app.include_router(router) diff --git a/gently/ui/web/routes/agent_ws.py b/gently/ui/web/routes/agent_ws.py index df59aaeb..46472d19 100644 --- a/gently/ui/web/routes/agent_ws.py +++ b/gently/ui/web/routes/agent_ws.py @@ -9,11 +9,13 @@ import asyncio import json import logging +from collections.abc import Callable from datetime import datetime -from typing import Dict, Optional from fastapi import APIRouter, WebSocket, WebSocketDisconnect +from gently.settings import settings + logger = logging.getLogger(__name__) @@ -29,15 +31,181 @@ def create_router(server) -> APIRouter: router = APIRouter() # Pending choice futures keyed by request_id - _choice_futures: Dict[str, asyncio.Future] = {} + _choice_futures: dict[str, asyncio.Future] = {} + + # ── Single-driver control arbitration ───────────────────── + # Shared across all /ws/agent clients (the router is created once). + # Only the control holder may drive the agent (chat/command/cancel); + # everyone else is an observer until they take control. This is the + # seed of the multi-user control lock and also prevents the shared + # agent conversation from being corrupted when >1 client connects. + _control: dict[str, str | None] = {"holder": None} + _clients: dict[str, Callable] = {} + _client_labels: dict[str, str] = {} + _client_counter = {"n": 0} + _raw_clients: dict[str, WebSocket] = {} # client_id -> websocket (broadcast) + + # ── Uniform display transcript ──────────────────────────── + # A single conversation history shared by every client of this session. + # Persisted to /chat_display.json so it survives reconnects and + # restarts; broadcast live so all instances stay in sync. + _history: list = [] + _history_state = {"sid": None, "path": None, "agent_buf": None, "autonomous": False} + + async def _broadcast_control_status(): + """Tell every connected agent client who currently holds control.""" + holder = _control["holder"] + holder_label = _client_labels.get(holder) if holder else None + for cid, fn in list(_clients.items()): + try: + await fn( + { + "type": "control_status", + "holder": holder, + "holder_label": holder_label, + "you_have_control": (cid == holder), + } + ) + except Exception: + pass + + def _load_history_for_session(bridge): + """Load the current session's display history, reloading if the + session changed (e.g. after a resume from the Sessions tab).""" + try: + agent = bridge.agent + store = getattr(agent, "store", None) + sid = getattr(agent, "session_id", None) + except Exception: + return + if sid == _history_state["sid"]: + return # already loaded for this session + # Session changed (or first load): reset and reload from disk. + _history.clear() + _history_state["sid"] = sid + _history_state["path"] = None + _history_state["agent_buf"] = None + _history_state["autonomous"] = False + try: + if store and sid: + sdir = store._session_dir(sid) + if sdir: + p = sdir / "chat_display.json" + _history_state["path"] = p + if p.exists(): + loaded = json.loads(p.read_text(encoding="utf-8")) or [] + if isinstance(loaded, list): + _history.extend(loaded) + except Exception: + logger.debug("Could not load chat history", exc_info=True) + + # Fallback: sessions created before chat_display.json existed (or any + # session resumed for the first time) — derive a best-effort transcript + # from the saved Claude conversation so the chat still shows history. + if not _history and store and sid: + try: + snap = store.load_session_snapshot(sid) or {} + for m in snap.get("conversation_history") or []: + role = m.get("role") + content = m.get("content") + if isinstance(content, list): + text = "".join( + b.get("text", "") + for b in content + if isinstance(b, dict) and b.get("type") == "text" + ) + else: + text = content if isinstance(content, str) else "" + text = (text or "").strip() + if not text: + continue + if role == "user": + _history.append({"role": "user", "text": text}) + elif role == "assistant": + _history.append({"role": "agent", "text": text}) + except Exception: + logger.debug("Could not derive history from conversation", exc_info=True) + + def _save_history(): + p = _history_state["path"] + if not p: + return + try: + tmp = p.with_suffix(".json.tmp") + tmp.write_text(json.dumps(_history[-500:]), encoding="utf-8") + tmp.replace(p) + except Exception: + pass - async def _run_wizard(wizard, websocket, send_fn, _choice_futures, bridge=None, log_transcript=None): + def _record(item): + _history.append(item) + if len(_history) > 500: + del _history[: len(_history) - 500] + _save_history() + + def _flush_agent_buf(): + buf = _history_state["agent_buf"] + if buf: + # An autonomous (wake) turn's text is recorded distinctly so replay + # shows it as "Gently · autonomous", not an ordinary agent reply. + role = "autonomous" if _history_state.get("autonomous") else "agent" + _record({"role": role, "text": buf}) + _history_state["agent_buf"] = None + + def _record_display(msg): + """Fold a streamed chunk into the persistent display history.""" + t = msg.get("type") + if t == "user_message": + _flush_agent_buf() + _history_state["autonomous"] = False + _record( + { + "role": "user", + "text": msg.get("text", ""), + "author": msg.get("author"), + "author_id": msg.get("author_id"), + } + ) + elif t == "autonomous_start": + # An autonomous wake turn is beginning — record the trigger banner + # and mark following text as autonomous until stream_end. + _flush_agent_buf() + _history_state["autonomous"] = True + _record({"role": "autonomous_start", "trigger": msg.get("trigger", "")}) + elif t == "text": + _history_state["agent_buf"] = (_history_state["agent_buf"] or "") + msg.get("text", "") + elif t == "tool_call": + _flush_agent_buf() + _record( + { + "role": "tool", + "name": msg.get("tool_name"), + "duration": msg.get("duration"), + "summary": msg.get("result_summary"), + } + ) + elif t == "stream_end": + _flush_agent_buf() + _history_state["autonomous"] = False + + async def _broadcast(msg): + """Record to history + send a display message to ALL clients.""" + _record_display(msg) + for _cid, ws in list(_raw_clients.items()): + try: + await ws.send_json(msg) + except Exception: + pass + + async def _run_wizard( + wizard, websocket, send_fn, _choice_futures, bridge=None, log_transcript=None + ): """Run the wizard's interactive loop. Returns the wizard task so callers can check for exceptions. Used both at startup and for the /wizard command. """ - _wizard_input_future: Optional[asyncio.Future] = None + _wizard_input_future: asyncio.Future | None = None async def _wizard_wait_for_input() -> str: nonlocal _wizard_input_future @@ -48,11 +216,13 @@ async def _wizard_wait_for_input() -> str: async def _wizard_wait_for_choice(choice_data: dict) -> str: request_id = _make_request_id() choice_data["request_id"] = request_id - await send_fn({ - "type": "choice_request", - "choice_data": choice_data, - "request_id": request_id, - }) + await send_fn( + { + "type": "choice_request", + "choice_data": choice_data, + "request_id": request_id, + } + ) loop = asyncio.get_event_loop() future = loop.create_future() _choice_futures[request_id] = future @@ -65,7 +235,8 @@ async def _wizard_wait_for_choice(choice_data: dict) -> str: while not wizard_task.done(): try: raw = await asyncio.wait_for( - websocket.receive_text(), timeout=60.0, + websocket.receive_text(), + timeout=60.0, ) except asyncio.TimeoutError: await websocket.send_json({"type": "ping"}) @@ -101,12 +272,18 @@ async def _wizard_wait_for_choice(choice_data: dict) -> str: # /reset-context kills the wizard — context is gone if command.strip().lower() == "/reset-context": wizard_task.cancel() - await send_fn({ - "type": "stream_end", - "tokens": {"input_tokens": 0, "output_tokens": 0, - "total_tokens": 0, "api_calls": 0}, - "wizard_complete": True, - }) + await send_fn( + { + "type": "stream_end", + "tokens": { + "input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "api_calls": 0, + }, + "wizard_complete": True, + } + ) return wizard_task elif msg_type == "ping": @@ -129,18 +306,55 @@ async def agent_websocket(websocket: WebSocket): bridge = getattr(server, "agent_bridge", None) if bridge is None: - await websocket.send_json({ - "type": "error", - "error": "Agent bridge not initialized", - }) + await websocket.send_json( + { + "type": "error", + "error": "Agent bridge not initialized", + } + ) await websocket.close() return - # Send connection metadata (version, tokens, embryo count, commands) + # Route autonomous (wake-router) turns through this router's _broadcast so + # they stream to all chat clients + persist to the display transcript. + # Idempotent; _broadcast is router-scoped and fans out to whoever is live. + bridge.register_display_broadcaster(_broadcast) + + # ── Authenticate the connection (account mode) ──────────── + # When user accounts are configured, identity comes from the signed + # session cookie (set at login). Viewers may watch but not drive; + # operators/admins may take the control lock. With no accounts + # configured we fall back to the legacy "anyone connected can drive". + from gently.ui.web.accounts import CONTROL_ROLES, get_account_store + from gently.ui.web.auth import SESSION_COOKIE + + _acct = get_account_store() + username = None + can_control = True # legacy default when no accounts are configured + if _acct is not None and _acct.has_users(): + # Viewing is open: anonymous clients may connect and *watch* the + # conversation. Only authenticated operators/admins can hold or + # take the control lock (enforced on the drive actions below). + _token = websocket.cookies.get(SESSION_COOKIE) + username = _acct.verify_session(_token) if _token else None + role = _acct.get_role(username) if username else None + can_control = role in CONTROL_ROLES + + # Assign a stable id for control arbitration. The label shown to other + # clients is the username when authenticated, else "Anonymous". The UI + # renders "You" for the viewer's own messages by matching client_id, so + # anonymous participants don't need disambiguating numbers. + _client_counter["n"] += 1 + client_id = f"agent_client_{_client_counter['n']}" + client_label = username or "Anonymous" + + # Send connection metadata (version, tokens, embryo count, commands). + # you_id lets the client label its own messages "You". meta = bridge.get_connect_metadata() _connected_msg = { "type": "connected", **meta, + "you_id": client_id, "timestamp": datetime.now().isoformat(), } await websocket.send_json(_connected_msg) @@ -189,10 +403,16 @@ async def _push_peer_discovered(event): "_type": "single", "question": f"New peer discovered: {hostname}", "options": [ - {"id": "pair", "label": "Pair", - "description": f"Start pairing with {hostname}"}, - {"id": "ignore", "label": "Ignore", - "description": "Dismiss (you can pair later via /pair)"}, + { + "id": "pair", + "label": "Pair", + "description": f"Start pairing with {hostname}", + }, + { + "id": "ignore", + "label": "Ignore", + "description": "Dismiss (you can pair later via /pair)", + }, ], "allow_multiple": False, }, @@ -241,12 +461,20 @@ async def _push_pairing_requested(event): "type": "choice_request", "choice_data": { "_type": "single", - "question": f"{hostname} wants to pair\nVerify this code matches: {pin}", + "question": ( + f"{hostname} wants to pair\nVerify this code matches: {pin}" + ), "options": [ - {"id": "accept", "label": "Accept pairing", - "description": f"Trust {hostname} and allow mesh communication"}, - {"id": "reject", "label": "Reject", - "description": "Decline this pairing request"}, + { + "id": "accept", + "label": "Accept pairing", + "description": f"Trust {hostname} and allow mesh communication", + }, + { + "id": "reject", + "label": "Reject", + "description": "Decline this pairing request", + }, ], "allow_multiple": False, }, @@ -327,7 +555,9 @@ async def _push_scope_denied(event): pass # Peer discovery - unsub = server.event_bus.subscribe_async(_ET.MESH_PEER_DISCOVERED, _push_peer_discovered) + unsub = server.event_bus.subscribe_async( + _ET.MESH_PEER_DISCOVERED, _push_peer_discovered + ) _mesh_unsubs.append(unsub) unsub = server.event_bus.subscribe_async(_ET.MESH_PEER_LOST, _push_peer_lost) _mesh_unsubs.append(unsub) @@ -335,23 +565,29 @@ async def _push_scope_denied(event): _mesh_unsubs.append(unsub) # Pairing events - unsub = server.event_bus.subscribe_async(_ET.MESH_PAIRING_REQUESTED, _push_pairing_requested) + unsub = server.event_bus.subscribe_async( + _ET.MESH_PAIRING_REQUESTED, _push_pairing_requested + ) _mesh_unsubs.append(unsub) - unsub = server.event_bus.subscribe_async(_ET.MESH_PAIRING_COMPLETED, _push_pairing_completed) + unsub = server.event_bus.subscribe_async( + _ET.MESH_PAIRING_COMPLETED, _push_pairing_completed + ) _mesh_unsubs.append(unsub) # Security events unsub = server.event_bus.subscribe_async(_ET.MESH_AUTH_FAILURE, _push_auth_failure) _mesh_unsubs.append(unsub) - unsub = server.event_bus.subscribe_async(_ET.MESH_CERT_PIN_FAILURE, _push_cert_pin_failure) + unsub = server.event_bus.subscribe_async( + _ET.MESH_CERT_PIN_FAILURE, _push_cert_pin_failure + ) _mesh_unsubs.append(unsub) unsub = server.event_bus.subscribe_async(_ET.MESH_SCOPE_DENIED, _push_scope_denied) _mesh_unsubs.append(unsub) # Active streaming task (so we can cancel on disconnect) - active_task: Optional[asyncio.Task] = None + active_task: asyncio.Task | None = None wizard_task = None - bootstrap_task: Optional[asyncio.Task] = None + bootstrap_task: asyncio.Task | None = None # ── Session transcript ──────────────────────────────── # Log every WebSocket message (both directions) to a JSONL @@ -367,7 +603,9 @@ async def _push_scope_denied(event): sdir = store._session_dir(sid) if sdir and sdir.exists(): _transcript_file = open( - sdir / "transcript.jsonl", "a", encoding="utf-8", + sdir / "transcript.jsonl", + "a", + encoding="utf-8", ) logger.info("Transcript logging to %s", sdir / "transcript.jsonl") except Exception as e: @@ -409,24 +647,65 @@ def choice_future_factory(choice_data: dict) -> asyncio.Future: _choice_futures[request_id] = future return future + def _discard_choice(request_id: str) -> None: + _choice_futures.pop(request_id, None) + + # Give the bridge the choice-factory + discard too, so ASK-mode autonomous + # turns can round-trip an approval picker through this connection's channel + # and clean up the future on timeout/cancel. + bridge.register_display_broadcaster(_broadcast, choice_future_factory, _discard_choice) + + # Register this client for control arbitration; grant control if free + # (only to clients allowed to drive — viewers never auto-hold). + _clients[client_id] = send_fn + _client_labels[client_id] = client_label + _raw_clients[client_id] = websocket + if _control["holder"] is None and can_control: + _control["holder"] = client_id + await _broadcast_control_status() + + # Replay the uniform session transcript so every client (and every + # reconnect/refresh) shows the same conversation. + _load_history_for_session(bridge) + if _history: + try: + await websocket.send_json({"type": "history", "items": list(_history)}) + except Exception: + pass + try: # ── Wizard phase ────────────────────────────────────── - # Run startup wizard (if needed) before entering the REPL. + # The startup wizard no longer auto-pops in the chat — setup is now + # launched on demand from the Home page (which sends /wizard) or via + # the /wizard command. Re-enable auto-run by setting + # server.wizard_autorun = True. NOTE: wizard_ran below is still + # derived from wizard.needed, so the briefing/resolution path is + # unaffected by this gate. wizard = getattr(bridge, "_wizard", None) - if wizard is not None and wizard.needed: + if wizard is not None and wizard.needed and getattr(server, "wizard_autorun", False): wizard_task = await _run_wizard( - wizard, websocket, send_fn, _choice_futures, bridge, + wizard, + websocket, + send_fn, + _choice_futures, + bridge, log_transcript=_log_transcript, ) exc = _handle_wizard_result(wizard_task) if exc: logger.error(f"Wizard error: {exc}", exc_info=exc) - await send_fn({ - "type": "stream_end", - "tokens": {"input_tokens": 0, "output_tokens": 0, - "total_tokens": 0, "api_calls": 0}, - "wizard_complete": True, - }) + await send_fn( + { + "type": "stream_end", + "tokens": { + "input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "api_calls": 0, + }, + "wizard_complete": True, + } + ) # ── Auto-briefing or resolution picker ──────────────── # New sessions with multiple unblocked imaging candidates @@ -441,48 +720,71 @@ def choice_future_factory(choice_data: dict) -> asyncio.Future: async def _run_resolution_bootstrap(): try: await bridge.bootstrap_resolution_picker( - send_fn, choice_future_factory, + send_fn, + choice_future_factory, ) except asyncio.CancelledError: raise except Exception as exc: logger.error( - "Resolution picker failed; falling back to " - "static briefing: %s", - exc, exc_info=exc, + "Resolution picker failed; falling back to static briefing: %s", + exc, + exc_info=exc, ) try: briefing = bridge.get_session_briefing() if briefing: await send_fn({"type": "stream_start"}) await send_fn({"type": "text", "text": briefing}) - await send_fn({ - "type": "stream_end", - "tokens": {"input_tokens": 0, "output_tokens": 0, - "total_tokens": 0, "api_calls": 0}, - }) + await send_fn( + { + "type": "stream_end", + "tokens": { + "input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "api_calls": 0, + }, + } + ) except Exception: pass if not wizard_ran: - if bridge.should_enter_resolution(): + enter_resolution = bridge.should_enter_resolution() + # Under ux_v2 the agent-first landing owns the session-entry + # decision ("Plan an experiment" / "Take a quick look"), so the + # legacy connect-time resolution picker would just duplicate it — + # and contradict it, by offering "Standalone" after the user has + # already chosen to plan. Stay quiet on connect for new sessions; + # the landing drives plan-mode (/plan) or standalone instead. + if enter_resolution and not settings.ui.ux_v2: bootstrap_task = asyncio.create_task(_run_resolution_bootstrap()) - else: + elif not enter_resolution: + # Resume / already-resolved sessions still get their briefing + # (it sits behind the landing overlay until dismissed). briefing = bridge.get_session_briefing() if briefing: await send_fn({"type": "stream_start"}) await send_fn({"type": "text", "text": briefing}) - await send_fn({ - "type": "stream_end", - "tokens": {"input_tokens": 0, "output_tokens": 0, - "total_tokens": 0, "api_calls": 0}, - }) + await send_fn( + { + "type": "stream_end", + "tokens": { + "input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "api_calls": 0, + }, + } + ) # ── Main REPL loop ──────────────────────────────────── while True: try: raw = await asyncio.wait_for( - websocket.receive_text(), timeout=60.0, + websocket.receive_text(), + timeout=60.0, ) except asyncio.TimeoutError: await websocket.send_json({"type": "ping"}) @@ -497,6 +799,53 @@ async def _run_resolution_bootstrap(): _log_transcript("in", data) msg_type = data.get("type") + # ── Control arbitration ─────────────────────────── + # A client requesting the wheel. + if msg_type == "take_control": + if not can_control: + await send_fn( + { + "type": "notification", + "level": "warning", + "title": "View-only role", + "body": "Your account can watch but not control the microscope.", + } + ) + await _broadcast_control_status() + continue + prev = _control["holder"] + _control["holder"] = client_id + if prev and prev != client_id and prev in _clients: + try: + await _clients[prev]( + { + "type": "notification", + "level": "warning", + "title": f"Control taken by {client_label}", + "body": "You are now viewing.", + } + ) + except Exception: + pass + await _broadcast_control_status() + continue + + # Only the holder may drive the agent. Observers are told + # to take control rather than silently corrupting the + # single shared conversation. + if msg_type in ("chat", "command", "cancel") and client_id != _control["holder"]: + holder_label = _client_labels.get(_control["holder"] or "") or "another client" + await send_fn( + { + "type": "notification", + "level": "info", + "title": f"Viewing only — control is held by {holder_label}", + "body": "Take control to drive the microscope.", + } + ) + await _broadcast_control_status() + continue + if msg_type == "chat": text = data.get("text", "").strip() if not text: @@ -506,11 +855,27 @@ async def _run_resolution_bootstrap(): if active_task and not active_task.done(): active_task.cancel() + # Echo the user's message to ALL clients (so observers see + # what was asked), then stream the reply to everyone. author + # is the display name (username or "Anonymous"); author_id + # lets each client render its own messages as "You". + await _broadcast( + { + "type": "user_message", + "text": text, + "author": client_label, + "author_id": client_id, + } + ) active_task = asyncio.create_task( - bridge.stream_response(text, send_fn, choice_future_factory) + bridge.stream_response(text, _broadcast, choice_future_factory) ) elif msg_type == "choice_response": + # Only the control holder answers pickers (observers see + # them read-only). + if _control["holder"] != client_id: + continue request_id = data.get("request_id", "") selected = data.get("selected", "") # Check if bridge owns this choice (e.g. /import-embryos picker) @@ -525,6 +890,10 @@ async def _run_resolution_bootstrap(): if active_task and not active_task.done(): active_task.cancel() active_task = None + # A cancelled stream emits no stream_end of its own, so + # tell every client the turn is over — otherwise their + # "Working…" indicator spins forever after Stop. + await _broadcast({"type": "stream_end"}) elif msg_type == "command": command = data.get("command", "").strip() @@ -535,11 +904,13 @@ async def _run_resolution_bootstrap(): if command.lower() in ("/wizard",): w = getattr(bridge, "_wizard", None) if w is None: - await send_fn({ - "type": "command_result", - "command": "/wizard", - "error": "Wizard not available", - }) + await send_fn( + { + "type": "command_result", + "command": "/wizard", + "error": "Wizard not available", + } + ) else: # Re-create wizard so it re-assesses gaps cs = getattr(bridge, "_context_store", None) @@ -548,40 +919,60 @@ async def _run_resolution_bootstrap(): w = bridge._wizard # Tell TUI we're entering wizard mode - await send_fn({ - "type": "command_result", - "command": "/wizard", - "content": {"wizard_active": True}, - }) + await send_fn( + { + "type": "command_result", + "command": "/wizard", + "content": {"wizard_active": True}, + } + ) wizard_task = await _run_wizard( - w, websocket, send_fn, _choice_futures, bridge, + w, + websocket, + send_fn, + _choice_futures, + bridge, log_transcript=_log_transcript, ) exc = _handle_wizard_result(wizard_task) if exc: logger.error(f"Wizard error: {exc}", exc_info=exc) - await send_fn({ - "type": "stream_end", - "tokens": {"input_tokens": 0, "output_tokens": 0, - "total_tokens": 0, "api_calls": 0}, - "wizard_complete": True, - }) + await send_fn( + { + "type": "stream_end", + "tokens": { + "input_tokens": 0, + "output_tokens": 0, + "total_tokens": 0, + "api_calls": 0, + }, + "wizard_complete": True, + } + ) else: try: - await bridge.handle_command(command, send_fn, choice_futures=_choice_futures) + await bridge.handle_command( + command, send_fn, choice_futures=_choice_futures + ) except Exception as e: logger.error("Command '%s' failed: %s", command, e, exc_info=True) - await send_fn({ - "type": "command_result", - "command": command, - "error": str(e), - }) + await send_fn( + { + "type": "command_result", + "command": command, + "error": str(e), + } + ) elif msg_type == "browse": target = data.get("target", "") await _handle_browse( - target, data, server, bridge, send_fn, + target, + data, + server, + bridge, + send_fn, ) elif msg_type == "ping": @@ -591,7 +982,7 @@ async def _run_resolution_bootstrap(): pass # response to our ping except WebSocketDisconnect: - logger.info("TUI client disconnected") + logger.info("Agent websocket client disconnected") except asyncio.CancelledError: pass except Exception as e: @@ -613,13 +1004,35 @@ async def _run_resolution_bootstrap(): wizard_task.cancel() if active_task and not active_task.done(): active_task.cancel() + # This connection was mid-stream — persist whatever the agent + # generated so far, otherwise a reload loses the in-progress + # reply (it's only committed on stream_end). Guarded to the + # owning connection so an observer's disconnect can't split a + # still-streaming reply into two history entries. + try: + _flush_agent_buf() + except Exception: + logger.debug("Could not flush agent buffer on disconnect", exc_info=True) if bootstrap_task is not None and not bootstrap_task.done(): bootstrap_task.cancel() - # Clean up pending futures - for future in _choice_futures.values(): - if not future.done(): - future.cancel() - _choice_futures.clear() + # Release control arbitration for this client; hand the wheel + # to any remaining client (or free it) and resync everyone. + _clients.pop(client_id, None) + _client_labels.pop(client_id, None) + _raw_clients.pop(client_id, None) + if _control["holder"] == client_id: + _control["holder"] = next(iter(_clients), None) + try: + await _broadcast_control_status() + except Exception: + pass + # Clean up pending futures only when the last client leaves — + # otherwise we'd cancel another connected client's pending choices. + if not _clients: + for future in _choice_futures.values(): + if not future.done(): + future.cancel() + _choice_futures.clear() return router @@ -643,13 +1056,20 @@ def serialize_campaign(c): else: children = [] items_raw = cs.get_plan_items(campaign_id=c.id) - items = [{ - "id": item.id, - "title": item.title, - "status": item.status.value if hasattr(item.status, "value") else str(item.status), - "type": item.type.value if hasattr(item.type, "value") else str(item.type), - "claimed_by_hostname": getattr(item, "claimed_by_hostname", None), - } for item in items_raw] + items = [ + { + "id": item.id, + "title": item.title, + "status": item.status.value + if hasattr(item.status, "value") + else str(item.status), + "type": item.type.value + if hasattr(item.type, "value") + else str(item.type), + "claimed_by_hostname": getattr(item, "claimed_by_hostname", None), + } + for item in items_raw + ] return { "id": c.id, "shorthand": c.shorthand or "", @@ -676,17 +1096,19 @@ def serialize_campaign(c): peers = mesh_svc.get_peers() result = [] for p in peers: - result.append({ - "instance_id": p.instance_id, - "hostname": p.hostname, - "ip_address": p.ip_address, - "viz_port": p.viz_port, - "mode": p.status.agent_mode if p.status else "unknown", - "embryo_count": p.status.embryo_count if p.status else 0, - "is_trusted": p.is_trusted, - "tls_enabled": p.tls_enabled, - "shared_campaigns": [], - }) + result.append( + { + "instance_id": p.instance_id, + "hostname": p.hostname, + "ip_address": p.ip_address, + "viz_port": p.viz_port, + "mode": p.status.agent_mode if p.status else "unknown", + "embryo_count": p.status.embryo_count if p.status else 0, + "is_trusted": p.is_trusted, + "tls_enabled": p.tls_enabled, + "shared_campaigns": [], + } + ) await send_fn({"type": "browse_result", "target": "peers", "data": result}) elif target == "peer_campaigns": @@ -708,25 +1130,29 @@ def serialize_campaign(c): campaigns = [] if p.instance_id == peer.instance_id: for c in shared: - campaigns.append({ - "id": c.get("id", ""), - "shorthand": c.get("shorthand", ""), - "description": c.get("description", ""), - "total": c.get("item_count", 0), - "completed": c.get("completed_count", 0), - "items": [], - }) - result.append({ - "instance_id": p.instance_id, - "hostname": p.hostname, - "ip_address": p.ip_address, - "viz_port": p.viz_port, - "mode": p.status.agent_mode if p.status else "unknown", - "embryo_count": p.status.embryo_count if p.status else 0, - "is_trusted": p.is_trusted, - "tls_enabled": p.tls_enabled, - "shared_campaigns": campaigns, - }) + campaigns.append( + { + "id": c.get("id", ""), + "shorthand": c.get("shorthand", ""), + "description": c.get("description", ""), + "total": c.get("item_count", 0), + "completed": c.get("completed_count", 0), + "items": [], + } + ) + result.append( + { + "instance_id": p.instance_id, + "hostname": p.hostname, + "ip_address": p.ip_address, + "viz_port": p.viz_port, + "mode": p.status.agent_mode if p.status else "unknown", + "embryo_count": p.status.embryo_count if p.status else 0, + "is_trusted": p.is_trusted, + "tls_enabled": p.tls_enabled, + "shared_campaigns": campaigns, + } + ) await send_fn({"type": "browse_result", "target": "peer_campaigns", "data": result}) elif target == "peer_campaign_items": @@ -734,31 +1160,53 @@ def serialize_campaign(c): campaign_id = data.get("campaign_id", "") mesh_svc = getattr(server, "mesh_service", None) if not mesh_svc or not hostname or not campaign_id: - await send_fn({"type": "browse_result", "target": "peer_campaign_items", "data": []}) + await send_fn( + { + "type": "browse_result", + "target": "peer_campaign_items", + "data": [], + } + ) return peer = mesh_svc.find_peer_by_hostname(hostname) if not peer or not mesh_svc.peer_client: - await send_fn({"type": "browse_result", "target": "peer_campaign_items", "data": []}) + await send_fn( + { + "type": "browse_result", + "target": "peer_campaign_items", + "data": [], + } + ) return export = await mesh_svc.peer_client.fetch_campaign_export(peer, campaign_id) if not export: - await send_fn({"type": "browse_result", "target": "peer_campaign_items", "data": []}) + await send_fn( + { + "type": "browse_result", + "target": "peer_campaign_items", + "data": [], + } + ) return items = [] for item in export.get("items", []): - items.append({ - "id": item.get("id", ""), - "title": item.get("title", ""), - "status": item.get("status", "planned"), - "claimed_by_hostname": item.get("claimed_by_hostname"), - }) - await send_fn({ - "type": "browse_result", - "target": "peer_campaign_items", - "data": items, - "campaign_id": campaign_id, - "hostname": hostname, - }) + items.append( + { + "id": item.get("id", ""), + "title": item.get("title", ""), + "status": item.get("status", "planned"), + "claimed_by_hostname": item.get("claimed_by_hostname"), + } + ) + await send_fn( + { + "type": "browse_result", + "target": "peer_campaign_items", + "data": items, + "campaign_id": campaign_id, + "hostname": hostname, + } + ) except Exception as e: logger.debug(f"Browse error ({target}): {e}") diff --git a/gently/ui/web/routes/auth_routes.py b/gently/ui/web/routes/auth_routes.py new file mode 100644 index 00000000..07e338f1 --- /dev/null +++ b/gently/ui/web/routes/auth_routes.py @@ -0,0 +1,123 @@ +"""Auth routes — login / logout / me, plus the login page. + +Self-managed accounts (see gently/ui/web/accounts.py). Login issues a signed +session cookie; roles (viewer/operator/admin) gate control elsewhere via +gently.ui.web.auth.resolve_role and the /ws/agent control lock. +""" + +import logging + +from fastapi import APIRouter, Request +from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse + +from gently.ui.web.accounts import ( + _SESSION_TTL_SECONDS, + CONTROL_ROLES, + ROLES, + get_account_store, +) +from gently.ui.web.auth import SESSION_COOKIE, current_username + +logger = logging.getLogger(__name__) + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _secure(request: Request) -> bool: + # Only mark the cookie Secure over HTTPS, else the browser drops it on + # plain-HTTP LAN deployments. + return request.url.scheme == "https" + + @router.get("/login", response_class=HTMLResponse) + async def login_page(request: Request): + store = get_account_store() + if store is None or not store.has_users(): + return RedirectResponse("/", status_code=302) + if current_username(request): + return RedirectResponse("/", status_code=302) + return server.templates.TemplateResponse(request, "login.html") + + @router.post("/api/auth/login") + async def login(request: Request): + store = get_account_store() + if store is None or not store.has_users(): + return JSONResponse({"error": "accounts not configured"}, status_code=400) + try: + body = await request.json() + except Exception: + body = {} + username = (body.get("username") or "").strip() + password = body.get("password") or "" + role = store.verify_password(username, password) + if not role: + host = request.client.host if request.client else "?" + logger.warning("login failed for %r from %s", username, host) + return JSONResponse({"error": "Invalid username or password"}, status_code=401) + token = store.issue_session(username) + resp = JSONResponse({"ok": True, "username": username, "role": role}) + resp.set_cookie( + SESSION_COOKIE, + token, + httponly=True, + samesite="lax", + secure=_secure(request), + max_age=_SESSION_TTL_SECONDS, + path="/", + ) + logger.info("login ok: %s (%s)", username, role) + return resp + + @router.post("/api/auth/logout") + async def logout(request: Request): + resp = JSONResponse({"ok": True}) + resp.delete_cookie(SESSION_COOKIE, path="/") + return resp + + @router.get("/api/auth/me") + async def me(request: Request): + store = get_account_store() + if store is None or not store.has_users(): + return JSONResponse({"accounts": False, "authenticated": False}) + username = current_username(request) + if not username: + return JSONResponse({"accounts": True, "authenticated": False}) + role = store.get_role(username) + return JSONResponse( + { + "accounts": True, + "authenticated": True, + "username": username, + "role": role, + "can_control": role in CONTROL_ROLES, + } + ) + + @router.post("/api/auth/users") + async def create_user(request: Request): + """Admin-only: provision a new account.""" + store = get_account_store() + if store is None: + return JSONResponse({"error": "accounts not configured"}, status_code=400) + requester = current_username(request) + if not requester or store.get_role(requester) != "admin": + return JSONResponse({"error": "admin role required"}, status_code=403) + try: + body = await request.json() + except Exception: + body = {} + new_user = (body.get("username") or "").strip() + password = body.get("password") or "" + role = body.get("role") or "viewer" + if not new_user or not password: + return JSONResponse({"error": "username and password required"}, status_code=400) + if role not in ROLES: + return JSONResponse({"error": f"role must be one of {list(ROLES)}"}, status_code=400) + try: + store.create_user(new_user, password, role) + except ValueError as e: + return JSONResponse({"error": str(e)}, status_code=400) + logger.info("admin %s created user %s (%s)", requester, new_user, role) + return JSONResponse({"ok": True, "username": new_user, "role": role}) + + return router diff --git a/gently/ui/web/routes/campaigns.py b/gently/ui/web/routes/campaigns.py index e5e59611..7d5f6e62 100644 --- a/gently/ui/web/routes/campaigns.py +++ b/gently/ui/web/routes/campaigns.py @@ -4,7 +4,7 @@ import logging from dataclasses import asdict from datetime import datetime -from typing import Any, Dict, List, Optional +from typing import Any from fastapi import APIRouter, Depends, HTTPException, Request @@ -115,19 +115,19 @@ async def get_campaign_document(campaign_id: str): raise HTTPException(status_code=404, detail="Campaign not found") # Collect all items across the tree and enrich with deps/dependents - bibliography = [] + bibliography: list[Any] = [] ref_index = {} # dedup by (source, key) # Pre-index every item in the tree once. The naive enrichment used to call # cs.get_plan_item(...) per dep + per dependent, each one walking the on-disk # campaign index — O(items × deps × campaigns) YAML reads per request. - items_by_id: Dict[str, Dict] = {} - dependents_map: Dict[str, List[str]] = {} + items_by_id: dict[str, dict] = {} + dependents_map: dict[str, list[str]] = {} def _index(node): for it in node.get("items", []): items_by_id[it["id"]] = it - for dep_id in (it.get("depends_on") or []): + for dep_id in it.get("depends_on") or []: dependents_map.setdefault(dep_id, []).append(it["id"]) for child in node.get("children", []): _index(child) @@ -147,16 +147,12 @@ def _enrich_tree(node): for item in node.get("items", []): item_id = item["id"] dep_ids = list(item.get("depends_on") or []) - item["dependencies"] = [ - {"id": d, "title": _resolve_title(d)} for d in dep_ids - ] + item["dependencies"] = [{"id": d, "title": _resolve_title(d)} for d in dep_ids] dnt_ids = dependents_map.get(item_id, []) - item["dependents"] = [ - {"id": d, "title": _resolve_title(d)} for d in dnt_ids - ] + item["dependents"] = [{"id": d, "title": _resolve_title(d)} for d in dnt_ids] # Collect references into bibliography - for ref in (item.get("references") or []): + for ref in item.get("references") or []: source = ref.get("source", "") key = ref.get("key", ref.get("id", ref.get("title", ""))) dedup_key = (source, key) @@ -227,19 +223,33 @@ async def get_item_detail(campaign_id: str, item_id: str): dependencies = [] for did in dep_ids: dep = cs.get_plan_item(did) - dependencies.append({"id": did, "title": dep.title if dep else did[:8], - "status": dep.status.value if dep else None}) + dependencies.append( + { + "id": did, + "title": dep.title if dep else did[:8], + "status": dep.status.value if dep else None, + } + ) # Dependents with titles dnt_ids = cs.get_plan_item_dependents(item_id) dependents = [] for did in dnt_ids: dnt = cs.get_plan_item(did) - dependents.append({"id": did, "title": dnt.title if dnt else did[:8], - "status": dnt.status.value if dnt else None}) - - # Sessions linked to this campaign - sessions = cs.get_sessions_for_campaign(item.campaign_id) + dependents.append( + { + "id": did, + "title": dnt.title if dnt else did[:8], + "status": dnt.status.value if dnt else None, + } + ) + + # Sessions — return only those linked to this specific item (item.session_ids), + # not all campaign sessions. The frontend uses item.session_ids as the canonical + # list and this pool as metadata (name, created_at) for display. + item_sids = set(item.session_ids or []) + all_sessions = cs.get_sessions_for_campaign(item.campaign_id) + sessions = [s for s in all_sessions if s.session_id in item_sids] return { "item": _serialize(item), @@ -248,6 +258,43 @@ async def get_item_detail(campaign_id: str, item_id: str): "sessions": [_serialize(s) for s in sessions], } + @router.post("/api/campaigns/{campaign_id}/items/{item_id}/sessions") + async def link_session_to_item(campaign_id: str, item_id: str, request: Request): + """Link a session to a plan item (appends) and record it against the campaign.""" + cs = _get_store() + campaign = _resolve(cs, campaign_id) + item = cs.get_plan_item(item_id) + if not item: + raise HTTPException(status_code=404, detail="Plan item not found") + + body = await request.json() + session_id = body.get("session_id") + if not session_id: + raise HTTPException(status_code=400, detail="session_id required") + + cs.link_plan_item_session(item_id, session_id) + cs.link_session_campaign(session_id, campaign.id) + + # Re-fetch item so session_ids reflects the just-added link; then filter + # exactly as get_item_detail does — POST and GET return the same scope. + item = cs.get_plan_item(item_id) + item_sids = set(item.session_ids or []) + all_sessions = cs.get_sessions_for_campaign(item.campaign_id) + sessions = [s for s in all_sessions if s.session_id in item_sids] + return {"sessions": [_serialize(s) for s in sessions]} + + @router.delete("/api/campaigns/{campaign_id}/items/{item_id}/sessions/{session_id}") + async def unlink_session_from_item(campaign_id: str, item_id: str, session_id: str): + """Remove a session link from a plan item. Returns {unlinked: bool}.""" + cs = _get_store() + _resolve(cs, campaign_id) + item = cs.get_plan_item(item_id) + if not item: + raise HTTPException(status_code=404, detail="Plan item not found") + + unlinked = cs.unlink_plan_item_session(item_id, session_id) + return {"unlinked": unlinked} + @router.get("/api/campaigns/{campaign_id}/planned-sessions") async def get_planned_sessions(campaign_id: str): """Planned sessions linked to a campaign.""" @@ -280,9 +327,12 @@ async def _require(request: Request): if required_scope not in scopes: if _audit: from gently.mesh.audit import AuditEvent + _audit.log( - AuditEvent.SCOPE_DENIED, outcome="deny", - peer_id=peer_id, ip=host, + AuditEvent.SCOPE_DENIED, + outcome="deny", + peer_id=peer_id, + ip=host, detail=f"scope={required_scope} path={request.url.path}", ) raise HTTPException( @@ -291,36 +341,103 @@ async def _require(request: Request): ) if _audit: from gently.mesh.audit import AuditEvent + _audit.log( - AuditEvent.AUTH_SUCCESS, outcome="allow", - peer_id=peer_id, ip=host, + AuditEvent.AUTH_SUCCESS, + outcome="allow", + peer_id=peer_id, + ip=host, ) return if _audit: from gently.mesh.audit import AuditEvent + _audit.log( - AuditEvent.AUTH_FAILURE, outcome="deny", - ip=host, detail=f"path={request.url.path}", + AuditEvent.AUTH_FAILURE, + outcome="deny", + ip=host, + detail=f"path={request.url.path}", ) raise HTTPException(status_code=403, detail="Mesh authentication required") return _require - @router.post("/api/campaigns/{campaign_id}/share", dependencies=[Depends(_make_campaign_auth("campaigns:admin"))]) + @router.patch( + "/api/campaigns/{campaign_id}/items/{item_id}", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) + async def update_item(campaign_id: str, item_id: str, request: Request): + """Edit plan-item fields and/or imaging-spec fields inline. + + Send only the fields you're changing. Spec edits are *merged* into the + existing spec, so the UI can PATCH a single field (e.g. laser_power_pct) + without losing the rest. An empty string clears a spec field to null. + Persists via update_plan_item, which fires PLAN_UPDATED for live refresh. + """ + cs = _get_store() + _resolve(cs, campaign_id) + item = cs.get_plan_item(item_id) + if not item: + raise HTTPException(status_code=404, detail="Plan item not found") + + body = await request.json() + if not isinstance(body, dict): + raise HTTPException(status_code=400, detail="Body must be a JSON object") + + kwargs: dict[str, Any] = {} + for f in ("title", "description", "outcome"): + if isinstance(body.get(f), str): + kwargs[f] = body[f] + if body.get("estimated_days") is not None: + kwargs["estimated_days"] = body["estimated_days"] + + if body.get("status"): + try: + kwargs["status"] = PlanItemStatus(body["status"]) + except ValueError as err: + raise HTTPException( + status_code=400, detail=f"Invalid status: {body['status']}" + ) from err + + spec_patch = body.get("spec") + if isinstance(spec_patch, dict): + current = item.imaging_spec or item.bench_spec + merged = asdict(current) if current else {} + for k, v in spec_patch.items(): + merged[k] = None if v == "" else v + kwargs["spec"] = merged + + if not kwargs: + raise HTTPException(status_code=400, detail="No editable fields supplied") + + cs.update_plan_item(item_id=item_id, **kwargs) # fires PLAN_UPDATED + updated = cs.get_plan_item(item_id) + return {"ok": True, "item": _serialize(updated)} + + @router.post( + "/api/campaigns/{campaign_id}/share", + dependencies=[Depends(_make_campaign_auth("campaigns:admin"))], + ) async def share_campaign(campaign_id: str): cs = _get_store() campaign = _resolve(cs, campaign_id) cs.share_campaign(campaign.id) return {"ok": True} - @router.post("/api/campaigns/{campaign_id}/unshare", dependencies=[Depends(_make_campaign_auth("campaigns:admin"))]) + @router.post( + "/api/campaigns/{campaign_id}/unshare", + dependencies=[Depends(_make_campaign_auth("campaigns:admin"))], + ) async def unshare_campaign(campaign_id: str): cs = _get_store() campaign = _resolve(cs, campaign_id) cs.unshare_campaign(campaign.id) return {"ok": True} - @router.get("/api/campaigns/{campaign_id}/export", dependencies=[Depends(_make_campaign_auth("campaigns"))]) + @router.get( + "/api/campaigns/{campaign_id}/export", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) async def export_campaign(campaign_id: str): cs = _get_store() campaign = _resolve(cs, campaign_id) @@ -328,7 +445,10 @@ async def export_campaign(campaign_id: str): _enrich_export_with_claims(tree, cs, campaign.id) return tree - @router.post("/api/campaigns/{campaign_id}/join", dependencies=[Depends(_make_campaign_auth("campaigns"))]) + @router.post( + "/api/campaigns/{campaign_id}/join", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) async def join_campaign(campaign_id: str, request: Request): cs = _get_store() campaign = _resolve(cs, campaign_id) @@ -340,14 +460,20 @@ async def join_campaign(campaign_id: str, request: Request): cs.add_campaign_participant(campaign.id, instance_id, hostname) return {"ok": True} - @router.get("/api/campaigns/{campaign_id}/participants", dependencies=[Depends(_make_campaign_auth("campaigns"))]) + @router.get( + "/api/campaigns/{campaign_id}/participants", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) async def get_participants(campaign_id: str): cs = _get_store() campaign = _resolve(cs, campaign_id) participants = cs.get_campaign_participants(campaign.id) return {"participants": participants} - @router.post("/api/campaigns/{campaign_id}/items/{item_id}/claim", dependencies=[Depends(_make_campaign_auth("campaigns"))]) + @router.post( + "/api/campaigns/{campaign_id}/items/{item_id}/claim", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) async def claim_item(campaign_id: str, item_id: str, request: Request): cs = _get_store() _resolve(cs, campaign_id) @@ -361,14 +487,20 @@ async def claim_item(campaign_id: str, item_id: str, request: Request): raise HTTPException(status_code=409, detail="Item already claimed by another node") return {"ok": True} - @router.post("/api/campaigns/{campaign_id}/items/{item_id}/unclaim", dependencies=[Depends(_make_campaign_auth("campaigns"))]) + @router.post( + "/api/campaigns/{campaign_id}/items/{item_id}/unclaim", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) async def unclaim_item(campaign_id: str, item_id: str): cs = _get_store() _resolve(cs, campaign_id) cs.unclaim_plan_item(item_id) return {"ok": True} - @router.post("/api/campaigns/{campaign_id}/items/{item_id}/status", dependencies=[Depends(_make_campaign_auth("campaigns"))]) + @router.post( + "/api/campaigns/{campaign_id}/items/{item_id}/status", + dependencies=[Depends(_make_campaign_auth("campaigns"))], + ) async def update_item_status(campaign_id: str, item_id: str, request: Request): cs = _get_store() _resolve(cs, campaign_id) @@ -380,7 +512,7 @@ async def update_item_status(campaign_id: str, item_id: str, request: Request): try: item_status = PlanItemStatus(status_str) except ValueError: - raise HTTPException(status_code=400, detail=f"Invalid status: {status_str}") + raise HTTPException(status_code=400, detail=f"Invalid status: {status_str}") from None cs.update_plan_item(item_id, status=item_status, outcome=outcome) return {"ok": True} @@ -388,7 +520,7 @@ async def update_item_status(campaign_id: str, item_id: str, request: Request): # Helpers # ------------------------------------------------------------------ - def _build_campaign_tree(cs, campaign_id: str) -> Optional[Dict]: + def _build_campaign_tree(cs, campaign_id: str) -> dict | None: """Recursively build campaign tree with plan items and status.""" campaign = cs.get_campaign(campaign_id) if not campaign: @@ -410,13 +542,10 @@ def _build_campaign_tree(cs, campaign_id: str) -> Optional[Dict]: "in_progress": status["in_progress"], "planned": status["planned"], }, - "children": [ - _build_campaign_tree(cs, child.id) - for child in children - ], + "children": [_build_campaign_tree(cs, child.id) for child in children], } - def _enrich_export_with_claims(tree: Dict, cs, campaign_id: str): + def _enrich_export_with_claims(tree: dict, cs, campaign_id: str): """Walk a serialized campaign tree and annotate items with IDs and claim info.""" items = cs.get_plan_items(campaign_id=campaign_id) items.sort(key=lambda x: x.phase_order) diff --git a/gently/ui/web/routes/chat.py b/gently/ui/web/routes/chat.py index 12833b13..e07d4501 100644 --- a/gently/ui/web/routes/chat.py +++ b/gently/ui/web/routes/chat.py @@ -14,15 +14,18 @@ import logging from datetime import datetime from pathlib import Path -from typing import Optional -from fastapi import APIRouter, HTTPException +from fastapi import APIRouter, Depends, HTTPException from fastapi.responses import StreamingResponse from pydantic import BaseModel +from gently.settings import settings +from gently.ui.web.auth import require_control + logger = logging.getLogger(__name__) -CHAT_MODEL = "claude-opus-4-7" +# Per-timepoint VLM chat → perception tier (Opus 4.8); centralized, not hardcoded. +CHAT_MODEL = settings.models.perception SYSTEM_PROMPT = ( "You are helping a biologist interpret a microscopy perception " "assessment of a C. elegans embryo at a specific timepoint. You can " @@ -43,21 +46,21 @@ class ChatRequest(BaseModel): message: str -def _resolve_session_dir(server, sid: str) -> Optional[Path]: +def _resolve_session_dir(server, sid: str) -> Path | None: store = getattr(server, "gently_store", None) if store is None: return None return store._session_dir(sid) -def _trace_path(server, sid: str, eid: str, tp: int) -> Optional[Path]: +def _trace_path(server, sid: str, eid: str, tp: int) -> Path | None: sd = _resolve_session_dir(server, sid) if sd is None: return None return sd / "embryos" / eid / "traces" / f"t{tp:04d}.json" -def _chat_path(server, sid: str, eid: str, tp: int) -> Optional[Path]: +def _chat_path(server, sid: str, eid: str, tp: int) -> Path | None: sd = _resolve_session_dir(server, sid) if sd is None: return None @@ -68,7 +71,7 @@ def _load_history(path: Path) -> list[dict]: if not path.exists(): return [] turns: list[dict] = [] - with open(path, "r", encoding="utf-8") as f: + with open(path, encoding="utf-8") as f: for line in f: line = line.strip() if not line: @@ -106,7 +109,13 @@ async def get_chat(sid: str, eid: str, tp: int): return {"turns": _load_history(path)} @router.post("/api/perception/chat/{sid}/{eid}/{tp}") - async def post_chat(sid: str, eid: str, tp: int, body: ChatRequest): + async def post_chat( + sid: str, + eid: str, + tp: int, + body: ChatRequest, + _control=Depends(require_control), # noqa: B008 + ): """Append a user message and stream the assistant reply as SSE. Each SSE event is JSON: ``{"type": "delta", "text": "..."}`` for @@ -126,7 +135,7 @@ async def post_chat(sid: str, eid: str, tp: int, body: ChatRequest): detail=f"No perception trace for T{tp}", ) - with open(trace_path, "r", encoding="utf-8") as f: + with open(trace_path, encoding="utf-8") as f: trace = json.load(f) stage = trace.get("predicted_stage", "unknown") reasoning = trace.get("reasoning", "") @@ -164,9 +173,7 @@ async def post_chat(sid: str, eid: str, tp: int, body: ChatRequest): } seed_assistant = { "role": "assistant", - "content": [ - {"type": "text", "text": f"Stage: {stage}\n\n{reasoning}"} - ], + "content": [{"type": "text", "text": f"Stage: {stage}\n\n{reasoning}"}], } messages: list[dict] = [seed_user, seed_assistant] @@ -174,9 +181,7 @@ async def post_chat(sid: str, eid: str, tp: int, body: ChatRequest): role = turn.get("role") content = turn.get("content", "") if role in ("user", "assistant") and content: - messages.append( - {"role": role, "content": [{"type": "text", "text": content}]} - ) + messages.append({"role": role, "content": [{"type": "text", "text": content}]}) messages.append( { "role": "user", diff --git a/gently/ui/web/routes/context.py b/gently/ui/web/routes/context.py new file mode 100644 index 00000000..66c7f25c --- /dev/null +++ b/gently/ui/web/routes/context.py @@ -0,0 +1,74 @@ +"""Context (shared-visibility) routes. + +Exposes the agent's "mind" — its open questions (uncertainty), active +watchpoints (attention), and pending expectations (beliefs) — read by anyone, +resolvable only by the control holder. Live updates ride the CONTEXT_UPDATED +event the FileContextStore emits on the global bus, which the server already +broadcasts to /ws; the client just re-fetches /api/context on it (no polling). +""" + +from fastapi import APIRouter, Body, Depends + +from gently.ui.web.auth import require_control + +from .campaigns import _serialize + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _store(): + # Defensive: the store is wired after construction; tolerate cold start. + return getattr(server, "context_store", None) + + @router.get("/api/context") + async def get_context(): + cs = _store() + empty = {"available": False, "expectations": [], "watchpoints": [], "questions": []} + if cs is None: + return empty + try: + return { + "available": True, + "questions": [_serialize(q) for q in cs.get_open_questions()], + "watchpoints": [_serialize(w) for w in cs.get_active_watchpoints()], + "expectations": [_serialize(e) for e in cs.get_pending_expectations()], + } + except Exception: + return empty + + @router.post("/api/context/questions/{q_id}/resolve", dependencies=[Depends(require_control)]) + async def resolve_question(q_id: str, resolution: str = Body("", embed=True)): + cs = _store() + if cs is None: + return {"ok": False, "error": "context store unavailable"} + cs.resolve_question(q_id, resolution or "") + return {"ok": True} + + @router.post( + "/api/context/watchpoints/{wp_id}/resolve", dependencies=[Depends(require_control)] + ) + async def resolve_watchpoint(wp_id: str): + cs = _store() + if cs is None: + return {"ok": False, "error": "context store unavailable"} + cs.resolve_watchpoint(wp_id) + return {"ok": True} + + @router.post( + "/api/context/expectations/{exp_id}/resolve", dependencies=[Depends(require_control)] + ) + async def resolve_expectation(exp_id: str, status: str = Body("confirmed", embed=True)): + cs = _store() + if cs is None: + return {"ok": False, "error": "context store unavailable"} + from gently.harness.memory.model import ExpectationStatus + + try: + st = ExpectationStatus(status) + except ValueError: + st = ExpectationStatus.CONFIRMED + cs.resolve_expectation(exp_id, st) + return {"ok": True} + + return router diff --git a/gently/ui/web/routes/data.py b/gently/ui/web/routes/data.py index 3e66763f..302d1701 100644 --- a/gently/ui/web/routes/data.py +++ b/gently/ui/web/routes/data.py @@ -3,10 +3,11 @@ import logging from datetime import datetime from pathlib import Path -from typing import Optional import yaml -from fastapi import APIRouter, HTTPException +from fastapi import APIRouter, Body, Depends, HTTPException + +from gently.ui.web.auth import require_control logger = logging.getLogger(__name__) @@ -15,6 +16,30 @@ _HARDWARE_CONFIG_PATH = Path(__file__).resolve().parents[4] / "config" / "hardware.yaml" +def _json_safe(obj): + """Make an acquisition result JSON-encodable for FastAPI. + + ``client.acquire_volume``/``acquire_burst`` return the pixel data under + ``volume``/``image`` as numpy arrays (internal callers like the timelapse + orchestrator need them). FastAPI's ``jsonable_encoder`` can't serialize a + raw ndarray — it tries ``dict(arr)`` and blows up — and the web UI only + needs paths + metadata anyway. Replace arrays with a small shape/dtype + hint and coerce numpy scalars to native types; recurse so the burst + ``frames`` list is covered too. + """ + import numpy as np + + if isinstance(obj, np.ndarray): + return {"shape": list(obj.shape), "dtype": str(obj.dtype)} + if isinstance(obj, np.generic): + return obj.item() + if isinstance(obj, dict): + return {k: _json_safe(v) for k, v in obj.items()} + if isinstance(obj, (list, tuple)): + return [_json_safe(v) for v in obj] + return obj + + def create_router(server) -> APIRouter: router = APIRouter() @@ -26,7 +51,7 @@ async def get_status(): "status": "running", "connections": len(server.manager.active_connections), **stats, - "timestamp": datetime.now().isoformat() + "timestamp": datetime.now().isoformat(), } @router.get("/api/device-status") @@ -63,6 +88,142 @@ async def get_device_status(): "microscope": microscope_up, } + def _require_agent_with_experiment(): + """Resolve the live agent from the server bridge, or 503. + + Edit endpoints write through ExperimentState so the notify hook fires + EMBRYOS_UPDATE and the Map re-renders without a follow-up fetch. + """ + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + if agent is None or not hasattr(agent, "experiment"): + raise HTTPException(status_code=503, detail="Agent not ready") + return agent + + @router.put("/api/embryos/{embryo_id}/position", dependencies=[Depends(require_control)]) + async def update_embryo_position( + embryo_id: str, + body: dict = Body(...), # noqa: B008 + ): + """Update an embryo's coarse XY position. + + Map-side edits write to the coarse stage and CLEAR any prior fine + position — the operator is overriding the sighting, so any + SPIM-objective fine alignment derived from the old coarse is no + longer trustworthy and must be re-run. + + Publishes OPERATOR_EDITED_EMBRYO with both the old and new + positions so candidates can reason about the magnitude of the + correction and trigger re-calibration suggestions. + """ + agent = _require_agent_with_experiment() + emb = agent.experiment.embryos.get(embryo_id) + if emb is None: + raise HTTPException(status_code=404, detail=f"Embryo {embryo_id} not found") + try: + x = float(body.get("x")) # type: ignore[arg-type] # None -> TypeError caught below + y = float(body.get("y")) # type: ignore[arg-type] + except (TypeError, ValueError): + raise HTTPException(status_code=400, detail="Body needs numeric x and y") from None + old_coarse = dict(emb.position_coarse) if emb.position_coarse else None + had_fine = bool(emb.position_fine) + emb.position_coarse = {"x": x, "y": y} + emb.position_fine = {} + agent.experiment.notify_embryos_changed() + + bus = getattr(agent, "_event_bus", None) + if bus is not None: + from gently.core.event_bus import EventType + + try: + bus.publish( + event_type=EventType.OPERATOR_EDITED_EMBRYO, + data={ + "embryo_id": embryo_id, + "old_position_coarse": old_coarse, + "new_position_coarse": {"x": x, "y": y}, + "fine_position_invalidated": had_fine, + }, + source="web:map-edit", + ) + except Exception: + logger.exception("Failed to publish OPERATOR_EDITED_EMBRYO") + return emb.to_dict() + + @router.delete("/api/embryos/{embryo_id}", dependencies=[Depends(require_control)]) + async def delete_embryo(embryo_id: str): + """Remove an embryo from the experiment. + + Goes through ExperimentState.remove_embryo so the observer hook + fires EMBRYOS_UPDATE automatically. Also publishes + OPERATOR_REMOVED_EMBRYO carrying the embryo's last known position + — candidates can use that to e.g. clean up associated cache or + log the deletion in their own world model. + """ + agent = _require_agent_with_experiment() + emb = agent.experiment.embryos.get(embryo_id) + last_position = None + if emb is not None: + last_position = { + "coarse": dict(emb.position_coarse) if emb.position_coarse else None, + "fine": dict(emb.position_fine) if emb.position_fine else None, + } + if not agent.experiment.remove_embryo(embryo_id): + raise HTTPException(status_code=404, detail=f"Embryo {embryo_id} not found") + + # Also drop it from the session files, or a false positive deleted here + # would reappear on the next restart (embryos are reloaded from disk on + # resume). Best-effort — the in-memory removal already succeeded. + store = getattr(agent, "store", None) + sid = getattr(agent, "session_id", None) + if store is not None and sid and hasattr(store, "delete_embryo"): + try: + store.delete_embryo(sid, embryo_id) + except Exception: + logger.exception("Failed to delete embryo %s from session files", embryo_id) + + bus = getattr(agent, "_event_bus", None) + if bus is not None: + from gently.core.event_bus import EventType + + try: + bus.publish( + event_type=EventType.OPERATOR_REMOVED_EMBRYO, + data={ + "embryo_id": embryo_id, + "last_position": last_position, + }, + source="web:map-delete", + ) + except Exception: + logger.exception("Failed to publish OPERATOR_REMOVED_EMBRYO") + return {"ok": True, "embryo_id": embryo_id} + + @router.get("/api/embryos/current") + async def get_current_embryos(): + """Return the agent's current embryo list as an EMBRYOS_UPDATE payload. + + EMBRYOS_UPDATE is published only on mutation, so a Map page opened + mid-session would otherwise see an empty embryo layer until the next + add/remove/edit. This endpoint serves the same payload shape as the + event so clients can bootstrap and then switch to the live stream. + """ + empty = {"embryos": [], "count": 0, "session_id": None} + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + if agent is None or not hasattr(agent, "experiment"): + return empty + try: + embryos = [e.to_dict() for e in agent.experiment.embryos.values()] + except Exception: + logger.exception("Failed to serialise embryos for snapshot") + return empty + return { + "embryos": embryos, + "count": len(embryos), + "session_id": getattr(agent, "session_id", None), + } + @router.get("/api/devices/coverslip") async def get_coverslip(): """Return the coverslip outline metadata for the Map view. @@ -74,17 +235,61 @@ async def get_coverslip(): block in this config and no zone endpoint here. """ try: - with open(_HARDWARE_CONFIG_PATH, "r") as f: + with open(_HARDWARE_CONFIG_PATH) as f: cfg = yaml.safe_load(f) or {} except FileNotFoundError: return {"coverslip": None} cs = cfg.get("coverslip") if not isinstance(cs, dict): return {"coverslip": None} - return {"coverslip": { - "center_um": list(cs.get("center_um") or [0.0, 0.0]), - "size_mm": list(cs.get("size_mm") or [50.0, 24.0]), - }} + return { + "coverslip": { + "center_um": list(cs.get("center_um") or [0.0, 0.0]), + "size_mm": list(cs.get("size_mm") or [50.0, 24.0]), + } + } + + @router.get("/api/devices/scan_geometry") + async def get_scan_geometry(): + """Return the most recent scan geometry for the 3D optical-space view. + + SCAN_GEOMETRY_UPDATE is published only when a volume is acquired, so a + page opened before the first acquisition would have no cuboid to draw. + This serves the last emitted payload (stashed on the agent by + acquisition_tools._publish_scan_geometry), or nominal defaults so the + scene is never empty. + """ + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + last = getattr(agent, "last_scan_geometry", None) if agent else None + if isinstance(last, dict): + return last + # Nominal defaults (calibration defaults; no acquisition yet). + num_slices = 50 + piezo_amplitude = 25.0 + piezo_center = 50.0 + z_extent = 2.0 * piezo_amplitude + return { + "embryo_id": None, + "stage_position_um": {"x": None, "y": None}, + "scan": { + "num_slices": num_slices, + "exposure_ms": 10.0, + "galvo_amplitude_deg": 0.5, + "galvo_center_deg": 0.0, + "piezo_amplitude_um": piezo_amplitude, + "piezo_center_um": piezo_center, + }, + "derived": { + "z_extent_um": z_extent, + "slice_spacing_um": z_extent / (num_slices - 1), + "z_min_um": piezo_center - piezo_amplitude, + "z_max_um": piezo_center + piezo_amplitude, + }, + "mode": "sheet", + "ts": None, + "is_default": True, + } @router.get("/api/devices/bottom_camera/status") async def get_bottom_camera_status(): @@ -98,7 +303,10 @@ async def get_bottom_camera_status(): "last_frame_ts": getattr(monitor, "_last_frame_ts", None) if monitor else None, } - @router.post("/api/devices/bottom_camera/stream/start") + @router.post( + "/api/devices/bottom_camera/stream/start", + dependencies=[Depends(require_control)], + ) async def start_bottom_camera_stream(): """Start the bottom-camera stream bridge. @@ -116,10 +324,13 @@ async def start_bottom_camera_stream(): await monitor.start() except Exception as exc: logger.exception("Failed to start bottom-camera monitor") - raise HTTPException(status_code=500, detail=f"start failed: {exc}") + raise HTTPException(status_code=500, detail=f"start failed: {exc}") from exc return {"streaming": monitor.running} - @router.post("/api/devices/bottom_camera/stream/stop") + @router.post( + "/api/devices/bottom_camera/stream/stop", + dependencies=[Depends(require_control)], + ) async def stop_bottom_camera_stream(): """Stop the bottom-camera stream bridge. Idempotent.""" bridge = getattr(server, "agent_bridge", None) @@ -131,42 +342,1376 @@ async def stop_bottom_camera_stream(): await monitor.stop() except Exception as exc: logger.exception("Failed to stop bottom-camera monitor") - raise HTTPException(status_code=500, detail=f"stop failed: {exc}") + raise HTTPException(status_code=500, detail=f"stop failed: {exc}") from exc return {"streaming": False} + def _resolve_client(): + """Resolve the live microscope client from the agent bridge, or None.""" + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + return getattr(agent, "client", None) if agent else None + + @router.get("/api/devices/room_light/status") + async def get_room_light_status(): + """Cached on/off state of the room-light SwitchBot (cheap to poll).""" + client = _resolve_client() + if client is None: + return {"available": False, "state": "unknown"} + try: + res = await client.get_room_light_status() + except Exception as exc: + logger.debug("room light status fetch failed: %s", exc) + return {"available": False, "state": "unknown"} + return { + "available": bool(res.get("available", res.get("success", False))), + "state": res.get("state", "unknown"), + } + + @router.post("/api/devices/room_light/set", dependencies=[Depends(require_control)]) + async def set_room_light(payload: dict = Body(...)): # noqa: B008 + """Switch the room light on/off. Body: {"state": "on"|"off"|"press"}.""" + state = str(payload.get("state", "")).lower() + if state not in ("on", "off", "press"): + raise HTTPException(status_code=400, detail="state must be on, off, or press") + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + res = await client.set_room_light(state) + except Exception as exc: + logger.exception("Room light command failed") + raise HTTPException( + status_code=502, detail=f"room light command failed: {exc}" + ) from exc + if not res.get("success"): + raise HTTPException( + status_code=502, detail=res.get("error", "room light command failed") + ) + return {"state": res.get("state", state)} + + @router.get("/api/devices/temperature/status") + async def get_temperature_status(): + """Live water temperature, setpoint, and lock state (cheap to poll). + + Cached at the device layer (no per-call hardware round trip), so the + Devices header can poll it like the room light. ``available`` is false + when no controller is configured/connected, which hides the control. + """ + client = _resolve_client() + if client is None: + return {"available": False, "state": "unknown"} + try: + res = await client.get_temperature() + except Exception as exc: + logger.debug("temperature status fetch failed: %s", exc) + return {"available": False, "state": "unknown"} + return { + "available": bool(res.get("success", False)), + "temperature_c": res.get("temperature_c"), + "setpoint_c": res.get("setpoint_c"), + "state": res.get("state", "unknown"), + "peltier_c": res.get("peltier_c"), + } + + @router.post("/api/devices/temperature/set", dependencies=[Depends(require_control)]) + async def set_temperature(payload: dict = Body(...)): # noqa: B008 + """Command the temperature setpoint. Body: {"target_c": float}. + + Non-blocking: the controller ramps and the status poll reflects progress + (and the SYSTEM LOCKED state once it stabilizes). + """ + try: + target = float(payload.get("target_c")) # type: ignore[arg-type] # None -> caught + except (TypeError, ValueError): + raise HTTPException(status_code=400, detail="target_c must be a number") from None + if not (0.0 <= target <= 99.9): + raise HTTPException(status_code=400, detail="target_c must be between 0.0 and 99.9 C") + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + res = await client.set_temperature(target) + except Exception as exc: + logger.exception("Temperature command failed") + raise HTTPException( + status_code=502, detail=f"temperature command failed: {exc}" + ) from exc + if not res.get("success"): + raise HTTPException( + status_code=502, detail=res.get("error", "temperature command failed") + ) + return { + "target_c": res.get("target_c", target), + "temperature_c": res.get("temperature_c"), + "state": res.get("state", "unknown"), + "waited": res.get("waited", False), + } + + @router.get("/api/devices/temperature/config") + async def get_temperature_config(): + """Thermalizer connection config (password redacted) + live backend/state + for the Settings panel. Read-only, so no control elevation required.""" + client = _resolve_client() + if client is None: + return {"available": False} + try: + res = await client.get_temperature_config() + except Exception as exc: + logger.debug("temperature config fetch failed: %s", exc) + return {"available": False} + return {"available": bool(res.get("success", False)), **res} + + @router.post("/api/devices/temperature/config/test", dependencies=[Depends(require_control)]) + async def test_temperature_config(payload: dict = Body(...)): # noqa: B008 + """Probe a candidate thermalizer config without committing it.""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + res = await client.test_temperature_config(payload) + except Exception as exc: + logger.exception("Thermalizer test failed") + raise HTTPException(status_code=502, detail=f"thermalizer test failed: {exc}") from exc + return res + + @router.post("/api/devices/temperature/config", dependencies=[Depends(require_control)]) + async def set_temperature_config(payload: dict = Body(...)): # noqa: B008 + """Reconfigure the thermalizer (serial/mqtt/mock). Live hot-swap where + possible; otherwise persisted for the next device-layer restart.""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + res = await client.set_temperature_config(payload) + except Exception as exc: + logger.exception("Thermalizer reconfigure failed") + raise HTTPException( + status_code=502, detail=f"thermalizer reconfigure failed: {exc}" + ) from exc + return res + + @router.get("/api/config/effective") + async def get_effective_config(): + """Read-only view of the effective server config, secrets redacted. + + settings.py values are frozen at import (resolved from env vars once), so + changing them requires a process restart — surfaced here for visibility, + not editing. Secrets are shown only as present/absent booleans. + """ + import os + + from gently.settings import settings as S + + return { + "note": "settings.py values are read from env at startup; " + "changing them needs a restart.", + "network": { + "viz_host": S.network.viz_host, + "viz_port": S.network.viz_port, + "device_host": S.network.device_host, + "device_port": S.network.device_port, + "mesh_port": S.network.mesh_port, + }, + "models": { + "main": S.models.main, + "perception": S.models.perception, + "fast": S.models.fast, + "medium": S.models.medium, + "refusal_fallback": S.models.refusal_fallback, + }, + "storage": {"base_path": str(S.storage.base_path)}, + "timeouts": { + "plan_execution": S.timeouts.plan_execution, + "volume_acquisition": S.timeouts.volume_acquisition, + "api_call": S.timeouts.api_call, + }, + "ml": { + "default_batch_size": S.ml.default_batch_size, + "default_epochs": S.ml.default_epochs, + "default_lr": S.ml.default_lr, + "model_cache_dir": str(S.ml.model_cache_dir), + }, + "transfer": { + "transfer_port": S.transfer.transfer_port, + "chunk_size": S.transfer.chunk_size, + "max_concurrent_transfers": S.transfer.max_concurrent_transfers, + }, + "mesh": { + "broadcast_interval_s": S.mesh.broadcast_interval_s, + "stale_threshold_s": S.mesh.stale_threshold_s, + "dead_threshold_s": S.mesh.dead_threshold_s, + }, + "ui": {"ux_v2": S.ui.ux_v2}, + "api": {"ncbi_tool": S.api.ncbi_tool}, + "secrets_present": { + "anthropic_api_key": bool(os.getenv("ANTHROPIC_API_KEY")), + "control_token": bool(os.getenv("GENTLY_CONTROL_TOKEN")), + }, + } + + # --- Rig-wide dashboard-preference defaults (layered UNDER per-browser localStorage) --- + _dashboard_defaults_path = _HARDWARE_CONFIG_PATH.parent / "dashboard_defaults.json" + + @router.get("/api/config/dashboard-defaults") + async def get_dashboard_defaults(): + """Rig-wide dashboard-pref defaults (JSON). The browser layers localStorage + over these, so a fresh browser inherits the rig's defaults.""" + import json + + if not _dashboard_defaults_path.exists(): + return {} + try: + return json.loads(_dashboard_defaults_path.read_text()) + except Exception: + return {} + + @router.put("/api/config/dashboard-defaults", dependencies=[Depends(require_control)]) + async def put_dashboard_defaults(payload: dict = Body(...)): # noqa: B008 + """Save the current dashboard prefs as the rig-wide defaults.""" + import json + + if not isinstance(payload, dict): + raise HTTPException(status_code=400, detail="body must be an object") + _dashboard_defaults_path.write_text(json.dumps(payload, indent=2)) + return {"saved": True} + + # --- Restart-required settings.py editors (persisted to config/settings.local.yml) --- + # Allowlist of knobs that are ACTUALLY consumed by the runtime (verified live + # readers) + grouped for the UI. Never expose ports/hosts/model-IDs/storage/ + # secrets. Deliberately omitted: timeouts.rpc_call (removed — RPyC-era dead), + # timeouts.plan_execution and ml.* defaults (currently 0 readers — editing + # would be a silent no-op). + _override_keys = [ + { + "env": "GENTLY_TIMEOUT_VOLUME", + "label": "Volume acquisition (s)", + "type": "int", + "group": "Timeouts", + "get": lambda S: S.timeouts.volume_acquisition, + }, + { + "env": "GENTLY_TIMEOUT_API", + "label": "External API call (s)", + "type": "int", + "group": "Timeouts", + "get": lambda S: S.timeouts.api_call, + }, + { + "env": "GENTLY_MESH_BROADCAST_INTERVAL", + "label": "Broadcast interval (s)", + "type": "float", + "group": "Mesh network", + "get": lambda S: S.mesh.broadcast_interval_s, + }, + { + "env": "GENTLY_MESH_STALE_THRESHOLD", + "label": "Stale threshold (s)", + "type": "float", + "group": "Mesh network", + "get": lambda S: S.mesh.stale_threshold_s, + }, + { + "env": "GENTLY_MESH_DEAD_THRESHOLD", + "label": "Dead threshold (s)", + "type": "float", + "group": "Mesh network", + "get": lambda S: S.mesh.dead_threshold_s, + }, + { + "env": "GENTLY_UX_V2", + "label": "UX v2 dashboard", + "type": "bool", + "group": "Interface", + "get": lambda S: S.ui.ux_v2, + }, + { + "env": "GENTLY_NCBI_TOOL", + "label": "Tool name", + "type": "str", + "group": "NCBI (Entrez)", + "get": lambda S: S.api.ncbi_tool, + }, + { + "env": "GENTLY_NCBI_EMAIL", + "label": "Contact email", + "type": "str", + "group": "NCBI (Entrez)", + "get": lambda S: S.api.ncbi_email, + }, + ] + _settings_local_path = _HARDWARE_CONFIG_PATH.parent / "settings.local.yml" + + def _coerce_override(typ, val): + if typ == "int": + return int(val) + if typ == "float": + return float(val) + if typ == "bool": + return val if isinstance(val, bool) else str(val).lower() in ("1", "true", "yes", "on") + return str(val) + + def _read_settings_local(): + if not _settings_local_path.exists(): + return {} + try: + return yaml.safe_load(_settings_local_path.read_text()) or {} + except Exception: + return {} + + @router.get("/api/config/settings-overrides") + async def get_settings_overrides(): + """Editable (restart-required) settings.py knobs: current effective value + + whether an override file entry exists.""" + from gently.settings import settings as S + + file_over = _read_settings_local() + items = [ + { + "env": k["env"], + "label": k["label"], + "type": k["type"], + "group": k["group"], + "current": k["get"](S), + "overridden": k["env"] in file_over, + } + for k in _override_keys + ] + return {"note": "changes take effect on the next process restart", "items": items} + + @router.put("/api/config/settings-overrides", dependencies=[Depends(require_control)]) + async def put_settings_overrides(payload: dict = Body(...)): # noqa: B008 + """Persist restart-required overrides to config/settings.local.yml. Only + allowlisted keys; never mutates the frozen settings singleton live.""" + allowed = {k["env"]: k["type"] for k in _override_keys} + updates = {} + for k, v in (payload or {}).items(): + if k not in allowed: + raise HTTPException(status_code=400, detail=f"unknown or non-editable key: {k}") + if v is None or v == "": + continue + try: + updates[k] = _coerce_override(allowed[k], v) + except (TypeError, ValueError): + raise HTTPException( + status_code=400, detail=f"{k}: invalid {allowed[k]} value" + ) from None + existing = _read_settings_local() + existing.update(updates) + _settings_local_path.write_text( + yaml.safe_dump(existing, default_flow_style=False, sort_keys=True) + ) + return { + "saved": list(updates.keys()), + "restart_required": True, + "note": "restart the server for these to take effect", + } + + # ------------------------------------------------------------------ + # Lightsheet live stream + # ------------------------------------------------------------------ + + @router.get("/api/devices/lightsheet/live/status") + async def get_lightsheet_live_status(): + """Return whether the lightsheet live stream bridge is running.""" + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + monitor = getattr(agent, "lightsheet_monitor", None) if agent else None + return { + "available": monitor is not None, + "streaming": bool(monitor and monitor.running), + "last_frame_ts": getattr(monitor, "_last_frame_ts", None) if monitor else None, + } + + @router.post( + "/api/devices/lightsheet/live/start", + dependencies=[Depends(require_control)], + ) + async def start_lightsheet_live_stream(): + """Start the lightsheet live stream bridge. + + Idempotent — calling start() while already running is a no-op. + """ + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + monitor = getattr(agent, "lightsheet_monitor", None) if agent else None + if monitor is None: + raise HTTPException( + status_code=503, + detail="Lightsheet monitor not initialised (agent or microscope not ready)", + ) + try: + await monitor.start() + except Exception as exc: + logger.exception("Failed to start lightsheet monitor") + raise HTTPException(status_code=500, detail=f"start failed: {exc}") from exc + return {"streaming": monitor.running} + + @router.post( + "/api/devices/lightsheet/live/stop", + dependencies=[Depends(require_control)], + ) + async def stop_lightsheet_live_stream(): + """Stop the lightsheet live stream bridge. Idempotent.""" + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + monitor = getattr(agent, "lightsheet_monitor", None) if agent else None + if monitor is None: + return {"streaming": False} + try: + await monitor.stop() + except Exception as exc: + logger.exception("Failed to stop lightsheet monitor") + raise HTTPException(status_code=500, detail=f"stop failed: {exc}") from exc + return {"streaming": False} + + # ------------------------------------------------------------------ + # Lightsheet live params + # ------------------------------------------------------------------ + + @router.post("/api/devices/lightsheet/live/params", dependencies=[Depends(require_control)]) + async def lightsheet_live_params(payload: dict = Body(...)): # noqa: B008 + """Forward galvo/piezo/exposure/side params to the device-layer lightsheet streamer.""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + res = await client.set_lightsheet_live_params( + galvo=payload.get("galvo"), + piezo=payload.get("piezo"), + exposure=payload.get("exposure"), + side=payload.get("side"), + ) + except Exception as exc: + logger.exception("lightsheet live params failed") + raise HTTPException(status_code=502, detail=f"params failed: {exc}") from exc + return res + + # ------------------------------------------------------------------ + # LED / laser / camera + # ------------------------------------------------------------------ + + @router.post("/api/devices/led/set", dependencies=[Depends(require_control)]) + async def led_set(payload: dict = Body(...)): # noqa: B008 + """Set the LED shutter state. Body: {"state": "Open"|"Closed"}.""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_led(str(payload.get("state", "Closed"))) + except Exception as exc: + logger.exception("LED set command failed") + raise HTTPException(status_code=502, detail=f"led failed: {exc}") from exc + + @router.post("/api/devices/laser/off", dependencies=[Depends(require_control)]) + async def laser_off(): + """Gate ALL laser lines off via the Laser config group "ALL OFF" preset. + + Uses setConfig("Laser", "ALL OFF") which drives the PLogic + OutputChannel to "none of outputs 5-8" — this gates every line + (488, 561, 405, 637) off, not just the 488 nm setpoint. + Required for safe brightfield live-view (spec §2.7). + """ + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_laser_config("ALL OFF") + except Exception as exc: + logger.exception("Laser off command failed") + raise HTTPException(status_code=502, detail=f"laser off failed: {exc}") from exc + + @router.get("/api/devices/laser/configs") + async def laser_configs(): + """Return the available Laser config-group presets from the device layer. + + No require_control — read-only status route, mirrors GET status + routes like room_light/status and temperature/status. + """ + client = _resolve_client() + if client is None or not client.is_connected: + # Device layer offline is an expected state (e.g. UI open without the + # device process) — a quiet 503, not an ERROR traceback per poll. + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.get_laser_configs() + except Exception as exc: + logger.exception("Laser configs fetch failed") + raise HTTPException(status_code=502, detail=f"laser configs failed: {exc}") from exc + + @router.post("/api/devices/laser/config", dependencies=[Depends(require_control)]) + async def laser_config_set(payload: dict = Body(...)): # noqa: B008 + """Apply a named Laser config-group preset (e.g. "ALL OFF", "488 only"). + + Body: {"config": ""} + + Returns the device layer response. 400 if config is missing/empty; + 503 if the microscope is not connected; 502 on device error. + """ + config = payload.get("config") + if not config or not isinstance(config, str): + raise HTTPException(status_code=400, detail="config must be a non-empty string") + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_laser_config(config) + except Exception as exc: + logger.exception("Laser config set command failed") + raise HTTPException(status_code=502, detail=f"laser config failed: {exc}") from exc + + @router.get("/api/devices/cameras") + async def cameras_list(): + """Return the available SPIM camera roles (A always; B if camera_b registered). + + No require_control — read-only status route, mirrors GET /api/devices/laser/configs. + """ + client = _resolve_client() + if client is None or not client.is_connected: + # Device layer offline is expected — quiet 503, no ERROR traceback. + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.get_cameras() + except Exception as exc: + logger.exception("Cameras list fetch failed") + raise HTTPException(status_code=502, detail=f"cameras failed: {exc}") from exc + + @router.post("/api/devices/camera/led_mode", dependencies=[Depends(require_control)]) + async def camera_led_mode(payload: dict = Body(...)): # noqa: B008 + """Enable/disable automatic LED for bottom-camera captures. Body: {"use_led": bool}.""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.set_camera_led_mode(bool(payload.get("use_led", False))) + except Exception as exc: + logger.exception("Camera LED mode command failed") + raise HTTPException(status_code=502, detail=f"camera led mode failed: {exc}") from exc + + # ------------------------------------------------------------------ + # Stage + # ------------------------------------------------------------------ + + @router.post("/api/devices/stage/move", dependencies=[Depends(require_control)]) + async def stage_move(payload: dict = Body(...)): # noqa: B008 + """Move the stage to an absolute XY position. Body: {"x": float, "y": float}.""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.move_to_position(float(payload["x"]), float(payload["y"])) + except KeyError: + raise HTTPException(status_code=400, detail="x and y required") from None + except Exception as exc: + logger.exception("Stage move command failed") + raise HTTPException(status_code=502, detail=f"stage move failed: {exc}") from exc + + def _num(v): + try: + return float(v) + except (TypeError, ValueError): + return None + + def _persist_detection_labels(agent, payload: dict, markers: list) -> bool: + """Persist the annotated bottom-cam frame + per-marker pixel/stage coords + as a labelled snapshot — localization training data (sub-project B). + + Best-effort: requires a client-supplied image and an active session; any + failure (no session, missing deps) is swallowed so it never blocks the + embryo registration. Reuses FileStore.register_snapshot so labels live in + the standard snapshots/ sidecar, not a bespoke store. + """ + image_b64 = payload.get("image_b64") + store = getattr(agent, "store", None) + session_id = getattr(agent, "session_id", None) + if not image_b64 or store is None or not session_id: + return False + try: + import base64 + import io + import tempfile + import uuid as _uuid + + import numpy as np + import tifffile + from PIL import Image + + from gently.core.coordinates import ( + DEFAULT_OBJECTIVE_MAG, + DEFAULT_PIXEL_SIZE_UM, + ) + + raw = base64.b64decode(image_b64.split(",")[-1]) + img = np.asarray(Image.open(io.BytesIO(raw))) + tmp = Path(tempfile.gettempdir()) / f"operate_{_uuid.uuid4().hex[:12]}.tif" + tifffile.imwrite(str(tmp), img) + + frame = payload.get("frame") or {} + pos = payload.get("stage_position") or [None, None] + meta = { + "kind": "operate_marking", + "stage_position": list(pos), + "frame": { + "width": _num(frame.get("w")), + "height": _num(frame.get("h")), + "downsample": _num(frame.get("downsample")), + }, + "transform": { + "pixel_size_um": DEFAULT_PIXEL_SIZE_UM, + "objective_mag": DEFAULT_OBJECTIVE_MAG, + }, + "embryos": [ + { + "pixel_x": _num(m.get("pixel_x")), + "pixel_y": _num(m.get("pixel_y")), + "stage_x_um": _num(m.get("stage_x_um")), + "stage_y_um": _num(m.get("stage_y_um")), + "source": m.get("source", "manual"), + } + for m in markers + ], + } + store.register_snapshot(session_id, "operate_marked", tmp, metadata=meta) + return True + except Exception: + logger.debug("detection-label persistence skipped", exc_info=True) + return False + + @router.post("/api/devices/detect_embryos", dependencies=[Depends(require_control)]) + async def detect_embryos(payload: dict = Body(default={})): # noqa: B008 + """Run bottom-camera embryo detection and RETURN the candidates for the + Operate-view marking canvas. Does NOT register — the operator confirms + (add/remove/relocate) on a frozen frame, then POSTs /api/devices/embryos/ + confirm. This keeps the human in the loop and the canonical embryo list + clean (a marking step, not a blind auto-register). + + Body (all optional): {exposure_ms, min_confidence, brightness_percentile, + min_area, max_area, use_claude_review, use_last_frame}. Claude review + defaults OFF. use_last_frame detects on the last streamed frame (if any) + instead of capturing a fresh image. + + Returns: {success, count, stage_position: [x, y] | null, + embryos: [{embryo_id, pixel_x, pixel_y, stage_x_um, stage_y_um, + confidence, area_pixels, bbox_pixel}]}. + """ + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + if not getattr(client, "has_sam", False): + raise HTTPException( + status_code=503, detail="SAM detection not available on device layer" + ) + + kw: dict = { + "use_claude_review": bool(payload.get("use_claude_review", False)), + "use_last_frame": bool(payload.get("use_last_frame", False)), + "capture_only": bool(payload.get("capture_only", False)), + } + for key, cast in ( + ("exposure_ms", float), + ("min_confidence", float), + ("brightness_percentile", float), + ("min_area", int), + ("max_area", int), + ): + if payload.get(key) is not None: + kw[key] = cast(payload[key]) + try: + result = await client.detect_embryos(**kw) + except Exception as exc: + logger.exception("Embryo detection failed") + raise HTTPException(status_code=502, detail=f"detection failed: {exc}") from exc + + if not result.get("success"): + raise HTTPException( + status_code=502, detail=str(result.get("error", "detection failed")) + ) + + embryos = [] + for emb in result.get("embryos", []) or []: + bbox = emb.get("bbox_pixel") + embryos.append( + { + "embryo_id": emb.get("embryo_id"), + "pixel_x": _num(emb.get("pixel_x")), + "pixel_y": _num(emb.get("pixel_y")), + "stage_x_um": _num(emb.get("stage_x_um")), + "stage_y_um": _num(emb.get("stage_y_um")), + "confidence": _num(emb.get("confidence")), + "area_pixels": emb.get("area_pixels"), + "bbox_pixel": list(bbox) if bbox is not None else None, + } + ) + pos = result.get("stage_position") + return { + "success": True, + "count": len(embryos), + "stage_position": list(pos) if pos is not None else None, + "embryos": embryos, + # The JPEG-encoded frame SAM ran on, so the Operate view can display + # the image the candidates came from (esp. a fresh capture). + "frame": result.get("frame"), + } + + @router.post("/api/devices/embryos/confirm", dependencies=[Depends(require_control)]) + async def confirm_embryos(payload: dict = Body(...)): # noqa: B008 + """Register operator-confirmed markers into the canonical embryo list. + + Agent-free commit step for the Operate-view marking canvas. The client + computes each marker's stage XY from the frozen frame (SAM candidates + carry server-computed stage coords; clicked markers are converted with + the frame's downsample-aware transform), so the server just registers + via ExperimentState.add_embryo(role='unassigned'), firing EMBRYOS_UPDATE. + + Body: {markers: [{stage_x_um, stage_y_um, pixel_x?, pixel_y?, source?, + confidence?}], image_b64?, frame? {w,h,downsample}, stage_position?}. + When image_b64 is present the annotated frame + per-marker pixel/stage + coords are persisted as a labelled snapshot (sub-project B: localization + training data) — best-effort, never blocks registration. + Returns {success, registered: [embryo_id, ...], labelled: bool}. + """ + agent = _require_agent_with_experiment() + markers = payload.get("markers") or [] + + def _next_embryo_id(taken: set) -> str: + n = 1 + while f"embryo_{n}" in agent.experiment.embryos or f"embryo_{n}" in taken: + n += 1 + return f"embryo_{n}" + + import uuid + + store = getattr(agent, "store", None) + sid = getattr(agent, "session_id", None) + + registered: list[str] = [] + taken: set = set() + for m in markers: + sx, sy = _num(m.get("stage_x_um")), _num(m.get("stage_y_um")) + if sx is None or sy is None: + continue + emb_id = _next_embryo_id(taken) + taken.add(emb_id) + emb_uid = str(uuid.uuid4()) + agent.experiment.add_embryo( + embryo_id=emb_id, + position={"x": sx, "y": sy}, + confidence=_num(m.get("confidence")) or 0.0, + uid=emb_uid, + role="unassigned", + ) + # Persist to the session files so embryos survive a restart — the + # experiment is otherwise memory-only until a volume is acquired. + # Best-effort: a storage hiccup must never fail the registration. + if store is not None and sid: + try: + store.register_embryo( + session_id=sid, + embryo_id=emb_id, + embryo_uid=emb_uid, + position_coarse={"x": sx, "y": sy}, + role="unassigned", + ) + except Exception: + logger.exception("Failed to persist embryo %s to session files", emb_id) + registered.append(emb_id) + + labelled = _persist_detection_labels(agent, payload, markers) + return {"success": True, "registered": registered, "labelled": labelled} + + @router.post( + "/api/devices/embryos/{embryo_id}/calibrate", + dependencies=[Depends(require_control)], + ) + async def calibrate_embryo_route(embryo_id: str, payload: dict = Body(default={})): # noqa: B008 + """Run piezo-galvo calibration for one embryo — the Operate B-cal step. + + Reuses the agent's proven ``calibrate_embryo`` tool (Claude-vision edge + detection + adaptive focus sweep, which sets the light-sheet laser config + per snap) rather than the bare device-layer plan, so the operate flow + gets the same calibration quality as the agent. No LLM orchestration — + the coroutine is called directly with an agent/client context. The SPIM + head should already be lowered + focused (operate reaches this step only + after B3). Persists the fit onto ``embryo.calibration``. + """ + agent = _require_agent_with_experiment() + if embryo_id not in agent.experiment.embryos: + raise HTTPException(status_code=404, detail=f"unknown embryo {embryo_id}") + client = _resolve_client() + if client is None or not getattr(client, "is_connected", False): + raise HTTPException(status_code=503, detail="Microscope not connected") + + # Ensure the calibration tools are registered on the global registry, + # then run via the registry so context (agent/client) is injected the + # same way the agent invokes it. Calling the @tool wrapper directly would + # drop the positional embryo_id. + import gently.app.tools.calibration_tools # noqa: F401 (registers the tool) + from gently.harness.tools.registry import get_tool_registry + + registry = get_tool_registry() + try: + message = await registry.execute( + "calibrate_embryo", + {"embryo_id": embryo_id}, + {"agent": agent, "client": client}, + ) + except Exception as exc: + logger.exception("Calibration failed for %s", embryo_id) + raise HTTPException(status_code=502, detail=f"calibration failed: {exc}") from exc + if isinstance(message, str) and message.startswith("Error"): + raise HTTPException(status_code=502, detail=message) + + emb = agent.experiment.embryos.get(embryo_id) + calibration = dict(getattr(emb, "calibration", {}) or {}) if emb else {} + agent.experiment.notify_embryos_changed() + return {"success": True, "message": message, "calibration": calibration} + + @router.post("/api/embryos/roles", dependencies=[Depends(require_control)]) + async def set_embryo_roles(payload: dict = Body(...)): # noqa: B008 + """Assign experimental roles to embryos — the Operate "Run" step. + + Body: {roles: {embryo_id: role_name}} where role_name is a key in + gently.harness.roles.REGISTRY (subject=='test', reference=='calibration', + plus 'lineaging'/'unassigned'). Sets EmbryoState.role and fires one + EMBRYOS_UPDATE (+ per-embryo STATUS_CHANGED) so every consumer refreshes. + Marking stays positions-only; roles are assigned here, not at marking. + Load-bearing: expression_monitoring scopes to role=='test', so the marked + set must be given roles before role-scoped monitoring matches anything. + """ + from gently.harness.roles import is_valid_role + + agent = _require_agent_with_experiment() + roles = payload.get("roles") or {} + if not isinstance(roles, dict) or not roles: + raise HTTPException(status_code=400, detail="roles map required") + embryos = agent.experiment.embryos + for eid, role in roles.items(): + if eid not in embryos: + raise HTTPException(status_code=400, detail=f"unknown embryo {eid}") + if not is_valid_role(str(role)): + raise HTTPException(status_code=400, detail=f"invalid role {role}") + + store = getattr(agent, "store", None) + sid = getattr(agent, "session_id", None) + bus = getattr(agent, "_event_bus", None) + updated: list[str] = [] + for eid, role in roles.items(): + emb = embryos[eid] + old = getattr(emb, "role", None) + emb.role = str(role) + if store is not None and sid: + try: + pos = getattr(emb, "position_coarse", {}) or {} + store.register_embryo( + sid, + eid, + position_x=pos.get("x"), + position_y=pos.get("y"), + calibration=getattr(emb, "calibration", {}) or {}, + role=str(role), + ) + except Exception: + logger.debug("role persist failed for %s", eid, exc_info=True) + if bus is not None: + try: + from gently.core.event_bus import EventType + + bus.publish( + event_type=EventType.STATUS_CHANGED, + data={ + "embryo_id": eid, + "change": "role_assigned", + "old_role": old, + "new_role": str(role), + }, + source="operate_roles", + ) + except Exception: + logger.debug("STATUS_CHANGED publish failed", exc_info=True) + updated.append(eid) + try: + agent.experiment.notify_embryos_changed() # fires EMBRYOS_UPDATE + except Exception: + logger.debug("notify_embryos_changed failed", exc_info=True) + return {"success": True, "updated": updated} + + @router.post("/api/operate/run-tactic", dependencies=[Depends(require_control)]) + async def operate_run_tactic(payload: dict = Body(...)): # noqa: B008 + """Append a tactic to the session Operation Plan and execute it via the + Tactic Executor (resolve scope → dispatch by kind to the orchestrator). + + Body: {tactic: {...}} OR {library_id: "..."} (instantiate a saved tactic); + optional {embryo_ids: [...]} to re-scope it to the marked set. + Returns {success, tactic_id, result}. + """ + from gently.app.orchestration.tactic_executor import ( + append_tactic_to_plan, + execute_tactic, + ) + + agent = _require_agent_with_experiment() + tactic = payload.get("tactic") + lib_id = payload.get("library_id") + if tactic is None and lib_id: + cs = getattr(agent, "context_store", None) + if cs is None: + raise HTTPException(status_code=503, detail="No context store") + tactic = cs.apply_tactic(lib_id) + if tactic is None: + raise HTTPException(status_code=404, detail=f"tactic '{lib_id}' not found") + if not isinstance(tactic, dict): + raise HTTPException(status_code=400, detail="tactic or library_id required") + + eids = payload.get("embryo_ids") + if eids: + tactic = dict(tactic) + tactic["scope"] = {"mode": "embryos", "embryo_ids": list(eids)} + + try: + stored = append_tactic_to_plan(agent, tactic) + except ValueError as exc: + raise HTTPException(status_code=400, detail=f"invalid tactic: {exc}") from exc + if stored is None: + raise HTTPException(status_code=503, detail="No session to attach the tactic to") + try: + result = await execute_tactic(agent, stored) + except Exception as exc: + logger.exception("run-tactic execution failed") + raise HTTPException(status_code=502, detail=f"tactic execution failed: {exc}") from exc + return {"success": bool(result.get("ok")), "tactic_id": stored.get("id"), "result": result} + + @router.get("/api/operation_plan") + async def get_operation_plan_route(): + """Current session's Operation Plan (the tactics document), for the + Operate run-spine. Returns {plan: {...}|null}. Never errors.""" + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + cs = getattr(agent, "context_store", None) if agent else None + sid = getattr(agent, "session_id", None) if agent else None + if cs is None or not sid: + return {"plan": None} + try: + return {"plan": cs.get_operation_plan(sid)} + except Exception: + logger.debug("get_operation_plan failed", exc_info=True) + return {"plan": None} + + # ------------------------------------------------------------------ + # Focus Z axes (Operate view) — fenced read + nudge + # ------------------------------------------------------------------ + + @router.get("/api/devices/stage/bottom_z") + async def get_bottom_z(): + """Bottom-camera focus Z position + limits (read-only).""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.get_bottom_z() + except Exception as exc: + logger.debug("bottom_z read failed: %s", exc) + raise HTTPException(status_code=502, detail=f"bottom_z read failed: {exc}") from exc + + @router.post("/api/devices/stage/bottom_z/nudge", dependencies=[Depends(require_control)]) + async def nudge_bottom_z(payload: dict = Body(...)): # noqa: B008 + """Nudge the bottom-camera focus Z by {delta} µm (fenced to its limits).""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + delta = _num(payload.get("delta")) + if delta is None: + raise HTTPException(status_code=400, detail="delta required") + try: + return await client.nudge_bottom_z(delta) + except Exception as exc: + logger.exception("bottom_z nudge failed") + raise HTTPException(status_code=502, detail=f"bottom_z nudge failed: {exc}") from exc + + @router.get("/api/devices/spim/fdrive") + async def get_fdrive(): + """SPIM-head F-drive position + limits + distance-to-floor (read-only).""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + return await client.get_fdrive() + except Exception as exc: + logger.debug("fdrive read failed: %s", exc) + raise HTTPException(status_code=502, detail=f"fdrive read failed: {exc}") from exc + + @router.post("/api/devices/spim/fdrive/nudge", dependencies=[Depends(require_control)]) + async def nudge_fdrive(payload: dict = Body(...)): # noqa: B008 + """Nudge the SPIM-head F-drive by {delta} µm (fenced; never below floor).""" + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + delta = _num(payload.get("delta")) + if delta is None: + raise HTTPException(status_code=400, detail="delta required") + try: + return await client.nudge_fdrive(delta) + except Exception as exc: + logger.exception("fdrive nudge failed") + raise HTTPException(status_code=502, detail=f"fdrive nudge failed: {exc}") from exc + + # ------------------------------------------------------------------ + # Acquisition + # ------------------------------------------------------------------ + + @router.post("/api/devices/acquire/burst", dependencies=[Depends(require_control)]) + async def acquire_burst(payload: dict = Body(...)): # noqa: B008 + """Trigger a burst acquisition. + + Body: {frames, mode, num_slices, exposure_ms, + laser_config?, piezo_center?, galvo_center?}. + laser_config is forwarded directly to the device client so callers + can send "ALL OFF" for brightfield-safe Manual-view captures. + piezo_center and galvo_center capture at the dialled focal plane. + """ + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + kw: dict = {} + if payload.get("laser_config") is not None: + kw["laser_config"] = str(payload["laser_config"]) + if payload.get("piezo_center") is not None: + kw["piezo_center"] = float(payload["piezo_center"]) + if payload.get("galvo_center") is not None: + kw["galvo_center"] = float(payload["galvo_center"]) + result = await client.acquire_burst( + frames=int(payload.get("frames", 60)), + mode=str(payload.get("mode", "1hz")), + num_slices=int(payload.get("num_slices", 1)), + exposure_ms=float(payload.get("exposure_ms", 5.0)), + **kw, + ) + return _json_safe(result) + except Exception as exc: + logger.exception("Burst acquisition failed") + raise HTTPException(status_code=502, detail=f"burst failed: {exc}") from exc + + @router.post("/api/devices/acquire/volume", dependencies=[Depends(require_control)]) + async def acquire_volume(payload: dict = Body(...)): # noqa: B008 + """Trigger a volume acquisition. + + Body: {num_slices, exposure_ms, + laser_config?, piezo_center?, galvo_center?}. + laser_config is forwarded directly to the device client so callers + can send "ALL OFF" for brightfield-safe Manual-view captures. + piezo_center and galvo_center capture at the dialled focal plane. + """ + client = _resolve_client() + if client is None: + raise HTTPException(status_code=503, detail="Microscope not connected") + try: + kw: dict = {} + if payload.get("laser_config") is not None: + kw["laser_config"] = str(payload["laser_config"]) + if payload.get("piezo_center") is not None: + kw["piezo_center"] = float(payload["piezo_center"]) + if payload.get("galvo_center") is not None: + kw["galvo_center"] = float(payload["galvo_center"]) + result = await client.acquire_volume( + num_slices=int(payload.get("num_slices", 50)), + exposure_ms=float(payload.get("exposure_ms", 10.0)), + **kw, + ) + return _json_safe(result) + except Exception as exc: + logger.exception("Volume acquisition failed") + raise HTTPException(status_code=502, detail=f"volume failed: {exc}") from exc + + # ------------------------------------------------------------------ + # Timelapse + # ------------------------------------------------------------------ + + @router.post("/api/devices/timelapse/start", dependencies=[Depends(require_control)]) + async def timelapse_start(payload: dict = Body(...)): # noqa: B008 + """Start an adaptive timelapse from the manual UI. + + Body fields (all optional except interval_seconds has a default): + interval_seconds (float, default 120) — cadence; must be > 0 + stop_condition (str, default "manual") — "manual", "timepoints", "duration" + embryo_ids (list[str] | null) — null = all active embryos + condition_value (int | null) — timepoints count or duration hours + monitoring_mode (str | null) — "idle" / "expression_monitoring" / + "pre_terminal_monitoring" + num_slices (int, default 50) — must be >= 1 if provided + exposure_ms (float, default 10.0) + galvo_amplitude (float, default 0.5) + galvo_center (float, default 0.0) + piezo_amplitude (float, default 25.0) + piezo_center (float, default 50.0) + laser_config (str | null) + + Validation: + - interval_seconds must be > 0 + - num_slices must be >= 1 + + Orchestrator access: server.agent_bridge.agent.timelapse_orchestrator + RIG-DEFERRED: the actual acquisition + galvo/piezo motion. + """ + # --- Validate --- + raw_interval = payload.get("interval_seconds", 120.0) + try: + interval_seconds = float(raw_interval) + except (TypeError, ValueError): + raise HTTPException( # B904 + status_code=400, detail="interval_seconds must be a number" + ) from None + if interval_seconds <= 0: + raise HTTPException(status_code=400, detail="interval_seconds must be > 0") + + raw_slices = payload.get("num_slices") + if raw_slices is not None: + try: + num_slices = int(raw_slices) + except (TypeError, ValueError): + raise HTTPException( # B904 + status_code=400, detail="num_slices must be an integer" + ) from None + if num_slices < 1: + raise HTTPException(status_code=400, detail="num_slices must be >= 1") + else: + num_slices = 50 + + stop_condition = str(payload.get("stop_condition") or "manual") + embryo_ids = payload.get("embryo_ids") or None + condition_value = payload.get("condition_value") + monitoring_mode = payload.get("monitoring_mode") or None + + # Volume geometry — passed through for context / future calibration write; + # not forwarded to orchestrator.start (which owns its own geometry via the + # per-embryo calibration). RIG-DEFERRED: real acquisition uses these. + volume_geometry = { + "num_slices": num_slices, + "exposure_ms": float(payload.get("exposure_ms", 10.0)), + "galvo_amplitude": float(payload.get("galvo_amplitude", 0.5)), + "galvo_center": float(payload.get("galvo_center", 0.0)), + "piezo_amplitude": float(payload.get("piezo_amplitude", 25.0)), + "piezo_center": float(payload.get("piezo_center", 50.0)), + "laser_config": payload.get("laser_config") or None, + } + + # --- Resolve orchestrator --- + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + orchestrator = getattr(agent, "timelapse_orchestrator", None) if agent else None + if orchestrator is None: + raise HTTPException( + status_code=503, + detail="Timelapse orchestrator not initialised (agent not running or no session)", + ) + + # --- Start timelapse (RIG-DEFERRED: real acquisition) --- + # TODO: UI-initiated timelapses skip the agent tool's plan auto-linking; + # this is intentional — the agent path wires the plan, this route does not. + try: + result = await orchestrator.start( + embryo_ids=embryo_ids, + stop_condition=stop_condition, + base_interval_seconds=interval_seconds, + condition_value=condition_value, + ) + except Exception as exc: + logger.exception("Timelapse start failed") + raise HTTPException(status_code=502, detail=f"timelapse start failed: {exc}") from exc + + # Optionally install a monitoring mode at startup (mirrors start_adaptive_timelapse) + mode_result = None + if monitoring_mode and monitoring_mode != "idle": + try: + mode_result = orchestrator.enable_monitoring_mode(monitoring_mode) + except Exception as exc: + mode_result = f"warning: failed to enable monitoring mode: {exc}" + + # Seed the session Operation Plan with a standing_timelapse tactic (+ a + # reactive_monitor when a monitoring mode is active) scoped to the marked + # set. Closes the historical "UI timelapses skip plan linking" gap so the + # Operate run-spine and the Operations tab show a real tactic. Best-effort: + # needs a live session + context store; never blocks the start. + seeded_tactics: list[str] = [] + cs = getattr(agent, "context_store", None) + sid = getattr(agent, "session_id", None) + # Skip seeding when start was a no-op ("already running") — don't append a + # phantom active tactic for a run we didn't start. + already_running = isinstance(result, str) and result.startswith("Timelapse already running") + if cs is not None and sid and not already_running: + try: + import uuid as _uuid + + from gently.app.tools.operation_plan_tools import _validate_tactics + + # Scope mirrors what orchestrator.start actually does: an omitted + # embryo_ids images ALL active embryos → record global, not []. + eids = list(embryo_ids or []) + seed_scope = {"mode": "embryos", "embryo_ids": eids} if eids else {"mode": "global"} + st_id = f"op_{_uuid.uuid4().hex[:8]}" + new_tactics: list[dict] = [ + { + "id": st_id, + "name": "Adaptive timelapse", + "kind": "standing_timelapse", + "state": "active", + "scope": dict(seed_scope), + "structure": { + "cadence_s": interval_seconds, + "interval": interval_seconds, + "stop_condition": stop_condition, + "condition_value": condition_value, + "monitoring_mode": monitoring_mode or "idle", + }, + "rationale": "Started from the Operate Run step.", + "live_bind": ["cadence"], + "relations": {}, + "live": {}, + "source": "operate", + } + ] + if monitoring_mode and monitoring_mode != "idle": + new_tactics.append( + { + "id": f"op_{_uuid.uuid4().hex[:8]}", + "name": "Monitor", + "kind": "reactive_monitor", + "state": "active", + "scope": dict(seed_scope), + "structure": {"monitoring_mode": monitoring_mode, "status": "armed"}, + "rationale": f"{monitoring_mode} on the marked subjects.", + "live_bind": ["signal"], + "relations": {"layered_on": [st_id]}, + "live": {}, + "source": "operate", + } + ) + new_tactics = _validate_tactics(new_tactics) + plan = cs.get_operation_plan(sid) or { + "session_id": sid, + "title": "Operate session", + "goal": "", + "tactics": [], + } + # Reconcile: retire any prior still-'active' operate-seeded tactics + # so repeated Start clicks don't accumulate stale active timelapses. + for t in plan.setdefault("tactics", []): + if t.get("source") == "operate" and t.get("state") == "active": + t["state"] = "done" + plan["tactics"].extend(new_tactics) + plan["updated_reason"] = "operate adaptive timelapse" + cs.set_operation_plan(sid, plan) + seeded_tactics = [t["id"] for t in new_tactics] + # Link the run to its tactics so stop/pause/resume can reconcile them. + try: + orchestrator._operate_tactic_ids = list(seeded_tactics) + except Exception: + pass + except Exception: + logger.debug("timelapse tactic seeding skipped", exc_info=True) + + return { + "started": True, + "result": result, + "tactics": seeded_tactics, + "monitoring_mode_result": mode_result, + "config": { + "interval_seconds": interval_seconds, + "stop_condition": stop_condition, + "embryo_ids": embryo_ids, + "condition_value": condition_value, + "monitoring_mode": monitoring_mode, + "volume_geometry": volume_geometry, + }, + } + + def _resolve_orch_and_agent(): + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + orch = getattr(agent, "timelapse_orchestrator", None) if agent else None + return orch, agent + + def _reconcile_operate_tactics(orch, agent, state: str, clear: bool): + """Transition the run's operate-seeded tactics to ``state`` so the + Operation Plan / run-spine reflect stop/pause/resume. Best-effort.""" + cs = getattr(agent, "context_store", None) + sid = getattr(agent, "session_id", None) + ids = list(getattr(orch, "_operate_tactic_ids", []) or []) + if cs is not None and sid and ids: + for tid in ids: + try: + cs.transition_tactic(sid, tid, state) + except Exception: + logger.debug("transition_tactic %s failed", tid, exc_info=True) + if clear: + try: + orch._operate_tactic_ids = [] + except Exception: + pass + + @router.post("/api/devices/timelapse/stop", dependencies=[Depends(require_control)]) + async def timelapse_stop(payload: dict = Body(default={})): # noqa: B008 + """Stop the running timelapse (Operate run-spine). Body: {reason?}.""" + orch, agent = _resolve_orch_and_agent() + if orch is None: + raise HTTPException(status_code=503, detail="No timelapse orchestrator") + try: + reason = str(payload.get("reason", "user_request")) + res = await orch.stop(reason=reason) + _reconcile_operate_tactics(orch, agent, "done", clear=True) + return {"stopped": True, "result": res} + except Exception as exc: + logger.exception("Timelapse stop failed") + raise HTTPException(status_code=502, detail=f"timelapse stop failed: {exc}") from exc + + @router.post("/api/devices/timelapse/pause", dependencies=[Depends(require_control)]) + async def timelapse_pause(): + """Pause the running timelapse.""" + orch, agent = _resolve_orch_and_agent() + if orch is None: + raise HTTPException(status_code=503, detail="No timelapse orchestrator") + try: + res = await orch.pause() + _reconcile_operate_tactics(orch, agent, "paused", clear=False) + return {"paused": True, "result": res} + except Exception as exc: + logger.exception("Timelapse pause failed") + raise HTTPException(status_code=502, detail=f"timelapse pause failed: {exc}") from exc + + @router.post("/api/devices/timelapse/resume", dependencies=[Depends(require_control)]) + async def timelapse_resume(): + """Resume a paused timelapse.""" + orch, agent = _resolve_orch_and_agent() + if orch is None: + raise HTTPException(status_code=503, detail="No timelapse orchestrator") + try: + res = await orch.resume() + _reconcile_operate_tactics(orch, agent, "active", clear=False) + return {"resumed": True, "result": res} + except Exception as exc: + logger.exception("Timelapse resume failed") + raise HTTPException(status_code=502, detail=f"timelapse resume failed: {exc}") from exc + @router.get("/api/calibration") - async def list_calibration(embryo_id: Optional[str] = None): + async def list_calibration(embryo_id: str | None = None): """Get calibration images""" images = server.store.get_all_calibration(embryo_id) - return { - "calibration": [img.to_dict() for img in images], - "count": len(images) - } + return {"calibration": [img.to_dict() for img in images], "count": len(images)} @router.get("/api/volumes") - async def list_volumes(embryo_id: Optional[str] = None): + async def list_volumes(embryo_id: str | None = None): """Get volume images""" images = server.store.get_all_volumes(embryo_id) - return { - "volumes": [img.to_dict() for img in images], - "count": len(images) - } + return {"volumes": [img.to_dict() for img in images], "count": len(images)} @router.get("/api/snapshots") - async def list_snapshots(embryo_id: Optional[str] = None): + async def list_snapshots(embryo_id: str | None = None): """Get snapshot images""" images = server.store.get_all_snapshots(embryo_id) - return { - "snapshots": [img.to_dict() for img in images], - "count": len(images) - } + return {"snapshots": [img.to_dict() for img in images], "count": len(images)} @router.get("/api/embryos") async def list_embryos(): """Get list of embryos with images""" - return { - "embryos": server.store.get_embryo_ids() - } + return {"embryos": server.store.get_embryo_ids()} @router.get("/api/embryos/positions") async def embryo_positions(): @@ -190,26 +1735,29 @@ async def embryo_positions(): # Embryo registered but no position yet (e.g. only the # ID arrived from another path). Skip — nothing to render. continue - points.append({ - "embryo_id": eid, - "uid": emb.get("uid"), - "x": float(x), - "y": float(y), - "role": emb.get("role", "test"), - "user_label": emb.get("user_label"), - "confidence": emb.get("confidence"), - "cadence_phase": emb.get("cadence_phase"), - "is_complete": bool(emb.get("is_complete")), - }) + points.append( + { + "embryo_id": eid, + "uid": emb.get("uid"), + "x": float(x), + "y": float(y), + "role": emb.get("role", "test"), + "strain": emb.get("strain"), + "user_label": emb.get("user_label"), + "confidence": emb.get("confidence"), + "cadence_phase": emb.get("cadence_phase"), + "is_complete": bool(emb.get("is_complete")), + } + ) return {"embryos": points} @router.get("/api/sequence/{embryo_id}") async def get_image_sequence( embryo_id: str, start: int = 0, - end: Optional[int] = None, + end: int | None = None, data_type: str = "volume_projection", - buffer_percent: float = 0.15 + buffer_percent: float = 0.15, ): """Get ordered sequence of images for timepoint range. @@ -230,7 +1778,7 @@ async def get_image_sequence( embryo_id=embryo_id, start=buffered_start, end=buffered_end, - data_type=data_type + data_type=data_type, ) # Return lightweight metadata (no base64 data) @@ -238,26 +1786,25 @@ async def get_image_sequence( seen_uids = set() for img in images: seen_uids.add(img.uid) - sequence.append({ - "uid": img.uid, - "timepoint": img.metadata.get("timepoint"), - "timestamp": img.timestamp, - "data_type": img.data_type, - "shape": img.shape, - "embryo_id": img.metadata.get("embryo_id") - }) + sequence.append( + { + "uid": img.uid, + "timepoint": img.metadata.get("timepoint"), + "timestamp": img.timestamp, + "data_type": img.data_type, + "shape": img.shape, + "embryo_id": img.metadata.get("embryo_id"), + } + ) # Fallback to persistent DataStore for missing timepoints if server.data_store and (len(sequence) == 0 or buffered_end is not None): try: - refs = server.data_store.query( - data_type=data_type, - embryo_id=embryo_id - ) + refs = server.data_store.query(data_type=data_type, embryo_id=embryo_id) for ref in refs: if ref.uid in seen_uids: continue - tp = ref.metadata.get('timepoint') + tp = ref.metadata.get("timepoint") if tp is None: continue tp = int(tp) @@ -266,16 +1813,18 @@ async def get_image_sequence( if buffered_end is not None and tp > buffered_end: continue seen_uids.add(ref.uid) - sequence.append({ - "uid": ref.uid, - "timepoint": tp, - "timestamp": ref.metadata.get('timestamp', ''), - "data_type": ref.data_type, - "shape": ref.metadata.get('shape'), - "embryo_id": embryo_id - }) + sequence.append( + { + "uid": ref.uid, + "timepoint": tp, + "timestamp": ref.metadata.get("timestamp", ""), + "data_type": ref.data_type, + "shape": ref.metadata.get("shape"), + "embryo_id": embryo_id, + } + ) # Re-sort by timepoint - sequence.sort(key=lambda x: x.get('timepoint') or 0) + sequence.sort(key=lambda x: x.get("timepoint") or 0) except Exception as e: logger.warning(f"DataStore fallback failed: {e}") @@ -284,14 +1833,12 @@ async def get_image_sequence( "requested_range": {"start": start, "end": end}, "buffered_range": {"start": buffered_start, "end": buffered_end}, "sequence": sequence, - "count": len(sequence) + "count": len(sequence), } @router.get("/api/events") async def list_events( - event_type: Optional[str] = None, - source: Optional[str] = None, - limit: int = 100 + event_type: str | None = None, source: str | None = None, limit: int = 100 ): """Get event history from EventBus""" if not server.event_bus: @@ -299,6 +1846,7 @@ async def list_events( # Get history from event bus from gently.core import EventType + et = None if event_type: try: @@ -306,24 +1854,22 @@ async def list_events( except KeyError: pass - events = server.event_bus.get_history( - event_type=et, - source=source, - limit=limit - ) + events = server.event_bus.get_history(event_type=et, source=source, limit=limit) return { "events": [ { - "event_type": e.event_type.name if hasattr(e.event_type, 'name') else str(e.event_type), + "event_type": e.event_type.name + if hasattr(e.event_type, "name") + else str(e.event_type), "data": e.data, "source": e.source, "timestamp": e.timestamp.isoformat(), - "event_id": e.event_id + "event_id": e.event_id, } for e in events ], - "total": len(events) + "total": len(events), } return router diff --git a/gently/ui/web/routes/device_layer.py b/gently/ui/web/routes/device_layer.py new file mode 100644 index 00000000..55faceda --- /dev/null +++ b/gently/ui/web/routes/device_layer.py @@ -0,0 +1,232 @@ +"""Device-layer supervision + launch gate routes. + +Two surfaces, both backed by ``DeviceLayerSupervisor``: + +- **Launch gate** (``GET /launch``) — the bare two-question start screen + (hardware on/off, agent on/off) with its persisted choices. +- **Devices panel** (``/api/device-layer/*``) — runtime status, log tail, and + Start / Stop controls that mirror the gate's hardware block. + +Plus the whole-backend shutdown handshake (``POST /api/shutdown``, issue #85) +used by the desktop shell on window-close — loopback-only, stops the managed +device layer gracefully, then asks the launcher to exit. + +Control-mutating endpoints (start/stop) are gated behind ``require_control``, +matching the rest of the hardware routes. Stop reuses the 409 + ``"blocked"`` +mid-run guard pattern: if hardware looks active, the caller must confirm. + +See ``docs/superpowers/specs/2026-07-02-unified-launcher-design.md`` (RFC #78). +""" + +from __future__ import annotations + +import asyncio +import logging + +from fastapi import APIRouter, Depends, Request +from fastapi.responses import HTMLResponse, JSONResponse + +from gently.ui.web.auth import require_control +from gently.ui.web.launch_prefs import detect_sam_device, load_prefs, save_prefs, stored_prefs + +logger = logging.getLogger(__name__) + + +def get_supervisor(server): + """Return the server's DeviceLayerSupervisor, creating one on first use. + + ``launch_gently`` attaches a supervisor to the viz server at boot, but we + lazily construct one if it's absent so these routes work standalone (and so + the panel degrades gracefully rather than 500-ing). + """ + sup = getattr(server, "device_supervisor", None) + if sup is None: + from gently.app.device_supervisor import DeviceLayerSupervisor + + sup = DeviceLayerSupervisor() + server.device_supervisor = sup + return sup + + +def _hardware_active(server) -> bool: + """Best-effort: is an acquisition / timelapse currently running? + + Used by the stop guard. Conservative — only reports active when we can see a + clearly-running timelapse; unknown state reads as inactive so a stuck flag + can't wedge the Stop button. + """ + tracker = getattr(server, "timelapse_tracker", None) + status = getattr(tracker, "status", None) + return str(status).lower() in ("running", "acquiring", "active") + + +def create_router(server) -> APIRouter: + router = APIRouter() + + # ── Launch gate ────────────────────────────────────────────────────── + + @router.get("/launch", response_class=HTMLResponse) + async def launch_gate(request: Request): + """The bare two-question launch screen (prefilled from last choice).""" + return server.templates.TemplateResponse(request, "launch.html", {}) + + @router.get("/api/launch/prefs") + async def get_launch_prefs(): + """Persisted launch choices (hardware/agent toggles + advanced defaults). + + Adds `sam_detected` (auto-detected GPU/CPU) and `sam_device_raw` (the + stored 'auto'/'cuda'/'cpu' before resolution) for the Settings UI. + """ + prefs = load_prefs() + prefs["sam_detected"] = detect_sam_device() + prefs["sam_device_raw"] = stored_prefs().get("sam_device", "auto") + return prefs + + @router.post("/api/launch/prefs", dependencies=[Depends(require_control)]) + async def set_launch_prefs(request: Request): + """Persist launch choices; returns the merged result.""" + body = await request.json() + return save_prefs(body if isinstance(body, dict) else {}) + + @router.post("/api/launch/go", dependencies=[Depends(require_control)]) + async def launch_go(request: Request): + """Submit the launch gate: persist the toggles, mark the gate passed + (so / stops bouncing to /launch), and — if the microscope toggle is on — + start the device layer. + + Returns fast: start() only *spawns* the child. MMCore init and its + per-stage progress happen in the background and are polled via + /api/device-layer/status, so the UI can hand off to the dashboard + immediately and follow the boot there. + """ + body = await _safe_json(request) + prefs = save_prefs(body if isinstance(body, dict) else {}) + server.gate_passed = True + device = None + if prefs.get("hardware"): + try: + device = get_supervisor(server).start() + except (FileNotFoundError, OSError) as e: + logger.error("device layer start failed from launch gate: %s", e) + return {"ok": True, "hardware": True, "error": str(e)} + return { + "ok": True, + "hardware": bool(prefs.get("hardware")), + "agent": bool(prefs.get("agent")), + "device": device, + } + + # ── Devices panel: device-layer supervision ────────────────────────── + + @router.get("/api/device-layer/status") + async def device_layer_status(): + """Live status: running / stopped / external / crashed (+ short log tail).""" + return get_supervisor(server).status() + + @router.get("/api/device-layer/log") + async def device_layer_log(limit: int = 200): + """Recent captured console output (oldest → newest) for the console view.""" + return {"lines": get_supervisor(server).log_tail(limit)} + + @router.post("/api/device-layer/start", dependencies=[Depends(require_control)]) + async def device_layer_start(request: Request): + """Spawn (or adopt-if-external) the device layer per the request body. + + Body (all optional): ``{"sam_device": "cuda"|"cpu", "config_path": "..."}``. + """ + body = await _safe_json(request) + sup = get_supervisor(server) + try: + status = sup.start( + sam_device=body.get("sam_device"), + config_path=body.get("config_path"), + ) + except (FileNotFoundError, OSError) as e: + logger.error("device layer start failed: %s", e) + return JSONResponse({"error": str(e)}, status_code=500) + return status + + @router.post("/api/device-layer/stop", dependencies=[Depends(require_control)]) + async def device_layer_stop(request: Request): + """Stop the managed device layer, with a mid-run confirmation guard. + + If hardware looks active and the body doesn't carry ``{"confirm": true}``, + returns 409 ``{"blocked": true, ...}`` instead of stopping (same shape the + thermalizer stop-guard uses). ``{"force": true}`` skips the grace period. + """ + body = await _safe_json(request) + sup = get_supervisor(server) + + if _hardware_active(server) and not body.get("confirm"): + return JSONResponse( + { + "blocked": True, + "reason": "hardware is active — an acquisition is running", + "hint": 'resend with {"confirm": true} to stop anyway', + }, + status_code=409, + ) + + return sup.stop(force=bool(body.get("force"))) + + # ── Whole-backend shutdown (desktop shell handshake, issue #85) ─────── + + @router.post("/api/shutdown") + async def shutdown_backend(request: Request): + """Gracefully stop the entire backend (device layer + agent + server). + + Called by the desktop shell on window-close so the backend can drain + state (e.g. session-replay final batches) and stop the device layer via + its clean SIGTERM path *before* the shell's kill / Job Object floor. + + Loopback-only: the shell carries no auth cookie, so instead of + ``require_control`` the guard is the connection source itself — only + 127.0.0.1/::1 may ask the process to die. Mid-run, the standard 409 + ``{"blocked": true}`` confirm guard applies (resend with + ``{"confirm": true}``); ``{"force": true}`` hard-kills the device layer. + """ + host = request.client.host if request.client else None + if host not in ("127.0.0.1", "::1"): + return JSONResponse( + {"error": "shutdown may only be requested from localhost"}, + status_code=403, + ) + + body = await _safe_json(request) + + if _hardware_active(server) and not body.get("confirm"): + return JSONResponse( + { + "blocked": True, + "reason": "hardware is active — an acquisition is running", + "hint": 'resend with {"confirm": true} to stop anyway', + }, + status_code=409, + ) + + # Stop a MANAGED device-layer child first (graceful path; no-ops for + # external/absent). stop() blocks in proc.wait() under a lock, so keep + # it off the event loop. + sup = get_supervisor(server) + try: + await asyncio.to_thread(sup.stop, force=bool(body.get("force"))) + except Exception: + logger.exception("device-layer stop during shutdown failed (continuing)") + + rs = getattr(server, "request_shutdown", None) + if rs is None: + return JSONResponse({"error": "shutdown not wired by launcher"}, status_code=501) + # Small delay so this response flushes before uvicorn starts exiting. + asyncio.get_running_loop().call_later(0.3, rs) + return {"ok": True, "stopping": True} + + return router + + +async def _safe_json(request: Request) -> dict: + """Parse a JSON body, tolerating an empty/absent one (returns {}).""" + try: + body = await request.json() + except (ValueError, TypeError): + return {} + return body if isinstance(body, dict) else {} diff --git a/gently/ui/web/routes/experiments.py b/gently/ui/web/routes/experiments.py index 3b8c9506..c2c5de77 100644 --- a/gently/ui/web/routes/experiments.py +++ b/gently/ui/web/routes/experiments.py @@ -61,6 +61,6 @@ async def get_strategy(session_id: str): raise HTTPException( status_code=500, detail=f"Failed to build strategy: {e}", - ) + ) from e return router diff --git a/gently/ui/web/routes/images.py b/gently/ui/web/routes/images.py index 2ecc805a..a9225d9b 100644 --- a/gently/ui/web/routes/images.py +++ b/gently/ui/web/routes/images.py @@ -2,16 +2,20 @@ import base64 import logging -from typing import Optional import numpy as np -from fastapi import APIRouter, HTTPException, Request -from fastapi.responses import Response, FileResponse +from fastapi import APIRouter, Depends, HTTPException, Request +from fastapi.responses import FileResponse, Response +from gently.ui.web.auth import require_control + +from ..upload_validation import decode_array_payload from ..volume_helpers import parse_volume_uid logger = logging.getLogger(__name__) +MAX_IMAGE_UPLOAD_BYTES = 64 * 1024 * 1024 + def create_router(server) -> APIRouter: router = APIRouter() @@ -66,17 +70,22 @@ async def get_image_png(uid: str): if data is None and parsed: embryo_id, timepoint = parsed if embryo_id in server.timelapse_tracker.projection_uids: - real_uid = server.timelapse_tracker.projection_uids[embryo_id].get(timepoint) + real_uid = server.timelapse_tracker.projection_uids[embryo_id].get( + timepoint + ) if real_uid: data = server.data_store.retrieve(real_uid) if data is not None: from io import BytesIO + from PIL import Image + from gently.core.imaging import ( - projection_three_view, - compute_crop_bounds, apply_crop_bounds, + compute_crop_bounds, + projection_three_view, ) + # Handle numpy array if isinstance(data, np.ndarray): # Handle 4D volumes (Views, Z, Y, X) - take View A @@ -84,67 +93,85 @@ async def get_image_png(uid: str): data = data[0] # Handle 3D volumes - generate three-view projection if data.ndim == 3: - z_depth, height, width = data.shape - # Handle dual-view format - if width > height * 2: - data = data[:, :, :width // 2] + # View A already selected by the 4D branch above; + # never split a 3D volume by aspect ratio. # Auto-crop and project bounds = compute_crop_bounds(data) data = apply_crop_bounds(data, bounds) data, _ = projection_three_view(data) # Normalize to uint8 if needed if data.dtype != np.uint8: - data = ((data - data.min()) / (data.max() - data.min() + 1e-8) * 255).astype(np.uint8) + data = ( + (data - data.min()) / (data.max() - data.min() + 1e-8) * 255 + ).astype(np.uint8) img = Image.fromarray(data) buf = BytesIO() - img.save(buf, format='PNG') - return Response(content=buf.getvalue(), media_type="image/png", headers=cache_headers) + img.save(buf, format="PNG") + return Response( + content=buf.getvalue(), + media_type="image/png", + headers=cache_headers, + ) except Exception as e: logger.warning(f"Failed to load image {uid} from DataStore: {e}") - # Fallback to FileStore JPEG projections (persistent on-disk) + # Fallback to FileStore JPEG projections (persistent on-disk). + # Unlike the in-memory base64 images, an on-disk projection CAN change + # (e.g. regenerated after a projection-format fix), so we must NOT mark + # it immutable with a content-independent (uid) ETag — that pins the + # browser to the stale image. Use a content-aware ETag (mtime+size) + # and a short max-age so a regeneration is picked up. if server.gently_store and parsed: embryo_id, timepoint = parsed proj_path = server._resolve_projection_path(embryo_id, timepoint) if proj_path: + st = proj_path.stat() return FileResponse( str(proj_path), media_type="image/jpeg", - headers=cache_headers, + headers={ + "Cache-Control": "public, max-age=3600", + "ETag": f'"{uid}-{int(st.st_mtime)}-{st.st_size}"', + }, ) raise HTTPException(status_code=404, detail=f"Image {uid} not found") - @router.post("/api/images") + @router.post("/api/images", dependencies=[Depends(require_control)]) async def push_image_http(request: Request): """Push a 2D image via HTTP (for CV subagent visualizations)""" try: data = await request.json() # Decode the image from base64 - image_b64 = data.get('image_b64') - uid = data.get('uid') - shape = data.get('shape') - dtype = data.get('dtype', 'uint8') - data_type = data.get('data_type', 'cv_visualization') - metadata = data.get('metadata', {}) + image_b64 = data.get("image_b64") + uid = data.get("uid") + shape = data.get("shape") + dtype = data.get("dtype", "uint8") + data_type = data.get("data_type", "cv_visualization") + metadata = data.get("metadata", {}) if not all([image_b64, uid, shape]): raise HTTPException(status_code=400, detail="Missing required fields") - # Decode array - array = np.frombuffer( - base64.b64decode(image_b64), - dtype=np.dtype(dtype) - ).reshape(shape) + # Decode array (validates shape/dtype and caps size before allocating) + array = decode_array_payload( + image_b64, + shape, + dtype, + max_nbytes=MAX_IMAGE_UPLOAD_BYTES, + label="image", + ) # Push using the existing method await server.push_image(array, uid, data_type, metadata) return {"status": "ok", "uid": uid, "data_type": data_type} + except HTTPException: + raise except Exception as e: logger.error(f"Failed to push image via HTTP: {e}") - raise HTTPException(status_code=500, detail=str(e)) + raise HTTPException(status_code=500, detail=str(e)) from e return router diff --git a/gently/ui/web/routes/logs.py b/gently/ui/web/routes/logs.py new file mode 100644 index 00000000..945337ab --- /dev/null +++ b/gently/ui/web/routes/logs.py @@ -0,0 +1,122 @@ +""" +Process console routes +====================== + +Serves the two process consoles the header drawer shows. + +The agent's stdout is genuinely unreachable in the packaged desktop app — the +Tauri shell spawns the backend with ``CREATE_NO_WINDOW`` in release, so there is +no console to look at when something misbehaves mid-session. Its output does +still land in ``{storage}/logs/gently_*.log``, so that file is the console. + +The device layer already has a live captured-stdout tail on its supervisor +(``/api/device-layer/log``); this module does not duplicate it. It only adds the +file-backed agent side, plus a fallback tail of the device layer's own log file +for the case where the layer runs externally and the supervisor captured +nothing. +""" + +from __future__ import annotations + +import logging +import re +from collections import deque +from pathlib import Path + +from fastapi import APIRouter, HTTPException, Query + +logger = logging.getLogger(__name__) + +MAX_LINES = 2000 +# Read only the tail of the file. Session logs can reach tens of MB and the +# drawer never shows more than a couple of thousand lines. +TAIL_BYTES = 512 * 1024 + +# "2026-07-19 11:04:22 gently.app.agent INFO message" — the default file format. +_LEVEL_RE = re.compile(r"\b(DEBUG|INFO|WARNING|ERROR|CRITICAL)\b") + +_SOURCES = { + "agent": "gently_*.log", + "device": "device_layer_*.log", +} + + +def _log_dir() -> Path: + from gently.settings import settings + + return Path(settings.storage.base_path) / "logs" + + +def _newest(pattern: str) -> Path | None: + d = _log_dir() + if not d.is_dir(): + return None + files = [p for p in d.glob(pattern) if p.is_file()] + if not files: + return None + return max(files, key=lambda p: p.stat().st_mtime) + + +def _tail(path: Path, limit: int) -> list[str]: + """Last ``limit`` lines, reading only the final chunk of the file.""" + with path.open("rb") as fh: + fh.seek(0, 2) + size = fh.tell() + fh.seek(max(0, size - TAIL_BYTES)) + chunk = fh.read() + text = chunk.decode("utf-8", errors="replace") + # A partial first line is likely when the file is larger than the window. + lines = text.splitlines() + if size > TAIL_BYTES and lines: + lines = lines[1:] + return list(deque(lines, maxlen=limit)) + + +def create_router(server) -> APIRouter: # noqa: ARG001 — parity with sibling modules + router = APIRouter() + + @router.get("/api/logs/{source}") + async def read_log( + source: str, + limit: int = Query(400, ge=1, le=MAX_LINES), + level: str | None = Query(None, description="Minimum level: INFO|WARNING|ERROR"), + ): + """Tail a process log (oldest → newest), optionally filtered by level. + + Returns ``{source, file, lines, truncated}``. A missing log directory or + no matching file is not an error — the console shows an empty state and + keeps polling, because the process may simply not have started yet. + """ + pattern = _SOURCES.get(source) + if pattern is None: + raise HTTPException(status_code=404, detail=f"unknown log source: {source}") + + path = _newest(pattern) + if path is None: + return {"source": source, "file": None, "lines": [], "truncated": False} + + try: + lines = _tail(path, limit) + except OSError as exc: + logger.warning("could not read %s log at %s: %s", source, path, exc) + raise HTTPException(status_code=502, detail=f"log read failed: {exc}") from exc + + if level: + wanted = level.upper() + order = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"] + if wanted in order: + keep = set(order[order.index(wanted) :]) + # Lines without a level are continuations (tracebacks) — keeping + # them preserves the stack under the ERROR that introduced it. + lines = [ + ln for ln in lines if not (m := _LEVEL_RE.search(ln)) or m.group(1) in keep + ] + + return { + "source": source, + "file": path.name, + "lines": lines, + "truncated": len(lines) >= limit, + } + + return router diff --git a/gently/ui/web/routes/notebook.py b/gently/ui/web/routes/notebook.py new file mode 100644 index 00000000..b9cb27da --- /dev/null +++ b/gently/ui/web/routes/notebook.py @@ -0,0 +1,99 @@ +"""Notebook (shared lab notebook) read routes. + +Exposes the notebook's Notes for the Notebook tab + Agent's-View live edge. +Read-only here; authoring/curation come in a later increment. +""" + +from fastapi import APIRouter, Body, HTTPException + +from gently.harness.memory.notebook import Author, NoteKind, NoteStatus, note_to_dict + + +def _coerce(enum_cls, value): + """Parse a query-param string into an enum; invalid/None → None (no filter).""" + if value is None: + return None + try: + return enum_cls(value) + except ValueError: + return None + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _nb(): + cs = getattr(server, "context_store", None) + return cs.notebook if cs is not None else None + + @router.get("/api/notebook/notes") + async def list_notes( + kind: str | None = None, + author: str | None = None, + status: str | None = None, + strain: str | None = None, + embryo: str | None = None, + thread: str | None = None, + limit: int | None = None, + ): + nb = _nb() + if nb is None: + return {"available": False, "notes": []} + notes = nb.query_notes( + kind=_coerce(NoteKind, kind), + author=_coerce(Author, author), + status=_coerce(NoteStatus, status), + strain=strain, + embryo=embryo, + thread=thread, + ) + if limit is not None and limit >= 0: + notes = notes[:limit] + return {"available": True, "notes": [note_to_dict(n) for n in notes]} + + @router.get("/api/notebook/notes/{note_id}") + async def get_note(note_id: str): + nb = _nb() + if nb is None: + raise HTTPException(status_code=404, detail="notebook unavailable") + note = nb.get_note(note_id) + if note is None: + raise HTTPException(status_code=404, detail="note not found") + return note_to_dict(note) + + @router.get("/api/notebook/threads") + async def list_threads(): + nb = _nb() + if nb is None: + return {"available": False, "threads": []} + counts: dict[str, int] = {} + for n in nb.query_notes(): + for t in n.threads: + counts[t] = counts.get(t, 0) + 1 + threads = [{"id": t, "count": c} for t, c in sorted(counts.items())] + return {"available": True, "threads": threads} + + @router.post("/api/notebook/ask") + async def ask( + question: str = Body(..., embed=True), + thread: str | None = Body(None, embed=True), + strain: str | None = Body(None, embed=True), + ): + nb = _nb() + if nb is None: + return {"available": False} + from gently.harness.memory.notebook_ask import answer_question, select_notes + from gently.settings import settings + + notes = select_notes(nb, thread=thread, strain=strain) + client = getattr(server, "claude_async", None) + if client is None: + import anthropic + + client = anthropic.AsyncAnthropic() + result = await answer_question(client, settings.models.main, question, notes) + result["available"] = True + result["note_ids"] = [n.id for n in notes] + return result + + return router diff --git a/gently/ui/web/routes/operation_plan.py b/gently/ui/web/routes/operation_plan.py new file mode 100644 index 00000000..a5e636eb --- /dev/null +++ b/gently/ui/web/routes/operation_plan.py @@ -0,0 +1,42 @@ +"""Operation Plan route. + +Returns the agent-authored Operation Plan for a session. +The plan is stored in FileContextStore (``server.context_store``) keyed by +session_id. ``session_id="current"`` resolves to the newest session via the +FileStore (``server.gently_store``). +""" + +from fastapi import APIRouter, HTTPException + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _resolve_session(session_id: str) -> str: + if session_id == "current": + store = getattr(server, "gently_store", None) + if store is None: + raise HTTPException( + status_code=503, detail="FileStore not configured on viz server" + ) + sessions = store.list_sessions() + if not sessions: + raise HTTPException(status_code=404, detail="No sessions in store") + session_id = sessions[0].get("session_id") + return session_id + + @router.get("/api/operation_plan/{session_id}") + async def get_operation_plan(session_id: str): + real_id = _resolve_session(session_id) + cs = getattr(server, "context_store", None) + if cs is None: + return {"session_id": real_id, "available": False, "plan": None} + try: + plan = cs.get_operation_plan(real_id) + except Exception: + plan = None + if plan is None: + return {"session_id": real_id, "available": False, "plan": None} + return {"session_id": real_id, "available": True, "plan": plan} + + return router diff --git a/gently/ui/web/routes/pages.py b/gently/ui/web/routes/pages.py index 1412e014..e52dbda5 100644 --- a/gently/ui/web/routes/pages.py +++ b/gently/ui/web/routes/pages.py @@ -1,18 +1,54 @@ """Page routes - HTML template rendering.""" +import time + from fastapi import APIRouter, Request from fastapi.responses import HTMLResponse, RedirectResponse +from gently.settings import settings + def create_router(server) -> APIRouter: router = APIRouter() @router.get("/", response_class=HTMLResponse) async def index(request: Request): - """Serve the main SPA page""" + """Serve the main SPA page. + + Viewing is open to everyone — the dashboard loads in view mode with no + login. Signing in is an *elevation* to control (handled in-app via the + chat window's "Sign in" affordance), not a gate on the page itself. + + The launch gate is the entry point: until it's submitted this session, + every visit to / bounces to /launch (RFC #78 defer-init boot). + """ + if not getattr(server, "gate_passed", False): + return RedirectResponse("/launch", status_code=302) + # The v2 landing ("what are we doing today?") is for STARTING fresh. Skip + # it when resuming a session (one-shot flag from the resume route) or when + # the live session already has work — so a resumed/underway session lands + # straight in the workspace instead of bouncing through the welcome screen. + # Resumed within the last few seconds? (time-window, not a one-shot, so + # all clients the resume-broadcast reloads skip the landing together.) + just_resumed = (time.monotonic() - getattr(server, "_resumed_at", 0.0)) < 15.0 + has_work = False + try: + bridge = getattr(server, "agent_bridge", None) + agent = getattr(bridge, "agent", None) if bridge else None + if agent is not None: + has_work = len(agent.experiment.embryos) > 0 + except Exception: + has_work = False + show_landing = bool(settings.ui.ux_v2) and not (just_resumed or has_work) return server.templates.TemplateResponse( + request, "index.html", - {"request": request, "active_section": "embryos", "is_live": True} + { + "active_section": "embryos", + "is_live": True, + "ux_v2": settings.ui.ux_v2, + "show_landing": show_landing, + }, ) # Standalone URLs redirect to SPA with hash fragment for tab routing @@ -32,8 +68,8 @@ async def plan_review_page(campaign_id: str): async def settings_page(request: Request): """Serve the dashboard settings page""" return server.templates.TemplateResponse( + request, "settings.html", - {"request": request} ) return router diff --git a/gently/ui/web/routes/replay.py b/gently/ui/web/routes/replay.py new file mode 100644 index 00000000..a0254a2c --- /dev/null +++ b/gently/ui/web/routes/replay.py @@ -0,0 +1,376 @@ +"""Session replay — rrweb ingest + post-hoc player. + +The recorder (``static/js/replay-recorder.js``) POSTs event batches here and +this module appends them to per-session JSONL files in the file store: + + sessions/{session}/ui-replay/rrweb-{tab}.jsonl full rrweb event stream + sessions/{session}/ui-replay/actions.jsonl semantic action log + sessions/{session}/ui-replay/meta.yaml tabs seen, user agents + +Batches that arrive while no session is active land in an unassigned bucket +(``{storage}/ui-replay/unassigned-{YYYYMMDD}/``) so nothing is dropped. + +Detached by design (docs/superpowers/specs/2026-07-13-session-replay-design.md): +nothing else in gently imports this module, and the ingest path never raises +into the UI — failures return JSON errors the recorder treats as backoff +signals. Ingest is unauthenticated like the viewing surface: recording must +cover view-role users, and the instrument is localhost-first. +""" + +import asyncio +import json +import logging +import re +from datetime import datetime +from pathlib import Path +from typing import Any + +from fastapi import APIRouter, Request +from fastapi.responses import HTMLResponse, JSONResponse + +from gently.core.file_store import _write_yaml +from gently.settings import settings + +logger = logging.getLogger(__name__) + +_TAB_RE = re.compile(r"^[a-f0-9]{4,16}$") +_UNASSIGNED_RE = re.compile(r"^unassigned-\d{8}$") +_MAX_BATCH_BYTES = 32 * 1024 * 1024 +# meta.yaml is only rewritten when a new (dir, tab) pair first appears. +_seen_tabs: set[tuple[str, str]] = set() +# (dir, tab) pairs whose rrweb file has hit the per-tab size cap — so the +# "capped" marker is logged once, not on every subsequent ingest. +_capped_tabs: set[tuple[str, str]] = set() +# Retention prune runs once per process, lazily on the first ingest (by which +# point the store is up), off the event loop. +_pruned = False + + +def _store(server) -> Any: + return getattr(server, "gently_store", None) + + +def _active_replay_dir(server) -> tuple[Path | None, str | None]: + """The active session's ui-replay dir, else the day's unassigned bucket.""" + store = _store(server) + if store is None: + return None, None + sid: str | None = None + try: + sid = server._current_session_id() + except Exception: # noqa: BLE001 — never let session lookup break ingest + sid = None + if sid: + sd = store._session_dir(sid) + if sd is not None and sd.exists(): + return sd / "ui-replay", sid + day = datetime.now().strftime("%Y%m%d") + return Path(store.root) / "ui-replay" / f"unassigned-{day}", None + + +def _resolve_replay_dir(server, session_id: str) -> Path | None: + """Map a player-facing id (session id or unassigned bucket) to its dir.""" + store = _store(server) + if store is None: + return None + if _UNASSIGNED_RE.match(session_id): + d = Path(store.root) / "ui-replay" / session_id + return d if d.exists() else None + sd = store._session_dir(session_id) + if sd is None: + return None + d = sd / "ui-replay" + return d if d.exists() else None + + +def _list_recordings(server) -> list[dict[str, Any]]: + """Every id (session or unassigned bucket) that has replay data.""" + store = _store(server) + if store is None: + return [] + out: list[dict[str, Any]] = [] + index: dict[str, str] = dict(getattr(store, "_index", {}) or {}) + for sid, folder in sorted(index.items(), key=lambda kv: kv[1], reverse=True): + d = Path(store.root) / "sessions" / folder / "ui-replay" + if d.is_dir(): + out.append({"id": sid, "folder": folder, **_dir_stats(d)}) + unassigned = Path(store.root) / "ui-replay" + if unassigned.is_dir(): + for d in sorted(unassigned.iterdir(), reverse=True): + if d.is_dir() and _UNASSIGNED_RE.match(d.name): + out.append({"id": d.name, "folder": d.name, **_dir_stats(d)}) + return out + + +def _dir_stats(d: Path) -> dict[str, Any]: + tabs = sorted(p.stem.replace("rrweb-", "") for p in d.glob("rrweb-*.jsonl")) + size = sum(p.stat().st_size for p in d.glob("*.jsonl")) + return {"tabs": tabs, "bytes": size} + + +def _append_lines(path: Path, records: list[Any]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + with open(path, "a", encoding="utf-8") as f: + for rec in records: + f.write(json.dumps(rec, ensure_ascii=False, default=str) + "\n") + + +def _update_meta(base: Path, tab: str, user_agent: str) -> None: + key = (str(base), tab) + if key in _seen_tabs: + return + _seen_tabs.add(key) + meta_path = base / "meta.yaml" + meta: dict[str, Any] = {} + if meta_path.exists(): + try: + import yaml + + meta = yaml.safe_load(meta_path.read_text(encoding="utf-8")) or {} + except Exception: # noqa: BLE001 — corrupt meta must not break ingest + meta = {} + tabs = meta.setdefault("tabs", {}) + tabs.setdefault( + tab, + {"first_seen": datetime.now().isoformat(), "user_agent": user_agent[:200]}, + ) + _write_yaml(meta_path, meta) + + +def _all_replay_dirs(store) -> list[Path]: + """Every ui-replay dir on disk (per-session + unassigned buckets).""" + root = Path(store.root) + dirs = [d for d in (root / "sessions").glob("*/ui-replay") if d.is_dir()] + unassigned = root / "ui-replay" + if unassigned.is_dir(): + dirs += [d for d in unassigned.iterdir() if d.is_dir() and _UNASSIGNED_RE.match(d.name)] + return dirs + + +def _prune_recordings(server) -> None: + """Keep total rrweb footprint under the configured budget by deleting the + oldest recordings first. Conservative: always keeps the newest few, skips + the active session, and never lets a failure escape into ingest.""" + store = _store(server) + if store is None: + return + budget = int(settings.ui.replay_total_budget_mb * 1024 * 1024) + active: str | None = None + try: + active = server._current_session_id() + except Exception: # noqa: BLE001 + active = None + entries: list[tuple[float, int, Path]] = [] + for d in _all_replay_dirs(store): + try: + files = list(d.glob("*.jsonl")) + size = sum(f.stat().st_size for f in files) + mtime = max((f.stat().st_mtime for f in files), default=0.0) + except OSError: + continue + # Never prune the active session's live recording. + if active and active in str(d): + continue + entries.append((mtime, size, d)) + total = sum(s for _, s, _ in entries) + if total <= budget: + return + entries.sort(key=lambda e: e[0]) # oldest first + keep_newest = 3 + prunable = entries[: max(0, len(entries) - keep_newest)] + import shutil + + for _mtime, size, d in prunable: + if total <= budget: + break + try: + shutil.rmtree(d) + total -= size + logger.info("replay: pruned old recording %s (%.0f MB)", d.name, size / 1048576) + except OSError: + logger.debug("replay: could not prune %s", d, exc_info=True) + + +def _read_jsonl(path: Path) -> list[Any]: + out = [] + with open(path, encoding="utf-8") as f: + for line in f: + line = line.strip() + if not line: + continue + try: + out.append(json.loads(line)) + except json.JSONDecodeError: + continue + return out + + +def create_router(server) -> APIRouter: + router = APIRouter() + + # One Jinja global so every template can conditionally include the + # recorder — the flag is the only coupling between replay and the pages. + try: + server.templates.env.globals["replay_enabled"] = settings.ui.replay + server.templates.env.globals["replay_fidelity"] = settings.ui.replay_fidelity + except Exception: # noqa: BLE001 — a missing templates attr is not fatal + pass + + @router.post("/replay/ingest") + async def ingest(request: Request): + if not settings.ui.replay: + return JSONResponse({"error": "replay disabled"}, status_code=403) + try: + length = int(request.headers.get("content-length") or 0) + except ValueError: + length = 0 + if length > _MAX_BATCH_BYTES: + return JSONResponse({"error": "batch too large"}, status_code=413) + try: + batch = await request.json() + except Exception: # noqa: BLE001 + return JSONResponse({"error": "bad json"}, status_code=400) + tab = str(batch.get("tab") or "") + if not _TAB_RE.match(tab): + return JSONResponse({"error": "bad tab id"}, status_code=400) + + base, sid = _active_replay_dir(server) + if base is None: + return JSONResponse({"error": "no store"}, status_code=503) + + rrweb_events = batch.get("rrweb") or [] + actions = list(batch.get("actions") or []) + for a in actions: + if isinstance(a, dict): + a["tab"] = tab + gap = batch.get("gap") + if gap: + actions.append( + { + "t": datetime.now().isoformat(), + "action": "gap", + "tab": tab, + "params": gap, + } + ) + + # Prune old recordings to the total budget once per process, off the + # event loop (the store is up by the time ingest first fires). + global _pruned + if not _pruned: + _pruned = True + asyncio.create_task(asyncio.to_thread(_prune_recordings, server)) + + cap_bytes = int(settings.ui.replay_max_tab_mb * 1024 * 1024) + + def _write() -> None: + rrweb_path = base / f"rrweb-{tab}.jsonl" + if rrweb_events: + try: + over_cap = rrweb_path.exists() and rrweb_path.stat().st_size >= cap_bytes + except OSError: + over_cap = False + if over_cap: + # A single tab hit the size cap — stop growing its rrweb + # stream (the live map/telemetry re-render at poll rate can + # run this to gigabytes). Keep the small action log going; + # mark the truncation once. + key = (str(base), tab) + if key not in _capped_tabs: + _capped_tabs.add(key) + actions.append( + { + "t": datetime.now().isoformat(), + "action": "rrweb-capped", + "tab": tab, + "params": {"cap_mb": settings.ui.replay_max_tab_mb}, + } + ) + logger.warning( + "replay: rrweb cap (%.0f MB) hit for tab %s — dropping further frames", + settings.ui.replay_max_tab_mb, + tab, + ) + else: + _append_lines(rrweb_path, rrweb_events) + if actions: + _append_lines(base / "actions.jsonl", actions) + _update_meta(base, tab, request.headers.get("user-agent", "")) + + try: + await asyncio.to_thread(_write) + except Exception: # noqa: BLE001 — degrade recording, never the app + logger.exception("replay ingest write failed") + return JSONResponse({"error": "write failed"}, status_code=500) + return {"ok": True, "session": sid} + + # ---- post-hoc player (read-only; available even when recording is off) -- + + @router.get("/replay", response_class=HTMLResponse) + async def replay_index(request: Request): + return server.templates.TemplateResponse( + request, + "replay.html", + {"session_id": None, "recordings": _list_recordings(server)}, + ) + + @router.get("/replay/api/recordings") + async def api_recordings(): + return {"recordings": _list_recordings(server)} + + @router.get("/replay/api/{session_id}/tabs") + async def api_tabs(session_id: str): + d = _resolve_replay_dir(server, session_id) + if d is None: + return JSONResponse({"error": "no replay data"}, status_code=404) + tabs = [] + for p in sorted(d.glob("rrweb-*.jsonl")): + tabs.append( + { + "tab": p.stem.replace("rrweb-", ""), + "bytes": p.stat().st_size, + "events": sum(1 for _ in open(p, encoding="utf-8")), + } + ) + return {"tabs": tabs} + + @router.get("/replay/api/{session_id}/events") + async def api_events(session_id: str, tab: str, start: int = 0, end: int = 0): + """rrweb events for one tab, optionally clipped to [start, end] ms.""" + d = _resolve_replay_dir(server, session_id) + if d is None: + return JSONResponse({"error": "no replay data"}, status_code=404) + path = d / f"rrweb-{tab}.jsonl" + if not _TAB_RE.match(tab) or not path.exists(): + return JSONResponse({"error": "unknown tab"}, status_code=404) + events = await asyncio.to_thread(_read_jsonl, path) + if start or end: + events = [ + e + for e in events + if isinstance(e, dict) + and (not start or e.get("timestamp", 0) >= start) + and (not end or e.get("timestamp", 0) <= end) + ] + return {"events": events} + + @router.get("/replay/api/{session_id}/actions") + async def api_actions(session_id: str): + d = _resolve_replay_dir(server, session_id) + if d is None: + return JSONResponse({"error": "no replay data"}, status_code=404) + path = d / "actions.jsonl" + actions = await asyncio.to_thread(_read_jsonl, path) if path.exists() else [] + return {"actions": actions} + + @router.get("/replay/{session_id}", response_class=HTMLResponse) + async def replay_player(request: Request, session_id: str): + if _resolve_replay_dir(server, session_id) is None: + return JSONResponse({"error": "no replay data"}, status_code=404) + return server.templates.TemplateResponse( + request, + "replay.html", + {"session_id": session_id, "recordings": []}, + ) + + return router diff --git a/gently/ui/web/routes/roles.py b/gently/ui/web/routes/roles.py new file mode 100644 index 00000000..057a97e4 --- /dev/null +++ b/gently/ui/web/routes/roles.py @@ -0,0 +1,30 @@ +"""Embryo Role Registry route. + +Returns the static REGISTRY of embryo roles from ``gently.harness.roles``. +Never raises a 500 — the registry is global and read-only. +""" + +from fastapi import APIRouter + +from gently.harness.roles import REGISTRY + + +def create_router(server) -> APIRouter: # noqa: ARG001 (server not needed; registry is global) + router = APIRouter() + + @router.get("/api/roles") + async def get_roles(): + roles = [ + { + "name": role.name, + "description": role.description, + "role_class": role.role_class, + "ui_color": role.ui_color, + "ui_icon": role.ui_icon, + "default_cadence_seconds": role.default_cadence_seconds, + } + for role in REGISTRY.values() + ] + return {"roles": roles} + + return router diff --git a/gently/ui/web/routes/sessions.py b/gently/ui/web/routes/sessions.py index 69d70d47..797003bc 100644 --- a/gently/ui/web/routes/sessions.py +++ b/gently/ui/web/routes/sessions.py @@ -1,9 +1,13 @@ -"""Session routes - list and retrieve saved sessions.""" +"""Session routes - list, retrieve, and resume saved sessions.""" -import json import logging +import time +from pathlib import Path -from fastapi import APIRouter, HTTPException +from fastapi import APIRouter, Depends, HTTPException +from fastapi.responses import FileResponse + +from gently.ui.web.auth import require_control logger = logging.getLogger(__name__) @@ -11,39 +15,254 @@ def create_router(server) -> APIRouter: router = APIRouter() + def _file_store(): + """The live FileStore (current Gently3 layout), via the agent.""" + bridge = getattr(server, "agent_bridge", None) + if bridge is not None and getattr(bridge, "agent", None) is not None: + st = getattr(bridge.agent, "store", None) + if st is not None: + return st + return getattr(server, "gently_store", None) + + def _active_session_id(): + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + return getattr(agent, "session_id", None) if agent is not None else None + @router.get("/api/sessions") async def list_sessions(): - """List available sessions with metadata""" + """List available sessions (from the live FileStore).""" + store = _file_store() + if store is None: + return {"sessions": []} + active_id = _active_session_id() sessions = [] - if server.sessions_dir.exists(): - for path in server.sessions_dir.glob("*.json"): + try: + for s in store.list_sessions(): + sid = s.get("session_id") try: - with open(path, encoding='utf-8') as f: - data = json.load(f) - sessions.append({ - 'session_id': data.get('session_id', path.stem), - 'name': data.get('name', path.stem), - 'created_at': data.get('created_at', ''), - 'last_active': data.get('last_active', ''), - 'embryo_count': len(data.get('embryo_states', {})), - 'description': data.get('description', '') - }) - except Exception as e: - logger.warning(f"Failed to read session {path}: {e}") - # Sort by created_at descending (newest first) - sessions.sort(key=lambda x: x.get('created_at', ''), reverse=True) - return {'sessions': sessions} + count = len(store.list_embryos(sid) or []) + except Exception: + count = 0 + sessions.append( + { + "session_id": sid, + "name": s.get("name") or sid, + "created_at": s.get("created_at", ""), + "last_active": s.get("last_active", ""), + "embryo_count": count, + "description": s.get("description", ""), + "active": sid == active_id, + } + ) + except Exception as e: + logger.warning("Failed to list sessions from FileStore: %s", e) + return {"sessions": sessions} - @router.get("/api/sessions/{session_id}") - async def get_session(session_id: str): - """Get full session state for review""" - path = server.sessions_dir / f"{session_id}.json" - if not path.exists(): + @router.get("/api/home/recent-images") + async def recent_images(limit: int = 8, scan: int = 200): + """Latest projection per embryo, aggregated across recent sessions. + + Unlike /api/snapshots (in-memory, current session only), this walks the + FileStore on disk so the home page can show imagery from *previous* + sessions. Cheap by construction: recent session IDs come from folder + names (no session.yaml parse), embryo IDs from directory names (no + embryo.yaml parse), timepoints from a filename glob (no pixel decode), + and the walk stops as soon as `limit` images are collected. + + `scan` is the *budget* of most-recent sessions to walk while hunting for + images, NOT a hard window — empty/aborted sessions (common at the head: + a rig accrues many no-capture sessions) are skipped nearly for free + (one iterdir each), so the default is generous enough to reach older + sessions that actually hold projections. Both bounds are clamped so a + crafted ?scan=/?limit= can't turn this unauthenticated read into an + unbounded scan. Returns components; the client builds the (encoded) URL. + """ + store = _file_store() + if store is None: + return {"images": []} + limit = max(1, min(int(limit), 48)) + scan = max(1, min(int(scan), 500)) + out = [] + try: + for sid in store.recent_session_ids(scan) or []: + try: + eids = store.list_embryo_ids(sid) + except Exception: + eids = [] + sname = None # parsed lazily, only if this session contributes + for eid in eids: + try: + tps = store.list_projection_timepoints(sid, eid) or [] + except Exception: + tps = [] + if not tps: + continue + if sname is None: + try: + info = store.get_session(sid) + except Exception: + info = None + sname = (info.get("name") if info else None) or sid + out.append( + { + "session_id": sid, + "session_name": sname, + "embryo_id": eid, + "timepoint": int(max(tps)), + } + ) + if len(out) >= limit: + break + if len(out) >= limit: + break + except Exception as e: + logger.warning("recent_images failed: %s", e) + return {"images": out[:limit]} + + @router.get("/api/sessions/{session_id}/projection") + async def get_session_projection(session_id: str, embryo: str, t: int): + """Serve a saved JPEG projection from any session on disk. + + Path-traversal safe: the resolved file must live inside the session's + own directory, so a crafted `embryo` (e.g. '../..') can't escape. + """ + store = _file_store() + if store is None: + raise HTTPException(status_code=503, detail="Store not available") + path = store.get_projection_path(session_id, embryo, t) + if path is None: + raise HTTPException(status_code=404, detail="Projection not found") + try: + sd = store._session_dir(session_id) + resolved = Path(path).resolve() + # Component-wise ancestor check (not str.startswith, which would + # let a sibling like `_evil` slip through the prefix match). + sd_resolved = Path(sd).resolve() if sd is not None else None + if sd_resolved is None or sd_resolved not in resolved.parents: + raise HTTPException(status_code=404, detail="Not found") + except HTTPException: + raise + except Exception: + raise HTTPException(status_code=404, detail="Not found") from None + try: + st = resolved.stat() + etag = f'"{int(st.st_mtime)}-{st.st_size}"' + except OSError: + etag = None + headers = {"Cache-Control": "private, max-age=60"} + if etag: + headers["ETag"] = etag + return FileResponse(str(resolved), media_type="image/jpeg", headers=headers) + + @router.post("/api/sessions/{session_id}/resume", dependencies=[Depends(require_control)]) + async def resume_session(session_id: str): + """Switch the live agent to a different saved session. + + Reuses the same machinery as CLI resume (saves the current session, + loads the target's embryos + conversation). Then nudges all browser + clients to reload so they pick up the new session's state and + transcript. + """ + bridge = getattr(server, "agent_bridge", None) + agent = bridge.agent if bridge is not None else None + if agent is None: + raise HTTPException(status_code=503, detail="Agent not ready") + store = getattr(agent, "store", None) + if store is None or store.get_session(session_id) is None: raise HTTPException(status_code=404, detail="Session not found") + if session_id == getattr(agent, "session_id", None): + return { + "ok": True, + "session_id": session_id, + "active": True, + "note": "already active", + } + try: + ok = agent.resume_session(session_id) + except Exception as e: + logger.exception("Session resume failed") + raise HTTPException(status_code=500, detail=f"resume failed: {e}") from e + if not ok: + raise HTTPException(status_code=500, detail="resume returned false") + # Rehydrate the viz image store from disk so the resumed session's + # projections/filmstrips show (pixels load lazily from the FileStore). + rehydrated = 0 + try: + rehydrated = server.rehydrate_session(session_id) + except Exception: + logger.exception("rehydrate_session failed") + # Resuming is an in-app action — the operator is already past the entry + # gate, so never bounce them back to /launch to re-answer hardware / + # assistant. gate_passed is in-memory and resets on any backend restart, + # which is exactly what made a resume-to-view land on the launch gate. + server.gate_passed = True + # Also skip the "what are we doing today?" landing overlay on the reload + # below — resuming existing work isn't starting fresh. A timestamp (not a + # one-shot bool) so EVERY client the broadcast reloads skips the landing, + # not just whichever one hits the index route first. + server._resumed_at = time.monotonic() + # Tell every connected browser to reload — they'll reconnect to the + # new session's state (embryos, transcript, rehydrated imagery). try: - with open(path, encoding='utf-8') as f: - return json.load(f) + await server.manager.broadcast({"type": "session_changed", "session_id": session_id}) + except Exception: + pass + return { + "ok": True, + "session_id": session_id, + "active": True, + "rehydrated_projections": rehydrated, + } + + @router.get("/api/sessions/{session_id}/plans") + async def get_session_plans(session_id: str): + """Plan items linked to a session, via the context store.""" + cs = getattr(server, "context_store", None) + if cs is None: + return {"plans": []} + try: + items = cs.get_plan_items_for_session(session_id) except Exception as e: - raise HTTPException(status_code=500, detail=f"Failed to load session: {e}") + logger.warning("get_plan_items_for_session failed for %s: %s", session_id, e) + return {"plans": []} + return { + "plans": [ + { + "id": item.id, + "title": item.title, + "campaign_id": item.campaign_id, + "status": item.status.value, + } + for item in items + ] + } + + @router.get("/api/sessions/{session_id}") + async def get_session(session_id: str): + """Get session state for review, from the live FileStore. + + Maps the FileStore session snapshot onto the shape the Sessions review + view expects (embryo_states / conversation). detection_history isn't + reconstructed here (per-timepoint predictions live elsewhere). + """ + store = _file_store() + if store is None: + raise HTTPException(status_code=503, detail="Store not available") + info = store.get_session(session_id) + if info is None: + raise HTTPException(status_code=404, detail="Session not found") + snapshot = store.load_session_snapshot(session_id) or {} + experiment = snapshot.get("experiment_data", {}) or {} + return { + "session_id": session_id, + "name": info.get("name") or session_id, + "description": info.get("description", ""), + "created_at": info.get("created_at", ""), + "last_active": info.get("last_active", ""), + "embryo_states": experiment.get("embryos", {}) or {}, + "conversation": snapshot.get("conversation_history", []) or [], + "detection_history": {}, + } return router diff --git a/gently/ui/web/routes/tactic_library.py b/gently/ui/web/routes/tactic_library.py new file mode 100644 index 00000000..5ef12804 --- /dev/null +++ b/gently/ui/web/routes/tactic_library.py @@ -0,0 +1,27 @@ +"""Tactic Library route. + +Returns the saved tactic library from FileContextStore (``server.context_store``). +Never raises a 500 — returns an empty list when the store is absent or has no +saved tactics. +""" + +from fastapi import APIRouter + + +def create_router(server) -> APIRouter: + router = APIRouter() + + @router.get("/api/tactic_library") + async def get_tactic_library(): + cs = getattr(server, "context_store", None) + if cs is None: + return {"tactics": []} + try: + tactics = cs.list_tactics() + except Exception: + tactics = [] + if not tactics: + return {"tactics": []} + return {"tactics": tactics} + + return router diff --git a/gently/ui/web/routes/temperature.py b/gently/ui/web/routes/temperature.py new file mode 100644 index 00000000..90916181 --- /dev/null +++ b/gently/ui/web/routes/temperature.py @@ -0,0 +1,47 @@ +"""Read-only temperature history for the live graph (backfill on mount/reload). + +Live updates ride the TEMPERATURE_UPDATE event channel; this route is backfill only. +Mirrors routes/experiments.py session resolution. +""" + +import urllib.parse + +from fastapi import APIRouter, HTTPException, Request + + +def create_router(server) -> APIRouter: + router = APIRouter() + + def _resolve_session(session_id: str): + store = getattr(server, "gently_store", None) + if store is None: + raise HTTPException(status_code=503, detail="FileStore not configured on viz server") + if session_id == "current": + sessions = store.list_sessions() + if not sessions: + raise HTTPException(status_code=404, detail="No sessions in store") + session_id = sessions[0].get("session_id") + if store._session_dir(session_id) is None: + raise HTTPException(status_code=404, detail=f"Session not found: {session_id}") + return session_id + + @router.get("/api/temperature/{session_id}/history") + async def get_history(session_id: str, request: Request): + # Parse `since` from raw query string using unquote (not unquote_plus) so that + # timezone offsets like +00:00 are preserved. Standard FastAPI query-param + # parsing applies unquote_plus, which converts + to a space. + raw_qs = request.scope.get("query_string", b"").decode() + since = None + for part in raw_qs.split("&"): + if "=" in part: + k, v = part.split("=", 1) + if urllib.parse.unquote(k) == "since": + since = urllib.parse.unquote(v) + break + + real_id = _resolve_session(session_id) + store = server.gently_store + samples = store.read_temperature_log(real_id, since=since) + return {"session_id": real_id, "samples": samples} + + return router diff --git a/gently/ui/web/routes/volumes.py b/gently/ui/web/routes/volumes.py index e48475c3..fa67c40b 100644 --- a/gently/ui/web/routes/volumes.py +++ b/gently/ui/web/routes/volumes.py @@ -3,18 +3,24 @@ import base64 import io import logging -from typing import Optional +from typing import Any import numpy as np -from fastapi import APIRouter, HTTPException, Request +from fastapi import APIRouter, Depends, HTTPException, Request from fastapi.responses import Response -from ..volume_helpers import load_volume_from_disk, image_to_base64_png +from gently.ui.web.auth import require_control + +from ..upload_validation import decode_array_payload +from ..volume_helpers import image_to_base64_png, load_volume_from_disk logger = logging.getLogger(__name__) +MAX_VOLUME_UPLOAD_BYTES = 512 * 1024 * 1024 + try: from PIL import Image + PIL_AVAILABLE = True except ImportError: PIL_AVAILABLE = False @@ -31,7 +37,8 @@ async def get_projections(embryo_id: str, timepoint: int, method: str = "all"): Args: embryo_id: Embryo identifier timepoint: Timepoint number (1-indexed) - method: Projection method - 'all', 'three_view', 'dual_view', 'depth_colored', 'multi_slice' + method: Projection method - 'all', 'three_view', 'dual_view', + 'depth_colored', 'multi_slice' Returns: List of projections with method name, description, and base64 PNG data @@ -41,7 +48,10 @@ async def get_projections(embryo_id: str, timepoint: int, method: str = "all"): # Look up volume path (timelapse tracker + FileStore fallback) volume_path = server._resolve_volume_path(embryo_id, timepoint) if not volume_path: - raise HTTPException(status_code=404, detail=f"No volume for {embryo_id} at timepoint {timepoint}") + raise HTTPException( + status_code=404, + detail=f"No volume for {embryo_id} at timepoint {timepoint}", + ) # Load volume from disk try: @@ -52,29 +62,32 @@ async def get_projections(embryo_id: str, timepoint: int, method: str = "all"): vol = (vol - vol.min()) / (vol.max() - vol.min() + 1e-8) except FileNotFoundError as e: - raise HTTPException(status_code=404, detail=str(e)) + raise HTTPException(status_code=404, detail=str(e)) from e except Exception as e: logger.error(f"Failed to load volume: {e}") - raise HTTPException(status_code=500, detail=f"Failed to load volume: {e}") + raise HTTPException(status_code=500, detail=f"Failed to load volume: {e}") from e - PROJECTION_METHODS = { - 'three_view': projection_three_view, + PROJECTION_METHODS: dict[str, Any] = { + "three_view": projection_three_view, } # Try to import additional projection methods from explorer try: from gently.dataset.explorer_server import ( - projection_dual_view, projection_depth_colored, + projection_dual_view, projection_multi_slice, projection_spin_3d, ) - PROJECTION_METHODS.update({ - 'dual_view': projection_dual_view, - 'depth_colored': projection_depth_colored, - 'multi_slice': projection_multi_slice, - 'spin_3d': projection_spin_3d, - }) + + PROJECTION_METHODS.update( + { + "dual_view": projection_dual_view, + "depth_colored": projection_depth_colored, + "multi_slice": projection_multi_slice, + "spin_3d": projection_spin_3d, + } + ) except ImportError: pass # Explorer projections not available @@ -84,22 +97,31 @@ async def get_projections(embryo_id: str, timepoint: int, method: str = "all"): for method_name, method_func in PROJECTION_METHODS.items(): try: proj_img, desc = method_func(vol) - projections.append({ - "method": method_name, - "description": desc, - "data": image_to_base64_png(proj_img), - }) + projections.append( + { + "method": method_name, + "description": desc, + "data": image_to_base64_png(proj_img), + } + ) except Exception as e: logger.warning(f"Projection {method_name} failed: {e}") else: if method not in PROJECTION_METHODS: - raise HTTPException(status_code=400, detail=f"Unknown method: {method}. Available: {list(PROJECTION_METHODS.keys())}") + raise HTTPException( + status_code=400, + detail=( + f"Unknown method: {method}. Available: {list(PROJECTION_METHODS.keys())}" + ), + ) proj_img, desc = PROJECTION_METHODS[method](vol) - projections.append({ - "method": method, - "description": desc, - "data": image_to_base64_png(proj_img), - }) + projections.append( + { + "method": method, + "description": desc, + "data": image_to_base64_png(proj_img), + } + ) return { "embryo_id": embryo_id, @@ -120,7 +142,10 @@ async def get_volume_raw(embryo_id: str, timepoint: int): # Look up volume path (timelapse tracker + FileStore fallback) volume_path = server._resolve_volume_path(embryo_id, timepoint) if not volume_path: - raise HTTPException(status_code=404, detail=f"No volume for {embryo_id} at timepoint {timepoint}") + raise HTTPException( + status_code=404, + detail=f"No volume for {embryo_id} at timepoint {timepoint}", + ) try: vol = load_volume_from_disk(volume_path) @@ -137,7 +162,7 @@ async def get_volume_raw(embryo_id: str, timepoint: int): # Encode as base64 vol_bytes = vol_uint8.tobytes() - vol_b64 = base64.b64encode(vol_bytes).decode('utf-8') + vol_b64 = base64.b64encode(vol_bytes).decode("utf-8") # Physical voxel size for isometric 3D rendering. # Matches the default in gently.core.imaging.projection_three_view: @@ -155,17 +180,17 @@ async def get_volume_raw(embryo_id: str, timepoint: int): } except FileNotFoundError as e: - raise HTTPException(status_code=404, detail=str(e)) + raise HTTPException(status_code=404, detail=str(e)) from e except Exception as e: logger.error(f"Failed to load volume: {e}") - raise HTTPException(status_code=500, detail=f"Failed to load volume: {e}") + raise HTTPException(status_code=500, detail=f"Failed to load volume: {e}") from e @router.get("/api/volumes3d") async def list_volumes_3d(): """Get list of 3D volumes (without heavy data)""" return { "volumes_3d": server.store.get_all_volumes_3d(), - "count": len(server.store._volumes_3d) + "count": len(server.store._volumes_3d), } @router.get("/api/volumes3d/{uid}") @@ -188,7 +213,7 @@ async def get_volume_3d_slice(uid: str, z: int): if PIL_AVAILABLE: img = Image.fromarray(rgb) buffer = io.BytesIO() - img.save(buffer, format='PNG') + img.save(buffer, format="PNG") return Response(content=buffer.getvalue(), media_type="image/png") raise HTTPException(status_code=500, detail="PIL not available") @@ -209,53 +234,64 @@ async def get_volume_data_for_3d_viewer(uid: str): volume = np.zeros(volume.shape, dtype=np.uint8) return { "shape": list(volume.shape), - "data": base64.b64encode(volume.tobytes()).decode('utf-8'), - "uid": uid + "data": base64.b64encode(volume.tobytes()).decode("utf-8"), + "uid": uid, } # Check if it's a regular image with stored volume data image = server.store.get_image_by_uid(uid) if image and image.shape and len(image.shape) == 3: - raise HTTPException(status_code=404, detail=f"Volume data for {uid} not available - only segmented volumes supported") + raise HTTPException( + status_code=404, + detail=f"Volume data for {uid} not available - only segmented volumes supported", + ) raise HTTPException(status_code=404, detail=f"Volume {uid} not found") - @router.post("/api/volumes3d") + @router.post("/api/volumes3d", dependencies=[Depends(require_control)]) async def push_volume_3d_http(request: Request): """Push a 3D volume with segmentation via HTTP (for CV subagent)""" try: data = await request.json() # Decode the volume and masks from base64 - volume_b64 = data.get('volume_b64') - masks_b64 = data.get('masks_b64') - uid = data.get('uid') - shape = data.get('shape') - dtype_vol = data.get('dtype_vol', 'uint16') - dtype_mask = data.get('dtype_mask', 'uint16') - metadata = data.get('metadata', {}) + volume_b64 = data.get("volume_b64") + masks_b64 = data.get("masks_b64") + uid = data.get("uid") + shape = data.get("shape") + dtype_vol = data.get("dtype_vol", "uint16") + dtype_mask = data.get("dtype_mask", "uint16") + metadata = data.get("metadata", {}) if not all([volume_b64, masks_b64, uid, shape]): raise HTTPException(status_code=400, detail="Missing required fields") - # Decode arrays - volume = np.frombuffer( - base64.b64decode(volume_b64), - dtype=np.dtype(dtype_vol) - ).reshape(shape) + # Decode arrays (validates shape/dtype and caps size before allocating) + volume = decode_array_payload( + volume_b64, + shape, + dtype_vol, + max_nbytes=MAX_VOLUME_UPLOAD_BYTES, + label="volume", + ) - masks = np.frombuffer( - base64.b64decode(masks_b64), - dtype=np.dtype(dtype_mask) - ).reshape(shape) + masks = decode_array_payload( + masks_b64, + shape, + dtype_mask, + max_nbytes=MAX_VOLUME_UPLOAD_BYTES, + label="masks", + ) # Push using the existing method await server.push_volume_3d(volume, masks, uid, metadata) return {"status": "ok", "uid": uid, "shape": shape} + except HTTPException: + raise except Exception as e: logger.error(f"Failed to push 3D volume via HTTP: {e}") - raise HTTPException(status_code=500, detail=str(e)) + raise HTTPException(status_code=500, detail=str(e)) from e return router diff --git a/gently/ui/web/routes/websocket.py b/gently/ui/web/routes/websocket.py index b49518e0..0de827fd 100644 --- a/gently/ui/web/routes/websocket.py +++ b/gently/ui/web/routes/websocket.py @@ -4,6 +4,7 @@ import json import logging from datetime import datetime +from typing import Any from fastapi import APIRouter, WebSocket, WebSocketDisconnect @@ -11,6 +12,36 @@ logger = logging.getLogger(__name__) +# /ws message types that mutate experiment state (define what gets imaged). +# These are control actions and are gated by role; pure read/presence +# messages stay open so anyone can watch. +_MARKING_TYPES = frozenset( + { + "embryo_marked", + "marking_update", + "marking_done", + "marking_redetect", + } +) + + +def _ws_can_control(websocket: WebSocket) -> bool: + """Whether this /ws client may perform control actions (marking). + + Account mode: operators/admins (by session cookie) only. Legacy mode + (no accounts configured): open, preserving prior behavior. + """ + from gently.ui.web.accounts import CONTROL_ROLES, get_account_store + from gently.ui.web.auth import SESSION_COOKIE + + store = get_account_store() + if store is None or not store.has_users(): + return True + token = websocket.cookies.get(SESSION_COOKIE) + user = store.verify_session(token) if token else None + role = store.get_role(user) if user else None + return role in CONTROL_ROLES + def create_router(server) -> APIRouter: router = APIRouter() @@ -23,27 +54,30 @@ async def websocket_endpoint(websocket: WebSocket): try: # Send current status on connect stats = server.store.get_stats() - await websocket.send_json({ - "type": "connected", - **stats, - "timestamp": datetime.now().isoformat() - }) + await websocket.send_json( + {"type": "connected", **stats, "timestamp": datetime.now().isoformat()} + ) # Always send timelapse state on connect so client can reconcile # (if IDLE with no session_id, client will clear stale cached state) timelapse_state = server.timelapse_tracker.to_dict() - await websocket.send_json({ - "type": "timelapse_state", - "data": timelapse_state - }) + # The header's session id is driven by this payload; the tracker's + # session_id goes stale after a resume with no active timelapse, so + # override it with the live agent session (the source of truth). + try: + bridge = getattr(server, "agent_bridge", None) + if bridge is not None and getattr(bridge, "agent", None) is not None: + live_sid = bridge.agent.session_id + if live_sid: + timelapse_state["session_id"] = live_sid + except Exception: + pass + await websocket.send_json({"type": "timelapse_state", "data": timelapse_state}) # Keep connection alive and handle incoming messages while True: try: - data = await asyncio.wait_for( - websocket.receive_text(), - timeout=30.0 - ) + data = await asyncio.wait_for(websocket.receive_text(), timeout=30.0) # Handle client messages (e.g., requests) await _handle_ws_message(server, websocket, data) except asyncio.TimeoutError: @@ -77,41 +111,37 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): msg_type = data.get("type") embryo_id = data.get("embryo_id") + # Gate control actions (marking) by role; viewing/presence stays open. + if msg_type in _MARKING_TYPES and not _ws_can_control(websocket): + logger.warning("Ignored %s from a view-only /ws client", msg_type) + return + if msg_type == "get_calibration": images = server.store.get_all_calibration(embryo_id) - await websocket.send_json({ - "type": "calibration", - "data": [img.to_dict() for img in images] - }) + await websocket.send_json( + {"type": "calibration", "data": [img.to_dict() for img in images]} + ) elif msg_type == "get_volumes": images = server.store.get_all_volumes(embryo_id) - await websocket.send_json({ - "type": "volumes", - "data": [img.to_dict() for img in images] - }) + await websocket.send_json( + {"type": "volumes", "data": [img.to_dict() for img in images]} + ) elif msg_type == "get_snapshots": images = server.store.get_all_snapshots(embryo_id) - await websocket.send_json({ - "type": "snapshots", - "data": [img.to_dict() for img in images] - }) + await websocket.send_json( + {"type": "snapshots", "data": [img.to_dict() for img in images]} + ) elif msg_type == "get_embryos": - await websocket.send_json({ - "type": "embryos", - "data": server.store.get_embryo_ids() - }) + await websocket.send_json({"type": "embryos", "data": server.store.get_embryo_ids()}) elif msg_type == "get_image": uid = data.get("uid") image = server.store.get_image_by_uid(uid) if image: - await websocket.send_json({ - "type": "image", - "data": image.to_dict() - }) + await websocket.send_json({"type": "image", "data": image.to_dict()}) elif msg_type == "pong": pass # Client responding to ping @@ -125,7 +155,8 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): # Sanitize name: strip HTML tags, limit length if name: import re - name = re.sub(r'<[^>]+>', '', name)[:50] + + name = re.sub(r"<[^>]+>", "", name)[:50] # Update the client's info async with server.manager._lock: if websocket in server.manager.active_connections: @@ -134,7 +165,7 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): client_id=client_id, name=name or old_info.name, color=server.manager._generate_color(client_id), - connected_at=old_info.connected_at + connected_at=old_info.connected_at, ) await server.manager.broadcast_presence() @@ -144,7 +175,8 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): if name: # Sanitize name: strip HTML tags, limit length import re - name = re.sub(r'<[^>]+>', '', name)[:50] + + name = re.sub(r"<[^>]+>", "", name)[:50] await server.manager.update_client_name(websocket, name) elif msg_type == "get_presence": @@ -155,16 +187,19 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): elif msg_type == "embryo_marked": session_id = data.get("session_id") marker = data.get("marker") - if session_id and marker and hasattr(server, '_marking_sessions'): + if session_id and marker and hasattr(server, "_marking_sessions"): session = server._marking_sessions.get(session_id) if session: session["markers"].append(marker) - logger.info(f"Embryo marked: #{marker['number']} at ({marker['pixelX']}, {marker['pixelY']})") + logger.info( + f"Embryo marked: #{marker['number']}" + f" at ({marker['pixelX']}, {marker['pixelY']})" + ) elif msg_type == "marking_update": session_id = data.get("session_id") markers = data.get("markers", []) - if session_id and hasattr(server, '_marking_sessions'): + if session_id and hasattr(server, "_marking_sessions"): session = server._marking_sessions.get(session_id) if session: session["markers"] = markers @@ -173,18 +208,17 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): elif msg_type == "marking_done": session_id = data.get("session_id") markers = data.get("markers", []) - if session_id and hasattr(server, '_marking_sessions'): + if session_id and hasattr(server, "_marking_sessions"): session = server._marking_sessions.get(session_id) if session: session["markers"] = markers session["complete"].set() - role_summary = {} + role_summary: dict[str, Any] = {} for m in markers: r = m.get("role", "test") role_summary[r] = role_summary.get(r, 0) + 1 logger.info( - f"Marking complete: {len(markers)} embryo(s) " - f"(roles: {role_summary})" + f"Marking complete: {len(markers)} embryo(s) (roles: {role_summary})" ) elif msg_type == "marking_redetect": @@ -194,13 +228,14 @@ async def _handle_ws_message(server, websocket: WebSocket, message: str): # listen for. Once recapture lands, the agent calls # start_marking_session again with the new image + markers. session_id = data.get("session_id") - if session_id and hasattr(server, '_marking_sessions'): + if session_id and hasattr(server, "_marking_sessions"): session = server._marking_sessions.get(session_id) if session is not None: session["redetect_requested"] = True logger.info(f"Marking redetect requested for session {session_id}") try: from gently.core import EventType, get_event_bus + get_event_bus().publish( event_type=EventType.STATUS_CHANGED, data={ diff --git a/gently/ui/web/server.py b/gently/ui/web/server.py index ff14f9bb..153a598b 100644 --- a/gently/ui/web/server.py +++ b/gently/ui/web/server.py @@ -24,7 +24,7 @@ import sys from datetime import datetime from pathlib import Path -from typing import Any, Dict, List, Optional +from typing import Any import numpy as np @@ -35,11 +35,12 @@ # Optional imports try: + import uvicorn from fastapi import FastAPI + from fastapi.middleware.cors import CORSMiddleware from fastapi.staticfiles import StaticFiles from fastapi.templating import Jinja2Templates - from fastapi.middleware.cors import CORSMiddleware - import uvicorn + FASTAPI_AVAILABLE = True except ImportError: FASTAPI_AVAILABLE = False @@ -60,8 +61,7 @@ class _InvalidHttpFilter(logging.Filter): def filter(self, record: logging.LogRecord) -> bool: msg = record.getMessage() if "Invalid HTTP request received" in msg: - logger.debug("uvicorn dropped non-HTTP bytes on viz port " - "(probable TLS/peer mismatch)") + logger.debug("uvicorn dropped non-HTTP bytes on viz port (probable TLS/peer mismatch)") return False return True @@ -76,25 +76,39 @@ def filter(self, record: logging.LogRecord) -> bool: try: from PIL import Image + PIL_AVAILABLE = True except ImportError: PIL_AVAILABLE = False # Import data models and components -from .models import ( - ClientInfo, Volume3DData, ImageData, EmbryoImageCache, - CALIBRATION_TYPES, VOLUME_TYPES, ANALYSIS_TYPES, VOLUME_3D_TYPES, +from .connection_manager import ConnectionManager # noqa: E402 +from .image_store import ImageStore # noqa: E402 +from .models import ( # noqa: E402 + ANALYSIS_TYPES, + CALIBRATION_TYPES, + VOLUME_3D_TYPES, + VOLUME_TYPES, + ClientInfo, + EmbryoImageCache, + ImageData, + Volume3DData, ) -from .image_store import ImageStore -from .timelapse_tracker import TimelapseStateTracker -from .connection_manager import ConnectionManager +from .timelapse_tracker import TimelapseStateTracker # noqa: E402 # Re-export for backward compatibility __all__ = [ - 'VisualizationServer', 'create_visualization_server', - 'ClientInfo', 'Volume3DData', 'ImageData', 'EmbryoImageCache', - 'CALIBRATION_TYPES', 'VOLUME_TYPES', 'ANALYSIS_TYPES', 'VOLUME_3D_TYPES', - 'ImageStore', + "VisualizationServer", + "create_visualization_server", + "ClientInfo", + "Volume3DData", + "ImageData", + "EmbryoImageCache", + "CALIBRATION_TYPES", + "VOLUME_TYPES", + "ANALYSIS_TYPES", + "VOLUME_3D_TYPES", + "ImageStore", ] @@ -130,8 +144,8 @@ def __init__( event_bus=None, sessions_dir: str = str(settings.storage.sessions_dir), gently_store=None, - ssl_certfile: str = None, - ssl_keyfile: str = None, + ssl_certfile: str | None = None, + ssl_keyfile: str | None = None, ): super().__init__(name="visualization", service_type="http", host=host, port=port) if not FASTAPI_AVAILABLE: @@ -147,6 +161,16 @@ def __init__( self.sessions_dir = Path(sessions_dir) self.gently_store = gently_store # FileStore for persistent volume/projection access self.context_store = None # FileContextStore — set via set_context_store() + # Wired in by launch_gently after construction (optional subsystems). + self.agent_bridge: Any = None + self.mesh_service: Any = None + self.device_supervisor: Any = None # DeviceLayerSupervisor (RFC #78) + # Callable that stops the WHOLE backend (launcher keep-alive included); + # POST /api/shutdown uses it for the desktop shell handshake (issue #85). + self.request_shutdown: Any = None + # False until the launch gate is submitted this session; while False, / + # bounces to /launch so the gate is the entry point (RFC #78). + self.gate_passed: bool = False # Connection manager for WebSocket clients self.manager = ConnectionManager() @@ -161,13 +185,26 @@ def __init__( self.app = FastAPI( title="Gently Visualization Server", description="Real-time microscopy visualization", - version="2.0.0" + version="2.0.0", ) # Setup templates and static files self.templates = Jinja2Templates(directory=str(TEMPLATES_DIR)) self.app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static") + # Static assets are served live (CLAUDE.md: "refresh the window — served + # live by Python, no rebuild"). Default StaticFiles sends ETag + + # Last-Modified but no Cache-Control, so browsers apply *heuristic* + # freshness and can serve a stale .js/.css after an edit — breaking that + # promise. Force revalidation on every load: with the ETag still present + # an unchanged file returns a cheap 304, a changed one returns fresh bytes. + @self.app.middleware("http") + async def _revalidate_static(request, call_next): + response = await call_next(request) + if request.url.path.startswith("/static"): + response.headers["Cache-Control"] = "no-cache" + return response + # Add CORS middleware self.app.add_middleware( CORSMiddleware, @@ -179,6 +216,7 @@ def __init__( # Register route groups from .routes import register_all_routes + register_all_routes(self) # Subscribe to events if event bus provided @@ -193,7 +231,7 @@ def set_context_store(self, context_store) -> None: """Set the FileContextStore for campaign/plan data access.""" self.context_store = context_store - def _resolve_volume_path(self, embryo_id: str, timepoint: int) -> Optional[str]: + def _resolve_volume_path(self, embryo_id: str, timepoint: int) -> str | None: """Resolve volume file path from timelapse tracker or FileStore.""" # 1. Try timelapse tracker (in-memory, fastest) if embryo_id in self.timelapse_tracker.volume_paths: @@ -201,11 +239,17 @@ def _resolve_volume_path(self, embryo_id: str, timepoint: int) -> Optional[str]: if path: return path - # 2. Try FileStore (file-based, persistent) - if self.gently_store and self.timelapse_tracker.session_id: + # 2. Try FileStore (file-based, persistent). Key on the LIVE agent + # session, not the tracker's (which goes stale after a resume with no + # active timelapse) — mirrors _resolve_projection_path so an agent-driven + # open_volume hand-off doesn't 404 after a /resume. + sid = self._current_session_id() + if self.gently_store and sid: try: vol_path = self.gently_store.get_volume_path( - self.timelapse_tracker.session_id, embryo_id, timepoint, + sid, + embryo_id, + timepoint, ) if vol_path and vol_path.exists(): return str(vol_path) @@ -214,12 +258,26 @@ def _resolve_volume_path(self, embryo_id: str, timepoint: int) -> Optional[str]: return None - def _resolve_projection_path(self, embryo_id: str, timepoint: int) -> Optional[Path]: - """Resolve projection file path from FileStore.""" - if self.gently_store and self.timelapse_tracker.session_id: + def _current_session_id(self) -> str | None: + """The live agent session (source of truth), falling back to the + timelapse tracker. The tracker's session_id goes stale after a resume + with no active timelapse, so the live agent session is preferred.""" + bridge = getattr(self, "agent_bridge", None) + if bridge is not None and getattr(bridge, "agent", None) is not None: + sid = getattr(bridge.agent, "session_id", None) + if sid: + return sid + return self.timelapse_tracker.session_id + + def _resolve_projection_path(self, embryo_id: str, timepoint: int) -> Path | None: + """Resolve projection file path from FileStore (current session).""" + sid = self._current_session_id() + if self.gently_store and sid: try: proj_path = self.gently_store.get_projection_path( - self.timelapse_tracker.session_id, embryo_id, timepoint, + sid, + embryo_id, + timepoint, ) if proj_path and proj_path.exists(): return proj_path @@ -227,6 +285,109 @@ def _resolve_projection_path(self, embryo_id: str, timepoint: int) -> Optional[P logger.debug(f"FileStore projection path lookup failed: {e}") return None + def rehydrate_session(self, session_id: str) -> int: + """Repopulate the in-memory image store with the FileStore's persisted + projections for a (resumed) session, so galleries and filmstrips show + its historical data. + + Lightweight: only metadata-bearing ImageData entries are created (uid + ``volume_{embryo}_t{NNNN}``); the JPEG pixels load lazily on demand via + /api/images/{uid}/png (which falls back to the FileStore projection). + Resets the store first so the previous session's images don't linger. + Returns the number of projection entries added. + """ + if self.gently_store is None or not session_id: + return 0 + self.store = ImageStore() # drop the previous session's images + added = 0 + try: + embryos = self.gently_store.list_embryos(session_id) or [] + except Exception: + embryos = [] + for emb in embryos: + eid = emb.get("embryo_id") if isinstance(emb, dict) else getattr(emb, "embryo_id", None) + if not eid: + continue + try: + tps = self.gently_store.list_projection_timepoints(session_id, eid) + except Exception: + tps = [] + for tp in tps: + self.store.add_image( + ImageData( + uid=f"volume_{eid}_t{tp:04d}", + data_type="volume_projection", + timestamp=f"{tp:06d}", # monotonic with timepoint for ordering + metadata={"embryo_id": eid, "timepoint": tp}, + ) + ) + added += 1 + + # Rehydrate the timelapse tracker's per-embryo perception state from + # predictions.jsonl so the Default / Film / reasoning views populate + # (those are driven by detection_reasoning, not the raw image store). + # Thumbnails resolve via the projection uids added above. + tracker = self.timelapse_tracker + try: + tracker.session_id = session_id + tracker.detection_reasoning = {} + tracker.projection_uids = {} + for emb in embryos: + eid = ( + emb.get("embryo_id") + if isinstance(emb, dict) + else getattr(emb, "embryo_id", None) + ) + if not eid: + continue + try: + preds = self.gently_store.get_predictions(session_id, eid) or [] + except Exception: + preds = [] + if not preds: + continue + items, puids, last_stage = [], {}, None + for p in preds: + tp = p.get("timepoint") + if tp is None: + continue + uid = f"volume_{eid}_t{tp:04d}" + puids[tp] = uid + stage = p.get("predicted_stage") + last_stage = stage or last_stage + items.append( + { + "timepoint": tp, + "stage": stage, + "detected_stage": stage, + "reasoning": p.get("reasoning"), + "confidence": p.get("confidence"), + "projection_uid": uid, + "image_uid": uid, + "detector_name": "perception", + } + ) + tracker.detection_reasoning[eid] = items + tracker.projection_uids[eid] = puids + entry = tracker.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "detections": {}, + "current_stage": None, + }, + ) + entry["timepoints"] = max((it["timepoint"] for it in items), default=0) + entry["current_stage"] = last_stage + tracker.total_timepoints = sum(len(v) for v in tracker.detection_reasoning.values()) + except Exception: + logger.exception("Tracker perception rehydration failed") + + logger.info("Rehydrated %d projections for session %s", added, session_id) + return added + def _subscribe_to_events(self): """Subscribe to EventBus for automatic updates - broadcasts ALL events""" @@ -236,7 +397,11 @@ def _subscribe_to_events(self): async def on_event_async(event): """Async handler for all events - broadcasts to WebSocket clients""" - event_type_str = event.event_type.name if hasattr(event.event_type, 'name') else str(event.event_type) + event_type_str = ( + event.event_type.name + if hasattr(event.event_type, "name") + else str(event.event_type) + ) # Update timelapse state tracker self.timelapse_tracker.handle_event(event_type_str, event.data) @@ -246,15 +411,17 @@ async def on_event_async(event): event_type=event_type_str, data=event.data, source=event.source, - event_id=event.event_id + event_id=event.event_id, ) # For session events, also broadcast updated timelapse_state so clients can sync if event_type_str in ("SESSION_STARTED", "SESSION_RESTORED"): - await self.manager.broadcast({ - "type": "timelapse_state", - "data": self.timelapse_tracker.to_dict() - }) + await self.manager.broadcast( + { + "type": "timelapse_state", + "data": self.timelapse_tracker.to_dict(), + } + ) # Subscribe to ALL events using wildcard with async handler self.event_bus.subscribe_async("*", on_event_async) @@ -272,11 +439,19 @@ def _init_from_event_history(self): # Process events in chronological order (history is newest-first) for event in reversed(history): - event_type_str = event.event_type.name if hasattr(event.event_type, 'name') else str(event.event_type) + event_type_str = ( + event.event_type.name + if hasattr(event.event_type, "name") + else str(event.event_type) + ) self.timelapse_tracker.handle_event(event_type_str, event.data) if self.timelapse_tracker.session_id: - logger.info(f"Initialized timelapse state from history: session={self.timelapse_tracker.session_id}, status={self.timelapse_tracker.status}") + logger.info( + f"Initialized timelapse state from history:" + f" session={self.timelapse_tracker.session_id}," + f" status={self.timelapse_tracker.status}" + ) except Exception as e: logger.warning(f"Failed to initialize from event history: {e}") @@ -285,13 +460,13 @@ def _array_to_image_data( array: np.ndarray, uid: str, data_type: str, - metadata: Optional[Dict] = None + metadata: dict | None = None, ) -> ImageData: """Convert numpy array to ImageData with base64 PNG""" from gently.core.imaging import ( - projection_three_view, - compute_crop_bounds, apply_crop_bounds, + compute_crop_bounds, + projection_three_view, ) # Handle 4D arrays (Views, Z, Y, X) - select View A only @@ -305,10 +480,9 @@ def _array_to_image_data( pass else: # It's a volume (Z, H, W) - generate three-view projection - z_depth, height, width = array.shape - # Handle dual-view format (width > 2*height) - if width > height * 2: - array = array[:, :, :width // 2] + # View selection already happened via the 4D branch above; a 3D + # array is one view. Do not split by aspect ratio (2048x512 + # native frames would be halved). # Auto-crop to embryo region bounds = compute_crop_bounds(array) array = apply_crop_bounds(array, bounds) @@ -328,8 +502,8 @@ def _array_to_image_data( if PIL_AVAILABLE: img = Image.fromarray(array) buffer = io.BytesIO() - img.save(buffer, format='PNG') - base64_png = base64.b64encode(buffer.getvalue()).decode('utf-8') + img.save(buffer, format="PNG") + base64_png = base64.b64encode(buffer.getvalue()).decode("utf-8") return ImageData( uid=uid, @@ -337,7 +511,7 @@ def _array_to_image_data( timestamp=datetime.now().isoformat(), metadata=metadata or {}, base64_png=base64_png, - shape=array.shape + shape=array.shape, ) async def push_image( @@ -345,7 +519,7 @@ async def push_image( array: np.ndarray, uid: str, data_type: str = "image", - metadata: Optional[Dict] = None, + metadata: dict | None = None, ): """ Push an image to connected clients @@ -369,14 +543,16 @@ async def push_image( # Broadcast to clients await self.manager.send_image(image_data) - logger.debug(f"Pushed image {uid} ({data_type}) to {len(self.manager.active_connections)} clients") + logger.debug( + f"Pushed image {uid} ({data_type}) to {len(self.manager.active_connections)} clients" + ) async def start_marking_session( self, image: np.ndarray, initial_stage_position: tuple = (0.0, 0.0), pixel_size_um: float = 0.65, - initial_markers: Optional[list] = None, + initial_markers: list | None = None, default_role: str = "test", ) -> str: """ @@ -412,8 +588,8 @@ async def start_marking_session( """ import uuid - if not hasattr(self, '_marking_sessions'): - self._marking_sessions = {} + if not hasattr(self, "_marking_sessions"): + self._marking_sessions: dict[str, dict[str, Any]] = {} session_id = str(uuid.uuid4())[:8] @@ -424,15 +600,17 @@ async def start_marking_session( py = m.get("pixel_y", m.get("pixelY")) if px is None or py is None: continue - normalized.append({ - "number": i + 1, - "pixelX": round(float(px), 1), - "pixelY": round(float(py), 1), - "role": m.get("role", default_role), - "source": m.get("source", "sam"), - "embryo_id": m.get("embryo_id"), - "confidence": m.get("confidence"), - }) + normalized.append( + { + "number": i + 1, + "pixelX": round(float(px), 1), + "pixelY": round(float(py), 1), + "role": m.get("role", default_role), + "source": m.get("source", "sam"), + "embryo_id": m.get("embryo_id"), + "confidence": m.get("confidence"), + } + ) self._marking_sessions[session_id] = { "markers": list(normalized), @@ -445,31 +623,34 @@ async def start_marking_session( # Encode image as base64 PNG from PIL import Image as PILImage + img = image if img.dtype != np.uint8: img = ((img - img.min()) / max(img.max() - img.min(), 1) * 255).astype(np.uint8) pil_img = PILImage.fromarray(img) buf = io.BytesIO() - pil_img.save(buf, format='PNG') - b64 = base64.b64encode(buf.getvalue()).decode('ascii') + pil_img.save(buf, format="PNG") + b64 = base64.b64encode(buf.getvalue()).decode("ascii") h, w = image.shape[:2] # Broadcast to all clients - await self.manager.broadcast({ - "type": "marking_image", - "data": { - "session_id": session_id, - "image_b64": b64, - "width": w, - "height": h, - "initial_markers": normalized, - "default_role": default_role, - "stage_x_um": float(initial_stage_position[0]), - "stage_y_um": float(initial_stage_position[1]), - "pixel_size_um": pixel_size_um, + await self.manager.broadcast( + { + "type": "marking_image", + "data": { + "session_id": session_id, + "image_b64": b64, + "width": w, + "height": h, + "initial_markers": normalized, + "default_role": default_role, + "stage_x_um": float(initial_stage_position[0]), + "stage_y_um": float(initial_stage_position[1]), + "pixel_size_um": pixel_size_um, + }, } - }) + ) logger.info( f"Marking session {session_id} started, image {w}x{h}, " @@ -478,7 +659,7 @@ async def start_marking_session( ) return session_id - async def wait_for_marking(self, session_id: str, timeout: float = None) -> list: + async def wait_for_marking(self, session_id: str, timeout: float | None = None) -> list: """ Wait for a marking session to complete. @@ -505,9 +686,9 @@ async def wait_for_marking(self, session_id: str, timeout: float = None) -> list markers = session["markers"] initial_pos = session["initial_stage_position"] - pixel_size = session["pixel_size_um"] + session["pixel_size_um"] h, w = session["image_shape"][:2] - center_x, center_y = w / 2, h / 2 + _center_x, _center_y = w / 2, h / 2 # Convert to embryo entries. Carries role + source so callers can # register each embryo with the right experimental classification. @@ -515,18 +696,24 @@ async def wait_for_marking(self, session_id: str, timeout: float = None) -> list embryos = [] for m in markers: px, py = m["pixelX"], m["pixelY"] - embryos.append({ - "embryo_number": m["number"], - "embryo_id": m.get("embryo_id") or f"embryo_{m['number']:03d}", - "pixel_position": (px, py), - "pixel_x": px, - "pixel_y": py, - "initial_stage_position": initial_pos, - "role": m.get("role", default_role), - "source": m.get("source", "manual"), - "confidence": m.get("confidence"), - "marking_timestamp": m.get("timestamp", datetime.now().isoformat()), - }) + embryos.append( + { + "embryo_number": m["number"], + # Unpadded to match the live convention used everywhere else + # (detection_tools registers embryos as f"embryo_{n}"). A + # zero-padded fallback here produced ids like "embryo_002" + # that never matched the stored "embryo_2". + "embryo_id": m.get("embryo_id") or f"embryo_{m['number']}", + "pixel_position": (px, py), + "pixel_x": px, + "pixel_y": py, + "initial_stage_position": initial_pos, + "role": m.get("role", default_role), + "source": m.get("source", "manual"), + "confidence": m.get("confidence"), + "marking_timestamp": m.get("timestamp", datetime.now().isoformat()), + } + ) # Clean up del self._marking_sessions[session_id] @@ -538,7 +725,7 @@ async def push_volume_3d( volume: np.ndarray, masks: np.ndarray, uid: str, - metadata: Optional[Dict] = None, + metadata: dict | None = None, ): """ Push a 3D segmentation volume to connected clients @@ -562,24 +749,54 @@ async def push_volume_3d( volume_data = Volume3DData( uid=uid, - data_type='segmentation_3d', + data_type="segmentation_3d", timestamp=datetime.now().isoformat(), volume=volume, masks=masks, colors=colors, - metadata=metadata or {} + metadata=metadata or {}, ) # Store the 3D volume self.store.add_volume_3d(volume_data) # Broadcast notification to clients (without the heavy data) - await self.manager.broadcast({ - 'type': 'volume_3d', - 'data': volume_data.to_info_dict() - }) + await self.manager.broadcast({"type": "volume_3d", "data": volume_data.to_info_dict()}) - logger.info(f"Pushed 3D volume {uid} ({volume.shape}) to {len(self.manager.active_connections)} clients") + logger.info( + f"Pushed 3D volume {uid} ({volume.shape}) to" + f" {len(self.manager.active_connections)} clients" + ) + + async def open_volume_in_browser( + self, + embryo_id: str, + timepoint: int, + view: str = "3d_viewer", + ) -> int: + """Ask every connected browser to open the in-browser volume viewer. + + This is the web-native replacement for the old napari ``view_volume``: + the agent triggers the existing ProjectionViewer (WebGL raymarcher + + projections) instead of launching a desktop Qt window that would block + the shared agent/web event loop. Returns the number of clients notified. + """ + await self.manager.broadcast( + { + "type": "open_volume", + "embryo_id": embryo_id, + "timepoint": timepoint, + "view": view, + } + ) + n = len(self.manager.active_connections) + logger.info( + "Requested browser open_volume for %s t%s (%d client(s))", + embryo_id, + timepoint, + n, + ) + return n async def on_start(self): """Start the visualization server""" @@ -592,15 +809,25 @@ async def on_start(self): # off to uvicorn (whose bind error surfaces inside a background # task and produces an unhelpful log line). import socket + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + # Match uvicorn's own bind semantics. uvicorn sets SO_REUSEADDR before it + # binds, so a bare preflight bind WITHOUT it is *stricter* than the real + # server: when a previous instance has just exited, its browser/websocket + # connections linger in TIME_WAIT holding this local port, and a plain + # bind() fails with EADDRINUSE even though uvicorn would bind fine. That + # false positive was the recurring "port in use" on quick restarts. With + # SO_REUSEADDR the preflight now fails only on a genuine live listener + # (a real second instance) — exactly when uvicorn would also fail. + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) try: sock.bind((self.host, self.port)) except OSError: raise OSError( - f"Port {self.port} is already in use. " - "Is another instance of the agent running? " - "Close it first and try again." - ) + f"Port {self.port} is already in use — another instance may be running. " + f"Free it with: fuser -k {self.port}/tcp " + f"(or: lsof -ti:{self.port} | xargs -r kill), then try again." + ) from None finally: sock.close() @@ -662,10 +889,10 @@ async def on_stop(self): self._server_task = None logger.info("Visualization server stopped") - async def health_check(self) -> Dict: + async def health_check(self) -> dict: """Return health status with connected client count.""" base = await super().health_check() - base['connected_clients'] = len(self.manager.active_connections) + base["connected_clients"] = len(self.manager.active_connections) return base async def run_forever(self): @@ -682,20 +909,20 @@ def signal_handler(*args): loop = asyncio.get_running_loop() signals_installed = False - if hasattr(signal, 'SIGINT'): + if hasattr(signal, "SIGINT"): try: loop.add_signal_handler(signal.SIGINT, signal_handler) signals_installed = True except NotImplementedError: pass - if hasattr(signal, 'SIGTERM'): + if hasattr(signal, "SIGTERM"): try: loop.add_signal_handler(signal.SIGTERM, signal_handler) except NotImplementedError: pass - if sys.platform == 'win32' and not signals_installed: + if sys.platform == "win32" and not signals_installed: signal.signal(signal.SIGINT, signal_handler) signal.signal(signal.SIGTERM, signal_handler) @@ -704,21 +931,17 @@ def signal_handler(*args): logger.info(f"Server running at http://{self.host}:{self.port} - Press Ctrl+C to stop") try: - if sys.platform == 'win32': + if sys.platform == "win32": while not stop_event.is_set(): try: - await asyncio.wait_for( - asyncio.shield(self._server_task), - timeout=0.5 - ) + await asyncio.wait_for(asyncio.shield(self._server_task), timeout=0.5) break except asyncio.TimeoutError: continue else: stop_task = asyncio.create_task(stop_event.wait()) done, pending = await asyncio.wait( - [self._server_task, stop_task], - return_when=asyncio.FIRST_COMPLETED + [self._server_task, stop_task], return_when=asyncio.FIRST_COMPLETED ) for task in pending: task.cancel() diff --git a/gently/ui/web/static/css/agent-chat.css b/gently/ui/web/static/css/agent-chat.css new file mode 100644 index 00000000..3fd7fc82 --- /dev/null +++ b/gently/ui/web/static/css/agent-chat.css @@ -0,0 +1,507 @@ +/* Floating agent-chat window — the web-side control surface. + Restrained, professional styling for a lab instrument. */ + +/* ── Collapsed rail (always-present agent affordance) ────────── + When the docked panel is collapsed it shrinks to this thin vertical rail: + a spark glyph (click to open), a vertical "Agent" label, plus the connection + dot + unseen-activity badge so a tucked-away agent can still signal it woke + or has something pending. Shown only via the body.chat-docked rules below. */ +.agent-rail { + display: none; /* shown only when collapsed (see :not(.open) rule) */ + flex-direction: column; align-items: center; + width: 100%; height: 100%; padding: 8px 0 0; + border: none; background: transparent; + cursor: pointer; +} +/* Icon-only activity-bar button: contained rounded hover target (VS Code/Google + style), with the connection state + activity count attached to the icon. */ +.agent-rail-icon { + position: relative; + display: flex; align-items: center; justify-content: center; + width: 40px; height: 40px; border-radius: 10px; + color: var(--text-muted); + transition: background 0.15s ease, color 0.15s ease; +} +.agent-rail:hover .agent-rail-icon { background: var(--bg-hover); color: var(--text); } +.agent-rail-spark { display: flex; } +/* Presence dot in the icon's corner (avatar-with-status pattern). */ +.agent-rail-dot { + position: absolute; right: 5px; bottom: 5px; + width: 8px; height: 8px; border-radius: 50%; + background: var(--text-muted); + border: 2px solid var(--bg-card); box-sizing: content-box; +} +.agent-rail-dot.ok { background: var(--accent-green); } +/* Unseen-activity count badge in the icon's top-right corner. */ +.agent-rail-badge { + position: absolute; top: -4px; right: -4px; + min-width: 16px; height: 16px; padding: 0 4px; + border-radius: 999px; background: var(--accent-purple); color: #fff; + font-size: 10px; font-weight: 700; line-height: 16px; text-align: center; + border: 2px solid var(--bg-card); box-sizing: content-box; +} +.agent-rail-badge.hidden { display: none; } + +/* ── Docked agent panel ──────────────────────────────────── + The panel is always docked (body.chat-docked, set on load): a real column + that pushes .app-main, never a float over content. Collapsing it (header + Agent toggle / Ctrl+J / ×) drops it to width 0 to reclaim space. + + The base .agent-chat rules below (absolute, translateX slide) are the + pre-JS / no-chat-docked fallback so the panel stays parked off-screen until + restorePrefs() docks it; body.chat-docked overrides them into the column. */ +.agent-chat { + position: absolute; + top: 0; right: 0; bottom: 0; + width: var(--chat-w, 460px); + max-width: 92vw; + display: flex; + flex-direction: column; + background: var(--bg-card); + border-left: 1px solid var(--border); + box-shadow: -16px 0 40px -16px var(--panel-edge-shadow); + z-index: 50; + overflow: hidden; + transform: translateX(100%); + transition: transform 0.22s cubic-bezier(0.22, 1, 0.36, 1); + will-change: transform; + font-family: 'Inter Tight', -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif; +} +.agent-chat.open { transform: translateX(0); } + +/* Docked: a real pushing column — no float shadow, just a seam. */ +body.chat-docked .agent-chat { + position: relative; + transform: none; + box-shadow: none; + border-left: 1px solid var(--border-strong); + flex: 0 0 auto; + transition: none; + z-index: auto; +} +/* Collapsed: shrink to the rail. Hide every child, then re-show only the rail. */ +body.chat-docked .agent-chat:not(.open) { + width: 48px; flex: 0 0 48px; overflow: hidden; +} +body.chat-docked .agent-chat:not(.open) > * { display: none; } +body.chat-docked .agent-chat:not(.open) > .agent-rail { display: flex; } + +@media (prefers-reduced-motion: reduce) { + .agent-chat { transition: opacity 0.12s ease; } +} + +/* Left-edge resize handle (thin seam, generous hit area). */ +.agent-chat-resize { + position: absolute; left: -2px; top: 0; bottom: 0; width: 6px; + cursor: ew-resize; z-index: 3; +} +.agent-chat-resize::after { + content: ''; position: absolute; left: 2px; top: 0; bottom: 0; width: 1px; + background: transparent; transition: background 0.12s ease; +} +.agent-chat-resize:hover::after, .agent-chat-resize.dragging::after { background: var(--accent); } + +.agent-control-banner.hidden { display: none; } + +/* ── Header ─────────────────────────────────────────────── */ +.agent-chat-header { + display: flex; + align-items: center; + gap: 10px; + padding: 12px 14px; + border-bottom: 1px solid var(--border); +} +.agent-chat-id { display: flex; align-items: center; gap: 9px; } +.agent-chat-mark { display: block; } +.agent-chat-title { font-weight: 600; font-size: 14px; color: var(--text); letter-spacing: 0.01em; } +.agent-chat-user { + font-size: 11px; color: var(--text-muted); + padding-left: 7px; margin-left: 1px; + border-left: 1px solid var(--border); +} +.agent-chat-user:empty { display: none; } +.agent-chat-signout { + background: none; border: none; + color: var(--text-muted); font-size: 11px; cursor: pointer; + padding: 0 4px; font-family: inherit; +} +.agent-chat-signout:hover { color: var(--text); text-decoration: underline; } + +.agent-chat-conn { + margin-left: auto; + font-size: 11px; + font-weight: 500; + padding: 3px 9px; + border-radius: 999px; + border: 1px solid var(--border); + color: var(--text-muted); + white-space: nowrap; +} +.agent-chat-conn.ac-conn-ok { color: var(--accent-green); border-color: rgba(74, 222, 128, 0.35); } +.agent-chat-conn.ac-conn-bad { color: var(--accent-orange, #fb923c); border-color: rgba(251, 146, 60, 0.35); } +.agent-chat-close { + background: none; border: none; + color: var(--text-muted); line-height: 1; + cursor: pointer; padding: 2px; border-radius: 5px; + display: inline-flex; align-items: center; +} +.agent-chat-close svg { display: block; } +.agent-chat-close:hover { color: var(--text); background: var(--bg-hover); } + +/* ── Control banner ─────────────────────────────────────── */ +.agent-control-banner { + display: flex; align-items: center; gap: 10px; + padding: 9px 14px; + background: rgba(251, 146, 60, 0.10); + border-bottom: 1px solid var(--border); + color: var(--accent-orange, #fb923c); + font-size: 12.5px; +} +.ac-take-control { + margin-left: auto; + padding: 4px 12px; border-radius: 7px; + border: 1px solid var(--accent); + background: var(--accent); color: #fff; + cursor: pointer; font-size: 12px; font-weight: 600; +} +.ac-take-control:hover { background: var(--accent-hover); } + +/* ── Transcript ─────────────────────────────────────────── */ +.agent-chat-log { + flex: 1 1 auto; + overflow-y: auto; + padding: 16px; + display: flex; flex-direction: column; gap: 14px; + font-size: 13.5px; line-height: 1.6; + color: var(--text); +} + +.ac-turn { display: flex; flex-direction: column; } +.ac-role { + font-size: 10.5px; font-weight: 600; + letter-spacing: 0.06em; text-transform: uppercase; + color: var(--accent-purple); + margin-bottom: 4px; +} +/* Sender name on user bubbles: muted so it doesn't compete with the message. */ +.ac-role-user { color: var(--text-muted); } +.ac-turn-agent .ac-content { color: var(--text); } +.ac-turn-agent .ac-content code { + font-family: 'JetBrains Mono', ui-monospace, monospace; + font-size: 12px; + background: rgba(127, 127, 127, 0.14); + padding: 1px 5px; border-radius: 4px; +} + +/* User: right-aligned, subtle accent block (not a loud bubble). Your own + messages use the blue accent; other participants get a neutral bubble so a + shared chat is easy to read at a glance. */ +.ac-turn-user { align-items: flex-end; } +.ac-turn-user .ac-content { + background: rgba(96, 165, 250, 0.12); + border: 1px solid rgba(96, 165, 250, 0.22); + color: var(--text); + padding: 7px 11px; + border-radius: 10px 10px 2px 10px; + max-width: 88%; + white-space: pre-wrap; word-wrap: break-word; +} +.ac-turn-user.ac-from-other .ac-content { + background: rgba(127, 127, 127, 0.12); + border-color: rgba(127, 127, 127, 0.24); +} + +/* ── Autonomous (wake) turns ────────────────────────────── */ +.ac-autonomous-banner { + display: flex; align-items: center; gap: 8px; + align-self: stretch; + margin: 2px 0; + padding: 6px 10px; + font-size: 11.5px; font-weight: 500; + color: var(--accent-purple); + background: rgba(167, 139, 250, 0.10); + border: 1px solid rgba(167, 139, 250, 0.28); + border-radius: 8px; +} +.ac-autonomous-dot { + width: 7px; height: 7px; border-radius: 50%; + background: var(--accent-purple); + box-shadow: 0 0 0 3px rgba(167, 139, 250, 0.20); + flex: 0 0 auto; +} +/* Autonomous agent bubbles get an accent rail + a distinct role label. */ +.ac-turn-autonomous { border-left: 2px solid rgba(167, 139, 250, 0.45); padding-left: 8px; } +.ac-turn-autonomous .ac-role { color: var(--accent-purple); } + +/* ── Activity indicator ─────────────────────────────────── */ +.ac-activity { + display: flex; align-items: center; gap: 9px; + color: var(--text-muted); font-size: 12.5px; +} +.ac-dots { display: inline-flex; gap: 4px; } +.ac-dots i { + width: 5px; height: 5px; border-radius: 50%; + background: var(--accent); + display: inline-block; + animation: ac-blink 1.2s infinite both; +} +.ac-dots i:nth-child(2) { animation-delay: 0.18s; } +.ac-dots i:nth-child(3) { animation-delay: 0.36s; } +@keyframes ac-blink { 0%, 80%, 100% { opacity: 0.22; } 40% { opacity: 1; } } + +/* ── Tool calls ─────────────────────────────────────────── */ +.ac-tool { + display: flex; align-items: center; gap: 8px; + font-family: 'JetBrains Mono', ui-monospace, monospace; + font-size: 11.5px; + color: var(--text-muted); + padding: 6px 10px; + border: 1px solid var(--border); + border-radius: 8px; + background: rgba(127, 127, 127, 0.05); +} +.ac-tool-name { color: var(--text); } +.ac-tool-meta { color: var(--text-muted); } +.ac-tool-check { color: var(--accent-green); } + +/* Multi-line tool rows: head (icon + name + meta) over args / summary. */ +.ac-tool { flex-direction: column; align-items: stretch; gap: 4px; } +.ac-tool-head { display: flex; align-items: center; gap: 8px; } +.ac-tool-args { color: var(--text-muted); padding-left: 19px; word-break: break-word; } +.ac-tool-summary { color: var(--text-muted); padding-left: 19px; word-break: break-word; } +.ac-tool-summary-err, .ac-tool-warn { color: var(--accent-orange, #fb923c); } +.ac-tool-err { border-color: rgba(251, 146, 60, 0.35); } +.ac-tool-spin { + width: 11px; height: 11px; border-radius: 50%; + border: 1.6px solid var(--border); + border-top-color: var(--accent); + display: inline-block; + animation: ac-spin 0.7s linear infinite; +} +@keyframes ac-spin { to { transform: rotate(360deg); } } + +/* ── System lines / notifications ───────────────────────── */ +.ac-system { + align-self: center; + font-size: 11.5px; color: var(--text-muted); + text-align: center; max-width: 95%; +} +.ac-level-error { color: var(--color-danger, #f87171); } +.ac-level-warning { color: var(--accent-orange, #fb923c); } +.ac-level-success { color: var(--accent-green); } + +/* ── Ask pointer (ux_v2) ─────────────────────────────────── */ +/* Compact transcript reference shown instead of full ask cards when the main + stage (AskStage / #ask-stage) owns the answer surface. */ +.ac-ask-pointer { + align-self: center; + font-size: 11.5px; color: var(--text-muted); + text-align: center; font-style: italic; + opacity: 0.75; +} +.ac-ask-pointer-answered { opacity: 0.35; } + +/* ── Choice picker ──────────────────────────────────────── */ +.ac-choice { + display: flex; flex-direction: column; gap: 7px; + padding: 12px; + border: 1px solid var(--border); + border-radius: 10px; + background: rgba(127, 127, 127, 0.04); +} +.ac-choice-q { color: var(--text); font-weight: 500; } +.ac-choice-opt { + text-align: left; + padding: 9px 12px; border-radius: 8px; + border: 1px solid var(--border); + background: var(--bg-card); color: var(--text); + cursor: pointer; + display: flex; flex-direction: column; gap: 2px; + transition: border-color 0.12s ease, background 0.12s ease; +} +.ac-choice-opt:hover:not(:disabled) { border-color: var(--accent); background: var(--bg-hover); } +.ac-choice-opt:disabled { opacity: 0.5; cursor: default; } +.ac-choice-label { font-weight: 600; font-size: 13px; } +.ac-choice-desc { font-size: 12px; color: var(--text-muted); } +.ac-choice-picked { border-color: var(--accent-green); background: rgba(74, 222, 128, 0.08); } +.ac-choice-wake { + border-color: rgba(167, 139, 250, 0.45); + border-left: 3px solid var(--accent-purple); + background: rgba(167, 139, 250, 0.06); +} +.ac-choice-origin { + font-size: 10.5px; font-weight: 600; letter-spacing: 0.04em; text-transform: uppercase; + color: var(--accent-purple); margin-bottom: 2px; +} + +/* Sticky ASK-approval slot: pinned above the composer so it never scrolls away. */ +.ac-pending { + flex: 0 0 auto; + border-top: 1px solid var(--border); + background: var(--bg-card); + padding: 8px 12px 0; +} +.ac-pending.hidden { display: none; } +.ac-pending .ac-choice { margin-bottom: 8px; } + +/* "↓ N new" jump-to-bottom pill (shown when scrolled up during streaming). */ +.ac-jump { + position: absolute; + left: 50%; transform: translateX(-50%); + bottom: 74px; + padding: 4px 12px; border-radius: 999px; + border: 1px solid var(--accent); + background: var(--bg-card); color: var(--accent); + font: 600 11.5px/1.4 'Inter Tight', -apple-system, sans-serif; + cursor: pointer; z-index: 4; + box-shadow: 0 4px 14px rgba(0, 0, 0, 0.35); +} +.ac-jump.hidden { display: none; } + +/* ── Applied-spec card ──────────────────────────────────── */ +.ac-spec { + border: 1px solid var(--border); + border-radius: 10px; + padding: 11px 13px; + background: rgba(127, 127, 127, 0.04); + font-size: 12.5px; +} +.ac-spec-title { + font-weight: 600; color: var(--accent-purple); + font-size: 11px; letter-spacing: 0.04em; text-transform: uppercase; + margin-bottom: 6px; +} +.ac-spec-row { display: flex; justify-content: space-between; gap: 16px; padding: 1px 0; color: var(--text-muted); } +.ac-spec-row span:last-child { color: var(--text); font-family: 'JetBrains Mono', ui-monospace, monospace; } + +/* ── Composer ───────────────────────────────────────────── */ +.agent-chat-input { + display: flex; gap: 8px; + padding: 12px; + border-top: 1px solid var(--border); + position: relative; /* anchor for the autocomplete dropdown */ +} + +/* ── Autocomplete dropdown ──────────────────────────────── */ +.ac-complete { + position: absolute; + left: 12px; right: 12px; bottom: calc(100% + 4px); + max-height: 240px; overflow-y: auto; + background: var(--bg-card); + border: 1px solid var(--border); + border-radius: 9px; + box-shadow: 0 -8px 28px rgba(0, 0, 0, 0.45); + padding: 4px; + z-index: 5; +} +.ac-complete.hidden { display: none; } +.ac-complete-item { + display: flex; flex-direction: column; gap: 1px; + padding: 6px 9px; border-radius: 6px; + cursor: pointer; +} +.ac-complete-item.active, +.ac-complete-item:hover { background: var(--bg-hover, rgba(127, 127, 127, 0.12)); } +.ac-complete-name { + font-family: 'JetBrains Mono', ui-monospace, monospace; + font-size: 12.5px; color: var(--accent); +} +.ac-complete-desc { + font-size: 11.5px; color: var(--text-muted); + white-space: nowrap; overflow: hidden; text-overflow: ellipsis; +} +.agent-chat-input textarea { + flex: 1 1 auto; resize: none; + border: 1px solid var(--border); border-radius: 9px; + background: var(--bg-dark); color: var(--text); + padding: 9px 11px; + font-family: inherit; font-size: 13.5px; line-height: 1.45; + max-height: 140px; +} +.agent-chat-input textarea::placeholder { color: var(--text-muted); } +.agent-chat-input textarea:focus { outline: none; border-color: var(--accent); } +.agent-chat-input textarea:disabled { opacity: 0.55; } +/* Circular icon button (ChatGPT/Claude style): an up-arrow to send, which + morphs into a stop square (.is-stop) while a cancellable turn is running. */ +.agent-chat-send { + flex: 0 0 auto; align-self: flex-end; + width: 36px; height: 36px; padding: 0; border-radius: 50%; + display: inline-flex; align-items: center; justify-content: center; + border: none; background: var(--accent); color: #fff; + cursor: pointer; transition: background 0.12s ease, opacity 0.12s ease; +} +.agent-chat-send:hover:not(:disabled) { background: var(--accent-hover); } +.agent-chat-send:disabled { opacity: 0.4; cursor: default; } +.agent-chat-send svg { display: block; } +/* Toggle which glyph shows; same filled circle in both states (the icon is the + signal), matching how Claude/ChatGPT morph the composer button. */ +.agent-chat-send .ac-icon-stop { display: none; } +.agent-chat-send.is-stop .ac-icon-send { display: none; } +.agent-chat-send.is-stop .ac-icon-stop { display: block; } + +/* ── Queued-message panel (type-while-busy) ─────────────── */ +.ac-queue { + margin: 0 12px 6px; + border: 1px solid var(--border); border-radius: 9px; + background: rgba(127, 127, 127, 0.06); + padding: 6px; font-size: 12px; +} +.ac-queue.hidden { display: none; } +.ac-queue-head { + display: flex; align-items: center; justify-content: space-between; + padding: 2px 4px 6px; color: var(--text-muted); +} +.ac-queue-clear { + background: none; border: none; color: var(--accent); + cursor: pointer; font-size: 11.5px; font-family: inherit; +} +.ac-queue-clear:hover { text-decoration: underline; } +.ac-queue-item { display: flex; align-items: center; gap: 8px; padding: 4px; } +.ac-queue-text { + flex: 1 1 auto; color: var(--text); + white-space: nowrap; overflow: hidden; text-overflow: ellipsis; +} +.ac-queue-remove { + flex: 0 0 auto; background: none; border: none; + color: var(--text-muted); cursor: pointer; font-size: 12px; line-height: 1; +} +.ac-queue-remove:hover { color: var(--color-danger, #f87171); } + +/* ── Rendered markdown (mdToHtml output, ac-md-* classes) ────────────────── + Shared by the chat transcript and the ux_v2 plan-wizard activity feed — the + same renderer feeds both, so these styles cover headings, lists, tables, + code blocks, quotes and links the agent emits. */ +.ac-md { line-height: 1.55; } +.ac-md > :first-child { margin-top: 0; } +.ac-md > :last-child { margin-bottom: 0; } +.ac-md-h1, .ac-md-h2, .ac-md-h3, .ac-md-h4, .ac-md-h5, .ac-md-h6 { + margin: 14px 0 6px; font-weight: 650; line-height: 1.3; letter-spacing: -.01em; color: var(--text); +} +.ac-md-h1 { font-size: 1.25em; } +.ac-md-h2 { font-size: 1.15em; } +.ac-md-h3 { font-size: 1.05em; } +.ac-md-h4, .ac-md-h5, .ac-md-h6 { font-size: 1em; } +.ac-md-p { margin: 7px 0; } +.ac-md-ul, .ac-md-ol { margin: 7px 0; padding-left: 22px; } +.ac-md-li { margin: 3px 0; } +.ac-md-quote { + margin: 8px 0; padding: 4px 12px; border-left: 3px solid var(--border, #e4e9f0); + color: var(--text-muted); font-style: italic; +} +.ac-md-hr { border: 0; border-top: 1px solid var(--border, #e4e9f0); margin: 12px 0; } +.ac-md-link { color: var(--accent, #2f6df6); text-decoration: underline; text-underline-offset: 2px; } +.ac-md-pre { + margin: 8px 0; padding: 10px 12px; border-radius: 8px; overflow-x: auto; + background: var(--bg, #f6f8fb); border: 1px solid var(--border, #e4e9f0); +} +.ac-md-pre .ac-md-code-block, .ac-md-pre code { + font-family: 'JetBrains Mono', ui-monospace, monospace; font-size: 12px; + color: var(--text); background: none; padding: 0; white-space: pre; +} +/* GFM tables — wrapped so a wide table scrolls instead of blowing out the column */ +.ac-md-table-wrap { margin: 9px 0; overflow-x: auto; border: 1px solid var(--border, #e4e9f0); border-radius: 8px; } +.ac-md-table { border-collapse: collapse; width: 100%; font-size: 12.5px; } +.ac-md-table th, .ac-md-table td { padding: 6px 10px; text-align: left; border-bottom: 1px solid var(--border, #e4e9f0); border-right: 1px solid var(--border, #e4e9f0); } +.ac-md-table th:last-child, .ac-md-table td:last-child { border-right: 0; } +.ac-md-table tr:last-child td { border-bottom: 0; } +.ac-md-table thead th { background: var(--bg, #f6f8fb); font-weight: 650; color: var(--text); } diff --git a/gently/ui/web/static/css/ask-stage.css b/gently/ui/web/static/css/ask-stage.css new file mode 100644 index 00000000..dc15c623 --- /dev/null +++ b/gently/ui/web/static/css/ask-stage.css @@ -0,0 +1,56 @@ +/* Main-stage ask surface (ux_v2): the agent's current pending ask, rendered + prominently outside the chat transcript. Reuses the .ac-choice card markup + from agent-chat.css; this file frames the stage container and adds the + shared free-text ("Something else…") escape styling. Only #ask-stage is + gated behind the flag, so loading this CSS unconditionally is harmless. */ + +.ask-stage { margin: 14px 16px 0; } +.ask-stage.hidden { display: none; } + +.ask-stage .ac-choice { + border: 1px solid var(--border, #e4e9f0); + border-radius: 14px; + padding: 16px 18px; + background: var(--surface, #fff); + box-shadow: 0 8px 28px rgba(15, 23, 42, .08); +} +.ask-stage .ac-choice-q { + font-size: 1.02rem; + font-weight: 600; + margin-bottom: 12px; +} + +/* Free-text "Something else…" escape — present on ask cards in BOTH surfaces. */ +.ac-choice-otherwrap { margin-top: 6px; } +.ac-choice-other.hidden, +.ac-choice-otherform.hidden { display: none; } +.ac-choice-otherform { display: flex; gap: 6px; align-items: center; margin-top: 4px; } +.ac-choice-otherinput { + flex: 1; min-width: 0; + padding: 8px 10px; + border: 1px solid var(--border, #cbd5e1); + border-radius: 8px; + font: inherit; + background: var(--surface, #fff); + color: inherit; +} +.ac-choice-otherinput:focus { outline: none; border-color: var(--accent, #2f6df6); } +.ac-choice-othergo { + border: 0; cursor: pointer; + background: var(--accent, #2f6df6); color: #fff; + border-radius: 8px; padding: 8px 12px; line-height: 1; +} + +/* Per-field provenance tag on imaging-spec rows (Phase 3b): shows where an + inferred value came from, e.g. "inferred · medium". */ +.ac-spec-src { + margin-left: 6px; + font-size: 10px; + letter-spacing: .02em; + color: var(--text-muted, #94a3b8); + background: var(--bg-hover, #f1f5f9); + border-radius: 999px; + padding: 1px 7px; + white-space: nowrap; + vertical-align: middle; +} diff --git a/gently/ui/web/static/css/boot-banner.css b/gently/ui/web/static/css/boot-banner.css new file mode 100644 index 00000000..1949805e --- /dev/null +++ b/gently/ui/web/static/css/boot-banner.css @@ -0,0 +1,85 @@ +/* Device-layer boot banner (boot-banner.js) - non-modal, follows MMCore boot. + A compact bottom-center pill; the Devices panel is the full on-demand console. */ +.boot-banner { + position: fixed; + left: 50%; + bottom: 22px; + transform: translateX(-50%); + z-index: 10020; + display: flex; + align-items: center; + gap: 0.6rem; + max-width: min(560px, 92vw); + padding: 0.55rem 0.7rem 0.55rem 0.9rem; + border: 1px solid var(--border); + border-radius: 999px; + background: var(--bg-card); + color: var(--text); + box-shadow: 0 10px 30px -10px rgba(0, 0, 0, 0.45); + font-size: 0.85rem; +} +/* Honor the hidden attribute: `display: flex` above otherwise overrides the UA + `[hidden] { display: none }`, so hide() (which sets el.hidden = true) left the + pill on screen as a bare dot. Higher specificity than `.boot-banner`, so it + wins and the banner fully disappears when hidden. */ +.boot-banner[hidden] { + display: none; +} +.boot-banner-dot { + width: 9px; + height: 9px; + border-radius: 50%; + flex: none; + background: var(--accent); +} +.boot-banner.booting .boot-banner-dot { + animation: boot-pulse 1.1s ease-in-out infinite; +} +@keyframes boot-pulse { + 0%, 100% { opacity: 0.35; } + 50% { opacity: 1; } +} +.boot-banner.ready { + border-color: var(--accent-green); +} +.boot-banner.ready .boot-banner-dot { + background: var(--accent-green); + animation: none; +} +.boot-banner.failed { + border-color: var(--accent-orange); +} +.boot-banner.failed .boot-banner-dot { + background: var(--accent-orange); + animation: none; +} +.boot-banner-text { + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +.boot-banner-btn { + flex: none; + border: 1px solid var(--border); + background: var(--bg-hover); + color: var(--text); + border-radius: 8px; + padding: 0.25rem 0.6rem; + font-size: 0.78rem; + cursor: pointer; +} +.boot-banner-btn:hover:not(:disabled) { background: var(--bg-card); } +.boot-banner-btn:disabled { opacity: 0.5; cursor: default; } +.boot-banner-x { + flex: none; + border: 0; + background: none; + color: var(--text-muted); + font-size: 1.1rem; + line-height: 1; + cursor: pointer; + padding: 0 0.15rem; +} +.boot-banner-x:hover { color: var(--text); } diff --git a/gently/ui/web/static/css/campaigns.css b/gently/ui/web/static/css/campaigns.css index bc4f36c2..b874ab44 100644 --- a/gently/ui/web/static/css/campaigns.css +++ b/gently/ui/web/static/css/campaigns.css @@ -1284,6 +1284,89 @@ font-size: 0.7rem; } +/* Section title with a right-aligned action (e.g. Edit) */ +.detail-section-title--row { + display: flex; + align-items: center; + justify-content: space-between; +} +.detail-section-action { display: inline-flex; } +.spec-edit-btn { + border: 1px solid var(--border); + background: transparent; + color: var(--text-muted); + border-radius: 6px; + padding: 2px 8px; + font: inherit; + font-size: 0.62rem; + letter-spacing: 0.3px; + cursor: pointer; +} +.spec-edit-btn:hover { color: var(--text); border-color: var(--text-muted); } + +/* Editable imaging-spec form */ +.spec-editor { display: flex; flex-direction: column; gap: 6px; } +.spec-edit-row { + display: grid; + grid-template-columns: 38% 1fr; + align-items: center; + gap: 10px; +} +.spec-edit-label { + color: var(--text-muted); + font-size: 0.7rem; +} +.spec-edit-row--empty .spec-edit-label::after { + content: ' • set'; + color: var(--accent-orange, #d98324); + font-size: 0.6rem; + opacity: 0.8; +} +.spec-edit-field { display: flex; align-items: center; gap: 5px; } +.spec-edit-input { + flex: 1; + min-width: 0; + background: var(--bg, #fff); + border: 1px solid var(--border); + border-radius: 6px; + padding: 4px 7px; + color: var(--text); + font-family: 'SF Mono', 'Fira Code', monospace; + font-size: 0.7rem; +} +.spec-edit-input:focus { + outline: none; + border-color: var(--accent, #2f6df6); +} +.spec-edit-row--empty .spec-edit-input { + border-style: dashed; +} +.spec-edit-unit { color: var(--text-muted); font-size: 0.66rem; } +.spec-edit-error { + color: var(--accent-orange, #d98324); + font-size: 0.68rem; + margin-top: 2px; +} +.spec-edit-actions { display: flex; gap: 8px; margin-top: 8px; } +.spec-save-btn, .spec-cancel-btn { + border-radius: 7px; + padding: 5px 14px; + font: inherit; + font-size: 0.72rem; + font-weight: 600; + cursor: pointer; +} +.spec-save-btn { + background: var(--accent, #2f6df6); + color: #fff; + border: 0; +} +.spec-cancel-btn { + background: transparent; + color: var(--text-muted); + border: 1px solid var(--border); +} + /* Dependencies / Dependents chips */ .dep-list { display: flex; @@ -1352,6 +1435,109 @@ font-size: 0.62rem; color: var(--text-muted); } +.detail-session-right { + display: flex; + align-items: center; + gap: 6px; +} +.detail-session-empty { + color: var(--text-muted); + font-size: 0.75rem; + font-style: italic; +} + +/* Session link button (header action) */ +.session-link-btn { + border: 1px solid var(--border); + background: transparent; + color: var(--text-muted); + border-radius: 6px; + padding: 2px 8px; + font: inherit; + font-size: 0.65rem; + cursor: pointer; + white-space: nowrap; + transition: color 0.15s, border-color 0.15s; +} +.session-link-btn:hover { + color: var(--text); + border-color: var(--accent, #2f6df6); +} + +/* Per-session delink button */ +.session-delink-btn { + background: transparent; + border: none; + color: var(--text-muted); + cursor: pointer; + font-size: 1rem; + line-height: 1; + padding: 1px 4px; + border-radius: 4px; + opacity: 0.45; + transition: opacity 0.15s, color 0.15s, background 0.15s; +} +.session-delink-btn:hover { + opacity: 1; + color: var(--accent-orange, #f97316); + background: rgba(249,115,22,0.12); +} + +/* Inline session picker */ +.session-picker { + margin-top: 8px; + padding: 10px; + background: rgba(0,0,0,0.18); + border: 1px solid var(--border); + border-radius: 8px; +} +.session-picker--loading { + font-size: 0.75rem; + color: var(--text-muted); + font-style: italic; +} +.session-picker-select { + width: 100%; + background: var(--bg-dark, #0f172a); + color: var(--text); + border: 1px solid var(--border); + border-radius: 6px; + padding: 6px 8px; + font: inherit; + font-size: 0.75rem; + margin-bottom: 8px; + box-sizing: border-box; +} +.session-picker-actions { + display: flex; + gap: 6px; +} +.session-picker-link-btn { + flex: 1; + background: var(--accent, #2f6df6); + color: #fff; + border: none; + border-radius: 6px; + padding: 5px 12px; + font: inherit; + font-size: 0.72rem; + font-weight: 600; + cursor: pointer; + transition: opacity 0.15s; +} +.session-picker-link-btn:hover { opacity: 0.85; } +.session-picker-cancel-btn { + background: transparent; + color: var(--text-muted); + border: 1px solid var(--border); + border-radius: 6px; + padding: 5px 12px; + font: inherit; + font-size: 0.72rem; + cursor: pointer; + transition: color 0.15s; +} +.session-picker-cancel-btn:hover { color: var(--text); } /* ================================================================ SHARED COMPONENTS diff --git a/gently/ui/web/static/css/experiment.css b/gently/ui/web/static/css/experiment.css index 9b343378..ee40cddd 100644 --- a/gently/ui/web/static/css/experiment.css +++ b/gently/ui/web/static/css/experiment.css @@ -634,3 +634,1205 @@ transform: rotate(45deg); flex-shrink: 0; } + +/* ============================================================ + Operation Spine — data-driven tactic plan renderer + All classes namespaced ops- to avoid collision. + ============================================================ */ + +:root { + --ops-done: #34d399; + --ops-active: #f5a623; + --ops-plan: #5aa9e6; + --ops-live: #22d3ee; + --ops-mono: ui-monospace, SFMono-Regular, Menlo, monospace; +} + +.ops-wrap { + max-width: 760px; + margin: 0 auto; + padding: 8px 4px 48px; +} + +.ops-crumb { + font-family: var(--ops-mono); + font-size: 11px; + letter-spacing: 0.16em; + color: var(--text-muted); + text-transform: uppercase; + margin-bottom: 6px; +} + +.ops-title { + font-size: 22px; + font-weight: 600; + margin: 0 0 4px; + letter-spacing: -0.01em; + color: var(--text); +} + +.ops-meta { + font-family: var(--ops-mono); + font-size: 12px; + color: var(--text-muted); + margin-bottom: 4px; +} + +.ops-legend { + display: flex; + gap: 16px; + font-family: var(--ops-mono); + font-size: 10.5px; + color: var(--text-muted); + text-transform: uppercase; + letter-spacing: 0.08em; + margin: 14px 0 20px; +} + +.ops-legend i { + display: inline-block; + width: 8px; + height: 8px; + border-radius: 2px; + margin-right: 6px; + vertical-align: 1px; +} + +.ops-empty { + font-family: var(--ops-mono); + color: var(--text-muted); + font-size: 13px; + border: 1px dashed var(--border); + border-radius: 12px; + padding: 30px; + text-align: center; + margin-top: 18px; +} + +/* ---- Idle CTA — "set up an operation" ---------------------- */ + +.ops-setup-cta { + margin-top: 28px; + display: flex; + flex-direction: column; + align-items: flex-start; + gap: 16px; +} + +.ops-brief-btn { + background: var(--ops-plan, #5aa9e6); + color: #05101e; + border: none; + border-radius: 8px; + padding: 10px 22px; + font: inherit; + font-weight: 700; + font-size: 14px; + letter-spacing: 0.01em; + cursor: pointer; + transition: opacity 0.15s; +} +.ops-brief-btn:hover { opacity: 0.85; } + +.ops-chips-label { + font-family: var(--ops-mono); + font-size: 11px; + letter-spacing: 0.1em; + text-transform: uppercase; + color: var(--text-muted); +} + +.ops-chips { + display: flex; + flex-wrap: wrap; + gap: 8px; +} + +.ops-chip { + background: transparent; + color: var(--text-muted); + border: 1px solid var(--border, #30363d); + border-radius: 20px; + padding: 6px 14px; + font: inherit; + font-family: var(--ops-mono); + font-size: 12px; + cursor: pointer; + transition: color 0.15s, border-color 0.15s; +} +.ops-chip:hover { + color: var(--text, #e6edf3); + border-color: var(--ops-plan, #5aa9e6); +} + +/* ---- Spine ------------------------------------------------- */ + +.ops-spine { + position: relative; + margin-left: 14px; + padding-left: 30px; + border-left: 2px solid var(--border); +} + +/* Tactic node */ +.ops-node { + position: relative; + margin-bottom: 14px; +} + +/* Timeline dot — overlaps the spine border-left */ +.ops-node::before { + content: ""; + position: absolute; + left: -39px; + top: 4px; + width: 14px; + height: 14px; + border-radius: 50%; + background: var(--bg-card, #161b22); + border: 2px solid var(--border); +} + +.ops-node.done::before { + border-color: var(--ops-done); + background: var(--ops-done); +} + +.ops-node.active::before { + border-color: var(--ops-active); + background: var(--ops-active); + box-shadow: 0 0 0 5px rgba(245, 166, 35, 0.14); +} + +/* AUDIT: queued = cocked instrument — dashed blue dot (not empty) */ +.ops-node.planned::before { + border-color: var(--ops-plan); + border-style: dashed; + background: var(--bg-card, #161b22); +} + +/* ---- Stage label: "01 · in use" ----------------------------- */ + +.ops-stagelab { + font-family: var(--ops-mono); + font-size: 10px; + letter-spacing: 0.14em; + text-transform: uppercase; + color: var(--text-muted); + margin-bottom: 4px; + display: flex; + align-items: center; + gap: 6px; +} + +/* AUDIT: active row's WHOLE LEFT COLUMN amber (dot + seq + "IN USE") */ +.ops-node.active .ops-stagelab { color: var(--ops-active); } +.ops-node.planned .ops-stagelab { color: var(--ops-plan); } + +/* AUDIT: "next" badge on first queued tactic — cocked-instrument marker */ +.ops-next-badge { + display: inline-block; + padding: 1px 7px; + border-radius: 9px; + background: rgba(90, 169, 230, 0.15); + border: 1px solid rgba(90, 169, 230, 0.45); + color: var(--ops-plan); + font-size: 9px; + font-weight: 700; + letter-spacing: 0.1em; + text-transform: uppercase; +} + +/* ---- Card --------------------------------------------------- */ + +.ops-card { + background: var(--bg-card, #161b22); + border: 1px solid var(--border, #30363d); + border-radius: 12px; + padding: 13px 16px; +} + +/* AUDIT: colored LEFT EDGE per state */ +/* done — green left edge, transparent bg */ +.ops-node.done .ops-card { + background: transparent; + border-color: rgba(48, 54, 61, 0.5); + padding: 9px 16px; + border-left: 3px solid var(--ops-done); +} + +/* active — amber left edge + tinted gradient + FLATTENED (no card-in-card) */ +.ops-node.active .ops-card { + border-color: rgba(245, 166, 35, 0.3); + border-left: 3px solid var(--ops-active); + background: linear-gradient(180deg, rgba(245, 166, 35, 0.05), transparent), + var(--bg-card, #161b22); +} + +/* planned — blue left edge, dashed border */ +.ops-node.planned .ops-card { + background: transparent; + border-style: dashed; + border-left: 3px solid var(--ops-plan); + border-left-style: solid; + opacity: 0.85; +} + +/* Hairline rule between card header and live readouts (flattened layout) */ +.ops-rule { + border: none; + border-top: 1px solid rgba(245, 166, 35, 0.18); + margin: 10px 0; +} + +/* ---- Card contents ----------------------------------------- */ + +.ops-row { + display: flex; + align-items: center; + gap: 10px; + flex-wrap: wrap; +} + +.ops-tname { + font-size: 14.5px; + font-weight: 600; + color: var(--text); +} + +.ops-node.done .ops-tname { + font-weight: 500; + color: var(--text-muted); +} + +.ops-target { + font-family: var(--ops-mono); + font-size: 12.5px; + color: var(--ops-active); + font-weight: 600; +} + +.ops-tsum { + font-family: var(--ops-mono); + font-size: 11.5px; + color: var(--text-muted); + margin-left: auto; +} + +.ops-desc { + font-family: var(--ops-mono); + font-size: 11px; + color: var(--text-muted); + opacity: 0.75; + margin: 4px 0 0; + line-height: 1.4; +} + +/* ---- Gauge strip — AUDIT: flat on panel face, no card-in-card */ + +.ops-live-strip { + display: flex; + gap: 10px; + flex-wrap: wrap; +} + +.ops-gauge { + flex: 1; + min-width: 140px; + background: var(--bg-hover, #21262d); + border: 1px solid var(--border, #30363d); + border-radius: 9px; + padding: 9px 11px; +} + +.ops-gl { + font-family: var(--ops-mono); + font-size: 9.5px; + letter-spacing: 0.1em; + text-transform: uppercase; + color: var(--text-muted); +} + +/* AUDIT: mono instrument values */ +.ops-gv { + font-family: var(--ops-mono); + font-size: 18px; + font-weight: 650; + font-variant-numeric: tabular-nums; + color: var(--text); + margin-top: 2px; +} + +.ops-set { color: var(--ops-active); font-size: 13px; } +.ops-u { color: var(--text-muted); font-size: 12px; } + +.ops-gsub { + font-family: var(--ops-mono); + font-size: 11px; + color: var(--text-muted); + margin-top: 6px; +} + +.ops-tempbar { + height: 5px; + border-radius: 3px; + background: var(--bg-hover, #21262d); + margin-top: 8px; + overflow: hidden; +} + +.ops-tempbar > i { + display: block; + height: 100%; + background: linear-gradient(90deg, var(--ops-live), var(--ops-active)); +} + +/* ---- Scripted protocol — phase stepper --------------------- */ + +.ops-phases { + display: flex; + gap: 7px; + flex-wrap: wrap; + margin-top: 8px; +} + +.ops-ph { + flex: 1; + min-width: 110px; + border: 1px solid var(--border, #30363d); + border-radius: 8px; + padding: 9px 10px; + background: var(--bg-hover, #21262d); +} + +.ops-ph.done { border-color: rgba(52, 211, 153, 0.35); background: rgba(52, 211, 153, 0.05); } +.ops-ph.active { border-color: rgba(245, 166, 35, 0.45); background: rgba(245, 166, 35, 0.08); } +.ops-ph.todo { opacity: 0.6; } + +.ops-pht { + font-family: var(--ops-mono); + font-size: 10px; + letter-spacing: 0.08em; + text-transform: uppercase; + display: flex; + gap: 6px; + align-items: center; + color: var(--text-muted); +} + +/* AUDIT: ACTIVE phase is the HEADLINE — largest in its phase */ +.ops-ph.active .ops-pht { + font-size: 12px; + font-weight: 700; + color: var(--ops-active); +} + +.ops-pi { + width: 15px; + height: 15px; + border-radius: 50%; + display: grid; + place-items: center; + font-size: 9px; + font-weight: 700; + flex-shrink: 0; +} + +.ops-ph.done .ops-pi { background: var(--ops-done); color: #062b1d; } +.ops-ph.active .ops-pi { background: var(--ops-active); color: #3a2607; } +.ops-ph.todo .ops-pi { background: var(--bg-hover, #21262d); color: var(--text-muted); border: 1px solid var(--border); } + +.ops-phc { + font-family: var(--ops-mono); + font-size: 11px; + color: var(--text-muted); + margin-top: 4px; +} + +/* AUDIT: active phase count also headline-sized */ +.ops-ph.active .ops-phc { font-size: 13px; font-weight: 600; color: var(--ops-active); } + +.ops-pips { display: flex; gap: 3px; margin-top: 7px; flex-wrap: wrap; } + +.ops-pip { width: 12px; height: 7px; border-radius: 2px; background: var(--bg-hover, #21262d); } +.ops-pip.before { background: var(--ops-plan); } +.ops-pip.during { background: var(--ops-active); } +.ops-pip.after { background: var(--ops-done); } +.ops-pip.pending { border: 1px dashed var(--border); background: transparent; } + +/* ---- Per-embryo cadence strip — standing_timelapse --------- */ + +.ops-cadence-strip { + display: flex; + flex-direction: column; + gap: 5px; + padding: 8px 10px; + background: var(--bg-hover, #21262d); + border: 1px solid var(--border, #30363d); + border-radius: 8px; +} + +.ops-cadence-embryo { + display: flex; + align-items: center; + gap: 10px; + font-family: var(--ops-mono); + font-size: 11.5px; +} + +.ops-cadence-id { + color: var(--text); + font-weight: 600; + min-width: 32px; +} + +.ops-cadence-phase { + padding: 2px 8px; + border-radius: 9px; + font-size: 10px; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.06em; +} + +.ops-cadence-phase.normal { background: rgba(139, 148, 158, 0.2); color: var(--text-muted); } +.ops-cadence-phase.fast { background: rgba(251, 146, 60, 0.2); color: var(--accent-orange, #fb923c); } +.ops-cadence-phase.burst { background: rgba(239, 68, 68, 0.2); color: #f87171; } +.ops-cadence-phase.paused { background: rgba(90, 169, 230, 0.15); color: var(--ops-plan); } + +.ops-cadence-val { color: var(--text-muted); margin-left: auto; } + +/* ---- Reactive monitor — armed/watching/fired badge --------- */ + +.ops-monitor-status { + display: inline-block; + margin-top: 8px; + padding: 3px 10px; + border-radius: 9px; + font-family: var(--ops-mono); + font-size: 10px; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.1em; +} + +.ops-monitor-armed { background: rgba(90, 169, 230, 0.15); color: var(--ops-plan); border: 1px solid rgba(90, 169, 230, 0.35); } +.ops-monitor-watching { background: rgba(245, 166, 35, 0.12); color: var(--ops-active); border: 1px solid rgba(245, 166, 35, 0.3); } +.ops-monitor-fired { background: rgba(52, 211, 153, 0.12); color: var(--ops-done); border: 1px solid rgba(52, 211, 153, 0.3); } + +/* Planned kind-detail snippets (cadence note, watch hint) */ +.ops-cadence-note, +.ops-monitor-watch { + display: inline-block; + margin-top: 6px; + font-family: var(--ops-mono); + font-size: 11px; + color: var(--text-muted); + opacity: 0.75; +} + +/* ---- Fix #1: bound live.* telemetry facts strip --------------- */ +/* Compact monospace key:value row rendered for active + done tactics + when the live object carries flat keys beyond readouts/phases/target. + Subtle instrument aesthetic — matches the ops-mono system, sits below + the structured readout strip without visual weight. */ + +.ops-livefacts { + display: flex; + flex-wrap: wrap; + gap: 6px 16px; + margin-top: 8px; + padding: 6px 10px; + background: var(--bg-hover, #21262d); + border: 1px solid var(--border, #30363d); + border-radius: 7px; + font-family: var(--ops-mono); + font-size: 11px; + line-height: 1.5; +} + +.ops-lf-pair { + display: inline-flex; + align-items: baseline; + gap: 5px; +} + +.ops-lf-k { + color: var(--text-muted); + font-size: 10px; + letter-spacing: 0.06em; + text-transform: lowercase; +} + +.ops-lf-v { + color: var(--text); + font-weight: 500; + font-variant-numeric: tabular-nums; +} + +/* ---- Fix #5: paused tactic state ----------------------------- */ +/* Muted grey dot + solid grey left edge — visually distinct from + done (green), active (amber), and queued (dashed blue). */ + +.ops-node.paused::before { + border-color: var(--text-muted, #8b949e); + background: var(--bg-card, #161b22); + opacity: 0.7; +} + +.ops-node.paused .ops-stagelab { + color: var(--text-muted, #8b949e); +} + +.ops-node.paused .ops-card { + background: transparent; + border-color: rgba(139, 148, 158, 0.3); + border-left: 3px solid var(--text-muted, #8b949e); + opacity: 0.8; +} + +/* ============================================================ + D2 — Roster Lens (.ops-roster* classes) + Role accent colors are injected at runtime via inline style + from /api/roles — never hardcoded here so the real REGISTRY + palette (magenta test #ff66cc, cyan calibration #00cccc, + teal lineaging #33cc88) drives the rendering. + ============================================================ */ + +/* Sub-label between roster block and spine */ +.ops-section-label { + font-family: var(--ops-mono); + font-size: 9.5px; + letter-spacing: 0.18em; + text-transform: uppercase; + color: var(--text-muted); + opacity: 0.65; + margin: 0 0 10px; +} + +/* Roster container */ +.ops-roster { + margin-bottom: 28px; + border: 1px solid var(--border, #30363d); + border-radius: 12px; + overflow: hidden; +} + +.ops-roster-head { + display: flex; + align-items: center; + gap: 10px; + padding: 9px 16px; + background: var(--bg-hover, #21262d); + border-bottom: 1px solid var(--border, #30363d); +} + +.ops-roster-title { + font-family: var(--ops-mono); + font-size: 10px; + letter-spacing: 0.16em; + text-transform: uppercase; + color: var(--text-muted); +} + +.ops-roster-count { + font-family: var(--ops-mono); + font-size: 10px; + color: var(--text-muted); + margin-left: auto; +} + +/* Class section (SUBJECTS / REFERENCES) */ +.ops-class-section { + border-bottom: 1px solid var(--border, #30363d); +} + +.ops-class-section:last-child { + border-bottom: none; +} + +.ops-class-header { + display: flex; + align-items: center; + gap: 10px; + padding: 7px 16px; + border-bottom: 1px solid var(--border, #30363d); +} + +.ops-class-header.subject { background: rgba(245, 166, 35, 0.04); } +.ops-class-header.reference { background: rgba(139, 148, 158, 0.06); } + +.ops-class-label { + font-family: var(--ops-mono); + font-size: 9px; + font-weight: 700; + letter-spacing: 0.2em; + text-transform: uppercase; +} + +.ops-class-header.subject .ops-class-label { color: var(--ops-active); } +.ops-class-header.reference .ops-class-label { color: var(--text-muted); } + +.ops-class-desc { + font-family: var(--ops-mono); + font-size: 9.5px; + color: var(--text-muted); + opacity: 0.65; +} + +/* Pulsing live dot in the SUBJECTS header */ +.ops-class-live-dot { + width: 6px; + height: 6px; + border-radius: 50%; + background: var(--ops-active); + box-shadow: 0 0 0 3px rgba(245, 166, 35, 0.2); + animation: ops-roster-pulse 2s ease-in-out infinite; + flex-shrink: 0; +} + +@keyframes ops-roster-pulse { + 0%, 100% { opacity: 1; } + 50% { opacity: 0.5; } +} + +/* Role group within a class section */ +.ops-role-group { + border-bottom: 1px solid rgba(48, 54, 61, 0.5); +} + +.ops-role-group:last-child { + border-bottom: none; +} + +/* Role group header — left edge color injected via inline style */ +.ops-role-header { + display: flex; + align-items: center; + gap: 8px; + padding: 6px 16px; + border-left: 3px solid transparent; +} + +.ops-role-name { + font-family: var(--ops-mono); + font-size: 10px; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.12em; + /* color injected via inline style */ +} + +.ops-role-sep { + font-family: var(--ops-mono); + font-size: 10px; + color: var(--text-muted); +} + +.ops-role-count { + font-family: var(--ops-mono); + font-size: 10px; + color: var(--text-muted); +} + +.ops-role-ids { + font-family: var(--ops-mono); + font-size: 10px; + color: var(--text-muted); + margin-left: 2px; +} + +/* Embryo rows — subjects full size, references compact */ +.ops-roster-embryo { + display: flex; + align-items: center; + gap: 10px; + padding: 6px 16px 6px 20px; + border-bottom: 1px solid rgba(48, 54, 61, 0.4); + font-family: var(--ops-mono); + font-size: 11.5px; + transition: background 0.1s; +} + +.ops-roster-embryo:last-child { border-bottom: none; } +.ops-roster-embryo:hover { background: var(--bg-hover, #21262d); } + +.ops-roster-embryo.compact { + padding: 4px 16px 4px 20px; + font-size: 10.5px; + opacity: 0.8; +} + +/* Embryo row fields */ +.ops-rem-id { + color: var(--text); + font-weight: 600; + min-width: 32px; +} + +/* Role chip — background/color/border via inline style from /api/roles */ +.ops-rem-role-chip { + display: inline-flex; + align-items: center; + gap: 4px; + padding: 1px 7px; + border-radius: 9px; + font-size: 9.5px; + font-weight: 700; + text-transform: lowercase; + letter-spacing: 0.04em; + border: 1px solid transparent; + white-space: nowrap; + flex-shrink: 0; +} + +.ops-rem-strain { + color: var(--text-muted); + font-size: 10.5px; + opacity: 0.8; + flex: 0 0 auto; + max-width: 160px; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +/* Cadence phase chip — same visual system as .ops-cadence-phase */ +.ops-rem-phase { + padding: 2px 8px; + border-radius: 9px; + font-size: 10px; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.06em; + flex-shrink: 0; +} + +.ops-rem-phase.normal { background: rgba(139, 148, 158, 0.2); color: var(--text-muted); } +.ops-rem-phase.fast { background: rgba(251, 146, 60, 0.2); color: var(--accent-orange, #fb923c); } +.ops-rem-phase.burst { background: rgba(239, 68, 68, 0.2); color: #f87171; } +.ops-rem-phase.paused { background: rgba(90, 169, 230, 0.15); color: var(--ops-plan); } + +.ops-rem-tactic { + color: var(--text); + font-size: 11px; + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.ops-rem-state { + font-size: 10px; + color: var(--text-muted); + text-transform: uppercase; + letter-spacing: 0.08em; + flex-shrink: 0; +} + +.ops-rem-state.active { color: var(--ops-active); } +.ops-rem-state.done { color: var(--ops-done); } + +/* Scope badge on tactic spine nodes */ +.ops-scope-badge { + display: inline-flex; + align-items: center; + gap: 4px; + padding: 2px 9px; + border-radius: 9px; + font-family: var(--ops-mono); + font-size: 10px; + font-weight: 600; + border: 1px solid transparent; + text-transform: lowercase; + letter-spacing: 0.02em; + white-space: nowrap; +} + +/* Global / explicit-embryos scope — static muted style */ +.ops-scope-global { + background: rgba(139, 148, 158, 0.15); + color: var(--text-muted); + border-color: rgba(139, 148, 158, 0.3); +} +/* Role-scoped badges use inline style (color/bg/border from /api/roles) */ + +/* ============================================================ + Expandable tactic detail — click-to-expand for queued/done/paused + (G — spine-cards) + ============================================================ */ + +/* Compact inline scope chip — always visible in collapsed header. + Distinct from .ops-scope-badge (which resolves embryo ids and is full-width). + Role-scoped chips use inline style for color/bg/border from /api/roles. */ +.ops-scope-chip { + display: inline-flex; + align-items: center; + padding: 1px 7px; + border-radius: 9px; + font-family: var(--ops-mono); + font-size: 9.5px; + font-weight: 600; + border: 1px solid transparent; + text-transform: lowercase; + letter-spacing: 0.02em; + white-space: nowrap; + flex-shrink: 0; +} +/* Neutral global/embryos variant */ +.ops-scope-chip.ops-scope-global { + background: rgba(139, 148, 158, 0.12); + color: var(--text-muted); + border-color: rgba(139, 148, 158, 0.25); +} + +/* Cursor affordance on the card itself */ +.ops-card[data-tactic-expand-id] { + cursor: pointer; + transition: border-color 0.15s; +} +.ops-node.planned .ops-card[data-tactic-expand-id]:hover { + border-color: rgba(90, 169, 230, 0.5); +} +.ops-node.done .ops-card[data-tactic-expand-id]:hover { + border-color: rgba(52, 211, 153, 0.4); +} +.ops-node.paused .ops-card[data-tactic-expand-id]:hover { + border-color: rgba(139, 148, 158, 0.45); +} + +/* Chevron toggle — pushes to the right edge of the header row */ +.ops-expand-chevron { + display: inline-flex; + align-items: center; + justify-content: center; + width: 20px; + height: 20px; + margin-left: auto; + flex-shrink: 0; + background: none; + border: none; + color: var(--text-muted); + font-size: 15px; + cursor: pointer; + transition: transform 0.2s ease, color 0.15s; + padding: 0; + line-height: 1; + opacity: 0.6; +} +.ops-expand-chevron.open { + transform: rotate(90deg); + color: var(--text); + opacity: 1; +} +.ops-card[data-tactic-expand-id]:hover .ops-expand-chevron { + opacity: 1; +} + +/* Expand body — hidden/visible via JS toggling .hidden */ +.ops-expand-body { + margin-top: 10px; +} +.ops-expand-body.hidden { + display: none; +} + +/* Hairline rule separating header from expand detail */ +.ops-expand-rule { + border: none; + border-top: 1px solid rgba(139, 148, 158, 0.15); + margin: 8px 0 10px; +} + +/* Key/value rows */ +.ops-expand-row { + display: flex; + align-items: flex-start; + gap: 10px; + margin-bottom: 6px; + font-family: var(--ops-mono); + font-size: 11px; + line-height: 1.45; +} + +.ops-expand-key { + flex: 0 0 76px; + text-transform: uppercase; + font-size: 9px; + letter-spacing: 0.12em; + color: var(--text-muted); + margin-top: 1px; + opacity: 0.7; +} + +.ops-expand-val { + flex: 1; + color: var(--text); + word-break: break-word; +} + +.ops-expand-mono { + font-variant-numeric: tabular-nums; +} + +/* Phase chain: ramp → hold */ +.ops-expand-phases { + display: inline-flex; + align-items: center; + flex-wrap: wrap; + gap: 4px; +} + +.ops-expand-phase { + display: inline-block; + padding: 1px 7px; + border-radius: 6px; + background: rgba(139, 148, 158, 0.1); + border: 1px solid rgba(139, 148, 158, 0.2); + color: var(--text-muted); + font-size: 10px; + text-transform: lowercase; +} + +.ops-expand-arrow { + color: var(--text-muted); + font-size: 10px; + opacity: 0.45; +} + +/* Readouts strip inside the expand body — inherits ops-gauge, slightly muted */ +.ops-expand-readouts { + display: flex; + gap: 8px; + flex-wrap: wrap; + margin-top: 8px; +} +.ops-expand-readouts .ops-gauge { + opacity: 0.85; +} + +/* ============================================================ + Linked-plans panel (ops-lp-*) — F / Task 4 + Session ↔ plan-item link/delink from the Operations view. + Appended below the tactic spine + roster; matches ops-* aesthetic. + ============================================================ */ + +.ops-lp { + margin-top: 28px; + border: 1px solid var(--border, #30363d); + border-radius: 12px; + overflow: hidden; +} + +/* Section header row: "Linked plans" label + "+ link to a plan" button */ +.ops-lp-head { + display: flex; + align-items: center; + gap: 10px; + padding: 9px 16px; + background: var(--bg-hover, #21262d); + border-bottom: 1px solid var(--border, #30363d); +} + +.ops-lp-title { + font-family: var(--ops-mono); + font-size: 10px; + letter-spacing: 0.16em; + text-transform: uppercase; + color: var(--text-muted); + flex: 1; +} + +/* "+ link to a plan" button — ghost style matching session-link-btn */ +.ops-lp-link-btn { + border: 1px solid var(--border, #30363d); + background: transparent; + color: var(--text-muted); + border-radius: 6px; + padding: 2px 9px; + font: inherit; + font-family: var(--ops-mono); + font-size: 10px; + cursor: pointer; + white-space: nowrap; + transition: color 0.15s, border-color 0.15s; +} +.ops-lp-link-btn:hover { + color: var(--text); + border-color: var(--ops-plan, #5aa9e6); +} + +/* Empty state */ +.ops-lp-empty { + padding: 14px 16px; + font-family: var(--ops-mono); + font-size: 11px; + color: var(--text-muted); + font-style: italic; +} + +/* Loading / transient state */ +.ops-lp-loading { + padding: 12px 16px; + font-family: var(--ops-mono); + font-size: 11px; + color: var(--text-muted); + font-style: italic; +} + +/* List of linked plan-item rows */ +.ops-lp-list { + padding: 6px 0; +} + +/* Single linked plan row: title · campaign · status · delink */ +.ops-lp-row { + display: flex; + align-items: center; + gap: 10px; + padding: 7px 14px; + border-bottom: 1px solid rgba(48, 54, 61, 0.5); + font-family: var(--ops-mono); + font-size: 11.5px; + transition: background 0.1s; +} +.ops-lp-row:last-child { border-bottom: none; } +.ops-lp-row:hover { background: var(--bg-hover, #21262d); } + +.ops-lp-row-title { + font-size: 12px; + color: var(--text); + font-weight: 500; + flex: 1; + min-width: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.ops-lp-row-campaign { + font-size: 10px; + color: var(--text-muted); + opacity: 0.8; + flex-shrink: 0; + max-width: 160px; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.ops-lp-row-status { + font-size: 9.5px; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.07em; + padding: 2px 7px; + border-radius: 9px; + flex-shrink: 0; +} + +/* Status colour variants mirroring ops- state palette */ +.ops-lp-status-planned { + background: rgba(90, 169, 230, 0.12); + color: var(--ops-plan, #5aa9e6); + border: 1px solid rgba(90, 169, 230, 0.3); +} +.ops-lp-status-active { + background: rgba(245, 166, 35, 0.12); + color: var(--ops-active, #f5a623); + border: 1px solid rgba(245, 166, 35, 0.3); +} +.ops-lp-status-done { + background: rgba(52, 211, 153, 0.1); + color: var(--ops-done, #34d399); + border: 1px solid rgba(52, 211, 153, 0.25); +} + +/* Per-row delink (×) button */ +.ops-lp-delink { + background: transparent; + border: none; + color: var(--text-muted); + cursor: pointer; + font-size: 1rem; + line-height: 1; + padding: 1px 4px; + border-radius: 4px; + opacity: 0.4; + flex-shrink: 0; + transition: opacity 0.15s, color 0.15s, background 0.15s; +} +.ops-lp-delink:hover { + opacity: 1; + color: var(--accent-orange, #f97316); + background: rgba(249, 115, 22, 0.12); +} + +/* Inline plan-item picker */ +.ops-lp-picker { + margin: 8px 14px 12px; + padding: 10px; + background: rgba(0, 0, 0, 0.18); + border: 1px solid var(--border, #30363d); + border-radius: 8px; +} +.ops-lp-picker--loading { + font-size: 11px; + color: var(--text-muted); + font-style: italic; + font-family: var(--ops-mono); + background: none; + border: none; + padding: 10px 14px 12px; + margin: 0; +} + +.ops-lp-picker-sel { + width: 100%; + background: var(--bg-dark, #0f172a); + color: var(--text); + border: 1px solid var(--border, #30363d); + border-radius: 6px; + padding: 6px 8px; + font: inherit; + font-size: 11.5px; + font-family: var(--ops-mono); + margin-bottom: 8px; + box-sizing: border-box; +} + +.ops-lp-picker-actions { + display: flex; + gap: 6px; +} + +.ops-lp-picker-link-btn { + flex: 1; + background: var(--ops-plan, #5aa9e6); + color: #05101e; + border: none; + border-radius: 6px; + padding: 5px 12px; + font: inherit; + font-family: var(--ops-mono); + font-size: 11px; + font-weight: 700; + cursor: pointer; + transition: opacity 0.15s; +} +.ops-lp-picker-link-btn:hover { opacity: 0.85; } + +.ops-lp-picker-cancel-btn { + background: transparent; + color: var(--text-muted); + border: 1px solid var(--border, #30363d); + border-radius: 6px; + padding: 5px 12px; + font: inherit; + font-family: var(--ops-mono); + font-size: 11px; + cursor: pointer; + transition: color 0.15s; +} +.ops-lp-picker-cancel-btn:hover { color: var(--text); } diff --git a/gently/ui/web/static/css/landing.css b/gently/ui/web/static/css/landing.css new file mode 100644 index 00000000..2f70129e --- /dev/null +++ b/gently/ui/web/static/css/landing.css @@ -0,0 +1,472 @@ +/* ux_v2 landing — the agent-first welcome that the prototype sketched, ported + into production. A full-bleed overlay shown on first entry that recedes into + the workspace once the user picks a path. Everything is scoped under + body.ux-v2 and the #v2-landing node only renders when the flag is on, so v1 + is byte-for-byte untouched. Visual language mirrors ux-prototype/landing.html + but reuses production's CSS variables (with the prototype hexes as fallback) + so it tracks the app theme. */ + +/* ux_v2 landing fills the gaps in the production token set (main.css defines + --bg-dark/-card/-hover, --border, --text, --text-muted, --accent, --accent-green + but NOT a page-bg alias, a secondary-text, or accent tints). Scope to + body.ux-v2 so v1 is untouched; both themes resolved here so landing.css can + reference these like any real token. dark is the default theme (main.css :root). */ +body.ux-v2 { + --bg: var(--bg-dark); /* page background, theme-aware */ + --text-secondary: var(--text-muted); + --accent-soft: rgba(96,165,250,.15); /* tint of dark --accent #60a5fa */ + --accent-green-soft: rgba(74,222,128,.15); /* tint of dark --accent-green */ + /* one disciplined type scale for the landing/plan surface */ + --v2-fs-body: 14px; + --v2-fs-sm: 13px; + --v2-fs-cap: 12px; + --v2-fs-eyebrow: 11px; +} +body.ux-v2[data-theme="light"] { + --accent-soft: rgba(59,130,246,.10); /* tint of light --accent #3b82f6 */ + --accent-green-soft: rgba(34,197,94,.12); /* tint of light --accent-green */ +} +/* accent-keyed glows can't put var() inside rgba channels, so the dark defaults + live on the elements (re-keyed off the dead #2f6df6 onto the real #60a5fa) and + light overrides ride here next to the tokens. */ +body.ux-v2[data-theme="light"] .v2-landing-orb { box-shadow: 0 6px 22px rgba(59,130,246,.35), inset 0 0 12px rgba(255,255,255,.6); } +body.ux-v2[data-theme="light"] .v2-escape-field input:focus { box-shadow: 0 0 0 4px rgba(59,130,246,.12); } +body.ux-v2[data-theme="light"] .v2-escape-send { box-shadow: 0 6px 16px rgba(59,130,246,.35); } + +/* Theme toggle floated in the overlay's top-right — the header's toggle is + hidden behind this full-bleed overlay, so the landing carries its own. + Positioned against .v2-landing (position:fixed below). */ +.v2-landing-theme { + position: absolute; + top: 22px; + right: 24px; + z-index: 3; +} + +.v2-landing { + position: fixed; + inset: 0; + z-index: 200; + display: flex; + align-items: flex-start; /* BOTH screens top-anchored — no discrete switch on swap */ + justify-content: center; + padding: 24px; + overflow: hidden; + background: + radial-gradient(1100px 700px at 78% -8%, var(--accent-soft) 0%, transparent 55%), + radial-gradient(900px 600px at 8% 108%, var(--accent-green-soft) 0%, transparent 55%), + var(--bg); + transition: opacity .5s cubic-bezier(.22,1,.36,1), transform .5s cubic-bezier(.22,1,.36,1), visibility .5s; +} +/* The calm screen "unfolds" into the workspace: fade + slight scale-up, then + the node is pulled from the layout (display:none set by JS after the + transition) so it never traps clicks. */ +.v2-landing.dismissed { + opacity: 0; + visibility: hidden; + transform: scale(1.015); + pointer-events: none; +} +.v2-landing::before { + content: ""; + position: absolute; + inset: -20vmax; + background: radial-gradient(closest-side, var(--accent-soft), transparent 70%); + filter: blur(30px); + animation: v2land-drift 26s cubic-bezier(.22,1,.36,1) infinite alternate; + will-change: transform; + pointer-events: none; +} +@keyframes v2land-drift { + 0% { transform: translate(-6vw,-4vh) scale(1); } + 100% { transform: translate(8vw,6vh) scale(1.15); } +} + +.v2-landing-inner { + position: relative; + z-index: 1; + display: flex; + flex-direction: column; + align-items: center; + max-width: 760px; + width: 100%; + margin-top: 7vh; /* shared anchor for welcome AND plan — orb stays put on swap */ + margin-bottom: 5vh; +} +.v2-landing-rise { animation: v2land-rise .6s cubic-bezier(.22,1,.36,1) backwards; } +.v2-landing-rise[data-i="1"] { animation-delay: .07s; } +.v2-landing-rise[data-i="2"] { animation-delay: .14s; } +.v2-landing-rise[data-i="3"] { animation-delay: .21s; } +@keyframes v2land-rise { from { opacity: 0; transform: translateY(14px); } to { opacity: 1; transform: none; } } + +/* agent presence */ +.v2-landing-agent { display: flex; flex-direction: column; align-items: center; gap: 16px; } +.v2-landing-orb { + width: 52px; height: 52px; border-radius: 50%; + background: radial-gradient(closest-side at 38% 34%, #ffffff, #bcd3ff 40%, var(--accent, #2f6df6) 100%); + box-shadow: 0 6px 22px rgba(96,165,250,.45), inset 0 0 12px rgba(255,255,255,.6); + animation: v2land-breathe 4s ease-in-out infinite; +} +@keyframes v2land-breathe { 0%,100% { transform: scale(1); } 50% { transform: scale(1.06); } } +.v2-landing-say { + font-size: clamp(20px, 3vw, 28px); font-weight: 600; letter-spacing: -.02em; + text-align: center; max-width: 22ch; line-height: 1.25; color: var(--text, #0f172a); +} +.v2-landing-say .dim { color: var(--text-muted, #94a3b8); font-weight: 500; } + +/* choice cards */ +.v2-landing-choices { + display: grid; grid-template-columns: repeat(2, minmax(0,1fr)); gap: 16px; + margin-top: 32px; width: min(720px, 92vw); +} +@media (max-width: 620px) { .v2-landing-choices { grid-template-columns: 1fr; } } +.v2-choice { + text-align: left; cursor: pointer; position: relative; overflow: hidden; + border: 1px solid var(--border, #e4e9f0); background: var(--bg-card, #fff); + border-radius: 18px; padding: 20px; + box-shadow: 0 1px 2px rgba(15,23,42,.04), 0 8px 28px rgba(15,23,42,.06); + font: inherit; color: var(--text, #0f172a); + transition: transform .26s cubic-bezier(.22,1,.36,1), box-shadow .26s cubic-bezier(.22,1,.36,1), border-color .26s; +} +.v2-choice:hover { + transform: translateY(-4px); + border-color: color-mix(in srgb, var(--accent) 45%, var(--border)); + box-shadow: 0 2px 6px rgba(15,23,42,.06), + 0 18px 50px color-mix(in srgb, var(--accent) 16%, transparent); +} +.v2-choice:active { transform: translateY(-1px) scale(.995); } + +/* Visible keyboard focus for every landing/plan control (mouse clicks get no ring) */ +#v2-landing :focus-visible { + outline: 2px solid var(--accent); + outline-offset: 2px; + border-radius: 12px; /* hug the pill/card corners */ +} +#v2-landing .v2-choice:focus-visible { outline-offset: -2px; } /* inset: the card clips outside outlines */ +#v2-landing .v2-escape-field input:focus-visible { outline-offset: 0; } +.v2-choice-ic { + width: 40px; height: 40px; border-radius: 11px; display: grid; place-items: center; + background: var(--accent-soft, #eaf1ff); color: var(--accent, #2f6df6); margin-bottom: 14px; +} +.v2-choice.alt .v2-choice-ic { background: var(--accent-green-soft, #e7f6ec); color: var(--accent-green, #16a34a); } +.v2-choice h3 { margin: 0 0 6px; font-size: 17px; letter-spacing: -.01em; } +.v2-choice p { margin: 0; color: var(--text-secondary, #475569); font-size: var(--v2-fs-sm); line-height: 1.5; } +.v2-choice-tag { + position: absolute; top: 16px; right: 16px; + font-size: var(--v2-fs-eyebrow); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted, #94a3b8); + border: 1px solid var(--border, #e4e9f0); border-radius: 999px; padding: 3px 9px; +} +.v2-choice-go { + margin-top: 16px; display: flex; align-items: center; gap: 6px; + color: var(--accent, #2f6df6); font-size: 13px; font-weight: 600; + opacity: 0; transform: translateX(-4px); transition: .26s cubic-bezier(.22,1,.36,1); +} +.v2-choice.alt .v2-choice-go { color: var(--accent-green, #16a34a); } +.v2-choice:hover .v2-choice-go { opacity: 1; transform: none; } + +/* escape hatch — chat is the last resort, an obvious pill */ +.v2-escape { margin-top: 24px; display: flex; flex-direction: column; align-items: center; } +.v2-escape-toggle { + display: inline-flex; align-items: center; gap: 7px; cursor: pointer; font: inherit; font-size: 13px; + background: var(--bg-card, #fff); border: 1px solid var(--border, #e4e9f0); color: var(--text-secondary, #475569); + padding: 11px 16px; border-radius: 999px; + box-shadow: 0 1px 2px rgba(15,23,42,.04), 0 8px 28px rgba(15,23,42,.06); + transition: color .2s, border-color .2s, transform .2s cubic-bezier(.22,1,.36,1); +} +.v2-escape-toggle:hover { color: var(--text, #0f172a); border-color: var(--border-strong); transform: translateY(-1px); } +.v2-escape-toggle .v2-escape-caret { display: inline-block; transition: transform .3s cubic-bezier(.22,1,.36,1); opacity: .55; } +.v2-escape.open .v2-escape-toggle .v2-escape-caret { transform: rotate(180deg); } +.v2-escape-field { + display: flex; align-items: center; gap: 8px; width: min(520px, 90vw); + max-height: 0; opacity: 0; overflow: hidden; + transition: max-height .4s cubic-bezier(.22,1,.36,1), opacity .4s cubic-bezier(.22,1,.36,1), margin .4s cubic-bezier(.22,1,.36,1); +} +.v2-escape.open .v2-escape-field { max-height: 64px; opacity: 1; margin-top: 12px; } +.v2-escape-field input { + flex: 1; min-width: 0; border: 1px solid var(--border, #e4e9f0); background: var(--bg-card, #fff); + border-radius: 12px; padding: 12px 14px; font: inherit; font-size: var(--v2-fs-body); color: var(--text, #0f172a); + outline: none; box-shadow: 0 1px 2px rgba(15,23,42,.04); + transition: border-color .2s, box-shadow .2s; +} +.v2-escape-field input:focus { border-color: var(--accent, #2f6df6); box-shadow: 0 0 0 4px rgba(96,165,250,.18); } +.v2-escape-send { + appearance: none; border: 0; cursor: pointer; flex: none; width: 42px; height: 42px; border-radius: 12px; + background: var(--accent, #2f6df6); color: #fff; display: grid; place-items: center; + box-shadow: 0 6px 16px rgba(96,165,250,.40); transition: transform .2s cubic-bezier(.22,1,.36,1), filter .2s; +} +.v2-escape-send:hover { transform: translateY(-1px); filter: brightness(1.05); } + +/* one-way skip into the workspace (e.g. a reload mid-session) */ +.v2-landing-skip { + margin-top: 24px; background: none; border: 0; cursor: pointer; font: inherit; font-size: var(--v2-fs-cap); + color: var(--text-muted, #94a3b8); padding: 10px 12px; border-radius: 8px; + transition: color .2s; +} +.v2-landing-skip:hover { color: var(--text-secondary, #475569); } + +/* Under ux_v2 the landing IS the welcome moment, so the legacy home hero + (static "Welcome to Gently" + start button) would be a duplicate behind it — + hide it. The recent-* cards and the context surface stay. */ +body.ux-v2 .home-hero { display: none; } + +/* ── Two-screen system: welcome ↔ in-place plan wizard ───────── */ +.v2-landing-inner { max-width: 980px; } /* widen for the plan layout */ +.v2-landing .v2-screen { display: none; width: 100%; } +.v2-landing[data-screen="welcome"] .v2-screen-welcome { + display: flex; flex-direction: column; align-items: center; + max-width: 760px; margin: 0 auto; + animation: v2land-plan-in .42s cubic-bezier(.22,1,.36,1) backwards; +} +.v2-landing[data-screen="plan"] .v2-screen-plan { + display: flex; flex-direction: column; + animation: v2land-plan-in .42s cubic-bezier(.22,1,.36,1) backwards; +} +/* One swap motion shared by both screens: a soft opacity + rise + settle. The + scale .992→1 echoes the dismissed-state scale(1.015) so the surface feels like + one continuous fabric folding, not two slides swapping. */ +@keyframes v2land-plan-in { + from { opacity: 0; transform: translateY(10px) scale(.992); } + to { opacity: 1; transform: none; } +} + +/* plan header */ +.v2-plan-head { display: flex; align-items: center; gap: 12px; margin-bottom: 16px; } +.v2-plan-orb { width: 40px; height: 40px; transition: width .42s cubic-bezier(.22,1,.36,1), height .42s cubic-bezier(.22,1,.36,1); } +.v2-plan-who { font-size: var(--v2-fs-eyebrow); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted, #94a3b8); } +.v2-plan-title { font-size: 18px; font-weight: 600; letter-spacing: -.01em; color: var(--text, #0f172a); } +.v2-plan-back { + margin-left: auto; background: none; border: 0; cursor: pointer; font: inherit; font-size: 13px; + color: var(--text-muted, #94a3b8); padding: 9px 12px; border-radius: 8px; transition: color .2s, background .2s; +} +.v2-plan-back:hover { color: var(--text, #0f172a); background: rgba(15,23,42,.05); } + +/* plan body: ask stage + assembling plan */ +.v2-plan-wrap { display: grid; grid-template-columns: 1.35fr .9fr; gap: 20px; align-items: start; } +@media (max-width: 720px) { + .v2-plan-wrap { grid-template-columns: 1fr; } + /* single column: THE PLAN sits BELOW the feed — drop the internal scroll + and let the whole plan screen scroll as one document instead. The + descendant selector outranks the plain `.v2-plan-main { max-height }` + rule that appears later in the file (equal specificity → source order), + so the cap is genuinely lifted here, not silently re-applied. */ + .v2-landing[data-screen="plan"] .v2-plan-main { height: auto; max-height: none; overflow-y: visible; padding-right: 0; } + .v2-landing[data-screen="plan"] { overflow-y: auto; } +} +.v2-plan-main { min-height: 220px; } +.v2-plan-ask:empty { display: none; } +.v2-plan-thinking { display: flex; align-items: center; gap: 9px; color: var(--text-muted, #94a3b8); font-size: var(--v2-fs-sm); padding: 20px 4px; } +.v2-plan-thinking.hidden { display: none; } +.v2-typing { display: inline-flex; gap: 5px; align-items: center; } +.v2-typing i { width: 7px; height: 7px; border-radius: 50%; background: var(--accent, #2f6df6); opacity: .4; animation: v2-blink 1.1s infinite; } +.v2-typing i:nth-child(2) { animation-delay: .15s; } +.v2-typing i:nth-child(3) { animation-delay: .3s; } +@keyframes v2-blink { 0%,100% { opacity: .25; transform: translateY(0); } 50% { opacity: 1; transform: translateY(-3px); } } + +.v2-plan-side { + background: var(--bg-card, #fff); border: 1px solid var(--border, #e4e9f0); border-radius: 14px; padding: 14px 16px; + box-shadow: 0 1px 2px rgba(15,23,42,.04); +} +.v2-plan-side-h { font-size: var(--v2-fs-eyebrow); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted, #94a3b8); margin-bottom: 10px; } +.v2-plan-side-empty { color: var(--text-muted, #94a3b8); font-size: var(--v2-fs-sm); font-style: italic; } +.v2-plan-row { display: flex; flex-direction: column; gap: 2px; padding: 9px 0; border-top: 1px dashed var(--border, #e4e9f0); } +.v2-plan-row:first-child { border-top: 0; } +.v2-plan-row .k { font-size: var(--v2-fs-eyebrow); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted, #94a3b8); } +.v2-plan-row .v { font-size: var(--v2-fs-body); color: var(--text, #0f172a); font-weight: 600; } + +/* plan footer: "Open conversation" (quiet, left), a spacer, then "Continue in + workspace" demoted to a text link (right). The agent's recommended option in + the ask card is the real primary action now — the footer no longer competes. */ +.v2-plan-foot { display: flex; align-items: center; gap: 10px; margin-top: 20px; padding-top: 16px; border-top: 1px solid var(--border, #e4e9f0); } +.v2-plan-foot-spacer { flex: 1; } +.v2-plan-chat { + background: none; border: 1px solid var(--border, #e4e9f0); color: var(--text-secondary, #475569); + border-radius: 999px; padding: 11px 16px; font: inherit; font-size: var(--v2-fs-sm); cursor: pointer; transition: border-color .2s, color .2s; +} +.v2-plan-chat:hover { border-color: var(--border-strong); color: var(--text, #0f172a); } +.v2-plan-skip { + background: none; border: 0; cursor: pointer; font: inherit; font-size: var(--v2-fs-sm); + color: var(--text-muted, #94a3b8); padding: 11px 10px; border-radius: 8px; transition: color .2s; +} +.v2-plan-skip:hover { color: var(--text-secondary, #475569); } +.v2-plan-export { + background: none; border: 1px solid var(--border, #e4e9f0); color: var(--text-secondary, #475569); + border-radius: 999px; padding: 11px 16px; font: inherit; font-size: var(--v2-fs-sm); + font-weight: 600; cursor: pointer; transition: border-color .2s, color .2s, background .2s; +} +.v2-plan-export:hover { border-color: var(--border-strong); color: var(--text, #0f172a); background: var(--bg-hover); } +.v2-plan-export:disabled { opacity: .6; cursor: default; } +.v2-plan-export[hidden] { display: none; } + +/* ── Plan-ready state: the design is done, signpost the finish line ───────── */ +.v2-screen-plan.ready .v2-plan-orb { + background: radial-gradient(closest-side at 38% 34%, #ffffff, #c8f0d4 40%, var(--accent-green, #16a34a) 100%); +} +.v2-screen-plan.ready .v2-plan-who { color: var(--accent-green, #16a34a); } +.v2-screen-plan.ready .v2-plan-foot { border-top-color: color-mix(in srgb, var(--accent-green) 35%, var(--border)); } +/* promote "open the workspace" from a quiet skip link to the primary action */ +.v2-screen-plan.ready #v2-plan-continue { + background: var(--accent-green, #16a34a); color: #fff; + border-radius: 999px; padding: 11px 20px; font-weight: 600; +} +.v2-screen-plan.ready #v2-plan-continue:hover { + color: #fff; background: color-mix(in srgb, var(--accent-green) 88%, #000); +} + +/* ── Agent-activity feed: claude.ai-style collapsible tool cards ──────────── */ +/* Both screens share the .v2-landing-inner top anchor (no per-screen align flip — + that was the welcome→plan lurch). The feed is a fixed-height viewport (height + set above) so the streaming activity column scrolls on its own without ever + reflowing the anchored header/footer around it. Short feeds stay compact + (min-height above); long feeds cap at 66vh and scroll internally. The header + never moves because the inner is top-anchored — only the footer rides down as + the feed grows, up to the cap. */ +.v2-plan-main { max-height: 66vh; overflow-y: auto; padding-right: 4px; } + +.v2-plan-activity { display: flex; flex-direction: column; gap: 8px; margin-bottom: 12px; } +.v2-plan-activity:empty { display: none; margin: 0; } + +/* Paginated feed — one agent step (turn) per page, flipped with ‹ Prev / Next ›. + The pager bar / dots reuse .v2-plan-pager / .v2-plan-dots styling. */ +.v2-feed-pages { display: flex; flex-direction: column; } +.v2-act-page { display: none; flex-direction: column; gap: 8px; } +.v2-act-page.active { display: flex; } +/* beat .v2-plan-pager/.v2-plan-dots { display:flex } so [hidden] actually hides */ +.v2-plan-pager[hidden], .v2-plan-dots[hidden] { display: none; } +.v2-feed-pager-bar { margin: 0 0 4px; } +.v2-feed-dots { margin-top: 10px; } +/* the current question, pinned below the paged feed, set off by a divider — + only once there are steps above it (no stray line on the first choice card) */ +#v2-plan-activity:has(.v2-act-page) + .v2-plan-ask:not(:empty) { + margin-top: 14px; padding-top: 14px; border-top: 1px solid var(--border, #e4e9f0); +} + +/* agent prose between tool calls */ +.v2-act-text { font-size: var(--v2-fs-sm); line-height: 1.55; color: var(--text-secondary, #475569); white-space: pre-wrap; } + +/* collapsed-by-default tool card; the header toggles .open to reveal the body */ +.v2-act-tool { border: 1px solid var(--border, #e4e9f0); border-radius: 11px; background: var(--bg-card, #fff); overflow: hidden; } +.v2-act-tool-head { + display: flex; align-items: center; gap: 8px; width: 100%; + background: none; border: 0; cursor: pointer; text-align: left; font: inherit; + padding: 11px 12px; color: var(--text, #0f172a); +} +.v2-act-tool-head:hover { background: var(--bg-hover, #f1f5f9); } +.v2-act-ic { width: 16px; flex: none; text-align: center; font-size: 12px; } +.v2-act-tool.done .v2-act-ic { color: var(--accent-green, #16a34a); } +.v2-act-tool.err .v2-act-ic { color: #ea580c; } +body.ux-v2[data-theme="dark"] .v2-act-tool.err .v2-act-ic { color: #fb923c; } +.v2-act-spin { + display: inline-block; width: 11px; height: 11px; border-radius: 50%; + border: 2px solid var(--border, #e4e9f0); border-top-color: var(--accent, #2f6df6); + animation: v2-act-spin .7s linear infinite; +} +@keyframes v2-act-spin { to { transform: rotate(360deg); } } +.v2-act-label { font-size: var(--v2-fs-sm); font-weight: 600; flex: none; } +.v2-act-summary { font-size: var(--v2-fs-cap); color: var(--text-muted, #94a3b8); flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.v2-act-chev { flex: none; color: var(--text-muted, #94a3b8); transition: transform .2s; font-size: 13px; } +.v2-act-tool.open .v2-act-chev { transform: rotate(90deg); } +/* animatable collapse: grid-template-rows 0fr→1fr eases in step with the chevron + (display:none isn't animatable). Needs exactly ONE min-height:0 child — landing.js + wraps the blocks in a single inner div for this. */ +.v2-act-tool-body { + display: grid; grid-template-rows: 0fr; opacity: 0; + padding: 0 12px 0 37px; + transition: grid-template-rows .26s cubic-bezier(.22,1,.36,1), + opacity .26s cubic-bezier(.22,1,.36,1), + padding-bottom .26s cubic-bezier(.22,1,.36,1); +} +.v2-act-tool-body > * { min-height: 0; overflow: hidden; } +.v2-act-tool.open .v2-act-tool-body { grid-template-rows: 1fr; opacity: 1; padding-bottom: 11px; } +.v2-act-tool.open .v2-act-summary { white-space: normal; } +.v2-act-block-label { font-size: var(--v2-fs-eyebrow); letter-spacing: .08em; text-transform: uppercase; color: var(--text-muted, #94a3b8); margin-top: 8px; } +.v2-act-block { + font: 12px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; + background: var(--bg-hover); border: 1px solid var(--border, #e4e9f0); border-radius: 8px; + padding: 8px 10px; margin-top: 4px; white-space: pre-wrap; word-break: break-word; + color: var(--text-secondary, #475569); max-height: 220px; overflow: auto; +} + +/* error + fallback states */ +.v2-plan-error { + font-size: var(--v2-fs-sm); color: #b91c1c; + background: rgba(239,68,68,.10); border: 1px solid rgba(239,68,68,.35); + border-radius: 11px; padding: 11px 13px; +} +body.ux-v2[data-theme="dark"] .v2-plan-error { + color: #fca5a5; background: rgba(239,68,68,.14); border-color: rgba(239,68,68,.40); +} +.v2-plan-error.hidden { display: none; } +.v2-plan-fallback { font-size: 13px; color: var(--text-muted, #94a3b8); padding: 8px 2px; } +.v2-plan-fallback a { color: var(--accent, #2f6df6); cursor: pointer; } + +/* plan-panel: phases + tasks (real plan), and a free-text-answer row variant */ +.v2-plan-phase { margin-top: 12px; } +.v2-plan-phase:first-child { margin-top: 0; } +.v2-plan-phase-h { + font-size: var(--v2-fs-eyebrow); font-weight: 700; letter-spacing: .06em; + text-transform: uppercase; color: var(--text-secondary, #475569); margin-bottom: 6px; +} +/* a plan item: "P.I" number · type-color dot · title · optional duration. + the type dot encodes the item kind (imaging/genetics/analysis/…) at a glance. */ +.v2-plan-task { + display: grid; grid-template-columns: auto 8px 1fr auto; align-items: baseline; + gap: 9px; font-size: var(--v2-fs-cap); color: var(--text-secondary, #475569); + padding: 6px 0; border-top: 1px solid color-mix(in srgb, var(--border, #e4e9f0) 55%, transparent); +} +.v2-plan-phase .v2-plan-task:first-child { border-top: 0; } +.v2-task-num { + font: 600 11px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; + color: var(--text-muted, #94a3b8); font-variant-numeric: tabular-nums; +} +.v2-task-dot { width: 8px; height: 8px; border-radius: 50%; align-self: center; background: var(--text-muted, #94a3b8); } +.v2-task-ttl { min-width: 0; color: var(--text, #0f172a); line-height: 1.45; } +.v2-task-days { + font-size: 10.5px; font-weight: 600; color: var(--text-muted, #94a3b8); + font-variant-numeric: tabular-nums; white-space: nowrap; +} +.v2-plan-task.type-imaging .v2-task-dot { background: var(--accent, #2f6df6); } +.v2-plan-task.type-genetics .v2-task-dot { background: #8b5cf6; } +.v2-plan-task.type-analysis .v2-task-dot { background: var(--accent-green, #16a34a); } +.v2-plan-task.type-bench .v2-task-dot { background: #d97706; } +/* decision points read as gates — a rotated square, not a round bead */ +.v2-plan-task.type-decision_point .v2-task-dot { background: #e11d48; border-radius: 1px; transform: rotate(45deg); } +.v2-plan-title-row { font-size: var(--v2-fs-sm); font-weight: 600; letter-spacing: -.01em; color: var(--text, #0f172a); margin-bottom: 8px; } +.v2-plan-row.v2-plan-row-freetext .v { font-style: italic; } +.v2-plan-task-empty { grid-column: 1 / -1; color: var(--text-muted, #94a3b8); font-style: italic; } + +/* THE PLAN pager: ‹ Prev · "Phase · i of N" · Next › + dots, one phase per page */ +.v2-plan-pager { display: flex; align-items: center; gap: 8px; margin: 2px 0 12px; } +.v2-plan-pager-btn { + flex: none; background: none; border: 0; cursor: pointer; font: inherit; + font-size: var(--v2-fs-cap); font-weight: 600; color: var(--accent, #2f6df6); + padding: 5px 7px; border-radius: 7px; transition: background .15s, color .15s, opacity .15s; +} +.v2-plan-pager-btn:hover:not(:disabled) { background: var(--accent-soft); } +.v2-plan-pager-btn:disabled { color: var(--text-muted, #94a3b8); opacity: .45; cursor: default; } +.v2-plan-pager-pos { + flex: 1; min-width: 0; text-align: center; font-size: var(--v2-fs-cap); font-weight: 600; + color: var(--text, #0f172a); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; +} +.v2-plan-dots { display: flex; gap: 6px; justify-content: center; margin-top: 12px; } +.v2-plan-dot { + width: 7px; height: 7px; padding: 0; border: 0; border-radius: 50%; cursor: pointer; + background: var(--border, #e4e9f0); transition: background .2s, transform .2s; +} +.v2-plan-dot:hover { transform: scale(1.25); } +.v2-plan-dot.active { background: var(--accent, #2f6df6); } + +@media (prefers-reduced-motion: reduce) { + .v2-landing, .v2-landing::before, .v2-landing-rise, .v2-landing-orb, .v2-plan-orb, + .v2-landing[data-screen="plan"] .v2-screen-plan, + .v2-landing[data-screen="welcome"] .v2-screen-welcome, + .v2-typing i, .v2-act-spin, .v2-act-chev, + .v2-act-tool-body, .v2-act-tool-head, + .v2-choice, .v2-escape-field, .v2-escape-toggle, + .v2-plan-pager-btn, .v2-plan-dot { + animation: none !important; + transition-duration: .12s !important; + } + /* keep the collapsible usable without the height tween */ + .v2-act-tool-body { transition: none !important; } + .v2-act-tool.open .v2-act-tool-body { grid-template-rows: 1fr; opacity: 1; } +} diff --git a/gently/ui/web/static/css/main.css b/gently/ui/web/static/css/main.css index 4d19c79d..985aa33a 100644 --- a/gently/ui/web/static/css/main.css +++ b/gently/ui/web/static/css/main.css @@ -23,6 +23,13 @@ --accent-pink: #f472b6; --accent-cyan: #22d3ee; + /* Temperature graph colors (root-level so the graph is portable outside + .devices-container-map — e.g. when reused in manual mode) */ + --temp-setpoint-color: #f59e0b; + --temp-water-color: #22d3ee; + --temp-grid-line-color: rgba(212, 221, 232, 0.20); + --temp-grid-label-color: #6a778a; + /* Gradients for modern feel */ --gradient-primary: linear-gradient(135deg, #60a5fa 0%, #c084fc 100%); --gradient-success: linear-gradient(135deg, #4ade80 0%, #22d3ee 100%); @@ -34,6 +41,10 @@ /* Image backgrounds */ --img-bg: #000; + + /* Docked agent panel */ + --panel-edge-shadow: rgba(0, 0, 0, 0.55); + --border-strong: #444c56; } /* ======================================== @@ -59,6 +70,12 @@ --accent-pink: #ec4899; --accent-cyan: #06b6d4; + /* Temperature graph colors (root-level; see dark block) */ + --temp-setpoint-color: #f59e0b; + --temp-water-color: #0e7490; + --temp-grid-line-color: rgba(29, 43, 58, 0.22); + --temp-grid-label-color: #6b7280; + /* Gradients */ --gradient-primary: linear-gradient(135deg, #3b82f6 0%, #a855f7 100%); --gradient-success: linear-gradient(135deg, #22c55e 0%, #06b6d4 100%); @@ -70,6 +87,10 @@ /* Image backgrounds */ --img-bg: #1e293b; + + /* Docked agent panel — softer shadow + stronger seam for light mode */ + --panel-edge-shadow: rgba(0, 0, 0, 0.18); + --border-strong: #cbd5e1; } * { margin: 0; padding: 0; box-sizing: border-box; } @@ -90,6 +111,25 @@ body { transition: background-color 0.3s ease, color 0.3s ease; } +/* App shell: main column + docked agent panel side by side. The flex row lets + the panel become a real column (pushing content) when pinned to dock; in the + default overlay mode the panel is position:fixed and sits out of this flow. */ +.app-shell { + flex: 1 1 auto; + min-height: 0; + display: flex; + flex-direction: row; + position: relative; /* anchor for the overlay-mode agent panel */ +} +.app-main { + flex: 1 1 auto; + min-width: 0; /* allow canvases to shrink (not overflow) when docked */ + min-height: 0; + display: flex; + flex-direction: column; + overflow: hidden; +} + /* Smooth theme transitions for key elements */ .header, .tabs, .tab-content, .panel, .gallery-item, .events-container, .lightbox-container, .shortcuts-content { @@ -625,6 +665,114 @@ a.tab-link.active { flex-direction: column; } +/* ── Home (landing) tab ───────────────────────────────────── + #home-content is a flex column with overflow:hidden, so the scroll lives on + .home-scroll. */ +.home-scroll { + flex: 1 1 auto; + min-height: 0; + overflow-y: auto; + padding: 24px; + display: flex; + flex-direction: column; + gap: 20px; +} +.home-hero { + display: flex; + align-items: center; + justify-content: space-between; + gap: 16px; + padding: 20px 22px; + border: 1px solid var(--border); + border-radius: 14px; + background: var(--bg-card); +} +.home-hero-title { font-size: 1.35rem; font-weight: 700; color: var(--text); margin: 0; } +.home-hero-status { font-size: 12.5px; color: var(--text-muted); margin-top: 4px; } +.home-start-btn { + flex: 0 0 auto; + padding: 10px 18px; + border: none; border-radius: 10px; + background: var(--gradient-primary, var(--accent)); + color: #fff; font-weight: 600; font-size: 13.5px; cursor: pointer; + box-shadow: var(--shadow-glow); + transition: transform 0.12s ease, box-shadow 0.12s ease; +} +.home-start-btn:hover { transform: translateY(-1px); box-shadow: var(--shadow-glow-strong); } + +.home-grid { + display: grid; + grid-template-columns: repeat(2, minmax(0, 1fr)); + gap: 16px; +} +.home-card-wide { grid-column: 1 / -1; } +@media (max-width: 820px) { + .home-grid { grid-template-columns: 1fr; } + .home-card-wide { grid-column: auto; } +} + +.home-card { + display: flex; flex-direction: column; + padding: 14px 16px; + border: 1px solid var(--border); + border-radius: 12px; + background: var(--bg-card); + min-height: 120px; +} +.home-card-head { + display: flex; align-items: center; justify-content: space-between; + margin-bottom: 10px; +} +.home-card-title { + font-size: 11px; font-weight: 600; letter-spacing: 0.06em; + text-transform: uppercase; color: var(--text-muted); +} +.home-card-link { font-size: 11.5px; color: var(--accent); text-decoration: none; } +.home-card-link:hover { text-decoration: underline; } +.home-card-body { display: flex; flex-direction: column; gap: 6px; } + +.home-item { + display: flex; align-items: center; justify-content: space-between; gap: 10px; + padding: 8px 10px; border-radius: 8px; + background: rgba(127, 127, 127, 0.05); + border: 1px solid transparent; +} +.home-item-clickable { cursor: pointer; } +.home-item-clickable:hover { border-color: var(--accent); background: var(--bg-hover); } +.home-item-main { display: flex; flex-direction: column; gap: 2px; min-width: 0; } +.home-item-row { display: flex; align-items: center; gap: 7px; } +.home-item-name { + font-size: 13px; color: var(--text); font-weight: 500; + white-space: nowrap; overflow: hidden; text-overflow: ellipsis; +} +.home-item-meta { font-size: 11.5px; color: var(--text-muted); } +.home-tag { + font-size: 9.5px; font-weight: 700; letter-spacing: 0.04em; text-transform: uppercase; + padding: 1px 6px; border-radius: 999px; +} +.home-tag-live { color: var(--accent-green); border: 1px solid rgba(74, 222, 128, 0.4); } +.home-resume { + flex: 0 0 auto; + padding: 4px 11px; border-radius: 7px; + border: 1px solid var(--accent); background: transparent; color: var(--accent); + font-size: 12px; font-weight: 600; cursor: pointer; +} +.home-resume:hover { background: var(--accent); color: #fff; } +.home-resume:disabled { opacity: 0.6; cursor: default; } +.home-chip { + flex: 0 0 auto; + font-size: 11px; font-weight: 600; + padding: 2px 8px; border-radius: 999px; + background: var(--bg-hover); color: var(--text-muted); +} + +.home-image-strip { display: flex; gap: 8px; flex-wrap: wrap; } +.home-image { + width: 84px; height: 84px; border-radius: 8px; overflow: hidden; + border: 1px solid var(--border); background: var(--img-bg); flex: 0 0 auto; +} +.home-image img { width: 100%; height: 100%; object-fit: cover; display: block; } + /* Live View - Clean full-width layout */ .live-view { display: flex; @@ -1482,12 +1630,200 @@ a.tab-link.active { background: #000; border-radius: 3px; border: 1px solid var(--border); - display: none; /* shown when frame arrives via .has-frame */ - opacity: 1; + display: inline-block; + opacity: 0.35; /* dim until a real frame arrives (.has-frame) */ } .cal-spim-thumb.has-frame { - display: inline-block; + opacity: 1; +} + +/* Thumb wrapped in a button so click pops out a larger live view. + Sized to match the thumb so it remains clickable even before the + first frame arrives. */ +.cal-spim-thumb-btn { + position: relative; + padding: 0; + background: none; + border: 0; + cursor: pointer; + display: inline-flex; + align-items: center; + line-height: 0; + color: inherit; + width: 96px; + height: 72px; +} + +.cal-spim-thumb-btn:focus-visible { + outline: 2px solid var(--accent, #4f8cff); + outline-offset: 2px; + border-radius: 4px; +} + +.cal-spim-expand-icon { + position: absolute; + top: 2px; + right: 2px; + background: rgba(0, 0, 0, 0.55); + color: #fff; + font-size: 11px; + line-height: 1; + padding: 2px 4px; + border-radius: 3px; + opacity: 0; + transition: opacity 0.12s ease; + pointer-events: none; +} + +.cal-spim-thumb-btn:hover .cal-spim-expand-icon, +.cal-spim-thumb-btn:focus-visible .cal-spim-expand-icon { + opacity: 1; +} + +/* Hide the expand chip when the thumb has no frame yet — nothing to expand. */ +.cal-spim-thumb-btn:has(.cal-spim-thumb:not(.has-frame)) .cal-spim-expand-icon { + display: none; +} + +/* ---------- Floating SPIM popout ---------- */ +.cal-spim-popout { + position: fixed; + top: 80px; + right: 24px; + width: 560px; + height: 480px; + min-width: 320px; + min-height: 260px; + z-index: 9000; + background: var(--bg-card); + border: 1px solid var(--border); + border-radius: 10px; + box-shadow: 0 16px 40px rgba(0, 0, 0, 0.45), + 0 2px 8px rgba(0, 0, 0, 0.25); + display: flex; + flex-direction: column; + overflow: hidden; + resize: both; +} + +.cal-spim-popout[hidden] { + display: none; +} + +.cal-spim-popout.dragging { + user-select: none; + cursor: grabbing; +} + +.cal-spim-popout-header { + flex: 0 0 auto; + display: flex; + align-items: center; + gap: 8px; + padding: 8px 12px; + background: var(--bg-elevated, var(--bg-card)); + border-bottom: 1px solid var(--border); + cursor: grab; + touch-action: none; +} + +.cal-spim-popout.dragging .cal-spim-popout-header { + cursor: grabbing; +} + +.cal-spim-popout-led { + width: 8px; + height: 8px; + border-radius: 50%; + background: #4ade80; + box-shadow: 0 0 6px rgba(74, 222, 128, 0.7); + animation: cal-spim-led-blink 1.6s ease-in-out infinite; +} + +.cal-spim-popout-led.idle { + background: var(--text-muted, #666); + box-shadow: none; + animation: none; +} + +.cal-spim-popout-title { + font-size: 11px; + font-weight: 700; + text-transform: uppercase; + letter-spacing: 0.6px; + color: var(--text); +} + +.cal-spim-popout-embryo { + font-family: 'JetBrains Mono', ui-monospace, monospace; + font-size: 11px; + color: var(--text-muted); +} + +.cal-spim-popout-spacer { + flex: 1; +} + +.cal-spim-popout-close { + background: transparent; + border: 0; + color: var(--text-muted); + font-size: 20px; + line-height: 1; + padding: 0 6px; + cursor: pointer; + border-radius: 4px; +} + +.cal-spim-popout-close:hover { + color: var(--text); + background: var(--border); +} + +.cal-spim-popout-body { + flex: 1; + min-height: 0; + display: flex; + align-items: center; + justify-content: center; + background: #000; + overflow: hidden; + padding: 4px; +} + +.cal-spim-popout-img { + max-width: 100%; + max-height: 100%; + object-fit: contain; + display: none; +} + +.cal-spim-popout-img.has-frame { + display: block; +} + +.cal-spim-popout-placeholder { + color: var(--text-muted); + font-size: 12px; + letter-spacing: 0.4px; +} + +.cal-spim-popout-placeholder[hidden] { + display: none; +} + +.cal-spim-popout-footer { + flex: 0 0 auto; + padding: 6px 12px; + border-top: 1px solid var(--border); + background: var(--bg-elevated, var(--bg-card)); + font-family: 'JetBrains Mono', ui-monospace, monospace; + font-size: 11px; + color: var(--text-muted); + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; } /* When a live frame is active, let the SPIM cell breathe a bit so the @@ -2036,6 +2372,20 @@ a.tab-link.active { .event-type-badge.error { background: rgba(248, 81, 73, 0.2); color: #f85149; } .event-type-badge.default { background: var(--bg-hover); color: var(--text-muted); } +/* Log-record badges per level. The level is the badge text (DEBUG / INFO / + WARN / ERROR) for log rows; the LOG_RECORD type itself is collapsed into + the level so the column doesn't read the same string for every line. */ +.event-type-badge.log-debug { background: rgba(125, 134, 145, 0.18); color: #9ba3b0; } +.event-type-badge.log-info { background: rgba(88, 166, 255, 0.16); color: var(--accent); } +.event-type-badge.log-warn { background: rgba(210, 153, 34, 0.22); color: var(--accent-orange); } +.event-type-badge.log-error { background: rgba(248, 81, 73, 0.22); color: #f85149; } + +/* Log line message: monospace, faint logger prefix, expandable trace. */ +.log-row .event-data { font-family: 'JetBrains Mono', ui-monospace, monospace; } +.log-logger { color: var(--text-muted); opacity: 0.85; margin-right: 0.5rem; } +.log-message { color: var(--text); } +.log-exc { color: #f85149; opacity: 0.85; } + .event-source { color: var(--text-muted); font-size: 0.75rem; @@ -3330,6 +3680,15 @@ kbd { padding: 0; } +/* Filmstrip: rows on the left, reasoning/detail panel pinned on the right. + (Recovered from the lost WIP commit 0269e18d.) */ +.view-filmstrip { + display: flex; + flex-direction: row; + align-items: stretch; + overflow: hidden; +} + /* ======================================== AMBIENT HEALTH PULSE ======================================== */ @@ -3405,12 +3764,24 @@ kbd { .board-col { padding: 0 0.5rem; } .board-col-embryo { width: 100px; flex-shrink: 0; } .board-col-stage { width: 130px; flex-shrink: 0; } -.board-col-conf { width: 60px; flex-shrink: 0; text-align: center; } -.board-col-rate { width: 70px; flex-shrink: 0; text-align: center; } -.board-col-eta { width: 70px; flex-shrink: 0; text-align: center; } +.board-col-clock { width: 72px; flex-shrink: 0; text-align: right; font-variant-numeric: tabular-nums; } +.board-col-stereo { width: 140px; flex-shrink: 0; font-variant-numeric: tabular-nums; } +.board-col-pace { width: 90px; flex-shrink: 0; text-align: center; font-variant-numeric: tabular-nums; } +.board-col-eta { width: 70px; flex-shrink: 0; text-align: right; font-variant-numeric: tabular-nums; } .board-col-spark { flex: 1; min-width: 100px; } .board-col-alert { width: 110px; flex-shrink: 0; text-align: right; } +/* Pace cell coloring — green when on reference, orange when slow, + red when seriously slow. Class names mirror _formatPace(). */ +.board-col-pace.pace-unknown { color: var(--text-muted); } +.board-col-pace.pace-normal { color: var(--accent-green, #4ade80); } +.board-col-pace.pace-slow { color: #fb923c; } +.board-col-pace.pace-slow-bad { color: #f87171; font-weight: 600; } + +/* Subtle overdue mark in the stereo cell when clock has run past the + expected stage duration. */ +.stereo-overdue { color: #fb923c; margin-left: 4px; } + .board-rows { flex: 1; } .board-row { @@ -3500,8 +3871,10 @@ kbd { scrollbar shared by all rows. Labels pin to the left via position:sticky inside each row. */ display: block; + flex: 1 1 0; /* flex-1 child: shrinks/grows as the panel opens */ + min-width: 0; overflow-x: auto; - overflow-y: hidden; + overflow-y: auto; scrollbar-width: thin; scrollbar-gutter: stable; position: relative; @@ -3634,6 +4007,10 @@ kbd { border-radius: 4px; border: 2px solid; object-fit: cover; + /* The stored projection is a three-view ([XY|YZ] over [XZ]); the embryo is + in the LEFT column (XY/XZ), and the centre is the black XY|YZ divider. + Crop to the left so the square thumbnail shows the embryo, not the gap. */ + object-position: left center; background: var(--bg-dark); } @@ -3658,15 +4035,25 @@ kbd { .filmstrip-detail { background: var(--bg-card); - /* Cap the detail panel so it always leaves the rows visible AND - has a scrollable body of its own. Without this, when an item is - expanded the detail panel can grow past the viewport bottom and - the parent's scroll is unintuitive (mouse wheel over the rows - converts to horizontal). max-height keeps it bounded; overflow-y - lets long VLM summaries scroll on their own. */ - max-height: 60vh; + /* Right-side panel: fixed-ish width that shrinks gracefully on narrow + viewports. Scrolls vertically inside itself so long VLM summaries + don't push the layout. When empty (no frame selected) it collapses + entirely so the rows get full width. (Recovered from WIP 0269e18d.) */ + flex: 0 0 auto; + width: clamp(360px, 32vw, 520px); + border-left: 1px solid var(--border); overflow-y: auto; overscroll-behavior: contain; + animation: filmstripDetailIn 0.18s ease-out; +} +.filmstrip-detail:empty { display: none; } +@keyframes filmstripDetailIn { + from { transform: translateX(8px); opacity: 0; } + to { transform: translateX(0); opacity: 1; } +} +/* In the narrow side panel, stack the image | VLM summary split vertically. */ +#filmstrip-detail .detail-split { + grid-template-columns: 1fr; } /* ======================================== @@ -8698,7 +9085,7 @@ body.modal-open { .settings-content { flex: 1; max-width: 720px; - padding: 2rem 2.5rem; + padding: 2rem 2.5rem 0; overflow-y: auto; } @@ -8820,6 +9207,46 @@ body.modal-open { color: var(--text-muted); } +/* Thermalizer / server-backed settings controls */ +.th-note { margin-bottom: 0.9rem; } +.th-ro { font-size: 0.8rem; font-weight: 400; color: var(--text-muted); } +.th-status { + font-size: 0.85rem; color: var(--text); font-variant-numeric: tabular-nums; + padding: 0.5rem 0.7rem; background: var(--bg-hover); + border: 1px solid var(--border); border-radius: 6px; +} +.th-actions { display: flex; gap: 0.6rem; } +.settings-btn { + background: var(--bg-hover); border: 1px solid var(--border); color: var(--text); + font: 600 0.82rem/1 'Inter Tight', system-ui, sans-serif; + padding: 0.55rem 0.9rem; border-radius: 7px; cursor: pointer; + transition: background 0.13s, border-color 0.13s, color 0.13s, opacity 0.13s; +} +.settings-btn:hover:not(:disabled) { border-color: var(--accent); color: var(--accent); } +.settings-btn:disabled { opacity: 0.5; cursor: progress; } +.settings-btn-primary { background: var(--accent); border-color: var(--accent); color: #fff; } +.settings-btn-primary:hover:not(:disabled) { background: var(--accent-hover); border-color: var(--accent-hover); color: #fff; } +.settings-result { font-size: 0.8rem; margin-top: 0.6rem; min-height: 1.1em; } +.settings-result.is-ok { color: var(--accent-green); } +.settings-result.is-err { color: var(--accent-orange); } +.settings-pre { + font: 0.78rem/1.5 ui-monospace, SFMono-Regular, monospace; + color: var(--text); background: var(--bg-dark); border: 1px solid var(--border); + border-radius: 8px; padding: 0.8rem 1rem; overflow-x: auto; white-space: pre; + max-height: 460px; overflow-y: auto; +} +.settings-defaults-bar { display: flex; gap: 0.5rem; margin-left: auto; flex-wrap: wrap; } +.settings-subhead { + font-size: 0.72rem; font-weight: 700; letter-spacing: 0.06em; text-transform: uppercase; + color: var(--text-muted); margin: 1.4rem 0 0.6rem; padding-bottom: 0.35rem; + border-bottom: 1px solid var(--border); +} +.settings-subhead:first-child { margin-top: 0.2rem; } +.settings-pre code, .settings-hint code { + font: 0.85em ui-monospace, SFMono-Regular, monospace; + background: var(--bg-hover); padding: 0.05rem 0.3rem; border-radius: 4px; +} + /* Range slider */ .settings-range { -webkit-appearance: none; @@ -8863,6 +9290,13 @@ body.modal-open { position: sticky; bottom: 0; padding: 0.75rem 0; + display: flex; + align-items: center; + gap: 0.75rem; + flex-wrap: wrap; + background: var(--bg-dark); + border-top: 1px solid var(--border); + z-index: 5; } .settings-save-status { @@ -8993,43 +9427,111 @@ body.modal-open { .devices-status-pill.stale { background: rgba(210, 153, 34, 0.2); color: var(--accent-orange, #d29922); } .devices-status-pill.error { background: rgba(248, 81, 73, 0.2); color: #f85149; } -.devices-positions { - display: grid; - grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); - gap: 0.75rem; -} - -.devices-card { +/* Device-layer supervision card (device-layer.js) */ +.devices-layer-card { background: var(--bg-card); border: 1px solid var(--border); border-radius: 6px; - padding: 0.75rem 1rem; + padding: 0.6rem 0.9rem; + margin: 0.75rem 0 0.25rem; } - -.devices-card-title { - font-size: 0.7rem; - text-transform: uppercase; - letter-spacing: 0.06em; - color: var(--text-muted); - margin-bottom: 0.5rem; +.devices-layer-head { + display: flex; + align-items: center; + gap: 0.6rem; + flex-wrap: wrap; } - -.devices-position-row { - display: grid; - grid-template-columns: 24px 1fr auto; - align-items: baseline; - gap: 0.5rem; - padding: 0.15rem 0; +.devices-layer-head .devices-card-title { margin-bottom: 0; } +.devices-layer-meta { + font-family: 'SF Mono', 'Consolas', monospace; + font-size: 0.72rem; + color: var(--text-muted); + font-variant-numeric: tabular-nums; } - -.devices-axis-label { +.devices-layer-hint { + font-size: 0.72rem; color: var(--text-muted); - font-size: 0.85rem; - font-weight: 500; } - -.devices-axis-value { - font-family: 'SF Mono', 'Consolas', monospace; +.devices-layer-actions { + display: flex; + align-items: center; + gap: 0.4rem; + margin-top: 0.55rem; + flex-wrap: wrap; +} +/* Stop tears down the running device layer — the destructive action, so it + reads red (filled + active when there's something to stop). When nothing is + running it stays red-tinted but clearly inactive, mirroring how Start is + blue-active/blue-faded. Overrides the generic .marking-action-btn:disabled + 0.4 opacity so the red stays legible while disabled. */ +#devices-layer-stop { + background: #f85149; + border-color: #f85149; + color: #fff; +} +#devices-layer-stop:hover:not(:disabled) { + filter: brightness(1.08); + background: #f85149; +} +#devices-layer-stop:disabled { + opacity: 1; + background: transparent; + border-color: rgba(248, 81, 73, 0.4); + color: rgba(248, 81, 73, 0.55); +} +.devices-layer-log { + margin: 0.55rem 0 0; + padding: 0.5rem 0.65rem; + max-height: 220px; + overflow: auto; + background: var(--bg-hover); + border: 1px solid var(--border); + border-radius: 5px; + font-family: 'SF Mono', 'Consolas', monospace; + font-size: 0.7rem; + line-height: 1.5; + color: var(--text-muted); + white-space: pre-wrap; + word-break: break-word; +} + +.devices-positions { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); + gap: 0.75rem; +} + +.devices-card { + background: var(--bg-card); + border: 1px solid var(--border); + border-radius: 6px; + padding: 0.75rem 1rem; +} + +.devices-card-title { + font-size: 0.7rem; + text-transform: uppercase; + letter-spacing: 0.06em; + color: var(--text-muted); + margin-bottom: 0.5rem; +} + +.devices-position-row { + display: grid; + grid-template-columns: 24px 1fr auto; + align-items: baseline; + gap: 0.5rem; + padding: 0.15rem 0; +} + +.devices-axis-label { + color: var(--text-muted); + font-size: 0.85rem; + font-weight: 500; +} + +.devices-axis-value { + font-family: 'SF Mono', 'Consolas', monospace; font-size: 1.05rem; color: var(--text); text-align: right; @@ -9172,6 +9674,7 @@ body.modal-open { --map-zone-green: 90, 168, 122; /* RGB triples for compositing */ --map-zone-orange: 215, 152, 84; --map-zone-red: 220, 96, 88; + --map-embryo: 156, 120, 220; /* lavender — distinct from zones and marker */ --map-overlay-bg: rgba(11, 14, 19, 0.78); --map-overlay-bg-2: rgba(11, 14, 19, 0.92); --map-overlay-edge: rgba(212, 221, 232, 0.18); @@ -9189,6 +9692,7 @@ body.modal-open { --map-accent: #0e7490; --map-accent-2: #155e75; --map-warm: #a16207; + --map-embryo: 100, 60, 180; /* deeper purple for cream paper */ --map-overlay-bg: rgba(246, 243, 236, 0.82); --map-overlay-bg-2: rgba(246, 243, 236, 0.96); --map-overlay-edge: rgba(29, 43, 58, 0.18); @@ -9252,6 +9756,179 @@ body.modal-open { .devices-status-led.paused::before{ background: var(--map-accent); } .devices-status-led.stale::before { background: var(--map-warm); } .devices-status-led.error::before { background: #f87171; } +/* Status sub-label (e.g. the live "Δt 0.20s"). The value updates every frame; + tabular figures + a reserved, right-aligned width keep its box from changing + size, so the rapidly-changing digits can't reflow and jitter the LIVE badge + and room-light button beside it (the cluster is right-anchored in the bar). */ +.devices-status-meta { + font-variant-numeric: tabular-nums; + min-width: 3.4rem; + text-align: right; +} + +/* --- Room-light toggle (header) -------------------------------------- */ +.devices-room-light { + display: inline-flex; + align-items: center; + gap: 0.4rem; + padding: 0.18rem 0.6rem 0.18rem 0.45rem; + border: 1px solid var(--map-overlay-edge); + background: var(--map-overlay-bg); + border-radius: 999px; + color: var(--map-ink-mute); + font-family: inherit; + font-size: 0.65rem; + font-weight: 600; + letter-spacing: 0.06em; + text-transform: uppercase; + cursor: pointer; + transition: color 0.15s, border-color 0.15s, background 0.15s; +} +.devices-room-light[hidden] { display: none; } +.devices-room-light:hover:not(:disabled) { + border-color: var(--map-accent); + color: var(--map-ink); +} +.devices-room-light:disabled { opacity: 0.5; cursor: default; } +.devices-room-light-bulb { + display: inline-flex; + align-items: center; + color: var(--map-ink-mute); + transition: color 0.15s, filter 0.15s; +} +/* "on" — warm glow on the bulb to read like a lit lamp */ +.devices-room-light.is-on { + border-color: rgba(255, 210, 74, 0.7); + color: #ffd24a; + background: rgba(255, 210, 74, 0.12); +} +.devices-room-light.is-on .devices-room-light-bulb { + color: #ffd24a; + filter: drop-shadow(0 0 5px rgba(255, 210, 74, 0.7)); +} +.devices-room-light.is-busy { opacity: 0.65; cursor: progress; } + +/* --- Temperature controller ------------------------------------------- */ +/* Readout + setpoint input + Set, styled as a pill to sit beside the + room-light toggle in the Devices header. Hidden until a controller is + available (mirrors the room light). */ +.devices-temp { + display: inline-flex; + align-items: center; + gap: 0.35rem; + padding: 0.12rem 0.28rem 0.12rem 0.5rem; + border: 1px solid var(--map-overlay-edge); + background: var(--map-overlay-bg); + border-radius: 999px; + color: var(--map-ink-mute); + font-family: inherit; + font-size: 0.65rem; + font-weight: 600; + letter-spacing: 0.06em; + text-transform: uppercase; +} +.devices-temp[hidden] { display: none; } +.devices-temp-icon { display: inline-flex; align-items: center; color: var(--map-ink-mute); transition: color 0.15s, filter 0.15s; } +/* "locked" — cool glow once the controller reports SYSTEM LOCKED */ +.devices-temp.is-locked { border-color: rgba(90, 200, 250, 0.6); color: #5ac8fa; background: rgba(90, 200, 250, 0.1); } +.devices-temp.is-locked .devices-temp-icon { color: #5ac8fa; filter: drop-shadow(0 0 5px rgba(90, 200, 250, 0.6)); } +.devices-temp.is-busy { opacity: 0.7; cursor: progress; } +.devices-temp-readout { + font-variant-numeric: tabular-nums; + letter-spacing: 0.02em; + min-width: 3.4em; + text-align: right; + white-space: nowrap; +} +.devices-temp-input { + width: 3.4em; + padding: 0.1rem 0.3rem; + border: 1px solid var(--map-overlay-edge); + background: var(--map-paper, rgba(0, 0, 0, 0.2)); + border-radius: 6px; + color: var(--map-ink); + font-family: inherit; + font-size: 0.72rem; + font-variant-numeric: tabular-nums; + text-align: right; + -moz-appearance: textfield; +} +.devices-temp-input:focus { outline: none; border-color: var(--map-accent); } +.devices-temp-input::-webkit-outer-spin-button, +.devices-temp-input::-webkit-inner-spin-button { -webkit-appearance: none; margin: 0; } +.devices-temp-set { + padding: 0.16rem 0.5rem; + border: 1px solid var(--map-overlay-edge); + background: var(--map-overlay-bg); + border-radius: 999px; + color: var(--map-ink-mute); + font-family: inherit; + font-size: 0.62rem; + font-weight: 700; + letter-spacing: 0.06em; + text-transform: uppercase; + cursor: pointer; + transition: color 0.15s, border-color 0.15s, background 0.15s; +} +.devices-temp-set:hover:not(:disabled) { border-color: var(--map-accent); color: var(--map-ink); } +.devices-temp-set:disabled { opacity: 0.5; cursor: default; } + +/* --- Temperature trajectory graph ------------------------------------- */ +/* SVG chart card mounted at the bottom of #devices-content. */ +.devices-temp-graph { + padding: 0.6rem 1rem 0.5rem; + border-top: 1px solid var(--map-overlay-edge); + min-height: 2rem; /* keeps the section visible even when empty */ +} +/* Solid line: water temperature */ +.temp-water { + stroke: var(--temp-water-color); + stroke-width: 1.5; + stroke-linejoin: round; + stroke-linecap: round; +} +/* Dashed line: setpoint (stepped) */ +.temp-setpoint { + stroke: var(--temp-setpoint-color); + stroke-width: 1.5; + stroke-dasharray: 5 3; + stroke-linejoin: round; + stroke-linecap: round; + opacity: 0.85; +} +/* Y-axis reference gridlines */ +.temp-grid-line { + stroke: var(--temp-grid-line-color); + stroke-width: 0.75; +} +/* Y-axis tick labels */ +.temp-grid-label { + fill: var(--temp-grid-label-color); + font-size: 8px; + font-family: ui-monospace, monospace; + text-anchor: end; + dominant-baseline: middle; +} +/* Calm empty state — never shows mock data */ +.temp-graph-empty { + padding: 1rem 0; + text-align: center; + color: var(--map-ink-mute); + font-size: 0.78rem; +} +/* Last-sample readout above the SVG */ +.temp-graph-readout { + font-size: 0.68rem; + font-weight: 600; + font-variant-numeric: tabular-nums; + letter-spacing: 0.04em; + color: var(--map-ink-mute); + margin-bottom: 0.2rem; + padding: 0 2px; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} /* --- Containers ------------------------------------------------------- */ .devices-view { display: flex; flex-direction: column; flex: 1; min-height: 0; } @@ -9429,6 +10106,53 @@ body.modal-open { 100% { opacity: 0; r: 28; } } +/* --- Embryo waypoints ------------------------------------------------ */ +/* Coarse = bottom-camera / manual placement; fine = SPIM-objective + alignment. Coarse reads as an outlined ring (provisional), fine as a + filled disc (committed). Same hue so the row of embryos still reads as + one cohort, but visual weight signals calibration state at a glance. */ +.devices-embryo-group { + cursor: pointer; +} +.devices-embryo-ring { + fill: rgba(var(--map-embryo), 0.08); + stroke: rgba(var(--map-embryo), 0.85); + stroke-width: 1.4; + vector-effect: non-scaling-stroke; +} +.devices-embryo-disc { + fill: rgba(var(--map-embryo), 0.65); + stroke: rgba(var(--map-embryo), 0.95); + stroke-width: 1.4; + vector-effect: non-scaling-stroke; +} +.devices-embryo-label { + fill: var(--map-ink); + font-family: 'JetBrains Mono', ui-monospace, monospace; + font-weight: 600; + text-anchor: middle; + dominant-baseline: central; + pointer-events: none; + paint-order: stroke; + stroke: var(--map-paper); + stroke-width: 2; + stroke-linejoin: round; +} + +/* Selected = "picked up" — outlined dashed, hollow fill, brighter label. + Click on empty map drops the picked-up embryo at that XY; Delete / + Backspace removes it; Escape deselects. */ +.devices-embryo-group.devices-embryo-selected .devices-embryo-ring, +.devices-embryo-group.devices-embryo-selected .devices-embryo-disc { + fill: rgba(var(--map-embryo), 0.12); + stroke: rgba(var(--map-embryo), 1); + stroke-width: 2; + stroke-dasharray: 4 3; +} +.devices-embryo-group.devices-embryo-selected .devices-embryo-label { + fill: rgba(var(--map-embryo), 1); +} + /* --- Overlay panels (compass, readout, scalebar, legend) ------------- */ .devices-compass, .devices-map-readout, @@ -9619,6 +10343,72 @@ body.modal-open { color: var(--map-paper); border-color: var(--map-accent); } + +/* ---- Top-right rail: the XY stage readout (Detect/mark moved to Operate) ---- */ +.devices-map-rail { + position: absolute; + top: 0.85rem; + right: 0.85rem; + width: 230px; + display: flex; + flex-direction: column; + gap: 0.5rem; + z-index: 4; +} +/* Readout drops its own absolute positioning inside the rail and fills it. */ +.devices-map-rail .devices-map-readout { + position: static; + top: auto; + right: auto; + width: 100%; + min-width: 0; +} + + +.devices-camera-head-btns { + display: inline-flex; + align-items: center; + gap: 0.3rem; +} +.devices-camera-expand { + background: transparent; + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink); + font-family: inherit; + font-size: 0.72rem; + line-height: 1; + padding: 0.16rem 0.4rem; + border-radius: 999px; + cursor: pointer; + transition: background 0.15s, border-color 0.15s, color 0.15s; +} +.devices-camera-expand:hover { + background: rgba(255,255,255,0.06); + border-color: var(--map-accent); + color: var(--map-accent); +} +.devices-camera-expand.active { + background: var(--map-accent); + color: var(--map-paper); + border-color: var(--map-accent); +} +/* Enlarged bottom-camera view — centred over the map, toggled by the + expand button. Reuses the same streaming ; sized by height so the + whole panel (header + stage + foot) stays within the map area. */ +.devices-camera-panel.expanded { + bottom: auto; + left: 50%; + top: 50%; + transform: translate(-50%, -50%); + width: auto; + max-width: calc(100% - 2rem); + z-index: 9; +} +.devices-camera-panel.expanded .devices-camera-stage { + height: min(64vh, 640px); + width: auto; + max-width: 100%; +} .devices-camera-stage { position: relative; width: 100%; @@ -9635,9 +10425,44 @@ body.modal-open { display: block; opacity: 0; transition: opacity 0.25s; + /* Zoom anchored at frame centre; scroll-wheel + cursor adjust translate + so the point under the cursor stays under the cursor. */ + transform-origin: center center; + will-change: transform; } .devices-camera-img.has-frame { opacity: 1; } +/* Cursor hints for zoom/pan mode. Default cursor stays untouched at zoom 1 + so the operator can still interact with overlays under the camera. */ +.devices-camera-stage.camera-zoomed { cursor: grab; } +.devices-camera-stage.camera-panning { cursor: grabbing; } + +/* Centre reticle — full-span horizontal + vertical hairline marking the + FOV centre IN the image. SVG is a sibling of ; the inner + receives the same translate/scale (in viewBox units) so the lines + track the camera image through zoom/pan instead of staying pinned to + the viewer rect. Transform lives on the , not the SVG element, so + the renderer re-rasterises at each zoom step — otherwise the strokes + get bitmap-scaled and go blurry. */ +.devices-camera-crosshair { + position: absolute; + inset: 0; + width: 100%; + height: 100%; + pointer-events: none; + opacity: 0; + transition: opacity 0.25s; +} +.devices-camera-stage:has(.devices-camera-img.has-frame) .devices-camera-crosshair { + opacity: 1; +} +.devices-camera-crosshair line { + stroke: var(--map-warm); + stroke-width: 1; + vector-effect: non-scaling-stroke; + stroke-opacity: 0.85; +} + .devices-camera-placeholder { position: absolute; inset: 0; @@ -9915,3 +10740,838 @@ body.modal-open { opacity: 0.5; cursor: not-allowed; } + +/* ========================================================================= + Manual view — lightsheet live panel + control rail + ========================================================================= */ + +/* Two-column layout: image stage (flex-grow) + fixed control rail */ +.ls-layout { + display: flex; + flex: 1; + min-height: 0; + gap: 1rem; + font-family: 'Inter Tight', system-ui, sans-serif; +} + +/* Left column — live image stage */ +.ls-image-col { + flex: 1; + min-width: 0; + display: flex; + flex-direction: column; + gap: 0.45rem; +} + +.ls-stage-head { + display: flex; + align-items: center; + gap: 0.6rem; + flex-wrap: wrap; +} + +/* The stage itself reuses .devices-camera-stage aspect + overflow */ +.ls-stage { + flex: 1; + min-height: 280px; + /* override aspect-ratio: let the flex column stretch it */ + aspect-ratio: unset; +} + +/* Right column — control rail. + Two-column masonry so every control stays visible without scrolling. + multicol packs the varying-height cards tighter than a grid would (no + row-alignment gaps); each .ls-group sets break-inside to stay whole. + overflow-y:auto is a safety net only — the default (collapsed) control + set fits with no scrollbar; it appears solely if Timelapse is expanded. */ +.ls-rail { + width: 380px; + flex-shrink: 0; + columns: 2; + column-gap: 0.7rem; + overflow-y: auto; + padding: 0.1rem 0 0.5rem; +} + +/* Control group block */ +.ls-group { + display: flex; + flex-direction: column; + gap: 0.32rem; + padding: 0.55rem 0.65rem; + background: var(--map-overlay-bg); + border: 1px solid var(--map-overlay-edge); + border-radius: 5px; + /* multicol: keep each card intact and space them (rail gap is gone) */ + break-inside: avoid; + margin-bottom: 0.7rem; +} + +/* Stream group: the live toggle, relocated from the viewer header to a + prominent full-width button at the top of the rail. column-span makes it + a header across both columns, whose top lines up with the image stage. */ +.ls-group-stream { + column-span: all; + /* no card frame — this is just the toggle, not a boxed control group */ + background: transparent; + border: none; + padding: 0; + /* top margin ≈ the viewer's header row height so the button lines up + with the top of the image-stage box (not the "LIGHTSHEET" label row). */ + margin-top: 1.05rem; + margin-bottom: 0.85rem; +} +/* Slim outline "● LIVE" toggle. Muted when off; glows green while streaming + (the .active class is toggled on by applyLightsheetState). */ +.ls-stream-btn { + display: inline-flex; + align-items: center; + justify-content: center; + gap: 0.45rem; + width: 100%; + padding: 0.34rem 0.6rem; + background: transparent; + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink-mute); + font-family: inherit; + font-size: 0.66rem; + font-weight: 600; + letter-spacing: 0.14em; + text-transform: uppercase; + border-radius: 5px; + cursor: pointer; + transition: background 0.12s, border-color 0.12s, color 0.12s; +} +.ls-stream-btn::before { + content: ''; + width: 6px; + height: 6px; + border-radius: 50%; + background: var(--map-ink-mute); + flex: none; + transition: background 0.12s, box-shadow 0.12s; +} +.ls-stream-btn:hover { border-color: var(--map-accent); color: var(--map-ink); } +.ls-stream-btn.active { + border-color: #4ade80; + color: #4ade80; + background: rgba(74, 222, 128, 0.10); +} +.ls-stream-btn.active::before { + background: #4ade80; + box-shadow: 0 0 8px rgba(74, 222, 128, 0.7); +} + +.ls-label { + font-size: 0.6rem; + font-weight: 600; + letter-spacing: 0.18em; + text-transform: uppercase; + color: var(--map-ink-mute); +} + +/* Generic inline row */ +.ls-row { + display: flex; + align-items: center; + gap: 0.4rem; +} + +/* Slider + number inline */ +.ls-slider-row { + display: flex; + align-items: center; + gap: 0.35rem; +} + +.ls-slider { + flex: 1; + min-width: 0; + height: 3px; + accent-color: var(--map-accent); + cursor: pointer; +} + +.ls-number { + width: 64px; + background: var(--map-paper-2); + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink); + border-radius: 4px; + padding: 0.2rem 0.35rem; + font-family: 'JetBrains Mono', monospace; + font-size: 0.7rem; + font-variant-numeric: tabular-nums; +} + +.ls-number-sm { + width: 50px; +} + +.ls-number:focus { + outline: none; + border-color: var(--map-accent); +} + +.ls-unit { + font-family: 'Inter Tight', system-ui, sans-serif; + font-size: 0.62rem; + color: var(--map-ink-mute); + flex-shrink: 0; +} + +/* Illumination button rows */ +.ls-btn-row { + display: flex; + gap: 0.35rem; + flex-wrap: wrap; +} + +.ls-illum-btn { + flex: 1; + /* Reserve enough width that labels wrap to a new row instead of + truncating to an ellipsis in the narrower 2-column rail. */ + min-width: 4.25rem; + background: transparent; + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink); + font-family: inherit; + font-size: 0.6rem; + font-weight: 600; + letter-spacing: 0.06em; + text-transform: uppercase; + padding: 0.28rem 0.4rem; + border-radius: 4px; + cursor: pointer; + transition: background 0.12s, border-color 0.12s, color 0.12s; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} + +.ls-illum-btn:hover { + background: rgba(255,255,255,0.06); + border-color: var(--map-accent); + color: var(--map-accent); +} + +.ls-illum-btn--active { + background: rgba(34, 211, 238, 0.14); + border-color: var(--map-accent); + color: var(--map-accent); +} + +/* Laser-off indicator */ +.ls-laser-indicator { + font-size: 0.6rem; + color: var(--map-ink-mute); + display: flex; + align-items: center; + gap: 0.35rem; + margin-top: 0.1rem; +} + +.ls-laser-dot { + width: 6px; + height: 6px; + border-radius: 50%; + background: var(--map-ink-mute); + flex-shrink: 0; +} + +.ls-laser-dot--on { + background: var(--map-accent); +} + +/* Laser preset row (B2) */ +.ls-laser-row { + display: flex; + align-items: center; + gap: 0.4rem; + margin-top: 0.2rem; +} + +.ls-laser-label { + font-size: 0.6rem; + font-weight: 600; + letter-spacing: 0.12em; + text-transform: uppercase; + color: var(--map-ink-mute); + flex-shrink: 0; +} + +.ls-laser-select { + flex: 1; + min-width: 0; + background: var(--map-paper-2); + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink); + font-family: inherit; + font-size: 0.62rem; + border-radius: 4px; + padding: 0.18rem 0.3rem; + cursor: pointer; + transition: border-color 0.12s; +} + +.ls-laser-select:focus { + outline: none; + border-color: var(--map-accent); +} + +/* Temperature setpoint button */ +.ls-set-btn { + background: transparent; + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink); + font-family: inherit; + font-size: 0.6rem; + font-weight: 600; + letter-spacing: 0.08em; + text-transform: uppercase; + padding: 0.2rem 0.55rem; + border-radius: 4px; + cursor: pointer; + transition: background 0.12s, border-color 0.12s; +} + +.ls-set-btn:hover { + background: rgba(255,255,255,0.06); + border-color: var(--map-accent); +} + +/* Temperature mini-graph */ +.ls-tempgraph { + min-height: 48px; + margin-top: 0.25rem; +} + +/* Acquire buttons */ +.ls-acquire-btn { + flex: 1; + background: transparent; + border: 1px solid var(--map-overlay-edge); + color: var(--map-ink); + font-family: inherit; + font-size: 0.62rem; + font-weight: 600; + letter-spacing: 0.06em; + text-transform: uppercase; + padding: 0.35rem 0.5rem; + border-radius: 4px; + cursor: pointer; + transition: background 0.12s, border-color 0.12s, color 0.12s; +} + +.ls-acquire-btn:hover:not(:disabled) { + background: rgba(255,255,255,0.06); + border-color: var(--map-accent); + color: var(--map-accent); +} + +.ls-acquire-btn:disabled { + opacity: 0.5; + cursor: not-allowed; +} + +/* Last-captured ref card */ +.ls-lastcap { + margin-top: 0.35rem; + padding: 0.35rem 0.45rem; + background: var(--map-paper-2); + border: 1px solid var(--map-overlay-edge); + border-radius: 4px; + display: flex; + flex-direction: column; + gap: 0.2rem; +} + +.ls-lastcap[hidden] { display: none; } + +.ls-lastcap-label { + font-size: 0.58rem; + letter-spacing: 0.14em; + text-transform: uppercase; + color: var(--map-ink-mute); +} + +.ls-lastcap-ref { + font-family: 'JetBrains Mono', monospace; + font-size: 0.65rem; + color: var(--map-ink); + word-break: break-all; +} + +/* ── Timelapse collapsible panel (B2 Task 3) ─────────────────────────────── */ + +/* Clickable header button (also carries .ls-label for font/color) */ +.ls-collapsible-head { + display: flex; + align-items: center; + justify-content: space-between; + width: 100%; + background: transparent; + border: none; + padding: 0; + cursor: pointer; + text-align: left; +} + +/* ▶/▼ caret — rotates when expanded */ +.ls-collapsible-arrow { + font-size: 0.5rem; + color: var(--map-ink-mute); + flex-shrink: 0; + transition: transform 0.15s; +} + +.ls-collapsible-head[aria-expanded="true"] .ls-collapsible-arrow { + transform: rotate(90deg); +} + +/* Collapsible body — hidden via [hidden] attribute; explicit rule prevents + display overrides from fighting the attribute (mirrors .ls-lastcap[hidden]) */ +.ls-collapsible-body { + display: flex; + flex-direction: column; + gap: 0.32rem; + margin-top: 0.35rem; +} + +.ls-collapsible-body[hidden] { display: none; } + +/* Sub-rows: like .ls-row but slightly indented for the body interior */ +.ls-sub-row { + display: flex; + align-items: center; + gap: 0.4rem; + padding-left: 0.25rem; +} + +/* Sub-label: like .ls-label but fixed-width to align inputs column */ +.ls-sub-label { + font-size: 0.58rem; + font-weight: 600; + letter-spacing: 0.12em; + text-transform: uppercase; + color: var(--map-ink-mute); + min-width: 4.5rem; + flex-shrink: 0; +} + +/* Volume-Geometry inset sub-group */ +.ls-sub-section { + display: flex; + flex-direction: column; + gap: 0.28rem; + padding: 0.4rem 0.5rem; + background: var(--map-paper-2); + border: 1px solid var(--map-overlay-edge); + border-radius: 4px; + margin: 0.15rem 0; +} + +/* "VOLUME GEOMETRY" sub-header */ +.ls-sub-section-label { + font-size: 0.52rem; + font-weight: 600; + letter-spacing: 0.2em; + text-transform: uppercase; + color: var(--map-ink-mute); + margin-bottom: 0.1rem; +} + +/* Start-result / status line — mono, muted */ +.ls-tl-status { + font-family: 'JetBrains Mono', monospace; + font-size: 0.62rem; + color: var(--map-ink-mute); + padding: 0.2rem 0.25rem; + word-break: break-all; +} + +.ls-tl-status[hidden] { display: none; } + +/* ── Timelapse accordion (active/inactive section states — B3 UX) ─────────── */ + +/* Wrapper for each accordion section */ +.ls-acc-section { + display: flex; + flex-direction: column; + border-bottom: 1px solid var(--map-rule-soft); + padding-bottom: 0.12rem; + margin-bottom: 0.08rem; +} +.ls-acc-section:last-of-type { + border-bottom: none; + margin-bottom: 0; +} + +/* Section header button */ +.ls-acc-head { + display: flex; + align-items: center; + gap: 0.32rem; + background: transparent; + border: none; + padding: 0.26rem 0; + cursor: pointer; + width: 100%; + text-align: left; + font-family: inherit; +} + +/* Status dot — hollow/muted when inactive */ +.ls-acc-dot { + width: 5px; + height: 5px; + border-radius: 50%; + border: 1px solid var(--map-ink-mute); + background: transparent; + flex-shrink: 0; + transition: background 0.15s, border-color 0.15s; +} + +/* Section label */ +.ls-acc-title { + font-size: 0.575rem; + font-weight: 600; + letter-spacing: 0.12em; + text-transform: uppercase; + color: var(--map-ink-mute); + flex-shrink: 0; + transition: color 0.15s; +} + +/* One-line summary — shown only when section is touched */ +.ls-acc-summary { + flex: 1; + font-size: 0.57rem; + font-family: 'JetBrains Mono', monospace; + letter-spacing: 0.02em; + color: var(--map-ink-mute); + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; + min-width: 0; +} +.ls-acc-summary[hidden] { display: none; } + +/* Caret arrow */ +.ls-acc-arrow { + font-size: 0.44rem; + color: var(--map-ink-mute); + flex-shrink: 0; + margin-left: auto; + transition: transform 0.15s, color 0.15s; +} +.ls-acc-head[aria-expanded="true"] .ls-acc-arrow { + transform: rotate(90deg); +} + +/* ── Active / touched state ────────────────────────────────────────────────── */ +.ls-acc-head.is-active .ls-acc-dot { + background: var(--map-accent); + border-color: var(--map-accent); +} +.ls-acc-head.is-active .ls-acc-title { + color: var(--map-accent); +} +.ls-acc-head.is-active .ls-acc-summary { + color: var(--map-accent); + opacity: 0.85; +} +.ls-acc-head.is-active .ls-acc-arrow { + color: var(--map-accent); +} + +/* Section body */ +.ls-acc-body { + display: flex; + flex-direction: column; + gap: 0.26rem; + padding: 0.18rem 0 0.32rem 0.5rem; +} +.ls-acc-body[hidden] { display: none; } + +/* Outer timelapse panel header dot — signals any section is set */ +.ls-tl-outer-dot { + width: 5px; + height: 5px; + border-radius: 50%; + border: 1px solid transparent; + background: transparent; + flex-shrink: 0; + transition: background 0.15s, border-color 0.15s; +} +.ls-tl-outer-dot.is-active { + background: var(--map-accent); + border-color: var(--map-accent); +} + +/* Start button reads "ready" (accent) once any section is configured */ +.ls-tl-start-btn { + transition: border-color 0.15s, color 0.15s, background 0.15s; +} +.ls-tl-start-btn.is-ready { + border-color: var(--map-accent); + color: var(--map-accent); + background: rgba(34, 211, 238, 0.08); +} + +/* Submit row — give the button its full row width */ +.ls-tl-submit-row { + margin-top: 0.3rem; +} + +/* ========================================== + Gallery Tab + ========================================== */ + +.gallery-tab-container { + display: flex; + flex-direction: column; + height: 100%; + overflow: hidden; + padding: 1.25rem 1.5rem 0; + box-sizing: border-box; +} + +.gallery-tab-header { + display: flex; + align-items: center; + gap: 1rem; + margin-bottom: 1rem; + flex-shrink: 0; +} + +.gallery-tab-title { + font-size: 1.35rem; + font-weight: 300; + letter-spacing: -0.01em; + color: var(--text); + margin: 0; + line-height: 1.2; + flex-shrink: 0; +} + +.gallery-tab-title-script { + font-family: 'Georgia', serif; + font-style: italic; + color: var(--text-muted); +} + +.gallery-tab-title-em { + font-style: normal; + font-weight: 500; + color: var(--accent); +} + +.gallery-tab-controls { + display: flex; + align-items: center; + gap: 0.5rem; + margin-left: auto; +} + +.gallery-filter-select { + background: var(--bg-card); + border: 1px solid var(--border); + border-radius: 6px; + color: var(--text); + font-size: 0.8rem; + padding: 0.3rem 0.6rem; + cursor: pointer; + transition: border-color 0.15s; +} + +.gallery-filter-select:hover, +.gallery-filter-select:focus { + border-color: var(--accent); + outline: none; +} + +.gallery-refresh-btn { + background: transparent; + border: 1px solid var(--border); + border-radius: 6px; + color: var(--text-muted); + font-size: 1rem; + width: 2rem; + height: 2rem; + display: flex; + align-items: center; + justify-content: center; + cursor: pointer; + transition: color 0.15s, border-color 0.15s; +} + +.gallery-refresh-btn:hover { + color: var(--accent); + border-color: var(--accent); +} + +.gallery-tab-body { + flex: 1; + overflow-y: auto; + padding-bottom: 1.5rem; +} + +.gallery-tab-empty, +.gallery-tab-loading, +.gallery-tab-error { + display: flex; + align-items: center; + justify-content: center; + height: 200px; + color: var(--text-muted); + font-size: 0.9rem; +} + +.gallery-tab-error { + color: var(--accent-red, #ef4444); +} + +.gallery-tab-grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(160px, 1fr)); + gap: 0.75rem; +} + +.gallery-tab-item { + background: var(--bg-card); + border: 1px solid var(--border); + border-radius: 8px; + overflow: hidden; + cursor: pointer; + transition: border-color 0.15s, transform 0.15s, box-shadow 0.15s; +} + +.gallery-tab-item:hover { + border-color: var(--accent); + transform: translateY(-2px); + box-shadow: 0 4px 16px rgba(0, 0, 0, 0.25); +} + +.gallery-tab-thumb { + display: block; + width: 100%; + aspect-ratio: 1; + object-fit: contain; + background: var(--bg-dark); +} + +.gallery-tab-thumb-placeholder { + display: flex; + align-items: center; + justify-content: center; + width: 100%; + aspect-ratio: 1; + background: var(--bg-dark); + color: var(--text-muted); + font-size: 0.7rem; + text-align: center; + padding: 0.5rem; + box-sizing: border-box; +} + +.gallery-tab-item-info { + padding: 0.4rem 0.5rem 0.45rem; + border-top: 1px solid var(--border); +} + +.gallery-tab-item-type { + font-size: 0.72rem; + font-weight: 600; + color: var(--accent); + text-transform: lowercase; + letter-spacing: 0.02em; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} + +.gallery-tab-item-embryo { + font-size: 0.68rem; + color: var(--text-muted); + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; + margin-top: 0.1rem; +} + +.gallery-tab-item-ts { + font-size: 0.62rem; + color: var(--text-muted); + opacity: 0.65; + margin-top: 0.1rem; + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} + +/* ========================================== + Gently Toast — volume acquired / burst acquired + ========================================== */ + +.gently-toast { + position: fixed; + bottom: 24px; + left: 50%; + transform: translateX(-50%) translateY(20px); + background: var(--bg-card); + border: 1px solid var(--accent-green); + border-radius: 10px; + padding: 0.6rem 1rem 0.6rem 1.1rem; + display: flex; + align-items: center; + gap: 0.65rem; + z-index: 10010; + box-shadow: 0 4px 20px rgba(0, 0, 0, 0.35); + opacity: 0; + transition: opacity 0.25s ease, transform 0.25s ease; + pointer-events: auto; +} + +.gently-toast.visible { + opacity: 1; + transform: translateX(-50%) translateY(0); +} + +.gently-toast-msg { + font-size: 0.85rem; + font-weight: 500; + color: var(--text); + white-space: nowrap; +} + +.gently-toast-action { + background: transparent; + border: 1px solid var(--accent); + border-radius: 5px; + color: var(--accent); + font-size: 0.78rem; + padding: 0.2rem 0.55rem; + cursor: pointer; + transition: background 0.15s, color 0.15s; + white-space: nowrap; +} + +.gently-toast-action:hover { + background: var(--accent); + color: var(--bg-dark); +} + +.gently-toast-dismiss { + background: transparent; + border: none; + color: var(--text-muted); + font-size: 1.1rem; + line-height: 1; + cursor: pointer; + padding: 0 0.1rem; + opacity: 0.7; + transition: opacity 0.15s; + margin-left: 0.2rem; +} +} diff --git a/gently/ui/web/static/css/notebook.css b/gently/ui/web/static/css/notebook.css new file mode 100644 index 00000000..fa4cebab --- /dev/null +++ b/gently/ui/web/static/css/notebook.css @@ -0,0 +1,104 @@ +/* ── Notebook tab (LIBRARY) ────────────────────────────────────────────── + The reading room for the shared lab notebook: thread rail + kind filter + + note cards. Tokens reuse the app theme (--accent, --accent-green, --border …). */ + +.nb-container { max-width: 1100px; margin: 0 auto; padding: 24px 28px; } + +.nb-header { + display: flex; align-items: center; justify-content: space-between; + gap: 16px; margin-bottom: 18px; flex-wrap: wrap; +} +.nb-title { font-size: 22px; font-weight: 600; letter-spacing: -.01em; color: var(--text, #0f172a); margin: 0; } + +.nb-kinds { display: flex; gap: 6px; } +.nb-kind { + background: none; border: 1px solid var(--border, #e4e9f0); color: var(--text-secondary, #475569); + border-radius: 999px; padding: 6px 13px; font: inherit; font-size: 13px; cursor: pointer; + transition: border-color .15s, color .15s, background .15s; +} +.nb-kind:hover { color: var(--text, #0f172a); border-color: var(--border-strong, #cbd5e1); } +.nb-kind.active { background: var(--accent, #2f6df6); border-color: var(--accent, #2f6df6); color: #fff; } + +/* Ask the notebook */ +.nb-ask { display: flex; gap: 9px; margin-bottom: 14px; } +.nb-ask-input { + flex: 1; border: 1px solid var(--border, #e4e9f0); border-radius: 11px; + padding: 11px 14px; font: inherit; font-size: 14px; color: var(--text, #0f172a); + background: var(--bg-card, #fff); +} +.nb-ask-input:focus { outline: none; border-color: var(--accent, #2f6df6); box-shadow: 0 0 0 4px rgba(96, 165, 250, .15); } +.nb-ask-go { + flex: none; border: 0; border-radius: 11px; padding: 11px 20px; font: inherit; font-size: 14px; + font-weight: 600; color: #fff; background: var(--accent, #2f6df6); cursor: pointer; transition: background .15s; +} +.nb-ask-go:hover { background: color-mix(in srgb, var(--accent) 88%, #000); } + +.nb-ask-result { + border: 1px solid var(--border, #e4e9f0); border-radius: 13px; padding: 16px 18px; margin-bottom: 18px; + background: var(--accent-soft, #f5f8ff); +} +.nb-ask-result[hidden] { display: none; } +.nb-ask-loading, .nb-ask-empty { color: var(--text-muted, #94a3b8); font-size: 14px; font-style: italic; } +.nb-ask-cov { + display: inline-block; font-size: 11px; font-weight: 700; letter-spacing: .04em; text-transform: uppercase; + padding: 2px 9px; border-radius: 999px; margin-bottom: 10px; color: #fff; background: var(--text-muted, #94a3b8); +} +.nb-cov-covered { background: var(--accent-green, #16a34a); } +.nb-cov-partial { background: #d97706; } +.nb-cov-not_in_notebook { background: var(--text-muted, #94a3b8); } +.nb-ask-answer { font-size: 14.5px; line-height: 1.55; color: var(--text, #0f172a); } +.nb-ask-h { + font-size: var(--v2-fs-eyebrow, 11px); font-weight: 700; letter-spacing: .06em; text-transform: uppercase; + color: var(--text-secondary, #475569); margin: 14px 0 6px; +} +.nb-ask-points { list-style: none; padding: 0; margin: 0; display: flex; flex-direction: column; gap: 7px; } +.nb-ask-point { font-size: 13.5px; line-height: 1.45; color: var(--text, #0f172a); display: flex; flex-wrap: wrap; align-items: baseline; gap: 6px; } +.nb-cite { + font: 600 11px/1.4 ui-monospace, SFMono-Regular, Menlo, monospace; + color: var(--accent, #2f6df6); background: var(--bg-card, #fff); + border: 1px solid var(--border, #e4e9f0); border-radius: 6px; padding: 1px 6px; +} +.nb-ask-next { margin: 0; padding-left: 18px; font-size: 13.5px; line-height: 1.5; color: var(--text-secondary, #475569); } +.nb-ask-next li { margin: 2px 0; } + +.nb-body-wrap { display: grid; grid-template-columns: 220px 1fr; gap: 22px; align-items: start; } +@media (max-width: 760px) { .nb-body-wrap { grid-template-columns: 1fr; } } + +/* thread rail (the inquiry spine) */ +.nb-threads { display: flex; flex-direction: column; gap: 4px; position: sticky; top: 12px; } +.nb-thread { + text-align: left; background: none; border: 0; cursor: pointer; font: inherit; font-size: 13px; + color: var(--text-secondary, #475569); padding: 8px 11px; border-radius: 9px; + transition: background .15s, color .15s; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; +} +.nb-thread:hover { background: var(--bg-hover, #f1f5f9); color: var(--text, #0f172a); } +.nb-thread.active { background: var(--accent-soft, #eaf1ff); color: var(--accent, #2f6df6); font-weight: 600; } + +/* notes */ +.nb-notes { display: flex; flex-direction: column; gap: 12px; } +.nb-empty { color: var(--text-muted, #94a3b8); font-size: 14px; font-style: italic; padding: 28px 4px; } + +.nb-card { + border: 1px solid var(--border, #e4e9f0); border-radius: 13px; padding: 14px 16px; + background: var(--bg-card, #fff); box-shadow: 0 1px 2px rgba(15, 23, 42, .04); +} +.nb-card-head { display: flex; align-items: center; gap: 10px; margin-bottom: 8px; } +.nb-badge { + font-size: 11px; font-weight: 700; letter-spacing: .04em; text-transform: uppercase; + padding: 2px 9px; border-radius: 999px; color: #fff; background: var(--text-muted, #94a3b8); +} +.nb-badge.nb-k-obs { background: var(--accent, #2f6df6); } +.nb-badge.nb-k-find { background: var(--accent-green, #16a34a); } +.nb-badge.nb-k-q { background: #d97706; } +.nb-author { font-size: 12px; color: var(--text-secondary, #475569); } +.nb-status { + margin-left: auto; font-size: 11px; color: var(--text-muted, #94a3b8); + text-transform: uppercase; letter-spacing: .05em; +} +.nb-body-text { font-size: 14px; line-height: 1.5; color: var(--text, #0f172a); white-space: pre-wrap; } + +.nb-chips { display: flex; flex-wrap: wrap; gap: 6px; margin-top: 10px; } +.nb-chip { + font-size: 11.5px; color: var(--text-secondary, #475569); + background: var(--bg-hover, #f1f5f9); border-radius: 7px; padding: 2px 8px; +} diff --git a/gently/ui/web/static/css/operate.css b/gently/ui/web/static/css/operate.css new file mode 100644 index 00000000..5b6c29b6 --- /dev/null +++ b/gently/ui/web/static/css/operate.css @@ -0,0 +1,662 @@ +/* ══════════════════════════════════════════════════════════════════════════ + Operate — three instrument surfaces. + + Bottom cam · SPIM head · Acquisition. Each pane is fully live whenever it is + visible; nothing here is disclosed or gated by "which step you are on", + because there are no steps. There is deliberately no rule in this file that + shows or hides a control based on anything but its own pane. + + Restyle scope: Operate only. Theme tokens are inherited from main.css so + light/dark keeps working; on top of them sits a local instrument layer with + THREE signal colours and nothing else. Measurements are monospace, spacing is + one 4px scale, surfaces are hairlines — no shadows, no gradients. + ══════════════════════════════════════════════════════════════════════════ */ + +.devices-view-operate { + container-type: inline-size; + container-name: operate; +} + +/* Every display rule below outranks the UA's `[hidden] { display: none }`, so + a class would silently defeat the attribute. Anything hidden stays hidden. */ +.operate [hidden] { display: none !important; } + +.operate { + /* instrument layer */ + --op-panel: #12161d; + --op-rule: #262c36; + --op-ink: #d7dee8; + --op-ink-dim: #7d8899; + /* the only three signal colours on this surface */ + --op-live: #4ade80; /* a stream is running */ + --op-warn: #f5a524; /* floor proximity, XY locked */ + --op-emit: #f4485d; /* LED or laser emitting — nothing else may use this */ + + --op-1: 4px; --op-2: 8px; --op-3: 12px; + --op-4: 16px; --op-5: 20px; --op-6: 24px; + + --op-mono: 'JetBrains Mono', ui-monospace, SFMono-Regular, Menlo, monospace; + --op-ui: 'Inter Tight', system-ui, sans-serif; + + display: flex; + flex-direction: column; + gap: var(--op-3); + /* Sized by the flex parent (.devices-view is flex:1; min-height:0). A hard + vh calc here fights it and overflows once the agent panel is open. */ + flex: 1 1 auto; + min-height: 0; + color: var(--op-ink); + font-family: var(--op-ui); +} + +[data-theme="light"] .operate { + --op-panel: #ffffff; + --op-rule: #dfe4ea; + --op-ink: #1c2430; + --op-ink-dim: #667081; + --op-live: #16a34a; + --op-warn: #b45309; + --op-emit: #dc2626; +} + +/* ── sub-navigation ─────────────────────────────────────────────────────── */ +.op-subnav { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--op-3); + padding-bottom: var(--op-2); + border-bottom: 1px solid var(--op-rule); + flex: 0 0 auto; +} +.op-subviews .view-btn { font-size: 0.72rem; } +.op-subnav-meta { + font-family: var(--op-mono); + font-size: 0.62rem; + color: var(--op-ink-dim); + letter-spacing: 0.04em; +} + +/* ── shared embryo rail + pane area ─────────────────────────────────────── */ +.op-body { + flex: 1 1 auto; + min-height: 0; + display: flex; + gap: var(--op-3); +} +.op-panearea { + flex: 1 1 auto; + min-width: 0; + min-height: 0; + display: flex; + flex-direction: column; +} +/* The rail is redundant on Acquisition, which carries its own richer roster. */ +.op-body[data-pane="acquire"] .op-embryos { display: none; } +.op-embryos { + flex: 0 0 208px; + min-height: 0; + display: flex; + flex-direction: column; + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--op-panel); + overflow: hidden; +} +.op-erail-head { + display: flex; + align-items: center; + justify-content: space-between; + padding: var(--op-2) var(--op-3); + border-bottom: 1px solid var(--op-rule); + font-size: 0.72rem; + letter-spacing: 0.06em; + text-transform: uppercase; + color: var(--op-ink-dim); + flex: 0 0 auto; +} +.op-erail-list { + flex: 1 1 auto; + min-height: 0; + overflow-y: auto; + padding: var(--op-1); + display: flex; + flex-direction: column; + gap: var(--op-1); +} +.op-erail-empty { + padding: var(--op-3); + font-size: 0.72rem; + line-height: 1.5; + color: var(--op-ink-dim); + text-align: center; +} +.op-erow { + display: grid; + grid-template-columns: 1fr auto; + align-items: center; + gap: var(--op-2); + padding: var(--op-2); + border: 1px solid transparent; + border-radius: 5px; + cursor: pointer; +} +.op-erow:hover { background: color-mix(in srgb, var(--op-ink) 6%, transparent); } +.op-erow.is-sel { + border-color: var(--accent); + background: color-mix(in srgb, var(--accent) 12%, transparent); +} +.op-erow:focus-visible { outline: 2px solid var(--accent); outline-offset: 1px; } +.op-erow-main { min-width: 0; display: flex; flex-direction: column; gap: 1px; } +.op-erow-label { font-size: 0.78rem; color: var(--op-ink); font-weight: 600; } +.op-erow-xy { font-family: var(--op-mono); font-size: 0.62rem; color: var(--op-ink-dim); } +.op-erow-del { + border: none; + background: none; + color: var(--op-ink-dim); + cursor: pointer; + font-size: 0.95rem; + line-height: 1; + padding: 2px 5px; + border-radius: 4px; +} +.op-erow-del:hover { color: var(--op-emit); background: color-mix(in srgb, var(--op-emit) 14%, transparent); } + +/* ── pane shell ─────────────────────────────────────────────────────────── */ +.op-pane { + flex: 1 1 auto; + min-height: 0; + display: grid; + grid-template-columns: minmax(0, 1fr) 260px; + grid-template-rows: auto minmax(0, 1fr); + gap: var(--op-3); + align-content: start; + /* Never triggers in the two-column layout (both children carry min-height:0); + it is what makes the stacked layout scroll instead of spilling. */ + overflow-y: auto; +} +.op-pane[hidden] { display: none; } + +.op-main { + grid-column: 1; + min-width: 0; + min-height: 0; + display: flex; + flex-direction: column; + gap: var(--op-2); +} +.op-inst { + grid-column: 2; + min-height: 0; + display: flex; + flex-direction: column; + gap: var(--op-2); + overflow-y: auto; + padding-right: var(--op-1); +} + +/* ── the XY interlock banner ──────────────────────────────────────────────── + The one piece of chrome that appears in more than one pane. It is rendered + from a single function into a per-pane host — deliberately NOT a shared + header abstraction. */ +.op-lock { + grid-column: 1 / -1; + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--op-3); + padding: var(--op-2) var(--op-3); + border: 1px solid var(--op-warn); + border-radius: 6px; + background: color-mix(in srgb, var(--op-warn) 12%, transparent); +} +.op-lock[hidden] { display: none; } +.op-lock-txt { + font-size: 0.68rem; + letter-spacing: 0.06em; + text-transform: uppercase; + font-weight: 600; + color: var(--op-warn); +} +.op-lock-txt i { font-style: normal; color: var(--op-ink-dim); } + +/* ── viewport ───────────────────────────────────────────────────────────── */ +/* The viewport is a centering box; the frame chrome (border, background, + placeholder) lives on .op-cam-fit, which is sized to the frame's own aspect + ratio (--cam-ar, set from the image once a frame arrives). That way the + border hugs the image instead of letterboxing it inside an oversized box. */ +.op-cam { + flex: 1 1 auto; + min-height: 0; + display: flex; + align-items: center; + justify-content: center; + overflow: hidden; +} +.op-cam-fit { + position: relative; + aspect-ratio: var(--cam-ar, 1 / 1); + /* Height drives (the main column is landscape, the frame square/portrait), + so the box fills the available height and its width follows the aspect — + capped by the column width. The narrow layout flips this below. */ + height: 100%; + width: auto; + max-width: 100%; + max-height: 100%; + min-height: 0; + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--img-bg); + overflow: hidden; +} +.op-cam-img { + width: 100%; + height: 100%; + object-fit: contain; + display: none; +} +.op-cam-img.has-frame { display: block; } +.op-mark-canvas { + position: absolute; + inset: 0; + width: 100%; + height: 100%; +} +/* Marking is always available on the bottom pane — it is not a mode you enter. */ +#op-cam-bottom .op-mark-canvas { cursor: crosshair; } +/* While the sample is at the objective the affordance withdraws itself, so the + rule is learned from the surface rather than from a toast after the click. */ +#op-cam-bottom.is-locked .op-mark-canvas { cursor: not-allowed; } +.op-cam-ph { + position: absolute; + inset: 0; + display: flex; + align-items: center; + justify-content: center; + font-size: 0.72rem; + color: var(--op-ink-dim); + letter-spacing: 0.06em; + text-transform: uppercase; +} + +/* Detection-in-progress overlay: sits over the frame so the wait is legible + on the image itself, not only in the button label. */ +.op-cam-busy { + position: absolute; + inset: 0; + display: flex; + align-items: center; + justify-content: center; + gap: var(--op-2); + background: color-mix(in srgb, var(--img-bg) 55%, transparent); + backdrop-filter: blur(1px); + z-index: 3; + pointer-events: none; +} +.op-cam-busy[hidden] { display: none; } +.op-cam-busy-txt { + font-size: 0.72rem; + color: var(--op-ink); + letter-spacing: 0.06em; + text-transform: uppercase; +} +.op-cam-spin { + width: 18px; + height: 18px; + border-radius: 50%; + border: 2px solid color-mix(in srgb, var(--op-ink) 30%, transparent); + border-top-color: var(--accent, currentColor); + animation: op-cam-spin 0.8s linear infinite; +} +@keyframes op-cam-spin { to { transform: rotate(360deg); } } +@media (prefers-reduced-motion: reduce) { + .op-cam-spin { animation-duration: 2s; } +} + +/* Action bar beneath a viewport: what you do TO the image lives next to it. */ +.op-bar { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--op-3); + flex-wrap: wrap; + flex: 0 0 auto; +} +.op-bar-r { display: flex; align-items: center; gap: var(--op-2); } +.op-bar-r .op-num { min-width: 22px; } + +/* ── instrument blocks ──────────────────────────────────────────────────── */ +.op-block, .op-gauge { + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--op-panel); + padding: var(--op-2) var(--op-3); + display: flex; + flex-direction: column; + gap: var(--op-2); +} +.op-block-head, .op-gauge-name, .op-label { + font-size: 0.62rem; + font-weight: 600; + letter-spacing: 0.10em; + text-transform: uppercase; + color: var(--op-ink-dim); +} +.op-block-head { color: var(--op-ink); } + +.op-row { + display: flex; + align-items: center; + justify-content: space-between; + gap: var(--op-2); +} +.op-actions { display: flex; gap: var(--op-2); } +.op-actions > .op-btn { flex: 1; } + +/* Every measurement is monospace and right-aligned; a bare number never + appears without its unit. */ +.op-num { + font-family: var(--op-mono); + font-size: 0.74rem; + font-weight: 500; + color: var(--op-ink); + text-align: right; + font-variant-numeric: tabular-nums; +} +.op-cap { + font-size: 0.68rem; + line-height: 1.45; + color: var(--op-ink-dim); + margin: 0; +} + +/* ── the Z instrument ─────────────────────────────────────────────────────── + The nudge ladder sits beside the track rather than under it, so the control + is spatially next to the travel it commands — ups above, downs below. */ +.op-gauge { + display: grid; + grid-template-columns: auto minmax(0, 1fr); + grid-template-areas: "head head" "body nudge" "foot foot"; + align-items: center; + column-gap: var(--op-3); +} +.op-gauge-head { grid-area: head; } +.op-gauge-body { grid-area: body; } +.op-gauge-nudge { grid-area: nudge; } +.op-gauge-foot { grid-area: foot; } + +.op-gauge-head { display: flex; align-items: baseline; justify-content: space-between; gap: var(--op-2); } +.op-gauge-read b { font-size: 0.92rem; } +.op-gauge-read i { + font-style: normal; + font-size: 0.6rem; + color: var(--op-ink-dim); + margin-left: 3px; +} +.op-gauge-body { display: flex; gap: var(--op-2); height: 84px; } +.op-gauge-track { + position: relative; + flex: 0 0 10px; + border: 1px solid var(--op-rule); + border-radius: 3px; + background: linear-gradient(to top, + color-mix(in srgb, var(--op-warn) 22%, transparent) 0%, + transparent 34%); +} +.op-gauge-mark { + position: absolute; + left: -4px; + right: -4px; + height: 2px; + background: var(--op-ink); + border-radius: 1px; + transition: bottom 0.18s ease; +} +.op-gauge-ticks { position: absolute; inset: 0; } +.op-gauge-ticks span { + position: absolute; + left: 100%; + transform: translateY(50%); + margin-left: 3px; + font-family: var(--op-mono); + font-size: 0.54rem; + color: var(--op-ink-dim); + white-space: nowrap; +} +.op-gauge-ticks span::before { + content: ''; + position: absolute; + right: 100%; + top: 50%; + width: 3px; + height: 1px; + background: var(--op-rule); +} +.op-gauge-scale { + display: flex; + flex-direction: column; + justify-content: space-between; + align-items: flex-start; + font-size: 0.58rem; + color: var(--op-ink-dim); + padding-left: 20px; /* clears the log-scale tick labels */ +} +.op-gauge-nudge { + display: grid; + grid-template-columns: 1fr 1fr; + gap: var(--op-1); +} +.op-gauge-foot { + display: flex; + justify-content: space-between; + gap: var(--op-2); + font-size: 0.6rem; + color: var(--op-ink-dim); +} +.op-gauge-foot-r { white-space: nowrap; } +.op-gauge.is-near-floor { border-color: var(--op-warn); } +.op-gauge.is-near-floor .op-gauge-mark { background: var(--op-warn); } + +/* An axis the rig does not have is a fact, not an error: dimmed, no marker, + nudges removed from the DOM entirely (there is nothing to press). */ +.op-gauge[data-status="absent"] { opacity: 0.55; } +.op-gauge[data-status="absent"] .op-gauge-track, +.op-gauge[data-status="error"] .op-gauge-track { background: none; } + +/* ── controls ───────────────────────────────────────────────────────────── */ +.op-btn, .op-nbtn, .op-segbtn { + font-family: var(--op-ui); + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--bg-hover); + color: var(--op-ink); + cursor: pointer; + transition: background-color 0.12s ease; +} +.op-btn { + padding: var(--op-2) var(--op-3); + font-size: 0.72rem; + font-weight: 500; +} +.op-btn-wide { width: 100%; } +.op-btn:hover:not(:disabled), .op-nbtn:hover:not(:disabled), .op-segbtn:hover { background: var(--border); } +.op-btn:disabled, .op-nbtn:disabled { opacity: 0.4; cursor: not-allowed; } +.op-btn-primary { background: var(--accent); border-color: var(--accent); color: #071018; font-weight: 600; } +.op-btn-primary:hover:not(:disabled) { background: var(--accent-hover); } +.op-btn-ghost { background: transparent; } +.op-btn-warn { background: var(--op-warn); border-color: var(--op-warn); color: #1a1204; font-weight: 600; } +.op-btn-toggle[aria-pressed="true"], .op-btn.is-on { + border-color: var(--op-live); + color: var(--op-live); +} +.op-btn.is-emitting { border-color: var(--op-emit); color: var(--op-emit); } + +.op-nbtn { + padding: var(--op-1) var(--op-2); + font-family: var(--op-mono); + font-size: 0.66rem; +} +.op-micro { display: inline-flex; align-items: center; gap: var(--op-1); } +.op-micro .op-num { min-width: 34px; } + +.op-seg { display: flex; flex-wrap: wrap; gap: var(--op-1); } +.op-segbtn { padding: var(--op-2) var(--op-3); font-size: 0.68rem; } +.op-segbtn.is-on { border-color: var(--accent); color: var(--accent); } + +.op-field { display: flex; align-items: center; justify-content: space-between; gap: var(--op-2); } +.op-num-in, .op-sel { + font-family: var(--op-mono); + font-size: 0.72rem; + padding: var(--op-1) var(--op-2); + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--bg-dark); + color: var(--op-ink); + width: 110px; +} +.op-modepanel { display: flex; flex-direction: column; gap: var(--op-2); } +.op-modepanel[hidden] { display: none; } + +/* Focus is visible on EVERY interactive element, including the canvas and the + gauge tracks. */ +.op-btn:focus-visible, .op-nbtn:focus-visible, .op-segbtn:focus-visible, +.op-num-in:focus-visible, .op-sel:focus-visible, .op-gauge-track:focus-visible, +.op-mark-canvas:focus-visible, .op-rrow:focus-visible, .op-libitem:focus-visible, +.op-rrole:focus-visible { + outline: 2px solid var(--accent); + outline-offset: 2px; +} + +/* ── acquisition pane ───────────────────────────────────────────────────── */ +.op-pane-acquire { grid-template-columns: minmax(0, 1fr); } +.op-main-cols { + display: grid; + grid-template-columns: minmax(0, 1fr) minmax(0, 1.1fr) minmax(0, 1fr); + gap: var(--op-4); + min-height: 0; +} +.op-col { + display: flex; + flex-direction: column; + gap: var(--op-2); + min-height: 0; + overflow-y: auto; +} + +.op-roster { display: flex; flex-direction: column; gap: var(--op-1); } +.op-rrow { + display: grid; + grid-template-columns: 1fr auto auto auto; + align-items: center; + gap: var(--op-2); + padding: var(--op-2); + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--op-panel); + cursor: pointer; + text-align: left; + color: var(--op-ink); + transition: background-color 0.12s ease; +} +.op-rrow:hover { background: var(--bg-hover); } +.op-rrow.is-sel { border-color: var(--accent); } +.op-rlabel { font-size: 0.72rem; font-weight: 500; } +.op-rxy { font-family: var(--op-mono); font-size: 0.64rem; color: var(--op-ink-dim); } +.op-rrole { + font-family: var(--op-ui); + font-size: 0.56rem; + font-weight: 600; + letter-spacing: 0.08em; + text-transform: uppercase; + padding: 2px var(--op-2); + border: 1px solid var(--op-rule); + border-radius: 999px; + background: transparent; + color: var(--op-ink-dim); + cursor: pointer; +} +.op-rrole.is-reference { border-color: var(--accent-purple); color: var(--accent-purple); } +.op-rcenter { + font-family: var(--op-ui); + font-size: 0.62rem; + padding: 3px var(--op-2); + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--bg-hover); + color: var(--op-ink); + cursor: pointer; + transition: background-color 0.12s ease; +} +.op-rcenter:hover { background: var(--border); } +.op-rcenter:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; } + +.op-lib-list { display: flex; flex-direction: column; gap: var(--op-1); } +.op-libitem { + display: flex; + flex-direction: column; + gap: 2px; + padding: var(--op-2); + border: 1px solid var(--op-rule); + border-radius: 6px; + background: var(--op-panel); + cursor: pointer; + text-align: left; + color: var(--op-ink); + font-size: 0.7rem; +} +.op-libitem small { font-family: var(--op-mono); font-size: 0.58rem; color: var(--op-ink-dim); } +.op-libitem.is-sel { border-color: var(--accent); } + +.op-runspine { display: flex; flex-direction: column; gap: var(--op-2); } +.op-tcard { + border: 1px solid var(--op-rule); + border-left-width: 2px; + border-radius: 6px; + background: var(--op-panel); + padding: var(--op-2) var(--op-3); + display: flex; + flex-direction: column; + gap: 2px; +} +.op-tcard.st-active { border-left-color: var(--op-live); } +.op-tcard.st-paused { border-left-color: var(--op-warn); } +.op-tcard.st-done { opacity: 0.6; } +.op-tcard-head { display: flex; justify-content: space-between; gap: var(--op-2); align-items: baseline; } +.op-tcard-name { font-size: 0.74rem; font-weight: 600; } +.op-tcard-state, .op-tcard-kind, .op-tcard-meta { + font-family: var(--op-mono); + font-size: 0.58rem; + color: var(--op-ink-dim); +} +.op-empty { + font-size: 0.68rem; + color: var(--op-ink-dim); + padding: var(--op-3); + border: 1px dashed var(--op-rule); + border-radius: 6px; + text-align: center; +} +.op-empty .op-btn { margin-top: var(--op-2); } + +/* ── narrow ─────────────────────────────────────────────────────────────── */ +@container operate (max-width: 900px) { + .op-pane { grid-template-columns: minmax(0, 1fr); grid-template-rows: auto auto auto; } + .op-main, .op-inst { grid-column: 1; } + .op-inst { overflow-y: visible; } + .op-cam { flex: 0 0 auto; min-height: 260px; } + /* Stacked/narrow: the box is width-limited, so width drives and height + follows the aspect (the parent's height is no longer definite here). */ + .op-cam-fit { width: 100%; height: auto; } + /* Keep the embryo rail beside the camera even here — just make it narrower. + It only stacks on top at the much smaller breakpoint below. */ + .op-embryos { flex: 0 0 168px; } + .op-col { overflow-y: visible; } + .op-main-cols { grid-template-columns: minmax(0, 1fr); } +} + +/* Only when there is genuinely no room for two columns does the rail stack on + top of the pane, capped so the camera keeps most of the height. */ +@container operate (max-width: 560px) { + .op-body { flex-direction: column; } + .op-embryos { flex: 0 0 auto; max-height: 148px; } +} diff --git a/gently/ui/web/static/css/review.css b/gently/ui/web/static/css/review.css index de2bd66d..735d42a5 100644 --- a/gently/ui/web/static/css/review.css +++ b/gently/ui/web/static/css/review.css @@ -443,3 +443,30 @@ color: var(--text-muted); } + +/* Resume-in-agent action on session list items */ +.session-resume-btn { + margin-top: 8px; + padding: 5px 10px; + border-radius: 7px; + border: 1px solid var(--accent, #60a5fa); + background: transparent; + color: var(--accent, #60a5fa); + font-size: 12px; + font-weight: 600; + cursor: pointer; +} +.session-resume-btn:hover { background: var(--accent, #60a5fa); color: #fff; } +.session-active-badge { + font-size: 10px; + font-weight: 600; + text-transform: uppercase; + letter-spacing: 0.05em; + color: var(--accent-green, #4ade80); + border: 1px solid var(--accent-green, #4ade80); + border-radius: 999px; + padding: 1px 7px; + margin-left: 6px; + vertical-align: middle; +} +.session-item.active-session { border-left: 2px solid var(--accent-green, #4ade80); } diff --git a/gently/ui/web/static/css/shell.css b/gently/ui/web/static/css/shell.css new file mode 100644 index 00000000..935b7006 --- /dev/null +++ b/gently/ui/web/static/css/shell.css @@ -0,0 +1,197 @@ +/* ux_v2 shell chrome: grouped left-rail nav + session-context strip. + Everything is scoped under body.ux-v2 so the v1 dashboard is byte-for-byte + untouched — no consolidation of the existing duplicate .tab rulesets here + (that cleanup is deferred to the final phase). */ + +/* Replace the flat 8-tab bar with the rail. */ +body.ux-v2 .tabs { display: none; } + +/* ── Left rail ─────────────────────────────────────────────── */ +body.ux-v2 .v2-rail { + flex: 0 0 212px; + display: flex; + flex-direction: column; + gap: 2px; + padding: 14px 10px; + border-right: 1px solid var(--border, #e4e9f0); + background: var(--bg-card, #fff); + overflow-y: auto; + animation: v2-rise .45s ease backwards; +} +body.ux-v2 .v2-nav-group { margin-bottom: 6px; } +body.ux-v2 .v2-nav-label { + font-size: 10px; letter-spacing: .1em; text-transform: uppercase; + color: var(--text-muted, #94a3b8); padding: 10px 10px 4px; +} +body.ux-v2 .v2-nav-item { + display: flex; align-items: center; gap: 8px; width: 100%; + background: none; border: 0; cursor: pointer; text-align: left; + padding: 8px 10px; border-radius: 8px; + font: inherit; font-size: 13.5px; + color: var(--text-secondary, #475569); + transition: background .15s, color .15s; +} +body.ux-v2 .v2-nav-item:hover { background: var(--bg-hover, #f1f5f9); color: var(--text, #0f172a); } +body.ux-v2 .v2-nav-item.active { + background: var(--accent-soft, #eaf1ff); + color: var(--accent, #2f6df6); + font-weight: 600; +} +body.ux-v2 .v2-rail-chat { + margin-top: auto; + display: flex; align-items: center; gap: 9px; + background: none; border: 1px solid var(--border, #e4e9f0); border-radius: 10px; + padding: 9px 12px; cursor: pointer; + font: inherit; font-size: 13px; + color: var(--text-secondary, #475569); + transition: border-color .15s, color .15s; +} +body.ux-v2 .v2-rail-chat:hover { border-color: var(--accent, #2f6df6); color: var(--accent, #2f6df6); } +body.ux-v2 .v2-rail-orb { + width: 18px; height: 18px; border-radius: 50%; flex: none; + background: radial-gradient(closest-side at 38% 34%, #fff, #bcd3ff 42%, var(--accent, #2f6df6) 100%); +} + +/* ── Session-context strip (top of main) ───────────────────── */ +body.ux-v2 .v2-strip { + flex: none; + display: flex; align-items: center; gap: 12px; + padding: 9px 16px; + border-bottom: 1px solid var(--border, #e4e9f0); + background: var(--bg-card, #fff); + font-size: 12.5px; color: var(--text-muted, #94a3b8); + animation: v2-rise .45s ease backwards .05s; +} +body.ux-v2 .v2-strip-live { + display: inline-flex; align-items: center; gap: 6px; + font-size: 10.5px; font-weight: 700; letter-spacing: .08em; color: #ef4444; +} +body.ux-v2 .v2-strip-dot { + width: 8px; height: 8px; border-radius: 50%; background: #ef4444; +} +body.ux-v2 .v2-strip-status { margin-left: auto; font-variant-numeric: tabular-nums; } + +body.ux-v2 .app-main { animation: v2-rise .5s ease backwards .1s; } + +@keyframes v2-rise { from { opacity: 0; transform: translateY(8px); } to { opacity: 1; transform: none; } } +@media (prefers-reduced-motion: reduce) { + body.ux-v2 .v2-rail, body.ux-v2 .v2-strip, body.ux-v2 .app-main { animation: none; } +} + +/* ── Shared-visibility surface (the agent's view) ──────────── */ +body.ux-v2 .cx-surface { + margin: 0 0 16px; + border: 1px solid var(--border, #e4e9f0); + border-radius: 14px; + background: var(--bg-card, #fff); + padding: 14px 16px; +} +body.ux-v2 .cx-surface.hidden { display: none; } +body.ux-v2 .cx-title { font-size: 11px; letter-spacing: .1em; text-transform: uppercase; color: var(--text-muted, #94a3b8); margin-bottom: 8px; } +body.ux-v2 .cx-lens { margin-bottom: 10px; } +body.ux-v2 .cx-lens-h { font-size: 11px; font-weight: 600; color: var(--text-secondary, #475569); margin: 6px 0 4px; } +body.ux-v2 .cx-item { display: flex; align-items: center; gap: 9px; padding: 5px 0; flex-wrap: wrap; } +body.ux-v2 .cx-text { flex: 1; min-width: 0; font-size: 13px; color: var(--text, #0f172a); } +body.ux-v2 .cx-dot { width: 7px; height: 7px; border-radius: 50%; flex: none; } +body.ux-v2 .cx-dot.cx-q { background: #d97706; } +body.ux-v2 .cx-dot.cx-w { background: var(--accent, #2f6df6); } +body.ux-v2 .cx-dot.cx-e { background: var(--accent-green, #16a34a); } +body.ux-v2 .cx-act { flex: none; border: 1px solid var(--border, #e4e9f0); background: none; color: var(--text-secondary, #475569); border-radius: 8px; padding: 3px 10px; font: inherit; font-size: 12px; cursor: pointer; } +body.ux-v2 .cx-act:hover { border-color: var(--accent, #2f6df6); color: var(--accent, #2f6df6); } +body.ux-v2 .cx-answer { display: flex; gap: 6px; align-items: center; flex: 1 0 100%; margin-top: 4px; } +body.ux-v2 .cx-answer.hidden { display: none; } +body.ux-v2 .cx-answer-input { flex: 1; min-width: 0; border: 1px solid var(--border, #cbd5e1); border-radius: 8px; padding: 6px 9px; font: inherit; font-size: 12px; } +body.ux-v2 .cx-answer-go { border: 0; background: var(--accent, #2f6df6); color: #fff; border-radius: 8px; padding: 6px 10px; cursor: pointer; } +body.ux-v2 .cx-empty { font-size: 12.5px; color: var(--text-muted, #94a3b8); font-style: italic; padding: 2px 0 4px; } + +/* ── PROCESS CONSOLE ────────────────────────────────────────────────────── + Header drawer over the agent + device-layer output. Monospace and dense on + purpose: this is a terminal the packaged app otherwise denies you, not a + styled log viewer. */ +.logc-open { position: relative; } +.logc-dot { + position: absolute; top: 3px; right: 3px; + width: 7px; height: 7px; border-radius: 50%; + background: var(--accent-orange); + box-shadow: 0 0 0 2px var(--bg-dark); +} + +.logc { position: fixed; inset: 0; z-index: 1200; } +.logc[hidden] { display: none; } +.logc-scrim { position: absolute; inset: 0; background: rgba(0, 0, 0, 0.35); } +.logc-panel { + position: absolute; left: 0; right: 0; bottom: 0; + height: min(62vh, 640px); + display: flex; flex-direction: column; + background: var(--bg-card); + border-top: 1px solid var(--border-strong); + box-shadow: 0 -12px 32px var(--panel-edge-shadow); +} +.logc-head { + display: flex; align-items: center; justify-content: space-between; gap: 1rem; + padding: 0.5rem 0.8rem; + border-bottom: 1px solid var(--border); +} +.logc-tabs { display: flex; gap: 0.3rem; } +.logc-tab { + font: 600 0.76rem/1 'Inter Tight', system-ui, sans-serif; + color: var(--text-muted); + background: transparent; border: 1px solid transparent; border-radius: 6px; + padding: 0.34rem 0.6rem; cursor: pointer; + transition: color 0.12s, background 0.12s, border-color 0.12s; +} +.logc-tab:hover { color: var(--text); } +.logc-tab.is-active { + color: var(--text); background: var(--bg-hover); border-color: var(--border); +} +.logc-tab:focus-visible, .logc-btn:focus-visible, .logc-level:focus-visible { + outline: 2px solid var(--accent); outline-offset: 2px; +} +.logc-tools { display: flex; align-items: center; gap: 0.35rem; } +.logc-level { + font: 500 0.72rem/1 'Inter Tight', system-ui, sans-serif; + color: var(--text); background: var(--bg-hover); + border: 1px solid var(--border); border-radius: 6px; padding: 0.3rem 0.4rem; +} +.logc-btn { + font: 600 0.72rem/1 'Inter Tight', system-ui, sans-serif; + color: var(--text-muted); background: transparent; + border: 1px solid var(--border); border-radius: 6px; + padding: 0.32rem 0.55rem; cursor: pointer; + transition: color 0.12s, border-color 0.12s; +} +.logc-btn:hover { color: var(--text); border-color: var(--border-strong); } +.logc-btn.is-on { color: var(--accent); border-color: var(--accent); } +.logc-close { font-size: 0.8rem; line-height: 1; } + +.logc-meta { + display: flex; align-items: baseline; justify-content: space-between; gap: 1rem; + padding: 0.3rem 0.85rem; + font-size: 0.68rem; color: var(--text-muted); + font-variant-numeric: tabular-nums; + border-bottom: 1px solid var(--border); +} + +.logc-body { + flex: 1 1 auto; min-height: 0; + margin: 0; padding: 0.6rem 0.85rem; + overflow: auto; + /* --bg-dark/--text are a matched pair in BOTH themes. --img-bg is not: it + stays dark under the light theme, which put dark text on a dark panel. */ + background: var(--bg-dark); + font: 12px/1.5 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; + color: var(--text); + white-space: pre; +} +/* Focused on open so keyboard scrolling works; the ring would otherwise frame + the whole panel on every open. */ +.logc-body:focus { outline: none; } +.logc-body:focus-visible { outline: 2px solid var(--accent); outline-offset: -2px; } +.logc-line { display: block; } +.logc-line.is-warn { color: var(--accent-orange); } +.logc-line.is-error { color: var(--accent-red, #ef4444); font-weight: 600; } + +@media (max-width: 720px) { + .logc-panel { height: 78vh; } + .logc-head { flex-wrap: wrap; gap: 0.5rem; } +} diff --git a/gently/ui/web/static/js/agent-chat.js b/gently/ui/web/static/js/agent-chat.js new file mode 100644 index 00000000..bfa50a8e --- /dev/null +++ b/gently/ui/web/static/js/agent-chat.js @@ -0,0 +1,1410 @@ +/** + * Floating agent-chat window — the web-side control surface. + * + * Connects to the same /ws/agent bridge the TUI uses, streams the agent's + * responses, and renders interactive choice pickers. A single-driver control + * lock on the server arbitrates who may drive the microscope; this client + * shows a banner and offers "Take control" when another client holds it. + * + * Self-contained IIFE (no build step). All untrusted text is escaped before + * insertion — never assign agent/user/tool strings to innerHTML directly. + */ +const AgentChat = (() => { + let ws = null; + let reconnectDelay = 1000; + // Commands/messages requested before the socket is open (e.g. from the + // landing, with the chat panel closed) — flushed in order on ws.onopen. + let pendingProgrammatic = []; + const MAX_DELAY = 30000; + + let panelOpen = false; + let hasControl = true; // optimistic until the server says otherwise + const answeredAsks = new Set(); // request_ids answered from EITHER surface (transcript / main stage) + let holderLabel = null; + let streaming = false; + let currentAgentEl = null; // the agent content element being streamed into + let activityEl = null; // the persistent "working…" indicator (reused) + let me = null; // { authenticated, username, role, can_control } + let myConnId = null; // this connection's id, for labelling own msgs "You" + + // Autocomplete: slash-command + @tool registries (pushed by the server on + // connect) and the live dropdown state. + let commands = []; // [{name, description, aliases, ...}] + let tools = []; // [{name, description, params, ...}] + let acItems = []; // current completion items shown in the dropdown + let acIdx = -1; // highlighted item index + let autonomousTurn = false; // true while rendering an autonomous (wake) turn + let agentBusy = false; // a turn (user or autonomous) is currently running + let busySource = null; // 'user' | 'wake' while busy + let askPending = false; // agent is paused waiting for user's ask answer + let msgQueue = []; // messages typed while busy, sent on idle + let queuePanel = null; // the "⏳ Queued (N)" panel element + + // DOM refs (resolved in init) + let panel, log, input, sendBtn, conn, banner, closeBtn, userEl, signoutBtn; + let railBtn, resizeEl, toggleDot, toggleBadge; // docked-panel chrome + let pendingSlot = null; // sticky slot for ASK approval proposals + let acComplete = null; // the autocomplete dropdown element + + // ── Safe rendering ──────────────────────────────────────── + function escapeHtml(s) { + const d = document.createElement('div'); + d.textContent = String(s == null ? '' : s); + return d.innerHTML; + } + + // ── Markdown → safe HTML ────────────────────────────────── + // Block-aware GFM subset: headings, pipe tables, fenced/indented code, + // ordered/unordered lists, blockquotes, hr, paragraphs; inline bold/italic/ + // strike/code/links. Designed for STREAMING: called repeatedly on a growing + // prefix, so it must never throw or hang on partial input (half a table, an + // unclosed fence, a dangling ** or `). All text is escaped before any markup + // is added — no raw HTML passthrough — and link hrefs are scheme-checked. + // + // All regexes are linear (no nested/adjacent unbounded quantifiers over the + // same class) to avoid catastrophic backtracking on adversarial tool output. + // + // Block class names (for styling): ac-md (wrapper), ac-md-h1..ac-md-h6, + // ac-md-p, ac-md-ul, ac-md-ol, ac-md-li, ac-md-quote, ac-md-hr, + // ac-md-pre, ac-md-code-block, ac-md-table-wrap, ac-md-table, ac-md-link. + + // escapeHtml() escapes & < > but not quotes; quote-escape for attribute use. + function escAttr(s) { + return escapeHtml(s).replace(/"/g, '"').replace(/'/g, '''); + } + + // Allow only safe link schemes. Reject javascript:/data:/vbscript: and any + // control chars an attacker might use to smuggle a scheme past the check. + // Relative URLs (no scheme) and #anchors are allowed. + function safeHref(raw) { + // Strip ASCII control chars + whitespace (incl. tab/newline) anywhere in + // the URL so they can't be used to break the scheme check below. + const url = String(raw || '').replace(/[\x00-\x20\x7f]/g, ''); + // A scheme is letters/digits/+/-/. followed by ':' before any / ? #. + const m = url.match(/^([a-zA-Z][a-zA-Z0-9+.\-]*):/); + if (m) { + const scheme = m[1].toLowerCase(); + if (scheme !== 'http' && scheme !== 'https' && scheme !== 'mailto') return null; + } + return url; + } + + // Inline spans, applied to ALREADY-ESCAPED text. Order matters: code spans + // are pulled out first (placeholdered) so their contents aren't re-processed, + // then links, then emphasis. Bounded quantifiers keep this linear-time. + function mdInline(escaped) { + const codes = []; + // `code` and ``code`` — non-greedy, capped run length, no newlines. + let s = escaped.replace(/(`{1,2})([^`\n]{0,500}?)\1/g, (_, _t, code) => { + codes.push(code); + return 'CODE' + (codes.length - 1) + ''; + }); + // [label](href) — label has no brackets, href no spaces/parens; bounded. + s = s.replace(/\[([^\]\n]{0,200})\]\(([^()\s]{0,500})\)/g, (m, label, href) => { + const safe = safeHref(href); + if (!safe) return label; // drop a rejected link, keep its text + return '' + label + ''; + }); + // Bold, italic, strikethrough. Each pattern is a single bounded run. + s = s.replace(/\*\*([^*\n]{1,500}?)\*\*/g, '$1'); + s = s.replace(/__([^_\n]{1,500}?)__/g, '$1'); + s = s.replace(/(^|[^*])\*([^*\n]{1,500}?)\*/g, '$1$2'); + s = s.replace(/(^|[^_\w])_([^_\n]{1,500}?)_(?![\w])/g, '$1$2'); + s = s.replace(/~~([^~\n]{1,500}?)~~/g, '$1'); + // Restore code spans as real elements. + s = s.replace(/CODE(\d+)/g, (_, i) => '' + codes[+i] + ''); + return s; + } + + // Split a GFM table row into escaped, inline-rendered cells. Handles escaped + // pipes (\|) and ignores the leading/trailing border pipes. + function tableCells(line) { + const cells = []; + let buf = ''; + for (let i = 0; i < line.length; i++) { + const ch = line[i]; + if (ch === '\\' && line[i + 1] === '|') { buf += '|'; i++; continue; } + if (ch === '|') { cells.push(buf); buf = ''; continue; } + buf += ch; + } + cells.push(buf); + // Drop empty edge cells produced by leading/trailing border pipes. + if (cells.length && cells[0].trim() === '') cells.shift(); + if (cells.length && cells[cells.length - 1].trim() === '') cells.pop(); + return cells.map(c => mdInline(escapeHtml(c.trim()))); + } + + // A table delimiter row: |---|:--:|--:| (each cell only - : and spaces, ≥1 -). + function isTableDivider(line) { + const t = line.trim().replace(/^\||\|$/g, ''); + if (!t) return false; + return t.split('|').every(c => /^\s*:?-+:?\s*$/.test(c)); + } + + function mdToHtml(text) { + const src = String(text == null ? '' : text); + const lines = src.split('\n'); + const out = []; + let i = 0; + + // Paragraph buffer: collect consecutive prose lines, flush on a block + // boundary. Soft line-breaks inside a paragraph become
. + let para = []; + const flushPara = () => { + if (!para.length) return; + const body = para.map(l => mdInline(escapeHtml(l))).join('
'); + out.push('

' + body + '

'); + para = []; + }; + + while (i < lines.length) { + const line = lines[i]; + const trimmed = line.trim(); + + // Fenced code block: ``` or ~~~ (optional language). Unterminated + // fences (mid-stream) consume to end-of-input — never hang. + const fence = trimmed.match(/^(`{3,}|~{3,})(.*)$/); + if (fence) { + flushPara(); + const marker = fence[1][0]; + const minLen = fence[1].length; + const langRaw = fence[2].trim().split(/\s+/)[0] || ''; + const lang = langRaw.replace(/[^a-zA-Z0-9_+\-.]/g, '').slice(0, 32); + const code = []; + i++; + while (i < lines.length) { + const cl = lines[i]; + const cm = cl.trim(); + // A closing fence: same marker char, length ≥ opening, nothing else. + if (cm[0] === marker && /^(`{3,}|~{3,})\s*$/.test(cm) && cm.replace(/\s+$/, '').length >= minLen) { + i++; + break; + } + code.push(cl); + i++; + } + const langClass = lang ? ' language-' + lang : ''; + out.push('
' +
+                    escapeHtml(code.join('\n')) + '
'); + continue; + } + + // Blank line: paragraph boundary. + if (trimmed === '') { flushPara(); i++; continue; } + + // ATX heading: #..###### text. + const h = trimmed.match(/^(#{1,6})\s+(.*?)\s*#*$/); + if (h) { + flushPara(); + const level = h[1].length; + out.push('' + + mdInline(escapeHtml(h[2])) + ''); + i++; + continue; + } + + // Horizontal rule: ---, ***, ___ (3+). + if (/^(-{3,}|\*{3,}|_{3,})$/.test(trimmed)) { + flushPara(); + out.push('
'); + i++; + continue; + } + + // GFM table: a header row followed by a delimiter row. Require the + // delimiter to confirm it's a table (so a lone pipe line stays prose), + // which also keeps a half-streamed table as plain text until ready. + if (trimmed.indexOf('|') !== -1 && i + 1 < lines.length && isTableDivider(lines[i + 1])) { + flushPara(); + const header = tableCells(line); + const colCount = header.length; + i += 2; // consume header + delimiter + const bodyRows = []; + while (i < lines.length) { + const rl = lines[i]; + if (rl.trim() === '' || rl.indexOf('|') === -1) break; + bodyRows.push(tableCells(rl)); + i++; + } + let tbl = '
'; + for (let c = 0; c < colCount; c++) tbl += ''; + tbl += ''; + bodyRows.forEach(row => { + tbl += ''; + for (let c = 0; c < colCount; c++) tbl += ''; + tbl += ''; + }); + tbl += '
' + (header[c] || '') + '
' + (row[c] || '') + '
'; + out.push(tbl); + continue; + } + + // Blockquote: one or more leading '>' lines (collected together). + if (/^>\s?/.test(line)) { + flushPara(); + const quote = []; + while (i < lines.length && /^>\s?/.test(lines[i])) { + quote.push(lines[i].replace(/^>\s?/, '')); + i++; + } + const body = quote.map(l => mdInline(escapeHtml(l))).join('
'); + out.push('
' + body + '
'); + continue; + } + + // Lists: a run of consecutive item lines. Ordered if the first item + // is "N." / "N)", else unordered (-, *, +). Nesting is rendered flat + // (depth via a leading-indent class isn't needed for the agent's output). + const ulMatch = line.match(/^(\s*)[-*+]\s+(.*)$/); + const olMatch = line.match(/^(\s*)\d{1,9}[.)]\s+(.*)$/); + if (ulMatch || olMatch) { + flushPara(); + const ordered = !!olMatch; + const tag = ordered ? 'ol' : 'ul'; + const cls = ordered ? 'ac-md-ol' : 'ac-md-ul'; + const items = []; + while (i < lines.length) { + const ul = lines[i].match(/^(\s*)[-*+]\s+(.*)$/); + const ol = lines[i].match(/^(\s*)\d{1,9}[.)]\s+(.*)$/); + const m = ordered ? ol : ul; + if (!m) { + // A blank line or non-item ends the list. + if (lines[i].trim() === '' || (!ul && !ol)) break; + break; + } + items.push(mdInline(escapeHtml(m[2]))); + i++; + } + out.push('<' + tag + ' class="' + cls + '">' + + items.map(it => '
  • ' + it + '
  • ').join('') + + ''); + continue; + } + + // Default: prose line — accumulate into the paragraph buffer. + para.push(line); + i++; + } + flushPara(); + return '
    ' + out.join('') + '
    '; + } + + // Pin-to-bottom autoscroll: only follow new content if the user is already + // near the bottom; otherwise count unseen items and show a "↓ N new" pill so + // a streaming agent never yanks the operator away from something they're reading. + let stickBottom = true; + let newCount = 0; + let jumpPill = null; + function nearBottom() { return (log.scrollHeight - log.scrollTop - log.clientHeight) < 60; } + function renderJumpPill() { + if (!jumpPill) return; + if (!stickBottom && newCount > 0) { + jumpPill.textContent = `↓ ${newCount} new`; + jumpPill.classList.remove('hidden'); + } else { + jumpPill.classList.add('hidden'); + } + } + function scrollToBottom(isNewItem = true) { + if (stickBottom) { log.scrollTop = log.scrollHeight; } + // Only count genuinely new items (bubbles/rows), not in-place streaming + // text edits — otherwise the "N new" pill inflates per chunk. + else { if (isNewItem) newCount += 1; renderJumpPill(); } + } + function jumpToBottom() { + stickBottom = true; newCount = 0; + log.scrollTop = log.scrollHeight; + renderJumpPill(); + } + + // ── Activity indicator ──────────────────────────────────── + // A single reusable "the agent is working" row, always pinned to the + // bottom of the log. This is the trust signal — something is happening. + function setActivity(label) { + if (!activityEl) { + activityEl = document.createElement('div'); + activityEl.className = 'ac-activity'; + activityEl.innerHTML = + '' + + ''; + } + activityEl.querySelector('.ac-activity-label').textContent = label; + log.appendChild(activityEl); // (re)pin to bottom + scrollToBottom(); + } + function hideActivity() { + if (activityEl && activityEl.parentNode) activityEl.parentNode.removeChild(activityEl); + } + + // ── Message elements ────────────────────────────────────── + function addTurn(role) { + const wrap = document.createElement('div'); + wrap.className = `ac-turn ac-turn-${role}`; + // No per-turn role label: the agent's replies are plain text and the + // user's sit in a bubble (modern chat convention). Autonomous (wake) + // turns are still marked by the banner + accent rail, not a label. + if (role === 'agent' && autonomousTurn) wrap.classList.add('ac-turn-autonomous'); + const content = document.createElement('div'); + content.className = 'ac-content'; + wrap.appendChild(content); + log.appendChild(wrap); + scrollToBottom(); + return content; + } + + /** Normalize an author for display: clean up legacy/anonymous labels. */ + function displayAuthor(author) { + if (!author) return 'Anonymous'; + // Legacy/per-connection labels ("window 3", "User 5") read as anonymous. + if (/^(window|user)\s+\d+$/i.test(author)) return 'Anonymous'; + return author; + } + + function addUserMessage(text, author, authorId) { + const wrap = document.createElement('div'); + wrap.className = 'ac-turn ac-turn-user'; + // Single shared chat, so every user message is labelled. It's "You" when + // it's from this connection (authorId match) or — once logged in — from + // your username (stable across reloads); otherwise the sender's name, or + // "Anonymous" for an unsigned-in participant. A local echo (no author + // info) is always you. + const mine = (!author && !authorId) + || (authorId && authorId === myConnId) + || (author && me && me.username && author === me.username); + const label = document.createElement('div'); + label.className = 'ac-role ac-role-user'; + label.textContent = mine ? 'You' : displayAuthor(author); + wrap.appendChild(label); + if (!mine) wrap.classList.add('ac-from-other'); + const content = document.createElement('div'); + content.className = 'ac-content'; + content.textContent = text; + wrap.appendChild(content); + log.appendChild(wrap); + scrollToBottom(); + } + + /** Rebuild the transcript from a persisted/replayed history list. */ + function renderHistory(items) { + log.innerHTML = ''; + currentAgentEl = null; + activityEl = null; + stickBottom = true; newCount = 0; // a full rebuild jumps to latest + (items || []).forEach(it => { + if (it.role === 'user') { + addUserMessage(it.text, it.author, it.author_id); + } else if (it.role === 'agent') { + const c = addTurn('agent'); + c._raw = it.text || ''; + c.innerHTML = mdToHtml(c._raw); + } else if (it.role === 'autonomous_start') { + addAutonomousBanner(it.trigger || ''); + } else if (it.role === 'autonomous') { + autonomousTurn = true; + const c = addTurn('agent'); + c._raw = it.text || ''; + c.innerHTML = mdToHtml(c._raw); + autonomousTurn = false; + } else if (it.role === 'tool') { + const el = document.createElement('div'); + el.className = 'ac-tool ac-tool-done'; + const dur = it.duration ? ` · ${(it.duration.toFixed ? it.duration.toFixed(1) : it.duration)}s` : ''; + const summary = it.summary ? ` — ${escapeHtml(it.summary)}` : ''; + el.innerHTML = `${escapeHtml(it.name || 'tool')}${dur}${summary}`; + log.appendChild(el); + } else if (it.role === 'system') { + addSystemLine(it.text, it.level || 'info'); + } + }); + scrollToBottom(); + } + + /** A divider announcing the agent woke itself, with the trigger reason. */ + function addAutonomousBanner(trigger) { + const el = document.createElement('div'); + el.className = 'ac-autonomous-banner'; + const t = trigger ? `Gently woke up — ${trigger}` : 'Gently woke up'; + el.innerHTML = `${escapeHtml(t)}`; + log.appendChild(el); + scrollToBottom(); + } + + function addSystemLine(text, level = 'info') { + const el = document.createElement('div'); + el.className = `ac-system ac-level-${level}`; + el.textContent = text; + log.appendChild(el); + scrollToBottom(); + } + + // ── Protocol handlers ───────────────────────────────────── + function handle(msg) { + switch (msg.type) { + case 'connected': + reconnectDelay = 1000; + myConnId = msg.you_id || myConnId; // for labelling own messages "You" + setConn(true, msg.version ? `Connected · v${msg.version}` : 'Connected'); + // The bridge ships the command + tool registries on connect. + // Capture them so the composer can offer autocomplete — the + // data was always on the wire; we just never used it. + commands = Array.isArray(msg.commands) ? msg.commands : []; + tools = Array.isArray(msg.tools) ? msg.tools : []; + break; + + case 'control_status': + hasControl = !!msg.you_have_control; + holderLabel = msg.holder_label || null; + renderControl(); + // The main-stage ask renderer re-renders read-only on control loss. + if (typeof ClientEventBus !== 'undefined') ClientEventBus.emit('AGENT_CONTROL', { hasControl }); + break; + + case 'history': + renderHistory(msg.items || []); + break; + + case 'user_message': + hideActivity(); + addUserMessage(msg.text, msg.author, msg.author_id); + break; + + case 'stream_start': + streaming = true; + currentAgentEl = null; // created lazily on first text + setBusy(true, 'user'); + setActivity('Working…'); + emitActivity('turn_start', { autonomous: false }); + break; + + case 'autonomous_start': + // The agent woke itself — render a distinct banner + label the + // following text as autonomous (no stream_start precedes this). + hideActivity(); + autonomousTurn = true; + currentAgentEl = null; + setBusy(true, 'wake'); + addAutonomousBanner(msg.trigger || ''); + bumpBadge(); + emitActivity('turn_start', { autonomous: true }); + break; + + case 'thinking': + if (streaming) setActivity('Thinking…'); + emitActivity('thinking', { text: msg.text || '' }); + break; + + case 'text': { + if (!currentAgentEl) { + hideActivity(); + currentAgentEl = addTurn('agent'); + currentAgentEl._raw = ''; + } + currentAgentEl._raw += (msg.text || ''); + currentAgentEl.innerHTML = mdToHtml(currentAgentEl._raw); + scrollToBottom(false); // in-place edit, not a new item + emitActivity('text', { text: msg.text || '' }); + break; + } + + case 'tool_start': { + hideActivity(); // the running tool row is the signal now + currentAgentEl = null; // text after a tool starts a fresh bubble + // ask_user_choice renders as a choice card via choice_request — + // skip the noisy spinning tool row for it. + if (msg.tool_name === 'ask_user_choice') break; + const label = msg.tool_label || msg.tool_name || 'tool'; + const args = fmtArgs(msg.tool_input); + const el = document.createElement('div'); + el.className = 'ac-tool ac-tool-running'; + el.dataset.tool = msg.tool_name || ''; + el.innerHTML = + `
    ` + + `${escapeHtml(label)}
    ` + + (args ? `
    ${escapeHtml(args)}
    ` : ''); + log.appendChild(el); + scrollToBottom(); + emitActivity('tool_start', { name: msg.tool_name || '', label: label, input: msg.tool_input }); + break; + } + + case 'tool_call': { + const running = [...log.querySelectorAll('.ac-tool-running')] + .filter(e => e.dataset.tool === (msg.tool_name || '')); + const el = running[running.length - 1]; + const label = msg.tool_name || 'tool'; + const dur = msg.duration + ? ` · ${(msg.duration.toFixed ? msg.duration.toFixed(1) : msg.duration)}s` : ''; + const args = fmtArgs(msg.tool_input); + const summary = msg.result_summary || ''; + // Show ⚠ instead of ✓ when the tool errored or its result reads + // like a failure — so the operator can tell when a tool did nothing. + const isErr = !!msg.is_error || looksLikeError(summary); + const icon = isErr + ? `` + : ``; + const html = + `
    ${icon}` + + `${escapeHtml(label)}` + + `${dur}
    ` + + (args ? `
    ${escapeHtml(args)}
    ` : '') + + (summary ? `
    ${escapeHtml(summary)}
    ` : ''); + if (el) { + el.className = 'ac-tool ac-tool-done' + (isErr ? ' ac-tool-err' : ''); + el.innerHTML = html; + } else { + // No matching running row (e.g. after a reconnect) — append fresh. + const fresh = document.createElement('div'); + fresh.className = 'ac-tool ac-tool-done' + (isErr ? ' ac-tool-err' : ''); + fresh.innerHTML = html; + log.appendChild(fresh); + } + if (streaming) setActivity('Working…'); // agent continues after the tool + scrollToBottom(); + emitActivity('tool_result', { + name: msg.tool_name || '', label: label, + input: msg.tool_input, duration: msg.duration, + summary: summary, full: msg.result_full, is_error: isErr, + }); + break; + } + + case 'choice_request': + hideActivity(); + setAskState(true); // waiting on user, not working + renderChoice(msg); + bumpBadge(); + break; + + case 'applied_spec': + renderSpec(msg.spec || {}); + break; + + case 'stream_end': + streaming = false; + currentAgentEl = null; + autonomousTurn = false; + hideActivity(); + setBusy(false); + emitActivity('turn_end'); + break; + + case 'command_result': + if (msg.error) addSystemLine(`${msg.command}: ${msg.error}`, 'error'); + else if (msg.content) addSystemLine(`${msg.command} ✓`, 'info'); + break; + + case 'notification': + addSystemLine(msg.body ? `${msg.title} — ${msg.body}` : msg.title, msg.level || 'info'); + bumpBadge(); + break; + + case 'error': + streaming = false; + hideActivity(); + setBusy(false); + addSystemLine(msg.error || 'Unknown error', 'error'); + emitActivity('turn_error', { error: msg.error || 'Unknown error' }); + clearPendingAsks(); // a cancelled/errored turn sends no choice_response + break; + + case 'ping': + send({ type: 'pong' }); + break; + + default: + break; // pong / state_update / browse_result / unknown — ignored + } + } + + // Build an ask card from a choice_data payload. Pure: the caller supplies + // hasControl + onPick, so the SAME builder renders in the chat transcript + // and on the main stage (#ask-stage) — one payload, two renderers. + function buildAskCard(data, opts) { + opts = opts || {}; + const reqId = opts.reqId || ''; + const isWake = !!opts.isWake; + const canAct = !!opts.hasControl && !answeredAsks.has(reqId); + const onPick = opts.onPick || function () {}; + + const wrap = document.createElement('div'); + wrap.className = 'ac-choice' + (isWake ? ' ac-choice-wake' : ''); + wrap.dataset.reqId = reqId; + if (isWake) { + const tag = document.createElement('div'); + tag.className = 'ac-choice-origin'; + tag.textContent = 'Autonomy proposal — your approval needed'; + wrap.appendChild(tag); + } + const q = document.createElement('div'); + q.className = 'ac-choice-q'; + q.innerHTML = mdToHtml(data.question || 'Choose:'); + wrap.appendChild(q); + + (data.options || []).forEach(opt => { + const btn = document.createElement('button'); + btn.className = 'ac-choice-opt'; + btn.disabled = !!opt.disabled || !canAct; // observers / already-answered → read-only + const desc = opt.description ? `${escapeHtml(opt.description)}` : ''; + btn.innerHTML = `${escapeHtml(opt.label)}${desc}`; + btn.addEventListener('click', () => onPick(opt.id)); + wrap.appendChild(btn); + }); + + // Free-text escape — the bridge routes an unknown selection to LLM + // resolution, so the agent's asend always unblocks. (The TUI had this; + // the web ask cards previously did not.) + if (canAct) { + const ow = document.createElement('div'); + ow.className = 'ac-choice-otherwrap'; + const otherBtn = document.createElement('button'); + otherBtn.className = 'ac-choice-opt ac-choice-other'; + otherBtn.innerHTML = 'Something else…'; + const form = document.createElement('div'); + form.className = 'ac-choice-otherform hidden'; + const ti = document.createElement('input'); + ti.type = 'text'; + ti.className = 'ac-choice-otherinput'; + ti.placeholder = 'Type your own answer…'; + const go = document.createElement('button'); + go.className = 'ac-choice-othergo'; + go.textContent = '→'; + const submitOther = () => { const v = ti.value.trim(); if (v) onPick(v); }; + otherBtn.addEventListener('click', () => { otherBtn.classList.add('hidden'); form.classList.remove('hidden'); ti.focus(); }); + go.addEventListener('click', submitOther); + ti.addEventListener('keydown', e => { if (e.key === 'Enter') { e.preventDefault(); submitOther(); } }); + form.appendChild(ti); form.appendChild(go); + ow.appendChild(otherBtn); ow.appendChild(form); + wrap.appendChild(ow); + } + + if (answeredAsks.has(reqId)) wrap.classList.add('ac-choice-answered'); + return wrap; + } + + // Send the answer ONCE (idempotent across both surfaces), then fire the + // clear off the CHOICE lifecycle — NOT stream_end (which lands after the + // answer for in-turn asks, and never for a cancelled turn). + function answerChoice(reqId, selected) { + if (!reqId || answeredAsks.has(reqId)) return; + answeredAsks.add(reqId); + send({ type: 'choice_response', request_id: reqId, selected }); + setAskState(false); // resume working-state visuals + if (streaming) setActivity('Working…'); + if (typeof ClientEventBus !== 'undefined') ClientEventBus.emit('ASK_CLEARED', { request_id: reqId }); + } + + // Disable + mark-answered any transcript / sticky-slot ask card for this + // request_id ('*' = all). The main stage clears itself via its own handler. + function markAnswered(reqId) { + [log, pendingSlot].forEach(scope => { + if (!scope) return; + scope.querySelectorAll('.ac-choice').forEach(card => { + if (reqId !== '*' && card.dataset.reqId !== reqId) return; + card.querySelectorAll('button').forEach(b => b.disabled = true); + card.classList.add('ac-choice-answered'); + }); + // Also fade compact pointers (ux_v2 mode — no buttons to disable). + scope.querySelectorAll('.ac-ask-pointer').forEach(ptr => { + if (reqId !== '*' && ptr.dataset.reqId !== reqId) return; + ptr.classList.add('ac-ask-pointer-answered'); + }); + }); + if (pendingSlot) { + const slotCard = pendingSlot.querySelector('.ac-choice'); + if (reqId === '*' || (slotCard && slotCard.dataset.reqId === reqId)) { + setTimeout(() => { pendingSlot.classList.add('hidden'); pendingSlot.innerHTML = ''; }, 700); + } + } + } + + // Retire all pending asks (turn cancelled/errored, or socket dropped). + function clearPendingAsks() { + if (typeof ClientEventBus !== 'undefined') ClientEventBus.emit('ASK_CLEARED', { request_id: '*' }); + else markAnswered('*'); + } + + function renderChoice(msg) { + const data = msg.choice_data || {}; + const reqId = msg.request_id || data.request_id || ''; + const isWake = msg.origin === 'wake'; + // Always mirror onto the main stage (AskStage / landing wizard). + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.emit('AGENT_ASK', { request_id: reqId, choice_data: data, origin: msg.origin }); + } + // Under ux_v2 (#ask-stage present), the main stage owns the full ask UI. + // Replace the duplicate card in the chat transcript with a compact pointer + // so the surrounding context (agent reasoning, tool calls) stays readable + // but the choice buttons aren't shown twice. + if (document.getElementById('ask-stage')) { + const ptr = document.createElement('div'); + ptr.className = 'ac-ask-pointer'; + ptr.dataset.reqId = reqId; + ptr.textContent = '↑ Gently is asking — answer above'; + log.appendChild(ptr); + scrollToBottom(); + return; + } + // v1 / non-ux_v2: render full card as before. + const card = buildAskCard(data, { + reqId, isWake, hasControl, + onPick: (sel) => answerChoice(reqId, sel), + }); + // ASK approvals pin to the sticky slot above the composer; ordinary + // choices stay inline in the transcript. + if (isWake && pendingSlot) { + pendingSlot.innerHTML = ''; + pendingSlot.appendChild(card); + pendingSlot.classList.remove('hidden'); + return; + } + log.appendChild(card); + scrollToBottom(); + } + + function renderSpec(spec) { + const prov = spec.provenance || {}; + const rows = []; + // (label, value, fieldKey) — fieldKey ties a row to its provenance entry. + const add = (label, value, key) => { + if (value === undefined || value === null || value === '') return; + rows.push({ label, value, src: key ? prov[key] : null }); + }; + add('Strain', spec.strain, 'strain'); + add('Genotype', spec.genotype, 'genotype'); + add('Reporter', spec.reporter, 'reporter'); + add('Channel', spec.laser_wavelength_nm != null ? `${spec.laser_wavelength_nm} nm` : null, 'laser_wavelength_nm'); + add('Temperature', spec.temperature_c != null ? `${spec.temperature_c} °C` : null, 'temperature_c'); + add('Slices', spec.num_slices, 'num_slices'); + add('Exposure', spec.exposure_ms != null ? `${spec.exposure_ms} ms` : null, 'exposure_ms'); + add('Interval', spec.interval_s != null ? `${spec.interval_s} s` : null, 'interval_s'); + add('Stop at', spec.stop_condition, 'stop_condition'); + if (!rows.length) return; + // A small "where did this come from" tag for inferred values. + const srcTag = (src) => { + if (!src || !src.source) return ''; + const where = String(src.source).split(':')[0]; + const conf = src.confidence ? ` · ${src.confidence}` : ''; + const title = escapeHtml(String(src.source) + (src.confidence ? ` (confidence: ${src.confidence})` : '')); + return ` ${escapeHtml(where + conf)}`; + }; + const el = document.createElement('div'); + el.className = 'ac-spec'; + el.innerHTML = '
    Imaging spec
    ' + + rows.map(r => `
    ${escapeHtml(r.label)}${escapeHtml(r.value)}${srcTag(r.src)}
    `).join(''); + log.appendChild(el); + scrollToBottom(); + } + + // ── Tool argument formatting ────────────────────────────── + /** Compact, escaped "key=value" rendering of a tool's input for the chat. */ + function fmtArgs(input) { + if (!input || typeof input !== 'object') return ''; + const parts = []; + for (const [k, v] of Object.entries(input)) { + if (k === 'context' || v === null || v === undefined || v === '') continue; + let val = (typeof v === 'object') ? JSON.stringify(v) : String(v); + if (val.length > 48) val = val.slice(0, 47) + '…'; + parts.push(`${k}=${val}`); + } + return parts.join(' '); + } + + /** Heuristic: does a tool's result summary read like a failure? + * Used to show ⚠ for tools that return an error STRING (the agent only + * flags raised exceptions). Avoids false alarms like "No errors found". */ + function looksLikeError(s) { + if (!s) return false; + const t = s.trim(); + if (/^no\s+(errors?|issues?|problems?|anomal|changes?|warnings?)\b/i.test(t)) return false; + if (/^(error|failed|failure|unable|cannot|can'?t|could\s?n'?t|could not|denied|invalid|no |not )/i.test(t)) return true; + // mid-string failure markers, e.g. "Timepoint 7 not found for embryo_2". + return /\bnot (found|available|connected|recognized|valid|supported)\b/i.test(t); + } + + // ── Autocomplete ────────────────────────────────────────── + /** The whitespace-delimited token immediately left of the caret. */ + function currentToken() { + const v = input.value; + const pos = (input.selectionStart != null) ? input.selectionStart : v.length; + const before = v.slice(0, pos); + const m = before.match(/(\S+)$/); + return { token: m ? m[1] : '', start: m ? pos - m[1].length : pos, pos }; + } + + /** Compute completion items for the current input/caret, or []. */ + function computeCompletions() { + const trimmed = input.value.trimStart().toLowerCase(); + // Slash commands: whole-input prefix (mirrors the TUI). A trailing space + // (i.e. typing args) naturally yields no matches and hides the menu. + if (trimmed.startsWith('/')) { + return commands.filter(c => + (c.name && c.name.toLowerCase().startsWith(trimmed)) || + (c.aliases || []).some(a => String(a).toLowerCase().startsWith(trimmed)) + ).slice(0, 8).map(c => ({ kind: 'command', name: c.name, desc: c.description || '' })); + } + // @tool mention: complete the token under the caret against tool names. + const tok = currentToken(); + if (tok.token.startsWith('@') && tools.length) { + const q = tok.token.slice(1).toLowerCase(); + return tools.filter(t => t.name.toLowerCase().includes(q)) + .slice(0, 8) + .map(t => ({ kind: 'tool', name: t.name, desc: t.description || '', token: tok })); + } + return []; + } + + function renderCompletions(items) { + acItems = items || []; + acIdx = acItems.length ? 0 : -1; + if (!acComplete) return; + if (!acItems.length) { hideCompletions(); return; } + acComplete.innerHTML = ''; + acItems.forEach((it, i) => { + const row = document.createElement('div'); + row.className = 'ac-complete-item' + (i === acIdx ? ' active' : ''); + row.innerHTML = + `${escapeHtml(it.name)}` + + (it.desc ? `${escapeHtml(it.desc)}` : ''); + // mousedown (not click) so it fires before the textarea blurs. + row.addEventListener('mousedown', (e) => { e.preventDefault(); acceptCompletion(it); }); + acComplete.appendChild(row); + }); + acComplete.classList.remove('hidden'); + } + + function hideCompletions() { + acItems = []; + acIdx = -1; + if (acComplete) { acComplete.classList.add('hidden'); acComplete.innerHTML = ''; } + } + + function updateCompletions() { + renderCompletions(computeCompletions()); + } + + function moveCompletion(delta) { + if (!acItems.length || !acComplete) return; + acIdx = (acIdx + delta + acItems.length) % acItems.length; + [...acComplete.children].forEach((c, i) => c.classList.toggle('active', i === acIdx)); + } + + function acceptCompletion(item) { + if (!item) return; + if (item.kind === 'command') { + input.value = item.name + ' '; + const p = input.value.length; + try { input.setSelectionRange(p, p); } catch (_) {} + } else if (item.kind === 'tool') { + const tok = item.token || currentToken(); + const v = input.value; + const insert = '@' + item.name + ' '; + input.value = v.slice(0, tok.start) + insert + v.slice(tok.pos); + const p = tok.start + insert.length; + try { input.setSelectionRange(p, p); } catch (_) {} + } + hideCompletions(); + input.focus(); + autosize(); + } + + // ── Control / UI state ──────────────────────────────────── + function renderControl() { + if (hasControl) { + banner.classList.add('hidden'); + banner.innerHTML = ''; + input.disabled = false; + input.placeholder = 'Message Gently… ( / commands · @ tools )'; + } else { + banner.classList.remove('hidden'); + const who = holderLabel || 'another session'; + input.disabled = true; + if (me && me.accounts && !me.authenticated) { + // Anonymous — viewing is open; sign in to control. + banner.innerHTML = `Viewing — sign in to control.`; + const btn = document.createElement('button'); + btn.className = 'ac-take-control'; + btn.textContent = 'Sign in'; + btn.addEventListener('click', () => { window.location.href = '/login'; }); + banner.appendChild(btn); + input.placeholder = 'Viewing — sign in to control…'; + } else if (me && me.authenticated && me.can_control === false) { + // Viewer-role account — watching is all this account can do. + banner.innerHTML = `View-only access — you can watch but not control.`; + input.placeholder = 'View-only access'; + } else { + banner.innerHTML = `Control held by ${escapeHtml(who)}`; + const btn = document.createElement('button'); + btn.className = 'ac-take-control'; + btn.textContent = 'Take control'; + btn.addEventListener('click', () => send({ type: 'take_control' })); + banner.appendChild(btn); + input.placeholder = 'Viewing only — take control to drive…'; + } + } + renderComposerButton(); // enable/disable + send/stop mode follow control + } + + function setBusy(busy, source) { + agentBusy = !!busy; + busySource = agentBusy ? (source || 'user') : null; + if (!agentBusy) askPending = false; // turn ended — reset ask state too + renderComposerButton(); // morph send <-> stop + if (agentBusy) { + input.placeholder = (busySource === 'wake') + ? 'Gently is acting autonomously — your message will queue' + : 'Gently is working — your message will queue'; + } else { + // Turn ended (completed, errored, or cancelled) — clear the working + // indicator. Cancel emits no stream_end of its own, so without this + // the "Working…" dots would spin forever after Stop. + hideActivity(); + if (hasControl) input.placeholder = 'Message Gently… ( / commands · @ tools )'; + drainQueue(); // a turn just ended — send the next queued message + } + } + + // While an ask is pending the agent is NOT working — it's blocked on user + // input. Override the "working" UI markers with a calm waiting hint WITHOUT + // changing agentBusy (queuing semantics stay intact; only visuals change). + function setAskState(waiting) { + askPending = waiting; + if (waiting) { + if (hasControl) input.placeholder = '↑ Type an answer or pick an option above…'; + } else if (agentBusy) { + // Restore working-state visuals now the ask has been answered. + if (hasControl) { + input.placeholder = (busySource === 'wake') + ? 'Gently is acting autonomously — your message will queue' + : 'Gently is working — your message will queue'; + } + } + renderComposerButton(); // reflect the send/stop mode for the new ask state + } + + /** Set the composer button to send (up-arrow) or stop (square) mode. */ + function renderComposerButton() { + if (!sendBtn) return; + // During a pending ask the agent is blocked on the user — offer Send (to + // answer the ask), not Stop, even though agentBusy is still set. + const stopMode = agentBusy && busySource === 'user' && !askPending; + sendBtn.classList.toggle('is-stop', stopMode); + if (stopMode) { + sendBtn.disabled = false; + sendBtn.setAttribute('aria-label', 'Stop'); + sendBtn.title = 'Stop the current turn'; + } else { + // Send is enabled only with control and some text to send. + sendBtn.disabled = !hasControl || input.value.trim() === ''; + sendBtn.setAttribute('aria-label', 'Send message'); + sendBtn.title = 'Send'; + } + } + + /** Abort the current cancellable turn and clear local busy/indicator state. */ + function cancelTurn() { + send({ type: 'cancel' }); + setBusy(false); + clearPendingAsks(); + } + + + // ── Message queue (type-while-busy) ─────────────────────── + function enqueue(text) { msgQueue.push(text); renderQueue(); } + function removeQueued(i) { + if (i >= 0 && i < msgQueue.length) { msgQueue.splice(i, 1); renderQueue(); } + } + function clearQueue() { msgQueue = []; renderQueue(); } + function drainQueue() { + if (agentBusy || !msgQueue.length) return; + if (!ws || ws.readyState !== WebSocket.OPEN) return; // keep queued until reconnect + const next = msgQueue.shift(); + renderQueue(); + actuallySend(next); + } + function renderQueue() { + if (!queuePanel) return; + if (!msgQueue.length) { queuePanel.classList.add('hidden'); queuePanel.innerHTML = ''; return; } + queuePanel.classList.remove('hidden'); + queuePanel.innerHTML = ''; + const head = document.createElement('div'); + head.className = 'ac-queue-head'; + const lbl = document.createElement('span'); + lbl.textContent = `⏳ Queued (${msgQueue.length})`; + const clear = document.createElement('button'); + clear.className = 'ac-queue-clear'; + clear.textContent = 'Clear all'; + clear.addEventListener('click', clearQueue); + head.appendChild(lbl); + head.appendChild(clear); + queuePanel.appendChild(head); + msgQueue.forEach((m, i) => { + const row = document.createElement('div'); + row.className = 'ac-queue-item'; + const span = document.createElement('span'); + span.className = 'ac-queue-text'; + span.textContent = m; + const x = document.createElement('button'); + x.className = 'ac-queue-remove'; + x.textContent = '✕'; + x.title = 'Remove from queue'; + x.addEventListener('click', () => removeQueued(i)); + row.appendChild(span); + row.appendChild(x); + queuePanel.appendChild(row); + }); + } + + function setConn(ok, label) { + conn.classList.toggle('ac-conn-ok', ok); + conn.classList.toggle('ac-conn-bad', !ok); + conn.textContent = label || (ok ? 'Connected' : 'Reconnecting…'); + if (toggleDot) toggleDot.classList.toggle('ok', ok); + // Feed the shared connection store (agent /ws/agent liveness). + if (typeof ConnectionStatus !== 'undefined') ConnectionStatus.setAgent(ok); + } + + // ── Transport ───────────────────────────────────────────── + function send(obj) { + if (ws && ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify(obj)); + } + + // Flush programmatic sends queued before the socket opened (see runCommand). + function flushProgrammatic() { + if (!pendingProgrammatic.length) return; + pendingProgrammatic.splice(0).forEach(t => actuallySend(t)); + } + + // Mirror the agent stream onto ClientEventBus so the ux_v2 plan wizard can + // render a tidy activity feed (collapsible tool cards) without a second + // socket. Additive — the chat-log rendering in handle() is unchanged. + function emitActivity(kind, extra) { + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.emit('AGENT_ACTIVITY', Object.assign({ kind }, extra || {})); + } + } + + function connect() { + const proto = location.protocol === 'https:' ? 'wss:' : 'ws:'; + setConn(false, 'Connecting…'); + ws = new WebSocket(`${proto}//${location.host}/ws/agent`); + ws.onopen = () => { reconnectDelay = 1000; setConn(true); flushProgrammatic(); }; + ws.onclose = () => { + setConn(false, 'Reconnecting…'); + setBusy(false); + streaming = false; + hideActivity(); + clearPendingAsks(); // stale asks: clear the stage on socket drop + setTimeout(connect, reconnectDelay); + reconnectDelay = Math.min(reconnectDelay * 2, MAX_DELAY); + }; + ws.onerror = () => {}; + ws.onmessage = (e) => { + let msg; + try { msg = JSON.parse(e.data); } catch { return; } + handle(msg); + }; + } + + // ── Input handling ──────────────────────────────────────── + function actuallySend(text) { + if (text.startsWith('/')) { + addUserMessage(text); // commands aren't broadcast; echo locally + send({ type: 'command', command: text }); // slash commands (e.g. /status) + // Most commands reply with a single 'command_result' and no stream — + // do NOT mark the composer busy, or the queue would stick forever. + // Commands that DO stream (e.g. /wizard) set busy via stream_start. + return; + } + send({ type: 'chat', text }); // echoed to all via 'user_message' + // Instant feedback before the first chunk arrives. + setBusy(true, 'user'); + setActivity('Working…'); + } + + function submit() { + hideCompletions(); + const text = input.value.trim(); + if (!text) return; + if (!hasControl) { renderControl(); return; } + input.value = ''; + autosize(); + // While the agent is busy (a user OR autonomous turn), queue instead of + // cancelling — Send no longer doubles as Stop. + if (agentBusy) { enqueue(text); return; } + actuallySend(text); + } + + function autosize() { + input.style.height = 'auto'; + input.style.height = Math.min(input.scrollHeight, 140) + 'px'; + } + + function togglePanel(open) { + panelOpen = (open === undefined) ? !panelOpen : open; + panel.classList.toggle('open', panelOpen); + // Remember collapse state so a reload restores it (defaults to open). + try { localStorage.setItem('gently-chat-open', panelOpen ? '1' : '0'); } catch (_) {} + if (railBtn) railBtn.setAttribute('aria-expanded', panelOpen ? 'true' : 'false'); + if (panelOpen) { + clearBadge(); + if (!ws) connect(); + // Re-pin to the latest content (it may have streamed while closed, + // where scroll events don't fire to keep stickBottom current). + setTimeout(() => { input.focus(); jumpToBottom(); }, 50); + } + // Opening/closing while docked reflows .app-main — tell viewers to resize. + if (document.body.classList.contains('chat-docked')) emitLayoutChanged(); + } + + // ── Layout: dock, resize, persistence ───────────────────── + const CHAT_MIN_W = 320; + const CHAT_DEFAULT_W = 460; + // Roomy ceiling: the panel shows agent reasoning, tool calls, approvals and + // pickers — content that wraps badly in a narrow column — so allow up to + // ~half the viewport (was min(560, 45vw), which capped power users too low). + function chatMaxW() { return Math.min(760, Math.round(window.innerWidth * 0.60)); } + + function emitLayoutChanged() { + // Let the CSS settle, then notify viewers (e.g. the 3D canvas) to resize. + requestAnimationFrame(() => window.dispatchEvent(new CustomEvent('gently:layout-changed'))); + } + + function curChatWidth() { + return parseInt(getComputedStyle(document.documentElement).getPropertyValue('--chat-w')) || CHAT_DEFAULT_W; + } + + function setChatWidth(px, persist) { + const w = Math.max(CHAT_MIN_W, Math.min(chatMaxW(), Math.round(px))); + document.documentElement.style.setProperty('--chat-w', w + 'px'); + if (persist) { try { localStorage.setItem('gently-chat-w', String(w)); } catch (_) {} } + return w; + } + + function setupResize() { + if (!resizeEl) return; + let startX = 0, startW = 0, dragging = false, rafId = 0, pid = null; + const onMove = (e) => { + if (!dragging) return; + setChatWidth(startW + (startX - e.clientX), false); // right panel: drag left = wider + if (document.body.classList.contains('chat-docked')) { + if (rafId) cancelAnimationFrame(rafId); + rafId = requestAnimationFrame(emitLayoutChanged); // coalesce dock reflow + } + }; + const onUp = () => { + if (!dragging) return; + dragging = false; + resizeEl.classList.remove('dragging'); + resizeEl.removeEventListener('pointermove', onMove); + resizeEl.removeEventListener('pointerup', onUp); + resizeEl.removeEventListener('pointercancel', onUp); + if (pid !== null && resizeEl.hasPointerCapture && resizeEl.hasPointerCapture(pid)) { + try { resizeEl.releasePointerCapture(pid); } catch (_) {} + } + pid = null; + document.body.style.userSelect = ''; + setChatWidth(curChatWidth(), true); + emitLayoutChanged(); + }; + resizeEl.addEventListener('pointerdown', (e) => { + if (e.button !== 0) return; // primary button only + e.preventDefault(); + dragging = true; + startX = e.clientX; + startW = curChatWidth(); + pid = e.pointerId; + // Capture so move/up/cancel always reach the handle (touch/pen-safe). + try { resizeEl.setPointerCapture(pid); } catch (_) {} + resizeEl.classList.add('dragging'); + document.body.style.userSelect = 'none'; + resizeEl.addEventListener('pointermove', onMove); + resizeEl.addEventListener('pointerup', onUp); + resizeEl.addEventListener('pointercancel', onUp); + }); + resizeEl.addEventListener('dblclick', () => { setChatWidth(CHAT_DEFAULT_W, true); emitLayoutChanged(); }); + } + + function restorePrefs() { + try { + const w = parseInt(localStorage.getItem('gently-chat-w')); + if (w) setChatWidth(w, false); + } catch (_) {} + // The agent panel is always docked — a real column that pushes content, + // not a float over it. It's open by default; the header Agent toggle / + // Ctrl+J / × collapse it to width 0 to reclaim space for the viewer. + document.body.classList.add('chat-docked'); + let open = true; + try { open = localStorage.getItem('gently-chat-open') !== '0'; } catch (_) {} + togglePanel(open); + } + + // Unseen-activity badge on the header toggle — so a closed panel still tells + // the operator the agent did something (woke, proposed an approval, notified). + let badgeCount = 0; + function bumpBadge() { + if (panelOpen) return; // they're watching; no badge needed + badgeCount += 1; + if (toggleBadge) { + toggleBadge.textContent = badgeCount > 9 ? '9+' : String(badgeCount); + toggleBadge.classList.remove('hidden'); + } + } + function clearBadge() { + badgeCount = 0; + if (toggleBadge) { toggleBadge.classList.add('hidden'); toggleBadge.textContent = ''; } + } + + // ── Identity ────────────────────────────────────────────── + function fetchMe() { + fetch('/api/auth/me').then(r => r.json()).then(m => { + me = m; + if (m && m.authenticated) { + userEl.textContent = m.username; + userEl.title = `Signed in as ${m.username} (${m.role})`; + signoutBtn.textContent = 'Sign out'; + signoutBtn.dataset.action = 'logout'; + signoutBtn.style.display = ''; + } else if (m && m.accounts) { + // Anonymous — viewing is open; sign in to gain control. + userEl.textContent = 'viewing'; + userEl.title = 'Not signed in — view-only'; + signoutBtn.textContent = 'Sign in'; + signoutBtn.dataset.action = 'login'; + signoutBtn.style.display = ''; + } else { + // No accounts configured (legacy mode). + userEl.textContent = ''; + signoutBtn.style.display = 'none'; + } + renderControl(); + }).catch(() => {}); + } + + // ── Init ────────────────────────────────────────────────── + function init() { + panel = document.getElementById('agent-chat'); + log = document.getElementById('agent-chat-log'); + input = document.getElementById('agent-chat-text'); + sendBtn = document.getElementById('agent-chat-send'); + conn = document.getElementById('agent-chat-conn'); + banner = document.getElementById('agent-control-banner'); + closeBtn = document.getElementById('agent-chat-close'); + userEl = document.getElementById('agent-chat-user'); + signoutBtn = document.getElementById('agent-chat-signout'); + railBtn = document.getElementById('agent-rail-toggle'); // collapsed-rail spark + resizeEl = document.getElementById('agent-chat-resize'); + // Connection dot + unseen-activity badge now live on the collapsed rail. + toggleDot = document.getElementById('agent-rail-dot'); + toggleBadge = document.getElementById('agent-rail-badge'); + if (!panel) return; // markup not present + + restorePrefs(); + // Dual-render: retire a transcript ask card when its ask is answered + // (from the transcript OR the main stage) or the turn is cancelled. + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('ASK_CLEARED', ({ request_id }) => { + markAnswered(request_id); + // If answered from the main stage (AskStage), restore working state. + if (askPending) setAskState(false); + }); + } + if (railBtn) railBtn.addEventListener('click', () => togglePanel(true)); + closeBtn.addEventListener('click', () => togglePanel(false)); + setupResize(); + // Ctrl/Cmd+J toggles the panel from anywhere. + document.addEventListener('keydown', (e) => { + if ((e.ctrlKey || e.metaKey) && (e.key === 'j' || e.key === 'J')) { + e.preventDefault(); // suppress browser default (downloads) always + if (e.repeat) return; // ignore held-key auto-repeat + if (document.activeElement === input) return; // don't toggle while composing + togglePanel(); + } + }); + signoutBtn.addEventListener('click', async () => { + if (signoutBtn.dataset.action === 'login') { + window.location.href = '/login'; + return; + } + try { await fetch('/api/auth/logout', { method: 'POST' }); } catch (_) {} + window.location.reload(); + }); + fetchMe(); + + // Build the autocomplete dropdown inside the composer (positioned above + // the textarea via CSS). + const inputWrap = input.parentNode; + if (inputWrap) { + acComplete = document.createElement('div'); + acComplete.className = 'ac-complete hidden'; + inputWrap.insertBefore(acComplete, inputWrap.firstChild); + + // Queued-message panel (above the composer) for type-while-busy. + queuePanel = document.createElement('div'); + queuePanel.className = 'ac-queue hidden'; + if (inputWrap.parentNode) inputWrap.parentNode.insertBefore(queuePanel, inputWrap); + + // (Stop is no longer a separate button — the composer send button + // morphs into a stop square while a cancellable turn runs.) + + // Sticky ASK-approval slot — above the queue + composer, never scrolls away. + pendingSlot = document.createElement('div'); + pendingSlot.className = 'ac-pending hidden'; + if (inputWrap.parentNode) inputWrap.parentNode.insertBefore(pendingSlot, queuePanel); + } + + // "↓ N new" jump pill + pin-to-bottom scroll tracking. + jumpPill = document.createElement('button'); + jumpPill.className = 'ac-jump hidden'; + jumpPill.addEventListener('click', jumpToBottom); + panel.appendChild(jumpPill); + log.addEventListener('scroll', () => { + stickBottom = nearBottom(); + if (stickBottom) newCount = 0; + renderJumpPill(); + }); + + // One button, two roles: stop a running cancellable turn, else send. + sendBtn.addEventListener('click', () => { + if (agentBusy && busySource === 'user') cancelTurn(); + else submit(); + }); + input.addEventListener('input', () => { autosize(); updateCompletions(); renderComposerButton(); }); + // Close the menu shortly after blur (delay lets a mousedown selection land). + input.addEventListener('blur', () => setTimeout(hideCompletions, 120)); + input.addEventListener('keydown', (e) => { + // While the completion menu is open it owns the navigation keys. + if (acItems.length) { + if (e.key === 'ArrowDown') { e.preventDefault(); moveCompletion(1); return; } + if (e.key === 'ArrowUp') { e.preventDefault(); moveCompletion(-1); return; } + if (e.key === 'Tab') { e.preventDefault(); acceptCompletion(acItems[acIdx]); return; } + if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); acceptCompletion(acItems[acIdx]); return; } + if (e.key === 'Escape') { e.preventDefault(); hideCompletions(); return; } + } + if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); submit(); } + // Escape mirrors Stop: cancel a cancellable (user) turn and clear busy + // (a cancelled turn emits no stream_end, so clear optimistically). + if (e.key === 'Escape' && agentBusy && busySource === 'user') { + e.preventDefault(); cancelTurn(); + } + }); + renderComposerButton(); // initial state: disabled until there's text + } + + document.addEventListener('DOMContentLoaded', init); + + // Public: programmatically send a message/command (e.g. the Home page's + // "Start / continue an experiment" button sends '/wizard'). + function runCommand(text) { + if (!text) return; + if (!hasControl) { renderControl(); return; } + // Works whether or not the chat panel is open, so the landing can drive + // the agent (enter plan mode) without foregrounding the chat REPL. If the + // socket isn't up yet, queue and connect — it flushes on open. + if (ws && ws.readyState === WebSocket.OPEN) { actuallySend(text); return; } + pendingProgrammatic.push(text); + if (!ws) connect(); + } + + return { togglePanel, runCommand, buildAskCard, answerChoice, mdToHtml, hasControl: () => hasControl }; +})(); diff --git a/gently/ui/web/static/js/app.js b/gently/ui/web/static/js/app.js index 203cea04..55443940 100644 --- a/gently/ui/web/static/js/app.js +++ b/gently/ui/web/static/js/app.js @@ -6,7 +6,7 @@ const state = { ws: null, connected: false, - tab: TABS.EMBRYOS, // Default to Embryos tab + tab: TABS.HOME, // Default to the Home landing tab snapshots: [], calibration: [], embryos: [], @@ -60,6 +60,8 @@ function updateCalibrationCount() { function switchTab(tabName) { if (!tabName) return; state.tab = tabName; + // ux_v2 grouped rail mirrors the active tab off this single chokepoint. + if (typeof ClientEventBus !== 'undefined') ClientEventBus.emit('TAB_CHANGED', tabName); // Update tab styling document.querySelectorAll('.tab').forEach(t => t.classList.remove('active')); @@ -71,6 +73,9 @@ function switchTab(tabName) { const content = document.getElementById(`${tabName}-content`); if (content) content.classList.add('active'); + // Lazy-init Home landing tab + if (tabName === TABS.HOME && typeof HomeApp !== 'undefined') HomeApp.init(); + // Render galleries if (tabName === TABS.CALIBRATION) renderCalibrationGallery(); if (tabName === TABS.EVENTS) renderEventsTable(); @@ -95,6 +100,16 @@ function switchTab(tabName) { ExperimentOverview.init(); } + // Lazy-init Notebook tab + if (tabName === TABS.NOTEBOOK && typeof NotebookApp !== 'undefined') { + NotebookApp.init(); + } + + // Lazy-init Gallery tab + if (tabName === TABS.GALLERY && typeof GalleryTab !== 'undefined') { + GalleryTab.init(); + } + // Update statusbar for context updateStatusbar(); } @@ -535,11 +550,13 @@ function fetchDeviceStatus() { .then(r => r.json()) .then(data => { _microscopeConnected = data.microscope; - _setBadge('status-microscope-badge', data.microscope, 'Online', 'Offline'); - updateTopLevelDot(); + ConnectionStatus.setMicroscope(data.microscope); }) .catch(() => { - _setBadge('status-microscope-badge', false, '', '--'); + // Transient poll failure: keep the last-known badge. The next + // successful poll re-renders via the store if the value changed + // (writing '--' here could stick, since the store only re-renders + // on an actual change, not on an unchanged success). }); } @@ -552,30 +569,41 @@ function _setBadge(id, isOn, onText, offText) { } function updateGentlyStatus(connected) { - _setBadge('status-gently-badge', connected, 'Online', 'Offline'); - updateTopLevelDot(); + // Feed the single source of truth; the header re-renders via the + // ConnectionStatus subscriber (renderConnectionUI). + ConnectionStatus.setGently(connected); } -function updateTopLevelDot() { +// Single renderer for the header connection UI, driven by a ConnectionStatus +// snapshot. Subscribed once at startup, so the pill, both popover badges, and +// the dot always reflect the same shared state. +function renderConnectionUI(s) { + _setBadge('status-gently-badge', s.gentlyConnected, 'Online', 'Offline'); + _setBadge('status-microscope-badge', s.microscopeConnected, 'Online', 'Offline'); const dot = document.getElementById('status-dot'); const text = document.getElementById('status-text'); if (!dot || !text) return; - const gentlyUp = state.connected; - const scopeUp = _microscopeConnected; - dot.classList.remove('connected', 'partial'); - if (gentlyUp && scopeUp) { + if (s.gentlyConnected && s.microscopeConnected) { dot.classList.add('connected'); text.textContent = 'Connected'; - } else if (gentlyUp) { + } else if (s.gentlyConnected) { + // Gently is up but the microscope isn't connected. "Online" here read as + // "all connected" and hid a disconnected scope — surface the operator's + // actual question instead (matches the popover's "Microscope: Offline"). dot.classList.add('partial'); - text.textContent = 'Online'; + text.textContent = 'Scope offline'; } else { text.textContent = 'Offline'; } } +// Back-compat shim: any legacy caller re-renders from the current snapshot. +function updateTopLevelDot() { + renderConnectionUI(ConnectionStatus.get()); +} + document.addEventListener('DOMContentLoaded', () => { // Initialize presence manager (before WebSocket so ID is ready) PresenceManager.init(); @@ -617,6 +645,24 @@ document.addEventListener('DOMContentLoaded', () => { } }); + // Connection status: one source of truth, three writers (this /ws, the + // device-status poll, and the agent /ws/agent). Subscribe the header + // renderer BEFORE connecting so the first handshake renders correctly. + ConnectionStatus.subscribe(renderConnectionUI); + + // Instant microscope availability: the single DEVICE_LAYER_AVAILABILITY + // signal (emitted by the launcher's device-layer watcher on every state + // transition) flips the microscope status the moment the layer attaches or + // detaches — no waiting for the 15s /api/device-status poll below, which + // stays on as a slow reconciler. + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('DEVICE_LAYER_AVAILABILITY', (d) => { + if (d && typeof d.available === 'boolean') { + ConnectionStatus.setMicroscope(d.available); + } + }); + } + // Start WebSocket connection connectWebSocket(); @@ -631,7 +677,7 @@ document.addEventListener('DOMContentLoaded', () => { const hash = window.location.hash.slice(1); // remove # if (hash) { const [tab, param] = hash.split(':'); - if (tab === TABS.PLANS || tab === TABS.SESSIONS || tab === TABS.EMBRYOS || tab === TABS.CALIBRATION || tab === TABS.EVENTS || tab === TABS.EXPERIMENT) { + if (tab === TABS.HOME || tab === TABS.PLANS || tab === TABS.SESSIONS || tab === TABS.EMBRYOS || tab === TABS.CALIBRATION || tab === TABS.EVENTS || tab === TABS.EXPERIMENT || tab === TABS.NOTEBOOK || tab === TABS.GALLERY) { switchTab(tab); if (tab === TABS.PLANS && param && typeof openCampaign === 'function') { setTimeout(() => openCampaign(param), 200); diff --git a/gently/ui/web/static/js/ask-stage.js b/gently/ui/web/static/js/ask-stage.js new file mode 100644 index 00000000..b5e63db1 --- /dev/null +++ b/gently/ui/web/static/js/ask-stage.js @@ -0,0 +1,53 @@ +/** + * AskStage (ux_v2) — renders the agent's CURRENT pending ask prominently on the + * main stage, in addition to the chat transcript. One payload, two renderers: + * it reuses AgentChat.buildAskCard so the stage and the transcript can't drift, + * and answering from either surface clears both (via the ASK_CLEARED event that + * AgentChat fires off the CHOICE lifecycle — not stream_end, which arrives only + * after an in-turn answer and never for a cancelled turn). + * + * No-ops unless #ask-stage is present (gated behind GENTLY_UX_V2 in the + * template), so it never affects the v1 dashboard. + */ +const AskStage = (() => { + let stageEl = null; + let current = null; // { reqId, data, isWake } + + function clear() { + current = null; + if (stageEl) { stageEl.innerHTML = ''; stageEl.classList.add('hidden'); } + } + + function render() { + if (!stageEl || !current || typeof AgentChat === 'undefined' || !AgentChat.buildAskCard) return; + const hasControl = AgentChat.hasControl ? AgentChat.hasControl() : true; + const card = AgentChat.buildAskCard(current.data, { + reqId: current.reqId, + isWake: current.isWake, + hasControl, + onPick: (sel) => AgentChat.answerChoice(current.reqId, sel), + }); + stageEl.innerHTML = ''; + stageEl.appendChild(card); + stageEl.classList.remove('hidden'); + } + + function init() { + stageEl = document.getElementById('ask-stage'); + if (!stageEl || typeof ClientEventBus === 'undefined') return; // ux_v2 off → no-op + + ClientEventBus.on('AGENT_ASK', ({ request_id, choice_data, origin }) => { + current = { reqId: request_id, data: choice_data || {}, isWake: origin === 'wake' }; + render(); + }); + ClientEventBus.on('ASK_CLEARED', ({ request_id }) => { + if (!current) return; + if (request_id === '*' || request_id === current.reqId) clear(); + }); + // Re-render read-only / actionable when control changes hands mid-ask. + ClientEventBus.on('AGENT_CONTROL', () => { if (current) render(); }); + } + + document.addEventListener('DOMContentLoaded', init); + return { clear }; +})(); diff --git a/gently/ui/web/static/js/boot-banner.js b/gently/ui/web/static/js/boot-banner.js new file mode 100644 index 00000000..b7a0342d --- /dev/null +++ b/gently/ui/web/static/js/boot-banner.js @@ -0,0 +1,173 @@ +/** + * Boot banner — a small, always-visible, non-modal summary of the device-layer + * boot, so the operator can follow MMCore startup from anywhere on the dashboard + * (the Devices panel is the on-demand console; this is its compact form). + * + * Polls /api/device-layer/status globally — fast (1s) while booting, calm (6s) + * otherwise so it also catches a device layer started later from the Devices + * panel. Also publishes a readiness signal (window.gentlyDeviceReady + + * ClientEventBus 'DEVICE_LAYER_STATE') that hardware-only controls can gate on. + */ +const BootBanner = (function () { + const STAGE_TOTAL = 5; + + let _el, _text, _details, _retry, _close; + let _timer = null; + let _readyTimer = null; // auto-dismiss timer for the "Microscope ready" flash + let _pollMs = 0; + let _lastState = null; + let _dom = false; + + function cacheDom() { + if (_dom) return; + _el = document.getElementById('gently-boot-banner'); + if (!_el) return; + _text = document.getElementById('boot-banner-text'); + _details = document.getElementById('boot-banner-details'); + _retry = document.getElementById('boot-banner-retry'); + _close = document.getElementById('boot-banner-close'); + _dom = true; + } + + function init() { + cacheDom(); + if (!_el) return; + _details.addEventListener('click', () => { + // The v2 landing overlay covers the workspace — dismiss it first, + // or the tab switch below happens invisibly behind it and the + // click feels dead (found via session replay: the operator + // clicked Details and saw nothing for over two minutes). + const landing = document.getElementById('v2-landing'); + if (landing && !landing.classList.contains('dismissed')) { + const skip = document.getElementById('v2-landing-skip'); + if (skip) skip.click(); else landing.classList.add('dismissed'); + } + if (typeof switchTab === 'function' && typeof TABS !== 'undefined') switchTab(TABS.DEVICES); + // The console we just opened IS the details — acknowledge so the + // click always has visible feedback (Retry lives on in the + // console's Start button; the poll re-shows on a state change). + acknowledge(); + }); + _retry.addEventListener('click', onRetry); + _close.addEventListener('click', acknowledge); + setPoll(1500); + } + + function setPoll(ms) { + if (ms === _pollMs && _timer) return; + _pollMs = ms; + if (_timer) clearInterval(_timer); + _timer = setInterval(poll, _pollMs); + } + + async function poll() { + try { + const r = await fetch('/api/device-layer/status'); + if (!r.ok) return; + render(await r.json()); + } catch (e) { + /* keep last-known UI */ + } + } + + // Failure state the user has dismissed (Details or ×). Without this the + // poll re-shows the banner every cycle — dismissal never sticks. + let _ackedState = null; + + function acknowledge() { + _ackedState = _lastState; + hide(); + } + + function render(d) { + const state = d.state; + if (state !== _lastState) _ackedState = null; // new state → new banner + + // Publish a readiness signal for hardware-only controls to gate on. + const ready = state === 'ready' || state === 'external'; + window.gentlyDeviceReady = ready; + if (state !== _lastState && typeof ClientEventBus !== 'undefined') { + ClientEventBus.emit('DEVICE_LAYER_STATE', { state, ready }); + } + + if (state === 'starting' || state === 'initializing') { + setPoll(1000); + // Dismissed by the operator? Keep it hidden. The progress label keeps + // changing while state stays 'initializing', so without this ack an × + // wouldn't stick and the banner would re-show every second. + if (_ackedState === state) { _lastState = state; return; } + show('booting'); + const p = d.progress || {}; + const step = p.i ? `step ${p.i}/${p.n || STAGE_TOTAL} · ` : ''; + _text.textContent = `Microscope warming up — ${step}${p.label || 'starting…'}`; + // Closable — a long or stuck boot must always be dismissible, not just + // via Details (which yanks you to the Devices tab). + btns({ details: true, retry: false, close: true }); + } else if (state === 'ready') { + setPoll(6000); + if (_ackedState === state) { _lastState = state; return; } + const justFinished = _lastState === 'starting' || _lastState === 'initializing'; + if (justFinished) { + // Flash "ready" briefly, then auto-dismiss. Unconditional hide via + // a tracked timer — the old class-guarded hide could leave the + // flash stuck on screen ("never disappears"); × is a backstop too. + show('ready'); + _text.textContent = 'Microscope ready'; + btns({ details: false, retry: false, close: true }); + if (_readyTimer) clearTimeout(_readyTimer); + _readyTimer = setTimeout(() => { _readyTimer = null; hide(); }, 3500); + } else if (!(_el.classList.contains('ready') && !_el.hidden)) { + hide(); // already ready on load, or the ready-flash was dismissed + } + } else if (state === 'failed' || state === 'crashed') { + if (_ackedState === state) { setPoll(6000); _lastState = state; return; } + show('failed'); + _text.textContent = + (d.failure && d.failure.summary) || + (state === 'crashed' + ? 'The device layer stopped unexpectedly.' + : "The microscope didn't start."); + btns({ details: true, retry: true, close: true }); + setPoll(6000); + } else { + // stopped (software-only session) or external with nothing to add. + hide(); + setPoll(6000); + } + _lastState = state; + } + + function show(kind) { + _el.hidden = false; + _el.className = 'boot-banner ' + kind; + } + function hide() { + _el.hidden = true; + } + function btns({ details, retry, close }) { + _details.hidden = !details; + _retry.hidden = !retry; + _close.hidden = !close; + } + + async function onRetry() { + _retry.disabled = true; + try { + await fetch('/api/device-layer/start', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }); + } catch (e) { + /* the status poll will reflect the outcome */ + } finally { + _retry.disabled = false; + setTimeout(poll, 400); + poll(); + } + } + + return { init }; +})(); + +document.addEventListener('DOMContentLoaded', () => BootBanner.init()); diff --git a/gently/ui/web/static/js/campaigns.js b/gently/ui/web/static/js/campaigns.js index 36d29623..d42cdbd4 100644 --- a/gently/ui/web/static/js/campaigns.js +++ b/gently/ui/web/static/js/campaigns.js @@ -40,6 +40,18 @@ const SPEC_UNITS = { laser_power_pct: '%', interval_s: 's', estimated_duration_h: ' hrs', estimated_days: ' days', }; +// Imaging-spec fields the inspector lets you edit/fill inline (ordered for the form). +// Empty ones still show \u2014 that's how you fill a TBD value like laser power. +const IMAGING_SPEC_FIELDS = [ + 'strain', 'genotype', 'reporter', 'sample_prep', 'temperature_c', 'num_embryos', + 'num_slices', 'exposure_ms', 'laser_wavelength_nm', 'laser_power_pct', 'interval_s', + 'target_window', 'start_stage', 'stop_condition', 'estimated_duration_h', + 'success_criteria', 'comparison_to', +]; +const SPEC_NUMERIC = new Set([ + 'temperature_c', 'num_embryos', 'num_slices', 'exposure_ms', 'laser_wavelength_nm', + 'laser_power_pct', 'interval_s', 'estimated_duration_h', +]); // ── State ──────────────────────────────────────────────── const state = { @@ -52,6 +64,11 @@ const state = { versions: [], // snapshots list viewingSnapshotId: null, allItemsFlat: {}, // id → item for quick lookup + editingSpec: false, // inspector imaging-spec edit mode + _inspectorData: null, // last item-detail payload (for re-render on edit toggle) + _specError: '', // inline save error in the spec editor + _sessionPickerOpen: false, // whether the session link picker is visible + _availableSessions: null, // null = not yet loaded, [] = loaded (for link picker) }; // ── DOM refs (cached on init) ──────────────────────────── @@ -117,11 +134,19 @@ function boot() { case 'select-item': selectItem(id); break; case 'open-campaign': openCampaign(id); break; case 'navigate-item': e.stopPropagation(); navigateToItem(id); break; + case 'run-item': e.stopPropagation(); runPlanItem(id); break; case 'filter-type': applyTypeFilter(el.dataset.filterType); break; case 'view-version': viewVersion(el.dataset.versionId, el.dataset.isCurrent === 'true'); break; case 'back-to-current': backToCurrent(); break; case 'scroll-to': e.stopPropagation(); scrollCanvasTo(el.dataset.target); break; case 'toggle-phase': toggleNavPhase(el); break; + case 'spec-edit': e.stopPropagation(); startSpecEdit(); break; + case 'spec-cancel': e.stopPropagation(); cancelSpecEdit(); break; + case 'spec-save': e.stopPropagation(); saveSpecEdit(); break; + case 'session-picker-open': e.stopPropagation(); openSessionPicker(); break; + case 'session-picker-cancel': e.stopPropagation(); cancelSessionPicker(); break; + case 'session-picker-link': e.stopPropagation(); submitSessionLink(); break; + case 'session-delink': e.stopPropagation(); handleSessionDelink(el.dataset.sessionId); break; } }); @@ -131,6 +156,13 @@ function boot() { // Plan view switcher setupPlanViewSwitcher(); + // Live refresh: re-fetch the active campaign when the plan changes (item status, + // session link, new item, progress). The store emits PLAN_UPDATED, the server + // broadcasts it to /ws, and websocket.js re-emits it on the client bus. + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('PLAN_UPDATED', () => scheduleCampaignRefresh()); + } + // Load campaigns — auto-selects first, or the specified one const initialId = window.INITIAL_CAMPAIGN_ID; if (initialId) { @@ -267,6 +299,26 @@ async function openCampaign(campaignId) { renderAll(); } +// Live refresh of the open campaign (debounced) — re-fetch its tree and re-render, +// preserving the selected item so the inspector reflects the change without a reload. +let _planRefreshTimer = null; +function scheduleCampaignRefresh() { + if (_planRefreshTimer) clearTimeout(_planRefreshTimer); + _planRefreshTimer = setTimeout(() => { + _planRefreshTimer = null; + refreshActiveCampaign().catch(() => {}); + }, 400); +} +async function refreshActiveCampaign() { + if (!state.activeCampaignId) return; + const keep = state.selectedItemId; + await loadDocument(state.activeCampaignId); + if (!state.docData) return; + renderAll(); + // Don't clobber an in-progress spec edit with a re-fetch. + if (keep && !state.editingSpec) selectItem(keep).catch(() => {}); +} + // Handle browser back/forward window.addEventListener('popstate', e => { const s = e.state; @@ -579,6 +631,14 @@ function renderVersionHistory() { // ══════════════════════════════════════════════════════════ async function selectItem(itemId) { + // A re-fetch of the same item (e.g. after saving) keeps view mode; switching + // to a different item always lands in read-only. + if (itemId !== state.selectedItemId) { + state.editingSpec = false; + state._specError = ''; + state._sessionPickerOpen = false; + state._availableSessions = null; + } state.selectedItemId = itemId; // Highlight in document @@ -617,10 +677,17 @@ async function selectItem(itemId) { } function renderInspector(data) { + state._inspectorData = data; const item = data.item; const deps = data.dependencies || []; const dnts = data.dependents || []; - const sessions = data.sessions || []; + // Build a metadata map from the campaign-level sessions included in the payload, + // then derive the per-item sessions list from item.session_ids (the true source + // of truth). data.sessions is the campaign pool; we only show sessions that are + // actually linked to THIS item. + const _sessionMeta = {}; + (data.sessions || []).forEach(s => { _sessionMeta[s.session_id || s.id] = s; }); + const sessions = (item.session_ids || []).map(sid => _sessionMeta[sid] || { session_id: sid, id: sid }); if ($inspectorTitle) $inspectorTitle.textContent = item.title; if ($inspectorStatus) { @@ -639,6 +706,19 @@ function renderInspector(data) { ${item.id}
    `; + // Run affordance — only for an actionable imaging item. Routes through the + // agent (it applies this item's spec via execute_plan_item), in keeping with + // the agent-first paradigm. + if (item.type === 'imaging' && item.status === 'planned') { + html += `
    + + Hands it to the agent to apply the spec and start +
    `; + } + // Description if (item.description) { html += section('Description', `
    ${esc(item.description)}
    `); @@ -649,9 +729,20 @@ function renderInspector(data) { html += section('Outcome', `
    ${esc(item.outcome)}
    `); } - // Imaging spec - if (item.imaging_spec) { - html += section('Imaging Specification', `${renderSpecTable(item.imaging_spec)}
    `); + // Imaging spec — view, or edit/fill inline (the laser-power loop). Shown for any + // imaging item even when no spec is set yet, so empty fields can be filled. + if (item.type === 'imaging' || item.imaging_spec) { + const spec = item.imaging_spec || {}; + if (state.editingSpec) { + html += section('Imaging Specification', renderSpecEditor(spec)); + } else { + const rows = renderSpecTable(spec); + const content = rows + ? `${rows}
    ` + : '
    No parameters set yet
    '; + const editBtn = ''; + html += section('Imaging Specification', content, editBtn); + } } // Bench spec @@ -709,24 +800,235 @@ function renderInspector(data) { html += section('References', refHtml); } - // Sessions + // Sessions — item-scoped (item.session_ids), with link/delink controls. + const _linkBtn = ``; + let sessHtml = ''; if (sessions.length > 0) { - let sessHtml = ''; sessions.forEach(s => { + const sid = s.session_id || s.id || ''; + const name = s.name || s.planned_intent || sid || 'Session'; sessHtml += `
    - ${esc(s.planned_intent || s.id || 'Session')} - ${s.created_at ? `${formatDate(s.created_at)}` : ''} + ${esc(name)} + + ${s.created_at ? `${formatDate(s.created_at)}` : ''} + +
    `; }); - html += section('Sessions', sessHtml); } else { - html += section('Sessions', - '
    No linked sessions
    '); + sessHtml = '
    No linked sessions
    '; + } + // Inline link picker — rendered when openSessionPicker() has set state flag + loaded data. + if (state._sessionPickerOpen) { + if (state._availableSessions === null) { + // Still loading — show spinner text; will re-render once fetch completes. + sessHtml += `
    Loading sessions…
    `; + } else { + const _linkedIds = new Set(item.session_ids || []); + const _available = state._availableSessions.filter(s => !_linkedIds.has(s.session_id)); + const _opts = _available.length === 0 + ? `` + : _available.map(s => ``).join(''); + sessHtml += `
    + +
    + + +
    +
    `; + } } + html += section('Sessions', sessHtml, _linkBtn); if ($inspectorBody) $inspectorBody.innerHTML = html; } +// Editable imaging-spec form. Lists every fillable field — empty ones included, +// flagged — so a TBD value (e.g. laser power) is obvious and one click away. +function renderSpecEditor(spec) { + let rows = ''; + for (const key of IMAGING_SPEC_FIELDS) { + const label = SPEC_LABELS[key] || key; + const val = spec[key]; + const has = val != null && val !== ''; + const numeric = SPEC_NUMERIC.has(key); + const unit = SPEC_UNITS[key] + ? `${esc(SPEC_UNITS[key].trim())}` : ''; + const rowCls = has ? 'spec-edit-row' : 'spec-edit-row spec-edit-row--empty'; + rows += `
    + + + ${unit} + +
    `; + } + const err = state._specError + ? `
    ${esc(state._specError)}
    ` : ''; + return `
    + ${rows} + ${err} +
    + + +
    +
    `; +} + +function startSpecEdit() { + if (!state._inspectorData) return; + state.editingSpec = true; + state._specError = ''; + renderInspector(state._inspectorData); +} + +function cancelSpecEdit() { + state.editingSpec = false; + state._specError = ''; + if (state._inspectorData) renderInspector(state._inspectorData); +} + +// Collect changed/filled fields and PATCH them. The store fires PLAN_UPDATED, +// which live-refreshes the plan; we also re-fetch the inspector for immediacy. +async function saveSpecEdit() { + const data = state._inspectorData; + const item = data && data.item; + const campaignId = state.activeCampaignId; + if (!item || !campaignId) return; + + const orig = item.imaging_spec || {}; + const specPatch = {}; + document.querySelectorAll('#inspector-body [data-spec-key]').forEach(inp => { + const key = inp.dataset.specKey; + const raw = inp.value.trim(); + const hadVal = orig[key] != null && orig[key] !== ''; + if (raw === '') { + if (hadVal) specPatch[key] = ''; // cleared an existing value → unset + return; // stayed empty → skip + } + let v = raw; + if (SPEC_NUMERIC.has(key)) { + const n = Number(raw); + if (!Number.isNaN(n)) v = n; + } + if (String(orig[key] ?? '') !== String(v)) specPatch[key] = v; + }); + + state.editingSpec = false; + state._specError = ''; + if (Object.keys(specPatch).length === 0) { + selectItem(item.id).catch(() => {}); // nothing changed — just leave edit mode + return; + } + + try { + const res = await fetch( + `/api/campaigns/${encodeURIComponent(campaignId)}/items/${encodeURIComponent(item.id)}`, + { + method: 'PATCH', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ spec: specPatch }), + }, + ); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + selectItem(item.id).catch(() => {}); // refresh inspector now; PLAN_UPDATED refreshes the plan + } catch (err) { + console.error('Failed to save spec:', err); + state.editingSpec = true; + state._specError = 'Could not save — try again.'; + renderInspector(data); + } +} + +// ── Session link / delink ───────────────────────────────────────────────────── + +// Open the inline session picker. Fetches /api/sessions and re-renders with the +// picker shown. Two-phase: immediate re-render with loading state, then again +// once the fetch resolves (mirrors the pattern of selectItem loading state). +async function openSessionPicker() { + state._sessionPickerOpen = true; + state._availableSessions = null; // triggers loading display + if (state._inspectorData) renderInspector(state._inspectorData); + try { + const res = await fetch('/api/sessions'); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + const body = await res.json(); + state._availableSessions = body.sessions || []; + } catch (err) { + console.error('Failed to load sessions for picker:', err); + state._availableSessions = []; + } + if (state._sessionPickerOpen && state._inspectorData) { + renderInspector(state._inspectorData); + } +} + +function cancelSessionPicker() { + state._sessionPickerOpen = false; + state._availableSessions = null; + if (state._inspectorData) renderInspector(state._inspectorData); +} + +// Read the picker ' : ''} + `).join(''); + const wHtml = watchpoints.map(it => ` +
    + + ${esc(it.target)}${it.condition ? ' — ' + esc(it.condition) : ''} + ${hc ? '' : ''} +
    `).join(''); + const eHtml = expectations.map(it => ` +
    + + ${esc(it.target)}${it.prediction ? ': ' + esc(it.prediction) : ''} + ${hc ? '' : ''} +
    `).join(''); + + // kind → existing cx-dot color: observation=blue, finding=green, question=amber + const dotFor = (k) => k === 'finding' ? 'cx-e' : (k === 'question' ? 'cx-q' : 'cx-w'); + const nHtml = notes.map(n => ` +
    + + ${esc(n.title || n.body)} +
    `).join(''); + + el.innerHTML = '
    Agent’s view
    ' + + section('Open questions', qHtml) + section('Watching', wHtml) + + section('Expectations', eHtml) + section('From the notebook', nHtml); + wire(); + } + + function wire() { + el.querySelectorAll('.cx-item').forEach(item => { + const kind = item.dataset.kind, id = item.dataset.id; + const actBtn = item.querySelector('.cx-act'); + if (!actBtn) return; + const act = actBtn.dataset.act; + if (act === 'answer') { + const box = item.querySelector('.cx-answer'); + const input = item.querySelector('.cx-answer-input'); + const submit = () => resolve(kind, id, { resolution: input.value.trim() }); + actBtn.addEventListener('click', () => { box.classList.toggle('hidden'); if (!box.classList.contains('hidden')) input.focus(); }); + item.querySelector('.cx-answer-go').addEventListener('click', submit); + input.addEventListener('keydown', e => { if (e.key === 'Enter') { e.preventDefault(); submit(); } }); + } else if (act === 'resolve') { + actBtn.addEventListener('click', () => resolve(kind, id, {})); + } else if (act === 'confirm') { + actBtn.addEventListener('click', () => resolve(kind, id, { status: 'confirmed' })); + } + }); + el.querySelectorAll('.cx-note').forEach(row => { + row.style.cursor = 'pointer'; + row.addEventListener('click', () => { + if (typeof switchTab === 'function') switchTab('notebook'); + }); + }); + } + + async function resolve(kind, id, body) { + try { + await fetch(`/api/context/${kind}/${encodeURIComponent(id)}/resolve`, { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body || {}), + }); + } catch (e) { /* ignore; surface stays as-is */ } + fetchAndRender(); // CONTEXT_UPDATED also re-fetches for every client + } + + function init() { + el = document.getElementById('context-surface'); + if (!el) return; // ux_v2 off → no-op + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('CONTEXT_UPDATED', () => fetchAndRender()); + ClientEventBus.on('AGENT_CONTROL', () => fetchAndRender()); // re-render with/without resolve controls + } + fetchAndRender(); + } + + document.addEventListener('DOMContentLoaded', init); + return { refresh: fetchAndRender }; +})(); diff --git a/gently/ui/web/static/js/control-auth.js b/gently/ui/web/static/js/control-auth.js new file mode 100644 index 00000000..359a1e4e --- /dev/null +++ b/gently/ui/web/static/js/control-auth.js @@ -0,0 +1,47 @@ +// control-auth.js — friendly, actionable hint when a control action is denied. +// +// Control routes (POST/PUT/DELETE that move hardware or mutate state) return 403 +// when the session lacks the control role — e.g. account mode with no operator +// logged in. Without this, the only signal is a bare 403 in the console and a +// button that silently does nothing. This installs a single global fetch wrapper +// that surfaces a throttled "Control required — Log in" toast instead. It only +// reads res.status (never consumes the body), so callers behave unchanged. +(function () { + if (window.__gentlyControlAuthPatched) return; + window.__gentlyControlAuthPatched = true; + + const origFetch = window.fetch.bind(window); + let lastHintAt = 0; + + function isControlRequest(input, init) { + let method = (init && init.method) || (typeof input === 'object' && input && input.method) || 'GET'; + method = String(method).toUpperCase(); + if (method === 'GET' || method === 'HEAD' || method === 'OPTIONS') return false; + const url = typeof input === 'string' ? input : (input && input.url) || ''; + return url.includes('/api/'); + } + + function showLoginHint() { + const now = Date.now(); + if (now - lastHintAt < 4000) return; // throttle so repeated clicks don't stack + lastHintAt = now; + const msg = 'Control required — you are in view-only mode.'; + if (typeof showGentlyToast === 'function') { + showGentlyToast(msg, 'Log in', () => { window.location.href = '/login'; }, 7000); + } else { + console.warn(msg + ' Log in at /login to drive hardware.'); + } + } + + window.fetch = async function (input, init) { + const res = await origFetch(input, init); + try { + if (res.status === 403 && isControlRequest(input, init)) { + showLoginHint(); + } + } catch (_) { + // Never let the hint interfere with the actual response. + } + return res; + }; +})(); diff --git a/gently/ui/web/static/js/device-layer.js b/gently/ui/web/static/js/device-layer.js new file mode 100644 index 00000000..2e3f7acc --- /dev/null +++ b/gently/ui/web/static/js/device-layer.js @@ -0,0 +1,333 @@ +/** + * Device-layer supervision card (Devices tab). + * + * The runtime mirror of the launch gate's hardware block: shows whether the + * device layer is running / stopped / external / crashed, lets an operator + * Start or Stop it from the UI, and tails its console. Consumes the routes + * added in gently/ui/web/routes/device_layer.py: + * + * GET /api/device-layer/status -> {state, managed, pid, port, port_open, + * sam_device, uptime_seconds, log_tail[]} + * GET /api/device-layer/log?limit=N -> {lines: [...]} + * POST /api/device-layer/start (control-gated) body {sam_device?, config_path?} + * POST /api/device-layer/stop (control-gated) body {confirm?, force?} + * -> 409 {blocked:true,...} mid-acquisition + * + * Self-contained IIFE (no devices.js internals). Polls only while the Devices + * tab is visible (ClientEventBus 'TAB_CHANGED'). Control auth is transparent: + * a same-origin fetch carries the gently_session cookie, and control-auth.js + * already toasts on a bare 403, so Start/Stop need no credential code. + */ +const DeviceLayerCard = (function () { + let _pollMs = 5000; // adaptive: 1s while booting, 5s otherwise + + // state -> [pill modifier class, label]. + const PILL = { + ready: ['live', 'ready'], + running: ['live', 'running'], // legacy alias + starting: ['paused', 'starting'], + initializing: ['paused', 'starting'], + external: ['paused', 'external'], + stopped: ['', 'stopped'], + crashed: ['error', 'crashed'], + failed: ['error', 'failed'], + }; + + let _card, _pill, _meta, _start, _stop, _logToggle, _log, _hint; + let _timer = null; + let _busy = false; // a start/stop is in flight + let _dom = false; + let _autoLog = false; // log pane was auto-opened for the boot phase + let _logUserOverride = false; // user toggled the log → stop auto-managing it this cycle + let _uptimeTimer = null; // 1s local ticker so uptime reads live between 5s polls + let _uptimeBase = 0; // uptime_seconds from the most recent poll + let _uptimeAnchor = 0; // performance.now() when that poll landed + let _samLabel = ''; // cached accelerator label for the local re-render + + function cacheDom() { + if (_dom) return; + _card = document.getElementById('devices-layer-card'); + if (!_card) return; // markup absent -> stay inert + _pill = document.getElementById('devices-layer-pill'); + _meta = document.getElementById('devices-layer-meta'); + _start = document.getElementById('devices-layer-start'); + _stop = document.getElementById('devices-layer-stop'); + _logToggle = document.getElementById('devices-layer-log-toggle'); + _log = document.getElementById('devices-layer-log'); + _hint = document.getElementById('devices-layer-hint'); + _dom = true; + } + + function init() { + cacheDom(); + if (!_card) return; + _start.addEventListener('click', onStart); + _stop.addEventListener('click', onStop); + _logToggle.addEventListener('click', toggleLog); + + // Poll only while the Devices tab is showing. switchTab() emits + // TAB_CHANGED with the *new* tab id (no leave signal), so compare + // against TABS.DEVICES to derive both enter and exit. + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('TAB_CHANGED', (tab) => { + if (tab === TABS.DEVICES) startPoll(); + else stopPoll(); + }); + } + // Seed if Devices is already the active tab (reload / deep link) — + // TAB_CHANGED does not fire for the landing tab. + const content = document.getElementById('devices-content'); + if (content && content.classList.contains('active')) startPoll(); + } + + // ── polling ────────────────────────────────────────────────────────── + + function startPoll() { + loadStatus(); // immediate + if (_timer) clearInterval(_timer); // clear-before-set + _timer = setInterval(loadStatus, _pollMs); + } + + function stopPoll() { + if (_timer) { clearInterval(_timer); _timer = null; } + stopUptimeTicker(); // no local ticking while the tab is hidden + } + + async function loadStatus() { + if (!_card) return; + try { + const res = await fetch('/api/device-layer/status'); + if (!res.ok) return; // keep last-known UI + apply(await res.json()); + } catch (e) { + console.debug('device-layer status poll failed', e); + } + } + + // ── render ─────────────────────────────────────────────────────────── + + function apply(d) { + _card.hidden = false; + const [mod, label] = PILL[d.state] || ['stale', d.state || 'unknown']; + _pill.className = 'devices-status-pill' + (mod ? ' ' + mod : ''); + _pill.textContent = label; + + const booting = d.state === 'starting' || d.state === 'initializing'; + + // Meta line: during boot show the current stage; otherwise uptime + SAM. + if (booting) { + stopUptimeTicker(); + _meta.textContent = (d.progress && d.progress.i && d.progress.n) + ? 'step ' + d.progress.i + '/' + d.progress.n + ' · ' + (d.progress.label || 'starting…') + : 'starting…'; + } else { + // Operator-friendly status line: uptime + the SAM accelerator in + // plain terms. pid, raw port, and "closed" are debug details (in the + // Log / status payload), kept off the glanceable line. + _samLabel = !d.sam_device ? '' + : d.sam_device === 'cuda' ? 'GPU' + : d.sam_device === 'cpu' ? 'CPU' : d.sam_device; + if (typeof d.uptime_seconds === 'number' && d.uptime_seconds > 0) { + // Anchor to the server's value, then tick locally every second so + // uptime counts smoothly instead of stepping by the 5s poll gap. + _uptimeBase = d.uptime_seconds; + _uptimeAnchor = (typeof performance !== 'undefined' ? performance.now() : 0); + renderSteadyMeta(); + startUptimeTicker(); + } else { + _uptimeBase = 0; + stopUptimeTicker(); + renderSteadyMeta(); + } + } + + // Hint line: failure reason, external note, or a reassurance during the + // slow MMCore step (step 2) so a long wait doesn't read as frozen. + if (d.state === 'failed' && d.failure) { + _hint.hidden = false; + const h = (d.failure.hints && d.failure.hints[0]) ? ' — ' + d.failure.hints[0] : ''; + _hint.textContent = (d.failure.summary || 'Startup failed') + h; + } else if (d.state === 'external') { + _hint.hidden = false; + _hint.textContent = 'running externally — not managed by gently'; + } else if (booting && d.progress && d.progress.i === 2) { + _hint.hidden = false; + _hint.textContent = 'Initializing Micro-Manager — this can take a minute.'; + } else { + _hint.hidden = true; + _hint.textContent = ''; + } + + // Enable/disable (a hint only; the server 403/409 is the real gate). + const canStart = d.state === 'stopped' || d.state === 'crashed' || d.state === 'failed'; + const canStop = + d.state === 'ready' || d.state === 'running' || booting; + _start.disabled = _busy || !canStart; + _stop.disabled = _busy || !canStop; + + // Adaptive cadence: poll fast (1s) while booting, calm (5s) otherwise. + const want = booting ? 1000 : 5000; + if (want !== _pollMs && _timer) { + _pollMs = want; + clearInterval(_timer); + _timer = setInterval(loadStatus, _pollMs); + } + + // During boot, auto-surface the trailing init log so the long MMCore + // step visibly progresses ("something is happening") instead of sitting + // on a static hint. Reuses the Log pane; auto-collapses once boot ends — + // unless the user has taken the log's open/closed state into their hands. + if (booting && _log.hidden && !_logUserOverride) { + _log.hidden = false; + _logToggle.setAttribute('aria-expanded', 'true'); + _logToggle.classList.add('active'); + _autoLog = true; + } else if (!booting && _autoLog && !_logUserOverride && !_log.hidden) { + _log.hidden = true; + _logToggle.setAttribute('aria-expanded', 'false'); + _logToggle.classList.remove('active'); + _autoLog = false; + } + + if (Array.isArray(d.log_tail) && d.log_tail.length && !_log.hidden) { + renderLog(d.log_tail); + } + } + + function fmtUptime(s) { + s = Math.max(0, Math.floor(s)); + if (s < 60) return s + 's'; + if (s < 3600) return Math.floor(s / 60) + 'm ' + (s % 60) + 's'; + return Math.floor(s / 3600) + 'h ' + Math.floor((s % 3600) / 60) + 'm'; + } + + // Re-render the steady-state meta from cached values, computing uptime live + // off the last poll's anchor so the local 1s ticker counts smoothly. + function renderSteadyMeta() { + if (!_meta) return; + const bits = []; + if (_uptimeBase > 0) { + const now = (typeof performance !== 'undefined' ? performance.now() : 0); + bits.push('up ' + fmtUptime(_uptimeBase + Math.max(0, now - _uptimeAnchor) / 1000)); + } + if (_samLabel) bits.push('SAM: ' + _samLabel); + _meta.textContent = bits.join(' · '); + } + + function startUptimeTicker() { + if (_uptimeTimer) return; // already ticking + _uptimeTimer = setInterval(renderSteadyMeta, 1000); + } + + function stopUptimeTicker() { + if (_uptimeTimer) { clearInterval(_uptimeTimer); _uptimeTimer = null; } + } + + // ── start / stop ───────────────────────────────────────────────────── + + async function onStart() { + if (_start.disabled) return; + // Fresh boot cycle — let the log pane auto-manage again. + _logUserOverride = false; + _autoLog = false; + _busy = true; + _start.disabled = true; + try { + await postJSON('/api/device-layer/start', {}); + toast('Device layer starting…'); + } catch (e) { + // 403 already surfaced by control-auth.js; don't double-toast. + if (e.status !== 403) toast('Start failed: ' + e.message); + } finally { + _busy = false; + // The layer takes a moment to bind its port; re-poll shortly. + setTimeout(loadStatus, 800); + loadStatus(); + } + } + + async function onStop() { + if (_stop.disabled) return; + if (!window.confirm('Stop the device layer?')) return; + _busy = true; + _stop.disabled = true; + try { + await postJSON('/api/device-layer/stop', { confirm: true }); + toast('Device layer stopping…'); + } catch (e) { + if (e.status === 409 && e.payload && e.payload.blocked) { + // Mid-acquisition soft block — offer an explicit force-stop. + if (window.confirm('A run is active. Force-stop the device layer anyway?')) { + try { + await postJSON('/api/device-layer/stop', { confirm: true, force: true }); + toast('Force-stopping…'); + } catch (e2) { + if (e2.status !== 403) toast('Force-stop failed: ' + e2.message); + } + } + } else if (e.status !== 403) { + toast('Stop failed: ' + e.message); + } + } finally { + _busy = false; + setTimeout(loadStatus, 500); + loadStatus(); + } + } + + // POST JSON; on !ok throw an Error carrying .status and parsed .payload so + // callers can branch on 409/403 (mirrors operate.js's postJSON convention). + async function postJSON(url, body) { + const res = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body || {}), + }); + if (!res.ok) { + let payload = {}; + try { payload = await res.json(); } catch (_) { /* non-JSON */ } + const err = new Error(payload.error || payload.reason || res.statusText); + err.status = res.status; + err.payload = payload; + throw err; + } + return res.json(); + } + + // ── log tail ───────────────────────────────────────────────────────── + + function toggleLog() { + _logUserOverride = true; // user owns the pane's open/closed state now + _autoLog = false; + const show = _log.hidden; + _log.hidden = !show; + _logToggle.setAttribute('aria-expanded', String(show)); + _logToggle.classList.toggle('active', show); + if (show) loadFullLog(); + } + + async function loadFullLog() { + try { + const res = await fetch('/api/device-layer/log?limit=200'); + if (!res.ok) return; + const d = await res.json(); + if (Array.isArray(d.lines)) renderLog(d.lines); + } catch (e) { + console.debug('device-layer log fetch failed', e); + } + } + + function renderLog(lines) { + if (!_log) return; + _log.textContent = lines.join('\n'); + _log.scrollTop = _log.scrollHeight; // pin to newest + } + + function toast(msg) { + if (typeof showGentlyToast === 'function') showGentlyToast(msg); + } + + return { init }; +})(); + +document.addEventListener('DOMContentLoaded', () => DeviceLayerCard.init()); diff --git a/gently/ui/web/static/js/devices.js b/gently/ui/web/static/js/devices.js index 0f2f5316..5bafaa38 100644 --- a/gently/ui/web/static/js/devices.js +++ b/gently/ui/web/static/js/devices.js @@ -14,7 +14,7 @@ */ const DevicesManager = (function () { const STALE_AFTER_MS = 4000; - const VIEWS = ['map', 'details']; + const VIEWS = ['operate', 'map', 'details', 'optical3d', 'manual']; const SVG_NS = 'http://www.w3.org/2000/svg'; // Status / details DOM @@ -32,11 +32,14 @@ const DevicesManager = (function () { let _mapWrap; let _scalebarLabel; - // Embryos overlay state: list of {embryo_id, x, y, role, ...}. - // Populated by /api/embryos/positions on init + EMBRYO_DETECTED / - // STATUS_CHANGED WS pushes thereafter. Roles drive the marker color + // Embryo waypoints — driven by EMBRYOS_UPDATE events (the canonical bulk + // mutation broadcast added by the embryos-broadcast commit) and the + // initial /api/embryos/current snapshot. Each entry mirrors + // EmbryoState.to_dict() (id, position_coarse, position_fine, + // has_fine_position, nickname, role, ...). Role drives marker color // (mirrors the marking-window legend: magenta=test, cyan=calibration, - // grey=unassigned). + // grey=unassigned). EMBRYO_DETECTED / STATUS_CHANGED listeners stay + // hooked as a belt-and-braces refresh path. let _embryos = []; const _ROLE_COLOR = { test: '#ff66cc', @@ -44,8 +47,14 @@ const DevicesManager = (function () { unassigned: '#888888', }; + // Map-side edit state. _selectedEmbryoId means "picked up": the next + // click on empty map space drops it there (with a confirm), Delete / + // Backspace removes it (with a confirm), Escape clears the selection. + let _selectedEmbryoId = null; + // Bottom-camera panel DOM + state - let _camPanel, _camToggle, _camImg, _camPlaceholder, _camLed, _camMeta; + let _camPanel, _camToggle, _camExpand, _camImg, _camPlaceholder, _camLed, _camMeta; + let _camStage, _camCrosshair, _camCrosshairGroup; let _camStreaming = false; let _camLastFrameTs = 0; let _camHasFrame = false; @@ -53,6 +62,83 @@ const DevicesManager = (function () { const _CAM_FPS_WINDOW = 12; let _camFrameTimes = []; + // Camera zoom / pan. Identity transform = (zoom 1, tx 0, ty 0); pan only + // engages once zoom > 1. Reset on double-click and on stream-off. + let _camZoom = 1; + let _camTx = 0; + let _camTy = 0; + let _camPanLast = null; // {x, y} clientX/Y of last pointermove during pan + const _CAM_ZOOM_MIN = 1; + const _CAM_ZOOM_MAX = 8; + const _CAM_ZOOM_STEP = 1.15; // multiplicative per wheel notch + + // Lightsheet live panel DOM + state (Manual view) + let _lsToggle, _lsImg, _lsPlaceholder, _lsLed, _lsMeta, _lsStage; + let _lsStreaming = false; + let _lsLastFrameTs = 0; + let _lsHasFrame = false; + let _lsStaleTimer = null; + const _LS_FPS_WINDOW = 12; + let _lsFrameTimes = []; + // Render throttle: decouple paint rate from frame-arrival rate. We keep + // only the latest frame, coalesce paints to one per animation frame, and + // hold a single decode in flight at a time. This stops a fast stream from + // swapping .src 100+ times/sec, which churns GPU texture uploads and + // can hang an older display driver (Video TDR). See handleLightsheetFrame. + let _lsPendingPayload = null; + let _lsRenderScheduled = false; + let _lsDecoding = false; + + // Lightsheet zoom / pan (mirrors camera zoom/pan) + let _lsZoom = 1; + let _lsTx = 0; + let _lsTy = 0; + let _lsPanLast = null; + + // Lightsheet live params — debounced POST to /api/devices/lightsheet/live/params + let _lsGalvo = 0; + let _lsPiezo = 0; + let _lsExposure = 20; // matches device-layer _ls_params default (20 ms) + let _lsSide = 'A'; // SPIM side — 'A' (HamCam1) or 'B' (HamCam2 if present) + let _lsParamTimer = null; + + // Lightsheet control inputs (rail) + let _lsGalvoSlider, _lsGalvoNum, _lsPiezoSlider, _lsPiezoNum, _lsExposureNum; + let _lsLedToggle, _lsRoomLightBtn; + let _lsLedIsOpen = false; // LED toggle state: false = Closed (safe default) + let _lsLaserToggle; + let _lsLaserOn = false; // Laser toggle state: false = OFF (entry-safe default) + let _lsSnapVolBtn, _lsBurstBtn, _lsLastcap, _lsLastcapRef; + let _lsLaserStatus; // span inside .ls-laser-indicator — driven by actual laser/off calls + let _lsLaserPreset; // — shown only when camera_b present + let _lsTempInput, _lsTempSet; + + // Timelapse form DOM refs (Manual view — #devices-tl-group) + let _tlToggle, _tlBody; + let _tlInterval, _tlStop, _tlCondRow, _tlCondLabel, _tlCondVal; + let _tlEmbryos, _tlMode; + let _tlSlices, _tlExposure, _tlGalvoAmp, _tlGalvoCtr, _tlPiezoAmp, _tlPiezoCtr, _tlLaser; + let _tlStart, _tlStatus, _tlStatusText; + // Accordion active-state per section: { sched, targets, geom } + let _tlTouched = { sched: false, targets: false, geom: false }; + + // Room-light toggle (header). Drives the SwitchBot Bot that switches the + // diSPIM room light. State is the bot's cached on/off; hidden until the + // device layer reports the accessory is configured. + let _roomLightToggle, _roomLightLabel; + let _roomLightState = 'unknown'; + let _roomLightAvailable = false; + let _roomLightBusy = false; + let _roomLightTimer = null; + + // Temperature-controller panel DOM + state + let _tempEl, _tempReadout, _tempInput, _tempSet; + let _tempState = 'unknown'; + let _tempAvailable = false; + let _tempBusy = false; + let _tempTimer = null; + let _lastTs = 0; let _previousTs = 0; let _lastWallTs = 0; @@ -60,7 +146,7 @@ const DevicesManager = (function () { let _filterText = ''; let _lastPropertyMap = {}; let _lastXY = null; // {X, Y} in stage µm, last seen - let _currentView = 'map'; + let _currentView = 'operate'; // Map geometry // _optimalBox: { x: [min, max], y: [min, max] } in stage µm, derived @@ -108,11 +194,69 @@ const DevicesManager = (function () { _camPanel = document.getElementById('devices-camera-panel'); _camToggle = document.getElementById('devices-camera-toggle'); + _camExpand = document.getElementById('devices-camera-expand'); _camImg = document.getElementById('devices-camera-img'); _camPlaceholder = document.getElementById('devices-camera-placeholder'); + _camStage = _camPanel ? _camPanel.querySelector('.devices-camera-stage') : null; + _camCrosshair = document.getElementById('devices-camera-crosshair'); + _camCrosshairGroup = document.getElementById('devices-camera-crosshair-group'); _camLed = document.getElementById('devices-camera-led'); _camMeta = document.getElementById('devices-camera-meta'); + // Manual / lightsheet panel + _lsToggle = document.getElementById('devices-ls-toggle'); + _lsImg = document.getElementById('devices-ls-img'); + _lsPlaceholder = document.getElementById('devices-ls-placeholder'); + _lsStage = document.getElementById('devices-ls-stage'); + _lsLed = document.getElementById('devices-ls-led'); + _lsMeta = document.getElementById('devices-ls-meta'); + _lsGalvoSlider = document.getElementById('devices-ls-galvo-slider'); + _lsGalvoNum = document.getElementById('devices-ls-galvo'); + _lsPiezoSlider = document.getElementById('devices-ls-piezo-slider'); + _lsPiezoNum = document.getElementById('devices-ls-piezo'); + _lsExposureNum = document.getElementById('devices-ls-exposure'); + _lsLedToggle = document.getElementById('devices-ls-led-toggle'); + _lsRoomLightBtn = document.getElementById('devices-ls-room-light-btn'); + _lsLaserToggle = document.getElementById('devices-ls-laser-toggle'); + _lsSnapVolBtn = document.getElementById('devices-ls-snap-volume'); + _lsBurstBtn = document.getElementById('devices-ls-burst'); + _lsLastcap = document.getElementById('devices-ls-lastcap'); + _lsLastcapRef = document.getElementById('devices-ls-lastcap-ref'); + _lsLaserStatus = document.getElementById('devices-ls-laser-status'); + _lsLaserPreset = document.getElementById('devices-laser-preset'); + _lsSideSelect = document.getElementById('devices-ls-side'); + _lsTempInput = document.getElementById('devices-ls-temp-input'); + _lsTempSet = document.getElementById('devices-ls-temp-set'); + + // Timelapse form + _tlToggle = document.getElementById('devices-tl-toggle'); + _tlBody = document.getElementById('devices-tl-body'); + _tlInterval = document.getElementById('devices-tl-interval'); + _tlStop = document.getElementById('devices-tl-stop'); + _tlCondRow = document.getElementById('devices-tl-cond-row'); + _tlCondLabel = document.getElementById('devices-tl-cond-label'); + _tlCondVal = document.getElementById('devices-tl-cond-val'); + _tlEmbryos = document.getElementById('devices-tl-embryos'); + _tlMode = document.getElementById('devices-tl-mode'); + _tlSlices = document.getElementById('devices-tl-slices'); + _tlExposure = document.getElementById('devices-tl-exposure'); + _tlGalvoAmp = document.getElementById('devices-tl-galvo-amp'); + _tlGalvoCtr = document.getElementById('devices-tl-galvo-ctr'); + _tlPiezoAmp = document.getElementById('devices-tl-piezo-amp'); + _tlPiezoCtr = document.getElementById('devices-tl-piezo-ctr'); + _tlLaser = document.getElementById('devices-tl-laser'); + _tlStart = document.getElementById('devices-tl-start'); + _tlStatus = document.getElementById('devices-tl-status'); + _tlStatusText = document.getElementById('devices-tl-status-text'); + + _roomLightToggle = document.getElementById('devices-room-light-toggle'); + _roomLightLabel = document.getElementById('devices-room-light-label'); + + _tempEl = document.getElementById('devices-temp'); + _tempReadout = document.getElementById('devices-temp-readout'); + _tempInput = document.getElementById('devices-temp-input'); + _tempSet = document.getElementById('devices-temp-set'); + // Recompute the scale bar caption whenever the canvas resizes. if (_mapSvg && window.ResizeObserver) { new ResizeObserver(() => updateScalebar()).observe(_mapSvg); @@ -257,6 +401,33 @@ const DevicesManager = (function () { } } + // Initial embryo snapshot — closes the gap for clients that connect + // mid-session, after the last EMBRYOS_UPDATE has already been broadcast + // and aged out of history. Subsequent updates arrive over the event bus. + async function loadEmbryosSnapshot() { + try { + const res = await fetch('/api/embryos/current'); + if (!res.ok) return; + const data = await res.json(); + handleEmbryosUpdate(data); + } catch (err) { + console.debug('embryos snapshot fetch failed:', err); + } + } + + function handleEmbryosUpdate(payload) { + _embryos = (payload && Array.isArray(payload.embryos)) ? payload.embryos : []; + // Recompute the frame so newly-arrived embryos are always in view. + // computeViewBox() returns true when the bounds shifted (incl. the + // first-ever compute); a wider frame needs a full redraw so grid/zones/ + // axes track the new bounds, otherwise just repaint the embryo layer. + if (computeViewBox()) { + renderMap(); + } else { + renderEmbryos(); + } + } + // ===================================================================== // Properties table (Details view) // ===================================================================== @@ -335,6 +506,18 @@ const DevicesManager = (function () { xMin = Math.min(xMin, _lastXY.X); xMax = Math.max(xMax, _lastXY.X); yMin = Math.min(yMin, _lastXY.Y); yMax = Math.max(yMax, _lastXY.Y); } + // Always frame the marked embryos — they can sit well outside the fence + // box / current stage position (e.g. SAM detections hundreds of µm away), + // and omitting them clips them to the map edge (looks like a wrong + // position even though their coords are correct). + if (_embryos && _embryos.length) { + _embryos.forEach(emb => { + const xy = embryoResolvedXY(emb); + if (!xy) return; + xMin = Math.min(xMin, xy.x); xMax = Math.max(xMax, xy.x); + yMin = Math.min(yMin, xy.y); yMax = Math.max(yMax, xy.y); + }); + } if (!isFinite(xMin) || !isFinite(yMin)) { xMin = -100; xMax = 100; yMin = -100; yMax = 100; } @@ -529,9 +712,16 @@ const DevicesManager = (function () { // "Forbidden" is implicit: paint the whole viewport with the red hatch // pattern. The optimal zone rect paints ABOVE this so the operator's safe // window looks carved out of a hatched danger envelope. + // + // Only paint the red envelope when we actually KNOW the safe window + // (_optimalBox, from XYStage fence telemetry). With no working-region data — + // device layer down, disconnected, or pre-calibration — an all-red map reads + // as "everything is off-limits" when the truth is "no position data yet". + // In that case leave a neutral grid instead of an alarming red field. function renderBeyond() { if (!_mapBeyond || !_viewBox) return; _mapBeyond.innerHTML = ''; + if (!_optimalBox) return; // no fence data → neutral empty-state, not red const { xMin, xMax, yMin, yMax } = _viewBox; const span = Math.max(xMax - xMin, yMax - yMin); const tile = Math.max(8, span / 50); @@ -744,6 +934,215 @@ const DevicesManager = (function () { return Math.round(v).toString(); } + // ===================================================================== + // Embryo waypoints + // ===================================================================== + + // "embryo_007" / "embryo_7" -> 7. Falls back to a 1-based index from the + // caller so the label always shows *something*, even for stray ids. + function embryoLabelText(id, fallbackIndex) { + const m = id && String(id).match(/(\d+)/); + if (m) { + const n = parseInt(m[1], 10); + if (Number.isFinite(n)) return String(n); + } + return String(fallbackIndex + 1); + } + + // Resolve XY for rendering — fine if SPIM-aligned, else coarse. Returns + // null when neither stage carries usable values so the entry is skipped + // (e.g. an embryo whose detection record came in malformed). + function embryoResolvedXY(emb) { + const f = emb && emb.position_fine; + if (f && Number.isFinite(f.x) && Number.isFinite(f.y)) return { x: f.x, y: f.y }; + const c = emb && emb.position_coarse; + if (c && Number.isFinite(c.x) && Number.isFinite(c.y)) return { x: c.x, y: c.y }; + return null; + } + + function renderEmbryos() { + if (!_mapEmbryos || !_viewBox) return; + _mapEmbryos.innerHTML = ''; + if (!_embryos || !_embryos.length) return; + const span = Math.max(_viewBox.xMax - _viewBox.xMin, + _viewBox.yMax - _viewBox.yMin); + const radius = span * 0.012; + const fontSize = span * 0.015; + + _embryos.forEach((emb, i) => { + const xy = embryoResolvedXY(emb); + if (!xy) return; + + const isFine = !!emb.has_fine_position; + const isSelected = _selectedEmbryoId !== null + && emb.id === _selectedEmbryoId; + + // Wrap circle + label in a group so a single closest() lookup + // finds the embryo regardless of which child the click hit. + const group = document.createElementNS(SVG_NS, 'g'); + group.setAttribute('class', + 'devices-embryo-group' + (isSelected ? ' devices-embryo-selected' : '')); + group.setAttribute('data-embryo-id', emb.id || ''); + group.setAttribute('data-embryo-stage', isFine ? 'fine' : 'coarse'); + + const circle = document.createElementNS(SVG_NS, 'circle'); + circle.setAttribute('cx', xy.x); + circle.setAttribute('cy', svgY(xy.y)); + circle.setAttribute('r', radius); + circle.setAttribute('class', + isFine ? 'devices-embryo-disc' : 'devices-embryo-ring'); + group.appendChild(circle); + + const label = document.createElementNS(SVG_NS, 'text'); + label.setAttribute('x', xy.x); + label.setAttribute('y', svgY(xy.y)); + label.setAttribute('class', 'devices-embryo-label'); + label.setAttribute('font-size', fontSize); + label.textContent = embryoLabelText(emb.id, i); + group.appendChild(label); + + _mapEmbryos.appendChild(group); + }); + } + + // ---- Map-side edit interactions ------------------------------------ + // Convert a pointer event's client coords into stage µm. SVG y axis is + // positive-down and stage y is positive-up, so the y component is + // negated to match the convention used elsewhere in this module. + function eventToStageXY(event) { + if (!_mapSvg || !_mapSvg.getScreenCTM) return null; + const ctm = _mapSvg.getScreenCTM(); + if (!ctm) return null; + const pt = _mapSvg.createSVGPoint(); + pt.x = event.clientX; + pt.y = event.clientY; + const local = pt.matrixTransform(ctm.inverse()); + return { x: local.x, y: -local.y }; + } + + function findEmbryoIdAt(target) { + if (!target) return null; + const node = target.closest && target.closest('[data-embryo-id]'); + return node ? node.getAttribute('data-embryo-id') : null; + } + + function embryoById(id) { + return _embryos.find(e => e.id === id) || null; + } + + function embryoNumberFor(emb) { + return embryoLabelText(emb.id, _embryos.indexOf(emb)); + } + + function setSelectedEmbryo(id) { + if (_selectedEmbryoId === id) return; + _selectedEmbryoId = id; + renderEmbryos(); + } + + function clearSelection() { + if (_selectedEmbryoId === null) return; + _selectedEmbryoId = null; + renderEmbryos(); + } + + async function attemptMoveSelected(targetStage) { + const id = _selectedEmbryoId; + if (!id) return; + const emb = embryoById(id); + if (!emb) { clearSelection(); return; } + const cur = embryoResolvedXY(emb); + const num = embryoNumberFor(emb); + const oldStr = cur ? `(${cur.x.toFixed(1)}, ${cur.y.toFixed(1)})` : '(unknown)'; + const newStr = `(${targetStage.x.toFixed(1)}, ${targetStage.y.toFixed(1)})`; + if (!window.confirm(`Move embryo ${num} from ${oldStr} to ${newStr}?`)) { + return; // keep the embryo picked up so they can try again + } + try { + const res = await fetch(`/api/embryos/${encodeURIComponent(id)}/position`, { + method: 'PUT', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ x: targetStage.x, y: targetStage.y }), + }); + if (!res.ok) { + window.alert(`Move failed (${res.status}): ${await res.text()}`); + return; + } + // EMBRYOS_UPDATE will arrive over the bus and refresh the layer; + // dropping clears the picked-up state regardless. + clearSelection(); + } catch (err) { + console.error('move embryo:', err); + window.alert(`Move failed: ${err.message}`); + } + } + + async function attemptDeleteSelected() { + const id = _selectedEmbryoId; + if (!id) return; + const emb = embryoById(id); + const num = emb ? embryoNumberFor(emb) : id; + if (!window.confirm(`Remove embryo ${num}?`)) return; + try { + const res = await fetch(`/api/embryos/${encodeURIComponent(id)}`, { + method: 'DELETE', + }); + if (!res.ok) { + window.alert(`Delete failed (${res.status}): ${await res.text()}`); + return; + } + // The embryo is gone from the server snapshot; EMBRYOS_UPDATE + // will arrive and drop it from _embryos. Clear locally too. + _selectedEmbryoId = null; + } catch (err) { + console.error('delete embryo:', err); + window.alert(`Delete failed: ${err.message}`); + } + } + + function onMapPointerDown(event) { + // Ignore non-primary buttons so right-clicks etc. don't trigger UI. + if (event.button !== undefined && event.button !== 0) return; + const id = findEmbryoIdAt(event.target); + if (id) { + setSelectedEmbryo(id); + return; + } + // Empty-space click: drop the picked-up embryo here. + if (_selectedEmbryoId !== null) { + const stage = eventToStageXY(event); + if (stage) attemptMoveSelected(stage); + } + } + + function onMapKeyDown(event) { + // Only honour keys when the operator is actually looking at the Map: + // not on another top-level tab, not on the Details subview, and not + // typing into an input / textarea / select / contenteditable. + if (typeof state !== 'undefined' && typeof TABS !== 'undefined' + && state.tab !== TABS.DEVICES) { + return; + } + if (_currentView !== 'map') return; + const a = document.activeElement; + if (a && (a.tagName === 'INPUT' || a.tagName === 'TEXTAREA' || + a.tagName === 'SELECT' || a.isContentEditable)) { + return; + } + if (event.key === 'Escape') { + if (_selectedEmbryoId !== null) { + clearSelection(); + event.preventDefault(); + } + return; + } + if (_selectedEmbryoId === null) return; + if (event.key === 'Delete' || event.key === 'Backspace') { + event.preventDefault(); // Backspace would otherwise navigate back + attemptDeleteSelected(); + } + } + function updateMapMarker() { if (!_mapMarker || !_lastXY) return; const sx = _lastXY.X; @@ -820,6 +1219,9 @@ const DevicesManager = (function () { if (_camPlaceholder) _camPlaceholder.hidden = false; if (_camMeta) _camMeta.textContent = 'stream off'; if (_camStaleTimer) { clearTimeout(_camStaleTimer); _camStaleTimer = null; } + // Operator may have zoomed in; reset so the next stream session + // starts at 1× rather than inheriting a stale view. + resetCameraZoom(); } else { _camFrameTimes = []; if (_camMeta) _camMeta.textContent = 'waiting…'; @@ -881,13 +1283,1131 @@ const DevicesManager = (function () { } } + // ---- Camera zoom / pan --------------------------------------------- + function applyCameraTransform() { + if (!_camImg) return; + _camImg.style.transform = + `translate(${_camTx}px, ${_camTy}px) scale(${_camZoom})`; + // Reticle uses an SVG transform attribute on the inner instead + // of a CSS transform on the SVG element — same geometric effect, + // but the SVG renderer re-rasterises at the new zoom so the 1px + // strokes stay crisp instead of getting bitmap-scaled. + if (_camCrosshairGroup && _camStage) { + const rect = _camStage.getBoundingClientRect(); + // Convert pixel-space translation to viewBox units (viewBox is + // 0..100 in both axes, preserveAspectRatio=none). + const txV = rect.width > 0 ? (_camTx * 100) / rect.width : 0; + const tyV = rect.height > 0 ? (_camTy * 100) / rect.height : 0; + // translate(50+tx, 50+ty) scale(zoom) translate(-50, -50) keeps + // the viewBox centre (50, 50) as the zoom anchor and offsets by + // the converted pixel translation. + _camCrosshairGroup.setAttribute( + 'transform', + `translate(${50 + txV} ${50 + tyV}) ` + + `scale(${_camZoom}) ` + + `translate(-50 -50)` + ); + } + } + + function resetCameraZoom() { + _camZoom = 1; + _camTx = 0; + _camTy = 0; + applyCameraTransform(); + if (_camStage) _camStage.classList.remove('camera-zoomed', 'camera-panning'); + } + + // Keep at least the image centre within the visible window so the + // operator can't accidentally pan the entire frame off-screen. At + // zoom 1 this collapses to (0, 0). + function clampCameraPan() { + if (!_camStage) return; + const rect = _camStage.getBoundingClientRect(); + const maxX = (rect.width * (_camZoom - 1)) / 2; + const maxY = (rect.height * (_camZoom - 1)) / 2; + _camTx = Math.max(-maxX, Math.min(maxX, _camTx)); + _camTy = Math.max(-maxY, Math.min(maxY, _camTy)); + } + + function onCameraWheel(event) { + if (!_camStage) return; + // Always preventDefault so the page doesn't scroll under the + // operator while they're framing a sample. + event.preventDefault(); + const rect = _camStage.getBoundingClientRect(); + const cx = event.clientX - rect.left - rect.width / 2; + const cy = event.clientY - rect.top - rect.height / 2; + const oldZoom = _camZoom; + const factor = event.deltaY < 0 ? _CAM_ZOOM_STEP : 1 / _CAM_ZOOM_STEP; + const newZoom = Math.max(_CAM_ZOOM_MIN, + Math.min(_CAM_ZOOM_MAX, oldZoom * factor)); + if (newZoom === oldZoom) return; + + // Keep the image point under the cursor anchored under the cursor + // across the zoom: cursor_new = cursor_old after the transform + // change, which means newT = cursor - (cursor - oldT) * (new/old). + const ratio = newZoom / oldZoom; + _camTx = cx - (cx - _camTx) * ratio; + _camTy = cy - (cy - _camTy) * ratio; + _camZoom = newZoom; + + if (Math.abs(_camZoom - 1) < 0.001) { + resetCameraZoom(); + return; + } + clampCameraPan(); + applyCameraTransform(); + _camStage.classList.add('camera-zoomed'); + } + + function onCameraPointerDown(event) { + if (event.button !== 0) return; + if (_camZoom <= 1) return; + _camPanLast = { x: event.clientX, y: event.clientY }; + try { _camStage.setPointerCapture(event.pointerId); } catch (_) {} + _camStage.classList.add('camera-panning'); + event.preventDefault(); + } + + function onCameraPointerMove(event) { + if (!_camPanLast) return; + _camTx += event.clientX - _camPanLast.x; + _camTy += event.clientY - _camPanLast.y; + _camPanLast = { x: event.clientX, y: event.clientY }; + clampCameraPan(); + applyCameraTransform(); + } + + function onCameraPointerEnd(event) { + if (!_camPanLast) return; + _camPanLast = null; + try { _camStage.releasePointerCapture(event.pointerId); } catch (_) {} + if (_camStage) _camStage.classList.remove('camera-panning'); + } + + function onCameraDoubleClick(event) { + if (_camZoom !== 1 || _camTx !== 0 || _camTy !== 0) { + event.preventDefault(); + resetCameraZoom(); + } + } + + function toggleCameraExpand() { + if (!_camPanel) return; + const expanded = _camPanel.classList.toggle('expanded'); + if (_camExpand) { + _camExpand.classList.toggle('active', expanded); + _camExpand.setAttribute('aria-pressed', expanded ? 'true' : 'false'); + _camExpand.title = expanded ? 'Shrink view' : 'Enlarge view'; + _camExpand.textContent = expanded ? '⤡' : '⤢'; + } + } + function setupCameraWiring() { if (!_camToggle) return; _camToggle.addEventListener('click', toggleCameraStream); + if (_camExpand) _camExpand.addEventListener('click', toggleCameraExpand); applyCameraState(false); if (typeof ClientEventBus !== 'undefined') { ClientEventBus.on('BOTTOM_CAMERA_FRAME', handleCameraFrame); } + // Camera zoom/pan. wheel needs passive:false so we can preventDefault + // and stop the page from scrolling beneath the FOV. + if (_camStage) { + _camStage.addEventListener('wheel', onCameraWheel, { passive: false }); + _camStage.addEventListener('pointerdown', onCameraPointerDown); + _camStage.addEventListener('pointermove', onCameraPointerMove); + _camStage.addEventListener('pointerup', onCameraPointerEnd); + _camStage.addEventListener('pointercancel', onCameraPointerEnd); + _camStage.addEventListener('dblclick', onCameraDoubleClick); + } + } + + // ===================================================================== + // Lightsheet live panel (Manual view) + // ===================================================================== + + /** Gate ALL lasers off via the Laser "ALL OFF" config-group preset. + * Updates the indicator span from the actual API result (not a static label). + * Fire-and-forget safe — failure shows a warning, never throws. */ + async function setLaserOff() { + cacheDom(); + try { + const res = await fetch('/api/devices/laser/off', { method: 'POST' }); + if (_lsLaserStatus) { + _lsLaserStatus.textContent = res.ok ? 'OFF (brightfield)' : 'warning: state unknown'; + } + if (res.ok) _setLaserToggleState(false); + } catch (err) { + if (_lsLaserStatus) _lsLaserStatus.textContent = 'warning: state unknown'; + console.debug('laser off call failed:', err); + } + } + + /** Fetch laser config-group presets and populate the #devices-laser-preset select. + * Selects "ALL OFF" by default (entry safety preset). + * Wires the change handler to POST the selected preset. + * Fire-and-forget safe — failure leaves the fallback "ALL OFF" option in place. */ + async function populateLaserPresets() { + cacheDom(); + if (!_lsLaserPreset) return; + try { + const res = await fetch('/api/devices/laser/configs'); + if (!res.ok) return; + const data = await res.json(); + // data may be an array of preset names or {configs: [...]} + const presets = Array.isArray(data) ? data : (data.configs || []); + if (!presets.length) return; + // Rebuild option list + _lsLaserPreset.innerHTML = ''; + for (const name of presets) { + const opt = document.createElement('option'); + opt.value = name; + opt.textContent = name; + _lsLaserPreset.appendChild(opt); + } + // Default to "ALL OFF" — entry safety state + if (presets.includes('ALL OFF')) _lsLaserPreset.value = 'ALL OFF'; + // Wire change handler — only POST if laser is currently ON; if OFF, it's + // just a selection that will be activated when the toggle is pressed. + _lsLaserPreset.onchange = () => { if (_lsLaserOn) setLaserPreset(_lsLaserPreset.value); }; + } catch (err) { + console.debug('laser preset fetch failed:', err); + } + } + + /** Fetch SPIM camera roles and show the Side A/B selector if camera_b is present. + * Called on manual-view entry. Hides the selector on single-camera rigs. + * Fire-and-forget safe — failure leaves the selector hidden (safe default). */ + async function populateCameraRoles() { + cacheDom(); + const group = document.getElementById('devices-ls-side-group'); + try { + const res = await fetch('/api/devices/cameras'); + if (!res.ok) return; + const data = await res.json(); + // data may be {cameras: [...]} or a raw array + const cameras = Array.isArray(data) ? data : (data.cameras || []); + const hasSideB = cameras.includes('B'); + if (group) group.style.display = hasSideB ? '' : 'none'; + if (_lsSideSelect && hasSideB) { + _lsSideSelect.onchange = () => { + _lsSide = _lsSideSelect.value; + postLightsheetParams(); + }; + } + } catch (err) { + console.debug('camera roles fetch failed:', err); + } + } + + /** POST a named laser preset to the device layer. + * Updates the status indicator on success. + * Fire-and-forget safe — failure shows a warning, never throws. */ + async function setLaserPreset(config) { + cacheDom(); + if (!config) return; + try { + const res = await fetch('/api/devices/laser/config', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ config }), + }); + if (_lsLaserStatus) { + _lsLaserStatus.textContent = res.ok ? config : 'warning: state unknown'; + } + if (res.ok) _setLaserToggleState(config !== 'ALL OFF'); + if (!res.ok) console.debug('laser preset set failed:', await res.text()); + } catch (err) { + if (_lsLaserStatus) _lsLaserStatus.textContent = 'warning: state unknown'; + console.debug('laser preset set failed:', err); + } + } + + // ===================================================================== + // Timelapse config form (Manual view) + // ===================================================================== + + // ── Accordion section summary builders ─────────────────────────────────── + + function _tlSchedSummary() { + const interval = (_tlInterval && _tlInterval.value) ? _tlInterval.value : '120'; + const stop = (_tlStop && _tlStop.value) ? _tlStop.value : 'manual'; + const condVal = (_tlCondVal && _tlCondVal.value) ? _tlCondVal.value : '10'; + if (stop === 'timepoints') return `${interval} s · ${condVal} frames`; + if (stop === 'duration') return `${interval} s · ${condVal} h`; + return `${interval} s · manual`; + } + + function _tlTargetsSummary() { + const embryos = (_tlEmbryos && _tlEmbryos.value.trim()) + ? _tlEmbryos.value.trim() + : 'all'; + const modeEl = _tlMode; + const modeText = (modeEl && modeEl.value) + ? modeEl.options[modeEl.selectedIndex].text + : 'none'; + return `${embryos} · ${modeText}`; + } + + function _tlGeomSummary() { + const slices = (_tlSlices && _tlSlices.value) ? _tlSlices.value : '50'; + const exposure = (_tlExposure && _tlExposure.value) ? _tlExposure.value : '10'; + const laser = (_tlLaser && _tlLaser.value) ? _tlLaser.value : 'ALL OFF'; + return `${slices} sl · ${exposure} ms · ${laser}`; + } + + /** Update a section's header active state and summary text, then sync the + * outer panel dot and the start button. sec = 'sched'|'targets'|'geom'. */ + function _tlUpdateSection(sec) { + const head = document.getElementById(`devices-tlacc-${sec}-head`); + const summary = document.getElementById(`devices-tlacc-${sec}-sum`); + const touched = _tlTouched[sec]; + + if (head) head.classList.toggle('is-active', touched); + if (summary) { + summary.hidden = !touched; + if (touched) { + if (sec === 'sched') summary.textContent = _tlSchedSummary(); + else if (sec === 'targets') summary.textContent = _tlTargetsSummary(); + else if (sec === 'geom') summary.textContent = _tlGeomSummary(); + } + } + + // Outer panel dot + start button "ready" state + const anyActive = Object.values(_tlTouched).some(Boolean); + const outerDot = document.getElementById('devices-tl-outer-dot'); + if (outerDot) outerDot.classList.toggle('is-active', anyActive); + if (_tlStart) _tlStart.classList.toggle('is-ready', anyActive); + } + + /** Wire the timelapse panel: outer collapsible toggle, accordion section + * toggles, touch listeners, and the submit button. + * Safe to call multiple times (re-assigns handlers idempotently). */ + function initTlForm() { + cacheDom(); + + // Reset touched state on each init (re-entering the manual view = fresh) + _tlTouched = { sched: false, targets: false, geom: false }; + // Clear any leftover active-state visuals from a previous visit + ['sched', 'targets', 'geom'].forEach(sec => _tlUpdateSection(sec)); + + // ── Outer collapsible toggle ────────────────────────────────────────── + if (_tlToggle && _tlBody) { + _tlToggle.onclick = () => { + const open = _tlBody.hidden; + _tlBody.hidden = !open; + _tlToggle.setAttribute('aria-expanded', String(open)); + const arrow = _tlToggle.querySelector('.ls-collapsible-arrow'); + if (arrow) arrow.textContent = open ? '▼' : '▶'; + }; + } + + // ── Accordion section toggles ───────────────────────────────────────── + ['sched', 'targets', 'geom'].forEach(sec => { + const head = document.getElementById(`devices-tlacc-${sec}-head`); + const body = document.getElementById(`devices-tlacc-${sec}-body`); + if (!head || !body) return; + head.onclick = () => { + const open = body.hidden; + body.hidden = !open; + head.setAttribute('aria-expanded', String(open)); + const arrow = head.querySelector('.ls-acc-arrow'); + if (arrow) arrow.textContent = open ? '▼' : '▶'; + }; + }); + + // ── Touch listeners ─────────────────────────────────────────────────── + const markTouched = sec => { + _tlTouched[sec] = true; + _tlUpdateSection(sec); + }; + + // Schedule — interval and stop condition drive summary; cond-row visibility unchanged + [_tlInterval, _tlCondVal].forEach(el => { + if (el) el.addEventListener('input', () => markTouched('sched')); + }); + if (_tlStop) { + _tlStop.addEventListener('change', () => { + const v = _tlStop.value; + const show = v === 'timepoints' || v === 'duration'; + if (_tlCondRow) _tlCondRow.hidden = !show; + if (_tlCondLabel) _tlCondLabel.textContent = v === 'duration' ? 'Hours' : 'Count'; + markTouched('sched'); + }); + } + + // Targets + if (_tlEmbryos) _tlEmbryos.addEventListener('input', () => markTouched('targets')); + if (_tlMode) _tlMode.addEventListener('change', () => markTouched('targets')); + + // Volume geometry + [_tlSlices, _tlExposure, _tlGalvoAmp, _tlGalvoCtr, _tlPiezoAmp, _tlPiezoCtr].forEach(el => { + if (el) el.addEventListener('input', () => markTouched('geom')); + }); + if (_tlLaser) _tlLaser.addEventListener('change', () => markTouched('geom')); + + // ── Submit ──────────────────────────────────────────────────────────── + if (_tlStart) _tlStart.onclick = startTimelapse; + } + + /** Populate timelapse volume-geometry defaults from GET /api/devices/scan_geometry, + * and populate the laser preset select from GET /api/devices/laser/configs. + * Fire-and-forget safe — failure leaves form-coded defaults in place. */ + async function populateTlDefaults() { + cacheDom(); + // Geometry defaults + try { + const res = await fetch('/api/devices/scan_geometry'); + if (res.ok) { + const data = await res.json(); + const scan = data.scan || {}; + if (_tlSlices && scan.num_slices != null) _tlSlices.value = scan.num_slices; + if (_tlExposure && scan.exposure_ms != null) _tlExposure.value = scan.exposure_ms; + if (_tlGalvoAmp && scan.galvo_amplitude_deg != null) _tlGalvoAmp.value = scan.galvo_amplitude_deg; + if (_tlGalvoCtr && scan.galvo_center_deg != null) _tlGalvoCtr.value = scan.galvo_center_deg; + if (_tlPiezoAmp && scan.piezo_amplitude_um != null) _tlPiezoAmp.value = scan.piezo_amplitude_um; + if (_tlPiezoCtr && scan.piezo_center_um != null) _tlPiezoCtr.value = scan.piezo_center_um; + } + } catch (err) { + console.debug('tl scan_geometry fetch failed:', err); + } + // Laser presets — reuse the shared endpoint; mirror populateLaserPresets() + if (!_tlLaser) return; + try { + const res = await fetch('/api/devices/laser/configs'); + if (!res.ok) return; + const data = await res.json(); + const presets = Array.isArray(data) ? data : (data.configs || []); + if (!presets.length) return; + _tlLaser.innerHTML = ''; + for (const name of presets) { + const opt = document.createElement('option'); + opt.value = name; + opt.textContent = name; + _tlLaser.appendChild(opt); + } + if (presets.includes('ALL OFF')) _tlLaser.value = 'ALL OFF'; + } catch (err) { + console.debug('tl laser configs fetch failed:', err); + } + } + + /** Gather form values, POST to /api/devices/timelapse/start, show result. */ + async function startTimelapse() { + cacheDom(); + if (!_tlStart) return; + _tlStart.disabled = true; + + // Build payload + const interval = parseFloat(_tlInterval ? _tlInterval.value : '120') || 120; + const stop_condition = _tlStop ? _tlStop.value : 'manual'; + const condRaw = _tlCondVal ? _tlCondVal.value : ''; + const condition_value = condRaw ? parseInt(condRaw, 10) : null; + const embryoRaw = _tlEmbryos ? _tlEmbryos.value.trim() : ''; + const embryo_ids = embryoRaw + ? embryoRaw.split(',').map(s => s.trim()).filter(Boolean) + : null; + const monitoring_mode = _tlMode ? (_tlMode.value || null) : null; + + const payload = { + interval_seconds: interval, + stop_condition, + embryo_ids, + condition_value, + monitoring_mode, + num_slices: _tlSlices ? parseInt(_tlSlices.value, 10) : 50, + exposure_ms: _tlExposure ? parseFloat(_tlExposure.value) : 10.0, + galvo_amplitude: _tlGalvoAmp ? parseFloat(_tlGalvoAmp.value) : 0.5, + galvo_center: _tlGalvoCtr ? parseFloat(_tlGalvoCtr.value) : 0.0, + piezo_amplitude: _tlPiezoAmp ? parseFloat(_tlPiezoAmp.value) : 25.0, + piezo_center: _tlPiezoCtr ? parseFloat(_tlPiezoCtr.value) : 50.0, + laser_config: _tlLaser ? (_tlLaser.value || null) : null, + }; + + if (_tlStatus) _tlStatus.hidden = false; + if (_tlStatusText) _tlStatusText.textContent = 'Starting…'; + + try { + const res = await fetch('/api/devices/timelapse/start', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(payload), + }); + const body = await res.json().catch(() => ({})); + if (res.ok) { + const msg = body.result || 'Timelapse started.'; + if (_tlStatusText) _tlStatusText.textContent = msg; + } else { + const detail = body.detail || `error ${res.status}`; + if (_tlStatusText) _tlStatusText.textContent = `Error: ${detail}`; + console.debug('timelapse start failed:', body); + } + } catch (err) { + if (_tlStatusText) _tlStatusText.textContent = `Network error: ${err.message}`; + console.debug('timelapse start failed:', err); + } finally { + if (_tlStart) _tlStart.disabled = false; + } + } + + async function toggleLightsheetStream() { + if (!_lsToggle) return; + _lsToggle.disabled = true; + try { + const starting = !_lsStreaming; + const endpoint = _lsStreaming + ? '/api/devices/lightsheet/live/stop' + : '/api/devices/lightsheet/live/start'; + const res = await fetch(endpoint, { method: 'POST' }); + if (!res.ok) { + const detail = await res.text(); + console.error('Lightsheet toggle failed:', detail); + if (_lsMeta) _lsMeta.textContent = `error: ${res.status}`; + return; + } + const data = await res.json(); + applyLightsheetState(!!data.streaming); + // Gate lasers off whenever live starts — brightfield-safe by default. + if (starting && data.streaming) setLaserOff(); + } catch (err) { + console.error('Lightsheet toggle failed:', err); + if (_lsMeta) _lsMeta.textContent = `error: ${err}`; + } finally { + _lsToggle.disabled = false; + } + } + + function applyLightsheetState(streaming) { + _lsStreaming = streaming; + if (_lsToggle) { + // Constant "Live" label; the .active class + status dot show whether + // the stream is currently running (muted = off, green glow = live). + _lsToggle.textContent = 'Live'; + _lsToggle.classList.toggle('active', streaming); + } + if (_lsLed) { + _lsLed.classList.toggle('live', streaming); + _lsLed.classList.remove('stale'); + } + if (!streaming) { + _lsHasFrame = false; + _lsFrameTimes = []; + _lsPendingPayload = null; + _lsRenderScheduled = false; + if (_lsImg) _lsImg.classList.remove('has-frame'); + if (_lsPlaceholder) _lsPlaceholder.hidden = false; + if (_lsMeta) _lsMeta.textContent = 'stream off'; + if (_lsStaleTimer) { clearTimeout(_lsStaleTimer); _lsStaleTimer = null; } + resetLightsheetZoom(); + } else { + _lsFrameTimes = []; + if (_lsMeta) _lsMeta.textContent = 'waiting…'; + } + } + + function handleLightsheetFrame(payload) { + if (!payload || !payload.jpeg_b64 || !_lsImg) return; + + // Lightweight bookkeeping runs per arriving frame so the FPS / live + // indicator reflects the true incoming rate. + const now = performance.now(); + _lsLastFrameTs = Date.now(); + _lsFrameTimes.push(now); + if (_lsFrameTimes.length > _LS_FPS_WINDOW) _lsFrameTimes.shift(); + if (_lsLed) { + _lsLed.classList.add('live'); + _lsLed.classList.remove('stale'); + } + if (_lsMeta) { + const shape = payload.shape || []; + const dims = shape.length === 2 ? `${shape[1]}×${shape[0]}` : ''; + const fps = computeLightsheetFps(); + _lsMeta.textContent = dims + ? `${dims} · ${fps != null ? fps.toFixed(1) + ' fps' : '…'}` + : (fps != null ? `${fps.toFixed(1)} fps` : 'live'); + } + scheduleLightsheetStaleCheck(); + + // Expensive path (decode + GPU texture upload) is throttled: keep only + // the newest frame and coalesce paints to one per animation frame. + _lsPendingPayload = payload; + if (!_lsRenderScheduled) { + _lsRenderScheduled = true; + requestAnimationFrame(renderLightsheetFrame); + } + } + + function renderLightsheetFrame() { + _lsRenderScheduled = false; + // One decode in flight at a time; a frame mid-decode means we skip this + // paint and let the decode's completion reschedule if newer data exists. + if (_lsDecoding) return; + const payload = _lsPendingPayload; + _lsPendingPayload = null; + if (!payload || !_lsImg) return; + + _lsDecoding = true; + _lsImg.src = `data:${payload.mime || 'image/jpeg'};base64,${payload.jpeg_b64}`; + + const done = () => { + _lsDecoding = false; + if (!_lsHasFrame) { + _lsHasFrame = true; + _lsImg.classList.add('has-frame'); + if (_lsPlaceholder) _lsPlaceholder.hidden = true; + } + // A newer frame may have landed during decode — paint it next frame. + if (_lsPendingPayload && !_lsRenderScheduled) { + _lsRenderScheduled = true; + requestAnimationFrame(renderLightsheetFrame); + } + }; + + // img.decode() resolves once the bitmap is ready (off the main thread), + // giving real backpressure. Fall back to a direct apply if unsupported. + if (typeof _lsImg.decode === 'function') { + _lsImg.decode().then(done).catch(done); + } else { + done(); + } + } + + function computeLightsheetFps() { + const n = _lsFrameTimes.length; + if (n < 2) return null; + const span = _lsFrameTimes[n - 1] - _lsFrameTimes[0]; + if (span <= 0) return null; + return ((n - 1) * 1000) / span; + } + + function scheduleLightsheetStaleCheck() { + if (_lsStaleTimer) clearTimeout(_lsStaleTimer); + _lsStaleTimer = setTimeout(() => { + const age = (Date.now() - _lsLastFrameTs) / 1000; + if (_lsMeta) _lsMeta.textContent = `last frame ${age.toFixed(1)}s ago`; + if (_lsLed) _lsLed.classList.add('stale'); + }, 1500); + } + + async function syncInitialLightsheetState() { + try { + const res = await fetch('/api/devices/lightsheet/live/status'); + if (!res.ok) return; + const data = await res.json(); + applyLightsheetState(!!data.streaming); + } catch (err) { + console.debug('lightsheet status check failed:', err); + } + } + + // ---- Lightsheet zoom / pan (mirrors camera zoom/pan) ---------------- + function applyLightsheetTransform() { + if (!_lsImg) return; + _lsImg.style.transform = + `translate(${_lsTx}px, ${_lsTy}px) scale(${_lsZoom})`; + } + + function resetLightsheetZoom() { + _lsZoom = 1; + _lsTx = 0; + _lsTy = 0; + applyLightsheetTransform(); + if (_lsStage) _lsStage.classList.remove('camera-zoomed', 'camera-panning'); + } + + function clampLightsheetPan() { + if (!_lsStage) return; + const rect = _lsStage.getBoundingClientRect(); + const maxX = (rect.width * (_lsZoom - 1)) / 2; + const maxY = (rect.height * (_lsZoom - 1)) / 2; + _lsTx = Math.max(-maxX, Math.min(maxX, _lsTx)); + _lsTy = Math.max(-maxY, Math.min(maxY, _lsTy)); + } + + function onLightsheetWheel(event) { + if (!_lsStage) return; + event.preventDefault(); + const rect = _lsStage.getBoundingClientRect(); + const cx = event.clientX - rect.left - rect.width / 2; + const cy = event.clientY - rect.top - rect.height / 2; + const oldZoom = _lsZoom; + const factor = event.deltaY < 0 ? _CAM_ZOOM_STEP : 1 / _CAM_ZOOM_STEP; + const newZoom = Math.max(_CAM_ZOOM_MIN, Math.min(_CAM_ZOOM_MAX, oldZoom * factor)); + if (newZoom === oldZoom) return; + const ratio = newZoom / oldZoom; + _lsTx = cx - (cx - _lsTx) * ratio; + _lsTy = cy - (cy - _lsTy) * ratio; + _lsZoom = newZoom; + if (Math.abs(_lsZoom - 1) < 0.001) { resetLightsheetZoom(); return; } + clampLightsheetPan(); + applyLightsheetTransform(); + _lsStage.classList.add('camera-zoomed'); + } + + function onLightsheetPointerDown(event) { + if (event.button !== 0) return; + if (_lsZoom <= 1) return; + _lsPanLast = { x: event.clientX, y: event.clientY }; + try { _lsStage.setPointerCapture(event.pointerId); } catch (_) {} + _lsStage.classList.add('camera-panning'); + event.preventDefault(); + } + + function onLightsheetPointerMove(event) { + if (!_lsPanLast) return; + _lsTx += event.clientX - _lsPanLast.x; + _lsTy += event.clientY - _lsPanLast.y; + _lsPanLast = { x: event.clientX, y: event.clientY }; + clampLightsheetPan(); + applyLightsheetTransform(); + } + + function onLightsheetPointerEnd(event) { + if (!_lsPanLast) return; + _lsPanLast = null; + try { _lsStage.releasePointerCapture(event.pointerId); } catch (_) {} + if (_lsStage) _lsStage.classList.remove('camera-panning'); + } + + function onLightsheetDoubleClick(event) { + if (_lsZoom !== 1 || _lsTx !== 0 || _lsTy !== 0) { + event.preventDefault(); + resetLightsheetZoom(); + } + } + + // ---- Lightsheet live params (debounced) ----------------------------- + function postLightsheetParams() { + fetch('/api/devices/lightsheet/live/params', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ galvo: _lsGalvo, piezo: _lsPiezo, exposure: _lsExposure, side: _lsSide }), + }).catch(err => console.debug('lightsheet params post failed:', err)); + } + + function scheduleLightsheetParamPost() { + if (_lsParamTimer) clearTimeout(_lsParamTimer); + _lsParamTimer = setTimeout(postLightsheetParams, 120); + } + + function onGalvoInput(src) { + const v = parseFloat(src.value); + if (isNaN(v)) return; + _lsGalvo = v; + // Sync the sibling control + if (src === _lsGalvoSlider && _lsGalvoNum) _lsGalvoNum.value = v; + if (src === _lsGalvoNum && _lsGalvoSlider) _lsGalvoSlider.value = v; + scheduleLightsheetParamPost(); + } + + function onPiezoInput(src) { + const v = parseFloat(src.value); + if (isNaN(v)) return; + _lsPiezo = v; + if (src === _lsPiezoSlider && _lsPiezoNum) _lsPiezoNum.value = v; + if (src === _lsPiezoNum && _lsPiezoSlider) _lsPiezoSlider.value = v; + scheduleLightsheetParamPost(); + } + + function onExposureInput() { + const v = parseFloat(_lsExposureNum && _lsExposureNum.value); + if (isNaN(v) || v < 1) return; + _lsExposure = v; + scheduleLightsheetParamPost(); + } + + // ---- Illumination toggles ------------------------------------------- + async function postLedPreset(preset) { + try { + await fetch('/api/devices/led/set', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ state: preset }), + }); + } catch (err) { console.debug('LED preset failed:', err); } + } + + /** Single LED toggle — mirrors Cam LED / Room Light aria-pressed pattern. + * Flips between Open (active) and Closed (inactive/safe default). */ + async function toggleLedPreset() { + _lsLedIsOpen = !_lsLedIsOpen; + if (_lsLedToggle) { + _lsLedToggle.classList.toggle('ls-illum-btn--active', _lsLedIsOpen); + _lsLedToggle.setAttribute('aria-pressed', _lsLedIsOpen ? 'true' : 'false'); + _lsLedToggle.textContent = _lsLedIsOpen ? 'LED: Open' : 'LED: Closed'; + } + await postLedPreset(_lsLedIsOpen ? 'Open' : 'Closed'); + } + + /** Update laser toggle button + dot to reflect on/off state. + * Called by setLaserOff() and setLaserPreset() after a successful API call. */ + function _setLaserToggleState(on) { + _lsLaserOn = on; + if (_lsLaserToggle) { + _lsLaserToggle.classList.toggle('ls-illum-btn--active', on); + _lsLaserToggle.setAttribute('aria-pressed', on ? 'true' : 'false'); + _lsLaserToggle.textContent = on ? 'Laser: ON' : 'Laser: OFF'; + } + const dot = document.querySelector('.ls-laser-dot'); + if (dot) dot.classList.toggle('ls-laser-dot--on', on); + } + + /** Laser on/off toggle — OFF fires laser/off; ON applies the selected preset. + * If selected preset is "ALL OFF", picks the first non-"ALL OFF" option. + * Entry safety: starts OFF (setLaserOff fires on manual-view entry). */ + async function toggleLaser() { + if (_lsLaserOn) { + await setLaserOff(); + } else { + let config = _lsLaserPreset ? _lsLaserPreset.value : null; + if (!config || config === 'ALL OFF') { + const opts = _lsLaserPreset ? Array.from(_lsLaserPreset.options) : []; + const first = opts.find(o => o.value !== 'ALL OFF'); + if (first) { + config = first.value; + _lsLaserPreset.value = config; + } else { + if (_lsLaserStatus) _lsLaserStatus.textContent = 'select a laser line first'; + return; + } + } + await setLaserPreset(config); + } + } + + async function toggleManualRoomLight() { + const nextState = _roomLightState === 'on' ? 'off' : 'on'; + if (_lsRoomLightBtn) { + _lsRoomLightBtn.classList.toggle('ls-illum-btn--active', nextState === 'on'); + _lsRoomLightBtn.setAttribute('aria-pressed', nextState === 'on' ? 'true' : 'false'); + } + try { + const res = await fetch('/api/devices/room_light/set', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ state: nextState }), + }); + if (res.ok) { + const data = await res.json(); + _roomLightState = data.state || nextState; + if (_lsRoomLightBtn) { + const on = _roomLightState === 'on'; + _lsRoomLightBtn.classList.toggle('ls-illum-btn--active', on); + _lsRoomLightBtn.setAttribute('aria-pressed', on ? 'true' : 'false'); + } + } + } catch (err) { console.debug('manual room light toggle failed:', err); } + } + + // ---- Acquire -------------------------------------------------------- + async function runLightsheetAcquire(mode) { + const btn = mode === 'burst' ? _lsBurstBtn : _lsSnapVolBtn; + if (btn) { btn.disabled = true; btn.textContent = 'acquiring…'; } + try { + let res; + if (mode === 'burst') { + res = await fetch('/api/devices/acquire/burst', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ frames: 10, mode: 'brightfield', + num_slices: 50, exposure_ms: _lsExposure, + laser_config: 'ALL OFF', + piezo_center: _lsPiezo, + galvo_center: _lsGalvo }), + }); + } else { + res = await fetch('/api/devices/acquire/volume', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ num_slices: 50, exposure_ms: _lsExposure, + laser_config: 'ALL OFF', + piezo_center: _lsPiezo, + galvo_center: _lsGalvo }), + }); + } + if (!res.ok) { + console.error('acquire failed:', res.status, await res.text()); + return; + } + const data = await res.json(); + if (_lsLastcap) _lsLastcap.hidden = false; + if (_lsLastcapRef) { + _lsLastcapRef.textContent = data.volume_path || data.path || data.id || 'done'; + } + // Show confirmation toast — no inline preview to keep manual mode uncluttered + if (typeof showGentlyToast === 'function') { + const label = mode === 'burst' ? 'Burst acquired' : 'Volume acquired'; + showGentlyToast(label, 'View in Gallery', () => { + if (typeof switchTab === 'function' && typeof TABS !== 'undefined') { + switchTab(TABS.GALLERY); + } + }); + } + } catch (err) { + console.error('acquire error:', err); + } finally { + if (btn) { + btn.disabled = false; + btn.textContent = mode === 'burst' ? 'Burst' : 'Snap Volume'; + } + } + } + + // ---- Temperature set (rail copy, delegates to same API) ------------- + async function setLightsheetTemperature() { + if (!_lsTempInput) return; + const target = parseFloat(_lsTempInput.value); + if (isNaN(target) || target < 0 || target > 99.9) return; + try { + await fetch('/api/devices/temperature/set', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ target_c: target }), + }); + } catch (err) { console.debug('ls temp set failed:', err); } + } + + function setupManualWiring() { + if (!_lsToggle) return; + _lsToggle.addEventListener('click', toggleLightsheetStream); + applyLightsheetState(false); + + // Param controls — slider ↔ number sync + debounced POST + if (_lsGalvoSlider) _lsGalvoSlider.addEventListener('input', () => onGalvoInput(_lsGalvoSlider)); + if (_lsGalvoNum) _lsGalvoNum.addEventListener('input', () => onGalvoInput(_lsGalvoNum)); + if (_lsPiezoSlider) _lsPiezoSlider.addEventListener('input', () => onPiezoInput(_lsPiezoSlider)); + if (_lsPiezoNum) _lsPiezoNum.addEventListener('input', () => onPiezoInput(_lsPiezoNum)); + if (_lsExposureNum) _lsExposureNum.addEventListener('input', onExposureInput); + + // Illumination + if (_lsLedToggle) _lsLedToggle.addEventListener('click', toggleLedPreset); + if (_lsRoomLightBtn) _lsRoomLightBtn.addEventListener('click', toggleManualRoomLight); + if (_lsLaserToggle) _lsLaserToggle.addEventListener('click', toggleLaser); + + // Acquire + if (_lsSnapVolBtn) _lsSnapVolBtn.addEventListener('click', () => runLightsheetAcquire('volume')); + if (_lsBurstBtn) _lsBurstBtn.addEventListener('click', () => runLightsheetAcquire('burst')); + + // Temperature + if (_lsTempSet) _lsTempSet.addEventListener('click', setLightsheetTemperature); + if (_lsTempInput) { + _lsTempInput.addEventListener('keydown', (e) => { + if (e.key === 'Enter') { e.preventDefault(); setLightsheetTemperature(); } + }); + } + + // Zoom / pan + if (_lsStage) { + _lsStage.addEventListener('wheel', onLightsheetWheel, { passive: false }); + _lsStage.addEventListener('pointerdown', onLightsheetPointerDown); + _lsStage.addEventListener('pointermove', onLightsheetPointerMove); + _lsStage.addEventListener('pointerup', onLightsheetPointerEnd); + _lsStage.addEventListener('pointercancel', onLightsheetPointerEnd); + _lsStage.addEventListener('dblclick', onLightsheetDoubleClick); + } + + // Subscribe to LIGHTSHEET_FRAME events + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('LIGHTSHEET_FRAME', handleLightsheetFrame); + } + + syncInitialLightsheetState(); + } + + // ===================================================================== + // Room-light toggle + // ===================================================================== + + function applyRoomLight(state, available) { + _roomLightState = state || 'unknown'; + _roomLightAvailable = !!available; + if (!_roomLightToggle) return; + _roomLightToggle.hidden = !_roomLightAvailable; + _roomLightToggle.disabled = !_roomLightAvailable || _roomLightBusy; + const on = _roomLightState === 'on'; + _roomLightToggle.classList.toggle('is-on', on); + _roomLightToggle.setAttribute('aria-pressed', on ? 'true' : 'false'); + if (_roomLightLabel && !_roomLightBusy) { + _roomLightLabel.textContent = on ? 'Room light: on' + : (_roomLightState === 'off' ? 'Room light: off' : 'Room light'); + } + } + + async function loadRoomLightStatus() { + if (!_roomLightToggle || _roomLightBusy) return; + try { + const res = await fetch('/api/devices/room_light/status'); + if (!res.ok) { applyRoomLight('unknown', false); return; } + const data = await res.json(); + applyRoomLight(data.state, data.available); + } catch (err) { + console.debug('room light status fetch failed:', err); + applyRoomLight('unknown', false); + } + } + + async function toggleRoomLight() { + if (!_roomLightToggle || _roomLightBusy || !_roomLightAvailable) return; + const next = _roomLightState === 'on' ? 'off' : 'on'; + _roomLightBusy = true; + _roomLightToggle.classList.add('is-busy'); + _roomLightToggle.disabled = true; + if (_roomLightLabel) { + _roomLightLabel.textContent = next === 'on' ? 'Turning on…' : 'Turning off…'; + } + + // Settle back to the resolved state, or surface a transient message + // (insufficient control / error) for 2 s before reverting. + const finish = (msg) => { + _roomLightBusy = false; + _roomLightToggle.classList.remove('is-busy'); + if (msg) { + if (_roomLightLabel) _roomLightLabel.textContent = msg; + _roomLightToggle.disabled = false; + setTimeout(() => applyRoomLight(_roomLightState, _roomLightAvailable), 2000); + } else { + applyRoomLight(_roomLightState, _roomLightAvailable); + } + }; + + try { + const res = await fetch('/api/devices/room_light/set', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ state: next }), + }); + if (res.status === 401 || res.status === 403) { finish('Need control'); return; } + if (!res.ok) { + console.error('room light set failed:', await res.text()); + finish('Error'); + return; + } + const data = await res.json(); + _roomLightState = data.state || next; + finish(null); + } catch (err) { + console.error('room light toggle failed:', err); + finish('Error'); + } + } + + function setupRoomLight() { + if (!_roomLightToggle) return; + _roomLightToggle.addEventListener('click', toggleRoomLight); + loadRoomLightStatus(); + // Light periodic refresh: state can also change from agent plans + // (e.g. brightfield imaging turns it on). Status read is cached at the + // device layer (no BLE), so polling is cheap; it also makes the toggle + // appear automatically once the device layer connects. + if (_roomLightTimer) clearInterval(_roomLightTimer); + _roomLightTimer = setInterval(loadRoomLightStatus, 15000); + } + + // ===================================================================== + // Temperature controller (ACUITYnano) — readout + setpoint + // ===================================================================== + + function fmtTemp(v) { + return (v === null || v === undefined || isNaN(v)) ? '—' : Number(v).toFixed(1) + '°'; + } + + function applyTemperature(data) { + _tempAvailable = !!(data && data.available); + if (!_tempEl) return; + _tempEl.hidden = !_tempAvailable; + if (!_tempAvailable) return; + _tempState = (data && data.state) || 'unknown'; + const locked = /LOCK/i.test(_tempState); + _tempEl.classList.toggle('is-locked', locked); + if (_tempBusy) return; // a set() is in flight; leave its transient label + const cur = fmtTemp(data.temperature_c); + const hasSp = data.setpoint_c !== null && data.setpoint_c !== undefined; + const sp = hasSp ? fmtTemp(data.setpoint_c) : null; + _tempReadout.textContent = sp ? (cur + ' → ' + sp) : cur; + _tempReadout.title = 'Water ' + cur + (sp ? (', setpoint ' + sp) : '') + + (locked ? ' (locked)' : ''); + // Seed the input with the current setpoint once, while untouched, so the + // operator sees where it is before nudging it. + if (_tempInput && document.activeElement !== _tempInput && _tempInput.value === '' && hasSp) { + _tempInput.value = Number(data.setpoint_c).toFixed(1); + } + } + + async function loadTemperatureStatus() { + if (!_tempEl || _tempBusy) return; + try { + const res = await fetch('/api/devices/temperature/status'); + if (!res.ok) { applyTemperature({ available: false }); return; } + applyTemperature(await res.json()); + } catch (err) { + console.debug('temperature status fetch failed:', err); + applyTemperature({ available: false }); + } + } + + async function setTemperature() { + if (!_tempEl || _tempBusy || !_tempAvailable) return; + const target = parseFloat(_tempInput && _tempInput.value); + if (isNaN(target) || target < 0 || target > 99.9) { + _tempReadout.textContent = '0–99.9 °C'; + setTimeout(loadTemperatureStatus, 1500); + return; + } + _tempBusy = true; + _tempEl.classList.add('is-busy'); + if (_tempSet) _tempSet.disabled = true; + _tempReadout.textContent = 'Set ' + target.toFixed(1) + '°…'; + + // Settle back to the resolved state, or surface a transient message + // (insufficient control / error) for 2 s before reverting. + const finish = (msg) => { + _tempBusy = false; + _tempEl.classList.remove('is-busy'); + if (_tempSet) _tempSet.disabled = false; + if (msg) { + _tempReadout.textContent = msg; + setTimeout(loadTemperatureStatus, 2000); + } else { + loadTemperatureStatus(); // controller ramps; poll shows progress + } + }; + + try { + const res = await fetch('/api/devices/temperature/set', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ target_c: target }), + }); + if (res.status === 401 || res.status === 403) { finish('Need control'); return; } + if (!res.ok) { + console.error('temperature set failed:', await res.text()); + finish('Error'); + return; + } + await res.json(); + finish(null); + } catch (err) { + console.error('temperature set failed:', err); + finish('Error'); + } + } + + function setupTemperature() { + if (!_tempEl) return; + if (_tempSet) _tempSet.addEventListener('click', setTemperature); + if (_tempInput) { + _tempInput.addEventListener('keydown', (e) => { + if (e.key === 'Enter') { e.preventDefault(); setTemperature(); } + }); + } + loadTemperatureStatus(); + // Periodic refresh: the setpoint can also change from agent plans, and a + // commanded ramp settles over time. Status is cached at the device layer, + // so polling is cheap; it also reveals the control once the layer connects. + if (_tempTimer) clearInterval(_tempTimer); + _tempTimer = setInterval(loadTemperatureStatus, 15000); } // ===================================================================== @@ -904,6 +2424,35 @@ const DevicesManager = (function () { if (typeof updateViewButtons === 'function') { updateViewButtons('devices-view-switcher', viewName); } + // The 3D optical-space view owns its own WebGL module. Build it lazily + // on first activation (the panel was display:none, so its container had + // no size until now); init() is idempotent and resizes on re-entry. + if (viewName === 'optical3d' && typeof Occupancy3DManager !== 'undefined') { + Occupancy3DManager.init(); + } + // Initialize temperature graph for the active view. The TemperatureGraph + // is a singleton; reinit on view switch ensures only one graph target + // is live at a time. ClientEventBus.off/on in TemperatureGraph.init + // makes re-init safe (idempotent). + if (window.TemperatureGraph) { + if (viewName === 'manual') { + const el = document.getElementById('devices-ls-tempgraph'); + if (el) TemperatureGraph.init(el, 'current'); + } else { + const el = document.getElementById('devices-temp-graph'); + if (el) TemperatureGraph.init(el, 'current'); + } + } + // Entering Manual view — gate lasers off immediately (brightfield-safe). + // populateLaserPresets() runs after setLaserOff() so the select is always + // seeded with the entry-safety state first. + if (viewName === 'manual') { setLaserOff(); populateLaserPresets(); populateCameraRoles(); initTlForm(); populateTlDefaults(); } + // The Operate view owns its own module; activate/deactivate it so its + // camera + SPIM streams only run while it's the visible view. + if (typeof OperateManager !== 'undefined') { + if (viewName === 'operate') OperateManager.activate(); + else OperateManager.deactivate(); + } } function setupViewSwitcher() { @@ -913,8 +2462,10 @@ const DevicesManager = (function () { document.addEventListener('keydown', (e) => { if (typeof state !== 'undefined' && typeof TABS !== 'undefined' && state.tab !== TABS.DEVICES) return; if (e.target.matches('input, textarea, select, [contenteditable]')) return; - if (e.key === 'm') { e.preventDefault(); switchView('map'); } + if (e.key === 'o') { e.preventDefault(); switchView('operate'); } + else if (e.key === 'm') { e.preventDefault(); switchView('map'); } else if (e.key === 'd') { e.preventDefault(); switchView('details'); } + else if (e.key === 'v') { e.preventDefault(); switchView('optical3d'); } }); } @@ -945,31 +2496,54 @@ const DevicesManager = (function () { scheduleStaleCheck(); } + // Single availability signal (from the launcher's device-layer watcher). + // When the layer goes away, drop the stale working-region so the map reverts + // to the neutral no-position state instead of a frozen red/green field. Live + // data repopulates _optimalBox via DEVICE_STATE_UPDATE once it's back. + function handleAvailability(d) { + if (d && d.available === false && _optimalBox) { + _optimalBox = null; + renderMap(); + } + } + function init() { cacheDom(); setupViewSwitcher(); setupCameraWiring(); + setupManualWiring(); + setupRoomLight(); + setupTemperature(); loadCoverslip(); - loadEmbryos(); + loadEmbryosSnapshot(); switchView(_currentView); if (typeof ClientEventBus !== 'undefined') { ClientEventBus.on('DEVICE_STATE_UPDATE', handlePayload); - // Embryo events: a fresh marking session emits one - // EMBRYO_DETECTED per registered embryo (via - // ExperimentState.add_embryo). assign_embryo_roles emits - // STATUS_CHANGED with change=role_assigned per change. + ClientEventBus.on('DEVICE_LAYER_AVAILABILITY', handleAvailability); + ClientEventBus.on('EMBRYOS_UPDATE', handleEmbryosUpdate); + // Belt-and-braces: also listen for the fine-grained events that + // existed before EMBRYOS_UPDATE so direct emitters still refresh. ClientEventBus.on('EMBRYO_DETECTED', handleEmbryoDetected); ClientEventBus.on('STATUS_CHANGED', handleStatusChanged); } + // Map-side edit handlers. Pointer events on the SVG cover both + // "click an embryo" (selects it) and "click empty map" (drops the + // selected embryo). Keyboard listener is document-wide but guards + // against firing while an input is focused. + if (_mapSvg) { + _mapSvg.addEventListener('pointerdown', onMapPointerDown); + } + document.addEventListener('keydown', onMapKeyDown); setStatus('stale', 'waiting', 'no payload yet'); syncInitialCameraState(); - // Stop the camera stream if the tab is closed while it's running, - // so MMCore isn't held by a disconnected browser. + // Stop the camera and lightsheet streams if the tab is closed while + // running, so MMCore isn't held by a disconnected browser. window.addEventListener('beforeunload', () => { if (_camStreaming) { - try { - navigator.sendBeacon('/api/devices/bottom_camera/stream/stop'); - } catch (_) {} + try { navigator.sendBeacon('/api/devices/bottom_camera/stream/stop'); } catch (_) {} + } + if (_lsStreaming) { + try { navigator.sendBeacon('/api/devices/lightsheet/live/stop'); } catch (_) {} } }); } diff --git a/gently/ui/web/static/js/embryos.js b/gently/ui/web/static/js/embryos.js index f8b7cea3..29340cf5 100644 --- a/gently/ui/web/static/js/embryos.js +++ b/gently/ui/web/static/js/embryos.js @@ -59,7 +59,7 @@ const EmbryosManager = { dashboardConfig: { defaultView: 'default', board: { - columns: ['stage', 'confidence', 'rate', 'eta', 'sparkline', 'alert'], + columns: ['stage', 'clock', 'stereo', 'pace', 'eta', 'sparkline', 'alert'], sparklineLength: 20, warnOvertimeRatio: 1.5, criticalOvertimeRatio: 2.5 @@ -266,6 +266,23 @@ const EmbryosManager = { // Deep merge with defaults this.dashboardConfig = this._deepMerge(this.dashboardConfig, parsed); } + // Migrate legacy board columns: drop the never-populated + // 'confidence' column and the misleading 'rate' column in + // favour of clock/stereo/pace. Idempotent — runs on every load. + const cols = this.dashboardConfig.board?.columns; + if (Array.isArray(cols)) { + const filtered = cols.filter(c => c !== 'confidence' && c !== 'rate'); + const ensure = (key, after) => { + if (filtered.includes(key)) return; + const idx = filtered.indexOf(after); + if (idx === -1) filtered.push(key); + else filtered.splice(idx + 1, 0, key); + }; + ensure('clock', 'stage'); + ensure('stereo', 'clock'); + ensure('pace', 'stereo'); + this.dashboardConfig.board.columns = filtered; + } } catch (e) { console.warn('Failed to load dashboard config:', e); } @@ -370,9 +387,10 @@ const EmbryosManager = {
    Embryo ${cols.includes('stage') ? 'Stage' : ''} - ${cols.includes('confidence') ? 'Conf' : ''} - ${cols.includes('rate') ? 'Rate' : ''} - ${cols.includes('eta') ? 'ETA' : ''} + ${cols.includes('clock') ? 'Clock' : ''} + ${cols.includes('stereo') ? 'Stereo' : ''} + ${cols.includes('pace') ? 'Pace' : ''} + ${cols.includes('eta') ? 'ETA' : ''} ${cols.includes('sparkline') ? 'Progression' : ''} ${cols.includes('alert') ? 'Alert' : ''}
    @@ -412,54 +430,27 @@ const EmbryosManager = { const latest = reasoning.length > 0 ? reasoning[reasoning.length - 1] : null; const cols = this.dashboardConfig.board.columns; - // Stage const stage = latest?.stage || embryo.current_stage || '—'; const stageIcon = this.getStageIcon(stage); const stageName = this.formatStageName(stage); - // Confidence - const conf = latest ? this.normalizeConfidence(latest.confidence) : 'unknown'; - const confDots = conf === 'high' ? '●●●' : conf === 'medium' ? '●●○' : conf === 'low' ? '●○○' : '○○○'; - const confClass = conf === 'high' ? 'conf-high' : conf === 'medium' ? 'conf-med' : 'conf-low'; + const align = this._computeAlignment(latest); + const overtime = align?.overtime; - // Rate - const overtime = latest?.temporal_analysis?.overtime_ratio; - let rateText = '—'; - let rateClass = ''; - if (overtime != null) { - const rate = (1 / overtime).toFixed(1); - rateText = overtime < 0.9 ? `${rate}x↑` : overtime > 1.1 ? `${rate}x↓` : `${rate}x→`; - rateClass = overtime < 0.9 ? 'rate-fast' : overtime > 1.5 ? 'rate-slow' : 'rate-normal'; - } + const clockText = align ? this._formatMinutes(align.inStageClockMin) : '—'; + const stereoText = align ? this._formatStereoLabel(align) : '—'; + const pace = align ? this._formatPace(align) : { text: '—', className: '' }; + const eta = align ? this._formatEta(align) : '—'; - // ETA - let eta = '—'; - if (stage && this.STAGE_TIMING[stage] != null) { - const stageMinutes = this.STAGE_TIMING[stage]; - const hatchMinutes = this.STAGE_TIMING['hatched'] || 570; - const remaining = hatchMinutes - stageMinutes; - if (remaining > 0) { - const hours = (remaining / 60).toFixed(1); - eta = `~${hours}h`; - } else { - eta = 'done'; - } - } - - // Sparkline const sparklineSvg = cols.includes('sparkline') ? this._renderBoardSparkline(reasoning) : ''; - // Alert const arrested = latest?.temporal_analysis?.is_potentially_arrested; const slow = overtime && overtime > (this.dashboardConfig.board.warnOvertimeRatio || 1.5); - const lowConf = conf === 'low'; let alertHtml = ''; if (arrested) { alertHtml = '⚠ arrested'; } else if (slow) { - alertHtml = `⚠ slow ${overtime.toFixed(1)}x`; - } else if (lowConf) { - alertHtml = '⚠ low conf'; + alertHtml = `⚠ slow ${overtime.toFixed(1)}×`; } const status = embryo.isComplete ? 'complete' : embryo.lastError ? 'error' : 'running'; @@ -472,8 +463,9 @@ const EmbryosManager = { ${embryo.embryoId.replace(/embryo_?/i, 'E')}
    ${cols.includes('stage') ? `${stageIcon} ${stageName}` : ''} - ${cols.includes('confidence') ? `${confDots}` : ''} - ${cols.includes('rate') ? `${rateText}` : ''} + ${cols.includes('clock') ? `${clockText}` : ''} + ${cols.includes('stereo') ? `${stereoText}` : ''} + ${cols.includes('pace') ? `${pace.text}` : ''} ${cols.includes('eta') ? `${eta}` : ''} ${cols.includes('sparkline') ? `${sparklineSvg}` : ''} ${cols.includes('alert') ? `${alertHtml}` : ''} @@ -481,6 +473,99 @@ const EmbryosManager = { `; }, + /** Compute clock↔stereotypic alignment from perception temporal_analysis. + * + * Definitions: + * inStageClockMin — wall-clock minutes elapsed in current stage + * inStageStereoMin — stereotypic minutes "used" within the stage, + * capped at the stage's expected duration. An + * overdue embryo is stuck at the stage end in + * stereo time while clock keeps ticking. + * overtime — ratio inStageClockMin / expected_duration. + * >1 means the embryo has spent more clock time + * in the stage than the reference 20°C textbook + * duration. <1 just means "still within stage" — + * no slow/fast signal yet. + * stereoAgeMin — total stereotypic age, anchored at the start + * minute of the current stage in the reference + * table plus the (capped) in-stage stereo offset. + */ + _computeAlignment(latest) { + const ta = latest?.temporal_analysis; + if (!ta || !ta.current_stage) return null; + const stage = ta.current_stage; + const stageStart = this.STAGE_TIMING[stage]; + if (stageStart == null) return null; + + const expDur = Number(ta.expected_duration_min) || 0; + const inClock = Number(ta.time_in_stage_min) || 0; + const overtime = Number(ta.overtime_ratio) || 0; + + const inStereo = expDur > 0 ? Math.min(inClock, expDur) : inClock; + const stereoAge = stageStart + inStereo; + + return { + stage, + stageStart, + expDur, + inStageClockMin: inClock, + inStageStereoMin: inStereo, + stereoAgeMin: stereoAge, + overtime, + }; + }, + + /** Render the stereo cell: "≈early", "≈bean +12m", or "≈comma +88m ⚠" + * when overdue (stereo capped at stage end while clock keeps running). */ + _formatStereoLabel(align) { + const stageName = this.formatStageName(align.stage); + const offsetMin = Math.round(align.inStageStereoMin); + const overdue = align.expDur > 0 && align.inStageClockMin > align.expDur + 1; + const offsetStr = offsetMin > 0 ? ` +${offsetMin}m` : ''; + const overdueMark = overdue ? ' ' : ''; + return `≈${stageName}${offsetStr}${overdueMark}`; + }, + + _formatPace(align) { + // Only emit a pace signal once we have meaningful clock data. + // Within the first few minutes the ratio is tiny and noisy — show + // a dashed placeholder so the column doesn't lie about precision. + const NORMAL_BAND = 1.05; + const SLOW_BAND = 1.5; + if (align.inStageClockMin < 1 || align.expDur <= 0) { + return { text: '—', className: 'pace-unknown' }; + } + const r = align.overtime; + if (r <= NORMAL_BAND) { + return { text: '1.0×', className: 'pace-normal' }; + } + if (r <= SLOW_BAND) { + return { text: `${r.toFixed(1)}× slow`, className: 'pace-slow' }; + } + return { text: `⚠ ${r.toFixed(1)}×`, className: 'pace-slow-bad' }; + }, + + /** ETA in hours from current stereotypic position to hatched, scaled + * by observed pace when the embryo is demonstrably slow. */ + _formatEta(align) { + const hatchStereo = this.STAGE_TIMING['hatched'] || 570; + const remainStereo = hatchStereo - align.stereoAgeMin; + if (remainStereo <= 0) return 'done'; + const paceFactor = align.overtime > 1.05 ? align.overtime : 1.0; + const remainClockMin = remainStereo * paceFactor; + return `~${(remainClockMin / 60).toFixed(1)}h`; + }, + + /** Compact minute formatter: "45s" / "10m" / "1h 22m" / "3h". */ + _formatMinutes(min) { + if (min == null || !isFinite(min)) return '—'; + if (min < 1) return `${Math.round(min * 60)}s`; + if (min < 60) return `${Math.round(min)}m`; + const h = Math.floor(min / 60); + const m = Math.round(min - h * 60); + return m > 0 ? `${h}h ${m}m` : `${h}h`; + }, + _renderBoardSparkline(reasoning) { if (!reasoning.length) return ''; const sorted = [...reasoning].sort((a, b) => (a.timepoint ?? 0) - (b.timepoint ?? 0)); @@ -565,12 +650,24 @@ const EmbryosManager = { const shortName = embryo.embryoId.replace(/embryo_?/i, 'E'); const latestStage = sorted.length > 0 ? this.formatStageName(sorted[sorted.length - 1].stage) : '—'; + const isTerminated = !!embryo.isComplete; + const termReason = embryo.completionReason || ''; + // Short label for the badge — humanise the no_object terminal + // reason, otherwise keep the first clause of whatever the + // backend sent so the user still gets a hint. + const termBadge = isTerminated + ? (termReason.includes('no_object') ? 'HATCHED?' : 'STOPPED') + : ''; + const termTooltip = isTerminated + ? `Terminated — ${termReason || 'no reason given'}` + : ''; - html += `
    `; + html += `
    `; html += `
    ${shortName} ${latestStage} ${reasoning.length} eval + ${isTerminated ? `${termBadge}` : ''}
    `; html += `
    `; @@ -1000,7 +1097,7 @@ const EmbryosManager = { intervalSeconds: embryoData.interval_seconds || this.state.baseInterval, timepoints: embryoData.timepoints || 0, isComplete: embryoData.is_complete || false, - completionReason: null, + completionReason: embryoData.completion_reason || null, firstAcquired: embryoData.first_acquired ? new Date(embryoData.first_acquired) : null, lastAcquired: embryoData.last_acquired ? new Date(embryoData.last_acquired) : null, detections: embryoData.detections || {}, @@ -2726,10 +2823,17 @@ const EmbryosManager = { `; } - // Format confidence display - const confDisplay = typeof item.confidence === 'number' - ? `${Math.round(item.confidence * 100)}%` - : (item.confidence || 'Unknown'); + // Format confidence display. Hide entirely when the detector + // doesn't emit a probabilistic confidence (e.g. dopaminergic_signal + // returns structured intensity/structure findings instead) — the + // string "Unknown confidence" was actively confusing. + const hasNumericConf = typeof item.confidence === 'number'; + const hasTextConf = typeof item.confidence === 'string' && item.confidence.trim() !== ''; + const confHtml = hasNumericConf + ? `${Math.round(item.confidence * 100)}% confidence` + : hasTextConf + ? `${item.confidence}` + : ''; return `
    @@ -2748,7 +2852,7 @@ const EmbryosManager = {
    ${item.stage ? this.formatStageName(item.stage) : (item.detected ? 'DETECTED' : 'Not detected')} - ${confDisplay} confidence + ${confHtml} ${transitionalHtml}
    ${detectorFindingsHtml} @@ -2979,13 +3083,22 @@ const EmbryosManager = { container.classList.remove('visible'); container.innerHTML = ''; } + // Filmstrip side panel — clearing innerHTML lets the :empty CSS + // rule collapse the panel and let the rows reclaim full width. + const filmstripDetail = document.getElementById('filmstrip-detail'); + if (filmstripDetail) { + filmstripDetail.innerHTML = ''; + } this.detailPanelVisible = false; this.currentDetailItem = null; - // Clear eval dot highlight + // Clear eval dot + filmstrip cell highlight document.querySelectorAll('.eval-dot.active').forEach(dot => { dot.classList.remove('active'); }); + document.querySelectorAll('.filmstrip-cell.active').forEach(cell => { + cell.classList.remove('active'); + }); }, // Navigate to previous/next item in detail panel diff --git a/gently/ui/web/static/js/events.js b/gently/ui/web/static/js/events.js index 02365e40..998431a1 100644 --- a/gently/ui/web/static/js/events.js +++ b/gently/ui/web/static/js/events.js @@ -41,6 +41,40 @@ function getEventBadgeClass(eventType) { return 'default'; } +// Log-record helpers -------------------------------------------------- +// LOG_RECORD events come from the Python logging bridge. We collapse the +// generic "LOG_RECORD" type into the actual level (DEBUG / INFO / WARN / +// ERROR) so the table is readable -- otherwise every row in a busy +// session reads the same string in the Type column. +function isLogEvent(event) { + return event && event.event_type === 'LOG_RECORD'; +} + +function logLevelLabel(d) { + // levelname is fastest path; fall back to numeric mapping if missing. + const lvl = (d && (d.level_name || '')).toString().toUpperCase(); + if (lvl) { + if (lvl === 'WARNING') return 'WARN'; + if (lvl === 'CRITICAL') return 'CRIT'; + return lvl; + } + const n = d && Number(d.level); + if (!isFinite(n)) return 'LOG'; + if (n >= 50) return 'CRIT'; + if (n >= 40) return 'ERROR'; + if (n >= 30) return 'WARN'; + if (n >= 20) return 'INFO'; + return 'DEBUG'; +} + +function logBadgeClass(d) { + const label = logLevelLabel(d); + if (label === 'DEBUG') return 'log-debug'; + if (label === 'INFO') return 'log-info'; + if (label === 'WARN') return 'log-warn'; + return 'log-error'; // ERROR / CRIT collapse together +} + // Search helper functions function escapeRegex(str) { return str.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); @@ -67,12 +101,17 @@ function eventMatchesSearch(event) { } function highlightSearchTerms(text) { - if (!searchQuery || !text) return text; + // Escape first — event keys/values/messages are arbitrary text (perception + // prose, file paths, agent output) and are inserted via innerHTML by the + // callers. Escaping here closes the XSS hole at every call site; the + // injected tags are the only markup we add. + const safe = escapeHtml(text == null ? '' : String(text)); + if (!searchQuery) return safe; try { const regex = new RegExp(`(${escapeRegex(searchQuery)})`, 'gi'); - return String(text).replace(regex, '$1'); + return safe.replace(regex, '$1'); } catch (e) { - return text; + return safe; } } @@ -212,45 +251,81 @@ function addEventToTable(event, prepend = true) { if (hasImage) tr.classList.add('has-image'); tr.dataset.eventId = event.event_id || ''; - const badgeClass = getEventBadgeClass(event.event_type); - - // Image indicator icon - const imageIndicator = hasImage - ? ` - - - - - - ` - : ''; - - // Thumbnail preview - const thumbnailHtml = hasImage - ? `Event image` - : ''; - - tr.innerHTML = ` - ${formatEventTime(event.timestamp)} - ${imageIndicator}${event.event_type} - ${event.source || '-'} - ${thumbnailHtml}
    ${formatEventData(event.data)}
    - `; - - // Click to expand data - tr.addEventListener('click', () => { - const dataDiv = tr.querySelector('.event-data'); - dataDiv.classList.toggle('expanded'); - if (dataDiv.classList.contains('expanded')) { - dataDiv.innerHTML = `
    ${JSON.stringify(event.data, null, 2)}
    `; - } else { - dataDiv.innerHTML = formatEventData(event.data); - } - }); + if (isLogEvent(event)) { + // Log rows have a compact, distinctive shape: level badge in the + // Type column, logger name + message in the Data column. Click to + // toggle a pre with the full payload (incl. exception trace). + tr.classList.add('log-row'); + const d = event.data || {}; + const badgeCls = logBadgeClass(d); + const label = logLevelLabel(d); + const message = highlightSearchTerms(d.message || ''); + const loggerName = highlightSearchTerms(d.logger || '-'); + const excTag = d.exc_text ? ' ⏎ trace…' : ''; + tr.innerHTML = ` + ${formatEventTime(event.timestamp)} + ${label} + ${event.source || '-'} +
    + ${loggerName}${message}${excTag} +
    + `; + tr.addEventListener('click', () => { + const dataDiv = tr.querySelector('.event-data'); + dataDiv.classList.toggle('expanded'); + if (dataDiv.classList.contains('expanded')) { + const tracePart = d.exc_text + ? `\n\n${d.exc_text}` : ''; + dataDiv.innerHTML = + `
    ${d.logger || ''}  ${d.func || ''}:${d.line || ''}\n` +
    +                    `${(d.message || '')}${tracePart}
    `; + } else { + dataDiv.innerHTML = + `${loggerName}` + + `${message}${excTag}`; + } + }); + } else { + const badgeClass = getEventBadgeClass(event.event_type); + + // Image indicator icon + const imageIndicator = hasImage + ? ` + + + + + + ` + : ''; + + // Thumbnail preview + const thumbnailHtml = hasImage + ? `Event image` + : ''; + + tr.innerHTML = ` + ${formatEventTime(event.timestamp)} + ${imageIndicator}${event.event_type} + ${event.source || '-'} + ${thumbnailHtml}
    ${formatEventData(event.data)}
    + `; + + // Click to expand data + tr.addEventListener('click', () => { + const dataDiv = tr.querySelector('.event-data'); + dataDiv.classList.toggle('expanded'); + if (dataDiv.classList.contains('expanded')) { + dataDiv.innerHTML = `
    ${JSON.stringify(event.data, null, 2)}
    `; + } else { + dataDiv.innerHTML = formatEventData(event.data); + } + }); + } if (prepend) { tbody.insertBefore(tr, tbody.firstChild); diff --git a/gently/ui/web/static/js/experiment-overview.js b/gently/ui/web/static/js/experiment-overview.js index 33032a40..a5be6444 100644 --- a/gently/ui/web/static/js/experiment-overview.js +++ b/gently/ui/web/static/js/experiment-overview.js @@ -1,161 +1,91 @@ /** - * Experiment Overview Tab — vector-graphics view of the planned timelapse. + * Experiment Overview Tab — vector-graphics view of the live imaging tactics + * (cadence patterns + reactive-monitoring rules) for the running experiment. * - * Data source priority: - * 1. GET /api/experiments/current/strategy — live snapshot from FileStore. - * 2. STUB_STRATEGY below — used when the live fetch - * fails or no session exists. - * - * The render path is data-shape-driven and doesn't care which source the - * snapshot came from — only ``ExperimentOverview.isLive`` differs so the - * header badge can say "live" or "mockup · stubbed data". + * Data source: GET /api/experiments/current/strategy — the live snapshot from + * FileStore. When there is no active experiment (or the fetch isn't ready), the + * view shows a calm empty state; it never renders stubbed/mock data. */ -const STUB_STRATEGY = { - session_id: "20260522_1430_dopaminergic_demo_a3f8e1c2", - session_name: "dopaminergic-reporter demo", - started_at: "2026-05-22T14:30:00", - now_offset_s: 8100, // 2h 15min into the run - horizon_s: 14400, // 4h total view window (past + projected) - base_interval_s: 120, - dose_budget_base_ms: 50000, - per_timepoint_ms: 500, // 50 slices × 10ms - monitoring_modes: [ - { - name: "expression_monitoring", - description: "Anticipating fluorescent-reporter onset on Test embryos: accelerate to 60s on signal, ramp 488 down on saturation.", - applies_to_roles: ["test"], - params: { - fast_interval: 60, - rampdown_step_pct: 1.0, - rampdown_floor_pct: 2.0, - rampdown_ceiling_pct: 6.0 - } - }, - { - name: "pre_terminal_monitoring", - description: "Anticipating organism pre-terminal stage (pretzel): accelerate to 30s on detection.", - applies_to_roles: ["test"], - params: { fast_interval: 30 } - } - ], - triggers: [ - { id: "t1", kind: "interval_rule", label: "signal onset", - when_text: "dopaminergic ≥ WEAK", then_text: "120s → 60s", - applies_to: ["test"], one_time: true }, - { id: "t2", kind: "power_rule", label: "488 ramp down", - when_text: "intensity = SATURATING (×3)", then_text: "488 ↓ 1%/step, floor 2%", - applies_to: ["test"] }, - { id: "t3", kind: "burst", label: "structure-triggered burst", - when_text: "structure_quality = GOOD", then_text: "burst 200 frames @ 20 Hz", - applies_to: ["test"] }, - { id: "t4", kind: "interval_rule", label: "pre-terminal speedup", - when_text: "stage = pretzel", then_text: "60s → 30s", - applies_to: ["test"], one_time: true } - ], - embryos: [ - { - id: "E1", role: "test", color: "#ff66cc", icon: "★", - dose_used_ms: 12500, dose_budget_ms: 50000, - tp_acquired: 25, - stop_condition: "hatching+3 OR 24h duration", - stop_kind: "bounded", - laser_488_pct_now: 3.0, - phases: [ - { mode: "base", start: 0, end: 1800, cadence_s: 120 }, - { mode: "fast", start: 1800, end: 3600, cadence_s: 60 }, - { mode: "burst", start: 3600, end: 3610, frames: 200, hz: 20 }, - { mode: "cooldown", start: 3610, end: 3640, cadence_s: 60 }, - { mode: "fast", start: 3640, end: 8100, cadence_s: 60 } - ], - trigger_events: [ - { trigger_id: "t1", at: 1800 }, - { trigger_id: "t3", at: 3600 }, - { trigger_id: "t2", at: 5400, count: 3 } - ], - power_history_488: [ - { at: 0, pct: 5.0 }, - { at: 5400, pct: 4.0 }, - { at: 5460, pct: 3.0 }, - { at: 8100, pct: 3.0 } - ], - // Future projection at current cadence (60s, fast). Hatching not - // deterministic so projected_end_s is null — render fades to ∞. - projected_cadence_s: 60, - projected_end_s: null - }, - { - id: "E2", role: "test", color: "#ff66cc", icon: "★", - dose_used_ms: 6500, dose_budget_ms: 50000, - tp_acquired: 13, - stop_condition: "hatching+3 OR 24h duration", - stop_kind: "bounded", - laser_488_pct_now: 5.0, - phases: [ - { mode: "base", start: 0, end: 8100, cadence_s: 120 } - ], - trigger_events: [], - power_history_488: [ - { at: 0, pct: 5.0 }, - { at: 8100, pct: 5.0 } - ], - projected_cadence_s: 120, - projected_end_s: null - }, - { - id: "E3", role: "test", color: "#ff66cc", icon: "★", - dose_used_ms: 38000, dose_budget_ms: 50000, - tp_acquired: 76, - stop_condition: "manual", - stop_kind: "open_ended", - laser_488_pct_now: 5.0, - phases: [ - { mode: "base", start: 0, end: 8100, cadence_s: 120 } - ], - trigger_events: [], - power_history_488: [ - { at: 0, pct: 5.0 }, - { at: 8100, pct: 5.0 } - ], - // Projected dose-exhaust horizon = 4.0h from now (warning condition) - projected_cadence_s: 120, - projected_end_s: null, - dose_exhaust_at_s: 12000 // budget will run out at this elapsed time - }, - { - id: "C1", role: "calibration", color: "#22d3ee", icon: "◆", - dose_used_ms: 33500, dose_budget_ms: 500000, // 10× multiplier - tp_acquired: 67, - stop_condition: "manual", - stop_kind: "open_ended", - laser_488_pct_now: 5.0, - phases: [ - { mode: "base", start: 0, end: 8100, cadence_s: 120 } - ], - trigger_events: [], - power_history_488: [ - { at: 0, pct: 5.0 }, - { at: 8100, pct: 5.0 } - ], - projected_cadence_s: 120, - projected_end_s: null - } - ] -}; const ExperimentOverview = { initialized: false, expandedMode: null, activeView: 'overview', // 'overview' | 'rules' - activeStrategy: null, // last fetched/loaded snapshot - isLive: false, // true when activeStrategy came from the API + activeStrategy: null, // last fetched strategy snapshot (rules view) + activePlan: null, // last fetched/loaded operation plan (overview spine) + isLive: false, // true when data came from the API + scenarioMode: false, // true when ?scenario= is active + _subscribed: false, // guard: prevents double-registration across tab re-clicks + _planRefreshTimer: null, // debounce handle for tactic-event-driven refetch + _tempUpdateHandler: null,// stored handler ref so it can be off()'d if needed + _rosterEmbryos: [], // embryos from /api/embryos/positions (D2 roster lens) + _rolesMap: null, // Map(role name → registry obj) from /api/roles (D2 roster lens) + _currentSessionId: null, // session_id from /api/operation_plan/current (always, even idle) + _planPickerOpen: false, // whether the plan-link picker is visible + _pickerItems: null, // flat plan items for the picker (null=not loaded) + _expandedTacticIds: new Set(), // tactic expand-keys for click-to-expand (survives refresh) async init() { console.log('[ExperimentOverview] init() called, view=', this.activeView); - const strategy = await this.loadStrategy(); + + // Scenario dev mode: ?scenario= renders a fixture with no fetch. + // Guard against double-registration when the tab is clicked repeatedly. + const scenarioParam = new URLSearchParams(location.search).get('scenario'); + if (scenarioParam && window.OPERATIONS_SCENARIOS && + Object.prototype.hasOwnProperty.call(window.OPERATIONS_SCENARIOS, scenarioParam)) { + this.scenarioMode = true; + this.activePlan = window.OPERATIONS_SCENARIOS[scenarioParam]; + this.activeStrategy = null; + this.isLive = false; + this.render(null); + this.initialized = true; + return; + } + + this.scenarioMode = false; + // Fetch plan (overview) and strategy (rules) in parallel so tab-switching + // between the two views doesn't require a second round-trip. + // D2: also fetch roster + roles for the roster lens. + const [plan, strategy, rosterEmbryos, rolesMap] = await Promise.all([ + this.loadPlan(), + this.loadStrategy(), + this._loadRoster(), + this._loadRolesMap(), + ]); + this.activePlan = plan; this.activeStrategy = strategy; + this._rosterEmbryos = rosterEmbryos; + this._rolesMap = rolesMap; + this.isLive = plan !== null || strategy !== null; this.render(strategy); this.initialized = true; + + // Subscribe to tactic-state events once per page load. + // Guard: _subscribed prevents double-registration across tab re-clicks. + // Skip entirely in scenario mode — no live backend, no websocket. + if (!this._subscribed) { + this._subscribed = true; + const refresh = () => this._debouncedRefresh(); + // Plan-changing events: re-fetch the whole plan after debounce. + // CONTEXT_UPDATED fires when OperationPlanUpdater patches the plan. + // The tactic-lifecycle events fire on transitions the updater also + // reacts to, so they all funnel into the same debounced refetch. + const TACTIC_EVENTS = [ + 'CONTEXT_UPDATED', + 'TEMP_PROTOCOL_STARTED', 'TEMP_PROTOCOL_COMPLETED', + 'BURST_START', 'BURST_COMPLETE', + 'EMBRYO_CADENCE_CHANGED', 'TEMPERATURE_SETPOINT_CHANGED', + 'POWER_RAMP_STEP', + ]; + TACTIC_EVENTS.forEach(ev => ClientEventBus.on(ev, refresh)); + + // High-frequency temperature binding (~1 Hz). + // Updates the active scripted_protocol tactic's temperature gauge + // IN PLACE — no plan refetch, no full re-render. + this._tempUpdateHandler = (data) => this._handleTempUpdate(data); + ClientEventBus.on('TEMPERATURE_UPDATE', this._tempUpdateHandler); + } }, async loadStrategy() { @@ -164,26 +94,98 @@ const ExperimentOverview = { cache: 'no-store' }); if (!resp.ok) { - console.warn( - '[ExperimentOverview] strategy fetch returned', - resp.status, '- falling back to stub' - ); - this.isLive = false; - return STUB_STRATEGY; + // No active experiment / not ready yet — show the empty state, + // never stubbed data. + console.warn('[ExperimentOverview] strategy fetch returned', resp.status); + return null; } const data = await resp.json(); - this.isLive = true; return data; } catch (e) { - console.warn( - '[ExperimentOverview] strategy fetch error - falling back to stub:', - e - ); - this.isLive = false; - return STUB_STRATEGY; + console.warn('[ExperimentOverview] strategy fetch error:', e); + return null; } }, + // Fetch the agent-authored Operation Plan for the current session. + // Returns the plan object (plan.tactics etc.) or null when unavailable. + // Always captures data.session_id in _currentSessionId so the Linked-plans + // panel can work even when no operation plan is active. + async loadPlan() { + try { + const resp = await fetch('/api/operation_plan/current', { cache: 'no-store' }); + if (!resp.ok) { + console.warn('[ExperimentOverview] plan fetch returned', resp.status); + return null; + } + const data = await resp.json(); + // Capture session_id regardless of plan availability — used by the + // Linked-plans panel which is session-scoped, not plan-scoped. + this._currentSessionId = data.session_id || null; + if (!data.available) return null; + return data.plan || null; + } catch (e) { + console.warn('[ExperimentOverview] plan fetch error:', e); + return null; + } + }, + + // Debounced plan refetch — coalesces rapid tactic-event bursts into a single + // fetch+render. 500 ms window matches experiment-strip.js convention. + // D2: also re-fetches the embryo roster so the lens stays current. + _debouncedRefresh() { + if (this._planRefreshTimer) clearTimeout(this._planRefreshTimer); + this._planRefreshTimer = setTimeout(async () => { + this._planRefreshTimer = null; + const [plan, rosterEmbryos] = await Promise.all([ + this.loadPlan(), + this._loadRoster(), + ]); + this.activePlan = plan; + this._rosterEmbryos = rosterEmbryos; + this.isLive = plan !== null; + this.render(this.activeStrategy); + }, 500); + }, + + // In-place temperature gauge update — called at ~1 Hz by TEMPERATURE_UPDATE. + // Finds the active scripted_protocol tactic's temperature readout in the DOM + // and rewrites only that element's value, never refetching the plan. + // No-op when there is no active scripted_protocol tactic with temperature binding. + _handleTempUpdate(data) { + if (!data || !data.sample) return; + const plan = this.activePlan; + if (!plan || !Array.isArray(plan.tactics)) return; + // Only act when an active scripted_protocol tactic declares temperature binding. + const activeTactic = plan.tactics.find( + t => t.state === 'active' + && t.kind === 'scripted_protocol' + && Array.isArray(t.live_bind) + && t.live_bind.includes('temperature') + ); + if (!activeTactic) return; + + const root = document.getElementById('experiment-overview-root'); + if (!root) return; + // _renderOpsReadout stamps data-livebind="temperature" on the gauge div + // when the readout label normalises to "temperature". + const gauge = root.querySelector('.ops-node.active .ops-gauge[data-livebind="temperature"]'); + if (!gauge) return; + const gv = gauge.querySelector('.ops-gv'); + if (!gv) return; + + const s = data.sample; + const water = s.water_c != null + ? parseFloat(s.water_c).toFixed(1) + '°C' + : '—'; + const sp = s.setpoint_c != null + ? ' → ' + + parseFloat(s.setpoint_c).toFixed(1) + + '°C' + : ''; + gv.innerHTML = water + sp; + }, + setView(view) { if (view === this.activeView) return; this.activeView = view; @@ -193,7 +195,7 @@ const ExperimentOverview = { }); // Re-render against the last fetched strategy (no re-fetch on tab // switch — refresh happens on tab activation in the bootstrap). - this.render(this.activeStrategy || STUB_STRATEGY); + this.render(this.activeStrategy); }, render(s) { @@ -204,13 +206,32 @@ const ExperimentOverview = { } // Tear down any prior ticker before we blow away the SVG it pointed at. this._stopNowTicker(); + // Reset plan-picker state on each full render so the picker doesn't persist + // across tactic-event-driven re-renders. + this._planPickerOpen = false; + this._pickerItems = null; + // Rules view requires the strategy snapshot; show an empty state when absent. + // Overview view uses this.activePlan — the null/empty case is handled inside + // _renderOperationSpine (it renders the idle state). + if (this.activeView === 'rules' && !s) { + root.innerHTML = '
    ' + + 'No active experiment — rules and monitoring modes will appear here once a run is live.
    '; + return; + } try { root.innerHTML = ''; if (this.activeView === 'rules') { this._renderRulesView(root, s); } else { - this._renderOverviewView(root, s); - this._startNowTicker(); + // Operation spine — data-driven tactic plan renderer. + // The swimlane view is retired; this renders this.activePlan. + this._renderOperationSpine(root, this.activePlan); + } + // Kick off the async Linked-plans panel (overview tab only). + // Fire-and-forget: appends a placeholder immediately, fills after fetch. + if (this.activeView === 'overview') { + this._initLinkedPlansPanel(root).catch(e => + console.warn('[ExperimentOverview] linked-plans panel error:', e)); } console.log('[ExperimentOverview] rendered OK, view=', this.activeView); } catch (err) { @@ -222,20 +243,6 @@ const ExperimentOverview = { } }, - // The "now" marker advances with wall-clock time and shows a countdown to - // the next base-interval acquisition. We update only the marker group's - // transform + the chip text, never re-rendering the whole SVG. Tick rate - // is ~4 Hz which keeps the line motion visibly smooth without burning - // cycles. Skipped while the tab is hidden. - _startNowTicker() { - this._stopNowTicker(); - const tick = () => { - if (!this._nowTickerCtx) return; - if (!document.hidden) this._updateNowMarker(); - this._nowTickerHandle = setTimeout(tick, 250); - }; - this._nowTickerHandle = setTimeout(tick, 250); - }, _stopNowTicker() { if (this._nowTickerHandle) { @@ -244,64 +251,256 @@ const ExperimentOverview = { } }, - _updateNowMarker() { - const ctx = this._nowTickerCtx; - if (!ctx || !ctx.marker.isConnected) return; - const elapsedRealS = (Date.now() - ctx.renderedAtMs) / 1000; - const effOffsetS = Math.min( - ctx.renderedOffsetS + elapsedRealS, - ctx.horizonS - ); - const x = ctx.xForT(effOffsetS); - ctx.marker.setAttribute('transform', `translate(${x},0)`); - - // Wall-clock from session-anchored time so the line and the clock - // can't drift apart even if the client clock is wrong. - const wallMs = ctx.startedAtMs + effOffsetS * 1000; - const d = new Date(wallMs); - const hh = String(d.getHours()).padStart(2, '0'); - const mm = String(d.getMinutes()).padStart(2, '0'); - const ss = String(d.getSeconds()).padStart(2, '0'); - - let label = `${hh}:${mm}:${ss}`; - if (ctx.baseIntervalS > 0) { - const nextTickS = Math.ceil(effOffsetS / ctx.baseIntervalS) * ctx.baseIntervalS; - const remainS = Math.max(0, Math.round(nextTickS - effOffsetS)); - const rm = Math.floor(remainS / 60); - const rs = String(remainS % 60).padStart(2, '0'); - label += ` · next ${rm}:${rs}`; + + // ================================================================= + // Linked-plans panel — session ↔ plan items (F / Task 4) + // Symmetric with the Plans-tab Sessions section (campaigns.js). + // Endpoints: GET /api/sessions/{id}/plans + // POST /api/campaigns/{cid}/items/{iid}/sessions + // DELETE .../sessions/{session_id} + // ================================================================= + + // Initialise and append the Linked-plans panel to the ops-wrap in root. + // Async: appends a loading placeholder immediately, fills after fetch. + // No-op when no session_id is known (e.g. store not yet initialised). + async _initLinkedPlansPanel(root) { + if (!this._currentSessionId) return; + const wrap = root.querySelector('.ops-wrap'); + if (!wrap) return; + + // Append placeholder before the async fetch so layout is stable. + const panelEl = document.createElement('div'); + panelEl.className = 'ops-lp'; + panelEl.innerHTML = '
    Loading linked plans…
    '; + wrap.appendChild(panelEl); + + const sid = this._currentSessionId; + try { + const [linkedData, campaignsData] = await Promise.all([ + fetch(`/api/sessions/${encodeURIComponent(sid)}/plans`, { cache: 'no-store' }) + .then(r => r.ok ? r.json() : { plans: [] }) + .catch(() => ({ plans: [] })), + fetch('/api/campaigns', { cache: 'no-store' }) + .then(r => r.ok ? r.json() : { campaigns: [] }) + .catch(() => ({ campaigns: [] })), + ]); + const plans = linkedData.plans || []; + const campaignNameMap = {}; + this._flattenCampaignNames(campaignsData.campaigns || [], campaignNameMap); + this._fillLinkedPlansPanel(panelEl, plans, campaignNameMap, sid); + } catch (e) { + console.warn('[ExperimentOverview] linked-plans init error:', e); + panelEl.innerHTML = '
    Could not load linked plans.
    '; } - ctx.chipText.textContent = label; - - // Size the chip to fit; flip to the left of the line if we're near - // the right edge so it stays on-screen. - const textLen = label.length * 6.2 + 12; - const nearEnd = x + textLen + 8 > ctx.laneRight; - if (nearEnd) { - ctx.chipBg.setAttribute('x', -textLen - 4); - ctx.chipBg.setAttribute('width', textLen); - ctx.chipText.setAttribute('x', -textLen + 2); + }, + + // Build a map of campaign_id → display name from the /api/campaigns tree. + _flattenCampaignNames(trees, map) { + const walk = (tree) => { + const c = tree.campaign; + if (c && c.id) map[c.id] = c.description || c.shorthand || c.id; + for (const child of (tree.children || [])) walk(child); + }; + for (const tree of (trees || [])) walk(tree); + }, + + // Build a flat list of {id, title, status, campaign_id, campaign_name} + // from the /api/campaigns tree, for the plan-item picker. + _flattenCampaignItems(trees, campaignNameMap) { + const items = []; + const walk = (tree, inheritedCid, inheritedName) => { + const c = tree.campaign; + const cid = (c && c.id) || inheritedCid; + const name = (c && (c.description || c.shorthand)) || inheritedName || cid; + for (const item of (tree.items || [])) { + items.push({ + id: item.id, + title: item.title || item.id, + status: typeof item.status === 'string' ? item.status + : (item.status && item.status.value) || 'planned', + campaign_id: cid, + campaign_name: name, + }); + } + for (const child of (tree.children || [])) walk(child, cid, name); + }; + for (const tree of (trees || [])) walk(tree, null, null); + return items; + }, + + // Render the linked-plans panel HTML into panelEl and wire button events. + _fillLinkedPlansPanel(panelEl, plans, campaignNameMap, sessionId) { + const ESC = this._opsESC.bind(this); + + // Header: section label + "+ link to a plan" button + let html = `
    + Linked plans + +
    `; + + // Linked plan-item rows (title · campaign · status · delink) + if (plans.length > 0) { + html += '
    '; + for (const p of plans) { + const cname = campaignNameMap[p.campaign_id] || p.campaign_id || '—'; + const sCls = p.status === 'completed' ? 'done' + : p.status === 'in_progress' ? 'active' : 'planned'; + html += `
    + ${ESC(p.title || p.id)} + ${ESC(cname)} + ${ESC(p.status || 'planned')} + +
    `; + } + html += '
    '; } else { - ctx.chipBg.setAttribute('x', 4); - ctx.chipBg.setAttribute('width', textLen); - ctx.chipText.setAttribute('x', 10); + html += '
    Not linked to any plan
    '; + } + + // Inline picker — shown when _planPickerOpen is set + if (this._planPickerOpen) { + if (this._pickerItems === null) { + // Still fetching — show loading state + html += '
    Loading plan items…
    '; + } else { + const linkedIds = new Set(plans.map(p => p.id)); + const available = this._pickerItems.filter(it => !linkedIds.has(it.id)); + const opts = available.length === 0 + ? '' + : available.map(it => + `` + ).join(''); + html += `
    + +
    + + +
    +
    `; + } + } + + panelEl.innerHTML = html; + + // Wire events directly on the rendered buttons. + const sid = sessionId; + const linkBtn = panelEl.querySelector('#ops-lp-link-btn'); + if (linkBtn) { + linkBtn.addEventListener('click', () => + this._openPlanPickerInPanel(panelEl, plans, campaignNameMap, sid)); + } + panelEl.querySelectorAll('.ops-lp-delink').forEach(btn => { + btn.addEventListener('click', () => + this._delinkPlanItem(panelEl, btn.dataset.itemId, btn.dataset.campaignId, sid)); + }); + const submitBtn = panelEl.querySelector('#ops-lp-picker-link'); + if (submitBtn) { + submitBtn.addEventListener('click', () => this._submitPlanLink(panelEl, sid)); + } + const cancelBtn = panelEl.querySelector('#ops-lp-picker-cancel'); + if (cancelBtn) { + cancelBtn.addEventListener('click', () => { + this._planPickerOpen = false; + this._pickerItems = null; + this._fillLinkedPlansPanel(panelEl, plans, campaignNameMap, sid); + }); } }, - _renderOverviewView(root, s) { - root.appendChild(this._renderHeader(s)); - root.appendChild(this._renderModes(s)); - root.appendChild(this._renderModeExpanded(s)); - root.appendChild(this._renderSwimlanes(s)); + // Open the inline plan-item picker: set loading state, fetch /api/campaigns, + // flatten to items, re-render with the picker populated. + async _openPlanPickerInPanel(panelEl, plans, campaignNameMap, sessionId) { + this._planPickerOpen = true; + this._pickerItems = null; // show loading + this._fillLinkedPlansPanel(panelEl, plans, campaignNameMap, sessionId); + try { + const res = await fetch('/api/campaigns', { cache: 'no-store' }); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + const data = await res.json(); + this._pickerItems = this._flattenCampaignItems(data.campaigns || [], campaignNameMap); + } catch (err) { + console.error('[ExperimentOverview] plan picker fetch error:', err); + this._pickerItems = []; + } + if (this._planPickerOpen) { + this._fillLinkedPlansPanel(panelEl, plans, campaignNameMap, sessionId); + } + }, + + // POST the selected plan item → session link, then refetch and re-render. + async _submitPlanLink(panelEl, sessionId) { + const select = panelEl.querySelector('#ops-lp-picker-sel'); + const value = select && select.value; + if (!value || !value.includes('::')) return; + const [campaignId, itemId] = value.split('::', 2); + if (!campaignId || !itemId) return; + + this._planPickerOpen = false; + this._pickerItems = null; + panelEl.innerHTML = '
    Linking…
    '; + try { + const res = await fetch( + `/api/campaigns/${encodeURIComponent(campaignId)}/items/${encodeURIComponent(itemId)}/sessions`, + { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ session_id: sessionId }), + }, + ); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + } catch (err) { + console.error('[ExperimentOverview] plan link error:', err); + } + await this._refetchLinkedPlans(panelEl, sessionId); + }, + + // DELETE the plan-item → session edge, then refetch and re-render. + async _delinkPlanItem(panelEl, itemId, campaignId, sessionId) { + panelEl.innerHTML = '
    Unlinking…
    '; + try { + const res = await fetch( + `/api/campaigns/${encodeURIComponent(campaignId)}/items/${encodeURIComponent(itemId)}/sessions/${encodeURIComponent(sessionId)}`, + { method: 'DELETE' }, + ); + if (!res.ok) throw new Error(`HTTP ${res.status}`); + } catch (err) { + console.error('[ExperimentOverview] plan delink error:', err); + } + await this._refetchLinkedPlans(panelEl, sessionId); + }, + + // Refetch linked plans + campaigns after a link/delink op, then re-render. + async _refetchLinkedPlans(panelEl, sessionId) { + try { + const [linkedData, campaignsData] = await Promise.all([ + fetch(`/api/sessions/${encodeURIComponent(sessionId)}/plans`, { cache: 'no-store' }) + .then(r => r.ok ? r.json() : { plans: [] }) + .catch(() => ({ plans: [] })), + fetch('/api/campaigns', { cache: 'no-store' }) + .then(r => r.ok ? r.json() : { campaigns: [] }) + .catch(() => ({ campaigns: [] })), + ]); + const plans = linkedData.plans || []; + const campaignNameMap = {}; + this._flattenCampaignNames(campaignsData.campaigns || [], campaignNameMap); + this._fillLinkedPlansPanel(panelEl, plans, campaignNameMap, sessionId); + } catch (e) { + console.warn('[ExperimentOverview] linked-plans refetch error:', e); + panelEl.innerHTML = '
    Could not reload linked plans.
    '; + } }, + + _renderRulesView(root, s) { // Compact header echoing the session identity const header = el('div', 'expov-header'); const metaRow = el('div', 'expov-header-row expov-header-row-meta'); metaRow.appendChild(elText('span', 'expov-session-name', s.session_name)); metaRow.appendChild(elText('span', 'expov-session-id', s.session_id)); - metaRow.appendChild(elText('span', 'expov-mockup-badge', 'mockup · stubbed data')); header.appendChild(metaRow); root.appendChild(header); @@ -313,52 +512,6 @@ const ExperimentOverview = { root.appendChild(this._renderRulesTable(s)); }, - // ----------------------------------------------------------------- - // Header — session identification + key metrics strip - // (page-level title lives in .experiment-header-bar above) - // ----------------------------------------------------------------- - _renderHeader(s) { - const elapsedH = Math.floor(s.now_offset_s / 3600); - const elapsedM = Math.floor((s.now_offset_s % 3600) / 60); - const wrap = el('div', 'expov-header'); - - // Session identification — the navbar already carries the id on - // every tab, so we only render a name line when it actually adds - // info (i.e. a human label, not a hash). The data-source badge is - // still useful and gets its own row so it stays visible. - const metaRow = el('div', 'expov-header-row expov-header-row-meta'); - if (s.session_name && s.session_name !== s.session_id) { - metaRow.appendChild(elText('span', 'expov-session-name', s.session_name)); - } - if (this.isLive) { - metaRow.appendChild(elText('span', 'expov-live-badge', 'live')); - } else { - metaRow.appendChild(elText('span', 'expov-mockup-badge', 'mockup · stubbed data')); - } - wrap.appendChild(metaRow); - - // Compact key-metric strip - const roleCounts = {}; - s.embryos.forEach(e => { roleCounts[e.role] = (roleCounts[e.role] || 0) + 1; }); - const roleStr = Object.entries(roleCounts).map(([r, n]) => `${n} ${r}`).join(' · '); - const metricsRow = el('div', 'expov-header-row expov-header-row-metrics'); - const metric = (label, val) => { - const m = el('span', 'expov-metric'); - m.appendChild(elText('span', 'expov-metric-val', val)); - m.appendChild(elText('span', 'expov-metric-lbl', label)); - return m; - }; - metricsRow.appendChild(metric('elapsed', `${elapsedH}h ${elapsedM}m`)); - metricsRow.appendChild(metric('base', `${s.base_interval_s}s`)); - const budgetText = (s.dose_budget_base_ms != null && isFinite(s.dose_budget_base_ms)) - ? `${(s.dose_budget_base_ms / 1000).toFixed(0)}s × role` - : 'no limit'; - metricsRow.appendChild(metric('budget', budgetText)); - metricsRow.appendChild(metric('embryos', `${s.embryos.length} · ${roleStr}`)); - wrap.appendChild(metricsRow); - - return wrap; - }, // ----------------------------------------------------------------- // Monitoring mode chips + expanded panel @@ -424,803 +577,653 @@ const ExperimentOverview = { return name.split('_').map(w => w.charAt(0).toUpperCase() + w.slice(1)).join(' '); }, - // ----------------------------------------------------------------- - // Legend - // ----------------------------------------------------------------- - _renderLegend() { - const wrap = el('div', 'expov-legend'); - const items = [ - ['base', 'base cadence'], - ['fast', 'fast cadence'], - ['burst', 'burst window'], - ['cooldown', 'cooldown'], - ['paused', 'paused'], - ]; - items.forEach(([cls, label]) => { - const item = el('span', 'expov-legend-item'); - item.appendChild(elClass('span', `expov-legend-swatch ${cls}`)); - item.appendChild(elText('span', '', label)); - wrap.appendChild(item); - }); - const projItem = el('span', 'expov-legend-item'); - projItem.appendChild(elClass('span', 'expov-legend-swatch projected')); - projItem.appendChild(elText('span', '', 'projected')); - wrap.appendChild(projItem); - - const glyphs = [ - ['◇', 'trigger fired'], - ['●', 'now'], - ['■', 'stop condition'], - ['∞', 'open-ended'], - ['▲', 'burst start'], - ['⚠', 'budget warning'] - ]; - glyphs.forEach(([g, label]) => { - const item = el('span', 'expov-legend-item'); - item.appendChild(elText('span', 'expov-legend-glyph', g)); - item.appendChild(elText('span', '', label)); - wrap.appendChild(item); - }); - return wrap; - }, - // ----------------------------------------------------------------- - // Swimlanes SVG — the main visualization - // ----------------------------------------------------------------- - _renderSwimlanes(s) { - const wrap = el('div', 'expov-swimlanes-wrap'); - - // Compact inline legend above the SVG - const legend = el('div', 'expov-mini-legend'); - const swatches = [ - ['base', 'base'], - ['fast', 'fast'], - ['burst', 'burst'], - ['cooldown', 'cooldown'] - ]; - swatches.forEach(([k, label]) => { - const item = el('span', 'expov-mini-legend-item'); - const sw = el('span', `expov-mini-legend-swatch ${k}`); - item.appendChild(sw); - item.appendChild(elText('span', '', label)); - legend.appendChild(item); - }); - const projItem = el('span', 'expov-mini-legend-item'); - projItem.appendChild(elClass('span', 'expov-mini-legend-swatch projected')); - projItem.appendChild(elText('span', '', 'projected')); - legend.appendChild(projItem); - wrap.appendChild(legend); - - // Layout constants (logical pixels in the SVG viewBox) - const LEFT = 180; // label gutter - const RIGHT = 80; // right gutter for stop icon + ∞ - const LANE_W = 900; // lane drawing area - const W = LEFT + LANE_W + RIGHT; - - const ROW_H = 100; // per-embryo row total height - const LANE_H = 28; // cadence lane height - const POWER_H = 22; // power strip height - const DOSE_H = 12; // dose gauge height - const ROW_PAD = 14; // top padding inside row - const TOP_AXIS_H = 36; // top axis area (time labels + wall-clock) - const BOTTOM_PAD = 8; - - const rows = s.embryos.length; - const H = TOP_AXIS_H + rows * ROW_H + BOTTOM_PAD; - - const svg = svgEl('svg', { - class: 'expov-swimlanes-svg', - viewBox: `0 0 ${W} ${H}`, - preserveAspectRatio: 'xMinYMin meet' - }); + // ================================================================= + // Operation Spine — data-driven plan renderer (replaces swimlanes) + // ================================================================= - // Time scale helpers - const xForT = (t) => LEFT + (t / s.horizon_s) * LANE_W; - const nowX = xForT(s.now_offset_s); - - // ----- top axis: hour ticks with wall-clock annotation - const startedAt = new Date(s.started_at); - const axisG = svgEl('g'); - for (let h = 0; h <= Math.ceil(s.horizon_s / 3600); h++) { - const x = xForT(h * 3600); - axisG.appendChild(svgEl('line', { - x1: x, x2: x, y1: TOP_AXIS_H - 6, y2: H - BOTTOM_PAD, - class: 'expov-svg-axis', 'stroke-opacity': h === 0 ? 0.55 : 0.12 - })); - axisG.appendChild(svgEl('text', { - x: x + 4, y: 12, class: 'expov-svg-axis-label' - }, `+${h}h`)); - // Wall-clock subtitle - const wallClock = new Date(startedAt.getTime() + h * 3600 * 1000); - const hh = String(wallClock.getHours()).padStart(2, '0'); - const mm = String(wallClock.getMinutes()).padStart(2, '0'); - axisG.appendChild(svgEl('text', { - x: x + 4, y: 22, - class: 'expov-svg-axis-wallclock' - }, `${hh}:${mm}`)); + // Minimal HTML escaper — values in readouts may contain trusted HTML + // (e.g. 32.0°C) so they are rendered with + // innerHTML; all other user/model strings go through _opsESC. + _opsESC(s) { + return String(s == null ? '' : s) + .replace(/[&<>"]/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c])); + }, + + // Entry point: render the operation spine into `root`. + // plan = the plan object (tactics array) or null for idle/unavailable. + _renderOperationSpine(root, plan) { + const ESC = this._opsESC.bind(this); + + if (!plan || !Array.isArray(plan.tactics) || plan.tactics.length === 0) { + root.innerHTML = ` +
    +
    Operations
    +

    No operation running

    +
    Brief the agent — it will declare a tactic plan and the spine renders live.
    +
    + +
    — or start from a template
    +
    + + + +
    +
    +
    `; + + // Wire CTA buttons — same open+send pattern as landing.js sendFreeform. + function _opsOpenAgent(prompt) { + if (typeof AgentChat === 'undefined' || !AgentChat.togglePanel) return; + AgentChat.togglePanel(true); + if (prompt && AgentChat.runCommand) setTimeout(() => AgentChat.runCommand(prompt), 300); + } + const briefBtn = root.querySelector('[data-ops-brief]'); + if (briefBtn) briefBtn.addEventListener('click', () => _opsOpenAgent('')); + root.querySelectorAll('[data-ops-prompt]').forEach(chip => { + chip.addEventListener('click', () => _opsOpenAgent(chip.dataset.opsPrompt)); + }); + return; } - svg.appendChild(axisG); - - // ----- "now" marker: translatable group containing the vertical line - // and a live clock chip. The chip advances every tick and shows the - // countdown to the next base-interval acquisition window. The group - // gets translated by _tickNow, so we don't rebuild SVG every second. - const nowMarker = svgEl('g', { class: 'expov-svg-now-marker' }); - nowMarker.appendChild(svgEl('line', { - x1: 0, x2: 0, y1: TOP_AXIS_H - 4, y2: H - BOTTOM_PAD, - class: 'expov-svg-now-line' - })); - // Chip sits just below the axis labels (which live at y=12 and y=22) - // so it doesn't sit on top of the "+0h / wallclock" annotation when - // the now-line is near the start of the timeline. - const chipBg = svgEl('rect', { - x: 4, y: TOP_AXIS_H - 6, width: 120, height: 14, rx: 7, - class: 'expov-svg-now-chip-bg' - }); - const chipText = svgEl('text', { - x: 10, y: TOP_AXIS_H + 4, class: 'expov-svg-now-label' - }, ''); - nowMarker.appendChild(chipBg); - nowMarker.appendChild(chipText); - svg.appendChild(nowMarker); - - // Stash the bits the ticker needs to update without re-rendering. - this._nowTickerCtx = { - marker: nowMarker, - chipBg: chipBg, - chipText: chipText, - xForT, - startedAtMs: new Date(s.started_at).getTime(), - renderedAtMs: Date.now(), - renderedOffsetS: s.now_offset_s, - baseIntervalS: s.base_interval_s || 0, - horizonS: s.horizon_s, - laneLeft: LEFT, - laneRight: LEFT + LANE_W, - }; - this._updateNowMarker(); - - // ----- one group per embryo - s.embryos.forEach((emb, i) => { - const rowTop = TOP_AXIS_H + i * ROW_H; - const rowG = svgEl('g'); - rowG.appendChild(this._renderLaneRow(s, emb, { - LEFT, LANE_W, RIGHT, W, ROW_H, LANE_H, POWER_H, DOSE_H, ROW_PAD, - TOP_AXIS_H, rowTop, xForT, nowX - })); - svg.appendChild(rowG); - }); - wrap.appendChild(svg); - return wrap; + const tactics = plan.tactics; + const hasActive = tactics.some(t => t.state === 'active'); + // Index of the first queued (planned) tactic — gets the "next" badge. + const firstPlannedIdx = tactics.findIndex(t => t.state === 'planned'); + + // Roster lens: visible when embryos + role metadata are both available. + // Gracefully absent if either fetch failed or returned nothing (backward compat + // with plans that pre-date D2 — spine renders exactly as before). + const rosterEmbryos = this._rosterEmbryos || []; + const rolesMap = this._rolesMap; + const rosterCtx = (rosterEmbryos.length > 0 && rolesMap && rolesMap.size > 0) + ? { embryos: rosterEmbryos, rolesMap } + : null; + const rosterHtml = rosterCtx + ? ` + ${this._renderRosterLens(rosterEmbryos, rolesMap, plan, ESC)}` + : ''; + const spineLabel = rosterCtx + ? '' + : ''; + + const spineNodes = tactics + .map((t, idx) => this._renderOpsTactic(t, idx, firstPlannedIdx, ESC, rosterCtx, tactics)) + .join(''); + + root.innerHTML = ` +
    +
    Operations · ${hasActive ? 'live' : 'idle'}
    +

    ${ESC(plan.title || '')}

    +
    ${ESC(plan.session_id || '')}${plan.goal ? ' · ' + ESC(plan.goal) : ''}
    +
    + done + in use + queued +
    + ${rosterHtml} + ${spineLabel} +
    ${spineNodes}
    +
    `; + this._wireSpineExpand(root); }, - _renderLaneRow(s, emb, dim) { - const { LEFT, LANE_W, W, ROW_H, LANE_H, POWER_H, DOSE_H, ROW_PAD, - rowTop, xForT, nowX } = dim; - const g = svgEl('g'); - - // Hairline divider above each row (except first) - if (rowTop > dim.TOP_AXIS_H) { - g.appendChild(svgEl('line', { - x1: 8, x2: W - 8, y1: rowTop, y2: rowTop, - class: 'expov-svg-row-divider' - })); + // Render a single tactic node. + // rosterCtx = { embryos, rolesMap } | null — when present, adds role-scope badge (D2). + // tactics = full tactics array — used to resolve relation ids to names. + _renderOpsTactic(t, idx, firstPlannedIdx, ESC, rosterCtx = null, tactics = []) { + const STATE_LABEL = { done: 'done', active: 'in use', planned: 'queued', paused: 'paused' }; + const seq = String(t.seq || idx + 1).padStart(2, '0'); + const stateLabel = STATE_LABEL[t.state] || t.state; + // First queued tactic gets a "next" badge — COCKED instrument marker. + const isFirstQueued = t.state === 'planned' && idx === firstPlannedIdx; + const nextBadge = isFirstQueued + ? 'next' + : ''; + + const live = t.live || {}; + const target = live.target || ''; + const summary = live.summary || ''; + const desc = live.desc || ''; + + // Scope chip/badge — compact chip for planned/done/paused (always visible, no embryo + // list needed); full badge with embryo resolution for active state (D2 roster lens). + const isExpandable = (t.state === 'planned' || t.state === 'done' || t.state === 'paused'); + const expandKey = this._tacticExpandKey(t); + const scopeBadge = isExpandable + ? this._renderOpsScopeChip(t.scope, ESC) + : (rosterCtx ? this._renderOpsScopeBadge(t.scope, rosterCtx.embryos, rosterCtx.rolesMap, ESC) : ''); + + // Chevron toggle — only for expandable (planned / done / paused) tactics. + const chevron = isExpandable + ? `` + : ''; + + // Header row: name · target · scope badge · summary · chevron + let inner = ` +
    + ${ESC(t.name)} + ${target ? `${ESC(target)}` : ''} + ${scopeBadge} + ${summary ? `${ESC(summary)}` : ''} + ${chevron} +
    + ${desc ? `
    ${ESC(desc)}
    ` : ''}`; + + // AUDIT: FLATTEN the active card — readouts on the panel face, separated by + // a hairline rule. No nested card-in-card boxes. + if (t.state === 'active' && live.readouts && live.readouts.length) { + inner += `
    +
    + ${live.readouts.map(r => this._renderOpsReadout(r, ESC)).join('')} +
    `; } - // ---- Left label gutter --------------------------------------- - // Single header line + phase pill. Power/dose labels are at the - // y-position of their respective sub-rows, right-aligned in the gutter. - const labelY = rowTop + ROW_PAD + 12; - - // Header line: icon · ID · role (· 10× hint for calibration) - g.appendChild(svgEl('text', { - x: 14, y: labelY + 1, class: 'expov-svg-role-icon', - fill: emb.color - }, emb.icon)); - g.appendChild(svgEl('text', { - x: 32, y: labelY, class: 'expov-svg-label' - }, emb.id)); - // role tag — eyebrow above the id (avoids colliding with long ids) - g.appendChild(svgEl('text', { - x: 14, y: rowTop + ROW_PAD - 1, - class: 'expov-svg-role-tag' - }, emb.role)); - - // Phase pill: current mode at glance - const currentPhase = emb.phases[emb.phases.length - 1]; - const pillY = labelY + 7; - const pillH = 14; - const phaseLabel = currentPhase.mode === 'burst' - ? `BURST · ${currentPhase.hz}Hz` - : `${currentPhase.mode.toUpperCase()} · ${currentPhase.cadence_s}s`; - const pillW = Math.max(70, phaseLabel.length * 6 + 12); - const pillX = 32; - const phaseColors = { - base: '#6b7280', - fast: '#fb923c', - burst: '#ef4444', - cooldown: '#a78bfa', - paused: '#3b82f6' - }; - const pillFill = phaseColors[currentPhase.mode] || '#6b7280'; - g.appendChild(svgEl('rect', { - x: pillX, y: pillY, width: pillW, height: pillH, rx: 7, - fill: pillFill, 'fill-opacity': 0.22, - stroke: pillFill, 'stroke-opacity': 0.65, 'stroke-width': 1 - })); - g.appendChild(svgEl('text', { - x: pillX + pillW / 2, y: pillY + 10, - 'text-anchor': 'middle', - fill: pillFill, 'font-size': 9.5, 'font-weight': 700, - 'font-family': "'JetBrains Mono', monospace" - }, phaseLabel)); - - // Tiny tp annotation under the pill (no stop — that's at lane right edge) - g.appendChild(svgEl('text', { - x: 32, y: pillY + pillH + 12, - class: 'expov-svg-sublabel' - }, `${emb.tp_acquired} tp acquired`)); - - // ---- Cadence lane -------------------------------------------- - const laneY = rowTop + ROW_PAD; - const laneMid = laneY + LANE_H / 2; - const laneBottom = laneY + LANE_H; - - // Phases — solid colored rects, no ticks. Cadence is read from the - // phase pill in the gutter and the optional inline cadence label. - // Min 4px visual width so micro-phases (burst, cooldown) stay visible. - emb.phases.forEach(ph => { - const x0 = xForT(ph.start); - const x1Raw = xForT(ph.end); - const x1 = Math.max(x1Raw, x0 + 4); - const width = x1 - x0; - const cls = `expov-svg-phase-${ph.mode}`; - g.appendChild(svgEl('rect', { - x: x0, y: laneY, width, height: LANE_H, rx: 2, - class: cls - })); - // Cadence text inside the rect is intentionally omitted — the - // gutter pill (currentPhase) and the colored rect (mode) already - // convey it. Keep an inline label only for cooldown, which is a - // transient state the pill won't be showing. - if (ph.mode === 'cooldown' && ph.cadence_s && width >= 42) { - g.appendChild(svgEl('text', { - x: x0 + width / 2, y: laneMid + 3.5, - 'text-anchor': 'middle', - class: 'expov-svg-phase-label' - }, `${ph.cadence_s}s · cool`)); - } - // Burst: keep the bright block + balloon since it's the most - // attention-worthy event in the lane - if (ph.mode === 'burst') { - const bx = (xForT(ph.start) + xForT(ph.end)) / 2; - const balloonY = laneY - 12; - const halfW = 30; - g.appendChild(svgEl('rect', { - x: bx - halfW, y: balloonY - 10, width: halfW * 2, height: 12, rx: 3, - class: 'expov-svg-burst-balloon-bg' - })); - g.appendChild(svgEl('text', { - x: bx, y: balloonY - 1, - 'text-anchor': 'middle', - class: 'expov-svg-burst-label' - }, `${ph.frames}f · ${ph.hz}Hz`)); - g.appendChild(svgEl('line', { - x1: bx, x2: bx, y1: balloonY + 2, y2: laneY, - stroke: '#ef4444', 'stroke-width': 1, 'stroke-opacity': 0.7 - })); - } - }); + // Kind-specific structure for the active state. + if (t.state === 'active') { + inner += this._renderOpsKindActive(t, live, ESC); + } else if (t.state === 'planned') { + inner += this._renderOpsKindPlanned(t, ESC); + inner += this._renderOpsExpandBody(t, ESC, tactics); + } - // ---- Acquisition density heatmap ---------------------------- - // Instead of one hairline per acquisition (reads as a barcode), - // we encode acquisition density as a luminance gradient over - // the past portion of the lane: sparse = lane stays muted, - // dense = a brighter band. The eye reads acquisition rate as - // brightness — no discrete marks, no clutter. - // - // Counts are derived from phase cadence then rescaled to match - // the authoritative `tp_acquired`, so the gradient never lies - // about how many timepoints fired even when the backend phase - // history is stale or incorrect. - const tickEnd = s.now_offset_s; - const ackPhases = emb.phases - .map((ph, i) => ({ ph, i })) - .filter(({ ph }) => ph.mode !== 'burst' && ph.cadence_s); - const predicted = ackPhases.map(({ ph }) => { - const phEnd = Math.min(ph.end ?? tickEnd, tickEnd); - const dur = phEnd - ph.start; - return dur > 0 ? Math.max(1, Math.floor(dur / ph.cadence_s) + 1) : 0; - }); - const predictedTotal = predicted.reduce((a, b) => a + b, 0); - const actualTotal = Number.isFinite(emb.tp_acquired) - ? emb.tp_acquired : predictedTotal; - const scale = predictedTotal > 0 ? actualTotal / predictedTotal : 0; - const acquisitions = []; - ackPhases.forEach(({ ph }, idx) => { - const phEnd = Math.min(ph.end ?? tickEnd, tickEnd); - const dur = phEnd - ph.start; - if (dur <= 0) return; - const n = Math.max(1, Math.round(predicted[idx] * scale)); - for (let j = 0; j < n; j++) { - acquisitions.push(ph.start + (dur * (j + 0.5)) / n); + // Fix #1: surface flat live.* telemetry keys not covered by structured + // readouts/phases. Render for active (in-progress telemetry) and done + // (completion data such as sustained_hz, mp4_path, last_fired). + // Skip planned — no live data is bound yet. + if (t.state === 'active' || t.state === 'done') { + const SKIP = new Set(['readouts', 'phases', 'target', 'summary', 'desc']); + const flatEntries = Object.entries(live).filter(([k]) => !SKIP.has(k)); + if (flatEntries.length) { + const humanKey = k => k.replace(/_/g, ' '); + const pairs = flatEntries.map(([k, v]) => { + const vStr = v == null ? '—' : String(v); + return `${ESC(humanKey(k))}${ESC(vStr)}`; + }).join(''); + inner += `
    ${pairs}
    `; } - }); + } - if (acquisitions.length > 0 && tickEnd > 0) { - // Layer 1 (background): smoothed luminance gradient - // encoding overall acquisition density along the lane. - // The triangular kernel kills aliasing stripes caused by - // evenly-spaced acquisitions falling into bins. - const BINS = 64; - const binSec = tickEnd / BINS; - const raw = new Array(BINS).fill(0); - for (const t of acquisitions) { - const bin = Math.min(BINS - 1, Math.max(0, Math.floor(t / binSec))); - raw[bin] += 1; - } - const kernel = [1, 2, 3, 4, 5, 4, 3, 2, 1]; - const kSum = kernel.reduce((a, b) => a + b, 0); - const kOff = Math.floor(kernel.length / 2); - const density = new Array(BINS).fill(0); - for (let i = 0; i < BINS; i++) { - let acc = 0, w = 0; - for (let k = 0; k < kernel.length; k++) { - const j = i + k - kOff; - if (j < 0 || j >= BINS) continue; - acc += raw[j] * kernel[k]; - w += kernel[k]; - } - density[i] = w > 0 ? acc / w * (kSum / w) : 0; - } - const maxD = Math.max(...density, 1e-6); - const gradId = `expov-density-${(emb.id || 'e').replace(/\W+/g, '_')}-r${rowTop}`; - const grad = svgEl('linearGradient', { - id: gradId, x1: '0%', x2: '100%', y1: '0%', y2: '0%' - }); - for (let i = 0; i < BINS; i++) { - const intensity = density[i] / maxD; - // Lower ceiling than the heatmap-only version (0.22 vs - // 0.38) because the dots above will carry the per-event - // signal; the band just hints at rate. - const alpha = 0.03 + 0.22 * intensity; - grad.appendChild(svgEl('stop', { - offset: `${(i / (BINS - 1)) * 100}%`, - 'stop-color': '#ffffff', - 'stop-opacity': alpha.toFixed(3), - })); - } - g.appendChild(grad); - const pastW = xForT(tickEnd) - LEFT; - if (pastW > 0) { - g.appendChild(svgEl('rect', { - x: LEFT, y: laneY, - width: pastW, height: LANE_H, - fill: `url(#${gradId})`, - rx: 2, - 'pointer-events': 'none' - })); - } - // Layer 2 (foreground): one soft round dot per acquisition - // along the top edge of the lane — keeps the per-event - // temporal discreteness the heatmap alone hides. - const dotY = laneY + 2; - for (const t of acquisitions) { - const tx = xForT(t); - g.appendChild(svgEl('circle', { - cx: tx, cy: dotY, r: 1.3, - class: 'expov-svg-acq-dot' - })); - } + // Expand body for done + paused tactics (appended after live-facts). + if (t.state === 'done' || t.state === 'paused') { + inner += this._renderOpsExpandBody(t, ESC, tactics); } - // ---- Cadence-change markers --------------------------------- - // Where consecutive phases differ in cadence (or mode), drop a - // vertical divider across the lane and a "300→60s · T34" chip - // above so the change is named and time-stamped in the lane. - // Apply the same scale factor used for tick rendering so the - // T# stamps shown on diamonds and cadence chips agree with the - // visible tick density and the authoritative tp_acquired count. - // Otherwise the chip might say "T118" on an embryo where we - // only drew 54 ticks — visually contradictory. - const tpIndexAt = (atS) => { - let count = 0; - for (const ph of emb.phases) { - if (!ph.cadence_s) continue; - if (atS < ph.start) break; - const phEnd = Math.min(atS, ph.end ?? atS); - count += Math.floor((phEnd - ph.start) / ph.cadence_s) + 1; - } - return Math.max(1, Math.round(count * scale)); - }; - // Track placed chip x-positions to avoid stacking chips on top - // of one another (e.g. the burst-balloon already sits there). - const placedChipX = []; - const burstXs = emb.phases - .filter(p => p.mode === 'burst') - .map(p => xForT(p.start)); - const isCollision = (x) => { - const min = 70; // px buffer - if (burstXs.some(bx => Math.abs(bx - x) < min)) return true; - return placedChipX.some(px => Math.abs(px - x) < min); - }; - // Walk phases and detect transitions, but COLLAPSE consecutive - // identical (mode + cadence) phases so a row of redundant phase - // records doesn't generate redundant dividers/chips. - let prevEffective = emb.phases[0]; - for (let i = 1; i < emb.phases.length; i++) { - const curr = emb.phases[i]; - const prev = prevEffective; - const sameCadence = prev.cadence_s === curr.cadence_s; - const sameMode = prev.mode === curr.mode; - if (sameCadence && sameMode) { - continue; // collapse: prevEffective stays the same + const cardExpandAttr = isExpandable + ? ` data-tactic-expand-id="${ESC(expandKey)}"` + : ''; + + return ` +
    +
    ${seq} · ${stateLabel}${nextBadge ? ' ' + nextBadge : ''}
    +
    ${inner}
    +
    `; + }, + + // ----------------------------------------------------------------- + // Expandable tactic detail — click-to-expand for queued/done/paused + // ----------------------------------------------------------------- + + // Stable key used to persist expand state across re-renders. + _tacticExpandKey(t) { + return String(t.id || t.seq || t.name || ''); + }, + + // Compact inline scope chip — always shown in the collapsed header. + // Works without embryo list; uses this._rolesMap for role color if available. + _renderOpsScopeChip(scope, ESC) { + const rolesMap = this._rolesMap; + if (!scope || scope.mode === 'global') { + return 'global'; + } + if (scope.mode === 'role') { + const role = scope.role || ''; + const roleInfo = rolesMap ? rolesMap.get(role) : null; + const color = (roleInfo && roleInfo.ui_color) || '#8b949e'; + const bg = this._hexToRgba(color, 0.12); + const border = this._hexToRgba(color, 0.35); + return `role: ${ESC(role)}`; + } + if (scope.mode === 'embryos') { + const ids = (scope.embryo_ids || []).join(', '); + return `${ESC(ids) || '—'}`; + } + return ''; + }, + + // Expanded detail body — rationale, scope, structure, relations, live.readouts. + // Rendered hidden by default; _wireSpineExpand toggles the `hidden` class. + _renderOpsExpandBody(t, ESC, tactics) { + const rows = []; + + // Rationale + if (t.rationale) { + rows.push(`
    + rationale + ${ESC(t.rationale)} +
    `); + } + + // Scope — resolved readable form + const scope = t.scope; + if (scope) { + let scopeText = 'all embryos'; + if (scope.mode === 'role') { + scopeText = `role: ${scope.role || ''}`; + } else if (scope.mode === 'embryos') { + scopeText = (scope.embryo_ids || []).join(', ') || '—'; } - prevEffective = curr; - if (prev.mode === 'burst' || curr.mode === 'burst') continue; - const cx = xForT(curr.start); - if (cx > xForT(s.now_offset_s)) continue; - // Divider is cheap to keep even on collision; the chip is what - // crowds the space, so we skip just the chip when crowded. - g.appendChild(svgEl('line', { - x1: cx, x2: cx, y1: laneY, y2: laneBottom, - class: 'expov-svg-cadence-divider' - })); - if (isCollision(cx)) continue; - const tp = tpIndexAt(curr.start); - const prevS = prev.cadence_s ?? '?'; - const currS = curr.cadence_s ?? '?'; - const chipText = `${prevS}→${currS}s · T${tp}`; - const chipW = chipText.length * 5.6 + 10; - const chipY = laneY - 13; - g.appendChild(svgEl('rect', { - x: cx - chipW / 2, y: chipY, - width: chipW, height: 12, rx: 3, - class: 'expov-svg-cadence-chip-bg' - })); - g.appendChild(svgEl('text', { - x: cx, y: chipY + 9, - 'text-anchor': 'middle', - class: 'expov-svg-cadence-chip' - }, chipText)); - placedChipX.push(cx); + rows.push(`
    + scope + ${ESC(scopeText)} +
    `); } - // ---- Power-change chips ------------------------------------- - // Same visual language as cadence chips, but parked in a row - // above so the two encodings stack neatly when they happen at - // the same trigger. Each chip names the rule outcome - // ("488 ↓ 5%→3%") and the timepoint it landed at. - const placedPowerChipX = []; - const hist488 = emb.power_history_488 || []; - // Walk the history, collect actual transitions (pairs where pct - // changes), then cluster consecutive close ones so a multi-step - // ramp gets a single annotation. - const transitions = []; - for (let k = 0; k < hist488.length - 1; k++) { - const a = hist488[k]; - const b = hist488[k + 1]; - if (a.pct === b.pct) continue; - transitions.push({ from: a, to: b }); + // Structure — kind-specific + const struct = t.structure || {}; + if (t.kind === 'standing_timelapse' && struct.cadence_s != null) { + rows.push(`
    + cadence + ${ESC(struct.cadence_s)}s +
    `); } - const CLUSTER_S = 60; - const clusters = []; - for (const tr of transitions) { - const last = clusters[clusters.length - 1]; - if (last && tr.to.at - last[last.length - 1].to.at <= CLUSTER_S) { - last.push(tr); - } else { - clusters.push([tr]); + if (t.kind === 'scripted_protocol') { + const phases = (struct.phases || []); + if (phases.length) { + const pHtml = phases.map(p => + `${ESC(p.name || p.state || '?')}` + ).join(''); + rows.push(`
    + phases + ${pHtml} +
    `); } } - for (const cluster of clusters) { - const first = cluster[0]; - const tail = cluster[cluster.length - 1]; - // Anchor the chip at the actual change time, not at any - // trailing anchor record (those can land past `now` and - // get hidden by the past-only guard). - const atS = Math.min(tail.to.at, s.now_offset_s); - const cx = xForT(atS); - const arrow = tail.to.pct < first.from.pct ? '↓' : '↑'; - const chipText = `488 ${arrow} ${first.from.pct}%→${tail.to.pct}% · T${tpIndexAt(atS)}`; - const chipW = chipText.length * 5.6 + 10; - // Sit just above the burst-balloon band (which lives at - // laneY-22..-10) so we stay within this row's vertical - // budget — laneY-40 would have crossed into the row above. - // Burst balloons live at separate x positions on every - // case I've seen, so dropping the burst-collision check - // lets the chip render even when a burst is on the same - // lane elsewhere. - const chipY = laneY - 25; - const crowded = - placedPowerChipX.some(px => Math.abs(px - cx) < 70); - g.appendChild(svgEl('line', { - x1: cx, x2: cx, y1: chipY + 12, y2: laneY, - class: 'expov-svg-power-chip-stem' - })); - if (crowded) continue; - g.appendChild(svgEl('rect', { - x: cx - chipW / 2, y: chipY, - width: chipW, height: 12, rx: 3, - class: 'expov-svg-power-chip-bg' - })); - g.appendChild(svgEl('text', { - x: cx, y: chipY + 9, - 'text-anchor': 'middle', - class: 'expov-svg-power-chip' - }, chipText)); - placedPowerChipX.push(cx); + if (t.kind === 'exclusive_burst' || t.kind === 'burst') { + if (struct.frames != null) { + rows.push(`
    + frames + ${ESC(struct.frames)} +
    `); + } + if (struct.mode) { + rows.push(`
    + mode + ${ESC(struct.mode)} +
    `); + } + } + if (t.kind === 'reactive_monitor' && struct.watch) { + rows.push(`
    + watch + ${ESC(struct.watch)} +
    `); } - // Projected future segment (dashed) past 'now' to a horizon — - // skipped entirely when the embryo has been terminated, since - // there's no future to project. projEndT is hoisted because - // downstream code (stop-icon, dose-exhaust line) anchors to it. - const isTerminated = emb.terminated_at_s != null - && emb.terminated_at_s <= s.now_offset_s; - const projStartT = s.now_offset_s; - let projEndT = isTerminated ? emb.terminated_at_s : s.horizon_s; - let projEndsAtBudget = false; - if (!isTerminated) { - if (emb.projected_end_s) projEndT = Math.min(projEndT, emb.projected_end_s); - if (emb.dose_exhaust_at_s && emb.dose_exhaust_at_s < projEndT) { - projEndT = emb.dose_exhaust_at_s; - projEndsAtBudget = true; - } - const xProjStart = xForT(projStartT); - const xProjEnd = xForT(projEndT); - if (xProjEnd > xProjStart) { - g.appendChild(svgEl('line', { - x1: xProjStart, y1: laneMid, x2: xProjEnd, y2: laneMid, - class: projEndsAtBudget - ? 'expov-svg-projected-bar warn' - : 'expov-svg-projected-bar' - })); + // Relations — resolve tactic IDs to names + const relations = t.relations || {}; + const afterIds = Array.isArray(relations.after) ? relations.after : []; + if (afterIds.length) { + const tacticIdMap = {}; + for (const tac of (tactics || [])) { + if (tac.id) tacticIdMap[tac.id] = tac.name || tac.id; } + const names = afterIds.map(id => tacticIdMap[id] || id); + rows.push(`
    + runs after + ${ESC(names.join(', '))} +
    `); } - // Terminated cap: small vertical stop bar at the termination - // point + a "■ DONE · T##" label below the lane so a finished - // embryo doesn't look like it's still acquiring. - if (isTerminated) { - const termX = xForT(emb.terminated_at_s); - g.appendChild(svgEl('line', { - x1: termX, x2: termX, - y1: laneY - 2, y2: laneBottom + 2, - class: 'expov-svg-terminated-bar' - })); - g.appendChild(svgEl('rect', { - x: termX - 2, y: laneY + LANE_H / 2 - 3, - width: 6, height: 6, - class: 'expov-svg-terminated-stop' - })); - const tp = tpIndexAt(emb.terminated_at_s); - const capText = `DONE · T${tp}`; - g.appendChild(svgEl('text', { - x: termX + 6, y: laneBottom + 9, - class: 'expov-svg-terminated-label' - }, capText)); + + // Live readouts (queued/done tactics may carry pre-set readout definitions) + const live = t.live || {}; + if (live.readouts && live.readouts.length) { + rows.push(`
    + ${live.readouts.map(r => this._renderOpsReadout(r, ESC)).join('')} +
    `); } - // Trigger diamonds — placed in the upper half of the lane to avoid - // colliding with the burst balloon above. Each diamond gets a tiny - // T# label below it so the user can see at which timepoint the - // rule fired without hovering. - (emb.trigger_events || []).forEach(te => { - const x = xForT(te.at); - const dy = laneY + 6; - const size = 4; - const trig = s.triggers.find(t => t.id === te.trigger_id); - const label = trig ? trig.label : te.trigger_id; - const dia = svgEl('polygon', { - points: `${x},${dy-size} ${x+size},${dy} ${x},${dy+size} ${x-size},${dy}`, - class: 'expov-svg-trigger-diamond expov-svg-tooltip-target' + if (!rows.length) return ''; + + const isOpen = this._expandedTacticIds.has(this._tacticExpandKey(t)); + return `
    +
    + ${rows.join('\n')} +
    `; + }, + + // Wire click-to-expand on all expandable tactic cards in root after innerHTML is set. + // Persist expand state in _expandedTacticIds so it survives debounced re-renders. + _wireSpineExpand(root) { + root.querySelectorAll('.ops-card[data-tactic-expand-id]').forEach(card => { + const expandId = card.dataset.tacticExpandId; + const body = card.querySelector('.ops-expand-body'); + const chevron = card.querySelector('.ops-expand-chevron'); + if (!body) return; + + card.addEventListener('click', (e) => { + // Let clicks on interactive elements inside the body bubble freely. + if (e.target.closest('a, button:not(.ops-expand-chevron)')) return; + const isExpanded = !body.classList.contains('hidden'); + if (isExpanded) { + body.classList.add('hidden'); + if (chevron) chevron.classList.remove('open'); + this._expandedTacticIds.delete(expandId); + } else { + body.classList.remove('hidden'); + if (chevron) chevron.classList.add('open'); + this._expandedTacticIds.add(expandId); + } }); - const tooltip = svgEl('title'); - tooltip.textContent = `${label}\n${trig?.when_text || ''} → ${trig?.then_text || ''}` + - (te.count ? ` (×${te.count})` : ''); - dia.appendChild(tooltip); - g.appendChild(dia); - g.appendChild(svgEl('text', { - x: x, y: laneBottom + 9, - 'text-anchor': 'middle', - class: 'expov-svg-trigger-tp' - }, `T${tpIndexAt(te.at)}`)); }); + }, + + // Render a readout gauge. `r.value` may contain trusted HTML (span markup). + // Stamps data-livebind on the outer div so _handleTempUpdate (and future + // live-binding) can find the gauge in-place without a full re-render. + // Priority: r.bind (explicit semantic key) > normalised r.label. + _renderOpsReadout(r, ESC) { + const bindKey = r.bind + ? r.bind + : (r.label + ? r.label.toLowerCase().replace(/[^a-z0-9]+/g, '_').replace(/^_|_$/g, '') + : ''); + const bindAttr = bindKey ? ` data-livebind="${ESC(bindKey)}"` : ''; + return `
    +
    ${ESC(r.label)}
    +
    ${r.value}
    + ${r.bar != null + ? `
    ` + : ''} + ${r.sub ? `
    ${ESC(r.sub)}
    ` : ''} +
    `; + }, - // Dose-exhaust warning: ⚠ + time-to-exhaust positioned ABOVE the lane - // so it doesn't overlap with the dashed projected bar - if (emb.dose_exhaust_at_s && emb.dose_exhaust_at_s < s.horizon_s) { - const exhX = xForT(emb.dose_exhaust_at_s); - const remain = emb.dose_exhaust_at_s - s.now_offset_s; - const rh = Math.floor(remain / 3600); - const rm = Math.floor((remain % 3600) / 60); - const exhText = `⚠ budget exhausts in ${rh > 0 ? rh + 'h ' : ''}${rm}m`; - g.appendChild(svgEl('text', { - x: exhX - 4, y: laneY - 5, - 'text-anchor': 'end', - fill: 'var(--accent-orange)', - 'font-size': 10, - 'font-weight': 600, - 'font-family': "'JetBrains Mono', monospace" - }, exhText)); - // Small dotted vertical marker so the user can see WHERE on the lane - g.appendChild(svgEl('line', { - x1: exhX, x2: exhX, y1: laneY, y2: laneY + LANE_H, - stroke: 'var(--accent-orange)', - 'stroke-width': 1.5, - 'stroke-dasharray': '2 2', - opacity: 0.7 - })); + // Render one phase in the scripted_protocol stepper. + // AUDIT: the active phase is the HEADLINE — CSS makes it larger. + _renderOpsPhase(p, ESC) { + const pips = (p.pips || []) + .map(k => ``) + .join(''); + const ic = p.state === 'done' ? '✓' + : p.state === 'active' ? '▶' + : (p.icon || '·'); + return `
    +
    ${ic}${ESC(p.name)}
    +
    ${ESC(p.count || '')}
    + ${pips ? `
    ${pips}
    ` : ''} +
    `; + }, + + // Kind-specific structure for ACTIVE tactics. + _renderOpsKindActive(t, live, ESC) { + if (!t.kind) return ''; + + // scripted_protocol → before/during/after phase stepper. + // Prefer live.phases (may carry pip/count state); fall back to structure.phases. + if (t.kind === 'scripted_protocol') { + const phases = live.phases || (t.structure && t.structure.phases) || []; + if (!phases.length) return ''; + return `
    + ${phases.map(p => this._renderOpsPhase(p, ESC)).join('')} +
    `; } - // Open-ended ∞ glyph at right edge - if (emb.stop_kind === 'open_ended') { - g.appendChild(svgEl('text', { - x: LEFT + LANE_W + 8, y: laneMid + 5, - class: 'expov-svg-infinity' - }, '∞')); - } else { - g.appendChild(svgEl('text', { - x: xForT(projEndT) + 6, y: laneMid + 4, - class: 'expov-svg-stop-icon expov-svg-stop-hatch' - }, '■')); + + // standing_timelapse → compact per-embryo cadence strip. + if (t.kind === 'standing_timelapse') { + const perEmbryo = t.structure && t.structure.per_embryo; + if (!perEmbryo || !perEmbryo.length) return ''; + const rows = perEmbryo.map(e => { + const intervalStr = e.interval_s != null ? `${ESC(e.interval_s)}s` : '—'; + return `
    + ${ESC(e.embryo_id)} + ${ESC(e.cadence_phase)} + ${intervalStr} +
    `; + }).join(''); + return `
    ${rows}
    `; + } + + // reactive_monitor → armed/watching/fired status badge. + if (t.kind === 'reactive_monitor') { + const st = (t.structure && t.structure.status) || 'armed'; + return `
    ${ESC(st)}
    `; + } + + // exclusive_burst / oneshot / custom — readouts only (already rendered above). + return ''; + }, + + // Kind-specific structure for PLANNED (queued) tactics — compact hints. + _renderOpsKindPlanned(t, ESC) { + if (!t.kind) return ''; + + if (t.kind === 'scripted_protocol') { + const phases = (t.structure && t.structure.phases) || []; + if (!phases.length) return ''; + return `
    + ${phases.map(p => this._renderOpsPhase(p, ESC)).join('')} +
    `; + } + + if (t.kind === 'standing_timelapse' && t.structure && t.structure.cadence_s) { + return `
    cadence · ${ESC(t.structure.cadence_s)}s
    `; + } + + if (t.kind === 'reactive_monitor' && t.structure && t.structure.watch) { + return `
    watch · ${ESC(t.structure.watch)}
    `; + } + + if ((t.kind === 'oneshot' || t.kind === 'custom') && t.structure && t.structure.note) { + return `
    ${ESC(t.structure.note)}
    `; + } + + return ''; + }, + + // ================================================================= + // D2 — Roster Lens: embryo population by role class + role + // ================================================================= + + // Fetch the current embryo roster from /api/embryos/positions. + // Returns [] on failure or when no embryos have positions yet. + async _loadRoster() { + try { + const resp = await fetch('/api/embryos/positions', { cache: 'no-store' }); + if (!resp.ok) return []; + const data = await resp.json(); + return Array.isArray(data.embryos) ? data.embryos : []; + } catch (e) { + console.warn('[ExperimentOverview] roster fetch error:', e); + return []; } + }, - // ---- Power strip --------------------------------------------- - // Visual encoding makes "steady" vs "ramping" obvious: - // • Steady segments = thin grey horizontal line - // • Ramping segments = bright cyan line + filled area + dot at each step - // • Step transitions get a small arrow (↓ or ↑) and a delta tag - const powerY = laneY + LANE_H + 6; - const powerH = POWER_H; - const powerYBase = powerY + powerH; - g.appendChild(svgEl('line', { - x1: LEFT, x2: LEFT + LANE_W, y1: powerYBase, y2: powerYBase, - class: 'expov-svg-power-baseline' - })); - const yForPct = (pct) => powerYBase - (pct / 10) * powerH; - - const hist = emb.power_history_488 || []; - if (hist.length > 1) { - // Detect ramp clusters: consecutive change-steps with x-spacing < - // CLUSTER_PX get grouped, annotated once at the cluster end. - const CLUSTER_PX = 20; - const stepEvents = []; // each: {fromIdx, toIdx, isRamp} - for (let k = 0; k < hist.length - 1; k++) { - if (hist[k].pct !== hist[k+1].pct) { - stepEvents.push({ fromIdx: k, toIdx: k+1 }); + // Fetch the roles registry from /api/roles. + // Returns a Map(name → {ui_color, ui_icon, role_class, default_cadence_seconds}). + // Returns empty Map on failure. + async _loadRolesMap() { + try { + const resp = await fetch('/api/roles', { cache: 'no-store' }); + if (!resp.ok) return new Map(); + const data = await resp.json(); + const map = new Map(); + if (Array.isArray(data.roles)) { + for (const r of data.roles) { + map.set(r.name, r); } } - // Group consecutive close steps into clusters - const clusters = []; - stepEvents.forEach(step => { - const last = clusters[clusters.length - 1]; - const stepX = xForT(hist[step.toIdx].at); - if (last) { - const lastX = xForT(hist[last[last.length-1].toIdx].at); - if (Math.abs(stepX - lastX) < CLUSTER_PX) { - last.push(step); - return; - } - } - clusters.push([step]); - }); + return map; + } catch (e) { + console.warn('[ExperimentOverview] roles fetch error:', e); + return new Map(); + } + }, - // Draw horizontal "steady" segments + vertical "step" lines for all - // adjacent (hist[k], hist[k+1]) pairs. - for (let k = 0; k < hist.length - 1; k++) { - const x0 = xForT(hist[k].at); - const x1 = xForT(hist[k+1].at); - const y = yForPct(hist[k].pct); - const yNext = yForPct(hist[k+1].pct); - g.appendChild(svgEl('line', { - x1: x0, x2: x1, y1: y, y2: y, - class: 'expov-svg-power-steady' - })); - if (hist[k].pct !== hist[k+1].pct) { - g.appendChild(svgEl('line', { - x1: x1, x2: x1, y1: y, y2: yNext, - class: 'expov-svg-power-step' - })); - g.appendChild(svgEl('circle', { - cx: x1, cy: yNext, r: 2.2, - class: 'expov-svg-power-stepdot' - })); - } + // Convert a hex color (#rrggbb) to rgba(r,g,b,alpha) for inline styles. + _hexToRgba(hex, alpha) { + const h = (hex || '#888888').replace('#', ''); + const r = parseInt(h.slice(0, 2), 16) || 0; + const g = parseInt(h.slice(2, 4), 16) || 0; + const b = parseInt(h.slice(4, 6), 16) || 0; + return `rgba(${r},${g},${b},${alpha})`; + }, + + // Cross-reference: find the name of the active (or most recently done) tactic + // that covers a given embryo (by embryo_id + role). + // Scope resolution: global → covers all; role → covers matching role; + // embryos → covers listed ids. Returns '' when no tactic found. + // NOTE: mirrors resolve_scope_embryos() in gently/app/orchestration/role_scope.py — keep in sync if a new scope mode is added. + _resolveCurrentTactic(embryoId, role, plan) { + if (!plan || !Array.isArray(plan.tactics)) return ''; + const covers = (t) => { + const scope = t.scope || { mode: 'global' }; + if (scope.mode === 'global') return true; + if (scope.mode === 'role') return scope.role === role; + if (scope.mode === 'embryos') { + return Array.isArray(scope.embryo_ids) && scope.embryo_ids.includes(embryoId); } - // Final tail to lane right edge - const last = hist[hist.length - 1]; - const lastX = xForT(last.at); - const lastY = yForPct(last.pct); - g.appendChild(svgEl('line', { - x1: lastX, x2: LEFT + LANE_W, y1: lastY, y2: lastY, - class: 'expov-svg-power-steady' - })); - - // One annotation per cluster — bracket + "5% → 3%" label - clusters.forEach(cluster => { - const first = cluster[0]; - const tail = cluster[cluster.length - 1]; - const xStart = xForT(hist[first.fromIdx].at); - const xEnd = xForT(hist[tail.toIdx].at); - const pctStart = hist[first.fromIdx].pct; - const pctEnd = hist[tail.toIdx].pct; - const arrow = pctEnd < pctStart ? '↓' : '↑'; - const yMid = (yForPct(pctStart) + yForPct(pctEnd)) / 2; - // Bracket: small horizontal line above the cluster steps - const bracketY = Math.min(yForPct(pctStart), yForPct(pctEnd)) - 6; - g.appendChild(svgEl('path', { - d: `M ${xStart} ${bracketY+3} L ${xStart} ${bracketY} L ${xEnd+2} ${bracketY} L ${xEnd+2} ${bracketY+3}`, - class: 'expov-svg-power-ramp-bracket' - })); - // Label "488 ↓ 5%→3%" anchored just right of the bracket end - g.appendChild(svgEl('text', { - x: xEnd + 6, y: bracketY + 4, - class: 'expov-svg-power-ramp-label' - }, `${arrow} ${pctStart}%→${pctEnd}%`)); - }); + return false; + }; + // Prefer the active tactic covering this embryo. + const active = plan.tactics.find(t => t.state === 'active' && covers(t)); + if (active) return active.name; + // Fall back to the most recently done tactic (last in array order). + for (let i = plan.tactics.length - 1; i >= 0; i--) { + if (plan.tactics[i].state === 'done' && covers(plan.tactics[i])) { + return plan.tactics[i].name; + } + } + return ''; + }, - // Subtle filled area under the curve — helps read overall level - const areaPts = [`${xForT(hist[0].at)},${powerYBase}`]; - for (let k = 0; k < hist.length; k++) { - const x = xForT(hist[k].at); - const y = yForPct(hist[k].pct); - areaPts.push(`${x},${y}`); - if (k < hist.length - 1) { - const xNext = xForT(hist[k+1].at); - areaPts.push(`${xNext},${y}`); - } + // Render a scope badge for a tactic node in the spine. + // Colors for role-scoped badges come from the roles registry (API), not CSS. + // Global and explicit-embryos scopes use the static `.ops-scope-global` class. + _renderOpsScopeBadge(scope, embryos, rolesMap, ESC) { + if (!scope || scope.mode === 'global') { + return '→ all embryos'; + } + if (scope.mode === 'role') { + const role = scope.role || ''; + const roleInfo = rolesMap ? rolesMap.get(role) : null; + const color = (roleInfo && roleInfo.ui_color) || '#8b949e'; + const matchIds = embryos + .filter(e => e.role === role) + .map(e => e.embryo_id) + .join(', '); + const label = matchIds + ? `→ ${ESC(role)} · ${ESC(matchIds)}` + : `→ ${ESC(role)}`; + const bg = this._hexToRgba(color, 0.12); + const border = this._hexToRgba(color, 0.35); + return `${label}`; + } + if (scope.mode === 'embryos') { + const ids = ESC((scope.embryo_ids || []).join(', ')); + return `→ ${ids}`; + } + return ''; + }, + + // Render the D2 roster lens: embryos grouped by role class then by role. + // SUBJECTS section is foregrounded; REFERENCES section is compact/muted. + // Role colors/icons come from the rolesMap (API data), not from CSS constants. + // Returns empty string when embryos array is empty (backward compat). + _renderRosterLens(embryos, rolesMap, plan, ESC) { + if (!embryos || embryos.length === 0) return ''; + + // Group embryos by role_class then by role, preserving first-seen order. + const CLASS_ORDER_PREF = ['subject', 'reference']; + const classOrder = []; + const byClass = {}; + + for (const emb of embryos) { + const roleInfo = rolesMap.get(emb.role); + const cls = (roleInfo && roleInfo.role_class) || 'subject'; + if (!byClass[cls]) { + byClass[cls] = { roleOrder: [], byRole: {} }; + classOrder.push(cls); + } + const section = byClass[cls]; + if (!section.byRole[emb.role]) { + section.byRole[emb.role] = []; + section.roleOrder.push(emb.role); } - areaPts.push(`${LEFT + LANE_W},${yForPct(last.pct)}`); - areaPts.push(`${LEFT + LANE_W},${powerYBase}`); - g.appendChild(svgEl('polygon', { - points: areaPts.join(' '), - class: 'expov-svg-power-area' - })); + section.byRole[emb.role].push(emb); } - // Power label with current value — "@" reads as "at this power" - // and avoids confusion with the bullet-separator used elsewhere - g.appendChild(svgEl('text', { - x: LEFT - 8, y: powerY + powerH / 2 + 3, - 'text-anchor': 'end', - class: 'expov-svg-sublabel' - }, `488 @ ${emb.laser_488_pct_now}%`)); - - // ---- Dose gauge ---------------------------------------------- - const doseY = powerYBase + 6; - const doseW = LANE_W; - g.appendChild(svgEl('rect', { - x: LEFT, y: doseY, width: doseW, height: DOSE_H, rx: 2, - class: 'expov-svg-dose-track' - })); - const dosePct = emb.dose_used_ms / emb.dose_budget_ms; - const fillCls = dosePct > 0.85 ? 'expov-svg-dose-fill-crit' - : dosePct > 0.60 ? 'expov-svg-dose-fill-warn' - : 'expov-svg-dose-fill-ok'; - g.appendChild(svgEl('rect', { - x: LEFT, y: doseY, width: Math.max(1, doseW * dosePct), height: DOSE_H, rx: 2, - class: fillCls - })); - // Dose label (shows 10× hint for calibration role) - const doseLabel = emb.role === 'calibration' ? 'dose (10×)' : 'dose'; - g.appendChild(svgEl('text', { - x: LEFT - 8, y: doseY + DOSE_H - 2, - 'text-anchor': 'end', - class: 'expov-svg-sublabel' - }, doseLabel)); - // Inside the bar: usage figure — "used of budget" is more scannable - // than "x / y s" which reads like a fraction - const usedS = (emb.dose_used_ms / 1000).toFixed(1); - const budgetS = (emb.dose_budget_ms / 1000).toFixed(1); - const doseText = emb.dose_budget_ms > 0 - ? `${usedS}s of ${budgetS}s (${Math.round(dosePct * 100)}%)` - : `${usedS}s used`; - g.appendChild(svgEl('text', { - x: LEFT + 6, y: doseY + DOSE_H - 2, - class: 'expov-svg-dose-text' - }, doseText)); - - return g; + + // Canonical class order: subjects first. + classOrder.sort((a, b) => { + const ai = CLASS_ORDER_PREF.indexOf(a); + const bi = CLASS_ORDER_PREF.indexOf(b); + return (ai < 0 ? 99 : ai) - (bi < 0 ? 99 : bi); + }); + + const totalCount = embryos.length; + const totalRoles = classOrder.reduce((n, cls) => n + byClass[cls].roleOrder.length, 0); + + const classSections = classOrder.map(cls => { + const { roleOrder, byRole } = byClass[cls]; + const isSubject = cls === 'subject'; + + const classHeaderInner = isSubject + ? `Subjects— adaptive tactics / scenarios` + : `References— steady acquisition`; + + const roleGroups = roleOrder.map(role => { + const roleEmbyros = byRole[role]; + const roleInfo = rolesMap.get(role); + const uiColor = (roleInfo && roleInfo.ui_color) || '#8b949e'; + const uiIcon = (roleInfo && roleInfo.ui_icon) || ''; + const bgRgba = this._hexToRgba(uiColor, 0.08); + const ids = roleEmbyros.map(e => e.embryo_id).join(', '); + + const embryoRows = roleEmbyros.map(emb => { + const cadencePhase = emb.cadence_phase || 'normal'; + const strain = emb.strain || '—'; + const label = emb.user_label || emb.embryo_id; + const tacticName = this._resolveCurrentTactic(emb.embryo_id, emb.role, plan); + const stateStr = emb.is_complete ? 'done' + : cadencePhase === 'paused' ? 'paused' + : 'active'; + const compact = isSubject ? '' : ' compact'; + const chipBg = this._hexToRgba(uiColor, 0.15); + const chipBorder = this._hexToRgba(uiColor, 0.4); + return `
    + ${ESC(label)} + ${uiIcon ? ESC(uiIcon) + ' ' : ''}${ESC(role)} + ${ESC(strain)} + ${ESC(cadencePhase)} + ${ESC(tacticName || '—')} + ${ESC(stateStr)} +
    `; + }).join(''); + + return `
    +
    + ${ESC(role.toUpperCase())} + · + ${roleEmbyros.length} embryo${roleEmbyros.length !== 1 ? 's' : ''} + ${ESC(ids)} +
    + ${embryoRows} +
    `; + }).join(''); + + return `
    +
    ${classHeaderInner}
    + ${roleGroups} +
    `; + }).join(''); + + return `
    +
    + Population roster + ${totalCount} embryo${totalCount !== 1 ? 's' : ''} · ${totalRoles} role${totalRoles !== 1 ? 's' : ''} +
    + ${classSections} +
    `; }, // ----------------------------------------------------------------- diff --git a/gently/ui/web/static/js/gallery.js b/gently/ui/web/static/js/gallery.js index 51ee4207..259e6636 100644 --- a/gently/ui/web/static/js/gallery.js +++ b/gently/ui/web/static/js/gallery.js @@ -422,13 +422,19 @@ const CalibrationProfileView = { /** Compact SPIM live indicator used inside the metrics strip. * Carries the same IDs as the old big preview so SpimLivePreview's - * apply-on-render logic continues to work unchanged. */ + * apply-on-render logic continues to work unchanged. The thumb is a + * button — click to open the floating popout for a larger view. */ _renderSpimIndicator() { return `
    SPIM - +
    @@ -1329,21 +1335,33 @@ const SpimLivePreview = { const placeholder = document.getElementById('cal-spim-placeholder'); const metaEl = document.getElementById('cal-spim-meta'); const led = document.getElementById('cal-spim-led'); - if (!img) return; // not in profile view const latest = embryoId ? this._latestByEmbryo[embryoId] : null; - if (latest) { - img.src = `data:image/png;base64,${latest.base64_png}`; - img.classList.add('has-frame'); - if (placeholder) placeholder.hidden = true; - if (metaEl) metaEl.textContent = this._formatMeta(latest); - if (led) led.classList.remove('idle'); - } else { - img.removeAttribute('src'); - img.classList.remove('has-frame'); - if (placeholder) placeholder.hidden = false; - if (metaEl) metaEl.textContent = '—'; - if (led) led.classList.add('idle'); + + if (img) { + if (latest) { + img.src = `data:image/png;base64,${latest.base64_png}`; + img.classList.add('has-frame'); + if (placeholder) placeholder.hidden = true; + if (metaEl) metaEl.textContent = this._formatMeta(latest); + if (led) led.classList.remove('idle'); + } else { + img.removeAttribute('src'); + img.classList.remove('has-frame'); + if (placeholder) placeholder.hidden = false; + if (metaEl) metaEl.textContent = '—'; + if (led) led.classList.add('idle'); + } + } + + // Mirror into popout if it's open — the popout lives outside the + // calibration panel's innerHTML reset, so we paint it independently. + if (typeof SpimPopout !== 'undefined') { + SpimPopout.paint(latest ? { + base64_png: latest.base64_png, + meta: this._formatMeta(latest), + embryoId, + } : null); } }, @@ -1375,6 +1393,227 @@ const SpimLivePreview = { document.addEventListener('DOMContentLoaded', () => SpimLivePreview.init()); +// ========================================== +// SPIM live popout (floating draggable window) +// ========================================== +// Lazy-built floating window that mirrors SpimLivePreview at a larger +// size. Draggable via the header bar, resizable from the bottom-right +// corner. Position and size persist in localStorage so the window +// re-opens where the operator last left it. Closes on Escape. +const SpimPopout = { + _STORAGE_KEY: 'gently.spimPopout.v1', + _root: null, + _isOpen: false, + + _ensureBuilt() { + if (this._root) return this._root; + + const el = document.createElement('div'); + el.className = 'cal-spim-popout'; + el.id = 'cal-spim-popout'; + el.hidden = true; + el.innerHTML = ` +
    + + SPIM Live + + + +
    +
    + +
    + Awaiting SPIM frame… +
    +
    + + `; + document.body.appendChild(el); + this._root = el; + + // Restore persisted geometry + const saved = this._loadGeometry(); + if (saved) { + el.style.left = `${saved.left}px`; + el.style.top = `${saved.top}px`; + el.style.width = `${saved.width}px`; + el.style.height = `${saved.height}px`; + } + + el.querySelector('#cal-spim-popout-close').addEventListener('click', () => this.close()); + this._wireDrag(el); + this._wireResizeObserver(el); + + return el; + }, + + open() { + const el = this._ensureBuilt(); + if (this._isOpen) return; + el.hidden = false; + this._isOpen = true; + + // Clamp into viewport in case window was resized while popout was hidden + this._clampIntoViewport(el); + + // Paint current frame for the selected embryo + const selected = (typeof CalibrationManager !== 'undefined') + ? CalibrationManager.selectedEmbryoId : null; + if (selected && typeof SpimLivePreview !== 'undefined') { + const latest = SpimLivePreview._latestByEmbryo[selected]; + this.paint(latest ? { + base64_png: latest.base64_png, + meta: SpimLivePreview._formatMeta(latest), + embryoId: selected, + } : null); + } else { + this.paint(null); + } + + document.addEventListener('keydown', this._onKey); + }, + + close() { + if (!this._root || !this._isOpen) return; + this._root.hidden = true; + this._isOpen = false; + document.removeEventListener('keydown', this._onKey); + }, + + toggle() { + this._isOpen ? this.close() : this.open(); + }, + + /** Called by SpimLivePreview whenever the current embryo's latest + * frame changes. Frame is {base64_png, meta, embryoId} or null. */ + paint(frame) { + if (!this._root || !this._isOpen) return; + const img = this._root.querySelector('#cal-spim-popout-img'); + const placeholder = this._root.querySelector('#cal-spim-popout-placeholder'); + const meta = this._root.querySelector('#cal-spim-popout-meta'); + const embryoEl = this._root.querySelector('#cal-spim-popout-embryo'); + const led = this._root.querySelector('#cal-spim-popout-led'); + + if (frame) { + img.src = `data:image/png;base64,${frame.base64_png}`; + img.classList.add('has-frame'); + placeholder.hidden = true; + meta.textContent = frame.meta || '—'; + embryoEl.textContent = frame.embryoId || ''; + led.classList.remove('idle'); + } else { + img.removeAttribute('src'); + img.classList.remove('has-frame'); + placeholder.hidden = false; + meta.textContent = '—'; + embryoEl.textContent = ''; + led.classList.add('idle'); + } + }, + + _onKey: (e) => { + if (e.key === 'Escape') SpimPopout.close(); + }, + + _wireDrag(el) { + const header = el.querySelector('#cal-spim-popout-header'); + let dragging = false; + let startX = 0, startY = 0, startLeft = 0, startTop = 0; + + header.addEventListener('pointerdown', (e) => { + // Don't start drag on the close button + if (e.target.closest('.cal-spim-popout-close')) return; + dragging = true; + const rect = el.getBoundingClientRect(); + startX = e.clientX; + startY = e.clientY; + startLeft = rect.left; + startTop = rect.top; + // Switch to absolute positioning if currently default + el.style.left = `${startLeft}px`; + el.style.top = `${startTop}px`; + el.style.right = 'auto'; + el.style.bottom = 'auto'; + header.setPointerCapture(e.pointerId); + el.classList.add('dragging'); + }); + + header.addEventListener('pointermove', (e) => { + if (!dragging) return; + const dx = e.clientX - startX; + const dy = e.clientY - startY; + let nextLeft = startLeft + dx; + let nextTop = startTop + dy; + // Keep at least 40px of header on-screen + const w = el.offsetWidth; + const h = el.offsetHeight; + nextLeft = Math.max(-(w - 80), Math.min(window.innerWidth - 80, nextLeft)); + nextTop = Math.max(0, Math.min(window.innerHeight - 40, nextTop)); + el.style.left = `${nextLeft}px`; + el.style.top = `${nextTop}px`; + }); + + const endDrag = (e) => { + if (!dragging) return; + dragging = false; + el.classList.remove('dragging'); + try { header.releasePointerCapture(e.pointerId); } catch (_) {} + this._saveGeometry(el); + }; + header.addEventListener('pointerup', endDrag); + header.addEventListener('pointercancel', endDrag); + }, + + _wireResizeObserver(el) { + if (typeof ResizeObserver === 'undefined') return; + let saveTimer = null; + const ro = new ResizeObserver(() => { + if (!this._isOpen) return; + clearTimeout(saveTimer); + saveTimer = setTimeout(() => this._saveGeometry(el), 250); + }); + ro.observe(el); + }, + + _clampIntoViewport(el) { + const rect = el.getBoundingClientRect(); + if (rect.left + 80 > window.innerWidth || rect.top + 40 > window.innerHeight + || rect.left < -(rect.width - 80) || rect.top < 0) { + // Recenter + const w = Math.min(rect.width || 520, window.innerWidth - 40); + const h = Math.min(rect.height || 440, window.innerHeight - 40); + el.style.width = `${w}px`; + el.style.height = `${h}px`; + el.style.left = `${Math.max(20, (window.innerWidth - w) / 2)}px`; + el.style.top = `${Math.max(20, (window.innerHeight - h) / 2)}px`; + } + }, + + _saveGeometry(el) { + const rect = el.getBoundingClientRect(); + const data = { + left: Math.round(rect.left), + top: Math.round(rect.top), + width: Math.round(rect.width), + height: Math.round(rect.height), + }; + try { localStorage.setItem(this._STORAGE_KEY, JSON.stringify(data)); } catch (_) {} + }, + + _loadGeometry() { + try { + const raw = localStorage.getItem(this._STORAGE_KEY); + if (!raw) return null; + const data = JSON.parse(raw); + if (typeof data.left !== 'number') return null; + return data; + } catch (_) { return null; } + }, +}; + // Legacy wrappers kept for backward compatibility function renderCalibrationGallery() { CalibrationManager.render(); } @@ -1392,3 +1631,173 @@ function filterByEmbryo(list) { if (!state.embryoFilter) return list; return list.filter(img => img.metadata?.embryo_id === state.embryoFilter); } + +// ========================================== +// GalleryTab — top-level Gallery tab controller +// ========================================== + +const GalleryTab = { + _allItems: [], + _embryoFilter: '', + _typeFilter: '', + + async init() { + const panel = document.getElementById('gallery-tab-body'); + if (!panel) return; + + // Wire filter controls (idempotent) + const embryoSel = document.getElementById('gallery-embryo-filter'); + const typeSel = document.getElementById('gallery-type-filter'); + const refreshBtn = document.getElementById('gallery-refresh-btn'); + if (embryoSel && !embryoSel._gtWired) { + embryoSel._gtWired = true; + embryoSel.addEventListener('change', () => { + this._embryoFilter = embryoSel.value; + this._renderGrid(); + }); + } + if (typeSel && !typeSel._gtWired) { + typeSel._gtWired = true; + typeSel.addEventListener('change', () => { + this._typeFilter = typeSel.value; + this._renderGrid(); + }); + } + if (refreshBtn && !refreshBtn._gtWired) { + refreshBtn._gtWired = true; + refreshBtn.addEventListener('click', () => this._load()); + } + + await this._load(); + }, + + async _load() { + const panel = document.getElementById('gallery-tab-body'); + if (!panel) return; + panel.innerHTML = ''; + try { + const data = await fetch('/api/snapshots').then(r => r.json()); + const items = data.snapshots || []; + // Sort newest first + items.sort((a, b) => (b.timestamp || '').localeCompare(a.timestamp || '')); + this._allItems = items; + this._populateEmbroyFilter(items); + this._renderGrid(); + } catch (err) { + panel.innerHTML = ''; + } + }, + + _populateEmbroyFilter(items) { + const sel = document.getElementById('gallery-embryo-filter'); + if (!sel) return; + const embryos = [...new Set(items.map(i => i.metadata?.embryo_id).filter(Boolean))].sort(); + const cur = sel.value; + // Rebuild options beyond the "All embryos" placeholder + while (sel.options.length > 1) sel.remove(1); + embryos.forEach(eid => { + const opt = document.createElement('option'); + opt.value = eid; + opt.textContent = eid; + sel.appendChild(opt); + }); + if (cur && embryos.includes(cur)) sel.value = cur; + }, + + _renderGrid() { + const panel = document.getElementById('gallery-tab-body'); + if (!panel) return; + + let items = this._allItems; + if (this._embryoFilter) { + items = items.filter(i => (i.metadata?.embryo_id || '') === this._embryoFilter); + } + if (this._typeFilter) { + items = items.filter(i => i.data_type === this._typeFilter); + } + + if (items.length === 0) { + panel.innerHTML = ''; + return; + } + + const html = ``; + panel.innerHTML = html; + + // Wire click handlers + panel.querySelectorAll('.gallery-tab-item').forEach(el => { + el.addEventListener('click', () => { + const idx = parseInt(el.dataset.idx, 10); + Lightbox.open(items, idx, 'gallery'); + }); + }); + }, + + _itemHtml(img, idx) { + const embryo = img.metadata?.embryo_id || ''; + const ts = img.timestamp ? img.timestamp.slice(0, 19).replace('T', ' ') : ''; + const typeLabel = img.data_type || 'image'; + const thumb = img.base64_png + ? `${typeLabel}` + : ``; + return ` + + `; + }, +}; + +// ========================================== +// showGentlyToast — lightweight global toast (volume acquired, etc.) +// ========================================== + +/** + * Show a brief toast notification with an optional action link. + * @param {string} message - Primary message text + * @param {string|null} actionLabel - Label for the action button (null = no button) + * @param {Function|null} actionFn - Callback invoked when the action is clicked + * @param {number} [duration=6000] - Auto-dismiss delay in ms + */ +function showGentlyToast(message, actionLabel, actionFn, duration = 6000) { + // Remove any existing gently-toast + document.querySelectorAll('.gently-toast').forEach(t => t.remove()); + + const toast = document.createElement('div'); + toast.className = 'gently-toast'; + + const msgSpan = document.createElement('span'); + msgSpan.className = 'gently-toast-msg'; + msgSpan.textContent = message; + toast.appendChild(msgSpan); + + if (actionLabel && actionFn) { + const actionBtn = document.createElement('button'); + actionBtn.className = 'gently-toast-action'; + actionBtn.textContent = actionLabel; + actionBtn.addEventListener('click', () => { + actionFn(); + toast.remove(); + }); + toast.appendChild(actionBtn); + } + + const dismiss = document.createElement('button'); + dismiss.className = 'gently-toast-dismiss'; + dismiss.setAttribute('aria-label', 'Dismiss'); + dismiss.textContent = '×'; + dismiss.addEventListener('click', () => toast.remove()); + toast.appendChild(dismiss); + + document.body.appendChild(toast); + // Trigger transition + requestAnimationFrame(() => toast.classList.add('visible')); + + const timer = setTimeout(() => toast.remove(), duration); + dismiss.addEventListener('click', () => clearTimeout(timer)); +} diff --git a/gently/ui/web/static/js/home.js b/gently/ui/web/static/js/home.js new file mode 100644 index 00000000..bf5cad17 --- /dev/null +++ b/gently/ui/web/static/js/home.js @@ -0,0 +1,189 @@ +/** + * HomeApp — the landing tab. + * + * A light at-a-glance landing surface: recent sessions, recent plans, recent + * images, a thin status line, and a "Start / continue an experiment" button + * that launches the setup flow (the wizard, which no longer auto-pops in chat). + * + * Read-only fetches against existing endpoints (/api/sessions, /api/campaigns, + * /api/home/recent-images); mirrors the ReviewApp/CampaignsApp module pattern. + */ +const HomeApp = (() => { + let _inited = false; + const SESSIONS_N = 5; + const CAMPAIGNS_N = 5; + const IMAGES_N = 8; + // Recent images are stable (latest projection per embryo). refresh() runs on + // every Home-tab entry, so guard against redundant disk-walking fetches: + // skip if one is in flight or the strip was loaded within IMAGES_TTL_MS. + const IMAGES_TTL_MS = 15000; + let _imgState = { at: 0, inflight: false }; + + function relTime(iso) { + if (!iso) return ''; + const t = Date.parse(iso); + if (isNaN(t)) return ''; + const s = Math.max(0, (Date.now() - t) / 1000); + if (s < 60) return 'just now'; + if (s < 3600) return `${Math.floor(s / 60)}m ago`; + if (s < 86400) return `${Math.floor(s / 3600)}h ago`; + const d = Math.floor(s / 86400); + return d < 30 ? `${d}d ago` : new Date(t).toLocaleDateString(); + } + + function empty(el, msg) { + el.innerHTML = `
    ${escapeHtml(msg)}
    `; + } + + function wireGoTab(scope) { + (scope || document).querySelectorAll('[data-go-tab]').forEach(el => { + if (el._goWired) return; + el._goWired = true; + el.addEventListener('click', (e) => { + e.preventDefault(); + if (typeof switchTab === 'function') switchTab(el.dataset.goTab); + }); + }); + } + + async function loadSessions() { + const el = document.getElementById('home-recent-sessions'); + if (!el) return; + try { + const data = await (await fetch('/api/sessions')).json(); + const sessions = (data.sessions || []).slice(0, SESSIONS_N); + if (!sessions.length) { empty(el, 'No sessions yet.'); return; } + el.innerHTML = sessions.map(s => { + const live = s.active ? 'live' : ''; + const resume = s.active ? '' : + ``; + return `
    +
    +
    ${escapeHtml(s.name || s.session_id)}${live}
    + ${escapeHtml(relTime(s.last_active))} · ${s.embryo_count || 0} embryos +
    ${resume} +
    `; + }).join(''); + el.querySelectorAll('[data-resume]').forEach(b => b.addEventListener('click', async () => { + b.disabled = true; + b.textContent = 'Resuming…'; + try { + await fetch(`/api/sessions/${encodeURIComponent(b.dataset.resume)}/resume`, { method: 'POST' }); + } catch (_) { b.disabled = false; b.textContent = 'Resume'; } + })); + } catch (e) { empty(el, 'Could not load sessions.'); } + } + + async function loadCampaigns() { + const el = document.getElementById('home-recent-campaigns'); + if (!el) return; + try { + const data = await (await fetch('/api/campaigns')).json(); + const items = (data.campaigns || []).slice(0, CAMPAIGNS_N); + if (!items.length) { empty(el, 'No plans yet.'); return; } + el.innerHTML = items.map(t => { + const c = t.campaign || {}; + const st = t.status || {}; + const name = c.shorthand || c.description || 'Untitled plan'; + const total = st.total || 0; + const chip = total ? `${st.completed || 0}/${total} done` : ''; + return `
    + ${escapeHtml(name)}${chip} +
    `; + }).join(''); + wireGoTab(el); + } catch (e) { empty(el, 'Could not load plans.'); } + } + + async function loadImages(force) { + const el = document.getElementById('home-recent-images'); + if (!el) return; + if (_imgState.inflight) return; + // _imgState.at is set only after a completed fetch (images or empty), + // never after an error — so failures still retry on the next entry. + if (!force && _imgState.at && (Date.now() - _imgState.at) < IMAGES_TTL_MS) return; + _imgState.inflight = true; + try { + const data = await (await fetch(`/api/home/recent-images?limit=${IMAGES_N}`)).json(); + // Latest projection per embryo across recent sessions (server orders + // most-recent session first). + const recent = (data.images || []).slice(0, IMAGES_N); + if (!recent.length) { + empty(el, 'No images yet — they appear once a session has captured volumes.'); + _imgState.at = Date.now(); + return; + } + el.innerHTML = '
    ' + recent.map(s => { + const tp = (s.timepoint != null) ? ` · t${s.timepoint}` : ''; + const label = `${s.embryo_id || ''}${tp}`; + const sub = s.session_name && s.session_name !== s.session_id + ? ` (${s.session_name})` : ''; + const src = `/api/sessions/${encodeURIComponent(s.session_id)}` + + `/projection?embryo=${encodeURIComponent(s.embryo_id)}` + + `&t=${encodeURIComponent(s.timepoint)}`; + return `
    + ${escapeHtml(label)} +
    `; + }).join('') + '
    '; + _imgState.at = Date.now(); + } catch (e) { + empty(el, 'Could not load images.'); + } finally { + _imgState.inflight = false; + } + } + + function updateStatus() { + const el = document.getElementById('home-status'); + if (!el) return; + // Read the shared ConnectionStatus store, not a one-shot snapshot of + // state.connected — the latter was read once at tab init (before the + // /ws handshake) and never corrected, showing "Offline" while the + // header pill said "Online". + const connected = (typeof ConnectionStatus !== 'undefined') + ? ConnectionStatus.get().gentlyConnected + : (typeof state !== 'undefined' && state.connected); + const n = (typeof state !== 'undefined' && Array.isArray(state.embryos)) ? state.embryos.length : 0; + el.textContent = connected + ? `Connected · ${n} embryo${n === 1 ? '' : 's'} in view` + : 'Offline — start the agent to connect.'; + } + + function refresh() { + updateStatus(); + loadSessions(); + loadCampaigns(); + loadImages(); + } + + function init() { + if (!_inited) { + _inited = true; + wireGoTab(document.getElementById('home-content')); + const start = document.getElementById('home-start-btn'); + if (start) start.addEventListener('click', () => { + if (typeof AgentChat !== 'undefined' && AgentChat.togglePanel) { + AgentChat.togglePanel(true); + // Let the panel's WS connect before sending the command. + if (AgentChat.runCommand) setTimeout(() => AgentChat.runCommand('/wizard'), 250); + } + }); + // Re-render the status line on every connection change. subscribe() + // replays the current snapshot immediately, so a late init still + // renders correct state. Registered once (inside the _inited guard). + if (typeof ConnectionStatus !== 'undefined') { + ConnectionStatus.subscribe(() => updateStatus()); + } + } + refresh(); // re-fetch on every entry to the tab + } + + // Self-initialise on load when Home is the default-active tab (switchTab's + // lazy-init hook only fires on a tab click / hash route, not initial paint). + document.addEventListener('DOMContentLoaded', () => { + const home = document.getElementById('home-content'); + if (home && home.classList.contains('active')) init(); + }); + + return { init, refresh }; +})(); diff --git a/gently/ui/web/static/js/landing.js b/gently/ui/web/static/js/landing.js new file mode 100644 index 00000000..7d82c553 --- /dev/null +++ b/gently/ui/web/static/js/landing.js @@ -0,0 +1,888 @@ +/** + * V2Landing (ux_v2): the agent-first welcome AND the in-place plan wizard. + * + * Clicking "Plan an experiment" switches the landing to a plan screen, enters + * plan mode, and renders the agent's work IN THE WIZARD — not the chat REPL: + * - the agent's reasoning + tool calls render as a tidy, claude.ai-style + * collapsible activity feed (#v2-plan-activity), fed by the AGENT_ACTIVITY + * event that agent-chat.js mirrors off the /ws/agent stream; + * - the agent's ask_user_choice questions render as button cards + * (#v2-plan-ask) via AgentChat.buildAskCard; + * - "THE PLAN" panel (#v2-plan-summary) mirrors the REAL plan (phases→tasks) + * fetched from /api/campaigns once a turn settles. + * Chat is the last resort (the escape pill / "Open conversation"). + * + * No-ops unless #v2-landing is present (flag off → v1 untouched, overlay absent). + */ +const V2Landing = (() => { + let el = null; + let current = null; // the ask currently in #v2-plan-ask + let feedTextEl = null; // current accumulating prose paragraph in the feed + let feedThinkingEl = null; // current accumulating reasoning (thinking) block + let runningTools = {}; // tool name -> stack of running card elements + let feedHadContent = false; // did this turn surface anything in the feed? + let capturedCampaignId = null; // best-effort id scraped from tool results + let planProposed = false; // propose_plan ran → plan is ready to commit + + const $ = (id) => document.getElementById(id); + + function greet() { + const g = $('v2-landing-greeting'); + if (!g) return; + const h = new Date().getHours(); + const t = h < 5 ? 'Still here.' : h < 12 ? 'Good morning.' + : h < 18 ? 'Good afternoon.' : 'Good evening.'; + g.innerHTML = t + '
    What are we doing today?'; + } + + function setScreen(name) { if (el) el.dataset.screen = name; } + function planActive() { + return !!el && el.dataset.screen === 'plan' && !el.classList.contains('dismissed') + && el.style.display !== 'none'; + } + + function dismiss() { + if (!el || el.classList.contains('dismissed')) return; + el.classList.add('dismissed'); + let done = false; + const finish = () => { + if (done) return; + done = true; + el.style.display = 'none'; + el.setAttribute('aria-hidden', 'true'); + }; + el.addEventListener('transitionend', finish, { once: true }); + setTimeout(finish, 650); + } + + // ── status / error helpers ──────────────────────────────────────── + function setThinkingLabel(text) { + const l = document.querySelector('#v2-plan-thinking .v2-plan-thinking-label'); + if (l && text) l.textContent = text; + } + // Elapsed-time counter so a long think reads as progress, not a hang. Starts + // when the thinking indicator first shows and runs until it's hidden (turn end). + let _thinkTimer = null; + let _thinkStart = 0; + function _thinkTick() { + const t = $('v2-plan-thinking'); + if (!t) return; + let el = t.querySelector('.v2-plan-elapsed'); + if (!el) { + el = document.createElement('span'); + el.className = 'v2-plan-elapsed'; + el.style.cssText = 'margin-left:6px;opacity:.55;font-variant-numeric:tabular-nums;'; + t.appendChild(el); + } + const s = Math.round((Date.now() - _thinkStart) / 1000); + el.textContent = s > 0 ? s + 's' : ''; + } + function showThinking(on, label) { + const t = $('v2-plan-thinking'); + if (t) t.classList.toggle('hidden', !on); + if (on && label) setThinkingLabel(label); + if (on) { + if (!_thinkTimer) { + _thinkStart = Date.now(); + _thinkTick(); + _thinkTimer = setInterval(_thinkTick, 1000); + } + } else if (_thinkTimer) { + clearInterval(_thinkTimer); + _thinkTimer = null; + const el = t && t.querySelector('.v2-plan-elapsed'); + if (el) el.textContent = ''; + } + } + // Human-readable "what's happening right now" from a tool activity event, + // so the status line names the live operation instead of a static string. + function prettyTool(act) { + const raw = (act && (act.label || act.name)) || 'the next step'; + const s = String(raw).replace(/_/g, ' ').trim(); + return s.charAt(0).toUpperCase() + s.slice(1) + '…'; + } + function errorVisible() { const e = $('v2-plan-error'); return !!e && !e.classList.contains('hidden'); } + function showPlanError(msg) { + const e = $('v2-plan-error'); if (!e) return; + e.textContent = msg; e.classList.remove('hidden'); + showThinking(false); + } + function hidePlanError() { const e = $('v2-plan-error'); if (e) e.classList.add('hidden'); } + + function clearAsk() { const m = $('v2-plan-ask'); if (m) m.innerHTML = ''; } + function resetSummary() { + const list = $('v2-plan-summary'); + if (list) list.innerHTML = '
    The plan will take shape here as Gently designs it.
    '; + planPage = 0; planPages = []; planTitleText = ''; + } + + // ── activity feed: paginated, ONE agent step (turn) per page ─────── + // Instead of one ever-growing scroll, each agent turn — its reasoning + + // the tool calls it made — is a page you flip through with ‹ Prev / Next ›. + // The current question stays pinned below the feed (#v2-plan-ask). A new + // turn auto-advances to its page; you can flip back to review earlier steps. + let feedPages = []; // .v2-act-page elements, one per turn + let feedPage = 0; // index currently shown + let curPageEl = null; // page receiving this turn's content + let pendingNewPage = false; // a turn started; open a fresh page on first content + + function feedEl() { return $('v2-plan-activity'); } + function feedPagesWrap() { return feedEl()?.querySelector('.v2-feed-pages'); } + function clearActivity() { + const f = feedEl(); + if (f) { + f.innerHTML = + '' + + '
    ' + + ''; + } + feedPages = []; feedPage = 0; curPageEl = null; pendingNewPage = false; + feedTextEl = null; feedThinkingEl = null; runningTools = {}; feedHadContent = false; + capturedCampaignId = null; planProposed = false; + clearPlanReady(); + hidePlanError(); + } + function newFeedPage() { + const wrap = feedPagesWrap(); if (!wrap) return null; + const page = document.createElement('div'); + page.className = 'v2-act-page'; + wrap.appendChild(page); + feedPages.push(page); + curPageEl = page; + feedPage = feedPages.length - 1; // auto-advance to the live step + feedTextEl = null; + drawFeedPager(); + return page; + } + // Where this turn's prose/tool cards land. Opens a fresh page the first time + // content arrives after a turn_start (so content-less command turns don't + // leave empty pages), and lazily on the very first content. + function feedTarget() { + if (pendingNewPage || !curPageEl) { newFeedPage(); pendingNewPage = false; } + return curPageEl; + } + function viewingLatest() { return feedPage >= feedPages.length - 1; } + function drawFeedPager() { + const f = feedEl(); if (!f) return; + const n = feedPages.length; + const i = Math.min(Math.max(feedPage, 0), Math.max(n - 1, 0)); + feedPages.forEach((p, idx) => p.classList.toggle('active', idx === i)); + const bar = f.querySelector('.v2-feed-pager-bar'); + const dots = f.querySelector('.v2-feed-dots'); + if (!bar || !dots) return; + if (n <= 1) { bar.hidden = true; dots.hidden = true; return; } + bar.hidden = false; dots.hidden = false; + bar.innerHTML = ''; + const mkBtn = (txt, disabled, fn) => { + const b = document.createElement('button'); + b.className = 'v2-plan-pager-btn'; b.type = 'button'; b.textContent = txt; + b.disabled = disabled; b.addEventListener('click', fn); + return b; + }; + const pos = document.createElement('span'); + pos.className = 'v2-plan-pager-pos'; pos.textContent = `Step ${i + 1} of ${n}`; + bar.append( + mkBtn('‹ Prev', i === 0, () => { if (feedPage > 0) { feedPage--; drawFeedPager(); } }), + pos, + mkBtn('Next ›', i === n - 1, () => { if (feedPage < n - 1) { feedPage++; drawFeedPager(); } }), + ); + dots.innerHTML = ''; + for (let d = 0; d < n; d++) { + const dot = document.createElement('button'); + dot.className = 'v2-plan-dot' + (d === i ? ' active' : ''); + dot.type = 'button'; + dot.setAttribute('aria-label', `Step ${d + 1} of ${n}`); + dot.addEventListener('click', () => { feedPage = d; drawFeedPager(); }); + dots.appendChild(dot); + } + } + function scrollFeedIfNearBottom() { + if (!viewingLatest()) return; // don't yank the user off an earlier step + const m = document.querySelector('.v2-screen-plan .v2-plan-main'); + if (m && (m.scrollHeight - m.scrollTop - m.clientHeight) < 140) m.scrollTop = m.scrollHeight; + } + function clearFallback() { feedEl()?.querySelectorAll('.v2-plan-fallback').forEach(n => n.remove()); } + + // Render the agent's prose like the chat does (reuses AgentChat.mdToHtml — + // escapes then renders bold/italic/code/line-breaks), so the feed isn't raw + // markdown. Falls back to escaped text if the helper isn't available. + function renderMd(s) { + if (typeof AgentChat !== 'undefined' && AgentChat.mdToHtml) return AgentChat.mdToHtml(s); + const esc = (typeof escapeHtml === 'function') ? escapeHtml(String(s)) : String(s); + return esc.replace(/\n/g, '
    '); + } + + // Plan-writing tools → refresh THE PLAN panel during the turn (debounced), + // not only at turn_end (ask_user_choice pauses the turn before it ends). + const PLAN_TOOLS = new Set([ + 'create_campaign', 'create_plan_item', 'link_plan_items', 'update_plan_item', + 'delete_plan_item', 'propose_plan', 'get_plan_status', 'validate_plan', + ]); + let planRefreshTimer = null; + function schedulePlanRefresh() { + if (planRefreshTimer) clearTimeout(planRefreshTimer); + planRefreshTimer = setTimeout(() => { planRefreshTimer = null; refreshPlanPanel(); }, 600); + } + + function safeStringify(v) { + try { + const s = (typeof v === 'string') ? v : JSON.stringify(v, null, 2); + return s.length > 4000 ? s.slice(0, 4000) + '\n…' : s; + } catch (e) { return String(v); } + } + function fillToolBody(body, act) { + body.innerHTML = ''; + // grid-template-rows reveal (landing.css) needs ONE collapsible child — + // append blocks into a single inner wrapper, not directly onto body. + const inner = document.createElement('div'); + body.appendChild(inner); + const inputStr = (act.input != null) ? safeStringify(act.input) : ''; + const full = act.full || act.summary || ''; + const block = (label, text) => { + const l = document.createElement('div'); l.className = 'v2-act-block-label'; l.textContent = label; + const b = document.createElement('pre'); b.className = 'v2-act-block'; b.textContent = text; + inner.append(l, b); + }; + if (inputStr) block('input', inputStr); + if (full) block('result', full); + if (!inputStr && !full) { + const e = document.createElement('div'); e.className = 'v2-act-block-label'; e.textContent = 'no details'; + inner.append(e); + } + } + function buildToolCard(act, done) { + const card = document.createElement('div'); + card.className = 'v2-act-tool' + (done ? (act.is_error ? ' done err' : ' done') : ''); + const head = document.createElement('button'); + head.className = 'v2-act-tool-head'; head.type = 'button'; + head.setAttribute('aria-expanded', 'false'); + const ic = document.createElement('span'); ic.className = 'v2-act-ic'; + ic.innerHTML = done ? (act.is_error ? '⚠' : '✓') : ''; + const label = document.createElement('span'); label.className = 'v2-act-label'; + label.textContent = act.label || act.name || 'tool'; + const sum = document.createElement('span'); sum.className = 'v2-act-summary'; + sum.textContent = done ? (act.summary || '') : ''; + const chev = document.createElement('span'); chev.className = 'v2-act-chev'; chev.textContent = '›'; + head.append(ic, label, sum, chev); + const body = document.createElement('div'); body.className = 'v2-act-tool-body'; + fillToolBody(body, act); + head.addEventListener('click', () => { + const open = card.classList.toggle('open'); + head.setAttribute('aria-expanded', open ? 'true' : 'false'); + }); + card.append(head, body); + return card; + } + function updateToolCard(card, act) { + card.classList.add('done'); + if (act.is_error) card.classList.add('err'); + const ic = card.querySelector('.v2-act-ic'); if (ic) ic.textContent = act.is_error ? '⚠' : '✓'; + const sum = card.querySelector('.v2-act-summary'); if (sum) sum.textContent = act.summary || ''; + const body = card.querySelector('.v2-act-tool-body'); if (body) fillToolBody(body, act); + } + function captureCampaignId(text) { + if (!text) return; + const s = String(text); + const m = s.match(/campaign_id[=:\s]+([0-9a-f]{6,})/i) || s.match(/\(id:\s*([0-9a-f]{6,})/i); + if (m) capturedCampaignId = m[1]; + } + + function applyActivity(act) { + if (!planActive() || !act) return; + const f = feedEl(); if (!f) return; + switch (act.kind) { + case 'turn_start': + feedTextEl = null; feedThinkingEl = null; pendingNewPage = true; hidePlanError(); clearFallback(); + clearPlanReady(); // new work in flight — drop any "ready" state + showThinking(true, 'reviewing your campaign and plan…'); + break; + case 'thinking': { + // Stream the model's reasoning summary live into the feed as a dim + // block, so the wait shows what the agent is actually considering. + showThinking(true); + const chunk = act.text || ''; + if (!chunk) { setThinkingLabel('thinking through the next step…'); break; } + if (!feedThinkingEl) { + feedThinkingEl = document.createElement('div'); + feedThinkingEl.className = 'v2-act-think'; + feedThinkingEl.style.cssText = + 'font-style:italic;opacity:.7;white-space:pre-wrap;margin:2px 0 8px;font-size:12.5px;line-height:1.5;'; + feedThinkingEl._raw = ''; + feedTarget().appendChild(feedThinkingEl); + } + feedThinkingEl._raw += chunk; + feedThinkingEl.textContent = feedThinkingEl._raw; + feedHadContent = true; + setThinkingLabel('reasoning…'); + scrollFeedIfNearBottom(); + break; + } + case 'text': { + const chunk = act.text || ''; + if (!chunk) break; + // The reasoning that immediately precedes the spoken answer is + // wrap-up meta ("let me wrap this up concisely and offer to + // export…") — drop the block entirely so the feed keeps the + // answer, not the narration of getting there. Reasoning that + // precedes a TOOL is left in place (tool_start only nulls the + // pointer) as the rationale for that action. + if (feedThinkingEl) { feedThinkingEl.remove(); feedThinkingEl = null; } + if (!feedTextEl) { + feedTextEl = document.createElement('div'); + feedTextEl.className = 'v2-act-text'; + feedTextEl._raw = ''; + feedTarget().appendChild(feedTextEl); + } + feedTextEl._raw += chunk; + feedTextEl.innerHTML = renderMd(feedTextEl._raw); + feedHadContent = true; showThinking(true, 'composing the response…'); scrollFeedIfNearBottom(); + break; + } + case 'tool_start': { + // ask_user_choice IS the active question (rendered separately in + // #v2-plan-ask) — don't also show it as a feed card. + if (act.name === 'ask_user_choice') break; + feedTextEl = null; feedThinkingEl = null; + const card = buildToolCard(act, false); + feedTarget().appendChild(card); + (runningTools[act.name] = runningTools[act.name] || []).push(card); + feedHadContent = true; showThinking(true, prettyTool(act)); scrollFeedIfNearBottom(); + break; + } + case 'tool_result': { + captureCampaignId(act.summary); + captureCampaignId(act.full); + if (PLAN_TOOLS.has(act.name)) schedulePlanRefresh(); + if (act.name === 'propose_plan' && !act.is_error) planProposed = true; + if (act.name === 'ask_user_choice') break; + feedTextEl = null; feedThinkingEl = null; + const arr = runningTools[act.name] || []; + const card = arr.pop(); + if (card) updateToolCard(card, act); + else feedTarget().appendChild(buildToolCard(act, true)); + feedHadContent = true; setThinkingLabel('working through the next step…'); scrollFeedIfNearBottom(); + break; + } + case 'turn_end': + showThinking(false); feedTextEl = null; feedThinkingEl = null; + refreshPlanPanel(); + if (!current && !feedHadContent) showFallback(); + // Plan proposed and the agent has settled (no pending question) → + // the design is done. Surface a clear "ready" state instead of + // leaving the user parked on the last wizard step. + if (planProposed && !current) showPlanReady(); + break; + case 'turn_error': + showPlanError(act.error || 'Something went wrong — open the conversation for detail.'); + break; + } + } + + function showFallback() { + const f = feedEl(); if (!f || f.querySelector('.v2-plan-fallback')) return; + const d = document.createElement('div'); + d.className = 'v2-plan-fallback'; + d.innerHTML = 'Gently replied in prose — open the conversation to read it.'; + d.querySelector('a').addEventListener('click', openChat); + feedTarget().appendChild(d); + } + + // ── plan-ready state ─────────────────────────────────────────────── + // Once the agent has proposed the plan and gone quiet, the wizard is done. + // Mark the screen "ready": rename the header, count phases/items from the + // panel, and promote "open workspace" to the obvious primary action — so the + // finish line is signposted instead of looking like one more wizard step. + function planCounts() { + let phases = 0, items = 0; + planPages.forEach(p => { + if (p.name !== 'Tasks') phases++; + items += (p.items || []).length; + }); + return { phases, items }; + } + function showPlanReady() { + const sec = document.querySelector('.v2-screen-plan'); + if (!sec) return; + sec.classList.add('ready'); + showThinking(false); + const who = sec.querySelector('.v2-plan-who'); + const title = sec.querySelector('.v2-plan-title'); + if (who) who.textContent = 'Gently · plan ready'; + if (title) { + const { phases, items } = planCounts(); + title.textContent = items + ? `Your plan is ready — ${items} item${items === 1 ? '' : 's'} across ${phases} phase${phases === 1 ? '' : 's'}` + : 'Your plan is ready'; + } + const cont = $('v2-plan-continue'); + if (cont) cont.textContent = 'Open the workspace ›'; + const exp = $('v2-plan-export'); + if (exp) exp.hidden = false; // the plan is final → offer the download + } + function clearPlanReady() { + const sec = document.querySelector('.v2-screen-plan'); + if (!sec || !sec.classList.contains('ready')) return; + sec.classList.remove('ready'); + const who = sec.querySelector('.v2-plan-who'); + const title = sec.querySelector('.v2-plan-title'); + if (who) who.textContent = 'Gently · planning'; + if (title) title.textContent = "Let's design your run"; + const cont = $('v2-plan-continue'); + if (cont) cont.textContent = 'Continue in workspace ›'; + const exp = $('v2-plan-export'); + if (exp) exp.hidden = true; + } + + // ── export the finished plan as a shareable markdown doc ──────────── + // Replaces the agent's end-of-plan "want me to export this?" prose with a + // real action: pull the enriched plan tree (/export) and render it to + // markdown client-side so the biologist can drop it in a doc or share it. + function specToLines(spec) { + let s = spec; + if (typeof s === 'string') { try { s = JSON.parse(s); } catch { return s ? ['- ' + s] : []; } } + if (!s || typeof s !== 'object') return []; + const out = []; + const fmt = (v) => Array.isArray(v) ? v.join(', ') : (typeof v === 'object' ? JSON.stringify(v) : String(v)); + const pick = (k, label) => { if (s[k] != null && s[k] !== '') out.push(`- **${label}:** ${fmt(s[k])}`); }; + pick('strain', 'Strain'); pick('goal', 'Goal'); + if (Array.isArray(s.channels) && s.channels.length) { + out.push('- **Channels:** ' + s.channels.map(c => `${c.name || '?'} (${c.excitation_nm || '?'} nm${c.exposure_ms ? `, ${c.exposure_ms} ms` : ''})`).join(', ')); + } + pick('num_slices', 'Slices'); pick('interval_s', 'Interval (s)'); pick('temperature_c', 'Temperature (°C)'); + pick('num_embryos', 'Embryos'); pick('start_stage', 'Start stage'); pick('stop_condition', 'Stop condition'); + pick('criteria', 'Criteria'); pick('success_criteria', 'Success criteria'); + return out; + } + function buildPlanMarkdown(tree) { + const L = []; + L.push(`# ${tree.description || tree.shorthand || 'Experimental plan'}`, ''); + if (tree.target) L.push(`**Goal:** ${tree.target}`, ''); + if (tree.shorthand) L.push(`**Plan ID:** \`${tree.shorthand}\``, ''); + L.push(`_Exported from Gently — ${new Date().toLocaleString()}_`, ''); + const renderItems = (items, prefix) => { + (items || []).slice().sort((a, b) => (a.phase_order || 0) - (b.phase_order || 0)).forEach((it, idx) => { + L.push(`### ${prefix}${idx + 1} ${it.title || '(task)'} \`${it.type || 'task'}\``, ''); + if (it.description) L.push(it.description, ''); + const sl = specToLines(it.spec); + if (sl.length) L.push(...sl, ''); + const refs = it.references || []; + if (refs.length) { + L.push('**References:**'); + refs.forEach((r, i) => L.push(`${i + 1}. ${r.citation || r.id || ''}${r.source ? ` _(${r.source})_` : ''}`)); + L.push(''); + } + }); + }; + if ((tree.items || []).length) { L.push('## Tasks', ''); renderItems(tree.items, ''); } + (tree.children || []).forEach((ph, pi) => { + if (!ph) return; + L.push(`## ${ph.display_name || ph.description || ph.shorthand || `Phase ${pi + 1}`}`, ''); + if (ph.target) L.push(ph.target, ''); + renderItems(ph.items, `${pi + 1}.`); + }); + return L.join('\n').replace(/\n{3,}/g, '\n\n').trimEnd() + '\n'; + } + async function resolveCampaignId() { + if (capturedCampaignId) return capturedCampaignId; + try { + const r = await fetch('/api/campaigns'); + if (r.ok) { const d = await r.json(); const t = (d.campaigns || [])[0]; return (t && t.campaign && t.campaign.id) || null; } + } catch (e) { /* offline */ } + return null; + } + async function exportPlan() { + const btn = $('v2-plan-export'); + const id = await resolveCampaignId(); + if (!id) { showPlanError('No plan to export yet.'); return; } + if (btn) { btn.disabled = true; btn.textContent = '↓ Exporting…'; } + try { + const r = await fetch(`/api/campaigns/${encodeURIComponent(id)}/export`); + if (!r.ok) throw new Error(`export ${r.status}`); + const tree = await r.json(); + const md = buildPlanMarkdown(tree); + const blob = new Blob([md], { type: 'text/markdown' }); + const url = URL.createObjectURL(blob); + const a = document.createElement('a'); + a.href = url; + a.download = `${(tree.shorthand || 'plan').replace(/[^\w.-]+/g, '_')}.md`; + document.body.appendChild(a); a.click(); a.remove(); + setTimeout(() => URL.revokeObjectURL(url), 1000); + } catch (e) { + showPlanError('Could not export the plan — open the conversation to export it manually.'); + } finally { + if (btn) { btn.disabled = false; btn.textContent = '↓ Export plan'; } + } + } + + // ── THE PLAN panel: mirror the real campaign tree ────────────────── + async function refreshPlanPanel() { + try { + let tree = null; + if (capturedCampaignId) { + const r = await fetch(`/api/campaigns/${encodeURIComponent(capturedCampaignId)}/tree`); + if (r.ok) tree = await r.json(); + } + if (!tree) { + const r = await fetch('/api/campaigns'); + if (r.ok) { const d = await r.json(); tree = (d.campaigns || [])[0] || null; } + } + if (tree) renderPlanTree(tree); + } catch (e) { /* keep whatever is shown */ } + } + function planName(c) { + c = c || {}; + return c.shorthand || c.display_name || c.description || 'Plan'; + } + // Phases read better by their human name ("Phase 1 — Reporter validation") + // than by their code shorthand ("nrp-p1"), which looks like a machine id. + function phaseName(c) { + c = c || {}; + return c.display_name || c.description || c.shorthand || 'Phase'; + } + // THE PLAN renders as a pager — one phase per page with ‹ Prev / Next ›, + // a position label, and dots — instead of one long scroll. planPage is held + // across re-renders (the panel refetches on every plan-writing tool) so the + // page you're reading doesn't snap back to the start mid-design. + let planPage = 0; + let planPages = []; // [{ name, items }] + let planTitleText = ''; + + function renderPlanTree(tree) { + if (!tree) return; + const phases = tree.children || []; + const rootItems = tree.items || []; + if (!phases.length && !rootItems.length) return; // nothing to show yet — keep placeholder + const pages = []; + if (rootItems.length) pages.push({ name: 'Tasks', items: rootItems }); + phases.forEach(phase => { + if (!phase) return; + pages.push({ name: phaseName(phase.campaign), items: phase.items || [] }); + }); + planPages = pages; + planTitleText = planName(tree.campaign); + if (planPage >= pages.length) planPage = pages.length - 1; + if (planPage < 0) planPage = 0; + drawPlanPage(); + } + + function drawPlanPage() { + const list = $('v2-plan-summary'); + if (!list) return; + const pages = planPages; + const n = pages.length; + if (!n) return; + const i = Math.min(Math.max(planPage, 0), n - 1); + const page = pages[i]; + + list.innerHTML = ''; + const title = document.createElement('div'); + title.className = 'v2-plan-title-row'; + title.textContent = planTitleText; + list.appendChild(title); + + if (n > 1) { + const bar = document.createElement('div'); + bar.className = 'v2-plan-pager'; + const prev = document.createElement('button'); + prev.className = 'v2-plan-pager-btn'; prev.type = 'button'; prev.textContent = '‹ Prev'; + prev.disabled = i === 0; + prev.addEventListener('click', () => { if (planPage > 0) { planPage--; drawPlanPage(); } }); + const pos = document.createElement('span'); + pos.className = 'v2-plan-pager-pos'; + pos.textContent = page.name; // position shown by the dots below + pos.title = `${page.name} · ${i + 1} of ${n}`; + const next = document.createElement('button'); + next.className = 'v2-plan-pager-btn'; next.type = 'button'; next.textContent = 'Next ›'; + next.disabled = i === n - 1; + next.addEventListener('click', () => { if (planPage < n - 1) { planPage++; drawPlanPage(); } }); + bar.append(prev, pos, next); + list.appendChild(bar); + } else { + const h = document.createElement('div'); + h.className = 'v2-plan-phase-h'; + h.textContent = page.name; + list.appendChild(h); + } + + const tasks = document.createElement('div'); + tasks.className = 'v2-plan-phase'; + const items = page.items || []; + // phase ordinal (1-based) for "P.I" numbering; the rootItems "Tasks" page + // isn't a phase, so it numbers items bare (1, 2, …). + const phaseOrd = pages.slice(0, i + 1).filter(p => p.name !== 'Tasks').length; + if (items.length) { + items.forEach((it, idx) => { + const type = String(it.type || '').toLowerCase(); + const t = document.createElement('div'); + t.className = 'v2-plan-task type-' + (type || 'other'); + const num = document.createElement('span'); + num.className = 'v2-task-num'; + num.textContent = phaseOrd ? `${phaseOrd}.${idx + 1}` : `${idx + 1}`; + const dot = document.createElement('span'); + dot.className = 'v2-task-dot'; + dot.title = type || 'task'; + const ttl = document.createElement('span'); + ttl.className = 'v2-task-ttl'; + ttl.textContent = it.title || it.shorthand || '(task)'; + t.append(num, dot, ttl); + if (it.estimated_days) { + const d = document.createElement('span'); + d.className = 'v2-task-days'; + d.textContent = `${it.estimated_days}d`; + t.append(d); + } + tasks.appendChild(t); + }); + } else { + const e = document.createElement('div'); + e.className = 'v2-plan-task v2-plan-task-empty'; + e.textContent = 'no items in this phase yet'; + tasks.appendChild(e); + } + list.appendChild(tasks); + + if (n > 1) { + const dots = document.createElement('div'); + dots.className = 'v2-plan-dots'; + for (let d = 0; d < n; d++) { + const dot = document.createElement('button'); + dot.className = 'v2-plan-dot' + (d === i ? ' active' : ''); + dot.type = 'button'; + dot.setAttribute('aria-label', `Go to ${pages[d].name} (${d + 1} of ${n})`); + dot.addEventListener('click', () => { planPage = d; drawPlanPage(); }); + dots.appendChild(dot); + } + list.appendChild(dots); + } + } + + // ── ask rendering (the active question) ──────────────────────────── + function labelFor(data, sel) { + const opts = (data && data.options) || []; + const one = (s) => { + const o = opts.find(o => o && (o.id === s || o.value === s || o.label === s)); + return o ? o.label : String(s); + }; + return Array.isArray(sel) ? sel.map(one).join(', ') : one(sel); + } + function recordPick(data, sel) { + const list = $('v2-plan-summary'); + if (!list) return; + const empty = list.querySelector('.v2-plan-side-empty'); + if (empty) empty.remove(); + const matched = (data && data.options || []).some(o => o && (o.id === sel || o.value === sel || o.label === sel)); + const row = document.createElement('div'); + row.className = 'v2-plan-row' + (matched ? '' : ' v2-plan-row-freetext'); + row.innerHTML = ''; + row.querySelector('.k').textContent = (data && data.question) || 'Choice'; + row.querySelector('.v').textContent = labelFor(data, sel); + list.appendChild(row); + } + function renderAsk() { + const mount = $('v2-plan-ask'); + if (!mount || !current || typeof AgentChat === 'undefined' || !AgentChat.buildAskCard) return; + showThinking(false); hidePlanError(); clearFallback(); + const data = current.data, reqId = current.reqId; + const hasControl = AgentChat.hasControl ? AgentChat.hasControl() : true; + const card = AgentChat.buildAskCard(data, { + reqId, isWake: current.isWake, hasControl, + onPick: (sel) => { + recordPick(data, sel); + AgentChat.answerChoice(reqId, sel); + current = null; clearAsk(); showThinking(true); + }, + }); + mount.innerHTML = ''; + mount.appendChild(card); + const first = mount.querySelector('button:not([disabled])'); + if (first) setTimeout(() => first.focus(), 30); + } + + let planKickedOff = false; // guard: design-kickoff fires once per session + async function startPlan() { + setScreen('plan'); + // Re-entering the wizard (Back → Plan again) must NOT re-fire the + // kickoff — that stacked duplicate "/plan" + design turns. Just show + // the wizard with its existing state. + if (planKickedOff) return; + planKickedOff = true; + resetSummary(); clearAsk(); clearActivity(); + current = null; + showThinking(true); + // Campaigns are persistent agent memory (not session state), so the + // agent always builds on an existing one — which leaves a user wanting a + // fresh plan stuck. So if an active campaign exists, ask up front: + // continue it (the default) or start a brand-new one. With NO campaign + // there's nothing to continue, so skip the gate and design straight away + // (that path is fresh anyway). + let campaign = null; + try { + const r = await fetch('/api/campaigns'); + if (r.ok) { const d = await r.json(); campaign = (d.campaigns || [])[0] || null; } + } catch (e) { /* offline / no API — just design */ } + if (campaign) renderCampaignChoice(campaign); + else kickoffDesign('continue'); + } + + // Enter plan mode, then prompt design. The prompt differs by intent: build + // on the active campaign, or set it aside and create a new one. A free-typed + // answer from the choice card becomes the design brief directly. + function kickoffDesign(mode) { + showThinking(true); + if (typeof AgentChat === 'undefined' || !AgentChat.runCommand) return; + AgentChat.runCommand('/plan'); + if (mode === 'fresh') { + AgentChat.runCommand( + "I want to start a brand-new experiment, not continue any existing " + + "campaign. Create a new campaign and let's design it from scratch — " + + "what should we capture?" + ); + } else if (mode === 'continue') { + AgentChat.runCommand("Let's design this run — what should it capture?"); + } else { + // free text from the choice card's "Something else…" escape + AgentChat.runCommand(String(mode)); + } + } + + // Continue-vs-fresh gate, shown only when an active campaign exists. Reuses + // the agent ask-card styling so it's visually identical to the agent's own + // questions; picking routes into kickoffDesign rather than the agent bridge. + function renderCampaignChoice(tree) { + const mount = $('v2-plan-ask'); + if (!mount || typeof AgentChat === 'undefined' || !AgentChat.buildAskCard) { + kickoffDesign('continue'); + return; + } + showThinking(false); hidePlanError(); clearFallback(); + const name = planName((tree && tree.campaign) || {}); + const data = { + question: `You have an active campaign — **${name}**. Design the next run inside it, or start something new?`, + options: [ + { id: 'continue', label: `Continue ${name}`, description: 'Design the next run inside your existing campaign' }, + { id: 'fresh', label: 'Start a brand-new campaign', description: 'Set the existing plan aside and plan from scratch' }, + ], + }; + const hasControl = AgentChat.hasControl ? AgentChat.hasControl() : true; + const card = AgentChat.buildAskCard(data, { + reqId: 'landing-campaign-choice', isWake: false, hasControl, + onPick: (sel) => { clearAsk(); kickoffDesign(sel); }, + }); + mount.innerHTML = ''; + mount.appendChild(card); + const first = mount.querySelector('button:not([disabled])'); + if (first) setTimeout(() => first.focus(), 30); + } + + function openScope() { + dismiss(); + if (typeof switchTab === 'function') switchTab('devices'); + } + function openChat() { + dismiss(); + if (typeof AgentChat !== 'undefined' && AgentChat.togglePanel) { + setTimeout(() => AgentChat.togglePanel(true), 300); + } + } + function sendFreeform(text) { + const v = (text || '').trim(); + dismiss(); + if (typeof AgentChat !== 'undefined' && AgentChat.togglePanel) { + AgentChat.togglePanel(true); + if (v && AgentChat.runCommand) setTimeout(() => AgentChat.runCommand(v), 300); + } + } + + function init() { + el = $('v2-landing'); + if (!el || typeof ClientEventBus === 'undefined') return; // flag off → no-op + greet(); + + el.querySelectorAll('[data-landing]').forEach(btn => btn.addEventListener('click', () => { + const kind = btn.dataset.landing; + if (kind === 'plan') startPlan(); + else if (kind === 'standalone') openScope(); + })); + + const esc = $('v2-escape'), escToggle = $('v2-escape-toggle'), + escInput = $('v2-escape-input'), escSend = $('v2-escape-send'); + if (escToggle && esc && escInput) { + escToggle.addEventListener('click', () => { + const open = esc.classList.toggle('open'); + escToggle.setAttribute('aria-expanded', open ? 'true' : 'false'); + if (open) setTimeout(() => escInput.focus(), 120); + }); + const submit = () => sendFreeform(escInput.value); + if (escSend) escSend.addEventListener('click', submit); + escInput.addEventListener('keydown', e => { + if (e.key === 'Enter') { e.preventDefault(); submit(); } + else if (e.key === 'Escape') { e.stopPropagation(); esc.classList.remove('open'); escToggle.setAttribute('aria-expanded', 'false'); } + }); + } + + const skip = $('v2-landing-skip'); + if (skip) skip.addEventListener('click', dismiss); + + // Theme toggle (header's is occluded by the overlay). Mirrors + // _header.html: flip data-theme on both roots + persist. + const themeBtn = $('v2-landing-theme'); + if (themeBtn) themeBtn.addEventListener('click', () => { + const cur = document.documentElement.getAttribute('data-theme') + || document.body.getAttribute('data-theme') || 'light'; + const next = cur === 'dark' ? 'light' : 'dark'; + document.documentElement.setAttribute('data-theme', next); + document.body.setAttribute('data-theme', next); + localStorage.setItem('gently-theme', next); + }); + + const back = $('v2-plan-back'); + if (back) back.addEventListener('click', () => setScreen('welcome')); + const planChat = $('v2-plan-chat'); + if (planChat) planChat.addEventListener('click', openChat); + const cont = $('v2-plan-continue'); + if (cont) cont.addEventListener('click', dismiss); + const exp = $('v2-plan-export'); + if (exp) exp.addEventListener('click', exportPlan); + + // The agent's questions + work render in the plan stage while it's active; + // once we've receded into the workspace, AskStage (#ask-stage) takes over. + ClientEventBus.on('AGENT_ASK', ({ request_id, choice_data, origin }) => { + if (!planActive()) return; + current = { reqId: request_id, data: choice_data || {}, isWake: origin === 'wake' }; + renderAsk(); + }); + ClientEventBus.on('ASK_CLEARED', ({ request_id }) => { + if (request_id === '*' || (current && request_id === current.reqId)) { + current = null; clearAsk(); + if (planActive() && !errorVisible()) showThinking(true); + // A question was just answered — the agent's continuation is the + // next step, so open a fresh feed page for it. (A turn stays one + // stream across an ask_user_choice pause, so turn_start alone + // would lump every step of the design into a single page.) + pendingNewPage = true; + } + }); + ClientEventBus.on('AGENT_CONTROL', () => { if (current && planActive()) renderAsk(); }); + ClientEventBus.on('AGENT_ACTIVITY', (act) => applyActivity(act)); + + document.addEventListener('keydown', e => { + if (e.key !== 'Escape' || !el || el.classList.contains('dismissed')) return; + if (el.dataset.screen === 'plan') setScreen('welcome'); // step back, don't bail + else dismiss(); + }); + } + + document.addEventListener('DOMContentLoaded', init); + + return { + dismiss, + show: () => { + if (!el) return; + el.style.display = ''; + el.removeAttribute('aria-hidden'); + el.classList.remove('dismissed'); + setScreen('welcome'); + greet(); + }, + }; +})(); diff --git a/gently/ui/web/static/js/logconsole.js b/gently/ui/web/static/js/logconsole.js new file mode 100644 index 00000000..36fac475 --- /dev/null +++ b/gently/ui/web/static/js/logconsole.js @@ -0,0 +1,217 @@ +/** + * Process console — the header drawer over the agent and device-layer output. + * + * The desktop shell spawns the backend with no console window, so when + * something misbehaves mid-session there is nowhere to look. This reads the + * same two streams the operator would have had on a terminal: + * + * Agent {storage}/logs/gently_*.log via /api/logs/agent + * Device layer supervisor-captured stdout via /api/device-layer/log, + * falling back to the layer's own log file when it runs + * externally and the supervisor captured nothing. + * + * Polling runs ONLY while the drawer is open. A console that keeps fetching + * behind a closed panel is exactly the kind of hidden work that made the camera + * streams fight each other. + */ +const LogConsole = (function () { + const POLL_MS = 2000; + const LIMIT = 600; + + let _wired = false, _open = false, _src = 'agent', _follow = true; + let _timer = null, _inFlight = false, _dirty = false, _lastKey = ''; + const D = {}; + + const $ = id => document.getElementById(id); + + function cacheDom() { + ['logc', 'logc-open', 'logc-scrim', 'logc-close', 'logc-body', 'logc-file', + 'logc-status', 'logc-level', 'logc-follow', 'logc-copy', 'logc-dot'] + .forEach(id => { D[id] = $(id); }); + } + + // ── fetching ────────────────────────────────────────────────────────── + async function fetchAgent(level) { + const q = new URLSearchParams({ limit: String(LIMIT) }); + if (level) q.set('level', level); + const r = await fetch(`/api/logs/agent?${q}`); + if (!r.ok) throw new Error(`${r.status}`); + return r.json(); + } + + async function fetchDevice(level) { + // Prefer the supervisor's live captured stdout — it exists even before + // anything has been flushed to a file. + try { + const r = await fetch(`/api/device-layer/log?limit=${LIMIT}`); + if (r.ok) { + const d = await r.json(); + if (d && Array.isArray(d.lines) && d.lines.length) { + return { file: 'device layer (captured stdout)', lines: d.lines }; + } + } + } catch (_) { /* fall through to the file */ } + const q = new URLSearchParams({ limit: String(LIMIT) }); + if (level) q.set('level', level); + const r2 = await fetch(`/api/logs/device?${q}`); + if (!r2.ok) throw new Error(`${r2.status}`); + return r2.json(); + } + + async function refresh() { + if (!_open) return; + // A refresh asked for while a poll is in flight must not be dropped — + // changing the tab or the level filter would otherwise appear to do + // nothing until the next tick. Remember it and re-run on completion. + if (_inFlight) { _dirty = true; return; } + _inFlight = true; + const level = D['logc-level'] ? D['logc-level'].value : ''; + try { + const d = _src === 'agent' ? await fetchAgent(level) : await fetchDevice(level); + render(d); + setStatus(''); + } catch (e) { + setStatus(`could not read log (${e.message})`); + } finally { + _inFlight = false; + if (_dirty) { _dirty = false; refresh(); } + } + } + + // ── rendering ───────────────────────────────────────────────────────── + function render(d) { + const lines = (d && d.lines) || []; + D['logc-file'].textContent = d && d.file ? d.file : 'no log file yet'; + // Skip the DOM write when nothing changed, so following does not fight + // a user who has scrolled up to read something. + const key = `${lines.length}|${lines[lines.length - 1] || ''}`; + if (key === _lastKey) return; + _lastKey = key; + + if (!lines.length) { + // An empty result under a filter is not an empty log — say which. + const level = D['logc-level'] ? D['logc-level'].value : ''; + if (level) { + D['logc-body'].textContent = + `No ${level.toLowerCase()} lines in the last ${LIMIT}. Set the filter to All to see everything.`; + } else { + D['logc-body'].textContent = _src === 'agent' + ? 'Nothing logged yet. The agent writes to {storage}/logs/gently_*.log.' + : 'Nothing logged yet. Start the device layer from the Devices tab.'; + } + return; + } + + const frag = document.createDocumentFragment(); + for (const ln of lines) { + const row = document.createElement('span'); + row.className = 'logc-line' + severityClass(ln); + row.textContent = ln; + frag.appendChild(row); + } + D['logc-body'].replaceChildren(frag); + + if (_follow) D['logc-body'].scrollTop = D['logc-body'].scrollHeight; + flagProblems(lines); + } + + function severityClass(ln) { + if (/\b(ERROR|CRITICAL|Traceback)\b/.test(ln)) return ' is-error'; + if (/\bWARNING\b/.test(ln)) return ' is-warn'; + return ''; + } + + // A quiet dot on the header button when the tail contains errors, so a + // failure that happens while the drawer is closed is still noticed. + function flagProblems(lines) { + if (!D['logc-dot']) return; + const bad = lines.slice(-80).some(ln => /\b(ERROR|CRITICAL|Traceback)\b/.test(ln)); + D['logc-dot'].hidden = !bad; + } + + function setStatus(msg) { + if (D['logc-status']) D['logc-status'].textContent = msg || ''; + } + + // ── open / close ────────────────────────────────────────────────────── + function open() { + if (_open) return; + _open = true; + D['logc'].hidden = false; + D['logc-open'].setAttribute('aria-expanded', 'true'); + _lastKey = ''; + refresh(); + _timer = setInterval(refresh, POLL_MS); + D['logc-body'].focus(); + } + + function close() { + if (!_open) return; + _open = false; + D['logc'].hidden = true; + D['logc-open'].setAttribute('aria-expanded', 'false'); + if (_timer) { clearInterval(_timer); _timer = null; } + } + + function toggle() { _open ? close() : open(); } + + function selectSource(src) { + if (src === _src) return; + _src = src; + _lastKey = ''; + document.querySelectorAll('.logc-tab').forEach(t => + t.classList.toggle('is-active', t.dataset.src === src)); + D['logc-body'].textContent = ''; + refresh(); + } + + function wire() { + if (_wired) return; + cacheDom(); + if (!D['logc'] || !D['logc-open']) return; // page without the header + _wired = true; + + D['logc-open'].addEventListener('click', toggle); + D['logc-close'].addEventListener('click', close); + D['logc-scrim'].addEventListener('click', close); + document.querySelectorAll('.logc-tab').forEach(t => + t.addEventListener('click', () => selectSource(t.dataset.src))); + D['logc-level'].addEventListener('change', () => { _lastKey = ''; refresh(); }); + + D['logc-follow'].addEventListener('click', () => { + _follow = !_follow; + D['logc-follow'].setAttribute('aria-pressed', String(_follow)); + D['logc-follow'].classList.toggle('is-on', _follow); + if (_follow) D['logc-body'].scrollTop = D['logc-body'].scrollHeight; + }); + D['logc-follow'].classList.add('is-on'); + + // Scrolling up is an intent to read — stop yanking the view to the end. + D['logc-body'].addEventListener('scroll', () => { + const el = D['logc-body']; + const atEnd = el.scrollHeight - el.scrollTop - el.clientHeight < 24; + if (_follow !== atEnd) { + _follow = atEnd; + D['logc-follow'].setAttribute('aria-pressed', String(_follow)); + D['logc-follow'].classList.toggle('is-on', _follow); + } + }); + + D['logc-copy'].addEventListener('click', async () => { + try { + await navigator.clipboard.writeText(D['logc-body'].textContent || ''); + setStatus('copied'); + setTimeout(() => setStatus(''), 1500); + } catch (_) { setStatus('copy blocked by the browser'); } + }); + + document.addEventListener('keydown', e => { + if (e.key === 'Escape' && _open) { e.preventDefault(); close(); return; } + // Ctrl/Cmd+` — the shell convention for "show me the console". + if ((e.ctrlKey || e.metaKey) && e.key === '`') { e.preventDefault(); toggle(); } + }); + } + + document.addEventListener('DOMContentLoaded', wire); + return { open, close, toggle }; +})(); diff --git a/gently/ui/web/static/js/notebook.js b/gently/ui/web/static/js/notebook.js new file mode 100644 index 00000000..1130a31a --- /dev/null +++ b/gently/ui/web/static/js/notebook.js @@ -0,0 +1,196 @@ +/** + * NotebookApp — the LIBRARY "Notebook" tab. + * + * The reading room for the shared lab notebook: a thread rail (the inquiry + * spine) + kind filter, rendering Notes from the read API (/api/notebook). + * Read-only for now; authoring/curation arrive in a later increment. + */ +const NotebookApp = (() => { + let inited = false; + let kindFilter = ''; // '' | observation | finding | question + let threadFilter = ''; // '' = all notes + + const $ = (id) => document.getElementById(id); + + async function fetchJSON(url) { + try { + const r = await fetch(url); + if (!r.ok) return null; + return await r.json(); + } catch (e) { + return null; + } + } + + function kindMeta(kind) { + return ({ + observation: { label: 'Observation', cls: 'nb-k-obs' }, + finding: { label: 'Finding', cls: 'nb-k-find' }, + question: { label: 'Question', cls: 'nb-k-q' }, + })[kind] || { label: kind || 'note', cls: '' }; + } + + async function loadThreads() { + const rail = $('nb-threads'); + if (!rail) return; + const data = await fetchJSON('/api/notebook/threads'); + const threads = (data && data.threads) || []; + rail.innerHTML = ''; + const mk = (id, label, count, active) => { + const b = document.createElement('button'); + b.className = 'nb-thread' + (active ? ' active' : ''); + b.textContent = label + (count != null ? ` ${count}` : ''); + b.addEventListener('click', () => { threadFilter = id; loadThreads(); loadNotes(); }); + return b; + }; + rail.appendChild(mk('', 'All notes', null, threadFilter === '')); + threads.forEach(t => rail.appendChild(mk(t.id, t.id, t.count, threadFilter === t.id))); + } + + function card(n) { + const km = kindMeta(n.kind); + const el = document.createElement('div'); + el.className = 'nb-card'; + + const head = document.createElement('div'); + head.className = 'nb-card-head'; + const badge = document.createElement('span'); + badge.className = 'nb-badge ' + km.cls; + badge.textContent = km.label; + const author = document.createElement('span'); + author.className = 'nb-author'; + author.textContent = n.author || ''; + const status = document.createElement('span'); + status.className = 'nb-status'; + status.textContent = n.status || ''; + head.append(badge, author, status); + + const body = document.createElement('div'); + body.className = 'nb-body-text'; + body.textContent = n.title || n.body || ''; + + el.append(head, body); + + const chips = [] + .concat((n.strains || []).map(s => '🧬 ' + s)) + .concat((n.embryos || []).map(e => '◌ ' + e)) + .concat((n.threads || []).map(t => '# ' + t)); + if (chips.length) { + const row = document.createElement('div'); + row.className = 'nb-chips'; + chips.forEach(text => { + const c = document.createElement('span'); + c.className = 'nb-chip'; + c.textContent = text; + row.appendChild(c); + }); + el.appendChild(row); + } + return el; + } + + async function loadNotes() { + const list = $('nb-notes'); + if (!list) return; + const params = new URLSearchParams(); + if (kindFilter) params.set('kind', kindFilter); + if (threadFilter) params.set('thread', threadFilter); + const qs = params.toString(); + const data = await fetchJSON('/api/notebook/notes' + (qs ? `?${qs}` : '')); + if (!data || data.available === false) { + list.innerHTML = '
    Notebook unavailable.
    '; + return; + } + const notes = data.notes || []; + if (!notes.length) { + list.innerHTML = + '
    No notes yet — the notebook fills as the agent ' + + 'records observations, findings, and open questions.
    '; + return; + } + list.innerHTML = ''; + notes.forEach(n => list.appendChild(card(n))); + } + + function setupFilters() { + document.querySelectorAll('#notebook-content [data-nb-kind]').forEach(btn => { + btn.addEventListener('click', () => { + kindFilter = btn.dataset.nbKind; + document.querySelectorAll('#notebook-content [data-nb-kind]') + .forEach(b => b.classList.toggle('active', b === btn)); + loadNotes(); + }); + }); + } + + // ── Ask the notebook ─────────────────────────────────────────────── + function renderAskResult(data) { + const box = $('nb-ask-result'); + if (!box) return; + box.hidden = false; + if (!data || data.available === false) { + box.innerHTML = '
    The notebook is unavailable right now.
    '; + return; + } + const cov = data.coverage || 'covered'; + const covLabel = { covered: 'Grounded in the notebook', partial: 'Partially covered', not_in_notebook: 'Not in the notebook yet' }[cov] || cov; + const points = (data.points || []).map(p => ` +
  • + ${esc(p.text)} + ${(p.note_ids || []).map(id => `${esc(id)}`).join('')} +
  • `).join(''); + const nexts = (data.suggested_next || []).map(s => `
  • ${esc(s)}
  • `).join(''); + box.innerHTML = + `
    ${esc(covLabel)}
    ` + + `
    ${esc(data.answer || '')}
    ` + + (points ? `
    Why
      ${points}
    ` : '') + + (nexts ? `
    Try next
      ${nexts}
    ` : ''); + } + + async function ask() { + const input = $('nb-ask-input'); + const box = $('nb-ask-result'); + const q = (input && input.value || '').trim(); + if (!q) return; + if (box) { box.hidden = false; box.innerHTML = '
    Thinking over the notebook…
    '; } + const body = { question: q }; + if (threadFilter) body.thread = threadFilter; // ask within the selected thread + try { + const r = await fetch('/api/notebook/ask', { + method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body), + }); + renderAskResult(await r.json()); + } catch (e) { + if (box) box.innerHTML = '
    Something went wrong asking the notebook.
    '; + } + } + + function setupAsk() { + const go = $('nb-ask-go'), input = $('nb-ask-input'); + if (go) go.addEventListener('click', ask); + if (input) input.addEventListener('keydown', e => { if (e.key === 'Enter') { e.preventDefault(); ask(); } }); + } + + function esc(s) { + return (typeof escapeHtml === 'function') ? escapeHtml(String(s == null ? '' : s)) + : String(s == null ? '' : s).replace(/[&<>"]/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c])); + } + + function refresh() { loadThreads(); loadNotes(); } + + function init() { + if (inited) { refresh(); return; } + inited = true; + setupFilters(); + setupAsk(); + refresh(); + // Notebook writes ride the CONTEXT_UPDATED event — live-refresh if visible. + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('CONTEXT_UPDATED', () => { + if (typeof state !== 'undefined' && state.tab === 'notebook') refresh(); + }); + } + } + + return { init }; +})(); diff --git a/gently/ui/web/static/js/occupancy3d.js b/gently/ui/web/static/js/occupancy3d.js new file mode 100644 index 00000000..b9cfb7ee --- /dev/null +++ b/gently/ui/web/static/js/occupancy3d.js @@ -0,0 +1,489 @@ +// ══════════════════════════════════════════════════════════════════════ +// 3D Optical Space — live digital-twin of the addressable imaging volume +// +// Renders the acquisition cuboid (the box of voxels being scanned) with the +// live light-sheet plane inside it, plus a Z-neighbourhood reference frame. +// An HTML overlay (mode badge + readouts + a top-down minimap) carries the +// GLOBAL context: where in the addressable XY stage range this cuboid sits, +// and the embryos around it. +// +// Why two representations: the addressable stage XY (~tens of mm), the cuboid +// footprint (~hundreds of µm) and the piezo Z range (~µm) differ by ~100x, so +// a single literal-scale 3D box would draw the cuboid invisibly small. The 3D +// scene therefore stays in one µm scale around the cuboid; the minimap (2D) +// handles the much larger stage extent. Some scales are local by design — see +// FOV_UM / the outer-frame sizing below. +// +// Data: +// DEVICE_STATE_UPDATE → live Piezo.Position (sheet Z), Galvo.A/B, XYStage, +// and the firmware box (minimap extent). +// SCAN_GEOMETRY_UPDATE → cuboid extents, num_slices, pencil/sheet mode. +// EMBRYOS_UPDATE → minimap markers. +// Bootstrap via /api/devices/scan_geometry + /api/embryos/current. +// +// Mirrors the DevicesManager IIFE pattern (devices.js) and reuses the +// Three.js scaffold + drag-orbit from projection-viewer.js. +// ══════════════════════════════════════════════════════════════════════ + +const Occupancy3DManager = (function () { + 'use strict'; + + // --- Tunables / approximations (v1) -------------------------------- + // Camera FOV footprint of the SPIM cuboid in µm. SPIM is 0.1625 µm/px; + // a ~2048px sCMOS ROI ≈ 333 µm. Not currently streamed, so we use a + // constant until SCAN_GEOMETRY_UPDATE carries fov_um. (Documented approx.) + const FOV_UM = 333.0; + const MAX_SLICE_LINES = 30; // cap drawn slice outlines for perf + const COLORS = { + outer: 0x33414d, + cuboid: 0x14b8c4, + cuboidFace: 0x14b8c4, + sheet: 0x39d0ff, + slice: 0x2a6f78, + beam: 0xffd166, + }; + + // --- Module state -------------------------------------------------- + let _initialized = false; + let _scene = null, _camera = null, _renderer = null, _root = null; + let _animationId = null, _resizeObserver = null, _resizeRaf = null, _onLayoutChanged = null; + let _isDragging = false, _prevMouse = { x: 0, y: 0 }; + const _rot = { x: -0.6, y: 0.6 }; + let _zoom = 1.7; + + // Live data caches + let _geom = null; // last SCAN_GEOMETRY_UPDATE.data + let _firmwareBox = null; // {x:[min,max], y:[min,max]} µm + let _stage = { x: null, y: null }; + let _piezoZ = null; // live axial position (µm) + let _galvo = { a: null, b: null }; + let _embryos = []; // [{x,y,role,id}] + let _scaler = null; + + // Scene object handles (rebuilt as geometry changes) + let _outerBox = null, _cuboid = null, _cuboidEdges = null; + let _sheet = null, _beam = null, _sliceGroup = null; + + // DOM + let _container = null, _modeEl = null, _readoutsEl = null, _minimapEl = null, _demoBtn = null; + let _demoTimer = null; + + // =================================================================== + // Init / scene scaffold + // =================================================================== + function init() { + if (_initialized) { _resize(); return; } + if (typeof THREE === 'undefined') { + console.warn('[occupancy3d] THREE not loaded'); + return; + } + _container = document.getElementById('occ3d-container'); + _modeEl = document.getElementById('occ3d-mode'); + _readoutsEl = document.getElementById('occ3d-readouts'); + _minimapEl = document.getElementById('occ3d-minimap'); + _demoBtn = document.getElementById('occ3d-demo-btn'); + if (!_container) return; + + _buildScene(); + _wireInteraction(); + if (_demoBtn) _demoBtn.addEventListener('click', toggleDemo); + + // Subscribe to live data (mirror devices.js:1553-1559) + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('DEVICE_STATE_UPDATE', handleDeviceState); + ClientEventBus.on('SCAN_GEOMETRY_UPDATE', handleScanGeometry); + ClientEventBus.on('EMBRYOS_UPDATE', handleEmbryos); + } + _bootstrap(); + + _initialized = true; + _rebuildSceneObjects(); + _renderReadouts(); + _renderMinimap(); + _animate(); + // Container is 0×0 while the tab is hidden; size once it's visible. + requestAnimationFrame(_resize); + } + + function _buildScene() { + const w = _container.clientWidth || 600; + const h = _container.clientHeight || 460; + + _scene = new THREE.Scene(); + _camera = new THREE.PerspectiveCamera(45, w / h, 0.01, 100); + _camera.position.set(0, 0, _zoom); + + _renderer = new THREE.WebGLRenderer({ antialias: true }); + _renderer.setSize(w, h); + _renderer.setClearColor(0x0a0e12); + _container.innerHTML = ''; + _container.appendChild(_renderer.domElement); + + _root = new THREE.Group(); + _root.rotation.x = _rot.x; + _root.rotation.y = _rot.y; + _scene.add(_root); + + // Keep the canvas in sync with its container (chat dock / window resize). + if (_resizeObserver) _resizeObserver.disconnect(); + _resizeObserver = new ResizeObserver(() => { + if (_resizeRaf) cancelAnimationFrame(_resizeRaf); + _resizeRaf = requestAnimationFrame(_resize); + }); + _resizeObserver.observe(_container); + if (!_onLayoutChanged) { + _onLayoutChanged = () => _resize(); + window.addEventListener('gently:layout-changed', _onLayoutChanged); + } + } + + function _resize() { + if (!_renderer || !_container) return; + const w = _container.clientWidth || 600; + const h = _container.clientHeight || 460; + if (w === 0 || h === 0) return; + _camera.aspect = w / h; + _camera.updateProjectionMatrix(); + _renderer.setSize(w, h); + } + + function _wireInteraction() { + const el = _renderer.domElement; + el.addEventListener('mousedown', (e) => { + _isDragging = true; _prevMouse = { x: e.clientX, y: e.clientY }; + }); + el.addEventListener('mousemove', (e) => { + if (!_isDragging) return; + _root.rotation.y += (e.clientX - _prevMouse.x) * 0.01; + _root.rotation.x += (e.clientY - _prevMouse.y) * 0.01; + _rot.x = _root.rotation.x; _rot.y = _root.rotation.y; + _prevMouse = { x: e.clientX, y: e.clientY }; + }); + window.addEventListener('mouseup', () => { _isDragging = false; }); + el.addEventListener('wheel', (e) => { + e.preventDefault(); + _zoom = Math.max(0.4, Math.min(6, _zoom + e.deltaY * 0.002)); + _camera.position.z = _zoom; + }, { passive: false }); + el.addEventListener('dblclick', () => { + _rot.x = -0.6; _rot.y = 0.6; _zoom = 1.7; + _root.rotation.x = _rot.x; _root.rotation.y = _rot.y; + _camera.position.z = _zoom; + }); + } + + function _animate() { + _animationId = requestAnimationFrame(_animate); + if (_renderer && _scene && _camera) _renderer.render(_scene, _camera); + } + + // =================================================================== + // Scene geometry (rebuilt when scan geometry changes) + // =================================================================== + function _disposeObj(obj) { + if (!obj) return; + _root.remove(obj); + obj.traverse?.((c) => { + c.geometry?.dispose?.(); + if (c.material) (Array.isArray(c.material) ? c.material : [c.material]).forEach(m => m.dispose()); + }); + obj.geometry?.dispose?.(); + if (obj.material) (Array.isArray(obj.material) ? obj.material : [obj.material]).forEach(m => m.dispose()); + } + + function _currentGeom() { + // Fall back to nominal defaults so the scene is never empty. + const g = _geom || {}; + const scan = g.scan || {}; + const derived = g.derived || {}; + const piezoCenter = scan.piezo_center_um != null ? scan.piezo_center_um : 50.0; + const zExtent = derived.z_extent_um != null ? derived.z_extent_um : 50.0; + return { + numSlices: scan.num_slices != null ? scan.num_slices : 50, + piezoCenter, + zExtent, + mode: g.mode || 'sheet', + }; + } + + function _rebuildSceneObjects() { + if (!_root) return; + [_outerBox, _cuboid, _cuboidEdges, _sheet, _beam, _sliceGroup].forEach(_disposeObj); + _outerBox = _cuboid = _cuboidEdges = _sheet = _beam = _sliceGroup = null; + + const g = _currentGeom(); + const fov = FOV_UM; + // Outer Z neighbourhood centred on the cuboid so it's always framed. + const halfZ = Math.max(g.zExtent * 2.5, 75); + const zMin = g.piezoCenter - halfZ; + const zMax = g.piezoCenter + halfZ; + const halfXY = fov * 1.5; + + _scaler = makeSceneScaler({ + xRange: [-halfXY, halfXY], + yRange: [-halfXY, halfXY], + zRange: [zMin, zMax], + }); + const L = (um) => _scaler.scaleLen(um); + const Z = (um) => _scaler.toScene(um, 'z'); + + // --- Outer reference frame (addressable Z × local XY) ---------- + _outerBox = new THREE.LineSegments( + new THREE.EdgesGeometry(new THREE.BoxGeometry(L(2 * halfXY), L(zMax - zMin), L(2 * halfXY))), + new THREE.LineBasicMaterial({ color: COLORS.outer }) + ); + _outerBox.position.y = Z(g.piezoCenter); // box centred on its own midpoint == piezoCenter + _root.add(_outerBox); + + // --- Acquisition cuboid (footprint × z-extent) ----------------- + // Three.js Y is our axial (Z µm) axis; X/Z are the lateral footprint. + const cw = L(fov), cd = L(fov), ch = L(g.zExtent); + _cuboid = new THREE.Mesh( + new THREE.BoxGeometry(cw, ch, cd), + new THREE.MeshBasicMaterial({ + color: COLORS.cuboidFace, transparent: true, opacity: 0.06, + depthWrite: false, side: THREE.DoubleSide, + }) + ); + _cuboid.position.y = Z(g.piezoCenter); + _root.add(_cuboid); + _cuboidEdges = new THREE.LineSegments( + new THREE.EdgesGeometry(new THREE.BoxGeometry(cw, ch, cd)), + new THREE.LineBasicMaterial({ color: COLORS.cuboid }) + ); + _cuboidEdges.position.y = Z(g.piezoCenter); + _root.add(_cuboidEdges); + + // --- Slice planes (faint outlines through the cuboid) ---------- + _sliceGroup = new THREE.Group(); + const n = Math.max(1, Math.min(g.numSlices, MAX_SLICE_LINES)); + const sliceMat = new THREE.LineBasicMaterial({ color: COLORS.slice, transparent: true, opacity: 0.5 }); + for (let i = 0; i < n; i++) { + const frac = n === 1 ? 0.5 : i / (n - 1); + const zUm = (g.piezoCenter - g.zExtent / 2) + frac * g.zExtent; + const ring = new THREE.LineLoop(_rectXZ(cw, cd), sliceMat); + ring.position.y = Z(zUm); + _sliceGroup.add(ring); + } + _root.add(_sliceGroup); + + // --- Light sheet / pencil beam --------------------------------- + if (g.mode === 'pencil') { + // Pencil: a thin beam along the lateral axis through cuboid centre. + _beam = new THREE.LineSegments( + new THREE.BufferGeometry().setFromPoints([ + new THREE.Vector3(-cw / 2, 0, 0), new THREE.Vector3(cw / 2, 0, 0), + ]), + new THREE.LineBasicMaterial({ color: COLORS.beam }) + ); + _root.add(_beam); + } else { + _sheet = new THREE.Mesh( + new THREE.PlaneGeometry(cw, cd), + new THREE.MeshBasicMaterial({ + color: COLORS.sheet, transparent: true, opacity: 0.35, + side: THREE.DoubleSide, depthWrite: false, + }) + ); + _sheet.rotation.x = -Math.PI / 2; // lie in the lateral (X-Z) plane + _root.add(_sheet); + } + _updateSheetPosition(); + } + + // A rectangle outline in the lateral (X-Z) plane, centred at origin. + function _rectXZ(w, d) { + const hw = w / 2, hd = d / 2; + return new THREE.BufferGeometry().setFromPoints([ + new THREE.Vector3(-hw, 0, -hd), new THREE.Vector3(hw, 0, -hd), + new THREE.Vector3(hw, 0, hd), new THREE.Vector3(-hw, 0, hd), + ]); + } + + // Move the sheet/beam to the live axial position (piezo µm), clamped to + // the cuboid extent. Falls back to the cuboid centre when no live value. + function _updateSheetPosition() { + if (!_scaler) return; + const g = _currentGeom(); + const zMin = g.piezoCenter - g.zExtent / 2; + const zMax = g.piezoCenter + g.zExtent / 2; + let zUm = _piezoZ != null ? _piezoZ : g.piezoCenter; + zUm = Math.max(zMin, Math.min(zMax, zUm)); + const y = _scaler.toScene(zUm, 'z'); + if (_sheet) _sheet.position.y = y; + if (_beam) _beam.position.y = y; + } + + // =================================================================== + // Event handlers + // =================================================================== + function handleDeviceState(payload) { + if (!payload) return; + const pos = payload.positions || {}; + for (const name of Object.keys(pos)) { + const e = pos[name] || {}; + if (e.kind === 'xy_stage') { + if (e.X != null) _stage.x = e.X; + if (e.Y != null) _stage.y = e.Y; + } else if (e.kind === 'piezo') { + if (e.Position != null) _piezoZ = e.Position; + } else if (e.kind === 'galvo') { + if (e.A != null) _galvo.a = e.A; + if (e.B != null) _galvo.b = e.B; + } + } + const box = extractFirmwareBox(payload.properties); + if (box) _firmwareBox = box; + _updateSheetPosition(); + _renderReadouts(); + _renderMinimap(); + } + + function handleScanGeometry(payload) { + if (!payload) return; + _geom = payload; + if (payload.stage_position_um) { + if (payload.stage_position_um.x != null) _stage.x = payload.stage_position_um.x; + if (payload.stage_position_um.y != null) _stage.y = payload.stage_position_um.y; + } + _rebuildSceneObjects(); + _renderReadouts(); + _renderMinimap(); + } + + function handleEmbryos(payload) { + if (!payload || !Array.isArray(payload.embryos)) return; + _embryos = payload.embryos.map((e) => { + const fine = e.position_fine || {}; + const coarse = e.position_coarse || {}; + const x = fine.x != null ? fine.x : coarse.x; + const y = fine.y != null ? fine.y : coarse.y; + return { x, y, role: e.role, id: e.id }; + }).filter((e) => e.x != null && e.y != null); + _renderMinimap(); + } + + async function _bootstrap() { + try { + const r = await fetch('/api/devices/scan_geometry'); + if (r.ok) handleScanGeometry(await r.json()); + } catch (_) { /* offline — demo button covers it */ } + try { + const r = await fetch('/api/embryos/current'); + if (r.ok) handleEmbryos(await r.json()); + } catch (_) { /* ignore */ } + } + + // =================================================================== + // HTML overlay: mode badge, readouts, minimap + // =================================================================== + function _fmt(v, digits = 1, unit = '') { + return v == null ? '—' : (Number(v).toFixed(digits) + unit); + } + + function _renderReadouts() { + const g = _currentGeom(); + if (_modeEl) { + _modeEl.textContent = g.mode === 'pencil' ? 'PENCIL' : 'SHEET'; + _modeEl.classList.toggle('is-pencil', g.mode === 'pencil'); + } + if (!_readoutsEl) return; + const scan = (_geom && _geom.scan) || {}; + const derived = (_geom && _geom.derived) || {}; + const rows = [ + ['stage X', _fmt(_stage.x, 0, ' µm')], + ['stage Y', _fmt(_stage.y, 0, ' µm')], + ['piezo Z', _fmt(_piezoZ, 1, ' µm')], + ['galvo A/B', `${_fmt(_galvo.a, 3)} / ${_fmt(_galvo.b, 3)}°`], + ['slices', scan.num_slices != null ? String(scan.num_slices) : '—'], + ['Z extent', _fmt(derived.z_extent_um, 1, ' µm')], + ['slice step', _fmt(derived.slice_spacing_um, 3, ' µm')], + ]; + _readoutsEl.innerHTML = rows + .map(([k, v]) => `
    ${k}${escapeHtml(v)}
    `) + .join(''); + } + + function _renderMinimap() { + if (!_minimapEl) return; + const VB = { w: 200, h: 120, pad: 8 }; + const box = _firmwareBox || { x: [-25000, 25000], y: [-12000, 12000] }; + const bw = box.x[1] - box.x[0], bh = box.y[1] - box.y[0]; + if (!(bw > 0 && bh > 0)) return; + const sx = (VB.w - 2 * VB.pad) / bw; + const sy = (VB.h - 2 * VB.pad) / bh; + const s = Math.min(sx, sy); + const ox = VB.pad + (VB.w - 2 * VB.pad - bw * s) / 2; + const oy = VB.pad + (VB.h - 2 * VB.pad - bh * s) / 2; + const px = (x) => ox + (x - box.x[0]) * s; + const py = (y) => oy + (box.y[1] - y) * s; // flip Y for screen + + const parts = []; + parts.push(``); + for (const e of _embryos) { + parts.push(``); + } + if (_stage.x != null && _stage.y != null) { + const fovPx = FOV_UM * s; + parts.push(``); + parts.push(``); + } + _minimapEl.innerHTML = parts.join(''); + } + + // =================================================================== + // Demo driver — develop without live hardware (launch_gently.py --offline) + // =================================================================== + function toggleDemo() { + if (_demoTimer) { + clearInterval(_demoTimer); _demoTimer = null; + if (_demoBtn) _demoBtn.classList.remove('is-on'); + return; + } + if (_demoBtn) _demoBtn.classList.add('is-on'); + // Seed a firmware box, a scan geometry, and a few embryos. + _firmwareBox = { x: [-25000, 25000], y: [-12000, 12000] }; + handleScanGeometry({ + embryo_id: 'demo_2', + stage_position_um: { x: 4200, y: -1800 }, + scan: { + num_slices: 60, exposure_ms: 5.0, + galvo_amplitude_deg: 0.5, galvo_center_deg: 0.0, + piezo_amplitude_um: 25.0, piezo_center_um: 50.0, + }, + derived: { z_extent_um: 50.0, slice_spacing_um: 50 / 59, z_min_um: 25, z_max_um: 75 }, + mode: 'sheet', ts: 0, + }); + handleEmbryos({ + embryos: [ + { id: 'demo_1', role: 'test', position_coarse: { x: 4200, y: -1800 } }, + { id: 'demo_2', role: 'control', position_coarse: { x: -8000, y: 5200 } }, + { id: 'demo_3', role: 'test', position_coarse: { x: 12000, y: 2400 } }, + ], + }); + // Sweep the sheet in Z to animate the plane. + let t = 0; + _demoTimer = setInterval(() => { + t += 0.08; + const g = _currentGeom(); + _piezoZ = g.piezoCenter + (g.zExtent / 2) * Math.sin(t); + _galvo.a = 0.5 * Math.sin(t); + _updateSheetPosition(); + _renderReadouts(); + }, 60); + } + + function cleanup() { + if (_animationId) cancelAnimationFrame(_animationId); + if (_demoTimer) { clearInterval(_demoTimer); _demoTimer = null; } + if (_resizeObserver) _resizeObserver.disconnect(); + if (_renderer) { _renderer.dispose(); } + } + + return { init, cleanup, toggleDemo, handleDeviceState, handleScanGeometry, handleEmbryos }; +})(); + +document.addEventListener('DOMContentLoaded', () => { + // Build lazily on first tab activation (container is 0×0 while hidden), + // so init() is invoked from app.js switchTab(), not here. +}); diff --git a/gently/ui/web/static/js/operate-math.js b/gently/ui/web/static/js/operate-math.js new file mode 100644 index 00000000..250cf857 --- /dev/null +++ b/gently/ui/web/static/js/operate-math.js @@ -0,0 +1,155 @@ +/** + * Operate — pure geometry, banding and interlock logic. + * + * These four functions decide where an embryo is, how far the F-drive may move, + * where the gauge marker sits, and whether XY motion is safe. They take explicit + * arguments and touch no DOM, so they can be unit-tested (tests/js/operate-math.test.mjs) + * — which the rest of the surface, being wiring over already-covered endpoints, + * does not need. + * + * Loaded as a plain script in the browser (window.OperateMath) and required by + * the Node test runner; there is no root package.json, so `.js` here is CJS. + */ +const OperateMath = (function () { + // pixel_size_um / objective_mag, before frame downsampling. + const BASE_UM_PER_PX = 6.5 / 10.0; + + function umPerPx(frame, base) { + const b = (base == null) ? BASE_UM_PER_PX : base; + return b * ((frame && frame.downsample) || 1); + } + + /** + * Frame pixel → absolute stage µm. + * + * `captureStage` is the absolute XY the frame was taken at and must be real — + * callers block marking rather than defaulting it to [0,0], because that + * silently converts clicks into offsets from stage origin and lands embryos + * hundreds of µm away. Returns null rather than guessing. + * + * Stage +Y is up, image +Y is down, hence the sign flip on Y only. + */ + function frameToStage(fx, fy, frame, captureStage, base) { + if (!frame || !Number.isFinite(frame.w) || !Number.isFinite(frame.h)) return null; + if (!Array.isArray(captureStage) || captureStage.length !== 2) return null; + if (!Number.isFinite(captureStage[0]) || !Number.isFinite(captureStage[1])) return null; + const u = umPerPx(frame, base); + return [ + captureStage[0] + (fx - frame.w / 2) * u, + captureStage[1] - (fy - frame.h / 2) * u, + ]; + } + + // The F-drive travels from ~25000 µm down onto a sample sitting around 50-60. + // Operators close that in bands: a big jump, then thousands, hundreds, tens. + // Offering ±1 at 25000 µm is 2500 clicks; offering ±1000 at 200 µm is a crash. + // So the steps on offer follow the current height. + const FD_BANDS = [ + { above: 10000, steps: [5000, 1000], label: 'coarse approach' }, + { above: 2000, steps: [1000, 500], label: 'approach' }, + { above: 1000, steps: [500, 100], label: 'near sample' }, + { above: 200, steps: [100, 50], label: 'close' }, + { above: -Infinity, steps: [50, 10, 5], label: 'fine — at sample' }, + ]; + + // With no position yet the finest band is the safe answer. Note `>` is strict: + // exactly 200 falls through to 'fine', not 'close'. + function fdBand(pos) { + if (pos == null || !Number.isFinite(pos)) return FD_BANDS[FD_BANDS.length - 1]; + return FD_BANDS.find(b => pos > b.above) || FD_BANDS[FD_BANDS.length - 1]; + } + + /** + * Absolute stage µm → frame pixel. The inverse of frameToStage. + * + * Markers are stored in stage coordinates, not pixel coordinates, so they + * stay attached to the sample rather than to the viewport: the frame can + * keep streaming and the stage can move, and each marker re-projects onto + * whatever frame is current. That is what removes the old freeze-the-frame + * marking mode. Returns null on the same conditions frameToStage does. + */ + function stageToFrame(sx, sy, frame, captureStage, base) { + if (!frame || !Number.isFinite(frame.w) || !Number.isFinite(frame.h)) return null; + if (!Array.isArray(captureStage) || captureStage.length !== 2) return null; + if (!Number.isFinite(captureStage[0]) || !Number.isFinite(captureStage[1])) return null; + const u = umPerPx(frame, base); + if (!(u > 0)) return null; + return [ + frame.w / 2 + (sx - captureStage[0]) / u, + frame.h / 2 - (sy - captureStage[1]) / u, + ]; + } + + /** + * May this nudge be offered, given the remaining travel to the floor? + * + * Banding (fdBand) picks step sizes proportionate to height; this is the + * separate safety gate. Up-steps are always offered — the server fences the + * ceiling. A down-step may not exceed the travel that is left. + * + * With no telemetry the answer is "allow": the UI must not pretend to know a + * distance it has not been told, and the real backstop is the server-side + * fence (F_DRIVE_MIN_UM in hardware/dispim/devices/piezo.py), which cannot be + * removed from here. + */ + function stepAllowed(delta, distanceToFloor) { + if (!Number.isFinite(delta)) return false; + if (delta >= 0) return true; + if (distanceToFloor == null || !Number.isFinite(distanceToFloor)) return true; + return Math.abs(delta) <= distanceToFloor; + } + + /** + * Where the marker sits on a travel track, 0 (min) to 1 (max). Null means + * "don't draw a marker" — a marker parked at the bottom reads as "at the + * limit", which is a lie when the truth is that nothing is known yet. + * + * The F-drive uses scale 'log'. Linearly, the last 200 µm of a 30-25000 µm + * axis — the entire approach-and-crash region — is 0.7% of the track, under + * one pixel. Log makes the approach legible where it matters. + */ + function gaugeFraction(pos, min, max, scale) { + if (pos == null || min == null || max == null) return null; + if (!Number.isFinite(pos) || !Number.isFinite(min) || !Number.isFinite(max)) return null; + const span = max - min; + if (!(span > 0)) return null; + const clamp = v => Math.min(1, Math.max(0, v)); + if (scale === 'log') { + const denom = Math.log10(span + 1); + if (!(denom > 0)) return null; + return clamp(Math.log10(Math.max(0, pos - min) + 1) / denom); + } + return clamp((pos - min) / span); + } + + // RIG-NOTE: 1000 µm is the band where operators switch to hundred-µm steps. + // Confirm against the real geometry before trusting it on the rig. + const ENGAGED_WITHIN_UM = 1000; + + /** + * Is the sample close enough to the objective that XY motion is unsafe? + * + * `latch` is a BELIEF (persisted: we commanded the head down). `floor` is a + * MEASUREMENT (distance_to_floor from telemetry). Measurement wins when it is + * decisive in either direction; the latch only decides inside the hysteresis + * band, and is the sole signal when there is no measurement at all. + * + * Fail safe: with no telemetry, a set latch locks. The `> within * 2` clear is + * what lets someone raising the head at the controller box be noticed; the 2x + * band stops it chattering at the boundary. + */ + function isEngaged(latch, floor, within) { + const w = (within == null) ? ENGAGED_WITHIN_UM : within; + if (floor == null || !Number.isFinite(floor)) return !!latch; + if (floor < w) return true; + if (floor > w * 2) return false; + return !!latch; + } + + return { + BASE_UM_PER_PX, ENGAGED_WITHIN_UM, FD_BANDS, + umPerPx, frameToStage, stageToFrame, fdBand, stepAllowed, gaugeFraction, isEngaged, + }; +})(); + +if (typeof module !== 'undefined' && module.exports) module.exports = OperateMath; diff --git a/gently/ui/web/static/js/operate.js b/gently/ui/web/static/js/operate.js new file mode 100644 index 00000000..6587ff69 --- /dev/null +++ b/gently/ui/web/static/js/operate.js @@ -0,0 +1,1318 @@ +/** + * Operate — three independent instrument surfaces. + * + * Bottom cam see the dish, focus the bottom objective, find embryos + * SPIM head bring the objectives to height over the sample + * Acquisition the embryo roster and what to run on it + * + * There are no steps, no phases and no progress ladder. Every pane is fully + * live whenever it is visible. + * + * THE INVARIANT: no control anywhere reads `_pane`. It is consulted by exactly + * two things — render dispatch, and which camera stream owns MMCore. If a + * `disabled`, `hidden` or early-return ever starts keying off `_pane`, the step + * model has grown back and this rewrite has been undone. + * + * Gating comes from live hardware state only: the XY interlock (moveStageTo / + * OperateMath.isEngaged) and the F-drive floor (OperateMath.stepAllowed). + * Selection is a cursor — it parameterises requests and never disables anything. + * + * Pure geometry, banding and the interlock predicate live in operate-math.js so + * they can be unit-tested (tests/js/operate-math.test.mjs). + */ +const OperateManager = (function () { + const M = (typeof OperateMath !== 'undefined') ? OperateMath : null; + const MARK_HIT_PX = 14; + + let _wired = false, _active = false; + + // ── navigation: the ONLY navigation state in this file ────────────────── + let _pane = 'bottom'; + + // ── session facts (owned by the bus, mirrored here) ───────────────────── + let _embryos = []; + // A CURSOR, not a step. It parameterises request bodies and readouts. It + // must never appear in a `disabled` or visibility expression. + let _selected = null; + + // ── device state ──────────────────────────────────────────────────────── + let _xy = null; // {x, y} from DEVICE_STATE_UPDATE + // Last bottom-cam frame. Kept because the marking canvas needs its geometry + // and capture position; the SPIM frame is only ever displayed, so it isn't. + let _lastBottom = null; + + // ── stream ownership, per pane ────────────────────────────────────────── + let _bottomOn = false, _bottomWasOn = false; + let _spimOn = false, _spimWasOn = false; + + // ── the interlock latch ───────────────────────────────────────────────── + // Retract is a RELATIVE move, so there is no absolute "safe height" to + // derive this from — asking the hardware cannot answer it. Persist it, so + // the ordinary reaction to something looking stuck (F5) doesn't come back + // claiming the head is up and re-enable an absolute XY move with the + // objective down. Telemetry can still clear it (see OperateMath.isEngaged). + const HEAD_KEY = 'gently.operate.headLowered'; + function loadHeadLowered() { + try { return sessionStorage.getItem(HEAD_KEY) === '1'; } catch (_) { return false; } + } + let _headLowered = loadHeadLowered(); + function setHeadLowered(v) { + _headLowered = !!v; + try { sessionStorage.setItem(HEAD_KEY, _headLowered ? '1' : '0'); } catch (_) {} + renderLock(); + } + + // ── marking ───────────────────────────────────────────────────────────── + // Markers are held in STAGE coordinates and re-projected onto whatever frame + // is current, so they stay attached to the sample instead of the viewport. + // That is what lets marking be always-on rather than a mode you enter. + let _markers = []; + + // ── emitters / run ────────────────────────────────────────────────────── + let _ledOn = false, _acquiring = false; + let _galvo = 0.0, _piezo = 50.0; + let _mode = 'single'; + let _selectedLib = null; + let _runPaused = false; + + // ── primitives ────────────────────────────────────────────────────────── + function $(id) { return document.getElementById(id); } + function toast(m) { if (typeof showGentlyToast === 'function') showGentlyToast(m); } + function escapeHtml(s) { + return String(s == null ? '' : s).replace(/[&<>"]/g, c => + ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c])); + } + + async function postJSON(url, body) { + const res = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body || {}), + }); + const text = await res.text().catch(() => ''); + let data = {}; + try { data = text ? JSON.parse(text) : {}; } catch (_) { /* not JSON */ } + if (!res.ok) { + const e = new Error(`${res.status} ${data.error || text}`); + e.status = res.status; + // A fenced nudge answers 400 but still reports where the axis + // actually is. Carry the body so the caller can re-absorb it + // instead of leaving the gauge stale. + e.data = data; + throw e; + } + return data; + } + async function getJSON(url) { + const res = await fetch(url); + const text = await res.text().catch(() => ''); + let data = {}; + try { data = text ? JSON.parse(text) : {}; } catch (_) { /* not JSON */ } + if (!res.ok) { const e = new Error(String(res.status)); e.status = res.status; e.data = data; throw e; } + return data; + } + + function labelFor(emb) { + const m = emb && emb.id && String(emb.id).match(/(\d+)/); + return m ? m[1] : '?'; + } + function resolveXY(emb) { + const f = emb && emb.position_fine; + if (f && Number.isFinite(f.x) && Number.isFinite(f.y)) return { x: f.x, y: f.y }; + const c = emb && emb.position_coarse; + if (c && Number.isFinite(c.x) && Number.isFinite(c.y)) return { x: c.x, y: c.y }; + return null; + } + + // ══ THE XY CHOKEPOINT ═══════════════════════════════════════════════════ + // The ONLY caller of /api/devices/stage/move in this file. The server does + // NOT interlock XY against the F-drive — routes/data.py validates that x and + // y are floats and nothing else — so this predicate is the whole guard. + // Do not add a second fetch of this endpoint; grep the URL string before + // touching XY motion. + async function moveStageTo(x, y, why) { + if (isEngaged()) { + toast('Sample is at the objective — back it off before moving XY'); + return false; + } + try { + await postJSON('/api/devices/stage/move', { x, y }); + if (why) toast(why); + return true; + } catch (e) { + toast(`Move failed (${e.status || e.message})`); + return false; + } + } + function isEngaged() { + return M ? M.isEngaged(_headLowered, fd.floor()) : _headLowered; + } + + // ══ Z INSTRUMENT ════════════════════════════════════════════════════════ + // One factory, two instances. The axes differ enormously in scale — the + // bottom objective has a ~200 µm throw, the F-drive runs 30→25000 µm onto a + // sample sitting at ~50 — so the F-drive gets position-banded steps and a + // log track, and the bottom axis a fixed ladder on a linear one. + function makeZAxis(cfg) { + const st = { pos: null, min: null, max: null, floor: null, status: 'unknown' }; + let busy = false; + + const num = v => (v == null || !Number.isFinite(Number(v))) ? null : Number(v); + + function absorb(d) { + if (!d) return; + if (num(d.position) != null) st.pos = num(d.position); + if (num(d.min) != null) st.min = num(d.min); + if (num(d.max) != null) st.max = num(d.max); + if (num(d.distance_to_floor) != null) st.floor = num(d.distance_to_floor); + else if (st.pos != null && st.min != null) st.floor = st.pos - st.min; + if (st.pos != null) st.status = 'ok'; + render(); + } + // 1 Hz position stream. Without this the gauge goes stale the moment + // someone drives the axis at the controller box instead of from here. + function absorbTelemetry(val) { + if (!Number.isFinite(val)) return; + st.pos = val; + if (st.min != null) st.floor = val - st.min; + if (st.status !== 'absent') st.status = 'ok'; + render(); + } + function fail(e) { + const msg = String((e && e.data && e.data.error) || (e && e.message) || ''); + // An axis this rig does not have is a FACT, not an error: the device + // layer 503s with "device not found". Render it as absent, quietly. + st.status = (e && e.status === 503 && /not found/i.test(msg)) ? 'absent' : 'error'; + render(); + } + + function steps() { + if (!cfg.bands) return cfg.steps; + return (M ? M.fdBand(st.pos) : { steps: cfg.steps }).steps; + } + function bandLabel() { + if (!cfg.bands) return ''; + if (st.pos == null) return 'position unknown'; + return (M ? M.fdBand(st.pos).label : ''); + } + + function renderNudges() { + const host = $(cfg.root + '-nudge'); + if (!host) return; + if (st.status === 'absent') { host.innerHTML = ''; host.dataset.band = ''; return; } + const s = steps(); + const key = s.join(','); + if (host.dataset.band !== key) { + host.dataset.band = key; + host.style.gridTemplateColumns = `repeat(${s.length}, minmax(0, 1fr))`; + // Ups on the top row, downs on the bottom, columns aligned by + // magnitude, so the control maps to the motion. + host.innerHTML = + s.map(v => ``).join('') + + s.map(v => ``).join(''); + } + host.querySelectorAll('[data-nudge]').forEach(b => { + const d = Number(b.dataset.nudge); + b.disabled = busy || (M ? !M.stepAllowed(d, st.floor) : false); + }); + } + + function renderTicks() { + const host = $(cfg.root + '-ticks'); + if (!host || !cfg.ticks) return; + const key = `${st.min},${st.max}`; + if (host.dataset.k === key) return; + host.dataset.k = key; + if (st.status !== 'ok' || st.min == null || st.max == null) { host.innerHTML = ''; return; } + host.innerHTML = cfg.ticks + .filter(t => t > st.min && t < st.max) + .map(t => { + const f = M ? M.gaugeFraction(t, st.min, st.max, cfg.scale) : null; + return f == null ? '' : `${t}`; + }).join(''); + } + + function render() { + const g = $(cfg.gauge); + if (g) { + g.dataset.status = st.status; + g.classList.toggle('is-near-floor', + st.status === 'ok' && st.floor != null && st.floor < 100); + } + const read = $(cfg.root + '-pos'); + if (read) { + read.textContent = st.status === 'absent' ? 'n/a' + : (st.pos == null ? '—' : st.pos.toFixed(1)); + } + const mark = $(cfg.root + '-mark'); + // Null fraction means "do not draw a marker" — a marker parked at + // the bottom of the track reads as "at the limit", which is a lie + // when the truth is that nothing is known yet. + const frac = (st.status === 'ok' && M) + ? M.gaugeFraction(st.pos, st.min, st.max, cfg.scale) : null; + if (mark) { + if (frac == null) mark.style.display = 'none'; + else { mark.style.display = 'block'; mark.style.bottom = `${(frac * 100).toFixed(2)}%`; } + } + const lo = $(cfg.root + '-min'), hi = $(cfg.root + '-max'); + if (lo) lo.textContent = st.min == null ? '—' : Math.round(st.min); + if (hi) hi.textContent = st.max == null ? '—' : Math.round(st.max); + + const track = $(cfg.root + '-track'); + if (track) { + track.setAttribute('aria-valuetext', + st.status === 'absent' ? 'axis not present' + : st.pos == null ? 'unknown' : `${st.pos.toFixed(1)} micrometres`); + } + // An absent or unreachable axis says so wherever this gauge has room + // for a line of text — the banded axis uses its band caption, the + // plain one its foot. + const statusText = st.status === 'absent' ? 'axis not present on this rig' + : st.status === 'error' ? 'position unavailable' : null; + const band = $(cfg.root + '-band'); + if (band) band.textContent = statusText || bandLabel(); + const floor = $(cfg.root + '-floor'); + if (floor) floor.textContent = st.floor == null ? '—' : Math.round(st.floor); + const foot = $(cfg.root + '-foot'); + if (foot) foot.textContent = statusText || ''; + renderTicks(); + renderNudges(); + renderLock(); + } + + async function nudge(delta) { + if (M && !M.stepAllowed(delta, st.floor)) { + toast('Too close to the floor for that step'); + return; + } + busy = true; renderNudges(); + try { + absorb(await postJSON(cfg.nudge, { delta })); + if (cfg.onDown && delta < 0) cfg.onDown(); + if (cfg.onUp && delta > 0) cfg.onUp(st); + } catch (e) { + // Even a refused nudge reports the real position — take it. + if (e && e.data && e.data.position != null) absorb(e.data); + toast(`${cfg.label} nudge blocked (${e.status || e.message})`); + } finally { busy = false; renderNudges(); } + } + + async function refresh() { + try { absorb(await getJSON(cfg.get)); } catch (e) { fail(e); } + } + + function wire() { + const host = $(cfg.root + '-nudge'); + if (host) { + host.addEventListener('click', e => { + const b = e.target.closest('[data-nudge]'); + if (b && !b.disabled) nudge(Number(b.dataset.nudge)); + }); + } + const track = $(cfg.root + '-track'); + if (track) { + track.addEventListener('keydown', e => { + const s = steps(); + const fine = s[s.length - 1], coarse = s[0]; + const map = { ArrowUp: fine, ArrowDown: -fine, PageUp: coarse, PageDown: -coarse }; + if (map[e.key] == null) return; + e.preventDefault(); + nudge(map[e.key]); + }); + } + } + + return { + wire, refresh, absorb, absorbTelemetry, render, nudge, + floor: () => st.floor, + status: () => st.status, + }; + } + + const bz = makeZAxis({ + root: 'op-bz', gauge: 'op-gauge-bz', label: 'Bottom-Z', + get: '/api/devices/stage/bottom_z', + nudge: '/api/devices/stage/bottom_z/nudge', + steps: [10, 1], scale: 'linear', bands: false, + }); + const fd = makeZAxis({ + root: 'op-fd', gauge: 'op-gauge-fd', label: 'F-drive', + get: '/api/devices/spim/fdrive', + nudge: '/api/devices/spim/fdrive/nudge', + bands: true, scale: 'log', ticks: [100, 1000, 10000], + steps: [50, 10, 5], + onDown: () => setHeadLowered(true), + }); + + // ══ THE INTERLOCK, MADE VISIBLE ═════════════════════════════════════════ + // Enforcement alone is not enough: with click-to-center always available, + // the operator must be able to see WHY a click will not do anything. The + // affordance withdraws itself (locked cursor) and the banner says so. + function renderLock() { + const eng = isEngaged(); + const d = fd.floor(); + ['bottom', 'spim'].forEach(p => { + const el = $(`op-lock-${p}`); + if (el) el.hidden = !eng; + const dd = $(`op-lock-${p}-d`); + if (dd) dd.textContent = d == null ? '—' : Math.round(d); + }); + const cam = $('op-cam-bottom'); + if (cam) cam.classList.toggle('is-locked', eng); + drawMarkers(); + } + // Always reachable, unlike the old design where clearing the latch lived on + // a step you might never arrive at — lower the head, never reach it, and XY + // stayed locked forever with no escape short of clearing sessionStorage. + async function backOff() { + try { + fd.absorb(await postJSON('/api/devices/spim/fdrive/nudge', { delta: 100 })); + // A retract that FAILS must not report the head as up: that is the + // state the XY chokepoint trusts before commanding an absolute move. + setHeadLowered(false); + toast('Backed off 100 µm'); + } catch (e) { + if (e && e.data && e.data.position != null) fd.absorb(e.data); + toast(`Back-off failed (${e.status || e.message}) — head still down`); + } + } + + // ══ VIEWPORT ════════════════════════════════════════════════════════════ + function frameOf(p) { + if (!p || !Array.isArray(p.shape)) return null; + return { w: p.shape[1], h: p.shape[0], downsample: p.downsample || 1 }; + } + function stageOf(p) { + if (p && Array.isArray(p.stage_position) && p.stage_position.length === 2) return p.stage_position; + // The device layer OMITS stage_position rather than defaulting it to + // [0,0] when it cannot know it. Honour that: fall back to the position + // stream, never to the origin. + if (_xy && Number.isFinite(_xy.x) && Number.isFinite(_xy.y)) return [_xy.x, _xy.y]; + return null; + } + function setImg(imgId, phId, p) { + const img = $(imgId), ph = $(phId); + if (!img || !p || !p.jpeg_b64) return; + img.src = `data:${p.mime || 'image/jpeg'};base64,${p.jpeg_b64}`; + if (!img.classList.contains('has-frame')) { + img.classList.add('has-frame'); + if (ph) ph.style.display = 'none'; + } + // Match the viewport box to the frame's aspect so the border hugs the + // image. naturalWidth is 0 until the data URL decodes, so fall back to a + // one-shot load listener. + if (img.naturalWidth && img.naturalHeight) setCamAspect(img); + else img.addEventListener('load', () => setCamAspect(img), { once: true }); + } + function setCamAspect(img) { + const fit = img.closest('.op-cam-fit'); + if (fit && img.naturalWidth && img.naturalHeight) { + fit.style.setProperty('--cam-ar', `${img.naturalWidth} / ${img.naturalHeight}`); + } + } + function clearImg(imgId, phId, text) { + const img = $(imgId), ph = $(phId); + if (img) img.classList.remove('has-frame'); + if (ph) { ph.style.display = ''; if (text) ph.textContent = text; } + } + // Stopping the stream freezes the last frame in place rather than clearing + // it: an operator wants to keep reading what is on the sample surface after + // ending live view. Only fall back to the placeholder when no frame was + // ever shown. The "LIVE" badge dropping (renderSubnavMeta) is the cue that + // the frame is now static. + function freezeImg(imgId, phId, text) { + const img = $(imgId); + if (img && img.classList.contains('has-frame')) return; + clearImg(imgId, phId, text); + } + + // Letterbox geometry for an object-fit: contain image, in CSS pixels. + // Measured off the CANVAS, not its host: it is the element being drawn into, + // and using one source for geometry and for the backing store keeps them + // from disagreeing. + function renderedRect() { + const c = $('op-mark-canvas'); + if (!c) return null; + const sb = c.getBoundingClientRect(); + if (!(sb.width > 0 && sb.height > 0)) return null; + const f = frameOf(_lastBottom); + const fw = f ? f.w : sb.width, fh = f ? f.h : sb.height; + const ar = fw / fh, sar = sb.width / sb.height; + let w, h; + if (ar > sar) { w = sb.width; h = sb.width / ar; } + else { h = sb.height; w = sb.height * ar; } + return { x: (sb.width - w) / 2, y: (sb.height - h) / 2, w, h, fw, fh, sb }; + } + function canvasCtx() { + const c = $('op-mark-canvas'); + if (!c) return null; + const r = c.getBoundingClientRect(); + if (!(r.width > 0 && r.height > 0)) return null; + // The backing store must track the CSS box or everything drawn is + // scaled — a stale height renders circles as ellipses. Scaling by dpr + // keeps it crisp on fractional-ratio displays; the transform then lets + // every drawing call stay in CSS pixels. + const dpr = window.devicePixelRatio || 1; + const w = Math.round(r.width * dpr), h = Math.round(r.height * dpr); + if (c.width !== w || c.height !== h) { c.width = w; c.height = h; } + const ctx = c.getContext('2d'); + ctx.setTransform(dpr, 0, 0, dpr, 0, 0); + ctx.clearRect(0, 0, r.width, r.height); + return ctx; + } + // Project a stage-space marker onto the current frame, then onto the canvas. + function markerToCanvas(m, r) { + const f = frameOf(_lastBottom), cap = stageOf(_lastBottom); + if (!f || !cap || !M) return null; + const px = M.stageToFrame(m.stageX, m.stageY, f, cap); + if (!px) return null; + return { cx: r.x + (px[0] / r.fw) * r.w, cy: r.y + (px[1] / r.fh) * r.h, px }; + } + // Registered embryos, projected onto the current frame. These are the + // click-to-center targets; pending markers are a separate, editable set. + function embryoPoints(r) { + const f = frameOf(_lastBottom), cap = stageOf(_lastBottom); + if (!f || !cap || !M) return []; + const out = []; + _embryos.forEach(emb => { + const xy = resolveXY(emb); + if (!xy) return; + const px = M.stageToFrame(xy.x, xy.y, f, cap); + if (!px) return; + out.push({ emb, cx: r.x + (px[0] / r.fw) * r.w, cy: r.y + (px[1] / r.fh) * r.h }); + }); + return out; + } + + function drawMarkers() { + // Render dispatch, not gating: the canvas has no size while its pane is + // hidden, so there is nothing to draw onto. + if (_pane !== 'bottom') return; + const ctx = canvasCtx(); + if (!ctx) return; + const r = renderedRect(); + if (!r) return; + + // Registered embryos first, so pending markers draw over them. + embryoPoints(r).forEach(({ emb, cx, cy }) => { + const sel = emb.id === _selected; + ctx.save(); + ctx.strokeStyle = isEngaged() ? '#7d8899' : (sel ? '#93c5fd' : '#60a5fa'); + ctx.fillStyle = ctx.strokeStyle; + ctx.lineWidth = sel ? 2 : 1.2; + if (isEngaged()) ctx.setLineDash([4, 3]); + ctx.beginPath(); ctx.arc(cx, cy, 13, 0, Math.PI * 2); ctx.stroke(); + ctx.setLineDash([]); + ctx.beginPath(); ctx.arc(cx, cy, 2.5, 0, Math.PI * 2); ctx.fill(); + ctx.font = '600 11px Inter Tight, sans-serif'; + ctx.fillText(labelFor(emb), cx + 16, cy + 4); + ctx.restore(); + }); + const locked = isEngaged(); + const colour = locked ? '#7d8899' : '#4ade80'; + _markers.forEach((m, i) => { + const p = markerToCanvas(m, r); + if (!p) return; + const { cx, cy } = p; + ctx.save(); + ctx.strokeStyle = colour; + ctx.lineWidth = locked ? 1.2 : 2; + if (locked) ctx.setLineDash([4, 3]); + ctx.beginPath(); ctx.arc(cx, cy, 11, 0, Math.PI * 2); ctx.stroke(); + ctx.setLineDash([]); + ctx.beginPath(); + ctx.moveTo(cx - 6, cy); ctx.lineTo(cx + 6, cy); + ctx.moveTo(cx, cy - 6); ctx.lineTo(cx, cy + 6); + ctx.stroke(); + ctx.fillStyle = colour; + ctx.font = '600 11px Inter Tight, sans-serif'; + ctx.fillText(String(i + 1), cx + 13, cy - 8); + ctx.restore(); + }); + } + + function onCanvasClick(e) { + const r = renderedRect(); + const c = $('op-mark-canvas'); + if (!r || !c) return; + const rect = c.getBoundingClientRect(); + const cxv = e.clientX - rect.left, cyv = e.clientY - rect.top; + + // Click a pending marker to remove it. + for (let i = 0; i < _markers.length; i++) { + const p = markerToCanvas(_markers[i], r); + if (p && Math.hypot(cxv - p.cx, cyv - p.cy) <= MARK_HIT_PX) { + // Editing the set invalidates the "N candidates added" note — it + // was the detection-time count, not the current one. + _markers.splice(i, 1); drawMarkers(); renderMarkCount(); setDetectNote(''); return; + } + } + // Click a registered embryo to select it and centre the stage on it. + // Always available — the interlock lives in moveStageTo, not here. + for (const p of embryoPoints(r)) { + if (Math.hypot(cxv - p.cx, cyv - p.cy) <= MARK_HIT_PX) { + centerOnEmbryo(p.emb); + return; + } + } + if (cxv < r.x || cxv > r.x + r.w || cyv < r.y || cyv > r.y + r.h) return; + + const f = frameOf(_lastBottom), cap = stageOf(_lastBottom); + if (!f) { toast('Start the camera first'); return; } + // NEVER default the capture position to [0,0]: that silently converts + // clicks into offsets from stage origin, so embryos land hundreds of µm + // away and calibration images empty field. Refuse to mark instead. + if (!cap) { toast('Stage position unknown — wait for the readout, then mark'); return; } + + const fx = ((cxv - r.x) / r.w) * r.fw, fy = ((cyv - r.y) / r.h) * r.fh; + const s = M && M.frameToStage(fx, fy, f, cap); + if (!s) { toast('Cannot place a marker without a stage position'); return; } + _markers.push({ stageX: s[0], stageY: s[1], source: 'manual' }); + drawMarkers(); renderMarkCount(); + } + + async function centerOnEmbryo(emb) { + const xy = resolveXY(emb); + if (!xy) { toast('That embryo has no recorded position'); return; } + selectEmbryo(emb.id); + await moveStageTo(xy.x, xy.y, `Centred on embryo ${labelFor(emb)}`); + } + + function renderMarkCount() { + const n = _markers.length; + const c = $('op-mark-count'); if (c) c.textContent = n; + const ok = $('op-confirm'); if (ok) ok.disabled = n === 0; + const cl = $('op-clear'); if (cl) cl.disabled = n === 0; + } + function setDetectNote(text) { + const note = $('op-detect-note'); if (note) note.textContent = text || ''; + } + + // ══ BOTTOM PANE ═════════════════════════════════════════════════════════ + async function toggleBottomCam() { + const b = $('op-cam-toggle'); if (b) b.disabled = true; + try { + const ep = _bottomOn ? '/api/devices/bottom_camera/stream/stop' + : '/api/devices/bottom_camera/stream/start'; + const d = await postJSON(ep, {}); + applyBottomCam(!!d.streaming); + _bottomWasOn = _bottomOn; + } catch (e) { toast(`Camera toggle failed (${e.status || e.message})`); } + finally { if (b) b.disabled = false; } + } + function applyBottomCam(on) { + _bottomOn = on; + const b = $('op-cam-toggle'); + if (b) { b.textContent = on ? 'Stop camera' : 'Start camera'; b.classList.toggle('is-on', on); } + if (!on) freezeImg('op-img-bottom', 'op-ph-bottom', 'Camera off'); + renderSubnavMeta(); + } + + function setBusyText(t) { + const el = document.querySelector('#op-busy-bottom .op-cam-busy-txt'); + if (el) el.textContent = t; + } + async function runDetect() { + const b = $('op-detect'); + if (b) { b.disabled = true; b.textContent = 'Detecting…'; } + const busy = $('op-busy-bottom'); if (busy) busy.hidden = false; + const note = $('op-detect-note'); + // Detect on the frame already on screen when there is one — the operator + // is looking at it, and re-capturing would disturb the LED/room light. + const shown = $('op-img-bottom'); + let hasFrame = !!(shown && shown.classList.contains('has-frame')); + try { + // Phase 1 — when the viewport is empty, capture and SHOW the image + // FIRST (no SAM yet), so the operator sees what detection will run on + // before it runs, rather than the image appearing only at the end. + if (!hasFrame) { + setBusyText('Capturing…'); + const cap = await postJSON('/api/devices/detect_embryos', { capture_only: true }); + if (cap.frame && cap.frame.jpeg_b64) { + _lastBottom = cap.frame; + setImg('op-img-bottom', 'op-ph-bottom', cap.frame); + hasFrame = true; + } + } + // Phase 2 — run SAM on the frame now on screen, then overlay results. + setBusyText('Detecting…'); + const d = await postJSON('/api/devices/detect_embryos', { use_last_frame: hasFrame }); + if (d.frame && d.frame.jpeg_b64) { + _lastBottom = d.frame; + setImg('op-img-bottom', 'op-ph-bottom', d.frame); + } + const cands = Array.isArray(d.embryos) ? d.embryos : []; + const f = frameOf(_lastBottom); + const cap = d.stage_position || stageOf(_lastBottom); + // A fresh detection REPLACES the previous auto-detected set rather + // than piling onto it (re-running would otherwise double the marks). + // Manual marks are kept — only 'sam' ones are cleared. + _markers = _markers.filter(m => m.source !== 'sam'); + let added = 0; + cands.forEach(c => { + let sx = c.stage_x_um, sy = c.stage_y_um; + if ((sx == null || sy == null) && f && cap && M && c.pixel_x != null && c.pixel_y != null) { + const s = M.frameToStage(c.pixel_x / f.downsample, c.pixel_y / f.downsample, f, cap); + if (s) { sx = s[0]; sy = s[1]; } + } + if (sx == null || sy == null) return; + _markers.push({ stageX: sx, stageY: sy, source: 'sam' }); + added++; + }); + drawMarkers(); renderMarkCount(); + if (note) note.textContent = `${added} candidate${added === 1 ? '' : 's'} added — edit them on the image, then register.`; + toast(`Detected ${added} candidate${added === 1 ? '' : 's'}`); + } catch (e) { + if (e.status === 503) { + if (note) note.textContent = 'Automatic detection is unavailable on this rig — mark by clicking the image.'; + } else { + toast(`Detect failed (${e.status || e.message})`); + } + } finally { + if (b) { b.disabled = false; b.textContent = 'Detect automatically'; } + if (busy) busy.hidden = true; + } + } + + async function confirmMarks() { + if (!_markers.length) return; + const f = frameOf(_lastBottom), cap = stageOf(_lastBottom); + if (!cap) { toast('Stage position unknown — cannot register markers'); return; } + const b = $('op-confirm'); if (b) b.disabled = true; + try { + // The payload shape is a hard contract with _persist_detection_labels + // (routes/data.py), which turns every confirm into training data for + // the localiser that will replace SAM. Pixel coords are projected + // from stage space against the frame being submitted. + const markers = _markers.map(m => { + const px = (f && M) ? M.stageToFrame(m.stageX, m.stageY, f, cap) : null; + return { + stage_x_um: m.stageX, stage_y_um: m.stageY, + pixel_x: px ? px[0] : undefined, pixel_y: px ? px[1] : undefined, + source: m.source, + }; + }); + const d = await postJSON('/api/devices/embryos/confirm', { + markers, + image_b64: _lastBottom ? _lastBottom.jpeg_b64 : undefined, + frame: f ? { w: f.w, h: f.h, downsample: f.downsample } : undefined, + stage_position: cap, + }); + const n = (d.registered || []).length; + _markers = []; + drawMarkers(); renderMarkCount(); + setDetectNote(`Registered ${n} embryo${n === 1 ? '' : 's'} — they're in the roster and on the SPIM head.`); + toast(`Registered ${n} embryo${n === 1 ? '' : 's'}`); + } catch (e) { + toast(`Register failed (${e.status || e.message})`); + if (b) b.disabled = false; + } + } + + // ══ SPIM PANE ═══════════════════════════════════════════════════════════ + async function toggleSpim() { + const b = $('op-spim-toggle'); if (b) b.disabled = true; + try { + const ep = _spimOn ? '/api/devices/lightsheet/live/stop' : '/api/devices/lightsheet/live/start'; + const d = await postJSON(ep, {}); + applySpim(!!d.streaming); + _spimWasOn = _spimOn; + } catch (e) { toast(`SPIM view toggle failed (${e.status || e.message})`); } + finally { if (b) b.disabled = false; } + } + function applySpim(on) { + _spimOn = on; + const b = $('op-spim-toggle'); + if (b) { b.textContent = on ? 'Stop view' : 'Start view'; b.classList.toggle('is-on', on); } + if (!on) freezeImg('op-img-spim', 'op-ph-spim', 'View off'); + renderSubnavMeta(); + } + + let _lsTimer = null; + function postLsParams() { + if (_lsTimer) clearTimeout(_lsTimer); + _lsTimer = setTimeout(() => { + postJSON('/api/devices/lightsheet/live/params', + { galvo: _galvo, piezo: _piezo, exposure: 20, side: 'A' }).catch(() => {}); + }, 120); + } + function nudgeGalvo(d) { + _galvo = Math.max(-5, Math.min(5, _galvo + d)); + const el = $('op-gv'); if (el) el.textContent = _galvo.toFixed(1); + postLsParams(); + } + function nudgePiezo(d) { + _piezo = Math.max(0, Math.min(200, _piezo + d)); + const el = $('op-pz'); if (el) el.textContent = _piezo.toFixed(0); + postLsParams(); + } + async function toggleLed() { + _ledOn = !_ledOn; + applyLed(); + try { await postJSON('/api/devices/led/set', { state: _ledOn ? 'Open' : 'Closed' }); } + catch (e) { toast(`LED failed (${e.status || e.message})`); } + } + function applyLed() { + const b = $('op-led'); + if (b) { + b.setAttribute('aria-pressed', _ledOn ? 'true' : 'false'); + b.classList.toggle('is-emitting', _ledOn); + } + renderSubnavMeta(); + } + async function forceLedOff() { + if (!_ledOn) return; + _ledOn = false; applyLed(); + try { await postJSON('/api/devices/led/set', { state: 'Closed' }); } catch (_) {} + } + + async function calibrateSelected() { + if (!_selected) { toast('Select an embryo first'); return; } + const b = $('op-calibrate'), out = $('op-cal-result'); + if (b) { b.disabled = true; b.textContent = 'Calibrating…'; } + if (out) out.textContent = 'sweeping…'; + try { + const d = await postJSON(`/api/devices/embryos/${_selected}/calibrate`, {}); + const cal = d.calibration || {}; + const slope = cal.slope_um_per_deg, r2 = cal.r_squared; + if (out) { + out.textContent = (slope != null) + ? `${Number(slope).toFixed(1)} µm/deg${r2 != null ? ` · R² ${Number(r2).toFixed(2)}` : ''}` + : 'done'; + } + } catch (e) { + if (out) out.textContent = 'failed'; + toast(`Calibrate failed (${e.status || e.message})`); + } finally { if (b) { b.disabled = false; b.textContent = 'Calibrate piezo–galvo'; } } + } + + function renderSpimTarget() { + const el = $('op-spim-target'); + if (!el) return; + const emb = _embryos.find(e => e.id === _selected); + el.textContent = emb ? `Selected: embryo ${labelFor(emb)}` : 'No embryo selected'; + } + + // ══ ACQUISITION PANE ════════════════════════════════════════════════════ + function selectEmbryo(id) { + _selected = id; + renderRoster(); renderSpimTarget(); renderSingle(); renderEmbryoRail(); + } + + // Shared embryo list, left of every instrument surface. Reads the canonical + // _embryos (bootstrapped from /api/embryos/current, kept live by + // EMBRYOS_UPDATE), so it is the same set on Bottom / SPIM / Acquire and it + // survives a refresh. + function renderEmbryoRail() { + const host = $('op-erail-list'); + const count = $('op-erail-count'); + if (count) count.textContent = _embryos.length; + if (!host) return; + host.innerHTML = ''; + if (!_embryos.length) { + const box = document.createElement('div'); + box.className = 'op-erail-empty'; + box.textContent = 'No embryos yet — detect on the bottom camera, then register.'; + host.appendChild(box); + return; + } + _embryos.forEach(emb => { + const xy = resolveXY(emb); + const row = document.createElement('div'); + row.className = 'op-erow' + (emb.id === _selected ? ' is-sel' : ''); + row.tabIndex = 0; + row.dataset.embryo = emb.id; + row.innerHTML = + '' + + `Embryo ${escapeHtml(labelFor(emb))}` + + `${xy ? `${xy.x.toFixed(0)}, ${xy.y.toFixed(0)}` : '—'}` + + '' + + ``; + host.appendChild(row); + }); + } + + async function deleteEmbryo(id) { + try { + const res = await fetch(`/api/embryos/${encodeURIComponent(id)}`, { method: 'DELETE' }); + if (!res.ok) throw Object.assign(new Error('delete failed'), { status: res.status }); + // EMBRYOS_UPDATE will reconcile every view; prune optimistically so + // the row disappears immediately even before the event lands. + _embryos = _embryos.filter(e => e.id !== id); + if (_selected === id) _selected = _embryos.length ? _embryos[0].id : null; + renderEmbryoRail(); renderRoster(); renderSpimTarget(); renderSingle(); drawMarkers(); + } catch (e) { + toast(`Delete failed (${e.status || e.message})`); + } + } + + function renderRoster() { + const host = $('op-roster'); + const count = $('op-roster-count'); + if (count) count.textContent = _embryos.length; + if (!host) return; + host.innerHTML = ''; + if (!_embryos.length) { + const box = document.createElement('div'); + box.className = 'op-empty'; + box.innerHTML = 'No embryos marked yet.' + + ''; + host.appendChild(box); + return; + } + _embryos.forEach(emb => { + const xy = resolveXY(emb); + const role = (emb.role && emb.role !== 'unassigned') ? emb.role : 'test'; + const row = document.createElement('div'); + row.className = 'op-rrow' + (emb.id === _selected ? ' is-sel' : ''); + row.tabIndex = 0; + row.dataset.embryo = emb.id; + row.innerHTML = + `Embryo ${escapeHtml(labelFor(emb))}` + + `${xy ? `${xy.x.toFixed(0)}, ${xy.y.toFixed(0)}` : '—'}` + + `` + + ``; + host.appendChild(row); + }); + } + + // Roles are read from the canonical embryo list and written through the + // endpoint — deliberately NOT mirrored in a local map, which in the old + // design drifted from _embryos and needed a reconciliation loop. + async function toggleRole(id) { + const emb = _embryos.find(e => e.id === id); + if (!emb) return; + const cur = (emb.role && emb.role !== 'unassigned') ? emb.role : 'test'; + const next = cur === 'calibration' ? 'test' : 'calibration'; + emb.role = next; + renderRoster(); + const roles = {}; + _embryos.forEach(e => { roles[e.id] = (e.role && e.role !== 'unassigned') ? e.role : 'test'; }); + try { await postJSON('/api/embryos/roles', { roles }); } + catch (e) { toast(`Roles failed (${e.status || e.message})`); } + } + + function setMode(m) { + _mode = m; + document.querySelectorAll('#op-modes [data-mode]').forEach(b => + b.classList.toggle('is-on', b.dataset.mode === m)); + ['single', 'adaptive', 'library', 'agent'].forEach(k => { + const p = $(`op-panel-${k}`); + if (p) p.hidden = k !== m; + }); + if (m === 'library') loadLibrary(); + renderSingle(); + } + + function renderSingle() { + const t = $('op-single-target'), d = $('op-single-delta'); + const emb = _embryos.find(e => e.id === _selected); + if (t) t.textContent = emb ? `embryo ${labelFor(emb)}` : 'none selected'; + if (!d) return; + const xy = emb ? resolveXY(emb) : null; + // A fact, not a gate: the operator is told how far off the stage is and + // decides for themselves. Acquire is never disabled on this. + if (!xy || !_xy) { d.textContent = '—'; return; } + const dx = xy.x - _xy.x, dy = xy.y - _xy.y; + d.textContent = `${Math.round(Math.hypot(dx, dy))} µm away`; + } + + async function loadLibrary() { + const host = $('op-lib-list'); + if (!host) return; + try { + const d = await getJSON('/api/tactic_library'); + const items = (d && d.tactics) || []; + if (!items.length) { host.innerHTML = '
    No saved tactics
    '; return; } + host.innerHTML = items.map(t => + ``).join(''); + } catch (_) { host.innerHTML = '
    Library unavailable
    '; } + } + + function subjectIds() { + const subs = _embryos.filter(e => e.role !== 'calibration').map(e => e.id); + return subs.length ? subs : _embryos.map(e => e.id); + } + + async function startRun() { + const b = $('op-run-start'); + const done = () => { if (b) { b.disabled = false; b.textContent = 'Start'; } }; + if (b) { b.disabled = true; b.textContent = 'Starting…'; } + try { + if (_mode === 'single') { + if (!_selected) { toast('Select an embryo first'); return; } + _acquiring = true; renderSubnavMeta(); + try { + await postJSON('/api/devices/acquire/volume', { + num_slices: Math.max(1, Number(($('op-vol-slices') || {}).value) || 50), + exposure_ms: Math.max(1, Number(($('op-vol-exp') || {}).value) || 10), + }); + toast('Volume acquired'); + } finally { _acquiring = false; await forceLedOff(); renderSubnavMeta(); } + return; + } + if (_mode === 'adaptive') { + const interval = Math.max(1, Number(($('op-tl-interval') || {}).value) || 120); + const sel = ($('op-tl-stop') || {}).value || 'manual'; + const val = Math.max(1, Number(($('op-tl-condval') || {}).value) || 1); + // The orchestrator parses the COMBINED form ('timepoints:N' / + // 'duration:Xh'). A bare 'timepoints' silently degrades to manual, + // i.e. a timelapse that never stops. + let stop_condition = sel; + if (sel === 'timepoints') stop_condition = `timepoints:${val}`; + else if (sel === 'duration') stop_condition = `duration:${val}h`; + await postJSON('/api/devices/timelapse/start', { + embryo_ids: subjectIds(), + interval_seconds: interval, + stop_condition, + monitoring_mode: ($('op-tl-monitor') || {}).value || 'idle', + }); + toast('Adaptive timelapse started'); + renderRun(); + return; + } + if (_mode === 'library') { + if (!_selectedLib) { toast('Pick a saved tactic'); return; } + const d = await postJSON('/api/operate/run-tactic', + { library_id: _selectedLib, embryo_ids: subjectIds() }); + if (d.success) { toast('Tactic started'); renderRun(); } + else toast(`Run failed: ${(d.result && d.result.message) || '?'}`); + return; + } + if (_mode === 'agent') { + const roster = _embryos.map(e => { + const xy = resolveXY(e); + const r = e.role === 'calibration' ? 'reference' : 'subject'; + return `${labelFor(e)}${xy ? ` (${xy.x.toFixed(0)},${xy.y.toFixed(0)})` : ''} [${r}]`; + }).join(', '); + const prompt = `I marked ${_embryos.length} embryos: ${roster}. ` + + 'Propose and start an Operation Plan to image them.'; + if (typeof AgentChat !== 'undefined' && AgentChat.togglePanel) { + AgentChat.togglePanel(true); + if (AgentChat.runCommand) setTimeout(() => AgentChat.runCommand(prompt), 300); + } else toast('Agent chat unavailable'); + } + } catch (e) { + toast(`Start failed (${e.status || e.message})`); + } finally { done(); } + } + + // Run presence is DERIVED from the server, not from a client flag. The old + // design kept it in memory, so F5 during a running timelapse lost the whole + // panel — and left a client state machine that could re-grow into steps. + async function renderRun() { + const host = $('op-runspine'), actions = $('op-run-actions'); + if (!host) return; + let tactics = []; + try { + const d = await getJSON('/api/operation_plan'); + tactics = (d && d.plan && d.plan.tactics) || []; + } catch (_) { /* leave empty */ } + const live = tactics.filter(t => t.state === 'active' || t.state === 'paused'); + if (actions) actions.hidden = live.length === 0; + if (!tactics.length) { + host.innerHTML = '
    Nothing running.
    '; + return; + } + host.innerHTML = tactics.map(tacticCard).join(''); + _runPaused = live.some(t => t.state === 'paused'); + const p = $('op-run-pause'); + if (p) p.textContent = _runPaused ? 'Resume' : 'Pause'; + } + function tacticCard(t) { + const state = t.state || 'planned'; + const struct = t.structure || {}; + const meta = []; + if (struct.cadence_s != null) meta.push(`${struct.cadence_s}s`); + if (struct.interval != null) meta.push(`${struct.interval}s`); + if (struct.status) meta.push(struct.status); + if (t.live && t.live.signal != null) meta.push(`signal ${t.live.signal}`); + return `
    ` + + `
    ${escapeHtml(t.name || t.id)}` + + `${escapeHtml(state)}
    ` + + `
    ${escapeHtml(t.kind || '')}
    ` + + (meta.length ? `
    ${escapeHtml(meta.join(' · '))}
    ` : '') + + (t.rationale ? `
    ${escapeHtml(t.rationale)}
    ` : '') + + '
    '; + } + async function pauseRun() { + try { + await postJSON(_runPaused ? '/api/devices/timelapse/resume' : '/api/devices/timelapse/pause', {}); + toast(_runPaused ? 'Resumed' : 'Paused'); + } catch (e) { toast(`Pause/resume failed (${e.status || e.message})`); } + renderRun(); + } + async function stopRun() { + if (!window.confirm('Stop the run?')) return; + try { await postJSON('/api/devices/timelapse/stop', { reason: 'operator' }); toast('Run stopped'); } + catch (e) { toast(`Stop failed (${e.status || e.message})`); } + renderRun(); + } + + // ══ PANES ═══════════════════════════════════════════════════════════════ + // Camera ownership is keyed on VISIBILITY, not on a step. Both cameras + // contend for MMCore, and the client swaps .src per frame with no throttle, + // so two live decoders is the condition that risks a Video-TDR freeze. "The + // camera is live while you are looking at it" guarantees at most one. + const PANES = { + bottom: { + onEnter() { if (_bottomWasOn && !_bottomOn) toggleBottomCam(); drawMarkers(); }, + onLeave() { _bottomWasOn = _bottomOn; if (_bottomOn) stopBottom(); }, + render() { renderMarkCount(); drawMarkers(); bz.render(); }, + }, + spim: { + onEnter() { if (_spimWasOn && !_spimOn) toggleSpim(); }, + onLeave() { _spimWasOn = _spimOn; if (_spimOn) stopSpim(); forceLedOff(); }, + render() { renderSpimTarget(); fd.render(); }, + }, + acquire: { + onEnter() { renderRun(); }, + onLeave() {}, + render() { renderRoster(); renderSingle(); }, + }, + }; + function stopBottom() { + fetch('/api/devices/bottom_camera/stream/stop', { method: 'POST' }).catch(() => {}); + applyBottomCam(false); + } + function stopSpim() { + fetch('/api/devices/lightsheet/live/stop', { method: 'POST' }).catch(() => {}); + applySpim(false); + } + + function showPane(name) { + if (!PANES[name] || name === _pane) return; + const prev = _pane; + _pane = name; + if (PANES[prev]) PANES[prev].onLeave(); + ['bottom', 'spim', 'acquire'].forEach(p => { + const el = $(`op-pane-${p}`); + if (el) el.hidden = p !== name; + }); + // Drives the CSS that hides the shared rail on Acquisition (its own + // roster is richer) while keeping it on Bottom / SPIM. + const body = $('op-body'); if (body) body.dataset.pane = name; + if (typeof updateViewButtons === 'function') updateViewButtons('operate-subtab-switcher', name); + PANES[name].onEnter(); + PANES[name].render(); + renderEmbryoRail(); + renderLock(); + } + + function renderSubnavMeta() { + const el = $('op-subnav-meta'); + if (!el) return; + const bits = []; + if (_bottomOn) bits.push('BOTTOM ● LIVE'); + if (_spimOn) bits.push('SPIM ● LIVE'); + if (_ledOn) bits.push('LED EMITTING'); + if (_acquiring) bits.push('LASER EMITTING'); + el.textContent = bits.join(' · '); + } + + // ══ EVENTS ══════════════════════════════════════════════════════════════ + function onBottomFrame(p) { + // Bail when hidden, or a hidden Operate keeps base64-decoding every + // frame behind whatever is on screen and races for stream ownership. + if (!_active || _pane !== 'bottom' || !p || !p.jpeg_b64) return; + _lastBottom = p; + if (p.focus_score != null) { + const el = $('op-bz-score'); + if (el) el.textContent = Number(p.focus_score).toFixed(3); + } + setImg('op-img-bottom', 'op-ph-bottom', p); + drawMarkers(); + } + function onSpimFrame(p) { + if (!_active || _pane !== 'spim' || !p || !p.jpeg_b64) return; + if (p.focus_score != null) { + const el = $('op-spim-score'); + if (el) el.textContent = Number(p.focus_score).toFixed(3); + } + setImg('op-img-spim', 'op-ph-spim', p); + } + function onEmbryosUpdate(p) { + _embryos = (p && Array.isArray(p.embryos)) ? p.embryos : []; + if (_selected && !_embryos.some(e => e.id === _selected)) _selected = null; + // The embryo list is shared across all three panes; keep a live + // selection whenever it is non-empty so SPIM/Acquire aren't a dead-end + // ("No embryo selected") right after registering. The operator can still + // switch by clicking a registered embryo (bottom) or a roster row. + if (!_selected && _embryos.length) _selected = _embryos[0].id; + // Render the shared rail even when the Operate view isn't the active tab, + // so switching to it (or refreshing) shows the list immediately rather + // than waiting for the next mutation event. + renderEmbryoRail(); + if (!_active) return; + renderRoster(); renderSpimTarget(); renderSingle(); + } + + function wire() { + if (_wired) return; + _wired = true; + + if (typeof initViewSwitcher === 'function') { + // Click delegation only — deliberately NO `views` option. + // + // initViewSwitcher's `views` binds BARE number keys on document, and + // 1-6 are already global main-tab navigation (KeyboardShortcuts in + // app.js), with system/calibration/plan switchers claiming 1-3 too. + // Binding them here either steals a main-tab key or, with a guard + // tight enough to be safe, never fires at all — verified in-browser: + // app.js handles the keypress first and moves state.tab, so the + // guard then correctly refuses. Three peer surfaces are fine to + // click between; a shortcut would need a modifier to be honest. + initViewSwitcher('operate-subtab-switcher', showPane); + } + + bz.wire(); fd.wire(); + + const cam = $('op-cam-toggle'); if (cam) cam.addEventListener('click', toggleBottomCam); + const det = $('op-detect'); if (det) det.addEventListener('click', runDetect); + const ok = $('op-confirm'); if (ok) ok.addEventListener('click', confirmMarks); + const cl = $('op-clear'); + if (cl) cl.addEventListener('click', () => { _markers = []; drawMarkers(); renderMarkCount(); setDetectNote(''); }); + const canvas = $('op-mark-canvas'); if (canvas) canvas.addEventListener('click', onCanvasClick); + + // Shared embryo rail: click a row to select (delete button is guarded + // first so removing a false positive doesn't also select it). + const erail = $('op-erail-list'); + if (erail) { + erail.addEventListener('click', (e) => { + const del = e.target.closest('[data-del]'); + if (del) { e.stopPropagation(); deleteEmbryo(del.dataset.del); return; } + const row = e.target.closest('[data-embryo]'); + if (row) selectEmbryo(row.dataset.embryo); + }); + erail.addEventListener('keydown', (e) => { + if (e.key !== 'Enter' && e.key !== ' ') return; + const row = e.target.closest('[data-embryo]'); + if (row) { e.preventDefault(); selectEmbryo(row.dataset.embryo); } + }); + } + + const sp = $('op-spim-toggle'); if (sp) sp.addEventListener('click', toggleSpim); + const led = $('op-led'); if (led) led.addEventListener('click', toggleLed); + const cal = $('op-calibrate'); if (cal) cal.addEventListener('click', calibrateSelected); + document.querySelectorAll('[data-gv]').forEach(b => + b.addEventListener('click', () => nudgeGalvo(Number(b.dataset.gv)))); + document.querySelectorAll('[data-pz]').forEach(b => + b.addEventListener('click', () => nudgePiezo(Number(b.dataset.pz)))); + document.querySelectorAll('[data-backoff]').forEach(b => + b.addEventListener('click', backOff)); + + const modes = $('op-modes'); + if (modes) { + modes.addEventListener('click', e => { + const b = e.target.closest('[data-mode]'); + if (b) setMode(b.dataset.mode); + }); + } + const stop = $('op-tl-stop'); + if (stop) { + stop.addEventListener('change', () => { + const w = $('op-tl-condwrap'); + if (w) w.hidden = stop.value === 'manual'; + }); + } + const lib = $('op-lib-list'); + if (lib) { + lib.addEventListener('click', e => { + const b = e.target.closest('[data-lib]'); + if (b) { _selectedLib = b.dataset.lib; loadLibrary(); } + }); + } + const roster = $('op-roster'); + if (roster) { + roster.addEventListener('click', e => { + const r = e.target.closest('[data-role-for]'); + if (r) { e.stopPropagation(); toggleRole(r.dataset.roleFor); return; } + const c = e.target.closest('[data-center]'); + if (c) { + e.stopPropagation(); + const emb = _embryos.find(x => x.id === c.dataset.center); + if (emb) centerOnEmbryo(emb); + return; + } + const g = e.target.closest('[data-goto]'); + if (g) { showPane(g.dataset.goto); return; } + const row = e.target.closest('[data-embryo]'); + if (row) selectEmbryo(row.dataset.embryo); + }); + roster.addEventListener('keydown', e => { + if (e.key !== 'Enter' && e.key !== ' ') return; + const row = e.target.closest('[data-embryo]'); + if (row) { e.preventDefault(); selectEmbryo(row.dataset.embryo); } + }); + } + const start = $('op-run-start'); if (start) start.addEventListener('click', startRun); + const pause = $('op-run-pause'); if (pause) pause.addEventListener('click', pauseRun); + const stopb = $('op-run-stop'); if (stopb) stopb.addEventListener('click', stopRun); + + window.addEventListener('resize', () => { if (_active && _pane === 'bottom') drawMarkers(); }); + // The viewport also changes size without a window resize — revealing the + // pane, the agent panel opening, the first frame arriving. Without this + // the overlay keeps whatever size it was first drawn at and every marker + // sits in the wrong place until the next frame happens to redraw it. + const camBox = $('op-cam-bottom'); + if (camBox && typeof ResizeObserver !== 'undefined') { + new ResizeObserver(() => { if (_active && _pane === 'bottom') drawMarkers(); }).observe(camBox); + } + + if (typeof ClientEventBus !== 'undefined') { + ClientEventBus.on('BOTTOM_CAMERA_FRAME', onBottomFrame); + ClientEventBus.on('LIGHTSHEET_FRAME', onSpimFrame); + ClientEventBus.on('EMBRYOS_UPDATE', onEmbryosUpdate); + ClientEventBus.on('DEVICE_STATE_UPDATE', p => { + const pos = p && p.positions; + if (!pos) return; + if (Array.isArray(pos.xy_stage)) _xy = { x: pos.xy_stage[0], y: pos.xy_stage[1] }; + if (!_active) return; + for (const v of Object.values(pos)) { + if (!v || typeof v !== 'object' || v.Position == null) continue; + const val = Number(v.Position); + if (!Number.isFinite(val)) continue; + if (v.kind === 'fdrive') fd.absorbTelemetry(val); + else if (v.kind === 'bottom_z') bz.absorbTelemetry(val); + } + if (_pane === 'acquire') renderSingle(); + }); + } + } + + async function activate() { + wire(); + if (_active) return; + _active = true; + showPaneInitial(); + try { onEmbryosUpdate(await getJSON('/api/embryos/current')); } catch (_) {} + await Promise.all([bz.refresh(), fd.refresh()]); + renderLock(); + renderSubnavMeta(); + } + function showPaneInitial() { + ['bottom', 'spim', 'acquire'].forEach(p => { + const el = $(`op-pane-${p}`); + if (el) el.hidden = p !== _pane; + }); + if (typeof updateViewButtons === 'function') updateViewButtons('operate-subtab-switcher', _pane); + if (PANES[_pane]) { PANES[_pane].onEnter(); PANES[_pane].render(); } + } + function deactivate() { + if (!_active) return; + _active = false; + // Remember what was running so returning restores it, but leave nothing + // decoding behind a hidden view. + _bottomWasOn = _bottomOn; _spimWasOn = _spimOn; + if (_bottomOn) stopBottom(); + if (_spimOn) stopSpim(); + forceLedOff(); + } + + return { activate, deactivate }; +})(); diff --git a/gently/ui/web/static/js/operations-scenarios.js b/gently/ui/web/static/js/operations-scenarios.js new file mode 100644 index 00000000..0cd72fbf --- /dev/null +++ b/gently/ui/web/static/js/operations-scenarios.js @@ -0,0 +1,372 @@ +/** + * Operation Plan scenario fixtures — development and Chrome-MCP audit targets. + * + * Each entry is a plan object matching the real API schema returned by + * GET /api/operation_plan/{session_id} → { available, plan } + * The `.plan` is what gets passed to the renderer. Active tactics carry a + * `live` field (readouts + phases) that the API route merges from live + * telemetry; here they are baked into the fixture. + * + * Scenario dev mode: load via ?scenario= — ExperimentOverview reads + * window.OPERATIONS_SCENARIOS[name] and skips all fetches. + * + * Scenarios: + * temp_strain — scripted_protocol active (temp-change burst protocol) + * expression_onset — reactive_monitor active (reporter rising) + * hatching_detect — reactive_monitor active (watch=hatching, status=watching) + * transmission_survey — exclusive_burst active (brightfield only) + * decided_plan — all planned, nothing run yet + * async_multi — standing_timelapse per-embryo cadence + layered reactive_monitor + * idle — null (no operation running) + */ +window.OPERATIONS_SCENARIOS = { + + /* ------------------------------------------------------------------ */ + temp_strain: { + session_id: '20260628_1432_tempstrain_a', + title: 'Temperature-strain run · E01', + goal: 'Acquire volumes before, during, and after a +4 °C step to 32 °C; capture reporter response to heat stress.', + tactics: [ + { + id: 'ts-1', seq: 1, + name: 'Monitor — low cadence', + kind: 'standing_timelapse', state: 'done', + scope: { mode: 'global' }, + rationale: 'Baseline acquisition before thermal perturbation.', + structure: { cadence_s: 180 }, + live_bind: ['cadence'], + live: { summary: '22 min · ended on signal' }, + relations: {} + }, + { + id: 'ts-2', seq: 2, + name: 'Transmission burst — baseline', + kind: 'exclusive_burst', state: 'done', + scope: { mode: 'global' }, + rationale: 'Brightfield snapshot before setpoint change.', + structure: { frames: 3, mode: 'brightfield' }, + live_bind: [], + live: { summary: '3 bursts · brightfield' }, + relations: {} + }, + { + id: 'ts-3', seq: 3, + name: 'Temp-change burst protocol', + kind: 'scripted_protocol', state: 'active', + scope: { mode: 'global' }, + rationale: 'Systematic volume capture before, during ramp, and after thermal lock. Laser off during ramp to limit phototoxicity.', + structure: { + phases: [ + { name: 'before', state: 'done', count: '1/1 done' }, + { name: 'during', state: 'active', count: '2 · awaiting lock' }, + { name: 'after', state: 'todo', count: '0/1' } + ] + }, + live_bind: ['temperature', 'current_burst'], + live: { + target: '→ 32.0 °C', + summary: 'started 3m ago', + desc: 'bursts before · setpoint change · bursts through ramp · bursts after lock — laser off', + readouts: [ + { + label: 'stage temp', + bind: 'temperature', + value: '29.432.0°C', + bar: 62 + }, + { + label: 'current burst', + value: '#3 during', + sub: '60f · 1Hz · brightfield' + } + ], + phases: [ + { name: 'before', state: 'done', count: '1/1 done', pips: ['before'] }, + { name: 'during', state: 'active', count: '2 · awaiting lock', pips: ['during', 'during', 'pending'] }, + { name: 'after', state: 'todo', count: '0/1', pips: ['pending'] } + ] + }, + relations: {} + }, + { + id: 'ts-4', seq: 4, + name: 'Recovery monitor — low cadence', + kind: 'standing_timelapse', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Resume gentle monitoring once temperature settles.', + structure: { cadence_s: 180 }, + live_bind: ['cadence'], + live: { + summary: 'queued · 30 min after lock', + desc: 'resume gentle monitoring once temperature settles' + }, + relations: { after: ['ts-3'] } + } + ] + }, + + /* ------------------------------------------------------------------ */ + expression_onset: { + session_id: '20260628_0915_onset_b', + title: 'Reporter-onset watch · E04', + goal: 'Detect first appearance of the fluorescent reporter; capture onset dynamics at high temporal resolution.', + tactics: [ + { + id: 'eo-1', seq: 1, + name: 'Monitor — low cadence', + kind: 'standing_timelapse', state: 'done', + scope: { mode: 'global' }, + rationale: 'Baseline acquisition before signal appears.', + structure: { cadence_s: 180 }, + live_bind: ['cadence'], + live: { summary: '1h 40m · baseline' }, + relations: {} + }, + { + id: 'eo-2', seq: 2, + name: 'Expression monitoring', + kind: 'reactive_monitor', state: 'active', + scope: { mode: 'global' }, + rationale: 'Accelerate cadence on signal, ramp 488 down on saturation, burst on stable structure.', + structure: { watch: 'reporter onset', reaction: 'accelerate + ramp laser', status: 'watching' }, + live_bind: ['signal', 'cadence'], + live: { + target: 'reporter onset', + summary: 'signal rising', + desc: 'accelerate cadence on signal · ramp 488 down on saturation · burst on stable structure', + readouts: [ + { + label: 'reporter signal', + value: 'rising', + sub: '+14% over 6 min', + bar: 48 + }, + { + label: 'cadence', + value: '120s 30s', + sub: 'accelerated on onset' + }, + { + label: '488 power', + value: '5% 3%', + sub: 'ramped to limit saturation' + } + ] + }, + relations: {} + }, + { + id: 'eo-3', seq: 3, + name: 'Burst on good structure', + kind: 'exclusive_burst', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Capture a burst once the reporter pattern holds.', + structure: { frames: 60, mode: 'fluorescence' }, + live_bind: [], + live: { + summary: 'queued · when structure stable', + desc: 'capture a burst once the reporter pattern holds' + }, + relations: { after: ['eo-2'] } + } + ] + }, + + /* ------------------------------------------------------------------ */ + hatching_detect: { + session_id: '20260627_2210_hatch_c', + title: 'Pre-hatching vigil · E11', + goal: 'Detect and capture the hatching event; accelerate acquisition as hatching approaches.', + tactics: [ + { + id: 'hd-1', seq: 1, + name: 'Pre-terminal monitoring', + kind: 'reactive_monitor', state: 'active', + scope: { mode: 'global' }, + rationale: 'Low cadence now; speed up as hatching approaches.', + structure: { watch: 'hatching', reaction: 'accelerate near event', status: 'watching' }, + live_bind: ['cadence'], + live: { + target: 'hatching', + summary: 'watching', + desc: 'low cadence now · speed up as hatching approaches', + readouts: [ + { label: 'est. time to hatch', value: '~38 min', sub: 'from motion + morphology' }, + { label: 'cadence', value: '180s', sub: 'will speed up near event' } + ] + }, + relations: {} + }, + { + id: 'hd-2', seq: 2, + name: 'Hatching speedup', + kind: 'standing_timelapse', state: 'planned', + scope: { mode: 'global' }, + rationale: 'High cadence through hatching.', + structure: { cadence_s: 30 }, + live_bind: ['cadence'], + live: { + summary: 'queued · ~T-10 min', + desc: 'high cadence through hatching' + }, + relations: { after: ['hd-1'] } + }, + { + id: 'hd-3', seq: 3, + name: 'Post-hatch monitor', + kind: 'standing_timelapse', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Resume normal cadence after the event.', + structure: { cadence_s: 120 }, + live_bind: ['cadence'], + live: { summary: 'queued · after event' }, + relations: { after: ['hd-2'] } + } + ] + }, + + /* ------------------------------------------------------------------ */ + transmission_survey: { + session_id: '20260628_1100_survey_a', + title: 'Transmission survey · plate A', + goal: 'Survey all embryos with brightfield only; no laser excitation.', + tactics: [ + { + id: 'srv-1', seq: 1, + name: 'Transmission burst', + kind: 'exclusive_burst', state: 'active', + scope: { mode: 'global' }, + rationale: 'LED/brightfield bursts, no laser — DIC-like contrast.', + structure: { frames: 30, mode: 'brightfield', phase: 'capturing' }, + live_bind: ['current_burst'], + live: { + summary: 'capturing', + desc: 'LED/brightfield bursts, no laser — DIC-like contrast', + readouts: [ + { label: 'bursts captured', value: '7', sub: 'across 3 embryos' }, + { label: 'illumination', value: 'LED · laser off', sub: 'brightfield' } + ] + }, + relations: {} + }, + { + id: 'srv-2', seq: 2, + name: 'Volume at best plane', + kind: 'oneshot', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Full volume capture at the best focal plane after operator review.', + structure: { note: 'operator selects plane' }, + live_bind: [], + live: { summary: 'queued · operator review' }, + relations: { after: ['srv-1'] } + } + ] + }, + + /* ------------------------------------------------------------------ */ + decided_plan: { + session_id: '20260628_1500_tempstrain_b', + title: 'Temperature-strain run · E02', + goal: 'Repeat the thermal strain protocol on a second embryo cohort.', + tactics: [ + { + id: 'dp-1', seq: 1, + name: 'Transmission burst — baseline', + kind: 'exclusive_burst', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Brightfield baseline before any change.', + structure: { frames: 3, mode: 'brightfield' }, + live_bind: [], + live: { + summary: 'queued · first', + desc: 'brightfield baseline before any change' + }, + relations: {} + }, + { + id: 'dp-2', seq: 2, + name: 'Temp-change burst protocol', + kind: 'scripted_protocol', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Thermal step to 30.0 °C with phased acquisition.', + structure: { + phases: [ + { name: 'before', state: 'todo', count: '0/1' }, + { name: 'during', state: 'todo', count: '0/3' }, + { name: 'after', state: 'todo', count: '0/1' } + ] + }, + live_bind: ['temperature', 'current_burst'], + live: { target: '→ 30.0 °C', summary: 'queued · second' }, + relations: { after: ['dp-1'] } + }, + { + id: 'dp-3', seq: 3, + name: 'Recovery monitor', + kind: 'standing_timelapse', state: 'planned', + scope: { mode: 'global' }, + rationale: 'Low-cadence monitoring after temperature settles.', + structure: { cadence_s: 180 }, + live_bind: ['cadence'], + live: { summary: 'queued · last' }, + relations: { after: ['dp-2'] } + } + ] + }, + + /* ------------------------------------------------------------------ */ + async_multi: { + session_id: '20260628_1630_async_multi', + title: 'Async multi-embryo run · 4 embryos', + goal: 'Per-embryo asynchronous acquisition with individual cadence phases; overlay a hatching watch on the two most advanced.', + tactics: [ + { + id: 'am-1', seq: 1, + name: 'Async timelapse — per-embryo cadence', + kind: 'standing_timelapse', state: 'active', + scope: { mode: 'embryos', embryo_ids: ['E01', 'E02', 'E03', 'E04'] }, + rationale: 'Each embryo runs at its own cadence based on developmental stage and reporter state.', + structure: { + cadence_s: 120, + per_embryo: [ + { embryo_id: 'E01', cadence_phase: 'normal', interval_s: 180 }, + { embryo_id: 'E02', cadence_phase: 'fast', interval_s: 30 }, + { embryo_id: 'E03', cadence_phase: 'burst', interval_s: 0 }, + { embryo_id: 'E04', cadence_phase: 'paused', interval_s: null } + ] + }, + live_bind: ['cadence'], + live: { + summary: 'running · 4 embryos', + readouts: [ + { label: 'active embryos', value: '3 / 4', sub: 'E04 paused' }, + { label: 'cadence range', value: '30–180s', sub: 'per-embryo mode' } + ] + }, + relations: {} + }, + { + id: 'am-2', seq: 2, + name: 'Hatching watch — E01, E02', + kind: 'reactive_monitor', state: 'active', + scope: { mode: 'embryos', embryo_ids: ['E01', 'E02'] }, + rationale: 'Overlay a hatching detector on the two most advanced embryos.', + structure: { watch: 'hatching', reaction: 'accelerate + alert', status: 'armed' }, + live_bind: ['signal'], + live: { + target: 'hatching', + summary: 'armed · E01, E02', + desc: 'watching for hatching onset on the two most advanced embryos', + readouts: [ + { label: 'watch status', value: 'armed', sub: 'no event yet' }, + { label: 'scope', value: 'E01, E02', sub: '2 of 4 embryos' } + ] + }, + relations: { layered_on: ['am-1'] } + } + ] + }, + + /* ------------------------------------------------------------------ */ + idle: null +}; diff --git a/gently/ui/web/static/js/projection-viewer.js b/gently/ui/web/static/js/projection-viewer.js index 1f5f530e..5cf7719d 100644 --- a/gently/ui/web/static/js/projection-viewer.js +++ b/gently/ui/web/static/js/projection-viewer.js @@ -118,6 +118,10 @@ const ProjectionViewer = { this.projections = []; this.selectedMethod = null; this.isOpen = true; + // Clear any volume from a previous open so a failed /api/volume-raw fetch + // can't leave the prior embryo/timepoint's 3D data bound (stale-render). + this.volumeData = null; + this.volumeShape = null; const modal = document.getElementById('projection-viewer-modal'); const loading = document.getElementById('pv-loading'); @@ -280,6 +284,10 @@ const ProjectionViewer = { }, selectMethod(method) { + // If the 3D view is requested but no volume loaded (e.g. /api/volume-raw + // failed while projections succeeded), fall back to the projections grid + // rather than showing an empty, never-initialized 3D panel. + if (method === '3d_viewer' && !this.volumeData) method = null; this.selectedMethod = method; this.renderProjections(); this.renderTabs(); @@ -298,6 +306,19 @@ const ProjectionViewer = { this.updateViewerVisibility(); }, + // Resize the WebGL canvas + camera to the container's current width. + // (Height is fixed at 400px; only width tracks the layout.) The animation + // loop handles re-rendering. + _resize3D() { + const container = document.getElementById('pv-3d-container'); + if (!container || !this.renderer3d || !this.camera3d) return; + const w = container.clientWidth || 500; + const h = 400; + this.renderer3d.setSize(w, h); + this.camera3d.aspect = w / h; + this.camera3d.updateProjectionMatrix(); + }, + // 3D Viewer Methods init3DViewer() { const container = document.getElementById('pv-3d-container'); @@ -319,6 +340,22 @@ const ProjectionViewer = { container.innerHTML = ''; container.appendChild(this.renderer3d.domElement); + // Keep the WebGL canvas in sync with its container width — the chat + // panel can dock/resize and the window can resize. The animation loop + // re-renders every frame, so on a size change we only need to resize the + // renderer + camera (coalesced to one rAF). Also listen for the explicit + // layout-change event the chat dock fires on collapse/expand + resize. + if (this._resizeObserver) this._resizeObserver.disconnect(); + this._resizeObserver = new ResizeObserver(() => { + if (this._resizeRaf) cancelAnimationFrame(this._resizeRaf); + this._resizeRaf = requestAnimationFrame(() => this._resize3D()); + }); + this._resizeObserver.observe(container); + if (!this._onLayoutChanged) { + this._onLayoutChanged = () => this._resize3D(); + window.addEventListener('gently:layout-changed', this._onLayoutChanged); + } + // Root group is the object the user rotates. Raymarched volume // mesh is added here. The group scale flips Y so the image // orientation matches 2D projections. @@ -609,6 +646,18 @@ const ProjectionViewer = { cancelAnimationFrame(this.animationId); this.animationId = null; } + if (this._resizeObserver) { + this._resizeObserver.disconnect(); + this._resizeObserver = null; + } + if (this._resizeRaf) { + cancelAnimationFrame(this._resizeRaf); + this._resizeRaf = null; + } + if (this._onLayoutChanged) { + window.removeEventListener('gently:layout-changed', this._onLayoutChanged); + this._onLayoutChanged = null; + } // Dispose the volume cube's geometry, material, and 3D texture. if (this.volumeMesh) { this.volumeMesh.geometry?.dispose(); diff --git a/gently/ui/web/static/js/replay-recorder.js b/gently/ui/web/static/js/replay-recorder.js new file mode 100644 index 00000000..fbccd282 --- /dev/null +++ b/gently/ui/web/static/js/replay-recorder.js @@ -0,0 +1,461 @@ +/* Session replay recorder — rrweb capture + semantic action log. + * + * Design contract (docs/superpowers/specs/2026-07-13-session-replay-design.md): + * the app always wins over the recording. Everything here is wrapped so that any + * failure degrades or disables the recorder silently — it must never throw into + * the page, block the main thread, or surface errors to the UI. + * + * The client is deliberately blind: it does not know the gently session id. + * It POSTs batches to /replay/ingest and the server files them under the + * active session (or an unassigned bucket). Removal = delete this file and + * its template include. + */ +(function () { + "use strict"; + + if (window.__gentlyReplay) return; // double-include guard + var STATE = { + tab: Math.random().toString(16).slice(2, 10), + rrwebBuf: [], + actionBuf: [], + dropped: 0, + failures: 0, + disabled: false, + stopFn: null, + flushTimer: null, + }; + window.__gentlyReplay = STATE; + + var INGEST_URL = "/replay/ingest"; + var FLUSH_MS = 4000; // short interval: bounds loss at tab close / app quit + // Count-based cap, NOT bytes: sizing would mean JSON.stringify on every + // rrweb event on the main thread (mutation-storm hot path). Serialization + // happens exactly once, at flush. 20k events ≫ one flush interval's worth. + var MAX_BUF_EVENTS = 20000; // drop-oldest beyond this (app > data) + var MAX_FAILURES = 5; // consecutive ingest failures before self-disable + var CHECKOUT_MS = 5 * 60 * 1000; // periodic full snapshots: cheap seeking later + // Live camera gets base64 data-URI src swaps at frame rate — recording + // it would add ~100KB+ per frame on the main thread. Blocked from capture; + // the bus-summary action records that frames were flowing instead. + var BLOCK_SELECTOR = "#op-img-bottom, #op-img-spim"; + // Machine-driven, high-churn regions: the live map re-renders at stage-poll + // rate, the 3D occupancy canvas animates, the temperature graph redraws on + // every reading. In 'balanced' fidelity these are blocked from the DOM stream + // (a placeholder box replays in their place) — they're the bulk of the volume. + var HIGH_CHURN = + "#devices-map-svg, #occ3d-container, #occ3d-minimap, #devices-temp-graph"; + + // Recording fidelity, most-specific first: a ?replay= URL override (for + // an ad-hoc high/low-fidelity capture), else the server default stamped on the + // recorder's @@ -27,5 +28,6 @@ + diff --git a/gently/ui/web/templates/index.html b/gently/ui/web/templates/index.html index dd3bbfb0..bf088472 100644 --- a/gently/ui/web/templates/index.html +++ b/gently/ui/web/templates/index.html @@ -11,15 +11,201 @@ + + + + + + + - + + {% if show_landing %} + {# Agent-first welcome — shown on a FRESH entry, recedes into the workspace + once the user picks a path. Suppressed when resuming a session or when the + session already has work (show_landing computed in the index route) so + resuming doesn't bounce back through "what are we doing today?". Chat is + the last resort (the escape pill), not the first thing. #} +
    + {# The header's theme toggle is occluded by this full-bleed overlay + (z-index 200), so the welcome/plan screens carry their own. Reuses + .theme-toggle + .theme-icon (icon-swap + styling) from main.css. #} + +
    + + {# ── Screen 1: welcome ── #} +
    +
    +
    +
    Hello.
    What are we doing today?
    +
    + +
    + + + +
    + +
    + +
    + + +
    +
    + + +
    + + {# ── Screen 2: the plan wizard, hosted IN the landing. The agent's + ask_user_choice questions render here as button cards (#v2-plan-ask), + NOT in the chat panel. The plan assembles on the right as you pick. #} +
    +
    + +
    +
    Gently · planning
    +
    Let's design your run
    +
    + +
    +
    +
    +
    +
    +
    working through the next step…
    + +
    + +
    +
    + + + + +
    +
    + +
    +
    + {% endif %} {% include '_header.html' %} {% include '_navbar.html' %} + +
    + {% if ux_v2 %} + + {% endif %} +
    + + {% if ux_v2 %}
    LIVE
    {% endif %} + + {# ux_v2: the agent's current pending ask, dual-rendered here + in the chat. #} + {% if ux_v2 %}{% endif %} + +
    +
    + {% if ux_v2 %}{% endif %} +
    +
    +

    Welcome to Gently

    +
    Connecting…
    +
    + +
    +
    +
    +
    + Recent sessions + All +
    +
    +
    Loading…
    +
    +
    +
    +
    + Recent plans + All +
    +
    +
    Loading…
    +
    +
    +
    +
    + Recent images + All +
    +
    +
    No images yet — they appear once a session is active.
    +
    +
    +
    +
    +
    +

    Calibration

    @@ -122,7 +308,7 @@

    Calibration

    -
    +
    Monitoring
    @@ -224,7 +410,7 @@

    Embryo Monitoring

    -

    Experiment

    +

    Operations

    @@ -245,6 +431,31 @@

    Experiment

    {% include '_sessions_panel.html' %}
    + +
    +
    +
    +

    Notebook

    +
    + + + + +
    +
    +
    + + +
    + +
    + +
    +
    +
    +
    +
    @@ -252,18 +463,318 @@

    Experiment

    Device state

    - + + + +
    + + disconnected no data
    + + + + +
    +
    + + + +
    + + + +
    + + +
    + +
    +
    +
    + Bottom camera view + +
    Camera off
    + +
    +
    + +
    + Click to mark · click a marker to remove · click a registered embryo to centre on it + + Marked + 0 + + + + +
    +

    +
    + + +
    + + + + + + + +
    +
    + +
    +
    + -
    +
    + + + + + + + + +
    +
    +
    + + + @@ -498,26 +1346,106 @@

    Properties

    +
    + + + +
    + + + + + + + + + + + + + + + + + + + + + {% if replay_enabled %} + + + + {% endif %} diff --git a/gently/ui/web/templates/launch.html b/gently/ui/web/templates/launch.html new file mode 100644 index 00000000..ff3070d2 --- /dev/null +++ b/gently/ui/web/templates/launch.html @@ -0,0 +1,194 @@ + + + + + +Launch Gently + + + + + +
    +
    Gently
    +

    What are we doing today?

    + +
    + +
    +
    Microscope
    +
    +
    + +
    + +
    + +
    +
    Assistant
    +
    +
    + +
    + + +
    +
    Remembers your last choice
    +
    + + +{% if replay_enabled %} + + + +{% endif %} + + diff --git a/gently/ui/web/templates/login.html b/gently/ui/web/templates/login.html new file mode 100644 index 00000000..9a98be17 --- /dev/null +++ b/gently/ui/web/templates/login.html @@ -0,0 +1,118 @@ + + + + + + Sign in · Gently + + + + + + +
    +
    + +

    Gently

    +
    +

    Sign in to control the microscope — or keep watching in view-only mode.

    + + + + + +
    +
    or
    + Continue without signing in → +

    View-only: watch live sessions and imagery. You can sign in any time to take control.

    +
    + + {% if replay_enabled %} + + + {% endif %} + + diff --git a/gently/ui/web/templates/replay.html b/gently/ui/web/templates/replay.html new file mode 100644 index 00000000..93abd10d --- /dev/null +++ b/gently/ui/web/templates/replay.html @@ -0,0 +1,245 @@ + + + + + +Gently — Session Replay + + + + + +
    +

    Session Replay

    + {% if session_id %}{{ session_id }}{% endif %} + {{ '← all recordings' if session_id else '← back to gently' }} +
    + +{% if not session_id %} +
    + {% if recordings %} + + + {% for r in recordings %} + + + + + + {% endfor %} +
    SessionTabsSize
    {{ r.id }}{{ r.tabs | length }}{{ '%.1f' | format(r.bytes / 1048576) }} MB
    + {% else %} +
    No recordings yet — use the app with replay enabled and sessions will appear here.
    + {% endif %} +
    +{% else %} +
    +
    +
    +
    + + + 0:00 / 0:00 + +
    +
    Loading events…
    +
    +
    +

    Tabs

    + +

    Actions

    +
    +
    +
    + + + +{% endif %} + + diff --git a/gently/ui/web/templates/review.html b/gently/ui/web/templates/review.html index 7115aeb7..94298ed0 100644 --- a/gently/ui/web/templates/review.html +++ b/gently/ui/web/templates/review.html @@ -7,11 +7,12 @@ + {% include '_header.html' %} @@ -26,5 +27,6 @@ + diff --git a/gently/ui/web/templates/settings.html b/gently/ui/web/templates/settings.html index 9e552936..b24655fb 100644 --- a/gently/ui/web/templates/settings.html +++ b/gently/ui/web/templates/settings.html @@ -9,7 +9,7 @@ @@ -52,6 +52,13 @@

    Settings

    Vitals View Default View
    +
    @@ -88,6 +95,31 @@

    Alert Thresholds

    + +
    +

    Device layer

    +

    + Install-time settings for the microscope control process. Most rigs never + change these — they're kept off the launch screen on purpose. +

    +
    + +
    + +
    +
    +
    + +
    + + + +
    +

    +
    +

    +
    +

    Ambient Pulse

    @@ -172,11 +204,12 @@

    Filmstrip View

    Vitals View

    - +
    +
    C. elegans staging reference curve for the vitals chart — not the hardware controller (see Hardware → Thermalizer).
    +
    +

    Thermalizer (ACUITYnano)

    +
    Machine-wide hardware setting, saved on the server (not this browser). Requires control.
    + +
    + +
    Loading…
    +
    + +
    + +
    + + + +
    +
    + +
    +
    +
    +
    + + +
    +
    + +
    + + +
    +
    +
    + +
    +

    Effective config (read-only)

    +
    The server's live configuration, secrets redacted. Editing these needs a restart / env override.
    +
    Loading…
    +
    + +
    +

    Advanced (restart required)

    +
    System tunables persisted to config/settings.local.yml. Saved values take effect on the next server restart. Requires control.
    +
    Loading…
    +
    + +
    +
    +
    +
    + + + + + + +
    @@ -235,6 +330,53 @@

    Default View

    localStorage.setItem('gently-theme', next); }); } + + // Device-layer settings (port + SAM device) — persisted via /api/launch/prefs. + // Applies to the next device-layer start (a running one isn't reconfigured). + (function () { + const port = document.getElementById('dl-port'); + const detected = document.getElementById('dl-sam-detected'); + const status = document.getElementById('dl-status'); + if (!port) return; + let loaded = false; + fetch('/api/launch/prefs').then(r => (r.ok ? r.json() : null)).then(p => { + if (!p) return; + port.value = (p.port != null ? p.port : ''); + const sam = p.sam_device_raw || 'auto'; + const radio = document.querySelector('#dl-sam input[value="' + sam + '"]') + || document.querySelector('#dl-sam input[value="auto"]'); + if (radio) radio.checked = true; + if (detected) { + detected.textContent = 'Detected: ' + (p.sam_detected === 'cuda' + ? 'GPU (CUDA)' + : 'CPU — no GPU found, so image analysis will be slower'); + } + loaded = true; + }); + async function save() { + if (!loaded) return; + const sam = (document.querySelector('#dl-sam input:checked') || {}).value || 'auto'; + const body = { sam_device: sam }; + const pv = parseInt(port.value, 10); + if (pv) body.port = pv; + try { + const r = await fetch('/api/launch/prefs', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(body), + }); + if (status) status.textContent = r.ok + ? 'Saved — applies next time the device layer starts' + : (r.status === 403 ? 'Sign in to change device settings' : 'Failed to save'); + } catch (e) { if (status) status.textContent = 'Failed to save'; } + } + port.addEventListener('change', save); + document.querySelectorAll('#dl-sam input').forEach(r => r.addEventListener('change', save)); + })(); + {% if replay_enabled %} + + + {% endif %} diff --git a/gently/ui/web/timelapse_tracker.py b/gently/ui/web/timelapse_tracker.py index b57086f9..965e4019 100644 --- a/gently/ui/web/timelapse_tracker.py +++ b/gently/ui/web/timelapse_tracker.py @@ -6,7 +6,6 @@ """ from datetime import datetime -from typing import Dict, List, Optional class TimelapseStateTracker: @@ -21,15 +20,17 @@ class TimelapseStateTracker: """ def __init__(self): - self.session_id: Optional[str] = None # Unique ID per experiment + self.session_id: str | None = None # Unique ID per experiment self.status = "IDLE" # IDLE, RUNNING, PAUSED, COMPLETED - self.started_at: Optional[str] = None - self.embryos: Dict[str, dict] = {} # embryo_id -> state + self.started_at: str | None = None + self.embryos: dict[str, dict] = {} # embryo_id -> state self.total_timepoints = 0 self.base_interval = 120 - self.detection_reasoning: Dict[str, List[dict]] = {} # embryo_id -> list of detections - self.projection_uids: Dict[str, Dict[int, str]] = {} # embryo_id -> {timepoint -> projection_uid} - self.volume_paths: Dict[str, Dict[int, str]] = {} # embryo_id -> {timepoint -> volume_path} + self.detection_reasoning: dict[str, list[dict]] = {} # embryo_id -> list of detections + self.projection_uids: dict[ + str, dict[int, str] + ] = {} # embryo_id -> {timepoint -> projection_uid} + self.volume_paths: dict[str, dict[int, str]] = {} # embryo_id -> {timepoint -> volume_path} def handle_event(self, event_type: str, data: dict): """Update state based on incoming event""" @@ -128,6 +129,16 @@ def handle_event(self, event_type: str, data: dict): self.status = "STOPPED" # Don't mark embryos as complete - they were stopped, not finished + elif event_type == "EMBRYO_TERMINATED": + # A single embryo's imaging was halted by the orchestrator + # (no_object terminal, configured stop condition, errors, etc). + # Carry the completion_reason through so the UI can show why. + eid = data.get("embryo_id") + if eid and eid in self.embryos: + self.embryos[eid]["is_complete"] = True + self.embryos[eid]["completion_reason"] = data.get("completion_reason") + self.embryos[eid].setdefault("completed_at", datetime.now().isoformat()) + elif event_type == "DETECTOR_EVALUATED": # All detector/perception evaluations (with reasoning) - populates reasoning panel eid = data.get("embryo_id") @@ -148,7 +159,8 @@ def handle_event(self, event_type: str, data: dict): "description": data.get("description"), "timepoint": timepoint, "volume_uid": data.get("volume_uid"), - "projection_uid": data.get("projection_uid") or projection_uid, # Use stored UID as fallback + "projection_uid": data.get("projection_uid") + or projection_uid, # Use stored UID as fallback "timestamp": datetime.now().isoformat(), # Perception-specific fields "stage": data.get("stage"), @@ -186,21 +198,26 @@ def handle_event(self, event_type: str, data: dict): # before any acquisition has happened. eid = data.get("embryo_id") if eid: - emb = self.embryos.setdefault(eid, { - "embryo_id": eid, - "timepoints": 0, - "is_complete": False, - "first_acquired": None, - "last_acquired": None, - "detections": {}, - "current_stage": None, - }) + emb = self.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "first_acquired": None, + "last_acquired": None, + "detections": {}, + "current_stage": None, + }, + ) if data.get("x") is not None: emb["stage_x_um"] = data["x"] if data.get("y") is not None: emb["stage_y_um"] = data["y"] if data.get("role"): emb["role"] = data["role"] + if "strain" in data: + emb["strain"] = data.get("strain") if data.get("uid"): emb["uid"] = data["uid"] if data.get("user_label"): @@ -215,13 +232,11 @@ def handle_event(self, event_type: str, data: dict): detector_name = data.get("detector_name", "unknown") self.embryos[eid]["detections"][detector_name] = { "detected": True, - "confidence": data.get("confidence") + "confidence": data.get("confidence"), } if detector_name == "hatching": self.embryos[eid]["is_complete"] = True - self.embryos[eid].setdefault( - "completed_at", datetime.now().isoformat() - ) + self.embryos[eid].setdefault("completed_at", datetime.now().isoformat()) elif event_type == "VERIFICATION_STARTED": # Verification round started for embryo @@ -250,8 +265,12 @@ def handle_event(self, event_type: str, data: dict): # Progress update eid = data.get("embryo_id") if eid and eid in self.embryos and "verification" in self.embryos[eid]: - self.embryos[eid]["verification"]["strategies_complete"] = data.get("strategies_complete", 0) - self.embryos[eid]["verification"]["total_strategies"] = data.get("total_strategies", 5) + self.embryos[eid]["verification"]["strategies_complete"] = data.get( + "strategies_complete", 0 + ) + self.embryos[eid]["verification"]["total_strategies"] = data.get( + "total_strategies", 5 + ) elif event_type == "VERIFICATION_COMPLETED": # Final verification result @@ -292,10 +311,16 @@ def handle_event(self, event_type: str, data: dict): if data.get("change") == "role_assigned": eid = data.get("embryo_id") if eid: - emb = self.embryos.setdefault(eid, { - "embryo_id": eid, "timepoints": 0, "is_complete": False, - "detections": {}, "current_stage": None, - }) + emb = self.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "detections": {}, + "current_stage": None, + }, + ) if data.get("new_role"): emb["role"] = data["new_role"] @@ -304,10 +329,16 @@ def handle_event(self, event_type: str, data: dict): elif event_type == "EMBRYO_CADENCE_CHANGED": eid = data.get("embryo_id") if eid: - emb = self.embryos.setdefault(eid, { - "embryo_id": eid, "timepoints": 0, "is_complete": False, - "detections": {}, "current_stage": None, - }) + emb = self.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "detections": {}, + "current_stage": None, + }, + ) if data.get("new_phase") is not None: emb["cadence_phase"] = data["new_phase"] if data.get("new_interval_s") is not None: @@ -319,22 +350,30 @@ def handle_event(self, event_type: str, data: dict): elif event_type == "POWER_RAMP_STEP": eid = data.get("embryo_id") if eid: - emb = self.embryos.setdefault(eid, { - "embryo_id": eid, "timepoints": 0, "is_complete": False, - "detections": {}, "current_stage": None, - }) + emb = self.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "detections": {}, + "current_stage": None, + }, + ) wavelength = data.get("wavelength", 488) if wavelength == 488: emb["laser_power_488_pct"] = data.get("new_pct") - emb.setdefault("power_history", []).append({ - "wavelength": wavelength, - "old_pct": data.get("old_pct"), - "new_pct": data.get("new_pct"), - "direction": data.get("direction"), - "rule": data.get("rule"), - "intensity_level": data.get("intensity_level"), - "timestamp": datetime.now().isoformat(), - }) + emb.setdefault("power_history", []).append( + { + "wavelength": wavelength, + "old_pct": data.get("old_pct"), + "new_pct": data.get("new_pct"), + "direction": data.get("direction"), + "rule": data.get("rule"), + "intensity_level": data.get("intensity_level"), + "timestamp": datetime.now().isoformat(), + } + ) # cap history per embryo if len(emb["power_history"]) > 200: emb["power_history"] = emb["power_history"][-200:] @@ -342,10 +381,16 @@ def handle_event(self, event_type: str, data: dict): elif event_type == "CLAUDE_DETECTOR_RESULT": eid = data.get("embryo_id") if eid: - emb = self.embryos.setdefault(eid, { - "embryo_id": eid, "timepoints": 0, "is_complete": False, - "detections": {}, "current_stage": None, - }) + emb = self.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "detections": {}, + "current_stage": None, + }, + ) findings = data.get("findings") or {} emb["last_intensity_level"] = findings.get("intensity_level") emb["last_structure_quality"] = findings.get("structure_quality") @@ -353,13 +398,24 @@ def handle_event(self, event_type: str, data: dict): if findings.get("has_hatched"): emb["hatched"] = True - elif event_type in ("BURST_QUEUED", "BURST_START", "BURST_FRAME", "BURST_COMPLETE"): + elif event_type in ( + "BURST_QUEUED", + "BURST_START", + "BURST_FRAME", + "BURST_COMPLETE", + ): eid = data.get("embryo_id") if eid: - emb = self.embryos.setdefault(eid, { - "embryo_id": eid, "timepoints": 0, "is_complete": False, - "detections": {}, "current_stage": None, - }) + emb = self.embryos.setdefault( + eid, + { + "embryo_id": eid, + "timepoints": 0, + "is_complete": False, + "detections": {}, + "current_stage": None, + }, + ) emb.setdefault("burst", {}) burst_state = emb["burst"] if event_type == "BURST_QUEUED": @@ -397,7 +453,7 @@ def to_dict(self) -> dict: "embryos": self.embryos, "total_timepoints": self.total_timepoints, "base_interval": self.base_interval, - "detection_reasoning": self.detection_reasoning + "detection_reasoning": self.detection_reasoning, } def reset(self): @@ -424,14 +480,18 @@ def seed_from_experiment(self, experiment) -> int: y = pos.get("y") if isinstance(pos, dict) else None if x is None or y is None: continue - self.handle_event("EMBRYO_DETECTED", { - "embryo_id": eid, - "uid": getattr(emb, "uid", None), - "x": x, - "y": y, - "role": getattr(emb, "role", "test"), - "user_label": getattr(emb, "user_label", None), - "confidence": getattr(emb, "detection_confidence", None), - }) + self.handle_event( + "EMBRYO_DETECTED", + { + "embryo_id": eid, + "uid": getattr(emb, "uid", None), + "x": x, + "y": y, + "role": getattr(emb, "role", "test"), + "strain": getattr(emb, "strain", None), + "user_label": getattr(emb, "user_label", None), + "confidence": getattr(emb, "detection_confidence", None), + }, + ) seeded += 1 return seeded diff --git a/gently/ui/web/upload_validation.py b/gently/ui/web/upload_validation.py new file mode 100644 index 00000000..c4f2b282 --- /dev/null +++ b/gently/ui/web/upload_validation.py @@ -0,0 +1,71 @@ +"""Validation helpers for HTTP array upload routes.""" + +from __future__ import annotations + +import base64 +import binascii +import math +from collections.abc import Iterable + +import numpy as np +from fastapi import HTTPException + + +def decode_array_payload( + encoded: str, + shape: Iterable[int], + dtype_name: str, + *, + max_nbytes: int, + label: str, +) -> np.ndarray: + """Decode a base64 array after validating shape, dtype, and byte count. + + Guards the raw ``np.frombuffer(b64decode(...)).reshape(shape)`` path against + attacker-controlled input: bounds the dimension count, forbids object + dtypes, caps the decoded size *before* allocating, and requires the decoded + byte length to match shape x dtype exactly. + """ + if not isinstance(encoded, str) or not encoded: + raise HTTPException(status_code=400, detail=f"{label} payload must be base64 text") + if isinstance(shape, (str, bytes)) or not isinstance(shape, Iterable): + raise HTTPException(status_code=400, detail=f"{label} shape must be a list of dimensions") + + try: + shape_tuple = tuple(int(dim) for dim in shape) + except (TypeError, ValueError): + raise HTTPException( + status_code=400, detail=f"{label} shape must contain integers" + ) from None + if not shape_tuple or len(shape_tuple) > 4 or any(dim <= 0 for dim in shape_tuple): + raise HTTPException( + status_code=400, detail=f"{label} shape must have 1-4 positive dimensions" + ) + + try: + dtype = np.dtype(dtype_name) + except TypeError: + raise HTTPException(status_code=400, detail=f"{label} dtype is not supported") from None + if dtype.hasobject: + raise HTTPException(status_code=400, detail=f"{label} dtype may not contain Python objects") + + expected_nbytes = math.prod(shape_tuple) * dtype.itemsize + if expected_nbytes > max_nbytes: + raise HTTPException( + status_code=413, + detail=f"{label} payload is too large ({expected_nbytes} bytes > {max_nbytes} bytes)", + ) + + try: + raw = base64.b64decode(encoded, validate=True) + except (binascii.Error, ValueError): + raise HTTPException( + status_code=400, detail=f"{label} payload is not valid base64" + ) from None + if len(raw) != expected_nbytes: + raise HTTPException( + status_code=400, + detail=f"{label} byte length {len(raw)} does not match shape/dtype {expected_nbytes}", + ) + + return np.frombuffer(raw, dtype=dtype).reshape(shape_tuple) diff --git a/gently/ui/web/volume_helpers.py b/gently/ui/web/volume_helpers.py index e92954ae..2ff479d9 100644 --- a/gently/ui/web/volume_helpers.py +++ b/gently/ui/web/volume_helpers.py @@ -10,11 +10,10 @@ import re from io import BytesIO from pathlib import Path -from typing import Optional import numpy as np -from gently.core.imaging import normalize_to_uint8, image_to_base64 +from gently.core.imaging import image_to_base64, normalize_to_uint8 logger = logging.getLogger(__name__) @@ -22,7 +21,7 @@ VOLUME_UID_PATTERN = re.compile(r"volume_(.+)_t(\d+)$") -def parse_volume_uid(uid: str) -> Optional[tuple]: +def parse_volume_uid(uid: str) -> tuple | None: """Parse a volume UID into (embryo_id, timepoint) or return None.""" if not uid.startswith("volume_"): return None @@ -42,9 +41,9 @@ def load_volume_from_disk(volume_path: str) -> np.ndarray: Cropped 3D numpy array (Z, H, W) """ from gently.core.imaging import ( - load_volume, - compute_crop_bounds, apply_crop_bounds, + compute_crop_bounds, + load_volume, ) path = Path(volume_path) @@ -67,7 +66,7 @@ def array_to_png_bytes(img_array: np.ndarray) -> bytes: img_array = normalize_to_uint8(img_array, method="simple") img = Image.fromarray(img_array) buf = BytesIO() - img.save(buf, format='PNG') + img.save(buf, format="PNG") return buf.getvalue() diff --git a/launch_gently.py b/launch_gently.py index 1727dd2a..b63f353b 100644 --- a/launch_gently.py +++ b/launch_gently.py @@ -4,36 +4,58 @@ Conversational AI agent for diSPIM microscope control. +Starts the agent + web visualization server, then opens the browser UI. +The web UI is the control surface (the legacy Ink TUI is retired — its +source is kept in the tree but no longer launched). + Usage: - python launch_gently.py # Ink TUI (default) - python launch_gently.py --offline + python launch_gently.py # Start server + open browser + python launch_gently.py --no-browser # Start server, don't open a browser + python launch_gently.py --offline # Run without the device layer + python launch_gently.py --no-api # UI-only: boot the web UI without an API key python launch_gently.py --sessions # List sessions and exit - python launch_gently.py --resume # Interactive session picker + python launch_gently.py --resume # Resume most recent session python launch_gently.py --resume latest # Resume most recent session python launch_gently.py --resume # Resume specific session python launch_gently.py -v # Verbose (INFO) logging python launch_gently.py --debug # Debug logging """ +import argparse import asyncio -import json import logging import os -import sys import shutil import subprocess -import argparse -from pathlib import Path +import sys from datetime import datetime +from pathlib import Path import yaml -from gently.log_config import configure_logging +# Load a project-root .env (if present) so ANTHROPIC_API_KEY and other +# settings can live in a file instead of being exported every session. +# Existing environment variables take precedence. +try: + from dotenv import load_dotenv + + load_dotenv(Path(__file__).resolve().parent / ".env") +except ImportError: + pass + +# The gently imports below pull in heavy dependencies (anthropic, torch, scipy, +# perception) and take several seconds. Print immediate feedback first so the +# terminal isn't silent during that load. Skipped for --help/--version. +if not any(flag in sys.argv for flag in ("-h", "--help")): + print("Starting gently — loading modules (this can take a few seconds)...", flush=True) + from gently.app.agent import MicroscopyAgent +from gently.core.file_store import FileStore +from gently.core.log_bridge import configure_log_bridge +from gently.hardware import get_hardware, load_hardware +from gently.log_config import configure_logging from gently.organisms import load_organism -from gently.hardware import load_hardware, get_hardware from gently.settings import settings -from gently.core.file_store import FileStore logger = logging.getLogger(__name__) @@ -65,11 +87,13 @@ def _build_session_items(store: FileStore) -> list: session_id = session.get("session_id", "unknown") embryos = store.list_embryos(session_id) embryo_count = len(embryos) if embryos else 0 - items.append({ - "session_id": session_id, - "embryo_count": embryo_count, - "time": _format_elapsed(session.get("last_active", "")), - }) + items.append( + { + "session_id": session_id, + "embryo_count": embryo_count, + "time": _format_elapsed(session.get("last_active", "")), + } + ) return items @@ -89,10 +113,93 @@ def list_sessions(store: FileStore): print("Use: python launch_gently.py --resume ") +def _print_banner(viz_url, device_connected, offline, storage_dir, log_file, resumed, no_api=False): + """Print a human-readable launch banner to the terminal. + + This is the "what you see when you open it" surface now that the + server (not a TUI) is the long-running process. + """ + line = "─" * 56 + if offline: + dev = "○ offline (--offline)" + elif device_connected: + dev = "● connected" + else: + dev = "○ offline — run: python start_device_layer.py" + agent_status = "○ disabled — UI only (--no-api)" if no_api else "● enabled" + url = viz_url or "(viz server failed to start — check the log)" + tag = " [resumed session]" if resumed else "" + print() + print(f" ✦ Gently is running.{tag}") + print(f" {line}") + print(f" Open: {url}") + print(f" Agent: {agent_status}") + print(f" Device: {dev}") + print(f" Storage: {storage_dir}") + print(f" Logs: {log_file}") + print(" Stop: Ctrl-C") + print(f" {line}") + print() + + +def _open_browser(url: str) -> None: + """Open the web UI, preferring Google Chrome. + + Override with GENTLY_BROWSER (a webbrowser name like 'firefox', or a full + path to a browser executable). Falls back to the OS default browser if + Chrome can't be found, so this never blocks startup. + """ + import webbrowser + + override = os.environ.get("GENTLY_BROWSER", "").strip() + + # 1) Registered browser names (override first, then Chrome aliases). + for name in ([override] if override else []) + [ + "chrome", + "google-chrome", + "chromium", + ]: + try: + webbrowser.get(name).open(url) + return + except Exception: + pass + + # 2) Explicit executables (an override path, then known Chrome locations). + candidates: list[str | None] = [override] if override else [] + candidates += [ + shutil.which("chrome"), + r"C:\Program Files\Google\Chrome\Application\chrome.exe", + r"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe", + ] + for exe in candidates: + try: + if exe and Path(exe).exists(): + webbrowser.register( + "gently-browser", + None, + webbrowser.BackgroundBrowser(exe), + preferred=True, + ) + webbrowser.get("gently-browser").open(url) + return + except Exception: + pass + + # 3) Fall back to the OS default. + try: + webbrowser.open(url) + except Exception: + pass + + def run_ink_picker(tui_dist: Path, sessions_json: str) -> str | None: """ Spawn the Ink TUI in session-picker mode and capture the selection. + Retired: kept for reference / potential reuse by a future web session + picker. No longer called by the launcher. + Returns the selected session ID, or None for a new session. """ proc = subprocess.run( @@ -110,20 +217,155 @@ def run_ink_picker(tui_dist: Path, sessions_json: str) -> str | None: # Parse the SESSION: protocol line from stdout for line in (proc.stdout or "").splitlines(): if line.startswith("SESSION:"): - selected = line[len("SESSION:"):].strip() + selected = line[len("SESSION:") :].strip() return selected if selected else None return None -async def main(offline: bool = False, resume_session: str = None, show_sessions: bool = False, pick_session: bool = False, log_level: str = "WARNING"): +# launch_gently runs as `__main__`, whose logger is not wired to the configured +# handlers — so operator-facing lifecycle events (reconnect, tool registration) +# use a `gently.*`-namespaced logger that reaches the console + file logs. +_reconnect_log = logging.getLogger("gently.app.launch") + + +async def _attach_microscope(client, store) -> int: + """Configure the device session + register microscope tools for a connected + client. Used both at boot and when reconnecting to a device layer that was + started from the launch gate / Devices panel. Returns the tool count.""" + try: + await client.configure_device_session(str(store.incoming_dir)) + logger.info("Device session configured: volume_dir=%s", store.incoming_dir) + except Exception as e: + logger.error("Failed to configure device session (volumes will be slow): %s", e) + try: + from gently.harness.microscope import register_microscope_tools + + n = register_microscope_tools(client) + if n: + logger.info("Registered %d microscope tools from device layer", n) + return n + except Exception as e: + logger.debug("Auto-tool registration skipped: %s", e) + return 0 + + +async def _watch_device_layer(agent, client, viz_server, store) -> None: + """RFC #78 single availability watcher — the one producer of hardware + availability that every dependent surface derives from. + + Polls the device-layer supervisor and, on each state *transition*, drives the + agent's hardware wiring and emits a DEVICE_LAYER_AVAILABILITY signal: + + · layer usable ('ready' managed, or 'external'): connect the client if + needed, attach the agent (tools + session + live telemetry monitors), + announce available — so hardware started mid-session (launch gate, Devices + panel, or a separately-run device server) becomes usable without relaunch. + · layer down ('stopped'/'crashed'/'failed') or still booting: detach the + agent (stop telemetry monitors), disconnect the client, announce + unavailable. + + Runs for every session now that the client is always created (even 'offline'), + so start/stop from anywhere propagates cleanly instead of dead-ending on a + client that was never built.""" + from gently.core.event_bus import EventType + + def announce(state: str, usable: bool) -> None: + try: + agent._emit_event( + EventType.DEVICE_LAYER_AVAILABILITY, + { + "state": state, + "available": bool(usable and client and client.is_connected), + "connected": bool(client and client.is_connected), + }, + ) + except Exception: + _reconnect_log.debug("availability announce failed", exc_info=True) + + prev_state = None + # Boot may have already connected + attached (layer up at launch): reflect it + # so we don't redundantly re-attach on the first poll. + attached = bool(client and client.is_connected) + while True: + await asyncio.sleep(2.0) + sup = getattr(viz_server, "device_supervisor", None) if viz_server else None + if sup is None: + continue + try: + # status() does a short socket probe — off the event loop. + state = (await asyncio.to_thread(sup.status)).get("state") + except Exception: + continue + if state == prev_state: + continue + prev_state = state + + usable = state in ("ready", "external") + if usable: + # Connect the client to the now-usable layer, then attach the agent. + if client is not None and not client.is_connected: + try: + await client.disconnect() # drop any stale failed-at-boot session + except Exception: + pass + try: + await client.connect() + except Exception as e: + _reconnect_log.debug("client connect on '%s' failed: %s", state, e) + if client is not None and client.is_connected and not attached: + try: + await agent.attach_hardware() + attached = True + _reconnect_log.info("Device layer %s — agent attached to hardware", state) + except Exception as e: + _reconnect_log.warning("attach_hardware failed: %s", e) + else: + # Booting or gone — hardware not usable; detach if we were attached. + if attached: + try: + await agent.detach_hardware() + _reconnect_log.info("Device layer %s — agent detached from hardware", state) + except Exception as e: + _reconnect_log.warning("detach_hardware failed: %s", e) + attached = False + if ( + client is not None + and client.is_connected + and state in ("stopped", "crashed", "failed") + ): + try: + await client.disconnect() + except Exception: + pass + announce(state, usable) + + +async def main( + offline: bool = False, + resume_session: str | None = None, + show_sessions: bool = False, + pick_session: bool = False, + log_level: str = "WARNING", + no_browser: bool = False, + no_api: bool = False, + no_auth: bool = False, +): # Set up log file in storage directory - storage_base = Path(os.environ.get("GENTLY_STORAGE", "D:/Gently3")) + # Unified with FileStore: logs live under the same root as data + # (settings.storage.base_path reads GENTLY_STORAGE_PATH). Previously this + # read a separate GENTLY_STORAGE env var, so setting only one split logs + # from data. + storage_base = settings.storage.base_path log_dir = storage_base / "logs" log_dir.mkdir(parents=True, exist_ok=True) log_file = str(log_dir / f"gently_{datetime.now().strftime('%Y%m%d_%H%M%S')}.log") # File always gets INFO+, console uses the requested level configure_logging(level=log_level, log_file=log_file) + # Mirror gently / gently_perception log lines onto the EventBus so the + # Events page in the viz server shows them too. Env vars control level + # and whether to include noisy third-party loggers (off by default). + configure_log_bridge() logger.info("Logging to %s (console level: %s)", log_file, log_level) # Load organism module from config @@ -142,39 +384,54 @@ async def main(offline: bool = False, resume_session: str = None, show_sessions: # Create unified store (FileStore) early for session queries from gently.core.gently_manifest import write_manifest + write_manifest(storage_dir) store = FileStore(storage_dir) + # ── Accounts / auth ─────────────────────────────────────────── + # Self-managed user accounts gate microscope control on the LAN. On first + # run we bootstrap an admin and print its one-time password in the banner. + # Pass --no-auth (or set GENTLY_NO_AUTH=1) to disable accounts (localhost-control mode). + admin_creds = None + auth_disabled = no_auth or os.environ.get("GENTLY_NO_AUTH", "").strip().lower() in ( + "1", + "true", + "yes", + ) + if auth_disabled: + logger.warning("Accounts disabled (--no-auth) — control is open on this host") + if not auth_disabled: + try: + from gently.ui.web.accounts import AccountStore, set_account_store + + account_store = AccountStore(storage_dir / "auth") + set_account_store(account_store) + admin_creds = account_store.bootstrap_admin_if_empty() + except Exception as e: + logger.error("Account store init failed (continuing without auth): %s", e) + # Handle --sessions (just list and exit) if show_sessions: list_sessions(store) store.close() return - # Ensure TUI is available - tui_dist = Path(__file__).parent / "gently" / "tui" / "dist" / "index.js" - if not tui_dist.exists() or not shutil.which("node"): - print("Error: TUI not available.") - if not tui_dist.exists(): - print(" Run: cd gently/tui && npm install && npm run build") - if not shutil.which("node"): - print(" Node.js not found in PATH") - store.close() - return + # Web-only: the TUI is retired. The browser is the control surface and + # the launcher just starts the server — no Node/dist requirement. - # Handle --resume (interactive picker, "latest", or specific session) + # Handle --resume. Interactive session picking has moved to the browser; + # without an explicit ID ("latest" or bare --resume) we resume the most + # recent session. session_to_resume = None - if pick_session: - # Two-phase launch: spawn Ink picker to select a session - items = _build_session_items(store) - if not items: - print("No saved sessions found. Starting new session.") - else: - session_to_resume = run_ink_picker(tui_dist, json.dumps(items)) - elif resume_session == "latest": + if pick_session or resume_session == "latest": sessions = store.list_sessions() if sessions: session_to_resume = sessions[0].get("session_id") + if pick_session: + print( + f"Resuming most recent session: {session_to_resume} " + "(interactive session picking is moving into the browser)" + ) else: print("No sessions found - starting fresh") elif resume_session: @@ -186,49 +443,41 @@ async def main(offline: bool = False, resume_session: str = None, show_sessions: # Log file path for launch info session_name = datetime.now().strftime("%Y%m%d") - log_file = log_dir / f"{session_name}.log" - - # Connect to device layer via hardware module's client factory - client = None - if not offline: - hw = get_hardware() - http_url = f"http://{settings.network.device_host}:{settings.network.device_port}" - if hasattr(hw, 'create_client'): - client = hw.create_client(http_url=http_url) - else: - # Fallback for hardware modules without create_client - from gently.app.queue_server_client import QueueServerClient - client = QueueServerClient(http_url=http_url) + log_file = str(log_dir / f"{session_name}.log") + + # Always build the microscope client — even when "offline" (RFC #78). The + # client is cheap and already tolerates a down device layer: is_connected + # stays False, its tools stay in Claude's schema and return clear errors. So + # "offline" means DON'T auto-connect at boot, NOT "never build the client". + # Building it unconditionally is what lets the single device-layer watcher + # attach the agent whenever hardware is started mid-session (launch gate, + # Devices panel, or a separately-run device server) — the previous + # `client = None` path dead-ended every downstream surface with no recovery. + hw = get_hardware() + http_url = f"http://{settings.network.device_host}:{settings.network.device_port}" + if hasattr(hw, "create_client"): + client = hw.create_client(http_url=http_url) + else: + # Fallback for hardware modules without create_client + from gently.app.queue_server_client import QueueServerClient + + client = QueueServerClient(http_url=http_url) + if offline: + logger.info("Launched offline — microscope client built but not auto-connecting at boot") + else: connected = await client.connect() if not connected: - # Keep client object (not None) so microscope tools remain in - # Claude's tool schema. Tools check is_connected at runtime and - # return clear error messages. Setting client = None causes all - # requires_microscope tools to vanish from the schema, which - # makes Claude hallucinate XML tool calls as plain text. logger.debug( - "Device layer not reachable at %s — microscope tools " - "available but will return errors until connected", http_url, + "Device layer not reachable at %s — microscope tools available but " + "will return errors until the layer starts and the watcher attaches", + http_url, ) - # Configure device session for zero-copy volume transfer - if client and client.is_connected: - try: - incoming = str(store.incoming_dir) - resp = await client.configure_device_session(incoming) - logger.info("Device session configured: volume_dir=%s", incoming) - except Exception as e: - logger.error("Failed to configure device session (volumes will be slow): %s", e) - - # Register auto-generated microscope tools from device layer plan schemas + # Configure the device session + register microscope tools if the client + # connected at boot (an already-running device layer). If the device layer is + # started later, _watch_device_layer attaches the agent once it's usable. if client and client.is_connected: - try: - from gently.harness.microscope import register_microscope_tools - n = register_microscope_tools(client) - if n: - logger.info("Registered %d microscope tools from device layer", n) - except Exception as e: - logger.debug("Auto-tool registration skipped: %s", e) + await _attach_microscope(client, store) # Create agent agent = MicroscopyAgent( @@ -236,12 +485,14 @@ async def main(offline: bool = False, resume_session: str = None, show_sessions: storage_path=storage_dir, session_id=session_to_resume, store=store, + no_api=no_api, ) # Generate TLS certificate for mesh communication cert_path, key_path = None, None try: from gently.mesh.tls import ensure_tls_cert, get_cert_fingerprint + _config_dir = Path(__file__).parent / "config" cert_path, key_path = ensure_tls_cert(_config_dir) except Exception: @@ -251,15 +502,20 @@ async def main(offline: bool = False, resume_session: str = None, show_sessions: # self-signed certs trigger browser "unsafe" warnings for visitors). await agent.start_viz_server(port=settings.network.viz_port) scheme = "http" - viz_url = f"{scheme}://localhost:{settings.network.viz_port}" if agent.viz_server is not None else None + viz_url = ( + f"{scheme}://localhost:{settings.network.viz_port}" + if agent.viz_server is not None + else None + ) # ── Mesh discovery ────────────────────────────────────────────── mesh = None try: + import uuid as _uuid + from gently.mesh import MeshService, register_mesh_routes from gently.mesh.audit import MeshAuditLog from gently.mesh.pairing import PairingManager - import uuid as _uuid # Persistent instance ID instance_id_path = Path(__file__).parent / "config" / "mesh_instance_id" @@ -285,6 +541,7 @@ def _capability_provider(): # GPU detection — try torch first, fall back to nvidia-smi try: import torch + if torch.cuda.is_available(): caps["has_gpu"] = True caps["gpu_name"] = torch.cuda.get_device_name(0) @@ -294,10 +551,15 @@ def _capability_provider(): except ImportError: try: import subprocess as _sp + out = _sp.check_output( - ["nvidia-smi", "--query-gpu=name,memory.total", - "--format=csv,noheader,nounits"], - timeout=5, text=True, + [ + "nvidia-smi", + "--query-gpu=name,memory.total", + "--format=csv,noheader,nounits", + ], + timeout=5, + text=True, ).strip() if out: parts = out.split(",", 1) @@ -317,6 +579,7 @@ def _capability_provider(): def _status_provider(): import gently as _gently + return { "session_id": agent.session_id or "", "acquisition_status": "idle", @@ -329,6 +592,7 @@ def _status_provider(): } import socket as _socket + config_dir = Path(__file__).parent / "config" audit_log = MeshAuditLog(config_dir) pairing_mgr = PairingManager( @@ -361,27 +625,32 @@ def _status_provider(): await mesh.start() except Exception as e: import logging as _log + _log.getLogger(__name__).warning(f"Mesh discovery failed to start: {e}") mesh = None # ── End mesh ──────────────────────────────────────────────────── # Attach the agent bridge to the viz server from gently.harness.bridge import AgentBridge + bridge = AgentBridge(agent) - bridge.set_launch_info({ - "device_connected": client.is_connected if client else False, - "sam_available": client.has_sam if client else False, - "offline": offline or (client is None) or not client.is_connected, - "store_path": str(storage_dir), - "viz_url": viz_url, - "log_path": str(log_file), - "resumed": session_to_resume is not None, - "mesh_service": mesh, - }) + bridge.set_launch_info( + { + "device_connected": client.is_connected if client else False, + "sam_available": client.has_sam if client else False, + "offline": offline or (client is None) or not client.is_connected, + "store_path": str(storage_dir), + "viz_url": viz_url, + "log_path": str(log_file), + "resumed": session_to_resume is not None, + "mesh_service": mesh, + } + ) # Initialize startup wizard (gap-driven onboarding) from gently.harness.memory.file_store import FileContextStore + agent_dir = storage_dir / "agent" context_store = FileContextStore(agent_dir) agent.set_context_store(context_store) @@ -390,34 +659,95 @@ def _status_provider(): if agent.viz_server is not None: agent.viz_server.agent_bridge = bridge agent.viz_server.set_context_store(context_store) + # Device-layer supervisor (managed child subprocess) — lets the launch + # gate and the Devices panel start/stop start_device_layer.py from the + # UI, and kills it on exit so it never orphans. It auto-detects an + # already-running (external) device layer and leaves it alone. RFC #78. + try: + from gently.app.device_supervisor import DeviceLayerSupervisor - ws_url = f"ws://localhost:{settings.network.viz_port}/ws/agent" - - # Spawn the Node.js TUI — it inherits stdin/stdout/stderr so Ink - # takes over the terminal. - tui_proc = subprocess.Popen( - ["node", str(tui_dist), "--ws-url", ws_url], - stdin=sys.stdin, - stdout=sys.stdout, - stderr=sys.stderr, + agent.viz_server.device_supervisor = DeviceLayerSupervisor( + port=settings.network.device_port, + ) + except Exception: + logger.debug("DeviceLayerSupervisor init skipped", exc_info=True) + # If launched into an existing session, rehydrate its persisted + # imagery so the galleries/filmstrips show data from the start. + if session_to_resume: + try: + agent.viz_server.rehydrate_session(session_to_resume) + except Exception: + logger.debug("Startup rehydrate failed", exc_info=True) + + # Single device-layer availability watcher (RFC #78): attaches/detaches the + # agent and announces DEVICE_LAYER_AVAILABILITY as the layer comes and goes. + # Always runs now that the client is always created — so starting/stopping the + # device layer mid-session propagates to every hardware-dependent surface. + asyncio.create_task(_watch_device_layer(agent, client, agent.viz_server, store)) + + # ── Banner + serve ────────────────────────────────────────────── + # The viz server runs in-process (uvicorn in a background task). With + # the TUI retired, the launcher's job is to keep that server alive and + # point the operator at the browser. + _print_banner( + viz_url=viz_url, + device_connected=bool(client and client.is_connected), + offline=offline, + storage_dir=storage_dir, + log_file=log_file, + resumed=session_to_resume is not None, + no_api=no_api, ) + if admin_creds: + _u, _p = admin_creds + print(" First-run admin account created — sign in at the URL above:") + print(f" username: {_u}") + print(f" password: {_p}") + print(" (Save this now. Add users via the admin API; GENTLY_NO_AUTH=1 disables auth.)\n") + + if viz_url and not no_browser: + _open_browser(viz_url) + + # Keep the event loop alive so the in-process viz server keeps serving. + # On Windows the Proactor loop won't surface Ctrl-C while blocked on a + # bare Event().wait(), so install signal handlers and poll on a short + # interval (which also lets a pending KeyboardInterrupt surface). + import signal as _signal + + _loop = asyncio.get_running_loop() + _stop = asyncio.Event() try: - # Wait for TUI to exit (blocks the event loop in a thread so - # the asyncio loop stays responsive for the viz server). - exit_code = await asyncio.get_event_loop().run_in_executor( - None, tui_proc.wait - ) - except (KeyboardInterrupt, asyncio.CancelledError): - tui_proc.terminate() + _loop.add_signal_handler(_signal.SIGINT, _stop.set) + _loop.add_signal_handler(_signal.SIGTERM, _stop.set) + except (NotImplementedError, AttributeError, RuntimeError, ValueError): + # Windows Proactor: add_signal_handler is unsupported — fall back to + # signal.signal, waking the loop via call_soon_threadsafe. + def _sig(*_a): + _loop.call_soon_threadsafe(_stop.set) + try: - tui_proc.wait(timeout=5) - except Exception: + _signal.signal(_signal.SIGINT, _sig) + _signal.signal(_signal.SIGTERM, _sig) + except (ValueError, OSError): pass + + # Graceful-shutdown hook for the desktop shell (issue #85): the viz server's + # POST /api/shutdown calls this to stop the whole backend, running the same + # finally-block teardown as Ctrl-C (thread-safe from any caller). + if agent.viz_server is not None: + agent.viz_server.request_shutdown = lambda: _loop.call_soon_threadsafe(_stop.set) + + try: + while not _stop.is_set(): + await asyncio.sleep(0.3) + except (KeyboardInterrupt, asyncio.CancelledError): + pass finally: # Suppress noisy CancelledError / overlapped IO errors from # uvicorn during shutdown on Windows. import logging as _logging + _logging.getLogger("uvicorn.error").setLevel(_logging.CRITICAL) _logging.getLogger("uvicorn").setLevel(_logging.CRITICAL) # Cleanup: stop mesh service @@ -434,39 +764,129 @@ def _status_provider(): pass +def _serve(**main_kwargs) -> None: + """Run one backend lifetime — the reload child's entry point.""" + try: + asyncio.run(main(**main_kwargs)) + except (KeyboardInterrupt, RuntimeError, SystemExit): + pass + + +def _run_with_reload(main_kwargs: dict) -> None: + """Dev auto-restart: re-run the backend whenever a gently/*.py file changes. + + Uses watchfiles (the same watcher uvicorn --reload uses) to run the server in + a child process and restart it on any .py change under gently/ (or + launch_gently.py). This is a WHOLE-backend restart, not an in-place reload — + gently's app is built from runtime state (agent, store, mesh), so there is no + static import for uvicorn's native reloader to hot-swap. A running device + layer restarts too, so this is for UI / backend dev, not live hardware. + After a restart, refresh the browser / Tauri window (Ctrl+R) to see changes. + """ + from watchfiles import PythonFilter, run_process + + root = Path(__file__).resolve().parent + paths = [str(root / "gently"), str(root / "launch_gently.py")] + print( + "[reload] watching gently/ + launch_gently.py — edit a .py file and the " + "backend restarts (then refresh the page)", + flush=True, + ) + run_process(*paths, target=_serve, kwargs=main_kwargs, watch_filter=PythonFilter()) + + def cli_main(): """Sync entry point for ``gently`` console script (pyproject.toml).""" - if not os.getenv("ANTHROPIC_API_KEY"): - print("Error: ANTHROPIC_API_KEY not set") - print("Set with: set ANTHROPIC_API_KEY=your-key") - exit(1) - parser = argparse.ArgumentParser(description="Launch Microscopy Agent") parser.add_argument("--offline", action="store_true", help="Run without server connections") + parser.add_argument( + "--no-api", + action="store_true", + help="UI-only mode: boot the web UI without any Anthropic API key. " + "Chat, perception, and plan generation are disabled.", + ) parser.add_argument("--sessions", action="store_true", help="List available sessions and exit") - parser.add_argument("--resume", nargs="?", const="__PICK__", metavar="ID", - help="Resume a session. Without ID: shows picker. With ID: resumes that session.") - parser.add_argument("-v", "--verbose", action="store_true", help="Enable verbose (INFO) logging") + parser.add_argument( + "--resume", + nargs="?", + const="__PICK__", + metavar="ID", + help="Resume a session. Without ID: shows picker. With ID: resumes that session.", + ) + parser.add_argument( + "-v", "--verbose", action="store_true", help="Enable verbose (INFO) logging" + ) parser.add_argument("--debug", action="store_true", help="Enable debug logging (most verbose)") + parser.add_argument( + "--no-auth", + action="store_true", + help="Disable accounts/login (localhost-control mode; same as GENTLY_NO_AUTH=1)", + ) + parser.add_argument( + "--no-browser", + action="store_true", + help="Do not auto-open the web UI in a browser", + ) + parser.add_argument( + "--reload", + action="store_true", + help="Dev: auto-restart the backend when a gently/*.py file changes " + "(watchfiles). Restarts the whole backend — not for live hardware.", + ) args = parser.parse_args() + # Gate the agent + hardware by the launch gate's remembered choices (the gate + # persists them; they apply at boot). CLI flags can still force these OFF. + launch_no_api = args.no_api + launch_offline = args.offline + try: + from gently.ui.web.launch_prefs import load_prefs + + _lp = load_prefs() + launch_no_api = args.no_api or not _lp.get("agent", True) + launch_offline = args.offline or not _lp.get("hardware", True) + except Exception: + pass + + # An API key is required unless running in UI-only mode. + if not launch_no_api and not os.getenv("ANTHROPIC_API_KEY"): + print("Error: ANTHROPIC_API_KEY not set") + if os.name == "nt": + print("Set with: set ANTHROPIC_API_KEY=your-key") + else: + print("Set with: export ANTHROPIC_API_KEY=your-key") + print("Or add it to a .env file in the project root: ANTHROPIC_API_KEY=your-key") + print("Or run UI-only without a key: python launch_gently.py --no-api") + exit(1) + log_level = "WARNING" if args.verbose: log_level = "INFO" if args.debug: log_level = "DEBUG" - pick_session = (args.resume == "__PICK__") + pick_session = args.resume == "__PICK__" resume_id = args.resume if args.resume and args.resume != "__PICK__" else None + main_kwargs = dict( + offline=launch_offline, + show_sessions=args.sessions, + resume_session=resume_id, + pick_session=pick_session, + log_level=log_level, + no_browser=args.no_browser, + no_api=launch_no_api, + no_auth=args.no_auth, + ) + + # Dev: auto-restart the backend on gently/*.py changes (refresh the page to + # pick them up). Runs the server in a watchfiles-managed child process. + if args.reload: + _run_with_reload(main_kwargs) + return + try: - asyncio.run(main( - offline=args.offline, - show_sessions=args.sessions, - resume_session=resume_id, - pick_session=pick_session, - log_level=log_level, - )) + asyncio.run(main(**main_kwargs)) except (KeyboardInterrupt, RuntimeError, SystemExit): pass diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 00000000..b356873f --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,18 @@ +site_name: Gently +site_description: Full-stack microscopy documentation for Gently +docs_dir: docs +nav: + - Home: index.md + - Full Stack Microscopy: full-stack-microscopy.md + - Guides: + - Try Without Hardware: guides/try-offline.md + - What Gently Can Do: guides/capabilities.md + - Build a Plugin: guides/build-a-plugin.md + - Hardware Setup: guides/hardware-setup.md + - Architecture: + - Sample and Hardware Domains: architecture/sample-hardware-domains.md + - Sample Tracking Metrics: architecture/sample-tracking-metrics.md + - Hardware Profile Template: architecture/hardware-profile-template.md +markdown_extensions: + - admonition + - tables diff --git a/notes/REALTIME_HATCHING_DETECTION.md b/notes/REALTIME_HATCHING_DETECTION.md index e4efe763..55de6f83 100644 --- a/notes/REALTIME_HATCHING_DETECTION.md +++ b/notes/REALTIME_HATCHING_DETECTION.md @@ -52,14 +52,14 @@ Edit `HATCHING_DETECTION_CONFIG` in `run_multi_embryo_volumes_with_detection.py` ```python HATCHING_DETECTION_CONFIG = { - 'enabled': True, # Enable/disable detection - 'min_timepoints_before_detection': 50, # Don't check before this (~100min at 2min/tp) - 'confidence_threshold': 'HIGH', # HIGH/MEDIUM/LOW - 'image_history_window': 10, # Recent images to send to Claude - 'stop_when_all_hatched': True, # End when all embryos hatched - 'continue_after_hatching': 5, # Confirmation timepoints after hatching - 'save_processed_images': True, # Save max projections for debugging - 'detection_log_file': 'hatching_detection_log.json' + "enabled": True, # Enable/disable detection + "min_timepoints_before_detection": 50, # Don't check before this (~100min at 2min/tp) + "confidence_threshold": "HIGH", # HIGH/MEDIUM/LOW + "image_history_window": 10, # Recent images to send to Claude + "stop_when_all_hatched": True, # End when all embryos hatched + "continue_after_hatching": 5, # Confirmation timepoints after hatching + "save_processed_images": True, # Save max projections for debugging + "detection_log_file": "hatching_detection_log.json", } ``` @@ -235,9 +235,9 @@ Reduce images sent to Claude as development progresses: if timepoint < 100: window_size = 10 # Early: more context elif timepoint < 200: - window_size = 6 # Mid: less context needed + window_size = 6 # Mid: less context needed else: - window_size = 4 # Late: minimal context + window_size = 4 # Late: minimal context ``` ### Batch Processing (Advanced) @@ -307,14 +307,16 @@ Edit `_create_detection_content` in `realtime_hatching_detector.py`: ```python # Add embryo-specific context -content.append({ - "type": "text", - "text": f""" +content.append( + { + "type": "text", + "text": f""" This is embryo #{embryo_number} from position {position}. Previous detection showed pre-hatching signs. Focus on eggshell breach in upper-right quadrant. - """ -}) + """, + } +) ``` ### Add Pre-hatching Detection @@ -362,7 +364,7 @@ annotations = load_manual_annotations() # Compare with detector results for embryo_id in annotations: - manual_tp = annotations[embryo_id]['hatching_timepoint'] + manual_tp = annotations[embryo_id]["hatching_timepoint"] detected_tp = detector.get_hatching_timepoint(embryo_id) diff = abs(manual_tp - detected_tp) if detected_tp else None print(f"{embryo_id}: Manual={manual_tp}, Detected={detected_tp}, Diff={diff}") diff --git a/notes/biologist-readiness-plan.md b/notes/biologist-readiness-plan.md new file mode 100644 index 00000000..03c25844 --- /dev/null +++ b/notes/biologist-readiness-plan.md @@ -0,0 +1,342 @@ +# Gently — Biologist-Readiness Plan + +> Engineering plan to make Gently more robust, easier for a non-programmer biologist to operate, +> and to evolve it into a multi-user, web-first microscope control system. +> Compiled from a codebase audit (architecture map, complexity audit of all >200-line files in `gently/`, +> robustness + UX review, frontend audit, startup/topology trace, and auth/multi-user ground-truth). + +**Author:** engineering analysis · **Date:** 2026-05-28 · **Horizon:** 1 focused week + a multi-sprint convergence arc + +--- + +## 0. Strategic decisions (already made) + +These are settled and shape everything below: + +1. **Frontend → converge on web-only.** The browser becomes the single surface (a floating agent chat window + the existing rich visuals). The Ink TUI becomes **legacy / maintenance-only** and is retired once the web reaches control parity. → *Do not invest in TUI refactors.* +2. **Processes → keep the two-process split, improve feedback.** The device layer (`start_device_layer.py`) stays a separate process from the agent (`launch_gently.py`) — this isolation is a safety feature, not an accident. Fix the *visibility* of its state, not the topology. +3. **Multi-user → LAN deployment, pluggable auth (no IT dependency to start).** Auth is a thin pluggable layer. Start with **Gently-managed accounts** (or shared/role tokens as an MVP) — needs nothing from institute IT. **Institute SSO (e.g. Janelia/HHMI login via a reverse proxy) is an optional later upgrade** that slots into the same layer if/when IT provides an endpoint. Gently owns the **control arbitration + roles + audit**, regardless of which login backend is used. +4. **Roles → viewers vs operators.** Anyone authenticated can **watch** (today's read-only experience, unchanged). Only **operators** can take control and drive the microscope. **Admins** can force-release and manage roles. +5. **Permission model → an explicit observable-vs-inputable classification.** Every endpoint/WS-message is tagged `observable` (read-only) or `inputable` (control). One registry drives all gating: viewer = observable set; operator-with-lock = observable + inputable. Adding a new action forces a classification; the audit log falls out of the `inputable` tag. +6. **Plan shape → balanced.** Interleave robustness/UX hardening with safe, high-value refactors. Bold-but-safe: refactor where features *won't* break; add tests *before* touching anything that might. + +--- + +## 1. Executive summary + +Gently is in **good architectural shape**. The hard parts (async acquisition state machine, hardware-safety code, the LLM loop) are well-factored. The problems that matter are **not "too complex"** — they are a handful of **silent, high-consequence failure modes**, an **opt-in/jargon UX that assumes a programmer**, and the **operational friction** of starting and using a multi-process, dual-frontend system. The web-only + multi-user direction resolves much of the friction *by construction* (e.g. it dissolves the embryo-marking hand-off and removes the Node dependency). + +**Top priorities, in order:** + +1. **Fix the verified, provable bugs** (status-tool KeyError, non-atomic writes, the silent device-down, the env-var split). Low risk, immediate value. +2. **Wire crash/restart auto-resume** — the single biggest data-loss risk; the code already exists but is never called. +3. **Harden transient-failure handling** (device hiccups, perception/Claude outages) so a brief blip doesn't silently end a run or image a dead embryo. +4. **Make state visible** — live device heartbeat, connection banner, liveness line, acquisition-settings panel, armed-rules display. +5. **Begin the web-only + multi-user arc** — browser agent chat, then the auth + single-driver control lock (the control lock must land *with* browser control, not after). + +--- + +## 2. State of the codebase — legitimate vs. accidental complexity + +Most large files are **legitimately large** (broad-but-cohesive domain modules), not tangled. Accidental complexity is concentrated and well-localized. + +### Leave alone — legitimate complexity (high feature-break risk) +- `harness/state.py` (979L) — shared mutable `EmbryoState`/`ExperimentState`. Splitting *creates* the duplication the design avoids. **Riskiest refactor target in the repo.** +- `harness/conversation.py` (774L) — core LLM loop (asend-recursion, observed-failure guards). +- `hardware/dispim/devices/*` (stage/optical/scanner/acquisition/camera/piezo) — laser/stage safety constants + MMCore vocab. +- `hardware/dispim/plans/calibration.py` (958L) — irreducible multi-phase calibration state machine. +- `core/imaging.py`, `event_bus.py`, `service.py`; `app/device_state_monitor.py`; `organisms/celegans/stages.py`. + +### Top refactor targets — accidental complexity worth fixing + +| File | Verdict | Risk | Effort | The fix | +|---|---|---|---|---| +| `app/tools/timelapse_tools.py` (815L) | REFACTORABLE | low | ~4h | Contains the confirmed KeyError bug. `@timelapse_tool` decorator kills the 6-line preamble in 17 tools; stop reaching into `orchestrator._embryo_states`. | +| `app/tools/calibration_tools.py` (1504L) | REFACTORABLE | low | ~2h | Delete ~450 lines of **dead code** (`fast_calibrate_embryo`, `hybrid_focus_selection`, `binary_edge_search`, `_fine_focus_sweep`) — unregistered, uncalled, reference nonexistent agent attrs. | +| `harness/bridge.py` (2215L) | REFACTORABLE | med | ~10h | God-object: 720-line `handle_command` if/elif ladder + case-folding bug (lowercases session/embryo IDs). Dispatch table off `CommandRegistry`. **High value for web convergence** — the browser control surface leans on this. | +| `harness/detection/verifier.py` (1158L) | REFACTORABLE | med | ~6h | `verify()`/`verify_with_context()` + two `_evaluate_consensus*` are superset/subset dupes; 5 `_run_*` + 4 `_parse_*` copy-paste. ~250 lines. **Capture consensus truth-table fixtures first.** | +| `mesh/peer_client.py` (393L) | REFACTORABLE | low | ~4h | 11 near-identical authed methods → one `_authed_json` helper (~270→~80 lines). | +| `hardware/dispim/claude_client.py` (631L) | REFACTORABLE | low | ~3h | 4 vision methods copy-paste → one `_vision_call`. | +| `harness/memory/file_store.py` (2552L) | MIXED | med | ~10h | Mixin split + shared serde. Lower priority than deleting the SQLite twin. | + +### The dominant *reduction* opportunity — ~4000 lines of dead duplicate code +The **legacy SQLite store stack** is a complete duplicate of the live file stores (CLAUDE.md says "No SQLite databases"): +- `core/store.py` (1064L) twins `core/file_store.py` +- `harness/memory/{store,_intentions,_plans,_understanding,_ml_pipelines}.py` (~2960L) twin `harness/memory/file_store.py` + +Dead in production, pinned only by ~41 tests. Delete **after** migrating tests to the `file_context_store` fixture → ~4000 lines gone, zero runtime change. **Friday work** (gated on test migration). + +--- + +## 3. Verified bugs (confirmed in source, not just inferred) + +| # | Bug | Location | Impact | +|---|---|---|---| +| V1 | `get_timelapse_status` reads `next_embryo`/`next_acquisition_in_seconds` that `to_dict()` never emits → **KeyError every call**. Same dead keys in `detection_tools.py`. | `app/tools/timelapse_tools.py:145-146,154` | Biologist's primary "is it working?" tool is broken. | +| V2 | `load_state()` fully implemented, `save_state()` runs every acquisition — but `load_state()` has **zero callers**. | `app/orchestration/timelapse.py:1643` | **No crash/restart auto-resume.** Overnight crash = whole night lost. | +| V3 | `_write_yaml` does `unlink()` then `rename()`; `save_state()` writes with no temp file. | `core/file_store.py:123-125` | **Non-atomic on Windows** — a power blip corrupts the files `/resume` needs. | +| V4 | Launcher reads `GENTLY_STORAGE`; everything else uses `GENTLY_STORAGE_PATH`. | `launch_gently.py:121` | Logs and data silently split to different paths. | +| V5 | Device-layer-down is a `logger.debug` (invisible at default log level). | `launch_gently.py:209-212` | Biologist starts with scope off, gets a normal-looking startup, discovers it mid-conversation. | +| V6 | **XSS / HTML injection** — event key/value (perception prose, paths, agent text) assigned via `innerHTML` with no escaping. | `ui/web/static/js/events.js:69-77, 130-151, 237` | Real injection surface in the events table. `escapeHtml` exists and is used elsewhere. | +| V7 | `/ws/agent` has **no connection guard/lock**; conversation state is a single shared object. | `routes/agent_ws.py:128`, `bridge.py:565`, `agent.py:759` | Latent today (TUI is sole client); **becomes live corruption the moment a browser drives the agent.** Fixed by the control lock (§9). | +| V8 | `bridge.handle_command` does `command.strip().lower()` then branches on it. | `harness/bridge.py:647,696` | Case-sensitive args (session IDs, hostnames, embryo IDs) silently corrupted. | +| V9 | Embryo marking blocks forever; `wait_for_marking(timeout=None)`; TUI never shows the viz URL or signals a browser is needed. | `ui/web/embryo_marker.py:79`, `server.py:481`, `detection_tools.py` | **Worst operational friction** — hangs if no browser is open. Dissolved by web-only convergence. | +| V10 | Marking is global shared state broadcast to all `/ws` clients; any client's `marking_done` clobbers. | `server.py:459-472`, `websocket.py:164-188` | Two browsers marking simultaneously clobber each other. Fixed by driver-only gating (§9). | + +--- + +## 4. Robustness gaps (ranked, for unattended multi-hour sessions) + +1. **[CRITICAL] No crash/restart auto-resume** (V2). `_resume_session` (`manager.py:40-117`) restores embryos+conversation but never the orchestrator or runtime fields (stop_condition, cadence_phase, next_due_at, error_count). +2. **[CRITICAL] Device hiccup permanently drops embryos.** `_acquire_embryo` (`timelapse.py:712`) treats network/timeout as terminal; 3 strikes → `complete: errors`. No auto-reconnect in `client.py`. +3. **[CRITICAL] Silent perception/detector outage.** `_run_perception`/`_run_detector` are log-only, no retry, no event. A Claude outage silently freezes stage/hatching detection **while the laser keeps firing.** +4. **[HIGH] Non-atomic writes** (V3). +5. **[HIGH] No abort path for a hung device-layer plan** — one RunEngine, no abort endpoint; one stuck acquisition freezes the wheel each round. +6. **[HIGH] Disk-full silently stops persistence** — `save_state` failures are `logger.debug`. +7. **[HIGH] Orphaned volume TIFFs** — swallowed `register_volume` failure + 300s `cleanup_incoming` race deletes valid volumes. +8. **[HIGH] Unbounded fatal exception kills the whole session** — `_run_loop` top-level except → FAILED, no per-iteration recovery. +9. **[MEDIUM]** Perception task leak / no per-call timeout (`timelapse.py:2706, 2483`). +10. **[MEDIUM]** Startup picker / `wait_for_marking` block forever. +11. **[MEDIUM]** Advisory `session.lock`, no PID check. + +--- + +## 5. Biologist usability gaps (ranked) + +1. **[CRITICAL] "Microscope not connected" is silent** (V5). → persistent banner worded as consequence+fix; live heartbeat dot; `/reconnect`. +2. **[CRITICAL] Phototoxicity protection is opt-in, silent, expert-only** — only arms if Claude is passed `monitoring_mode='expression_monitoring'`. → make it **default** for reporter/hatching experiments; agent states plainly what it armed; show armed rules in plain English. +3. **[CRITICAL] No LLM-independent emergency stop** — pause/stop are only LLM tools. → `/stop` `/pause` that call the orchestrator directly (no API round-trip). +4. **[HIGH] Silent auto-complete / auto-pause** — biologist must inspect `completion_reason`. → push plain-language notice; distinguish hardware-error (offer retry) from biological endpoint. +5. **[HIGH] No liveness reassurance.** → "last volume 0:47 ago · next in 1:13" line, yellow/red when stalled. +6. **[HIGH] Marking blocks with no browser cue** (V9). +7. **[HIGH] Cryptic launch hard-stops** (`ANTHROPIC_API_KEY not set`, "TUI not available", Node/npm). +8. **[HIGH] First-run setup landmines** — stale model IDs (`settings.py:55-58`), env-var split, raw `ModuleNotFoundError` on bad organism, README version drift (v0.11.0 vs 0.20.0). → `--doctor` preflight. +9. **[MEDIUM]** Jargon mismatch (campaign/role=test/burst/SAM/photodose). → relabel human-facing strings. +10. **[MEDIUM]** Stop-condition vocabulary mismatch ("pretzel"/"2fold" shown but rejected as targets); casing drift. +11. **[MEDIUM]** Generic error strings (raw `str(e)`/tracebacks reach the biologist). + +--- + +## 6. Frontend audit + +### Web UI (`gently/ui/web`) — the future single surface +- **Stack:** vanilla JS, no build step, FastAPI + Jinja2, Three.js for 3D. ~15k JS / 21 ` + +