diff --git a/docs/product-ideation/ELN-USAGE.md b/docs/product-ideation/ELN-USAGE.md new file mode 100644 index 00000000..d78eb7c1 --- /dev/null +++ b/docs/product-ideation/ELN-USAGE.md @@ -0,0 +1,98 @@ +# ELN — iterative usage evaluation (biologist journey) + +How the integrated lab notebook is used *over time* — accumulating embryo datasets, +annotating incrementally, and running the perception→HuggingFace flywheel across many +sessions. Produced by Fable subagents (journey / iterative stories / story-flow specs) +and synthesized. Informs the ELN build, esp. the annotation flywheel. + +## Overview + +The ELN's value to this C. elegans developmental biologist (dopaminergic-outgrowth program on the diSPIM: early ventral-nerve-cord neuron vs. the late dat-1 neuron that lights ~3-fold, with the K-pump thermo strain as a second arm) is invisible in any single session and only legible over WEEKS. In one sitting an ELN is note-taking and she stays skeptical of the model's stage calls; the payoff is a trust curve, not a feature tour. The mechanism is a double-duty flywheel: every annotation she makes corrects THIS session's record AND becomes a training label for the NEXT model (predict -> annotate-in-ELN -> accuracy Result -> HuggingFace export -> retrain -> better predictions). That loop is what converts her posture from "verify every frame myself" to "let the agent switch cadence on its own call" — and trust is earned on a measured curve (predicted-vs-ground-truth accuracy per run, rising across exports), never asserted. The emotional spine runs skepticism -> grudging verification -> pattern-recognition -> delegation. The ELN's real job is to make accumulation legible: turn a drawer of disconnected .tif stacks into a Strain->Experiment->Hypothesis->Result spine where the model gets provably better and she can watch it happen. Grounding note: phase-1 Strain store has already landed in gently/harness/memory/file_store.py:174-231 (create_strain/get_strain/list_strains/resolve_strain, bare-string->None back-compat, covered by tests/test_strain_store.py); ranged ground-truth already exists in gently/core/file_store.py:1248 (set_ground_truth with start/end_timepoint+annotator); ImagingSpec.strain/genotype/reporter are still bare strings at gently/harness/memory/model.py:209-211. The 8 Playwright seed flows (US-45..US-52) therefore all land verdict=gap in the seeded baseline by design — the value is the baseline-diff catching gap->partial->works as phases 1-4 (Strain -> Experiment -> Hypothesis/Result+scoring -> HF connector) merge. + +## Biologist journey (over time) + +| Stage | Thought | Emotion | Friction | ELN opportunity | +|---|---|---|---|---| +| **STEP 1 — First session, Monday morning (skeptic mode)** | It found the embryos and started imaging, fine — but is it actually calling the stages right or just guessing? | Wary, arms-crossed — she's been burned by autofocus tools that overpromise. | Stage labels (comma -> 1.5-fold -> 2-fold) scroll past on the live Vitals strip with no way to say 'no, that's still comma'; today the ELN is a passive teleprompter of the model's opinion and her disagreement has nowhere to land except a sticky note. | This is the empty write-path the spine names: the human can watch the mind but not co-author the record. The first session's job isn't to be trusted — it's to become correctable. | +| **STEP 2 — First annotation pass, same evening (grudging verification)** | The 3-fold call came in three frames too early — the dat-1 puncta weren't there yet; that's still late 2-fold. | Mild irritation that turns to engagement once she realizes she can actually fix it. | The transitional frames are genuinely hard — twitching motion-blurs the 2-fold body and she's squinting at a max-projection to adjudicate a boundary the model also found hard. Real cognitive load, but spent on the 6 frames that matter, not all 240. | Assisted human-in-the-loop: the agent pre-screens, bulk-proposes the long unambiguous comma stretch (one-click accept) and surfaces only the ~6 transitional frames. Her 15 minutes produce ~40 ground-truth labels via set_ground_truth(stage, timepoint, annotator='human'). She doesn't yet know they're training data — to her they're just fixing the log. | +| **STEP 3 — Tuesday, second session (the drawer starts filling)** | Same strain, but yesterday's plate sat at 20C and this one's been at 22C — will onset shift? | A flicker of purpose; two sessions feel like the start of something rather than a one-off. | Nothing links the two sessions yet — they're two YAML folders under sessions/ with no scientific relationship declared. 'These are two replicates of the same arm' lives only in her head. | The agent recognizes the strain and offers to reuse yesterday's ImagingSpec (num_slices=80, 488@10%, interval_s=180, adaptive_intervals). Annotation is FASTER because the model already agrees on the easy stretches, so her uncertain-frame queue is shorter. The Experiment entity is the missing container that would say replicate_of and group these into an analyzable design. | +| **STEP 4 — End of week, cross-session comparison (pattern recognition)** | Across five embryos, dat-1 onset clusters near 3-fold but the two 22C plates onset noticeably earlier — and I keep correcting the model the SAME way; it systematically calls 3-fold early. | The first real spark — she's seeing structure she couldn't see one session at a time. | The comparison is still eyeballed across YAML; she wants 'onset timepoint per embryo, by arm' as a table and has to assemble it by hand from predictions.jsonl + ground_truth. | She creates an Experiment with arms {20C, 22C} and back-links the week's sessions. A Result of kind:measurement (onset-timepoint per embryo) computed from the accumulated inputs turns her manual cross-tab into a first-class record with provenance back to each embryo. | +| **STEP 5 — Hypothesis crystallizes (from noticing to claiming)** | It's not noise: dat-1 onset is temperature-advanced — and separately, my correction pattern IS the model's blind spot; it conflates the dim early dat-1 blob with 3-fold body morphology. | The good kind of tension — a claim she can be wrong about. | She has two entangled claims — a biology claim (temperature) and a model-quality claim (staging accuracy) — and needs the ELN to keep them separate so one doesn't contaminate the other. | She authors a Hypothesis (statement, status:proposed, prediction bound to an Expectation) so the agent auto-checks it as replicates land. The agent, a symmetric writer, proposes a SECOND hypothesis: that its 3-fold early-calls concentrate exactly in her correction range. The Hypothesis entity consolidates the diffuse Expectation/Question/Learning triad into a falsifiable statement whose status can flip to supported/refuted. | +| **STEP 6 — Accuracy becomes visible (the model's report card)** | Okay, but HOW wrong is it, exactly? Put a number on it. | Shift from anecdote to accountability — she wants the baseline before investing in a fix. | 54% on the exact 2-fold->3-fold transition she cares most about is sobering: it gates the low->high cadence switch that decides whether she CATCHES outgrowth — a wrong 3-fold call either burns photodose early or misses the dat-1 onset window entirely. | The agent computes a Result of kind:accuracy per run (e.g. 82% overall, 54% on boundary frames); annotated-count and few-shot pool tick live as she confirms. The accuracy Result makes trust quantitative instead of vibes — the pre-improvement baseline, the denominator of the trust curve. | +| **STEP 7 — Export to HuggingFace (the corrections leave the notebook, on her terms)** | I've got ~300 human-confirmed stage labels this week — a real dataset, not a toy. | A small pride, plus a beat of caution because this is a publish, not a save. | It's a real decision with real weight — a specimen-external publish — and she wants to be sure she isn't shipping the frames she was unsure about. The approval gate is friction that's correctly placed. (If HF_TOKEN were missing the connector would disable with a clear 'set HF_TOKEN', not fail silently.) | On the Experiment she configures export={target:hf, repo:'pskeshu/gently-perception-benchmark', split, revision}; the agent bundles {projection ref, model prediction, her ground truth, provenance, strain/organism} and PROPOSES the push, which she confirms with the same posture as a hardware action. The loop goes bidirectional: the model's own live calls became her annotation queue; her corrections now become the model's next training set — without leaving the session or opening the old disconnected gently/dataset annotator. | +| **STEP 8 — Retrain and the model improves (the payoff frame)** | If those 300 labels were worth anything, the boundary accuracy should move. | Genuine anticipation — the first moment the tedious annotation nights might visibly pay back. | She has to guard against fooling herself — is it really better, or did she just annotate an easier plate? She insists on scoring against sessions the model was NOT trained on. | gently-perception retrains on the exported revision; the 2-fold->3-fold boundary climbs from 54% toward the 80s and her uncertain-frame queue on the next live session is visibly shorter. Because every Result carries method+inputs+provenance (derived_from the specific revision), the improvement is auditable — she can point to which export drove which accuracy jump, and the benchmark accumulates into a rising curve. | +| **STEP 9 — Trust crosses the delegation threshold (she stops babysitting)** | Its 3-fold call is now good enough that I'll let IT trigger the cadence switch — I don't need to sit here to catch onset. | Relief bordering on liberation; the microscope becomes a colleague instead of a suspect. | The residual risk is now tail-risk, not baseline error — a rare motion-blurred false STRONG that burns a burst on noise — so she keeps the approval posture on the expensive actions (ramp_down, stop) even as she delegates the cheap ones. | She lets the dopaminergic_signal detector's findings (intensity MEDIUM/STRONG -> speed_up; structure_quality GOOD -> burst) drive the low->high cadence handoff unattended overnight, reviewing over coffee. Delegation is granted per-decision as accuracy earns it, not all-or-nothing — the accumulated accuracy Results are what licensed her to let go. | +| **STEP 10 — Second arm, compounding returns (the K-pump temperature program)** | Twitching wrecked my hardest boundary frames all along — Richard's K-pump strain stops twitching at 25C at 3-fold, so a warm imaging window gives me paralysis-free onset frames AND lets me study how imaging perturbs hatching timing. | The confidence of someone who now trusts the loop enough to point it at a harder question. | A new confound to keep honest — does thermalization itself shift onset/hatching? She needs the Experiment's control arm and replicate structure to separate temperature biology from imaging perturbation. | She opens a second Experiment arm with the K-pump strain, thermo-cycles to a paralysis window for clean high-cadence dat-1 tracking, and logs hatching-time-distribution Results across imaged vs. unimaged embryos. The clean paralyzed frames become premium ground truth — boundary accuracy jumps again on the next export because the labels are no longer motion-corrupted. One strain's clean data raises model quality that benefits every strain; the same accuracy machinery that graded staging now grades hatching-time distributions. | +| **STEP 11 — The longitudinal payoff (the ELN as a living lab record)** | A month ago I had a drawer of .tif stacks and a model I second-guessed; now I have a spine where the model is provably better than it was and I can show exactly why. | Quiet ownership — the record is co-authored, not dictated to a chat box. | The remaining work is scientific, not clerical — the tool has stopped being the bottleneck, which is the goal. | A confirmed Hypothesis verdict (temperature-advanced dat-1 onset, status:supported across replicates) auto-posts a FINDING note into the Notebook, cross-linked to the Results, sessions, embryos and Strain records that bear it — the object a figure or the outgrowth paper actually reads. She can traverse from a claim down to the individual annotated frame that supports it. Strain->Experiment->Hypothesis->Result turns weeks of disconnected sessions into a dependency-map of what she knows and how well the instrument knows it, with a trust curve she can point to. | + +## Iterative user stories + +- As a skeptical biologist, I want to co-author the session record — correct the model's stage call in the same surface where I watch it — so that my first-week disagreement lands as data instead of a sticky note, and the record becomes correctable before it becomes trusted. +- As an annotator, I want the agent to pre-screen each run and bulk-propose the long unambiguous stretches while surfacing ONLY the ~6 transitional frames to me, so that a week of nights produces hundreds of ground-truth labels I confirmed by range rather than typed frame-by-frame. +- As a researcher running the same strain across days, I want a fresh session to recognize the strain and offer to reuse the prior ImagingSpec, so that replicates accumulate with consistent acquisition parameters and my uncertain-frame queue shrinks every session as the model learns to agree with me. +- As a researcher, I want an Experiment entity between Campaign and Session with arms{name,strain_ref,condition}, controls[] and replicate_of, so that a week of runs stops being loose YAML folders and becomes an analyzable controlled comparison (20C vs 22C, WT vs mutant, imaged vs unimaged). +- As a researcher, I want a falsifiable Hypothesis record with a status that can flip proposed->supported->refuted and predictions bound to Expectations, so that my intuition ('dat-1 onset is temperature-advanced') becomes testable infrastructure the agent auto-checks as new replicates land — kept separate from any model-quality claim. +- As a researcher, I want an accuracy Result (predicted-vs-ground-truth per run) computed from my accumulated annotations and recomputed as I confirm, so that trust becomes a measured number with a pre-improvement baseline instead of a vibe, and I can see which boundary the model is worst on. +- As a researcher, I want to configure a per-Experiment HuggingFace export to pskeshu/gently-perception-benchmark and approve each push explicitly, so that my corrections leave the notebook as a versioned, provenant dataset — on my terms, never automatically, never silently on a missing token. +- As a researcher, I want the improvement after retrain to be auditable — every Result carrying method+inputs+derived_from(revision) and scored on sessions the model was NOT trained on — so that I can prove the accuracy jump was real and point to which export drove it, instead of fooling myself with an easier plate. +- As a biologist earning trust, I want to delegate the cheap decisions (low->high cadence switch) to the agent per-decision once accuracy warrants it while keeping approval on the expensive ones (ramp_down, stop), so that I stop babysitting onset at 2am and review the record over coffee — delegation granted on a curve, not all-or-nothing. +- As a researcher opening a second arm (K-pump thermo strain), I want the flywheel I built for one program to accelerate the next — clean paralyzed frames raising model quality for every strain, the same accuracy machinery now grading hatching-time distributions — so that compounding returns, not linear effort, is what a month of iterative use buys. +- As a biologist finishing a month, I want a confirmed Hypothesis verdict to auto-post a FINDING note cross-linked down to the Results, sessions, embryos, Strain and the individual annotated frames that support it, so that the ELN is a living, traversable lab record — the object a figure reads — rather than note-taking I dictated to a chat box. + +## The little things (that compound over time) + +- RESUME POINTER: reopen a 400-timepoint embryo and jump to the first not-yet-confirmed timepoint with confirmed ranges greyed — never re-scan the 280 frames I labelled last week just to find where I stopped. +- ONLY-THE-UNCERTAIN QUEUE, DEFAULT ON: surface only the frames the model is unsure about (UNCERTAIN / is_transitional / LOW confidence / boundary neighbours) so I confirm ~12 decisive frames per embryo, not 400 obvious ones — and it must default on, not be a filter I re-enable each session. +- BATCH-CONFIRM BY RANGE: one drag-select confirm ('t120-t245 = comma') instead of 125 per-frame clicks, with the agent pre-proposing the range edges and me nudging one and accepting. +- BOUNDARY-FIRST, AUTO-FILL FORWARD: label the transition ('bean->comma at t118') and carry it forward to the next boundary, making annotation O(transitions ~7) not O(timepoints ~400) — matches the (stage,start_timepoint,end_timepoint) range model already in set_ground_truth. +- FIX ONE EDGE WITHOUT NUKING THE RANGE: move a single mis-dragged boundary frame and have the two adjacent ranges re-resolve — today the UNIQUE(session,embryo,stage) overwrite in file_store.py:1248 means a fat-finger forces a redo of the whole 100-frame stage. +- TRAINABLE-SET METER: live 'annotated / distance-to-trainable' per stage-class (target >=30 each) so I see at a glance I'm gated on 3fold(9) and hatching(6) and can stop annotating well-covered comma(55). +- CLASS-BALANCE GAP DRIVES ACQUISITION: the meter names WHICH classes are thin and rare in the wild (3fold, hatching), so I bias the next diSPIM session's cadence/window to catch under-represented stages instead of collecting more comma I don't need. +- CROSS-SESSION UNCERTAIN BACKLOG: one uncertain-frame queue spanning every session of a strain, so a dedicated annotation hour clears last week's three OH904 runs in one sitting instead of context-switching between three session folders. +- DEDUP RE-IMAGED EMBRYOS: key on embryo_uid and warn 'already annotated under session 20260615 as e3' before I re-annotate, so the trainable-set count and accuracy Result aren't silently inflated by the same embryo counted twice. +- CROSS-SESSION TIMING COMPARISON: overlay stage-transition and hatch timing across sessions of the same strain/condition on one axis, so an embryo that hatched 90 min late jumps out as an outlier instead of hiding inside a per-session view I never compare. +- THERMAL-HISTORY ON EVERY EMBRYO: attach the actual setpoint-cycle history (incl. any 25C paralysis windows) to each embryo record, so I never blind-compare a thermo-cycled embryo against a constant-20C one and misread the perturbation as biology. +- PROVENANCE ON A LABEL, MONTH LATER: click any ground-truth label and see what it was judged from — projection/trace, annotator, model version, and the calibration in effect (two-point dark/flat vs the -100/÷4 fallback) — so an uncalibrated frame doesn't get re-read as biology. +- ACCURACY UPDATES AS I ANNOTATE: the accuracy Result, annotated-count and few-shot pool recompute the moment I confirm a range, so I catch a perception regression early ('the new version is now wrong on comma frames I just fixed') instead of after a full pass. +- DON'T RE-ANNOTATE WHAT THE MODEL NOW GETS RIGHT: after a new gently-perception version lands, re-surface ONLY frames where the new prediction disagrees with my confirmed ground truth — a model upgrade costs a few re-checks, not a full re-annotation. +- INCREMENTAL, DIFF-ONLY HF EXPORT: push only annotations confirmed since the last revision and show the concrete diff (N new frames, which embryos/strains) before I approve, so a re-push is a reviewable increment and the approval gate means something. +- PER-STRAIN / PER-EXPERIMENT SLICE COUNTERS: each dataset slice (WT staging vs K-pump thermo vs dat-1 outgrowth) shows its own growing annotated count via the Strain chip, so I know the dat-1 slice is still too thin to export while the WT slice is ready. +- STRAIN CHIP REVERSE-LINKS: a Strain record links out to every session, embryo, annotation and Result that used it, so 'show me all my dat-1 embryos and their outgrowth-onset frames' is one click — every strain touchpoint is a clickable chip, not a bare string. +- DRAFT VS FINAL SLICE GATE: a half-annotated embryo yields a draft accuracy Result excluded from export and the trainable-set meter until I mark it final, so incomplete slices don't silently pollute the benchmark or make the 'ready to train' meter lie. +- CADENCE-SWITCH PROVENANCE FOR OUTGROWTH: record the exact frame that triggered the low->high cadence switch (the 'signal appeared' timepoint with the perceiver prose and MEDIUM/STRONG classification that fired it) on the embryo, so I can judge whether the switch fired too late and missed the earliest outgrowth. +- PRIORITIZE RARE HIGH-VALUE FRAMES: flag frames near predicted hatching and near dat-1 onset for priority annotation, so the rarest, most load-bearing boundaries get clean human labels first instead of drowning in abundant mid-stage frames. +- AGENT-PROPOSED VERDICTS WAIT FOR ME: the agent's pre-screened range proposals and candidate Hypothesis verdicts queue with the evidence each was based on, so I confirm them in a batch when I have attention — automation assists, it never decides or blocks me to adjudicate during acquisition. +- PHYSICALLY-IMPOSSIBLE-ORDER GUARD: a quiet warning when a confirmed label implies an impossible stage regression (pretzel before comma, hatched then bean), so a mis-drag is caught at authoring instead of surfacing as a corrupt training example after export. +- STICKY FILTER + KEYBOARD-FIRST FLOW: my per-strain/per-experiment filter and view state persist across days and a 'jump to next uncertain frame' hotkey drives the confirm/next rhythm, so resuming the daily annotation grind takes zero re-setup and my hands never leave the keyboard. + +## Playwright story-flow specs (for the baseline-diff audit) + +To add under `tools/ui_crawler/stories/` once that harness lands on the base (via #76). + +| Flow | Asserts | Mode | +|---|---|---| +| `US-52 strain-record` | PHASE-1 KEYSTONE. goto; skip_landing; probe a Strain record surface: create affordance via count_text(r'new strainadd straincreate strainregister strain'), then a form carrying the real fields name(req)/genotype/markers[]/background/source_lab/organism_ref(->gently.organisms key)/stock{thawed,frozen,notes}. Then tab('plans'), open a plan item's imaging-spec and assert ImagingSpec.strain (model.py:209) renders as a clickable .strain-chip (proposed id) not bare text, and clicking it opens the record with reverse links (sessions/notes/ground_truth). Assert-intent: a first-class Strain entity exists and every touchpoint (ImagingSpec.strain/genotype/reporter, Note.strains[] model.py:265, ground_truth, EmbryoInfo.strain) resolves to it, unrecognized-string-stays-valid. Seed: OH904 (genotype otIs355, reporter 'rab-3p::GFP pan-neuronal', markers ['rab-3p::GFP'], organism_ref c_elegans) + K-pump thermo strain + dat1 reporter. Store side already landed (create_strain/resolve_strain in file_store.py:174, tests/test_strain_store.py) so this flow audits UI only. Baseline TODAY: rec.gap('no Strain record view/chip — strain appears only as free text on ImagingSpec/Note/ground_truth; no create-strain control, no reverse links'). rec.shot('no-strain-record'). Fully headless. | headless | +| `US-46 create-experiment` | tab('plans') (experiments live in the campaign navigator/inspector — reuse Plans/Operations family, no new top-level tab); present('#navigator') && present('#canvas-content'); create = count_text(r'new experimentcreate experiment\+\s*experimentadd experiment'); open composer and assert fields title(req)/campaign_ref/an ARMS editor adding rows {name,strain_ref(strain picker -> US-52 chip),condition}/controls[] adder/replicate_of. Assert-intent: the human can author the controlled-comparison unit (arms x replicates x controls) between Campaign and Session — the empty scientific middle. Concreteness: temperature program arms '20C'(hold 20C) vs '25C-paralysis'(thermo-cycle to 25C at 3-fold for twitch-free windows); mutant-screen arms 'WT' vs 'mutant'. Baseline TODAY (mirrors US-35: no create-campaign control): rec.gap('Plans shows the campaign viewer but no create-experiment control; the Experiment entity/arms editor does not exist in the UI'). rec.shot('no-experiment-surface'). Fully headless. Enters via shared helper open_experiment(page). | headless | +| `US-47 link-session-to-arm` | open_experiment(page); within the inspector locate per-arm session attach: add = count_text(r'add sessionattach sessionlink sessionassign to arm') + a session-picker; assert it writes onto arms[].session_ids[] and the picked session shows reverse membership ('in Experiment X / arm Y' chip). Reuse note: rides existing session-link plumbing (FileContextStore.get_sessions_for_campaign / get_campaign_ids_for_session), same wiring US-07 exercised (item inspector session link/delink=1); a Session belongs to zero-or-one arm (orphan = today's behavior). Assert-intent: sessions become analyzable members of a comparison arm, not loose runs. Baseline TODAY: rec.gap('no experiment/arm surface to attach sessions to — session grouping beyond campaigns is agent-only'). rec.shot('no-arm-attach'). Headless for the affordance; NOTE fully exercising the write needs >=1 seeded Experiment + >=1 Session in dev data (seed programs provide these) — else asserts the empty-arm affordance only. | headless | +| `US-49 author-hypothesis` | open_experiment(page); create = count_text(r'new hypothesisauthor hypothesisadd hypothesisstate a hypothesis'); open composer, assert statement(req free text)/status selector with the four enum values (proposedsupportedrefutedinconclusive)/a predictions editor adding rows {target,expected,optional expectation_ref}/experiment_refs/author stamp(humanagent). Assert cross-linking: a committed hypothesis surfaces on the Experiment AND is referenced in the Notebook. Assert-intent: a falsifiable claim consolidating the diffuse Expectation/Question/Learning triad. Concreteness: temperature hypothesis 'thermo-cycling to 25C at 3-fold yields twitch-free windows WITHOUT shifting hatching-time distribution' (predictions target='twitch rate@25C' expected~0; target='median hatch time' expected='unchanged vs 20C'); dopaminergic hypothesis 'dat1 onset at ~3-fold is a reliable low->high cadence trigger capturing late-dat1 outgrowth'. Baseline TODAY: rec.gap('no hypothesis composer; claims live only as prose Learnings/Expectations with no falsifiable status'). rec.shot('no-hypothesis-composer'). Fully headless. | headless | +| `US-50 record-result` | open_experiment(page); create = count_text(r'record resultnew resultadd resultlog finding'); assert composer fields kind selector(measurementderivedaccuracy)/valuetable/method/inputs[] multi-select over sessions/embryos/predictions/ground_truth/experiment_ref/hypothesis_ref/status(draftfinal)/provenance{author,ts,derived_from}; assert incomplete-input Result stays draft (spec edge). Assert-intent: a quantitative finding distinct from a prose Learning — 'the object a figure reads' — bound to a hypothesis verdict. Concreteness: kind=accuracy 'predicted-vs-ground-truth stage accuracy for run R' (inputs=predictions.jsonl+ground_truth.yaml); kind=measurement 'hatch-time distribution per temperature arm' (median/IQR table); kind=derived 'dat1 outgrowth onset-to-stable-bright interval'. Baseline TODAY: rec.gap('no Result composer; quantitative findings have no home distinct from prose Learnings'). rec.shot('no-result-composer'). Headless for manual authoring; NOTE the assisted-scoring loop (agent computes a candidate Result when an Experiment's sessions complete AND a bound Expectation.expected_time passes / ground_truth lands, then proposes a hypothesis verdict) is agent-blocked — companion rec.blocked('needs a live agent turn + completed sessions to compute a candidate Result'). | headless | +| `US-45 annotate-ground-truth (SUPERSEDES US-31)` | goto; skip_landing; tab('embryos'); view('vitals') (US-29 confirmed #view-vitals is the per-embryo stage strip across timepoints); assert a DRAG-SELECT range mechanism on the strip (proposed id #gt-range-select or a brush overlay — drive page.mouse.down/move/up across the strip, or exists('#gt-range-select')), a range stage-assignment control applied to the selected [start_timepoint,end_timepoint] span (store already supports ranges: file_store.set_ground_truth(session_id,embryo_id,stage,start_timepoint,end_timepoint,annotator,notes) at file_store.py:1248), and an ASSISTED BATCH-CONFIRM affordance accepting a high-agreement stretch in one click while surfacing only uncertain/transitional frames (count_text(r'confirm rangeaccept proposedbatch confirmconfirm \d+ framesapply to range')). Assert an annotator+provenance stamp (which projection/trace the call was judged from). Assert-intent: the in-ELN assisted human-in-the-loop write-path — agent narrows, human confirms by range — replacing the binary I-Agree/I-Disagree (localStorage-only, non-persisted) US-31 flagged. Baseline TODAY: rec.gap('Vitals strip has no drag-select range annotation nor assisted batch-confirm; only per-card binary I Agree/I Disagree saved to localStorage, not persisted as ground truth'). rec.shot('no-range-annotate'). Headless for the affordance; NOTE the agent PROPOSAL content (pre-screened high-agreement stretches) and the stage predictions being annotated both need a live perception run — end-to-end confirm-and-persist is rig/agent-blocked (companion rec.blocked). CAVEAT for the implementer: the UNIQUE(session,embryo,stage) overwrite in set_ground_truth means a single range per stage — 'fix one edge without nuking the range' needs a segment-keyed schema before this can go partial->works. | headless | +| `US-48 configure-hf-export (SUPERSEDES US-32)` | open_experiment(page); assert an export-config panel target='hf'/repo defaulting to 'pskeshu/gently-perception-benchmark'/split/revision; a PUSH button; and critically the GATE STATE: (a) approval-gated — the push uses the same assisted-approval posture as hardware actions (require_control + explicit confirm dialog; never automatic), so assert the button demands confirmation not a bare click; (b) token-absent state — with HF_TOKEN unset the connector must DISABLE with a clear 'set HF_TOKEN' message, never silent failure (count_text(r'set HF_TOKENtoken.*requiredexport disabled') + assert push control disabled). Assert-intent detail: incremental+provenant — the panel shows what was last exported (which annotations/revision) so re-pushes are diffs. Baseline TODAY: rec.gap('no per-experiment HF export config or approval-gated push; US-32 confirmed the only exports are plan-markdown + a settings JSON dump — acquired data/predictions/ground-truth cannot be published'). rec.shot('no-hf-export'). Headless for the config form + token-absent disabled state + approval-gate presence; NOTE the actual push (datasets.push_to_hub over the network, a specimen-external publish requiring human approval + HF_TOKEN) is blocked — companion rec.blocked('needs HF_TOKEN + human approval + network; push not exercised headless'). | headless | +| `US-51 accumulate-dataset-count` | after US-45's annotation surface, on the same Vitals/Experiment surface assert a live readout trio updating at authoring (spec flywheel step 3): annotated-ground-truth count + an accuracy readout (predicted-vs-ground-truth per run, the kind=accuracy Result) + few-shot pool size — count_text(r'annotatedground.?truth countaccuracyfew.?shot\d+ frames? annotated\d+ labelled') + probe a proposed id #gt-accumulator. Assert-intent: the human sees the dataset growing and live accuracy improving as corrections land — the readout that makes the flywheel legible and motivates the HF export (US-48). Assert-intent detail: the count should equal the export candidate size (what US-48 would push) so the two surfaces agree. Baseline TODAY: rec.gap('no annotated-count / accuracy / few-shot-pool readout; nothing surfaces dataset accumulation from annotations'). rec.shot('no-accumulator'). Headless for the readout affordance; NOTE the numbers only populate after a live perception run + real annotations (seed dev data can pre-populate a static count) — the incrementing-at-authoring behavior is rig/agent-blocked (companion rec.blocked). | headless | + +## Design implications for the build + +- The flywheel is the product, not a feature. Build every surface to prove the double-duty of an annotation (fixes THIS record AND trains the NEXT model): the annotate write-path (US-45), the accuracy Result (US-50 kind=accuracy), the HF connector (US-48), and the accumulator readout (US-51) are one loop — ship them as a set, because any one alone reverts the ELN to note-taking. +- Ranged ground-truth already exists (set_ground_truth with start/end_timepoint+annotator, file_store.py:1248) so US-45's range drag-select is UI-only on top of a real store method — BUT the UNIQUE(session,embryo,stage) overwrite-by-stage is a blocker for 'fix one edge without nuking the range'. Move to a segment-keyed model (session,embryo,stage,start_timepoint) or an explicit segment id before the annotate surface can go past 'one range per stage'. +- Default the uncertain-only queue ON and make the agent pre-screen every run into (a) long high-agreement stretches for one-click range accept and (b) the ~6 transitional frames that need a human. This turns annotation from O(timepoints ~400) into O(transitions ~7) and is the single biggest lever on whether she keeps annotating past week one. +- Phase-1 Strain store has landed (create_strain/resolve_strain, bare-string->None back-compat, tests/test_strain_store.py); the remaining work is the UI: a .strain-chip that renders everywhere strain appears (ImagingSpec.strain/genotype/reporter model.py:209-211, Note.strains[] :265, ground_truth, EmbryoInfo.strain) and resolves via resolve_strain, plus reverse-links from the Strain record out to sessions/embryos/annotations/Results. Keep the unrecognized-string-stays-valid fallback so nothing forces a migration. +- The Experiment is the 'empty scientific middle' between Campaign and Session — arms{name,strain_ref,condition} x replicate_of x controls[]. Reuse the existing session-link plumbing (get_sessions_for_campaign / get_campaign_ids_for_session) for arm membership rather than inventing new wiring, and surface it inside the Plans/Operations navigator (no new top-level tab). Add the shared open_experiment(page) helper to tools/ui_crawler/stories/_harness.py since 5 of the 8 flows enter through it. +- Keep the biology claim and the model-quality claim as separate Hypothesis records with the same status enum (proposed|supported|refuted|inconclusive). The agent must be a symmetric author here — it proposes its own hypothesis ('my 3-fold early-calls cluster in her correction range') alongside hers — but proposed verdicts QUEUE with their evidence for human batch-confirm; they never auto-flip and never block acquisition. +- Trust must be a measured curve, not asserted. Every accuracy Result carries method+inputs+provenance(derived_from the exact HF revision), and the scoring convention must be held-out: score against sessions the model was NOT trained on, or the whole trust curve is self-deception. Surface the pre-improvement baseline (e.g. 54% on the 2-fold->3-fold boundary) as the denominator every future export is measured against. +- Make the flywheel legible with a live accumulator trio (annotated-count + accuracy + few-shot pool) that recomputes at authoring — and make the count EQUAL the export candidate size so US-51 and US-48 never disagree about how much data there is. +- HF export is a specimen-external publish: gate it with the same posture as a hardware action (require_control + explicit confirm dialog, never automatic), make it incremental/diff-only (push only annotations since the last revision, show the concrete diff before approval), and on a missing HF_TOKEN DISABLE the control with a clear 'set HF_TOKEN' message — never fail silently. This is the correctly-placed friction that lets her trust the loop. +- Add a draft-vs-final gate: incomplete-input Results stay draft and are excluded from both export and the trainable-set meter, so a half-annotated embryo never silently pollutes pskeshu/gently-perception-benchmark or makes the 'ready to train' meter lie. +- Provenance on every label is what makes month-old records trustable: store which projection/trace it was judged from, the annotator, the model version, and the calibration in effect (two-point dark/flat vs the -100/÷4 fallback) — an uncalibrated frame looks different and must not be re-read as biology. Dedup on embryo_uid so a re-imaged/re-aggregated embryo can't inflate counts or accuracy. +- Build the trainable-set meter as PER-STAGE-CLASS with a target (>=30 each), not a grand total — it must name that she is gated on 3fold and hatching, and that class-balance gap should feed back into acquisition planning (bias the next diSPIM session's cadence/window toward rare stages), closing the loop from annotation back to what to image next. +- Delegation is per-decision, earned on the curve: let accumulated accuracy license the agent to trigger cheap actions (low->high cadence switch on dopaminergic_signal MEDIUM/STRONG) unattended while expensive actions (ramp_down, stop) keep the approval gate. Record the exact frame that fired each delegated switch on the embryo so she can audit whether it fired too late and missed the earliest outgrowth. +- The deliverable of iterative use is legible accumulation: a confirmed Hypothesis verdict should auto-post a FINDING note into the Notebook cross-linked down to Results, sessions, embryos, Strain and the individual annotated frames — a traversable dependency-map from claim to evidence. Also attach each embryo's thermal-history (setpoint cycles incl. 25C paralysis windows) so thermo-cycled and constant-20C embryos are never compared blind — the ELN's job is to keep the confounds honest, not just to store files. \ No newline at end of file diff --git a/docs/product-ideation/PLAN-ELN-STORIES.md b/docs/product-ideation/PLAN-ELN-STORIES.md new file mode 100644 index 00000000..f87e5611 --- /dev/null +++ b/docs/product-ideation/PLAN-ELN-STORIES.md @@ -0,0 +1,212 @@ +# Combined Plan-Mode + ELN user stories — seeing the fold + +15 stories across the 4 seed programs + a longitudinal/multi-user scenario (Fable), +written to SEE how plan mode and the ELN scientific spine fold together. Companion to +the fold study (`2026-07-02-planmode-eln-fold-study.md`); grounds the fold decision. + +## What the stories reveal (synthesis) + +### Overview + +Across all five scenarios (stage-classification, temperature-hatching, mutant-screen, dopaminergic-outgrowth, longitudinal-multiuser) and fifteen stories, Plan Mode and the ELN are never two activities — they are one scientific loop observed at two moments. The same perception call is a prospective decision gate when it fires live and a retrospective ground-truth label when the human corrects it (stage-cls "Catching hatch"; dopaminergic "Spend photons only once the neuron turns on"). The same drag-select annotation feeds both the model-agnostic flywheel and the numerator of the scientific metric (temperature "Where the hatch numbers come from"). The same phase carries the hypothesis, the arms, the results, and the gate that spawns the next arm. The stories show the seam bites in exactly the places the current two-tree split forces ONE fact to be modeled twice — arm/session membership, strain identity, a detector call, a verdict — and each bite is healed by making it one node on the plan tree. Collectively they land squarely on Option C and sharpen several of the fold study's own flagged risks from cautions into concrete requirements. The recurring failure mode named in nearly every story is the bottom-appended renderElnSection blob: a Result that scored a specific arm, or a hypothesis that belongs at a phase header, floating disconnected from the node it describes. The fold's core promise, validated story after story, is that a single shared node type lets the record interleave with the plan it records so a reviewer reads phase to hypothesis to arms to results-on-arms to verdict to gate to next-arm as one document. + +### Recurring plan ↔ ELN touchpoints + +- **Phase (child Campaign, role=experiment) header on the plan tree; the plan Doc already prints a 'Hypothesis:' line it cannot currently store.** ↔ **Hypothesis node whose predictions carry expectation_ref (reference, not a copy of target/expected), authored via author_hypothesis with an agent-proposed / human-confirmed verdict via set_hypothesis_verdict.** + - → Hypothesis becomes a native node anchored at the phase header, populating the Doc line instead of living in a parallel object. Recurs in every scenario (SC 'Paired fusion staging', temperature 'The paralysis gate', mutant 'Designing the two-gate screen', dopaminergic 'Spend photons only once'). +- **The imaging/acquisition PlanItem (arm) whose session actually ran.** ↔ **Result(kind=accuracy | measurement | derived) plus confusion matrix or distribution, recorded via record_result on the item that produced it.** + - → Result attaches inline on the arm PlanItem, dissolving the bottom-appended renderElnSection blob (longitudinal '2-fold/3-fold boundary', mutant 'Running Experiment 1', 'Calling genotype from the axon-guidance phenotype', dopaminergic 'Spend photons only once'). Its anchor (embryo/session/item) must be fixed up front so the flywheel-first ordering does not need rework when Experiment is demoted (temperature 'Where the hatch numbers come from'). +- **ImagingSpec.strain/genotype/reporter on each arm's spec; unverified strings stay valid as display-only back-compat.** ↔ **Strain record (genotype, markers, CGC status) authored via create_strain / resolve_strain.** + - → Strain is a resolved reference, not a bare string — but the stories make it LIVING state: a genetics PlanItem's completion MINTS the record and rewires the spec to resolve to it (mutant 'Calling genotype...', longitudinal 'A cross that mints a strain'), and a CGC-verification gate must block unverified strings like OH-otIs355 from the shared HF push while keeping them valid inside the plan (stage-cls 'Ubiquitous vs sparse'). +- **A decision_point PlanItem that gates or branches the tree — proceed to Exp B, unblock Phase 2, or pick the next arm.** ↔ **The data-borne Hypothesis verdict (supported/refuted) that the gate should read.** + - → Distinct nodes that REFERENCE each other — the gate CONSUMES the verdict, never merges with it (temperature 'The paralysis gate', mutant 'Designing the two-gate screen' + 'Running Experiment 1', longitudinal 'When the decision gate must consume the verdict'). Sharpened: when a gate also consumed a LIVE prediction (the 3-fold cadence switch), its recorded outcome must be linkable to the Result that later scores that same call, or an execution artifact silently confounds the evidence (stage-cls 'Catching hatch'). +- **ImagingSpec detectors that emit predictions.jsonl at session run; the active_plan_item_id -> detect_embryos/start_timelapse auto-configure path is the bridge into execution.** ↔ **set_ground_truth range authoring on the Vitals stage strip — agent batch-confirm of high-agreement stretches plus human adjudication of transitional frames — stamped with annotator + the projection/trace it was judged from.** + - → One range-confirm on the session surface serves BOTH the model-agnostic perception->HF flywheel AND the scientific metric's numerator (hatch time, genotype call). The fold converges the previously separate annotator (gently/dataset) onto the live session surface so a single act feeds the model and the claim at once (temperature 'Where the hatch numbers come from', mutant 'Calling genotype...', dopaminergic 'Spend photons only once'). +- **Export target (HF repo / split / revision) configured on the experiment-phase via configure_export; the push reuses the hardware-grade require_control approval posture.** ↔ **Approval-gated push of {projection ref, prediction, ground truth, provenance, strain/organism} as an incremental, provenant diff; missing HF_TOKEN degrades to a clear disabled state.** + - → Export config hangs on the phase, but the per-strain SLICE and the which-revision-was-exported record are Result attributes — where per-slice split config attaches once arms are PlanItems (not an Experiment with its own export block) is unresolved (stage-cls 'Ubiquitous vs sparse'). A specimen-external publish must never fire automatically (temperature 'Where the hatch numbers come from', mutant 'Running Experiment 1'). +- **A new imaging PlanItem or phase spawned as a same-tree child — a round-N+1 annotation-budget reallocation, the next thermo arm, a higher-cadence bracket over the residual-error stage.** ↔ **The accuracy / distribution Result and its confirmed verdict that trigger the new item.** + - → The evidential readout AUTHORS the next executable spec — under Option C this is a same-tree edit (a Result on one PlanItem growing a sibling), which is the fold's headline win, but no Result->PlanItem authoring tool is defined yet (stage-cls 'Ubiquitous vs sparse', longitudinal '2-fold/3-fold boundary', temperature 'Constant-25 vs cycled vs control'). +- **A non-imaging genetics PlanItem ('cross X -> homozygous line, ~10 days') whose depends_on edge gates the downstream imaging arms and phase.** ↔ **A Strain record minted or advanced from 'in construction' to real when the cross completes; the depends_on imaging PlanItem's ImagingSpec.strain rewired to resolve to it.** + - → A plan-outcome EVENT reaches across into the ELN Strain store; strain-in-construction is one fact that today lives twice (a blocked PlanItem AND a Strain yaml) and must be single-sourced (mutant 'Calling genotype...', longitudinal 'A cross that mints a strain', dopaminergic 'Which neuron comes up first' + 'Stop the twitch to catch the tip'). +- **Sibling control PlanItems — a 5-s BF hatch witness, a stage-matched-scoring control, a registration-error floor, a thermal-only developmental control.** ↔ **Their output is ground truth or a denominator for a DIFFERENT arm's Result; spanning/ordering Results attach to the phase that contains the arms.** + - → A plan node's product is an ELN input for another node — cross-arm provenance crossing the seam (temperature witness in 'Catching hatch'/'Constant-25...', stage-matched scoring in mutant 'Calling genotype...', common developmental clock in dopaminergic 'Which neuron comes up first'). The claim and the acquisition spec are inseparable, which is precisely why one shared tree beats two — but a Result that spans two CAMPAIGNS (dopaminergic 'Stop the twitch' reading the temperature program's hatching distributions) still has no home. + +### Fold insights + +- The fold is not prospective-versus-retrospective; it is one perception call wearing two hats. The load-bearing object in three scenarios — a stage/hatch/onset detector call — is simultaneously a prospective decision_point that reconfigures hardware live AND the retrospective ground_truth whose accuracy IS the Result. Two trees force the human to re-annotate by hand exactly what the detector already decided, and 'did the trigger fire correctly?' cannot be read off one place. Making it one node is the deepest justification for the fold (stage-cls 'Catching hatch', dopaminergic 'Spend photons only once', mutant 'Calling genotype...'). +- The loop closes across the seam: a Result must SPAWN the next PlanItem. The evidential accuracy/distribution readout is the trigger for a new executable spec. Under Option C this is a same-tree edit, which is the win — but it is also the one authoring path with no defined tool, so the fold is incomplete until Result->PlanItem authoring exists (stage-cls 'Ubiquitous vs sparse', longitudinal '2-fold/3-fold boundary', temperature 'Constant-25 vs cycled vs control'). +- Gate-consumes-verdict is the single most recurrent touchpoint and it confirms the fold study's step-6 'relate, do not merge.' Every scenario has a decision_point that must consume a Hypothesis verdict to branch. The gate is a plan node, the verdict an ELN node; they reference. The stories add a sharper requirement than the study stated: when the gate already consumed a live prediction, the gate outcome must be traceably linkable to the Result scoring that same call so a late-firing gate shows up as a confound rather than hiding inside the number (temperature 'The paralysis gate', mutant, longitudinal 'When the decision gate must consume the verdict', stage-cls 'Catching hatch'). +- One annotation, two consumers. A single set_ground_truth range-confirm feeds both the model-agnostic perception->HF flywheel (needs only Embryo/Prediction/ground_truth) AND the numerator of the ELN's scientific metric. This validates the study's flywheel-first sequencing precisely because the annotation act is shared and the flywheel is independent of Strain/Experiment (temperature 'Where the hatch numbers come from', all scenarios). +- 'Arms ARE imaging PlanItems' holds for the branching designs the study said to verify first — the two-gate mutant screen, the three thermo arms, the cross-strain ordering — but the paired within-embryo design breaks the assumed 1:1 arm-to-session mapping. Three arms (bf-only/nuclear-only/fusion) are ONE acquisition scored three ways: one session feeds N arms. So 'arm' and 'acquisition' must decouple, and session membership must be many-arms-to-one-session, not the arm.session_ids-model-runs assumption (stage-cls 'Paired fusion staging'). +- Strain is a shared lifecycle state, not a static reference. A genetics PlanItem's completion must mint the Strain the spec resolves to; strain-in-construction is one reality currently duplicated as a blocked PlanItem and a Strain record that can silently disagree; and a CGC-verification gate straddles the plan field and the HF export (valid in-plan, blocked from the shared benchmark). The fold must own this lifecycle, not just resolve a name (mutant 'Calling genotype...', longitudinal 'A cross that mints a strain', dopaminergic 'Stop the twitch'). +- Two clocks must be stitched. Plan history advances on a spec-VERSION axis; the ELN advances on an annotation-ROUND axis. Round-over-round accuracy only makes sense interleaved with the ImagingSpec version that produced each round — impossible until Result renders inline on the PlanItem. And ground_truth's single-annotator stamp is inadequate: two annotators systematically disagree at the hard boundaries and behavior adjudicates, so the fold needs a per-annotator stamp plus an adjudication node (longitudinal all three stories). +- Controls and comparisons are intrinsically cross-node: a witness item's only product is ground truth for another arm's Result; a stage-matched-scoring control is what makes a hypothesis defensible; an ordering Result spans two arms; a developmental-time Result reads another campaign's distributions. A single shared tree hangs spanning Results on the phase and makes control->arm provenance legible — but cross-campaign Results remain homeless in any single tree (dopaminergic 'Which neuron comes up first' + 'Stop the twitch', temperature, mutant). + +### Does it confirm Option C? + +The stories overwhelmingly CONFIRM Option C's direction and its explicit risk register, then REFINE it in about five concrete places — all extensions or sharpenings, never contradictions, and none argue for A/B/D/E. CONFIRMED: (1) Experiment demoted to a phase-role over imaging PlanItems works for exactly the case the study named as the make-or-break test — the mutant screen — where 'Designing the two-gate screen' shows arms x strain x condition + sibling controls + a cross-dependency all carried by PlanItems, and 'The paralysis gate' and 'Constant-25 vs cycled vs control' do the same for 3-4 thermo arms. (2) Strain/Hypothesis/Result are kept and additive: every story authors a Hypothesis at a phase header (the Doc line the plan already prints but cannot store), a Result on the arm, and a resolved Strain on the spec. (3) Gate-consumes-verdict as distinct-but-referencing nodes is validated as the most recurrent seam (step 6). (4) Flywheel-first with a fixed Result anchor is confirmed and its stated risk is echoed verbatim by 'Where the hatch numbers come from,' which flags that the accuracy Result must ship on a stable embryo/session anchor before the Experiment demotion moves anything. (5) The bottom-appended renderElnSection blob is named as the pain in nearly every story, confirming node-anchored inline rendering as the fix. REFINEMENTS that go beyond the study: (a) The paired within-embryo design ('Paired fusion staging') genuinely CHALLENGES the assumption that arm.session_ids model runs 1:1 — one session feeds three arms, so the fold must support spec-inheriting scoring conditions that never touch hardware and many-arms-to-one-session membership; this is the study's own PlanItem-overload risk made concrete. (b) The Result->next-PlanItem authoring path (the closing loop) is required by 'Ubiquitous vs sparse' and 'When the decision gate must consume the verdict' but has no defined tool. (c) Strain must be lifecycle state minted by a genetics PlanItem's completion, not merely a resolved reference ('A cross that mints a strain'). (d) Multi-annotator ground_truth with an adjudication node is required by the longitudinal stories and is absent from the study. (e) Cross-campaign spanning Results ('Stop the twitch') have no home even under a single tree. Net: the stories ratify Option C as the right fold and hand it a punch list of five under-specified edges. + +### The unified experience the stories point to + +One tree, read top to bottom as a single living document. In /plan the researcher (or the agent, using the same tool surface) designs a Campaign; a controlled comparison is a phase with role=experiment whose arms ARE imaging PlanItems carrying strain and condition. The agent authors Hypotheses at the phase header (finally populating the Doc's Hypothesis line), resolves each Strain onto its ImagingSpec, drops sibling control PlanItems, wires depends_on edges for genetic crosses and phase gates, and hangs the HF export config on the phase. On approval the SAME tree is executable: active_plan_item_id auto-configures the ImagingSpec to hardware — detect_embryos, start_timelapse, ACUITYnano setpoints, adaptive cadence — and live detector calls fire the decision_point gates. Back in the notebook, the Vitals stage strip is the same session surface where the human authors ground_truth: the agent batch-confirms the high-agreement stretches, the human adjudicates only the transitional frames, and that ONE range-confirm simultaneously feeds the perception->HF flywheel and the scientific metric. Results compute and render INLINE on the PlanItem that produced them; the agent proposes a verdict the human confirms; that verdict RELEASES the decision_point gate, which unblocks or spawns the next PlanItem — the evidence writing the next experiment in the same tree. A genetics item completing mints the Strain its downstream spec resolves to and unblocks the arms that waited on the cross. An approval-gated HF push (hardware-grade confirm, disabled-not-silent when the token is missing) exports the corrected rows with full provenance as an incremental diff. There is no bottom-appended ELN blob, no re-annotating what the detector already decided, no strain string orphaned from its record, and no verdict the human has to re-decide because the gate reads it directly. A reviewer opening the phase sees, in one place: hypothesis, arms with their specs and controls, results on the arms, the confirmed verdict, the gate it released, and the next arm it spawned. + +### Open questions (the punch list) + +- Paired within-embryo designs: are inheriting scoring conditions (bf-only, nuclear-only) modeled as imaging PlanItems that never touch hardware, or as Result variants on the single acquisition item? Either way the fold needs many-arms-to-one-session membership, which breaks the arm.session_ids-model-runs 1:1 assumption (stage-cls 'Paired fusion staging'). +- What is the Result->PlanItem authoring tool/path that closes the loop, letting a confirmed accuracy/distribution Result spawn the next arm as a same-tree child (stage-cls 'Ubiquitous vs sparse', temperature 'Constant-25 vs cycled vs control')? +- Where does per-strain / per-slice export split and revision config attach once arms are PlanItems rather than an Experiment with its own export block — and how does a Result carry which revision/annotations it exported so the next HF push is a provenant diff (stage-cls 'Ubiquitous vs sparse')? +- How is multi-annotator ground_truth modeled — a per-annotator stamp plus an adjudication node that records 'Keshu vs Ana/Hari, resolved by post-hatch behavior' — given that the current single-annotator stamp would overwrite or drop the second scorer's labels at exactly the hard boundaries (longitudinal '2-fold/3-fold boundary', 'A cross that mints a strain')? +- Where does a Result that spans two CAMPAIGNS live — e.g. a developmental-time Result that reads another campaign's hatching-time distributions — since even a single tree only hangs spanning Results on a phase within one campaign (dopaminergic 'Stop the twitch to catch the tip')? +- How is the gate/Result confound made visible: linking a decision_point's consumed prediction to the Result that later scores the same call, so a late-firing gate surfaces as an execution artifact instead of hiding inside the metric (stage-cls 'Catching hatch')? +- What event mints a Strain when a genetics PlanItem completes, and how is strain-in-construction state single-sourced between the blocked PlanItem and the Strain record so the two cannot silently disagree? Where exactly does the CGC-verification gate sit between the plan's strain field and the HF export (mutant 'Calling genotype...', longitudinal 'A cross that mints a strain', dopaminergic 'Stop the twitch')? +- How does the Expectation/ground_truth plumbing accept a DOWNSTREAM behavioral label (post-hatch Unc locomotion) bound back to an embryo imaged hours earlier — a provenance chain from the timelapse track end (hatch) to the larva (mutant 'Calling genotype...')? +- How does the phase-role view render 'these N sessions = R replicates of A arms + controls' at least as clearly as the retired Experiment.arms did, so demoting Experiment is not perceived as a regression (temperature 'Constant-25 vs cycled vs control')? +- Migration: link_session_to_arm today does not update the campaign link, so collapsing to the single link_session_campaign source of truth risks orphaning arm->session links already written by the seeds — how is the existing arm-link data migrated in the same change that drops arms[].session_ids[]? + +--- +## The stories + +### Scenario: stage-classification: plan the nuclear+BF fusion staging run, execute it on the diSPIM, annotate ground truth on the timelapse, score accuracy, push the flywheel to HuggingFace, and let the accuracy result inform the next plan item — traversed across both the prospective Plan Mode layer (Campaign→PlanItem task graph, ImagingSpec→hardware) and the retrospective ELN spine (Strain/Hypothesis/Result + perception→ground_truth→accuracy→HF flywheel), under the Option-C fold where Plan Mode absorbs the ELN as native nodes and Experiment is demoted to a phase-role over imaging PlanItems. + +#### Paired fusion staging: one acquisition, three arms, one push +*Keshu, dogfooding program #1 on the diSPIM — the researcher who both authors the plan and corrects the ground truth.* + +In plan mode Keshu says "set up the fusion-vs-single-modality comparison on the lineaging strain." The agent proposes a phase (child Campaign, role=experiment) 'Fusion vs single-modality (within-embryo paired)' under the stage-classification campaign, with three arms — bf-only, nuclear-only, fusion — on RW10226. But it flags the paired design: the three arms read the SAME volumes, so it commits ONE acquisition PlanItem (BF max-projection + 488nm his-72::GFP nuclear channel, 80 slices, 10-min interval, comma→1.5-fold→2-fold→3-fold→pretzel, 5 embryos) and marks bf-only and nuclear-only as scoring conditions that inherit that acquisition rather than re-imaging. It authors Hypothesis 1 at the phase header ('fusion > nuclear-only > bf-only, gain concentrated at the 2-fold↔3-fold and 3-fold↔pretzel boundaries where the BF outline saturates'). Keshu approves; the plan commits. At the bench the agent auto-configures from the ImagingSpec (active_plan_item_id → detect_embryos → start_timelapse) and runs the timelapse; gently-perception emits per-timepoint stage predictions on the BF, nuclear, and fused projections. Back in the notebook, on the Vitals stage strip, the agent pre-screens: it bulk-proposes the long comma and pretzel stretches as one-range confirms and surfaces only the 2-fold/3-fold transitional frames for Keshu to decide by drag-select; each correction persists via set_ground_truth stamped with annotator + the projection it was judged from. Three Result(kind=accuracy) records compute — top-1 accuracy plus a confusion matrix per arm — and confirm fusion collapses the off-diagonal 2-fold/3-fold mass that bf-only cannot. The agent proposes a 'supported' verdict on Hypothesis 1; Keshu confirms. He approval-gates the push of the corrected {projection ref, prediction, ground truth, provenance, strain} rows to pskeshu/gently-perception-benchmark. The residual pretzel-boundary error in the fusion confusion matrix seeds the next plan item. + +- **Plan touch:** Campaign 'stage-classification' → phase (child Campaign, role=experiment) whose arms are imaging PlanItems; ONE acquisition PlanItem carrying the ImagingSpec (strain=RW10226, BF + 488nm nuclear, 80 slices, 10-min interval, start_stage=comma, stop_condition=pretzel, num_embryos=5); execution via active_plan_item_id → detect_embryos/start_timelapse; the other two arms as spec-inheriting scoring conditions. +- **ELN touch:** Hypothesis authored at the phase header (Option-C node-anchored, populating the plan Doc's 'Hypothesis:' line it previously could not store); ground_truth range-authoring on the Vitals stage strip (agent batch-confirm of high-agreement stretches + human transitional frames) via set_ground_truth; three Result(kind=accuracy) with confusion matrices attached to the acquisition PlanItem; agent-proposed/human-confirmed Hypothesis verdict; approval-gated HF push to pskeshu/gently-perception-benchmark; Strain RW10226 resolved on ImagingSpec.strain. +- **Seam:** The within-embryo PAIRED design breaks the fold's 'arms ARE imaging PlanItems' assumption. The three arms are not three acquisitions — they are ONE session scored three ways. So Plan Mode wants three PlanItems, the hardware runs once, and the ELN wants three Results over one session. Where do bf-only and nuclear-only live — as imaging PlanItems whose spec inherits but never touches hardware, or as Result variants on the single acquisition item? The fold study assumed an arm's session_ids[] model its runs (1:1), but here one run feeds three arms (1:N), so session membership is many-arms-to-one-session. This is exactly the risk the study flagged: verify imaging PlanItems truly carry arms×condition before dropping Experiment.arms, or the paired-comparison structure is lost. + +#### Catching hatch: the cadence gate that becomes the score +*A postdoc in Hari Shroff's group running the hatching-detection experiment, where real-time cadence control and after-the-fact scoring collide.* + +In plan mode the postdoc plans the adaptive-high-fusion arm on RW10226: monitor at 10-min baseline until the 3-fold stage is detected, then switch to 30-s cadence with BF+nuclear fusion until hatch is called, where hatch = collapse of the ordered nuclear lattice plus the larva translating out of the eggshell. The agent encodes adaptive_intervals ({baseline: 600, after_3fold: 30}), detectors=[3-fold, hatch], and a decision_point 'if 3-fold detected → switch to 30-s cadence'. It adds two sibling control PlanItems: a 5-s BF-only over-sampled N2 witness through the peri-hatch window (to establish the true hatch timestamp with minimal photodose) and a near-unmonitored endpoint-only photodose control. It authors Hypothesis 2 at the phase ('hatch is recoverable only above a cadence threshold; the adaptive switch localizes it to ±1 frame while fixed 10-min cannot; the nuclear lattice flags hatch 1-2 frames before BF larval translation'). Execution: mid-run the live 3-fold detector fires the cadence switch straight off the ImagingSpec, and the 30-s capture localizes the fast stochastic hatch. Afterward, in the ELN, the postdoc annotates the witness's true hatch frame as ground_truth on the Vitals strip; the agent computes Result(kind=accuracy) = |predicted hatch frame − witness time| per arm and a second Result that the lattice collapse preceded BF translation. Hypothesis 2 gets an agent-proposed verdict the postdoc confirms, and the hatching split is pushed to HF. The photodose-control Result (did the 30-s load delay hatch versus the endpoint control?) informs whether the next plan item drops laser power. + +- **Plan touch:** ImagingSpec with adaptive_intervals + detectors=[3-fold, hatch] + a decision_point gate ('3-fold → 30-s switch') driving the real-time cadence controller during execution; sibling control PlanItems — the 5-s BF-only N2 hatch witness and the endpoint-only photodose control. +- **ELN touch:** ground_truth hatch-frame annotation on the witness timeline; Result(kind=accuracy) hatch-timing error per arm (adaptive-high-fusion vs fixed-low vs adaptive-high-bf); a derived Result for the nuclear-lead-over-BF cue; Hypothesis 2 verdict; approval-gated HF push of the hatching split. +- **Seam:** The SAME perception event lives in both layers with opposite roles. Prospectively the 3-fold call is a decision_point gate that fired the cadence switch during execution (Plan Mode); retrospectively the human corrects that same 3-fold/hatch call as ground_truth and its accuracy IS the Result (ELN). Two frictions. (1) The fold study insists decision_point gates and Hypothesis verdicts stay distinct-but-related — here the gate CONSUMED a prediction whose accuracy the Result later measures, so if the gate fired late (a staging error) the hatch-timing Result is confounded by the plan's own execution artifact: the prospective control decision contaminates the retrospective evidence. The gate outcome recorded on the PlanItem must be linkable to the Result scoring the same call. (2) The 5-s witness is a separate acquisition PlanItem whose only output is ground truth for a DIFFERENT arm's Result — a plan node whose product is an ELN input for another node, i.e. cross-arm provenance crossing the plan/ELN seam. + +#### Ubiquitous vs sparse: the accuracy result that writes the next plan item +*Keshu closing round N of the annotation flywheel and deciding round N+1 — the seam where evidence must spawn the next executable spec.* + +Keshu opens the generalization phase (Experiment 3): compare the dense whole-embryo nuclear lattice of RW10226 (his-72::GFP, 488nm) against the sparse pan-neuronal pattern of otIs355 (rab-3p::2xNLS::TagRFP, 561nm), matched geometry and cadence, equal annotation budget. Before commit, plan mode resolves the Strain references and the agent surfaces the data-quality flags baked into the seed program: RW10226's zuIs178/stIs10024 transgene strings and otIs355's placeholder CGC designation (OH-otIs355) must be verified against CGC before the first benchmark push. Keshu confirms/edits the Strain records. The agent commits two imaging PlanItems (ubiquitous, sparse) inheriting one matched acquisition spec, with the HF export target configured on the phase, and authors Hypothesis 3 ('at equal annotation budget, accuracy on sparse-neuronal nuclei is lower than on ubiquitous — a nuclear-density dependence — AND round-over-round accuracy rises as corrected frames accumulate'). Both arms execute. On the Vitals strip Keshu annotates each strain independently; the agent's high-agreement pre-screen matches worse on the sparse pattern, so more transitional frames fall to him. Two Result(kind=accuracy) records land as per-strain slices; the sparse slice lags the ubiquitous by a measurable margin, and the round-N accuracy sits above round N−1 as the few-shot pool grows. Keshu approval-gates the per-strain HF push with split/revision, recording exactly which annotations/revision were exported so the next push is a diff. Then the payoff: the sparse-lags-ubiquitous Result informs the next plan item — the agent proposes a round-N+1 imaging PlanItem that reallocates annotation budget toward the sparse strain (or adds embryos). Keshu approves; the plan tree grows a child directly from the finding. + +- **Plan touch:** Phase (role=experiment) with two imaging PlanItems as arms; Strain resolution on ImagingSpec.strain/genotype/reporter (RW10226 488nm vs otIs355 561nm); one matched inherited acquisition spec (geometry/cadence, equal annotation budget); HF export config (repo/split/revision) hung on the phase; the NEW round-N+1 PlanItem spawned from the Result. +- **ELN touch:** Strain records (genotype, markers, CGC-verification flags before first push); two per-strain Result(kind=accuracy) slices; Hypothesis 3 covering both the density gap and round-over-round convergence; incremental provenant per-strain HF split/revision push. +- **Seam:** This closes the loop across the prospective/retrospective boundary: a Result (retrospective) must SPAWN the next imaging PlanItem (prospective). In Option-C's single tree that is a same-tree edit — a Result on one PlanItem informing a sibling — which is the fold's win, but the loop still crosses layers: the evidential accuracy readout is the trigger for a new executable spec, and nothing yet defines that Result→PlanItem authoring path. Three frictions. (1) Export split/revision config lives on the phase (experiment-role), but the per-strain SLICE is a Result attribute — where does per-strain split config attach once arms are PlanItems, not an Experiment object with its own export block? (2) Round-over-round convergence requires each push to be an incremental provenant diff, so the Result must carry which revision/annotations it exported — a Result field the plan's export config depends on. (3) Strain data-quality is a gate between the plan's strain field and the benchmark export: an unverified string (OH-otIs355) is valid for the plan as display-only back-compat, but must NOT reach the shared HF benchmark push — a verification gate straddling plan field and ELN export. + +### Scenario: temperature-hatching — Richard Ikegami's K-pump thermo strain (RK-KP1: twk-18(cn110gf); his-72p::HIS-24::mCherry) that stops embryonic twitching at the restrictive 25 C from the 3-fold stage, turning the diSPIM's ACUITYnano temperature setpoint into a scientific instrument. The campaign quantifies the DISTRIBUTION of C. elegans hatching times and how the two moves the microscope makes on that setpoint — thermo-cycling to 25 C for paralysis-free windows, and the photodose of imaging itself — perturb it. These stories cover: (Exp A) certifying paralysis is real, K-channel-specific, and reversible before any distribution claim rests on it; (Exp C) planning constant-25 vs thermo-cycled vs constant-20 control arms, executing them on the setpoint, forming and scoring the Pareto / distribution-shift hypothesis, and deciding the next arm; and the hatch-detection perception -> ground-truth -> HuggingFace flywheel that underwrites the hatch-time numbers the ELN records. Every story is read against the Option C fold: Plan Mode is the single design substrate, an Experiment is demoted to a phase-role (child Campaign, role=experiment) whose arms ARE imaging PlanItems carrying strain/condition, and Strain/Hypothesis/Result attach as native nodes authored by the same plan tools. + +#### The paralysis gate: proving the setpoint is an instrument before trusting a single hatch time +*Richard Ikegami — creator of RK-KP1; needs the K-pump paralysis certified as real, mutant-specific, and reversible before he will let any hatching-distribution claim rest on the thermo-cycling protocol.* + +Richard says 'help me prove 25 C actually silences twitching in my strain, and that it is the channel and not just heat.' In plan mode the agent designs Exp A as its own phase: three imaging arms — KP restrictive (RK-KP1 held at constant 25 C), KP permissive (RK-KP1 at 20 C), WT restrictive (WT-HIS at 25 C) — plus a reversibility-control PlanItem (hold RK-KP1 at 25 C paralyzed, ramp back to 20 C, confirm twitching resumes within one thermal time-constant), n>=8 embryos per arm, motion index blinded to arm during scoring. Each arm's ImagingSpec is a 10-min 3-fold motion-assay window: bright-field + 488 nm nuclear frames every 30 s. Richard runs each arm; the agent auto-loads the spec, drives the ACUITYnano ramp/stabilize/hold and, for the control, the reversal. After scoring, motion index per arm becomes a Result; the agent scores the premise Hypothesis ('25 C abolishes twitching from 3-fold, K-channel-gated and reversible; WT twitches at both temperatures') to 'supported' — RK-KP1 motion drops >=90% vs its own 20 C value, WT-25 is indistinguishable from WT-20, twitching resumes on cooling. That verdict is what a Phase-A decision gate consumes to unlock Exp B and Exp C. If the strain still twitched at 25 C, or WT went silent, or paralysis did not reverse, the gate blocks the whole distribution campaign — no point measuring hatch-time shifts of a protocol whose premise failed. + +- **Plan touch:** A child Campaign with role=experiment (the demoted Experiment) holds the three motion-assay arms as imaging PlanItems, each carrying an arm/condition field and an ImagingSpec whose strain is a resolved reference (RK-KP1 vs WT-HIS) and whose temperature_c encodes 25 vs 20; the reversibility control is a sibling imaging PlanItem with a ramp-then-reverse condition. The ImagingSpec -> ACUITYnano setpoint execution path runs the ramp/hold/reverse on hardware. A decision_point PlanItem depends_on all Exp A arms and gates Exp B/C. +- **ELN touch:** The Strain records RK-KP1 and WT-HIS (genotype, markers, twk-18 status) are authored via create_strain and resolved onto each arm's ImagingSpec.strain. A Result (kind=measurement, the per-arm motion index, blinded provenance) is recorded via record_result on the PlanItem that produced it. The premise Hypothesis is authored via author_hypothesis with predictions carrying expectation_ref, and set_hypothesis_verdict marks it supported/refuted. +- **Seam:** The touchpoint is the Phase-A decision_point gate reading the Hypothesis verdict: plan mode's gate (proceed/block Exp B-C) and the ELN's data-borne verdict (supported/refuted) are two nodes that must reference — not merge into — each other, exactly the conflation the fold study flags in step 6. Friction under today's two-tree split: the decision gate lives in the plan tree and the verdict lives in the parallel ELN object with zero agent tools, so nothing can make the gate actually consume the verdict — Richard would eyeball the bottom-appended renderElnSection() blob and hand-flip the gate. Second seam: the motion-index Result attaches to a PlanItem (arm) while the reversibility evidence is a control PlanItem's outcome; the fold must let a Result anchor to the arm that produced it, which the parallel Experiment.arms[] cannot express as a plan node. + +#### Constant-25 vs cycled vs control: running the three arms and letting the distribution pick the next one +*Keshu — running the microscope on Richard's behalf, wanting the agent to plan the three thermo protocols, execute them on the setpoint, and tell him from the hatch-time distributions which arm to expand next.* + +This is the core scenario. In plan mode the agent lays out Exp C as a role=experiment phase with three arms as imaging PlanItems: Constant-25 imaged (RK-KP1 held at 25 C, continuous paralysis, ~10 h to hatch, high-cadence 3D through 3-fold->hatch), Thermo-cycled imaged (base 20 C, brief 25 C windows for imaging only, ~14 h development), and Constant-20 unimaged baseline (bright-field hatch detection only, the shape reference). n>=20 embryos per arm for distribution statistics; a WT-HIS constant-25 unimaged control arm to separate heat effects from K-channel-gating load; and a matched-cumulative-photodose constraint between the two imaged arms so the contrast isolates temperature protocol, not light. The two imaged arms inherit_from one fully-specified ImagingSpec (~1-min interval, 488 nm nuclear) and override only the temperature schedule. Keshu runs sessions; the agent auto-loads each arm's spec and drives the ACUITYnano — a flat 25 C hold for one arm, a 20<->25 C cycle (ramp to silence twitching, image, return to 20 C) for the other — detecting hatch by bright-field eggshell breach and timing it as minutes from the nuclear 3-fold-onset landmark. As the ~20 sessions per arm accrue, the agent computes the hatch-time distribution per arm (median, CV, 90th-percentile tail, arrest fraction) and, after regressing out the trivial ~10 h vs ~14 h developmental-rate shift, scores the Pareto Hypothesis: intermittent cycling perturbs the distribution SHAPE less than constant-25 at equal image quality. A decision_point then asks which arm to run next — expand thermo-cycled, or add the WT-25 heat-only control to attribute the constant-25 penalty to channel-gating load — and the choice is driven by the recorded Result. + +- **Plan touch:** The role=experiment phase's arms are imaging PlanItems with an arm/condition field (constant-25 / cycled / constant-20-control), inherit_from spec sharing plus per-arm temperature-schedule overrides, a matched-photodose constraint expressed across the two imaged items, and comparison_to encoding the arm relation. The ImagingSpec -> ACUITYnano setpoint path executes the constant-hold vs 20<->25 cycle schedule on hardware. A decision_point PlanItem ('which arm next?') depends_on the arms and picks the next PlanItem to run. +- **ELN touch:** Per-arm hatch-time distribution recorded via record_result (kind=derived: median, CV, 90th-pct tail, arrest fraction, hatch-success rate, mean motion-blur), with inputs[] listing the arm's member sessions/embryos. The Pareto Hypothesis authored and its verdict set from those Results; export config (repo pskeshu/gently-hatching-time) hung on the phase via configure_export. +- **Seam:** The load-bearing touchpoint is the 'decide next arm' gate: the plan's decision_point selects the next imaging PlanItem, but the selection is a function of the ELN Result/verdict — plan (what to run next) and ELN (what the data says) meet at one node. Friction under the two-tree split: 'these 20 sessions = one arm's replicates, and there are three arms' is exactly the readout the parallel Experiment gave via arms[].session_ids[]; the fold study warns that demoting Experiment must render arms/replicates/controls just as clearly from the single link_session_campaign source or Keshu perceives a regression. And because the current design has arms[].session_ids[] as a THIRD session store beside PlanItem.session_ids[], the matched-photodose 2-arm constraint and the member-session count can disagree between the plan item and the Experiment object — the fold collapses them to one, but the migration risks orphaning arm->session links already written by the seed. The gate cannot consume the verdict today because ELN has no tools and the render is a bottom-appended blob, not a node on the arm it describes. + +#### Where the hatch numbers come from: ground-truthing eggshell breach and pushing the flywheel +*Richard, checking that the hatching-time distribution he is about to publish rests on trustworthy hatch calls — because every hatch time is a bright-field eggshell-breach detection and a nuclear 3-fold-onset landmark, both perception outputs, not ground truth.* + +The distribution-shape Hypothesis (baseline hatch times are right-skewed / log-normal, and perturbations inflate the right tail and arrest fraction rather than translating the mode) is only as good as the hatch-detection underneath it. On the Vitals stage strip, Richard reviews each embryo's late window; the agent has pre-screened the bright-field frames, bulk-proposing the high-agreement eggshell-breach stretch as one range-confirm and surfacing only the ambiguous transitional frames (partial breach, debris) for him to decide. He drag-selects and confirms; set_ground_truth stamps annotator + provenance (the projection/trace it was judged from). The same pass confirms the nuclear 3-fold-onset landmark, since hatch time is measured as minutes from that landmark to breach. The agent computes a Result(kind=accuracy) — predicted breach frame vs Richard's ground truth per run — and, on Richard's explicit approval, prepares an approval-gated push of {bright-field hatch +- nuclear-stage frames, model prediction, human ground truth, provenance, strain/organism} to HuggingFace (pskeshu/gently-hatching-time) as hatch-detection training data. Only after the hatch calls are ground-truthed does the agent recompute the per-arm hatch-time distribution and re-score the log-normal / tail-inflation Hypothesis with AIC on the corrected data. + +- **Plan touch:** The ImagingSpec detectors for the imaged arms (bright-field eggshell breach + nuclear staging) are what generate the predictions being annotated; the export target (HF repo) is configured on the experiment phase. The auto-configure path that enabled those detectors at session start is the plan-mode -> execution bridge (active_plan_item_id -> start_timelapse). +- **ELN touch:** Ground-truth authoring via set_ground_truth (range drag-select + agent batch-confirm) on the stage strip; a Result(kind=accuracy) computed from predictions.jsonl vs ground_truth on the embryo/session; the approval-gated HuggingFace push (require_control + explicit confirm, HF_TOKEN from env, incremental/provenant). The corrected hatch times feed the distribution Result and the distribution-shape Hypothesis verdict. +- **Seam:** Two Results anchor at two different nodes, and the fold study says one of them must stay stable: the accuracy Result is model-agnostic and attaches to embryo/session (the flywheel that can ship FIRST, needing only Embryo/Prediction/ground_truth, no Strain/Experiment), while the distribution Result attaches to the experiment phase/arm. Friction: the flywheel-first ordering couples the accuracy Result to an attachment point that the later Experiment demotion (phase 5) might move, so the Result's anchor must be fixed up front or the flywheel needs rework. Deeper seam: the hatch-detection ground truth is BOTH the flywheel's training label AND the numerator of the ELN's hatch-time metric — a single annotation feeds the perception loop (retrospective/evidential, model-agnostic) and the plan's scientific outcome (the distribution that decides the next arm). Today those live in a disconnected annotator (gently/dataset) outside the live session and in the parallel ELN object; the fold converges them onto the session surface so one range-confirm serves the model and the claim at once — and the HF push, a specimen-external publish, must reuse hardware-grade approval so it never fires automatically. + +### Scenario: mutant-screen (blind WT vs mutant): plan a mixed-population imaging run, execute it, annotate which embryo is which, hypothesize a discriminating phenotype, record the classification result, and feed it back to HuggingFace. Grounded in seed/programs/mutant-screen.yaml — Experiment 1 (marker-based N2 vs otIs355) validates the multi-embryo classify-and-report machinery; Experiment 2 (phenotype-based: unc-6(+);otIs355 vs unc-6(ev400);otIs355, same TagRFP marker) is the real screen where genotype must be read from the UNC-6/netrin axon-guidance phenotype and confirmed by the post-hatch Unc locomotion phenotype. Stories are written against the fold study's Option C: Plan Mode stays the single design substrate, Experiment is demoted to a phase-role over imaging PlanItems, and Strain/Hypothesis/Result attach as native nodes on the plan tree. + +#### Designing the two-gate screen so Experiment 2 can't start before the cross exists +*Keshu, planning the mutant-screen campaign in plan mode with Hari's case study in hand (both are co-authors on the record)* + +Keshu opens /plan and says 'I want the agent to call genotype in a mixed dish of two C. elegans strains — first by marker, then by phenotype alone.' In the UNDERSTAND/RESEARCH phases the agent pulls the danienella outgrowth axis and the unc-6/netrin guidance literature, then proposes a Campaign 'Mutant screen — blind genotype calling' with two phases. Phase 1 (role=experiment, 'Marker-based call') gets two imaging PlanItems as its arms: a WT (N2) arm and a Reporter (otIs355) arm, both on one shared dish, each ImagingSpec = bright-field localization of every embryo + a single 488/561nm marker-check volume, no timelapse. It also drops three control PlanItems (pure-N2 blind, pure-otIs355 blind, 561-laser-off) and a decision_point gate 'Is the classify-and-report machinery trustworthy? (>=0.98 marker accuracy, <2% FP/FN)'. Phase 2 (role=experiment, 'Phenotype-based call') gets a unc-6(+) reporter arm and a unc-6(ev400);otIs355 mutant arm — BOTH TagRFP+, ImagingSpec = timelapse bean->3-fold continued to hatch, detectors=[comma, 1.5-fold, hatch], with stage-matched-scoring and imaged-vs-unimaged-hatch-band controls. Critically the agent inserts a genetics PlanItem 'Cross otIs355 x unc-6(ev400) -> homozygous line, ~10 days' and link_plan_items makes every Phase-2 imaging arm depend_on it, and makes Phase 2 depend_on the Phase-1 gate passing. As the agent writes the plan it also authors three Hypothesis nodes straight from the seed program — H1 (marker-separable at >=0.98), H2 (phenotype-separable >=0.75, monotone with stage), H3 (embryo call concordant with post-hatch Unc) — and attaches H1 at the Phase-1 header and H2/H3 at the Phase-2 header. Keshu approves; the plan commits and the startup briefing now reads 'Phase 2 blocked on genetic cross; Phase 2 blocked on Phase-1 gate.' + +- **Plan touch:** Campaign creation; two child-Campaign phases carrying role=experiment; imaging PlanItems as arms (WT/Reporter; unc-6(+)/unc-6(ev400)) each with a full ImagingSpec (marker-volume vs timelapse-to-hatch); sibling control PlanItems; a genetics PlanItem; link_plan_items dependency edges (Phase-2 arms depend_on the cross; Phase 2 depends_on the Phase-1 decision_point gate); the decision_point gate itself. +- **ELN touch:** Three Strain records resolved onto the arms' ImagingSpec.strain (N2, otIs355, unc-6(ev400);otIs355 — the last flagged 'confirm CGC designation / built by cross'); three Hypothesis nodes (H1/H2/H3 from the seed's hypotheses[]) authored by the agent and attached at their phase headers with predictions carrying expectation_refs (e.g. 'accuracy monotone across bean->3-fold'). +- **Seam:** This is the cleanest fold win AND its sharpest test. Under Option C the 'arms x strain x condition + controls' of each Experiment must be fully carried by imaging PlanItems (arm/condition field + comparison_to + sibling control items) — the fold study explicitly flags the mutant screen as the case to verify BEFORE Experiment.arms is dropped. Friction: H2's stage-dependence prediction and the stage-matched-scoring control are a Hypothesis-node concern and a PlanItem-spec concern respectively, so one scientific claim spans a claim node and an acquisition spec; and the decision_point gate ('machinery trustworthy?') and H1's verdict ('marker-separable — supported?') are two different nodes that must reference (not merge) each other — the gate consumes the verdict. Today's two-tree split would have put the cross-dependency in the plan tree and the hypotheses in a parallel ELN object with no way to express 'Phase 2 imaging is blocked until H1 is supported.' + +#### Running Experiment 1 and letting the marker channel score itself into HuggingFace +*Keshu, back in execution mode on the diSPIM the morning the shared N2+otIs355 dish is ready* + +At startup the agent resolves plan context, finds the one unblocked imaging item (the WT arm of Phase 1 is really 'localize every embryo on the shared dish + one marker volume'), and auto-configures from the ImagingSpec: bright-field localization of all embryos, then a single 488/561nm decision volume per embryo, 561-off confirmation pass per the control item. It detects 14 embryos, assigns each a genotype call with confidence from TagRFP presence (pan-neuronal nuclear signal = otIs355; dark = N2), and reports the population breakdown against the 50:50 plated ratio. Because ground truth here IS the marker channel, the agent pre-screens: on the per-embryo strip it bulk-proposes the confident TagRFP+ and TagRFP-dark calls as ground truth (one range-confirm) and surfaces only the two dim/early embryos near the detection threshold for Keshu to decide by eye. Keshu confirms. The agent then computes a candidate Result(kind=accuracy) — predicted genotype vs marker ground truth, per embryo — attaches it to the WT-arm PlanItem, and proposes the H1 verdict: accuracy 0.99, pure-N2 control FP 1.4%, pure-otIs355 FN 1.8% -> 'H1 supported.' Keshu confirms the verdict; that flips the Phase-1 decision_point gate to 'proceed', which unblocks Phase 2 (still waiting only on the cross). Keshu then approves the HF push: the {projection ref, model call, human ground truth, provenance, strain/organism} tuples format into the genotype-classification benchmark and push to pskeshu/gently-genotype-screen (falling back to pskeshu/gently-perception-benchmark if the repo check flags). + +- **Plan touch:** active_plan_item_id resolution auto-loads the WT-arm ImagingSpec (localization + single marker volume, no timelapse); the marker-check and 561-off control PlanItems execute; on completion the arm items are marked completed with outcomes and the Phase-1 decision_point gate fires once its dependencies (both arms + controls) are met. +- **ELN touch:** Ground-truth authoring on the per-embryo strip with agent batch-confirm of confident calls + human decision on threshold embryos; a Result(kind=accuracy) attached to the WT-arm PlanItem; the H1 Hypothesis verdict set (supported) with FP/FN predictions scored; the approval-gated HuggingFace export configured on the Phase-1 experiment-phase and pushed with provenance back to session/embryo/annotator. +- **Seam:** The touchpoint is the decision_point-gate <-> Hypothesis-verdict handoff: an ELN act (confirming H1 supported) must drive a plan act (opening the Phase-2 branch). The fold study warns these must relate, not merge — otherwise the plan loses its proceed/switch branch semantics or the science loses its data-borne verdict. Friction points: (1) the Result's attachment point — the flywheel is shipped first and the fold study says fix Result's anchor (embryo/session/item) as stable up front, but here the natural anchor is the WT-arm PlanItem that phase-5 might reshape; (2) HF export config lives on the phase (child Campaign) not a standalone Experiment object, so the approval gate + missing-HF_TOKEN disabled-state must ride the plan's require_control posture; (3) in today's split the accuracy Result sits in a parallel ELN store keyed only by campaign_ref, so it can't render inline on the arm item that produced it — the bolted-on renderElnSection appends it at the bottom instead of at the arm. + +#### Calling genotype from the axon-guidance phenotype and closing the ground-truth loop at hatch, unattended +*Hari (as reviewing collaborator/PI) checking the real screen after Keshu reports the unc-6(ev400);otIs355 cross is homozygous* + +Keshu reports conversationally 'the cross is done — homozygous unc-6(ev400);otIs355, Unc confirmed.' The agent marks the genetics PlanItem completed and, since the Phase-1 gate already passed, both Phase-2 arms unblock. On the shared dish (unc-6(+) reporter + mutant reporter, BOTH TagRFP+, so the marker is deliberately uninformative) the agent runs the Phase-2 timelapse ImagingSpec: bean->3-fold at adaptive cadence, continued to hatch, detectors=[comma, 1.5-fold, hatch]. For each embryo it calls genotype from reporter neuroanatomy alone — nerve-ring position + commissure guidance — expecting wild-type routing in one arm and a displaced/misshapen nerve ring + misrouted dorsoventral commissures in the mutant arm, and it deliberately holds low-confidence on pre-comma embryos where the phenotype hasn't manifested (H2's early mutant->WT miss). The hatch detector (field-clearing / vermiform escape) ends each embryo's track and triggers a short post-hatch locomotion observation: Unc vs coordinated. THIS behavior call is the ground truth — no blinding co-marker exists. The agent computes two candidate Results on the mutant-arm PlanItem: (a) accuracy(kind=accuracy) of embryo-morphology call vs behavior-confirmed genotype, stratified by stage (near chance at bean, rising after 1.5-fold), and (b) concordance between the two independent readouts. It proposes verdicts: H2 supported (0.81, monotone with stage, survives the stage-matched-scoring control), H3 supported (concordance >= morphology accuracy; disagreements localize to low-confidence embryos; hatch recall 0.92). Hari reviews at the Phase-2 header, edits H2's note to caveat expressivity, confirms both verdicts, and approves the incremental HF push of the phenotype-labeled slice. The final decision_point gate now reads 'unc-6 phenotype resolvable at gently's resolution -> expand or publish.' + +- **Plan touch:** Non-imaging plan update (cross reported done) completes the genetics PlanItem and advances the dependency graph to unblock both Phase-2 arms; the timelapse-to-hatch ImagingSpec auto-loads (adaptive intervals, hatch stop condition, detectors); the imaged-vs-unimaged hatch-band and stage-matched-scoring control PlanItems execute; the final decision_point gate ('expand or publish') fires when the arms + controls complete. +- **ELN touch:** Hatch-detection-triggered ground-truth authoring where the label comes from post-hatch Unc/WT behavior (not a marker); two Result nodes on the mutant-arm PlanItem (stage-stratified accuracy + cross-readout concordance); H2 and H3 verdicts set with predictions scored (stage-dependence, systematic early misses, hatch recall, Unc penetrance); incremental provenant HuggingFace push of the phenotype-based genotype benchmark slice. +- **Seam:** The seam is a genuinely two-layer loop: a PLAN detector (hatch) becomes the trigger for an ELN ground-truth label (locomotion), which scores an ELN Result, which sets an ELN verdict, which feeds a PLAN decision gate. The friction the fold targets: (1) H2's core claim ('signal is neuroanatomical, not developmental-timing') is only defensible because of a PlanItem control (stage-matched scoring) — the claim node and the acquisition spec are inseparable, which is exactly why a shared node tree (Option C) beats two parallel trees; (2) ground truth here has NO co-marker, so the Expectation/ground_truth plumbing must accept a downstream, post-hatch, behavioral label bound back to an embryo imaged hours earlier — a provenance chain that spans the timelapse track end (hatch) to the larva; (3) under today's split the two Results and the H2/H3 verdicts live in the parallel ELN store and can't render on the mutant-arm item or interleave with the timelapse phase, so a reviewer like Hari sees a bottom-appended blob instead of 'this arm, this claim, this score' in one place — the precise bolt-on Option C dissolves by anchoring Result-on-PlanItem and Hypothesis-at-phase-header. + +### Scenario: dopaminergic-outgrowth — "Catching Outgrowth": image the DEVELOPING dat-1 dopaminergic neuron (BZ555, dat-1p::GFP, onset ~3-fold) instead of its wired endpoint. Use expression onset (dopaminergic_signal.py: intensity_level>=MEDIUM -> speed_up; structure_quality=GOOD -> burst) as the trigger to switch from low-cadence low-dose monitoring to high-cadence capture of CEP dendrite extension, ramp_down_power on SATURATING, stop on has_hatched. Contrast a late dat-1 neuron against an early-slow ventral-nerve-cord DD neuron (juIs76, unc-25p::GFP) to reconstruct scaffold-then-follower ordering (a dependency-map unit), and fold in Richard's K-pump thermo-paralytic background (twitching stops at 25 C) to make motion-free fine-cadence capture through the twitchy 3-fold window possible. These stories traverse Plan Mode (prospective/executable Campaign->PlanItem graph, ImagingSpec->hardware) and the ELN spine (Strain/Hypothesis/Result + perception->ground_truth->accuracy->HuggingFace), flagging where the two current trees touch and where the seam bites, read against the Option-C fold recommendation. + +#### Spend photons only once the neuron turns on +*Keshu — designing and running the core method demo (Exp-1), wants the outgrowth trajectory without frying the embryo before dat-1 even switches on* + +As-a researcher, I open plan mode and say 'I want to catch dat-1 dopaminergic outgrowth but not dose the embryo for the ~2 hours before the neuron turns on.' The agent reasons through the biology (BZ555, dat-1p::GFP onset ~3-fold, CEP dendrite extension along the nose) and proposes a Campaign 'Catching Outgrowth' with an experiment-phase 'Exp-1 expression-triggered capture'. It lays down imaging PlanItems as the arms: an Adaptive (monitor->burst) item whose ImagingSpec is 488nm @5%, ~40 slices, interval 600s, adaptive_intervals {monitor:600, speed_up:120, burst:60}, detectors=[dopaminergic_signal], ramp_down_power on SATURATING, stop_condition=has_hatched; a Fixed high-cadence (naive) reference item at 488 @10% constant 120s; and three control PlanItems (low-cadence-only, minimal-imaging developmental, detector false-onset). I approve; a decision-gate item 'did the trigger fire before hatch?' closes the phase. Next morning startup auto-resolves active_plan_item_id to the Adaptive arm, pre-loads the ImagingSpec, I detect_embryos and start_timelapse. Live, dopaminergic_signal.py returns MEDIUM -> the loop drops to 2-min cadence, then GOOD -> fires 1-min bursts, catching CEP dendrite length-vs-time straight through hatch. Session completes and links back to the PlanItem. Now I flip to the evidential side: on the projection/stage strip I drag-select the frames where the CEP dendrite first emerges and confirm outgrowth-onset ground truth (the agent pre-screens the high-agreement stretch, I only adjudicate the transitional frames). I author a Hypothesis 'dat-1 onset (MEDIUM) precedes and brackets peak CEP extension, so onset is a usable capture trigger', with predictions bound to the run's Expectations. I record a Result(measurement) — onset ~1.5 h before hatch, the dendrite trajectory — and a Result(derived) comparing cumulative 488 dose Adaptive vs Fixed. The detector false-onset control feeds a Result(accuracy) on the detector's MEDIUM/GOOD calls, and the confirmed annotations queue for an approval-gated HuggingFace push. + +- **Plan touch:** Campaign -> experiment-phase -> imaging PlanItems as arms (Adaptive/Fixed) + control PlanItems; ImagingSpec.adaptive_intervals + detectors=[dopaminergic_signal] + ramp_down_power + stop_condition=has_hatched; active_plan_item_id -> detect_embryos/start_timelapse auto-config; the live detector event driving the low->high cadence switch (MEDIUM->speed_up, GOOD->burst); decision_point gate 'trigger fired before hatch?' +- **ELN touch:** ground_truth range-annotation of CEP dendrite onset on the stage/projection strip (agent batch-confirm, human adjudicates transitions); Hypothesis(onset-is-a-trigger) with predictions -> expectation_refs; Result(measurement: onset time + trajectory), Result(derived: Adaptive-vs-Fixed dose); Result(accuracy) from the false-onset control; approval-gated HuggingFace export of confirmed annotations +- **Seam:** The load-bearing object is ONE thing wearing two hats: the dopaminergic_signal.py onset event. In the plan tree it is a prospective adaptive-interval rule / decision-gate that fires live and reconfigures hardware; in the ELN tree the SAME onset is what the human re-marks as ground_truth and what the Result(accuracy) scores. With today's two trees there is no shared node, so the human re-annotates by hand exactly what the detector already decided, and 'did the trigger fire correctly?' can't be read off one place — the live trigger lives on the PlanItem/timelapse, the annotated onset lives in ground_truth/Result. The plan Doc even prints a 'Hypothesis:' line it cannot store, so the claim gets appended in the bolted-on ELN block at the bottom. Option C dissolves this: the Hypothesis attaches to the phase and the accuracy Result to the PlanItem that produced it, so trigger-as-plan and trigger-as-evidence become one node. + +#### Which neuron comes up first — the scaffold or the follower +*Keshu with Richard — building the dependency-map ordering (Exp-2): does the early ventral-nerve-cord neuron precede the late dopaminergic one?* + +As-a researcher I want to know whether the GABAergic DD ventral-nerve-cord neuron extends before the dat-1 dopaminergic neuron — the scaffold-then-follower ordering that is the unit of a developmental dependency map. In plan mode the agent builds an experiment-phase 'Exp-2 ordering' with two cross-strain imaging arms: an Early-VNC(DD) PlanItem on juIs76 (unc-25p::GFP) with adaptive monitor->burst tuned to that reporter, and a Late-dat-1 PlanItem on BZ555 that inherits_from the Exp-1 Adaptive spec. It adds a control PlanItem 'common developmental clock' (stage every embryo to time-before-hatch plus the 2-fold landmark so cross-strain times are comparable), and — because within-animal ordering needs Richard's two-color line — a genetics PlanItem 'build dat-1p::GFP; unc-25p::mCherry (Richard)' plus a Dual-label two-color imaging PlanItem (488+561) that depends_on it and sits blocked. A decision gate says 'run the two single-color arms now against the common clock; activate the dual-label arm when the strain lands.' I execute both single-color arms across sessions; each links its session_id to its arm. On the evidential side I annotate DD commissure onset and dat-1 onset per embryo against the common clock (outgrowth-timing ground_truth). I author the Hypothesis 'DD begins and completes earlier, over a slower and longer window, than dat-1 — VNC scaffold precedes dopaminergic follower' and record a Result(derived): DD onset before dat-1 onset, DD duration > dat-1 duration, DD rate < dat-1 rate — the ordering itself. Weeks later Richard's dual-label strain lands, the genetics PlanItem completes and unblocks the two-color PlanItem; I re-run within one animal and a second Result confirms VNC-before-DA inside individual embryos, not just across the population. + +- **Plan touch:** experiment-phase with two cross-strain imaging arms (juIs76 DD arm; BZ555 dat-1 arm via inherit_from Exp-1); a 'common developmental clock' control PlanItem; a genetics PlanItem (Richard's dual-label cross) + a blocked Dual-label two-color imaging PlanItem (depends_on the genetics item); decision_point gate on strain availability driving single-color-now vs dual-label-later +- **ELN touch:** Strain records BZ555 / juIs76 / dual-label (dual-label stock = in construction); per-embryo outgrowth-onset ground_truth staged on the common clock; Hypothesis(scaffold-then-follower); Result(derived: DD-vs-dat-1 onset ordering, duration, rate) as the dependency-map unit; a second within-animal Result once the dual-label arm runs +- **Seam:** The comparison is intrinsically cross-arm/cross-strain: the ordering Result and its Hypothesis must SPAN two imaging PlanItems (DD + dat-1) staged to a shared clock. In today's split the plan tree carries the two arm PlanItems with their session_ids[], but the ELN Experiment keeps its OWN arms[].session_ids[] — a third session-grouping store — so 'which sessions are the DD arm vs the dat-1 arm' is recorded twice and can drift (the fold study flags link_session_to_arm not updating the campaign link). The spanning Result has no single node to attach to and renders detached at the bottom, and the common-clock control (a plan PlanItem) is exactly what makes the two arms' ground_truth comparable, yet that dependency is invisible across the two trees. Option C — arms ARE the imaging PlanItems, one link_session_campaign source of truth, Result attached to the phase that spans them — is what makes 'these are 2 arms staged to one clock, ordered DD-before-dat-1' readable in one place. + +#### Stop the twitch to catch the tip +*Keshu + Richard — paralysis-assisted capture (Exp-3), folding the K-pump thermo strain and the temperature program in to beat 3-fold motion* + +As-a researcher I face the reason nobody has caught outgrowth at fine cadence: the 3-fold embryo twitches, corrupting frame-to-frame tip tracking. Richard offers his potassium-pump thermo-paralytic background — twitching stops at 25 C. In plan mode the agent builds an experiment-phase 'Exp-3 paralysis-assisted' with two arms: a 25C-thermo-paralysis PlanItem (strain dat-1p::GFP; K-pump — in construction, baseline substitute BZ555 held at 25 C) with ImagingSpec.temperature_c=25 and high 1-2 min cadence, and a 20C-twitching control PlanItem on BZ555 at identical cadence. It adds a thermal-only developmental control PlanItem (unimaged 25 C vs 20 C, measuring hatching-time distributions — this cross-links the temperature seed program's Campaign) and a registration-error control PlanItem (same volume acquired back-to-back to bound the non-biological jitter floor). A genetics PlanItem 'cross dat-1 into the K-pump background (Richard)' sits blocked, with a decision gate to run BZ555@25C as the baseline substitute until it lands. I execute: 25C (BZ555 baseline) high-cadence vs 20C twitching. On the evidential side I annotate tip-trackability and dendrite-length-vs-time per arm, capturing frame-to-frame displacement ground truth. I author the Hypothesis 'thermo-paralysis at 25 C suppresses twitching to near the registration-error floor and preserves outgrowth duration in developmental time — temperature rescales the clock without distorting the process' and record a Result(measurement): displacement at 25 C far below 20 C, approaching the registration floor; plus a Result(derived): outgrowth duration expressed as a fraction of embryogenesis, preserved 25 C vs 20 C, computed against the thermal-only control's hatching-time distributions. Finally the motion-free, tip-tracked trajectories and their onset ground_truth push to HuggingFace (pskeshu/gently-dopaminergic-outgrowth) behind an explicit approval gate. + +- **Plan touch:** experiment-phase with two temperature-arm imaging PlanItems (ImagingSpec.temperature_c setpoint driving the hardware/thermal axis); thermal-only developmental + registration-error control PlanItems; a genetics PlanItem (K-pump cross, blocked) + decision_point gate to substitute BZ555@25C; explicit cross-link to the temperature seed program's Campaign +- **ELN touch:** Strain records (dat-1;K-pump stock = in construction; BZ555 substitute); frame-to-frame tip-displacement + trajectory ground_truth per arm; Hypothesis(paralysis preserves developmental-time kinematics); Result(measurement: 25C-vs-20C displacement vs registration floor) and Result(derived: outgrowth-duration-as-fraction-of-embryogenesis) that READS the thermal-only control's hatching-time distributions; approval-gated HuggingFace push of the tracked trajectories +- **Seam:** Two seams bite here. (1) The strain-in-construction is ONE fact that must live in both trees at once: in the plan tree it is a genetics PlanItem + a blocked imaging PlanItem (depends_on) + a decision gate substituting BZ555@25C; in the ELN it is a Strain record whose stock reads 'in construction'. Unconnected, the plan's 'blocked until strain' and the ELN Strain's construction status describe the same reality twice and can silently disagree. (2) The temperature_c value is a prospective ImagingSpec field that drives hardware AND the independent variable the retrospective Result is conditioned on — yet nothing in the two-tree split ties the executed setpoint to the Result's condition axis, and the developmental-time Result reads hatching-time distributions from a DIFFERENT campaign (the temperature seed program), so a Result spanning two campaigns has no home in either tree. Option C helps by hanging the export config and Result on the phase and resolving strain as a reference on ImagingSpec, but the cross-campaign temperature link and the shared strain-construction state are exactly the friction the fold has to make coherent rather than duplicate. + +### Scenario: longitudinal-multiuser: work that stretches across many sessions, days, and weeks, and across more than one human. Datasets and ground-truth annotations accumulate round over round; a second annotator re-scores and systematically disagrees at exactly the hard boundaries; a quantitative Result from one experiment/campaign becomes the seed of the next plan (a certified protocol, a residual-confusion finding, or a freshly-crossed strain); and the agent proposes a hypothesis verdict that the human(s) confirm. The through-line is that PLAN HISTORY (prospective: campaign/phase/PlanItem specs evolving as inheritance propagates and gates open) and the ELN RECORD (retrospective: ground_truth, accuracy Results, hypothesis verdicts accumulating per round/embryo) advance on two different time axes that must stay stitched together — and the current two-tree split (Plan Mode's Campaign->PlanItem graph vs the ELN's parallel Experiment/Hypothesis/Result store) is where they tear. All three stories are read against Option C (Plan absorbs ELN as native nodes+fields; Experiment demoted to a phase-role over imaging PlanItems) to show where the fold heals the seam. + +#### The 2-fold/3-fold boundary two annotators keep fighting over +*Keshu (PI, primary annotator) + Ana (rotation student, second annotator), over ~6 weeks on the stage-classification program (RW10226 ubiquitous nuclear-GFP).* + +Keshu opens /plan and designs the stage-classification campaign: Experiment 1 as a phase with three imaging PlanItems that all read the SAME paired RW10226 embryos/volumes — bf-only, nuclear-only, fusion (input modality the only variable) — sharing one ImagingSpec with adaptive_intervals (comma->1.5-fold->2-fold->3-fold->pretzel), plus a Phase-1 decision_point 'does fusion beat single?'. Over several days on the diSPIM the sessions run and gently-perception writes predictions.jsonl. On the Vitals stage strip Keshu one-drag-confirms the agent's pre-screened high-agreement stretches (long comma and pretzel runs) and hand-labels the transitional frames. Weeks later Ana re-annotates a held-out subset as a second annotator and disagrees SYSTEMATICALLY at the 2-fold<->3-fold boundary — precisely where the bright-field outline saturates and only the nuclear lattice still moves. Each export round the agent computes a Result(kind=accuracy) + confusion matrix (fusion > nuclear-only > bf-only), notices the off-diagonal mass AND the inter-annotator disagreement both concentrate at 2-fold/3-fold, proposes the H1 verdict 'fusion gain is real and concentrated at late transitions — SUPPORTED,' and proposes a NEW hypothesis that residual error is annotation-limited, not model-limited. Keshu confirms. The round-1->round-4 accuracy trend then seeds the next campaign's plan: a new higher-cadence PlanItem bracketing 2-fold/3-fold and a double-annotate-then-adjudicate step before the approval-gated push to pskeshu/gently-perception-benchmark. + +- **Plan touch:** Campaign -> Experiment-1 phase -> three imaging PlanItems (the arms) sharing one ImagingSpec + adaptive_intervals; a Phase-1 decision_point gate; plan HISTORY as the fusion PlanItem's spec is re-versioned round over round; the result-seeds-next-plan step spawns a fresh high-cadence PlanItem + a new phase. +- **ELN touch:** set_ground_truth on the Vitals stage strip stamped per annotator (Keshu, then Ana); a Result(kind=accuracy)+confusion-matrix per export round; H1 verdict + the new annotation-limited hypothesis (agent proposes, human confirms); approval-gated HuggingFace export. +- **Seam:** The accuracy Result SCORES the fusion PlanItem but in the two-tree world it lands in the bottom-appended /eln blob (renderElnSection), disconnected from the item and from the spec version that produced it — you cannot read round-over-round accuracy next to the ImagingSpec history that generated each round. Worse, ground_truth stores a single annotator stamp per (stage,timepoint), so Ana's disagreeing labels either overwrite Keshu's or have nowhere to live — there is no adjudication node. Two clocks refuse to align: plan history advances on a spec-version axis, the ELN advances on an annotation-round axis. Option C heals it by attaching the Result inline on the fusion PlanItem and the hypothesis at the phase header, so the annotation-round Results interleave with the spec versions on one node. + +#### When the decision gate must consume the verdict (paralysis -> distributions) +*Keshu (imaging) + Richard Ikegami (owns RK-KP1, remote second scorer), across ~5 weeks on the temperature->hatching program.* + +In /plan Keshu lays out the campaign as three phases. Exp A (paralysis validation) is a phase of motion-assay imaging PlanItems — RK-KP1 at 25C, RK-KP1 at 20C, WT-HIS at 25C, plus a reversibility control — each driving the ACUITYnano setpoint. A decision_point gates whether to proceed: 'is the 25C paralysis real, K-channel-specific, and reversible?' Downstream, Exp B (the 2x2 factorial: temperature-protocol x imaging) is a phase whose thermo-cycling ImagingSpec is meant to INHERIT the certified 20<->25C schedule from Exp A. Over several days the agent runs Exp A, computing a motion index per embryo (blinded to arm). Richard, remote, independently scores twitch/still as the second annotator; he and the agent's motion-index threshold disagree on the borderline embryos scored within one thermal time-constant of 25C. The agent computes the motion Result, proposes the H1 verdict '25C abolishes twitching, mutant-specific, reversible — SUPPORTED'; Keshu and Richard confirm. That verdict is what should RELEASE the plan's decision_point, unblocking Exp B. Exp B then runs n>=20/arm over three weeks; hatch-time (3-fold onset -> bright-field eggshell breach) accumulates into distributions. Once enough embryos land, the agent proposes the log-normal / right-tail-inflation hypothesis and the 'light perturbs, thermalization doesn't' verdict from the 2x2; Keshu confirms, and hatch frames export to pskeshu/gently-hatching-time. + +- **Plan touch:** Exp A / Exp B / Exp C as phases; ImagingSpec carrying the temperature setpoint + thermo-cycling schedule; spec INHERITANCE flowing the certified 20<->25C schedule from Exp A into Exp B/C arms; the decision_point gate between the paralysis phase and the distribution phase; the plan status readout ('Exp B: 47/80 sessions'). +- **ELN touch:** the motion-index Result (Exp A) confirming H1; the accumulating hatch-time Results building per-arm distributions (median, CV, 90th-percentile tail) over weeks; the distribution-shape + photodose-dose-response hypotheses (agent proposes verdict, humans confirm); two annotators (agent motion index + Richard); approval-gated HF export. +- **Seam:** The plan's decision_point ('proceed to Exp B?') and the ELN Hypothesis verdict ('paralysis SUPPORTED') are the SAME judgement expressed twice — the fold study says relate, not merge (a gate consumes a verdict). In the two-tree world the gate lives in plan/current.yaml and the verdict in agent/hypotheses/ with no link, so the human re-decides what the ELN already concluded. Second tear: the certified thermo-cycling schedule is an ELN Result but must reach plan ImagingSpec.adaptive_intervals via inheritance — today a hand-copy across the seam. And the longitudinal distribution grows as ELN Results over weeks while the plan tree shows only a session counter, so 'these 60 sessions = 20 embryos x 3 arms of one distribution test' is legible in neither tree alone. Option C makes the gate reference the verdict and hangs the certified schedule where inheritance can pull it. + +#### A cross that mints a strain, a phenotype two experts read differently +*Keshu (imaging + primary genotype caller) + Hari Shroff (collaborator, second phenotype annotator), plus a ~10-day genetic cross, on the mutant-screen program.* + +In /plan Keshu builds the blind-genotype screen: Experiment 1 (marker-based validation) as a phase mixing N2 + otIs355 on one dish, a single 488/561nm marker-check volume PlanItem; a genetics PlanItem 'cross otIs355 x unc-6(ev400) -> homozygous Unc line, ~10 days'; and Experiment 2 (the real phenotype screen) as a phase whose imaging PlanItem depends_on that cross. Strain records for N2 and otIs355 already exist; the unc-6(ev400);otIs355 record does NOT yet exist. Exp 1 runs: the agent bright-field-localizes every embryo, calls genotype from TagRFP presence, scores a Result(kind=accuracy) ~0.99 against the marker ground truth -> H1 SUPPORTED (agent proposes, Keshu confirms), releasing the gate to Exp 2. Ten days later Keshu reports 'cross done, homozygous Unc line' — the genetics PlanItem completes AND a new unc-6(ev400);otIs355 Strain record is born, unblocking Exp 2's imaging PlanItem. Exp 2 images the mixed dish bean->3-fold->hatch; both strains carry the SAME TagRFP so marker is uninformative and the agent must call genotype from nerve-ring position + commissure routing. Keshu and Hari independently annotate the displaced-nerve-ring phenotype and disagree on subtle mid-elongation embryos. The post-hatch Unc locomotion — triggered by the hatch detector ending each embryo's track — is the tie-breaker ground truth. The agent computes stage-stratified accuracy + an embryo-vs-behavior concordance Result, proposes the H2 verdict 'genotype separable from neuroanatomy alone, above chance, monotone with stage — SUPPORTED'; Keshu and Hari confirm; the benchmark exports to pskeshu/gently-genotype-screen. + +- **Plan touch:** Exp1 / Exp2 phases; a genetics PlanItem whose depends_on gates the Exp2 imaging PlanItem; a decision_point on Exp1 accuracy; the execution path (bright-field localization + the hatch detector that ends each track and triggers the behavior assay). +- **ELN touch:** Strain records — including the one MINTED when the cross completes; the Exp1 accuracy Result -> H1; the Exp2 stage-stratified accuracy + concordance Results -> H2/H3; two phenotype annotators (Keshu, Hari); post-hatch Unc locomotion as adjudicating ground truth; approval-gated HF export. +- **Seam:** Three tears. (1) The genetics PlanItem completing must MINT an ELN Strain record and rewire the depends_on imaging PlanItem's ImagingSpec.strain to resolve to it — a plan-outcome event reaching across into the ELN Strain store; today the strain is a bare string on the spec and a disconnected Strain yaml, with nothing linking the cross's completion to the record's birth. (2) The unc-6 arm is literally one-thing-modeled-twice — an imaging PlanItem (with session_ids) AND an Experiment.arm (with its own session_ids) — the exact redundancy Option C collapses; session membership is triple-stored (PlanItem, Experiment.arm, link_session_campaign). (3) Two experts disagree and behavior adjudicates, but ground_truth carries a single annotator stamp and no node records 'Keshu vs Hari, resolved by post-hatch Unc.' Option C demotes Experiment to a phase-role so the arm IS the imaging PlanItem, makes the cross's completion mint the Strain the spec resolves to, and hangs the accuracy/concordance Results inline on the item they score. diff --git a/docs/superpowers/plans/2026-07-02-eln-scientific-spine.md b/docs/superpowers/plans/2026-07-02-eln-scientific-spine.md new file mode 100644 index 00000000..447c3b52 --- /dev/null +++ b/docs/superpowers/plans/2026-07-02-eln-scientific-spine.md @@ -0,0 +1,632 @@ +# ELN Scientific Spine Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add the co-authored ELN scientific spine (Strain → Experiment → Hypothesis → Result) + the perception-annotation → HuggingFace flywheel to gently, as optional file-based overlays. + +**Architecture:** New spine entities are file-based YAML under `agent/` via `FileContextStore` (mirrors the campaign store). ELN logic (accuracy scoring, HF export) lives in a new `gently/eln/` package that reads from `FileStore` (not the deprecated `gently/dataset`). Ground-truth authoring reuses the existing `FileStore.set_ground_truth` (already range-based). Scoring reuses the existing `Expectation` machinery. Everything is optional — nothing in the current flow requires a spine record. + +**Tech Stack:** Python 3.10+, FastAPI (viz routes), PyYAML (file store), pytest, ruff. HuggingFace export via lazy `datasets`/`huggingface_hub` (not base deps). Vanilla JS for the viz UI. + +## Global Constraints + +- Python `>=3.10,<3.13`; numpy `<2`; ruff line-length 100; `select = ["E","F","I","UP","B"]`. +- No SQLite for spine entities — file-based YAML under `agent/` (`FileContextStore`). +- Spine entities are OPTIONAL overlays; bare-string strain back-compat (no forced migration). +- HuggingFace export is APPROVAL-GATED (`require_control` + explicit confirm; never auto); `HF_TOKEN` from env; default repo `pskeshu/gently-perception-benchmark`. +- UI work: verify in-browser (Chrome MCP UI/UX audit — alignment/spacing/overflow/contrast) before marking done; only show coherent screens (per repo convention). +- Commit after every green step. + +--- + +## Status — already implemented on `feature/eln-scientific-spine` (backend, tested) + +These are DONE and merged on the branch; later tasks build on them: + +- **Strain store** — `FileContextStore.create_strain/get_strain/list_strains/resolve_strain` (`gently/harness/memory/file_store.py`); spine dir skeleton (`strains/experiments/hypotheses/results`). Tests: `tests/test_strain_store.py` (3). +- **HF connector** — `gently/eln/hf_connector.py`: `build_records` (pure), `push_dataset` (approval-gated, env `HF_TOKEN`, lazy `datasets`, `_push_fn` injection), `token_present`. Tests: `tests/test_hf_connector.py` (5). +- **Accuracy scoring** — `gently/eln/accuracy.py`: `stage_at_timepoint`, `score_accuracy`, `accuracy_result`. Tests: `tests/test_eln_accuracy.py` (4). +- **Ground-truth authoring route** — `POST /api/embryos/{id}/ground_truth` (`gently/ui/web/routes/data.py`, `require_control`, range-based → `set_ground_truth`). Tests: `tests/test_eln_ground_truth_route.py` (4). +- **Seed data** — `seed/programs/{stage-classification,temperature-hatching,mutant-screen,dopaminergic-outgrowth}.yaml` + README. +- **Usage evaluation** — `docs/product-ideation/ELN-USAGE.md` (biologist journey + iterative stories + story-flow specs + design implications). + +Run all: `.venv/bin/python -m pytest tests/test_strain_store.py tests/test_hf_connector.py tests/test_eln_accuracy.py tests/test_eln_ground_truth_route.py -q` → 16 passed. + +--- + +## File Structure (remaining work) + +- `gently/harness/memory/file_store.py` — add Experiment / Hypothesis / Result store methods (mirror Strain). +- `gently/eln/seed_loader.py` — CREATE: load `seed/programs/*.yaml` into `FileContextStore`. +- `gently/eln/strain_resolution.py` — CREATE: resolve strain refs on `ImagingSpec`/`Note` for display (non-mutating). +- `gently/eln/export_service.py` — CREATE: assemble annotation rows from `FileStore` for an experiment → `hf_connector`. +- `gently/ui/web/routes/data.py` — add Experiment/Result read routes + the approval-gated export route. +- `gently/harness/tools/…` — register agent tools (`create_experiment`, `author_hypothesis`, `record_result`, `link_session_to_arm`, `configure_export`). +- `gently/ui/web/static/js/…` + templates — Vitals stage-strip authoring UI, accuracy readout, experiment inspector, export button. +- `tools/ui_crawler/stories/US-*.py` — ELN story flows (after #76 lands the harness on the base). + +--- + +## Task 1: Experiment store + +**Files:** +- Modify: `gently/harness/memory/file_store.py` (after the Strain methods) +- Test: `tests/test_experiment_store.py` + +**Interfaces:** +- Consumes: `self._gen_id`, `self._now`, `self._write_yaml`, `self._read_yaml`, `self.agent_dir`. +- Produces: `create_experiment(title, campaign_ref=None, arms=None, controls=None, replicate_of=None, export=None, author=None, experiment_id=None) -> str`; `get_experiment(id) -> dict|None`; `list_experiments() -> list[dict]`; `link_session_to_arm(experiment_id, arm_name, session_id) -> bool`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_experiment_store.py +from gently.harness.memory.file_store import FileContextStore + + +def test_create_and_get_experiment(tmp_path): + s = FileContextStore(tmp_path / "agent") + eid = s.create_experiment( + "32C vs control", + arms=[{"name": "hot", "strain_ref": "OH904", "condition": "25C"}], + controls=["control"], + ) + e = s.get_experiment(eid) + assert e["title"] == "32C vs control" + assert e["arms"][0]["name"] == "hot" + assert e["controls"] == ["control"] + + +def test_link_session_to_arm(tmp_path): + s = FileContextStore(tmp_path / "agent") + eid = s.create_experiment("e", arms=[{"name": "hot", "strain_ref": "x", "condition": "c"}]) + assert s.link_session_to_arm(eid, "hot", "sess1") is True + assert "sess1" in s.get_experiment(eid)["arms"][0]["session_ids"] + assert s.link_session_to_arm(eid, "missing", "sess1") is False + + +def test_list_experiments(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_experiment("a"); s.create_experiment("b") + assert len(s.list_experiments()) == 2 +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `.venv/bin/python -m pytest tests/test_experiment_store.py -q` +Expected: FAIL (`AttributeError: ... 'create_experiment'`). + +- [ ] **Step 3: Write minimal implementation** + +Add to `FileContextStore` (after the Strain section): + +```python + # ------------------------------------------------------------------ + # Scientific spine — Experiment (ELN, phase 2). Optional overlay. + # ------------------------------------------------------------------ + + def create_experiment( + self, + title: str, + campaign_ref: str | None = None, + arms: list | None = None, + controls: list | None = None, + replicate_of: str | None = None, + export: dict | None = None, + author: str | None = None, + experiment_id: str | None = None, + ) -> str: + eid = experiment_id or self._gen_id() + now = self._now() + norm_arms = [] + for a in arms or []: + norm_arms.append({ + "name": a.get("name"), + "strain_ref": a.get("strain_ref") or a.get("strain"), + "condition": a.get("condition"), + "session_ids": list(a.get("session_ids") or []), + }) + data = { + "id": eid, "title": title, "campaign_ref": campaign_ref, + "hypothesis_refs": [], "arms": norm_arms, + "controls": list(controls or []), "replicate_of": replicate_of, + "export": export or {}, "notes": None, "status": "active", + "author": author, "created_at": now, "updated_at": now, + } + self._write_yaml(self.agent_dir / "experiments" / f"{eid}.yaml", data) + return eid + + def get_experiment(self, experiment_id: str) -> dict | None: + p = self.agent_dir / "experiments" / f"{experiment_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_experiments(self) -> list[dict]: + d = self.agent_dir / "experiments" + if not d.exists(): + return [] + return [e for e in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if e] + + def link_session_to_arm(self, experiment_id: str, arm_name: str, session_id: str) -> bool: + e = self.get_experiment(experiment_id) + if not e: + return False + for arm in e.get("arms", []): + if arm.get("name") == arm_name: + if session_id not in arm.setdefault("session_ids", []): + arm["session_ids"].append(session_id) + e["updated_at"] = self._now() + self._write_yaml(self.agent_dir / "experiments" / f"{experiment_id}.yaml", e) + return True + return False +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `.venv/bin/python -m pytest tests/test_experiment_store.py -q` → Expected: PASS (3). + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/file_store.py tests/test_experiment_store.py +git commit -m "feat(eln): Experiment store (spine phase 2)" +``` + +--- + +## Task 2: Hypothesis store + +**Files:** +- Modify: `gently/harness/memory/file_store.py` +- Test: `tests/test_hypothesis_store.py` + +**Interfaces:** +- Produces: `create_hypothesis(statement, predictions=None, experiment_refs=None, author=None, hypothesis_id=None) -> str`; `get_hypothesis(id) -> dict|None`; `list_hypotheses() -> list[dict]`; `set_hypothesis_status(id, status) -> bool` where status ∈ {proposed, supported, refuted, inconclusive}. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_hypothesis_store.py +import pytest +from gently.harness.memory.file_store import FileContextStore + + +def test_create_get_hypothesis(tmp_path): + s = FileContextStore(tmp_path / "agent") + hid = s.create_hypothesis( + "25C shifts hatching-time distribution later", + predictions=[{"target": "median_hatch", "expected": "+30min"}], + author="human", + ) + h = s.get_hypothesis(hid) + assert h["statement"].startswith("25C") + assert h["status"] == "proposed" + assert h["predictions"][0]["target"] == "median_hatch" + + +def test_set_status(tmp_path): + s = FileContextStore(tmp_path / "agent") + hid = s.create_hypothesis("h") + assert s.set_hypothesis_status(hid, "supported") is True + assert s.get_hypothesis(hid)["status"] == "supported" + with pytest.raises(ValueError): + s.set_hypothesis_status(hid, "bogus") +``` + +- [ ] **Step 2: Run test to verify it fails** — Run: `.venv/bin/python -m pytest tests/test_hypothesis_store.py -q` → FAIL. + +- [ ] **Step 3: Write minimal implementation** + +```python + # ------------------------------------------------------------------ + # Scientific spine — Hypothesis (ELN, phase 3). Optional overlay. + # ------------------------------------------------------------------ + _HYP_STATUSES = ("proposed", "supported", "refuted", "inconclusive") + + def create_hypothesis( + self, + statement: str, + predictions: list | None = None, + experiment_refs: list | None = None, + author: str | None = None, + hypothesis_id: str | None = None, + ) -> str: + hid = hypothesis_id or self._gen_id() + data = { + "id": hid, "statement": statement, "status": "proposed", + "predictions": list(predictions or []), + "experiment_refs": list(experiment_refs or []), + "author": author, "basis": [], "created_at": self._now(), + } + self._write_yaml(self.agent_dir / "hypotheses" / f"{hid}.yaml", data) + return hid + + def get_hypothesis(self, hypothesis_id: str) -> dict | None: + p = self.agent_dir / "hypotheses" / f"{hypothesis_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_hypotheses(self) -> list[dict]: + d = self.agent_dir / "hypotheses" + if not d.exists(): + return [] + return [h for h in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if h] + + def set_hypothesis_status(self, hypothesis_id: str, status: str) -> bool: + if status not in self._HYP_STATUSES: + raise ValueError(f"status must be one of {self._HYP_STATUSES}") + h = self.get_hypothesis(hypothesis_id) + if not h: + return False + h["status"] = status + self._write_yaml(self.agent_dir / "hypotheses" / f"{hypothesis_id}.yaml", h) + return True +``` + +- [ ] **Step 4: Run test to verify it passes** → PASS (2). + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/file_store.py tests/test_hypothesis_store.py +git commit -m "feat(eln): Hypothesis store (spine phase 3)" +``` + +--- + +## Task 3: Result store + persist accuracy Result + +**Files:** +- Modify: `gently/harness/memory/file_store.py` +- Test: `tests/test_result_store.py` + +**Interfaces:** +- Consumes: `gently.eln.accuracy.accuracy_result`. +- Produces: `save_result(result: dict, result_id=None) -> str`; `get_result(id) -> dict|None`; `list_results(experiment_ref=None) -> list[dict]`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_result_store.py +from gently.harness.memory.file_store import FileContextStore +from gently.eln.accuracy import accuracy_result + + +def test_save_and_filter_results(tmp_path): + s = FileContextStore(tmp_path / "agent") + r = accuracy_result("sess", "emb", [{"timepoint": 1, "predicted_stage": "bean"}], + [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 5}]) + r["experiment_ref"] = "exp1" + rid = s.save_result(r) + assert s.get_result(rid)["kind"] == "accuracy" + assert len(s.list_results(experiment_ref="exp1")) == 1 + assert len(s.list_results(experiment_ref="other")) == 0 +``` + +- [ ] **Step 2: Run test to verify it fails** → FAIL. + +- [ ] **Step 3: Write minimal implementation** + +```python + # ------------------------------------------------------------------ + # Scientific spine — Result (ELN, phase 3). Optional overlay. + # ------------------------------------------------------------------ + + def save_result(self, result: dict, result_id: str | None = None) -> str: + rid = result_id or result.get("id") or self._gen_id() + data = dict(result) + data["id"] = rid + data.setdefault("created_at", self._now()) + self._write_yaml(self.agent_dir / "results" / f"{rid}.yaml", data) + return rid + + def get_result(self, result_id: str) -> dict | None: + p = self.agent_dir / "results" / f"{result_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_results(self, experiment_ref: str | None = None) -> list[dict]: + d = self.agent_dir / "results" + if not d.exists(): + return [] + out = [r for r in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if r] + if experiment_ref is not None: + out = [r for r in out if r.get("experiment_ref") == experiment_ref] + return out +``` + +- [ ] **Step 4: Run test to verify it passes** → PASS. + +- [ ] **Step 5: Commit** + +```bash +git add gently/harness/memory/file_store.py tests/test_result_store.py +git commit -m "feat(eln): Result store + persist accuracy Result (spine phase 3)" +``` + +--- + +## Task 4: Strain resolution helper (display-side, non-mutating) + +**Files:** +- Create: `gently/eln/strain_resolution.py` +- Test: `tests/test_strain_resolution.py` + +**Interfaces:** +- Consumes: `FileContextStore.resolve_strain`. +- Produces: `resolve_ref(store, ref) -> dict|None` (returns `{"id","name",...}` record or a `{"name": ref}` shim for a bare unmatched string); `enrich_imaging_spec(store, spec_dict) -> dict` (adds `strain_record` alongside the raw `strain` string, never overwrites). + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_strain_resolution.py +from gently.harness.memory.file_store import FileContextStore +from gently.eln.strain_resolution import resolve_ref, enrich_imaging_spec + + +def test_resolve_ref_record_and_shim(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_strain("OH904", genotype="otIs355") + assert resolve_ref(s, "OH904")["genotype"] == "otIs355" # real record + shim = resolve_ref(s, "unknown-str") + assert shim == {"name": "unknown-str"} # bare-string shim + assert resolve_ref(s, None) is None + + +def test_enrich_imaging_spec_non_mutating(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_strain("N2") + spec = {"strain": "N2", "num_slices": 80} + out = enrich_imaging_spec(s, spec) + assert out["strain"] == "N2" # original preserved + assert out["strain_record"]["name"] == "N2" + assert "strain_record" not in spec # did not mutate input +``` + +- [ ] **Step 2: Run test to verify it fails** → FAIL (no module). + +- [ ] **Step 3: Write minimal implementation** + +```python +# gently/eln/strain_resolution.py +"""Display-side strain resolution — non-mutating, bare-string safe.""" +from __future__ import annotations + + +def resolve_ref(store, ref): + if not ref: + return None + rec = store.resolve_strain(ref) + return rec if rec else {"name": ref} + + +def enrich_imaging_spec(store, spec_dict): + out = dict(spec_dict) + ref = out.get("strain") + if ref: + out["strain_record"] = resolve_ref(store, ref) + return out +``` + +- [ ] **Step 4: Run test to verify it passes** → PASS. + +- [ ] **Step 5: Commit** + +```bash +git add gently/eln/strain_resolution.py tests/test_strain_resolution.py +git commit -m "feat(eln): non-mutating strain resolution helper" +``` + +--- + +## Task 5: Seed loader + +**Files:** +- Create: `gently/eln/seed_loader.py` +- Test: `tests/test_seed_loader.py` + +**Interfaces:** +- Consumes: `FileContextStore.create_strain/create_campaign/create_experiment/create_hypothesis`. +- Produces: `load_program(store, program_dict) -> dict` (returns `{campaign_id, strain_ids, experiment_ids, hypothesis_ids}`); `load_seed_dir(store, seed_dir) -> list[dict]`. + +- [ ] **Step 1: Write the failing test** + +```python +# tests/test_seed_loader.py +from gently.harness.memory.file_store import FileContextStore +from gently.eln.seed_loader import load_program + + +def test_load_program(tmp_path): + s = FileContextStore(tmp_path / "agent") + program = { + "slug": "demo", + "campaign": {"title": "T", "goal": "G", "description": "D"}, + "strains": [{"name": "OH904", "genotype": "otIs355"}], + "experiments": [{"title": "E1", "arms": [{"name": "a", "strain": "OH904", "condition": "c"}]}], + "hypotheses": [{"statement": "H1", "predictions": ["p"]}], + } + out = load_program(s, program) + assert s.get_campaign(out["campaign_id"]) is not None + assert len(out["strain_ids"]) == 1 and len(out["experiment_ids"]) == 1 + assert s.get_experiment(out["experiment_ids"][0])["arms"][0]["strain_ref"] == "OH904" +``` + +- [ ] **Step 2: Run test to verify it fails** → FAIL. + +- [ ] **Step 3: Write minimal implementation** + +```python +# gently/eln/seed_loader.py +"""Load seed research programs (seed/programs/*.yaml) into a FileContextStore.""" +from __future__ import annotations + +from pathlib import Path + +import yaml + + +def load_program(store, program: dict) -> dict: + c = program.get("campaign", {}) + campaign_id = store.create_campaign( + description=c.get("description") or c.get("title") or program.get("slug", "seed"), + shorthand=c.get("title"), + summary=c.get("goal"), + ) + strain_ids = [ + store.create_strain( + s["name"], genotype=s.get("genotype"), markers=s.get("markers"), + organism_ref=s.get("organism_ref"), author="seed", + ) + for s in program.get("strains", []) + ] + experiment_ids = [ + store.create_experiment( + e["title"], campaign_ref=campaign_id, arms=e.get("arms"), + controls=e.get("controls"), + export={"target": "hf", "repo": e["export_repo"]} if e.get("export_repo") else None, + author="seed", + ) + for e in program.get("experiments", []) + ] + hypothesis_ids = [ + store.create_hypothesis( + h["statement"], + predictions=[{"target": p, "expected": None} for p in h.get("predictions", [])], + experiment_refs=experiment_ids, author="seed", + ) + for h in program.get("hypotheses", []) + ] + return { + "campaign_id": campaign_id, "strain_ids": strain_ids, + "experiment_ids": experiment_ids, "hypothesis_ids": hypothesis_ids, + } + + +def load_seed_dir(store, seed_dir) -> list[dict]: + out = [] + for f in sorted(Path(seed_dir).glob("*.yaml")): + program = yaml.safe_load(f.read_text()) + if program and program.get("campaign"): + out.append(load_program(store, program)) + return out +``` + +- [ ] **Step 4: Run test to verify it passes** → PASS. + +- [ ] **Step 5: Commit** + +```bash +git add gently/eln/seed_loader.py tests/test_seed_loader.py +git commit -m "feat(eln): seed-program loader (seed/programs -> FileContextStore)" +``` + +--- + +## Task 6: Export service — assemble annotation rows for an experiment + +**Files:** +- Create: `gently/eln/export_service.py` +- Test: `tests/test_export_service.py` + +**Interfaces:** +- Consumes: `FileStore.get_ground_truth`, per-embryo predictions, `hf_connector.build_records`. +- Produces: `collect_annotations(file_store, session_id, embryo_ids, strain=None) -> list[dict]` — one row per ground-truth-covered prediction, ready for `build_records`. + +- [ ] **Step 1: Write the failing test** — with a fake file_store exposing `get_ground_truth` + a `get_predictions`-like accessor; assert rows pair prediction+GT+provenance. (Full code: mirror `accuracy.stage_at_timepoint` to attach `ground_truth_stage` to each prediction row; carry `session_id/embryo_id/annotator/strain`.) + +- [ ] **Step 2–5:** implement `collect_annotations`, test green, commit `feat(eln): export service — assemble annotation rows`. + +--- + +## Task 7: Approval-gated HF export route (viz) + +**Files:** +- Modify: `gently/ui/web/routes/data.py` +- Test: `tests/test_eln_export_route.py` + +**Interfaces:** +- Consumes: `export_service.collect_annotations`, `hf_connector.build_records/push_dataset`. +- Produces: `POST /api/experiments/{id}/export` (`require_control`) — body `{confirm: true}`; 400 if `confirm` is not true (approval gate); 503 if `HF_TOKEN` absent (via `hf_connector.token_present()`); returns the push summary. Mock `push_dataset` via monkeypatch in the test. + +- [ ] **Step 1: Write the failing test** (mirror `tests/test_eln_ground_truth_route.py` harness): + +```python +def test_export_requires_confirm(monkeypatch): + # build client (agent.store has get_ground_truth); POST without confirm → 400 + ... +def test_export_requires_token(monkeypatch): + # token_present() False → 503 + ... +def test_export_pushes_on_confirm(monkeypatch): + # monkeypatch hf_connector.push_dataset; confirm:true → 200 + summary + ... +``` + +- [ ] **Step 2–5:** implement route (assemble rows → `build_records` → `push_dataset`), green, commit `feat(eln): approval-gated HF export route`. + +--- + +## Task 8: Agent tools (create_experiment / author_hypothesis / record_result / link_session_to_arm / configure_export) + +**Files:** +- Modify: the agent tool registry (`gently/harness/tools/…` — follow the existing tool-registration pattern used by campaign/plan tools) + `gently/harness/prompts/manager.py` allow-list. +- Test: `tests/test_eln_agent_tools.py` + +**Interfaces:** +- Each tool is a thin wrapper over the corresponding `FileContextStore` method, stamping `author="agent"`. Mirror an existing store-backed tool (e.g. the campaign-creation tool) for schema + registration. + +- [ ] **Step 1:** test each tool calls the right store method with `author="agent"`. +- [ ] **Step 2–5:** implement, register in the allow-list, green, commit `feat(eln): agent tools for the scientific spine`. + +--- + +## Task 9: Vitals stage-strip ground-truth authoring UI (range drag-select) — IN-BROWSER + +**Files:** +- Modify: `gently/ui/web/static/js/…` (the Vitals / stage-strip view) + CSS. +- Verify: Chrome MCP + a new `US-` story flow (Task 12). + +**Interfaces:** +- Consumes: `POST /api/embryos/{id}/ground_truth` (Task done). DOM contract: a `ground_truth_control` element on the stage strip; corrected points render as a filled diamond vs the model's hollow dot. + +- [ ] **Step 1:** replace the localStorage-only Agree/Disagree with a stage-picker popover on click/drag-select over the stage strip; pre-fill the agent's predicted stage (one-click "confirm as ground truth"). +- [ ] **Step 2:** drag-select a range → one `POST` with `start_timepoint`/`end_timepoint`. +- [ ] **Step 3:** render corrected points distinctly; show annotator. +- [ ] **Step 4: Verify in-browser** — boot viz, drive the flow, run the Chrome MCP UI/UX audit (alignment/spacing/overflow/contrast); fix flaws before marking done. +- [ ] **Step 5: Commit** `feat(eln): ground-truth range authoring on the Vitals stage strip`. + +> This task requires the running app; do not mark done without in-browser verification. Design cues in `docs/product-ideation/CONSULT.md` (IDEA-01 deep design) + `ELN-USAGE.md`. + +--- + +## Task 10: Assisted batch-confirm + accuracy readout — IN-BROWSER + +**Files:** JS/CSS for the stage strip + a small viz route `GET /api/embryos/{id}/accuracy` (calls `accuracy.accuracy_result`). + +- [ ] **Step 1:** `GET /api/embryos/{id}/accuracy` returns the accuracy Result (predicted vs GT) — unit-test like the GT route. +- [ ] **Step 2:** agent pre-screens: bulk-propose high-agreement stretches (one range confirm); surface only uncertain frames for the human. (Backend: reuse `accuracy.score_accuracy` to find agreement runs.) +- [ ] **Step 3:** show a live "agrees M/N" readout + annotated-count that updates at authoring. +- [ ] **Step 4: Verify in-browser** (Chrome MCP audit). +- [ ] **Step 5: Commit** `feat(eln): assisted batch-confirm + live accuracy readout`. + +--- + +## Task 11: Experiment inspector + export button — IN-BROWSER + +**Files:** viz template + JS in the campaign navigator/inspector; `GET /api/experiments` + `GET /api/experiments/{id}` read routes (unit-tested). + +- [ ] Render experiments (arms, replicates, member sessions, linked hypotheses + status, export config). Export button → `POST /api/experiments/{id}/export` with an explicit confirm dialog. Verify in-browser. Commit `feat(eln): experiment inspector + approval-gated export button`. + +--- + +## Task 12: ELN US- story flows (after #76 lands the crawler on the base) + +**Files:** `tools/ui_crawler/stories/US-*.py` (contract: `META` + `async def flow(page, url, rec)`); update `tools/ui_crawler/baseline/status.json`. + +- [ ] Add the flows enumerated in `docs/product-ideation/ELN-USAGE.md` (story-flow specs): `annotate-ground-truth`, `create-experiment`, `link-session-to-arm`, `author-hypothesis`, `record-result`, `configure-hf-export`, `accumulate-dataset-count`, `strain-record`. Run `run_stories.py --update-baseline`, commit. (Blocked until the `tools/ui_crawler` harness is present on this branch's base — i.e. after PR #76 merges into #72.) + +--- + +## Self-Review + +- **Spec coverage:** Strain (done + Task 4 resolution), Experiment (Task 1), Hypothesis (Task 2), Result (Task 3 + done accuracy), assisted scoring loop (done accuracy + Task 10), HF connector (done + Tasks 6–7), seed programs (done + Task 5 loader), co-authored agent tools (Task 8), UI (Tasks 9–11), story flows (Task 12). All spec sections map to a task. +- **Placeholder scan:** Backend tasks (1–7) carry complete code; UI tasks (9–11) are explicitly flagged IN-BROWSER with concrete endpoints, DOM contract, and verification gates rather than blind JS — the honest boundary for unattended work. +- **Type consistency:** store methods return `dict|None` / `list[dict]`; `create_*` return `str` ids; `resolve_strain`/`resolve_ref` bare-string contract consistent across Task 4 and the shipped Strain store. diff --git a/docs/superpowers/specs/2026-07-02-eln-scientific-spine-design.md b/docs/superpowers/specs/2026-07-02-eln-scientific-spine-design.md new file mode 100644 index 00000000..bd061ae0 --- /dev/null +++ b/docs/superpowers/specs/2026-07-02-eln-scientific-spine-design.md @@ -0,0 +1,226 @@ +# ELN Scientific Spine — Strain → Experiment → Hypothesis → Result — Design + +Date: 2026-07-02 +Status: Approved (design) — implementation planned via writing-plans +Branch: feature/eln-scientific-spine (off #72; PR back into #72) +Origin: `docs/product-ideation/ENTITY-EVOLUTION.md` (Fable's entity-evolution assessment) + Hari Shroff's *Plan for Agentic Microscopy Paper* (6/3/26). + +## Problem + +gently lets a biologist **drive the hardware** but only **watch the mind**: the agent +holds a near-monopoly on authoring the shared record; the human's only write-path is +dictating to chat. The data model has an **empty scientific middle** — `Campaign` is a +program, `Session` is one run, but nothing expresses "these 6 sessions are 3 replicates +of two arms of one comparison testing a claim." The agent forms `Expectation`s (with +`expected_time` + auto-resolvable status; `gently/harness/memory/_understanding.py`) and +logs `Learning`s, but nothing states the **claim** they bear on or whether it +**survived**. And the most-referenced scientific thing — `strain` — is a bare string on +`ImagingSpec.strain/genotype/reporter` (`gently/harness/memory/model.py:200`), +`BenchSpec`, `Note.strains[]`, `ground_truth` (a partial `EmbryoInfo.strain` + +`tests/test_embryo_strain.py` already exist). + +This spec adds a thin **scientific spine** that turns data-collection into +claim-testing and gives the human a first-class, co-authored write-path. + +## Goals + +- **Strain** record (genotype, markers, stock, organism ref) that everything resolves + to — with bare-string back-compat. +- **Experiment** — the controlled-comparison unit (arms × replicates × controls) + between Campaign and Session; groups Sessions into an analyzable design. +- **Hypothesis** — a falsifiable claim with status, consolidating the diffuse + Expectation/Question/Learning triad around a testable statement. +- **Result** — a quantitative/derived finding (distinct from a prose Learning) with + method + provenance; the object a figure or paper reads. +- Close an **assisted claim-testing loop** on the *existing* `Expectation.resolve` + + `ground_truth` (agent proposes, human confirms). +- Co-authored: the human **or** the agent can create/edit any of these. + +## Primary near-term use case: in-ELN perception annotation → HuggingFace + +The active driver is the **embryo stage-classification** project. `gently-perception` +(Anthropic-built perceiver, a separate pip dependency) runs on live embryos and emits +per-timepoint stage predictions (`predictions.jsonl` + `traces/`). Today, turning those +into training data happens in a **separate, disconnected annotator** (`gently/dataset/` +— `aggregator.py`, `embryo_dataset.py`, `explorer_server.py`, `schema.py`; +`stage_annotator.py`; see `docs/PERCEPTION_V2_IMPROVEMENT_STRATEGY.md`) — the human's +feedback lives *outside* the live session. + +The spine's **ground-truth authoring** is the in-ELN mechanism for human feedback to the +data, and it should **converge** that annotator into the session surface rather than +duplicate it. The flywheel: + +1. **Predict** — `gently-perception` emits stage predictions on live embryos. +2. **Annotate in-ELN (assisted-human-in-the-loop)** — on the Vitals stage strip the + human corrects/confirms ground truth by **range** (drag-select); the **agent + pre-screens** — bulk-proposes high-agreement stretches as ground truth (one range + confirm) and surfaces ONLY the uncertain/transitional frames for the human to decide. + The agent narrows; the human confirms. Persists via `set_ground_truth(stage, + timepoint, annotator)` (`core/file_store.py:1255`), stamped with annotator + + provenance (the projection/trace it was judged from). +3. **Score** — a `Result` of `kind: accuracy` computes predicted-vs-ground-truth per + run; the accuracy readout, annotated-count, and few-shot pool update at authoring. +4. **Export → HuggingFace** — see the connector below. + +### The HuggingFace connector (approval-gated) + +An **Experiment** can be **configured with an export target** — a HuggingFace dataset +repo (default `pskeshu/gently-perception-benchmark`). On the human's **approval**, the +accumulated {projection/volume ref, model prediction, human ground truth, provenance, +strain/organism} formats into the dataset schema (reuse `gently/dataset` aggregator / +`benchmarks/perception/ground_truth.py`) and pushes via `huggingface_hub` / +`datasets.push_to_hub`. Design constraints: +- **Approval gate:** a push is a specimen-external publish — it uses the same + assisted-approval posture as hardware actions (`require_control` + an explicit + confirm; never automatic). The agent may *prepare* the export and *propose* the push; + the human approves. +- **Token from env:** `HF_TOKEN` (assume present in production). Absent token ⇒ the + connector disables with a clear "set HF_TOKEN" state (never a silent failure). +- **Config lives on the Experiment:** `export = {target: "hf", repo, split, revision}`. +- **Incremental + provenant:** each push records what was exported (which annotations, + which revision) so re-pushes are diffs, and the dataset carries provenance back to the + session/embryo/annotator. +- **Companion detail:** the exact dataset feature-schema + the migration of the + standalone `gently/dataset` tooling is an implementation detail carried in the plan; + this spec fixes the contract (approval-gated, env-token, per-experiment target). + +This makes the loop bidirectional: the model's live calls become the human's annotation +queue; the human's corrections become the model's next training set — without leaving +the notebook. + +## Seed / default programs (real data, version-controlled) + +Ship the repo with **real research programs** (not stubs) as seed `Campaign → +Experiment → Hypothesis` instances under `seed/programs/`, loadable into a fresh gently. +Purpose: (a) shape a *reusable* gently by building against actual cases; (b) real +notebook data to work with instead of stubs; (c) development seriousness. They are the +spine's dogfood + its best validation. The four (from Hari's paper plan + collaborator +notes): + +1. **Embryo stage classification** — nuclear (+ bright-field fusion) staging with + `gently-perception`; hatching-detection subproblem (high-cadence to catch hatching). + The perception-annotation→HuggingFace flywheel above is its engine. +2. **Temperature → hatching-time distributions (K-pump thermo strain)** — Richard's + strain targets the potassium pumps so that at **25 °C the embryo stops twitching** + (3-fold stage). The agent uses **thermo-cycling** creatively (paralysis-free imaging + windows) and studies hatching-time *distributions* and how imaging/thermalization + perturb them. Exercises the temperature setpoint + (future) Actuator axis. +3. **Mutant screen** — mixed WT + mutant populations; the agent reports which embryo is + which. Exercises multi-embryo roles + classification + reporting. +4. **Dopaminergic neuronal outgrowth ("catching outgrowth")** — the novel target + (Richard/`proposals/janelia/danienella`): image the *developing* neuron, not its + final state. `dat1` onset (~3-fold) marks the dopaminergic neuron and opens the + outgrowth window; the agent runs **low-cadence monitor until the signal appears, then + switches to high-cadence** (expression-triggered differential acquisition; + `gently/app/detectors/dopaminergic_signal.py` already exists), quantifying outgrowth + timing toward a dependency-map of neural-architecture development. Test case: an early + slow ventral-nerve-cord neuron vs a later `dat1` neuron. + +Real strains available in-repo/notes (e.g. OH904 `rab-3p::GFP` pan-neuronal, otIs355, +unc-6(ev400) crosses; the K-pump thermo strain). Seed programs instantiate real Strain +records referencing these. + +## Non-goals (YAGNI) + +- `Setpoint → typed Actuator` generalization — its own later spec (the temperature seed + program will motivate it). +- `Embryo → Specimen` rename — `SAMPLE_TERM` already covers the human-facing need. +- Modality/Device capability layer; multi-tenant (Team/Permission/Booking) — deferred to + the cloud-venture track. +- Re-inventing the organism model — it is already the `gently.organisms` plugin; Strain + references it. +- Full migration of the standalone `gently/dataset` annotator — the ELN owns in-session + annotation + the export trigger; the export pipeline + tool migration are planned + detail, not this design. + +## Entities & schemas + +File-based under `agent/` (`FileContextStore` conventions; no SQLite). **All fields +optional unless (req).** Every entity is an **optional overlay** — nothing in the +existing flow requires one. + +**Strain** — `agent/strains/{id}.yaml`: `id`, `name` (req), `genotype`, `markers[]`, +`background`, `source_lab`, `organism_ref` (→ `gently.organisms` key), `stock` +{`thawed`,`frozen`,`notes`}, `created_at`, `author`. `ImagingSpec.strain/genotype/reporter`, +`BenchSpec.strains[]`, `Note.strains[]`, `ground_truth`, and `EmbryoInfo.strain` gain +resolution to a Strain `id`; an unrecognized string stays valid (display-only). No forced +migration. + +**Experiment** — `agent/experiments/{id}.yaml`: `id`, `title` (req), `campaign_ref`, +`hypothesis_refs[]`, `arms` [{`name`, `strain_ref`, `condition`, `session_ids[]`}], +`controls[]`, `replicate_of`, `export` {`target`,`repo`,`split`,`revision`}, `notes`, +`status`, `created_at`, `author`. Reuses the campaign hierarchy + session-link plumbing +(`FileContextStore.get_sessions_for_campaign`, `get_campaign_ids_for_session`). A Session +belongs to zero or one Experiment arm. + +**Hypothesis** — `agent/hypotheses/{id}.yaml`: `id`, `statement` (req), `status` +(`proposed`|`supported`|`refuted`|`inconclusive`), `predictions` [{`target`,`expected`, +`expectation_ref?`}], `experiment_refs[]`, `author` (`human`|`agent`), `basis[]`, +`created_at`. + +**Result** — `agent/results/{id}.yaml`: `id`, `kind` (`measurement`|`derived`| +`accuracy`), `value`|`table`, `method`, `inputs[]` (sessions/embryos/predictions/ +ground_truth), `experiment_ref`, `hypothesis_ref`, `status` (`draft`|`final`), +`provenance` {`author`,`ts`,`derived_from`}. + +## Data flow — the assisted scoring loop + +When an Experiment's Sessions complete **and** a bound `Expectation`'s `expected_time` +passes (or `ground_truth` lands), the **agent computes a candidate `Result`** (hatch-rate +per arm, transition timing, predicted-vs-GT accuracy) and **proposes a Hypothesis +verdict**; both surface for the human to **confirm or edit**. Confirmed Results persist + +may promote to a calibrated `Learning`. Manual authoring + verdicts always available — +automation assists, never decides. + +## Authorship (co-authored) + +Human and agent are symmetric writers. Human: forms/composers + contextual seeds. Agent: +new tools (`create_experiment`, `author_hypothesis`, `record_result`, +`link_session_to_arm`, `configure_export`) alongside the campaign/plan tools. `author` +stamped on every record. + +## UI surfaces + +Reuse the Plans/Operations family — no new top-level tab unless it earns one. +**Experiments** in the campaign navigator/inspector (arms, replicates, member sessions, +hypotheses + status, export config + push button). **Strain** a light record view + +clickable chip everywhere strain appears (reverse links). **Hypothesis + Result** on the +Experiment + cross-linked into the Notebook (a confirmed verdict posts a FINDING note). +**Ground-truth authoring** on the Vitals stage strip (range drag-select + assisted +batch-confirm). + +## Error handling / edges + +Bare-string strain stays valid; optional membership (orphan Session = today's behavior); +cold-start (zero entities ⇒ identical to now); incomplete-input Result stays `draft`; +deleting a referenced Strain dereferences to display-string (no cascade); missing +`HF_TOKEN` disables the connector with a clear state. + +## Testing + +- Store round-trip + back-compat unit tests per entity; a scoring test on fixtures + (Expectation + ground_truth → candidate Result → proposed verdict); HF connector + tested with a mocked `push_to_hub` + a token-absent path. +- **New `US-` story flows** (the repo's audit paradigm, once the `tools/ui_crawler` + harness lands on the base via #76): `strain-record`, `create-experiment`, + `link-session-to-arm`, `author-hypothesis`, `record-result`, `annotate-ground-truth`, + `configure-hf-export` — each a verdict + trace + screenshot in the baseline-diff suite. + +## Phasing (per Fable's sequencing) + +1. **Strain record** + back-compat resolution (keystone). +2. **Experiment** (arms/replicates/controls; Session grouping; export config). +3. **Hypothesis + Result** + the assisted scoring loop. +4. **HF connector** (approval-gated push) — can co-develop with the annotation UI. + +**The annotation flywheel can lead.** The perception-annotation loop (ground-truth +range-authoring + assisted batch-confirm + `accuracy` Result + a first HuggingFace +export) needs only Embryo/Prediction/`ground_truth` — independent of Strain/Experiment — +and is the active project, so it can be built first. Strain→Experiment later enriches it +with per-strain / per-experiment dataset slices. + +## Open items + +- Experiments' own nav entry vs inside Plans (decide in phase-2 UI). +- The agent's candidate-Result computation set (phase-3 detail). +- Exact HF dataset feature-schema + `gently/dataset` convergence (planned detail). diff --git a/docs/superpowers/specs/2026-07-02-planmode-eln-fold-study.md b/docs/superpowers/specs/2026-07-02-planmode-eln-fold-study.md new file mode 100644 index 00000000..a9e670a0 --- /dev/null +++ b/docs/superpowers/specs/2026-07-02-planmode-eln-fold-study.md @@ -0,0 +1,67 @@ +# Study: how Plan Mode and the ELN scientific spine fold together + +Date: 2026-07-02 · Fable subagents (1M context) · commissioned when the ELN detail +looked grafted onto the Plans tab. Decision record — grounds the next ELN work. + +## The tension + +Plan Mode and the ELN spine model the same scientific work from two directions, and the overlap is concentrated in exactly one object. Plan Mode (AgentMode.PLAN) is the FORWARD/PROSPECTIVE + EXECUTABLE layer: a typed, dependency-aware Campaign→PlanItem task graph with spec inheritance, decision gates, version history, a 6-view Plans tab, a full agent tool surface, and — the thing nothing else does — the ImagingSpec→hardware auto-configure path (active_plan_item_id → detect_embryos/start_timelapse). The ELN spine is the BACKWARD/RETROSPECTIVE + EVIDENTIAL layer: Strain (canonical record), Experiment (controlled comparison), Hypothesis (falsifiable claim + verdict), Result (typed finding + provenance), plus the perception→ground_truth→accuracy→HuggingFace flywheel. Three of the four ELN entities are genuinely additive (Strain normalizes bare strings; Hypothesis is the missing claim node the plan Doc already prints but cannot store; Result is the quantitative finding a figure reads). But Experiment is one-thing-modeled-twice: a controlled comparison (arms × strain × condition + controls, grouping sessions) is already expressible as a phase (child Campaign) of imaging PlanItems whose imaging_spec carries strain/condition, whose comparison_to encodes the arm relation, and whose session_ids[] already model an arm's runs. The current design instead builds Experiment as a PARALLEL object with a THIRD session-grouping store (arms[].session_ids[] alongside PlanItem.session_ids[] and the link_session_campaign table), FLAT storage keyed only by campaign_ref, and NO agent tools. That parallel-graph choice is the direct cause of the bolted-on render: because there is no shared node type, the UI physically cannot interleave experiments with the plan items they describe, so renderElnSection() can only be appended to the bottom of the Doc view. The tension is therefore not prospective-vs-retrospective (those genuinely complement) — it is whether the retrospective record earns a parallel object tree or should attach as native nodes/fields onto the plan tree that already exists. + +## Options considered + +### A. Deprecate Plan Mode (ELN becomes the only spine) _(migration cost: high)_ +- **What:** Retire AgentMode.PLAN, its prompt, its replace-the-hardware-tools tool set, and the Plans tab; make Experiment/Hypothesis/Result the record of what the lab does. +- **Changes:** Model: delete Campaign/PlanItem authoring path or freeze it; Experiment becomes the primary unit. UI: remove the 6-view Plans tab (campaigns.js). Tools: delete plan_mode/tools/planning.py etc. CRITICAL rebuild: ELN has no execution path, so the ImagingSpec→hardware auto-load (active_plan_item_id → detect_embryos/start_timelapse) would have to be reimplemented on Experiment/arms from scratch, along with the dependency graph, decision gates, spec inheritance, and version history that only Plan Mode has. +- **Trade-offs:** Non-starter. Destroys the ONLY layer that turns a scientific question into microscope configuration and reverts gently to the pre-plan-mode state the design doc names (question → 'start a timelapse', each session fresh, no cross-session tracking). ELN is half-built, read-only, and has no scheduling/spec/hardware model. Removes the working incumbent to promote the unfinished overlay. + +### B. ELN absorbs Plan (Experiment becomes the master, plan items become children) _(migration cost: high)_ +- **What:** Invert the hierarchy: Experiment sits above the plan tree; PlanItems become children of Experiments/arms; the ELN entity graph becomes the primary model. +- **Changes:** Model: re-parent Campaign→PlanItem under Experiment; rewrite active_plan_item_id resolution to route through Experiment→arm→item. Migrate the campaign hierarchy and all plan snapshots. Tools: rewrite every plan tool to author through Experiment. UI: rebuild the 6 views around Experiment as root. +- **Trade-offs:** Backwards against the codebase and the minimal-model principle. Makes the half-built, untooled, read-only retrospective layer the master of the fully-built, tooled, hardware-wired prospective layer. Maximum rewrite of the part that works; contradicts 'reuse existing objects, wire what exists.' + +### C. Plan absorbs ELN as native nodes + fields (fold; Experiment demoted to a phase-role) _(migration cost: medium)_ +- **What:** Plan Mode stays the single design substrate. Keep the ALREADY-BUILT Strain/Hypothesis/Result stores but re-home them as native attachments on Campaign/PlanItem/Expectation, authored by the SAME plan tools and rendered inline. Retire Experiment-the-parallel-object: a controlled comparison becomes a phase (child Campaign) with role=experiment whose arms ARE its imaging PlanItems (a light arm/condition field), with session membership from the ONE existing link_session_campaign source of truth. Strain becomes a resolved reference on ImagingSpec.strain. +- **Changes:** Model (gently/harness/memory/model.py): add Hypothesis + Result as nodes referencing Campaign/PlanItem; Hypothesis.predictions carry expectation_ref (reference, not a copy of target/expected); Result wired to promote into a Learning; add optional arm/condition + role=experiment on a phase; wire resolve_strain into ImagingSpec.strain/genotype/reporter, BenchSpec.strains[], Note.strains[], ground_truth. Drop arms[].session_ids[] as an independent store — derive membership from PlanItem.session_ids[] / get_sessions_for_campaign. Keep export config, hang it on the phase/campaign. Tools (gently/harness/plan_mode/tools/planning.py): add author_hypothesis, record_result, set_hypothesis_verdict, create_strain/resolve_strain, configure_export — the missing surface. UI (gently/ui/web/static/js/campaigns.js): render each Hypothesis at its campaign/phase header and each Result on the PlanItem that produced it; delete renderElnSection() as a bottom-appended blob. Migrate seeds + the /api/campaigns/{id}/eln route + Experiment tests to the phase-role shape. +- **Trade-offs:** Honors the minimal-conceptual-model principle and keeps the genuinely-new value (Strain/Hypothesis/Result stores + the accuracy/HF flywheel are already built + tested and are REUSED, not rebuilt). One shared node type makes the render interleave naturally, dissolving the bolt-on. Cost: the Experiment object (store method create_experiment, the /eln route, seed_loader, test_experiment_store.py) gets reworked/demoted — some already-built code changes role, and imaging PlanItems must be verified to cleanly carry arms×controls before Experiment.arms is dropped. + +### D. Coexist with crisp boundaries (keep both trees; fix the seam + add tools) _(migration cost: medium)_ +- **What:** Keep Experiment/Hypothesis/Result as separate objects but define a clean contract: Plan = prospective/executable, ELN = retrospective/evidential, linked by ID. Fix the render by weaving via references, add the missing agent tools, and collapse session-grouping to one source of truth without re-parenting. +- **Changes:** Model: enforce references-not-copies (arms reference PlanItems/sessions via existing plumbing; Hypothesis.predictions.expectation_ref required; Result promotes to Learning); drop arms[].session_ids[], derive from link_session_campaign. Tools: add the 5 ELN tools onto the plan surface. UI: render ELN nodes anchored to their campaign_ref/item rather than appended at the bottom. +- **Trade-offs:** Lowest-disruption path that keeps ALL built code including Experiment. But it leaves TWO intermediate layers between Campaign and Session (phase child-Campaign vs Experiment) with undefined containment — the spec's own unresolved Open Item — so it preserves the near-duplicate object and risks re-accruing parallel-graph debt. It fixes the symptom (render, missing tools, triple session store) without resolving the root modeling redundancy the minimal-model principle targets. + +### E. Unify into one record-node model _(migration cost: high)_ +- **What:** Collapse plan and ELN into a single typed node that carries optional prospective spec (ImagingSpec) AND optional retrospective claim/finding (hypothesis/result) fields, over one reference+dependency graph. +- **Changes:** Rewrite model.py into a unified node schema; migrate all plan + ELN data into it; rewrite every plan and ELN tool; rebuild all 6 views + the campaign routes against the new node type. +- **Trade-offs:** Theoretically the cleanest, but a big-bang rewrite of a working incumbent and a brand-new abstraction — precisely what 'resist forced/new abstractions; wire what exists, simplest version first' warns against. Highest risk, largest blast radius, and unnecessary because C achieves the single-graph benefit incrementally by reusing the objects that already exist. + +## Recommendation + +**Option C — Plan Mode absorbs the ELN spine as native nodes + fields, with Experiment demoted from a parallel object to a phase-role over imaging PlanItems. Decouple and ship the perception→HuggingFace flywheel FIRST because it is model-agnostic (it needs only Embryo/Prediction/ground_truth, not Strain/Experiment), then land the fold on the design side.** + +### Rationale + +Three forces converge on C. (1) The minimal-conceptual-models principle: the fold audit shows three of four ELN entities are genuinely additive (Strain, Hypothesis, Result) and exactly one — Experiment — is one-thing-modeled-twice. A controlled comparison is already a phase (child Campaign) of imaging PlanItems whose imaging_spec carries strain/condition, whose comparison_to encodes arms, and whose session_ids[] already model an arm's runs; PLAN_MODE_DESIGN.md's own worked example builds controls as a sibling 'Matched controls' item. Adding Experiment as a parallel object with a THIRD session store and FLAT storage is the forced abstraction the principle says to resist. C reuses the plan tree and retires the duplicate. (2) What is already built favors reuse over rebuild: the Strain/Hypothesis/Result stores, accuracy scorer, HF connector, and seed loader are implemented AND tested (test_strain_store/test_hypothesis_store/test_result_store/test_eln_accuracy/test_hf_connector/test_seed_loader). C keeps all of that — it re-homes where those records attach and adds the missing agent tools; it does not throw work away. The only ELN code that changes role is Experiment (store method + /eln route + seeds + its test), which is the redundant part. Deprecate/absorb/unify (A/B/E) all discard or rewrite the working incumbent; D keeps the redundancy. (3) The paper + flywheel goals: Hari's agentic-microscopy paper plan and the spec both name the perception→ground_truth→accuracy→HF loop as the ACTIVE driver, and that loop is orthogonal to the plan-vs-ELN modeling decision. So the flywheel is not held hostage by the fold — ship it first as a Result(kind=accuracy) attached to the embryo/session with approval-gated HF push, while the design-side fold proceeds in parallel. C is also the only option that makes the spine co-authorable (the spec's core unmet goal — today there are zero ELN agent tools) by adding author_hypothesis/record_result/etc. onto the SAME plan tool surface, and it fixes the bolted-on render for free because a shared node type lets hypotheses/results interleave with the plan items they describe. C delivers the retrospective value (claim + verdict + typed finding + flywheel) at medium cost while keeping Plan Mode's irreplaceable prospective+executable half intact. + +## What to do now about the bolted-on rendering + +Treat the bolted-on renderElnSection() (campaigns.js:490, appended at line 463 after the phase blocks via a separate /api/campaigns/{id}/eln fetch) as a symptom of the parallel-graph modeling choice, not a CSS problem — do NOT invest in polishing it as a standalone panel. The correct fix is node-anchored inline rendering, which C enables: once Hypothesis attaches to a Campaign/phase and Result attaches to the PlanItem that produced it, render each Hypothesis at its campaign/phase header (the plan Doc already prints a 'Hypothesis:' line it currently cannot populate) and each Result inline on its item, and delete renderElnSection() as a bottom-appended blob plus its disconnected eln-* CSS. Immediate holding action if a reviewable PR is needed before the fold lands: relabel the trailing block as a single clearly-scoped, collapsible 'Scientific spine (preview)' card so it reads as intentional rather than broken — but do not add more eln-* styling; the block is deleted, not grown. First UI slice of the fold = converting that one fetch+append into per-node inline rendering keyed off campaign_ref/item id. + +## Sequencing + +1. Ship the flywheel first (model-agnostic, the paper's active driver): wire ground-truth range authoring on the Vitals stage strip (drag-select + agent batch-confirm of high-agreement stretches via set_ground_truth), compute a Result(kind=accuracy) from predictions.jsonl vs ground_truth (eln/accuracy.py, already built+tested), and the approval-gated HF push (eln/hf_connector.py). This needs only Embryo/Prediction/ground_truth — no Strain/Experiment — so it is independent of the fold and unblocks the stage-classification project immediately. +2. Make the spine co-authorable: add the missing ELN agent tools (author_hypothesis, record_result, set_hypothesis_verdict, create_strain, configure_export) onto the plan tool surface in gently/harness/plan_mode/tools/planning.py, backed by the existing file_store.py store methods. This closes the spec's biggest unmet gap (today zero ELN tools exist) without any modeling change. +3. Land Strain resolution (the keystone): wire resolve_strain into ImagingSpec.strain/genotype/reporter, BenchSpec.strains[], Note.strains[], and ground_truth, with unknown strings staying valid (already back-compat). Add clickable strain chips with reverse links. +4. Weave the render: convert renderElnSection() into per-node inline rendering — Hypothesis at its campaign/phase, Result on its PlanItem — and delete the bottom-appended block. This is the concrete death of the bolt-on. +5. Demote Experiment to a phase-role: model a controlled comparison as a phase (child Campaign, role=experiment) whose arms are its imaging PlanItems (light arm/condition field), with session membership from the single link_session_campaign / get_sessions_for_campaign source of truth. Migrate seed programs, the /api/campaigns/{id}/eln route, and test_experiment_store.py to the phase-role shape; drop arms[].session_ids[] as an independent store. +6. Enforce reference-not-copy invariants: Hypothesis.predictions.expectation_ref points at a real Expectation (never re-store target/expected); Result promotes into a calibrated Learning; relate (do not merge) decision_point gates and Hypothesis verdicts — a gate may consume a verdict. + +## Risks + +- Reworking already-built + tested Experiment code: the create_experiment store method, the /eln route, seed_loader, and test_experiment_store.py all assume the parallel object. Demoting it to a phase-role is partly throwaway work — be explicit that the Strain/Hypothesis/Result stores are KEPT and only Experiment changes role, so the medium cost is not mistaken for a rewrite. +- PlanItem overload: making imaging PlanItems double as experiment 'arms' assumes imaging_spec + comparison_to + an arm/condition field truly capture arms × strain × condition + controls. Verify against the seed programs (esp. the mutant screen and temperature-hatching arms) BEFORE deleting Experiment.arms, or a real comparison structure could be lost. +- Session-membership migration: link_session_to_arm today does not update the campaign link, so collapsing to the single link_session_campaign source of truth risks orphaning arm→session links already written by seeds. Migrate existing arm links in the same change that drops arms[].session_ids[]. +- Flywheel-first ordering couples Result(kind=accuracy) to a Result attachment point that phase 5 may move. Fix the Result's attachment (embryo/session/item) as stable up front so the flywheel does not need rework when Experiment is demoted. +- Losing the 'these 6 sessions = 3 replicates of 2 arms' readout: the retrospective clarity Experiment gave must be rendered just as clearly by the phase-role, or users perceive a regression. Make the phase-role view show arms/replicates/controls explicitly. +- PR scope creep: PR #77 is off #72; a full refold plus flywheel could balloon it. Split into a flywheel PR (steps 1-2, immediately valuable) and a fold PR (steps 4-6) to keep review tractable. +- HF push is a specimen-external publish: the approval gate must be airtight (require_control + explicit confirm, never automatic) and a missing HF_TOKEN must degrade to a clear disabled state, never a silent failure — regression here leaks data or fails invisibly. +- Decision-gate vs Hypothesis-verdict conflation: if step 6 merges them instead of relating them, the plan loses its workflow branch semantics (proceed/switch) or the science loses its data-borne verdict — keep them distinct nodes that reference each other. \ No newline at end of file diff --git a/gently/eln/__init__.py b/gently/eln/__init__.py new file mode 100644 index 00000000..c8baa7b2 --- /dev/null +++ b/gently/eln/__init__.py @@ -0,0 +1,5 @@ +"""Gently ELN — the scientific spine (Strain/Experiment/Hypothesis/Result) helpers + connectors. + +The Strain store lives in FileContextStore; this package holds the ELN logic that +reads from FileStore (predictions + ground_truth) — not the deprecated gently.dataset. +""" diff --git a/gently/eln/accuracy.py b/gently/eln/accuracy.py new file mode 100644 index 00000000..960b8cb9 --- /dev/null +++ b/gently/eln/accuracy.py @@ -0,0 +1,89 @@ +"""Perception accuracy scoring — the ELN flywheel's "Score" step. + +Compares gently-perception per-timepoint stage predictions against human-authored +ground truth (range-based: a stage spans [start_timepoint, end_timepoint]) and +produces an ``accuracy`` Result. Pure functions — no I/O — so fully unit-testable. +""" + +from __future__ import annotations + + +def stage_at_timepoint(ground_truth: list[dict], tp: int) -> str | None: + """The ground-truth stage covering timepoint ``tp``, or None. + + Ground-truth entries are ``{stage, start_timepoint, end_timepoint}`` where + ``end_timepoint`` None means open-ended (until the next stage / session end). + When ranges overlap, the latest-starting one wins (the more specific label). + """ + covering = None + best_start = -1 + for e in ground_truth: + s = e.get("start_timepoint") + if s is None: + continue + en = e.get("end_timepoint") + if tp >= s and (en is None or tp <= en) and s > best_start: + covering = e.get("stage") + best_start = s + return covering + + +def score_accuracy(predictions: list[dict], ground_truth: list[dict]) -> dict: + """Per-timepoint predicted-vs-ground-truth comparison. + + ``predictions``: ``[{timepoint, predicted_stage}]``; + ``ground_truth``: ``[{stage, start_timepoint, end_timepoint}]``. + Returns ``{n_scored, n_correct, accuracy, unscored, confusion}`` where + ``unscored`` counts predictions with no ground-truth coverage and ``accuracy`` + is None when nothing could be scored. + """ + n_correct = 0 + n_scored = 0 + unscored = 0 + confusion: dict[str, int] = {} + for p in predictions: + tp = p.get("timepoint") + pred = p.get("predicted_stage") + gt = stage_at_timepoint(ground_truth, tp) if tp is not None else None + if gt is None: + unscored += 1 + continue + n_scored += 1 + confusion[f"{gt}|{pred}"] = confusion.get(f"{gt}|{pred}", 0) + 1 + if gt == pred: + n_correct += 1 + accuracy = (n_correct / n_scored) if n_scored else None + return { + "n_scored": n_scored, + "n_correct": n_correct, + "accuracy": accuracy, + "unscored": unscored, + "confusion": confusion, + } + + +def accuracy_result( + session_id: str, + embryo_id: str, + predictions: list[dict], + ground_truth: list[dict], + run_id: str | None = None, + annotator: str | None = None, +) -> dict: + """Build an ``accuracy`` Result dict (matches the spine Result schema).""" + score = score_accuracy(predictions, ground_truth) + return { + "kind": "accuracy", + "value": score["accuracy"], + "table": score, + "method": "per-timepoint predicted-vs-ground-truth stage", + "inputs": { + "session_id": session_id, + "embryo_id": embryo_id, + "run_id": run_id, + "n_predictions": len(predictions), + "n_ground_truth": len(ground_truth), + }, + "provenance": {"author": annotator}, + "status": "draft" if score["n_scored"] == 0 else "final", + } diff --git a/gently/eln/export_service.py b/gently/eln/export_service.py new file mode 100644 index 00000000..e17437ce --- /dev/null +++ b/gently/eln/export_service.py @@ -0,0 +1,99 @@ +"""Assemble a per-embryo annotated timelapse into HuggingFace-ready rows. + +The flywheel's unit is one embryo's timelapse + its ground-truth annotations. This +pairs each prediction timepoint with the ground-truth stage covering it (range-based) +and emits rows ready for ``hf_connector.build_records``. Reads via the FileStore +accessors (``get_predictions`` returns dicts; ``get_ground_truth`` returns entries); +normalizes dict / dataclass / object so it's robust to either. +""" + +from __future__ import annotations + +from dataclasses import asdict, is_dataclass + +from gently.eln.accuracy import stage_at_timepoint + + +def _as_dict(x) -> dict: + if isinstance(x, dict): + return x + if is_dataclass(x) and not isinstance(x, type): + return asdict(x) + if hasattr(x, "__dict__"): + return dict(x.__dict__) + return {} + + +def _pred_stage(rec: dict): + findings = rec.get("findings") + return ( + rec.get("predicted_stage") + or rec.get("stage") + or (findings.get("stage") if isinstance(findings, dict) else None) + ) + + +def _annotator_at(gt: list[dict], tp): + """The annotator of the ground-truth range covering ``tp`` (latest-start wins).""" + if tp is None: + return None + best = None + best_start = -1 + for e in gt: + s = e.get("start_timepoint") + if s is None: + continue + en = e.get("end_timepoint") + if tp >= s and (en is None or tp <= en) and s > best_start: + best = e.get("annotator") + best_start = s + return best + + +def collect_embryo_annotations( + store, session_id: str, embryo_id: str, strain: str | None = None, annotated_only: bool = True +) -> list[dict]: + """Rows for one embryo's timelapse: each prediction paired with its ground-truth stage. + + ``annotated_only`` keeps only rows that have a human ground-truth label — the export + payload. Set False to include un-annotated predictions too. + """ + preds = store.get_predictions(session_id, embryo_id) or [] + gt = [_as_dict(e) for e in (store.get_ground_truth(session_id, embryo_id) or [])] + rows = [] + for p in preds: + rec = _as_dict(p) + tp = rec.get("timepoint") + gt_stage = stage_at_timepoint(gt, tp) if tp is not None else None + if annotated_only and gt_stage is None: + continue + rows.append( + { + "session_id": session_id, + "embryo_id": embryo_id, + "timepoint": tp, + "predicted_stage": _pred_stage(rec), + "ground_truth_stage": gt_stage, + "stage": gt_stage, # hf_connector.build_records falls back to `stage` + "annotator": _annotator_at(gt, tp), + "strain": strain, + "provenance": { + "session_id": session_id, + "embryo_id": embryo_id, + "source": "gently", + }, + } + ) + return rows + + +def annotation_summary(store, session_id: str, embryo_id: str) -> dict: + """``{n_predictions, n_annotated}`` for the per-embryo push affordance.""" + preds = store.get_predictions(session_id, embryo_id) or [] + gt = [_as_dict(e) for e in (store.get_ground_truth(session_id, embryo_id) or [])] + n_ann = 0 + for p in preds: + tp = _as_dict(p).get("timepoint") + if tp is not None and stage_at_timepoint(gt, tp) is not None: + n_ann += 1 + return {"n_predictions": len(preds), "n_annotated": n_ann} diff --git a/gently/eln/hf_connector.py b/gently/eln/hf_connector.py new file mode 100644 index 00000000..d5bbaa48 --- /dev/null +++ b/gently/eln/hf_connector.py @@ -0,0 +1,96 @@ +"""Approval-gated HuggingFace export connector for the ELN. + +Formats accumulated {image ref, model prediction, human ground truth, provenance, +strain} into dataset rows and pushes them to a HuggingFace dataset repo (default +``pskeshu/gently-perception-benchmark``). Design constraints (ELN spine spec): + +- **Approval-gated:** the CALLER (a ``require_control`` route + explicit human + confirm) decides to push; this module never auto-pushes. +- **Token from env** ``HF_TOKEN`` (assumed present in production). Absent → a clear + ``HFExportError``, never a silent failure. +- **Per-experiment target:** repo / split / revision are parameters. +- ``build_records`` is pure (no I/O) so it is fully unit-testable; the actual push + lazily imports ``datasets`` (not a base dependency) and accepts an injected + ``_push_fn`` for tests. +""" + +from __future__ import annotations + +import os + +DEFAULT_REPO = "pskeshu/gently-perception-benchmark" + + +class HFExportError(RuntimeError): + """Raised when an export cannot proceed (no token, no records, push failure).""" + + +def hf_token() -> str | None: + """The HuggingFace token from the environment, or None.""" + return os.environ.get("HF_TOKEN") or os.environ.get("HUGGING_FACE_HUB_TOKEN") + + +def token_present() -> bool: + return bool(hf_token()) + + +def build_records(annotations: list[dict]) -> list[dict]: + """Format annotation rows into flat, HF-ready dataset rows. Pure/no-I/O. + + Each input row carries the model prediction + the human ground truth + enough + provenance to trace back to the session/embryo/annotator (and strain). + """ + rows = [] + for a in annotations: + rows.append( + { + "session_id": a.get("session_id"), + "embryo_id": a.get("embryo_id"), + "image_ref": a.get("image_ref"), + "predicted_stage": a.get("predicted_stage"), + "ground_truth_stage": a.get("ground_truth_stage") or a.get("stage"), + "start_timepoint": a.get("start_timepoint"), + "end_timepoint": a.get("end_timepoint"), + "annotator": a.get("annotator"), + "strain": a.get("strain"), + "provenance": a.get("provenance") or {}, + } + ) + return rows + + +def push_dataset( + records: list[dict], + repo: str = DEFAULT_REPO, + split: str = "train", + revision: str | None = None, + token: str | None = None, + _push_fn=None, +) -> dict: + """Push formatted records to a HuggingFace dataset repo. + + APPROVAL-GATED: the caller MUST have obtained human confirmation first — this + function performs the publish, it does not decide to. ``token`` defaults to the + env ``HF_TOKEN``. ``_push_fn`` is a test injection point. Returns a summary; + raises ``HFExportError`` on no token / no records / push failure. + """ + token = token or hf_token() + if not token: + raise HFExportError( + "HF_TOKEN not set — set it in the environment to enable HuggingFace export." + ) + if not records: + raise HFExportError("no annotated records to export.") + if _push_fn is not None: + _push_fn(records, repo, split, revision, token) + else: + try: + from datasets import Dataset # lazy — not a base dependency + except ImportError as e: + raise HFExportError( + "the 'datasets' package is required for HuggingFace export " + "(pip install datasets huggingface_hub)." + ) from e + ds = Dataset.from_list(records) + ds.push_to_hub(repo, split=split, revision=revision, token=token) + return {"repo": repo, "split": split, "n": len(records), "revision": revision} diff --git a/gently/eln/seed_loader.py b/gently/eln/seed_loader.py new file mode 100644 index 00000000..30d4532e --- /dev/null +++ b/gently/eln/seed_loader.py @@ -0,0 +1,67 @@ +"""Load seed research programs (``seed/programs/*.yaml``) into a FileContextStore. + +Each program is imported as a Campaign + Strain(s) + Experiment(s) + Hypothesis(es). +Idempotency is the caller's concern — loading twice creates duplicates. +""" + +from __future__ import annotations + +from pathlib import Path + +import yaml + + +def load_program(store, program: dict) -> dict: + """Import one seed program dict; return the created entity ids.""" + c = program.get("campaign", {}) + campaign_id = store.create_campaign( + description=c.get("description") or c.get("title") or program.get("slug", "seed"), + shorthand=c.get("title"), + summary=c.get("goal"), + ) + strain_ids = [ + store.create_strain( + s["name"], + genotype=s.get("genotype"), + markers=s.get("markers"), + organism_ref=s.get("organism_ref"), + author="seed", + ) + for s in program.get("strains", []) + ] + experiment_ids = [ + store.create_experiment( + e["title"], + campaign_ref=campaign_id, + arms=e.get("arms"), + controls=e.get("controls"), + export={"target": "hf", "repo": e["export_repo"]} if e.get("export_repo") else None, + author="seed", + ) + for e in program.get("experiments", []) + ] + hypothesis_ids = [ + store.create_hypothesis( + h["statement"], + predictions=[{"target": p, "expected": None} for p in h.get("predictions", [])], + experiment_refs=experiment_ids, + author="seed", + ) + for h in program.get("hypotheses", []) + ] + return { + "campaign_id": campaign_id, + "strain_ids": strain_ids, + "experiment_ids": experiment_ids, + "hypothesis_ids": hypothesis_ids, + } + + +def load_seed_dir(store, seed_dir) -> list[dict]: + """Load every ``*.yaml`` program under ``seed_dir``. Returns per-program ids.""" + out = [] + for f in sorted(Path(seed_dir).glob("*.yaml")): + program = yaml.safe_load(f.read_text()) + if program and program.get("campaign"): + out.append(load_program(store, program)) + return out diff --git a/gently/eln/strain_resolution.py b/gently/eln/strain_resolution.py new file mode 100644 index 00000000..fa22a4b2 --- /dev/null +++ b/gently/eln/strain_resolution.py @@ -0,0 +1,27 @@ +"""Display-side strain resolution — non-mutating, bare-string safe. + +A strain reference that matches no record becomes a ``{"name": ref}`` shim, so the +UI always has something to show and no migration is forced. +""" + +from __future__ import annotations + + +def resolve_ref(store, ref): + """Resolve a strain ref to its record, or a ``{"name": ref}`` shim, or None.""" + if not ref: + return None + rec = store.resolve_strain(ref) + return rec if rec else {"name": ref} + + +def enrich_imaging_spec(store, spec_dict): + """Return a copy of an imaging-spec dict with a ``strain_record`` attached. + + Non-mutating: the input dict is untouched and the raw ``strain`` string stays. + """ + out = dict(spec_dict) + ref = out.get("strain") + if ref: + out["strain_record"] = resolve_ref(store, ref) + return out diff --git a/gently/harness/memory/file_store.py b/gently/harness/memory/file_store.py index a5a1580f..4305d401 100644 --- a/gently/harness/memory/file_store.py +++ b/gently/harness/memory/file_store.py @@ -136,6 +136,11 @@ def _ensure_dirs(self): "ml/pipelines", "ml/runs", "ml/assessments", + # ELN scientific spine (optional overlays) + "strains", + "experiments", + "hypotheses", + "results", ): (self.agent_dir / subdir).mkdir(parents=True, exist_ok=True) @@ -161,6 +166,211 @@ def _campaign_folder(self, campaign_id: str) -> Path | None: """Return the folder for a campaign, or None.""" return self._campaign_index.get(campaign_id) + # ------------------------------------------------------------------ + # Scientific spine — Strain (ELN, phase 1). Optional overlay: nothing in + # the existing flow requires a Strain record; bare strain strings still work. + # ------------------------------------------------------------------ + + def create_strain( + self, + name: str, + genotype: str | None = None, + markers: list | None = None, + background: str | None = None, + source_lab: str | None = None, + organism_ref: str | None = None, + stock: dict | None = None, + author: str | None = None, + strain_id: str | None = None, + ) -> str: + """Create (or overwrite by id) a Strain record at agent/strains/{id}.yaml.""" + sid = strain_id or self._gen_id() + now = self._now() + data = { + "id": sid, + "name": name, + "genotype": genotype, + "markers": list(markers or []), + "background": background, + "source_lab": source_lab, + "organism_ref": organism_ref, + "stock": stock or {}, + "author": author, + "created_at": now, + "updated_at": now, + } + self._write_yaml(self.agent_dir / "strains" / f"{sid}.yaml", data) + logger.info(f"Created strain {sid} [{name}]") + return sid + + def get_strain(self, strain_id: str) -> dict | None: + p = self.agent_dir / "strains" / f"{strain_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_strains(self) -> list[dict]: + d = self.agent_dir / "strains" + if not d.exists(): + return [] + return [s for s in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if s] + + def resolve_strain(self, ref: str | None) -> dict | None: + """Resolve a strain by id or (case-insensitive) name. + + Back-compat: a bare string that matches no record returns None, so + callers keep the raw string as a display value — no forced migration. + """ + if not ref: + return None + byid = self.get_strain(ref) + if byid: + return byid + rl = str(ref).strip().lower() + for s in self.list_strains(): + if str(s.get("name", "")).strip().lower() == rl: + return s + return None + + # ------------------------------------------------------------------ + # Scientific spine — Experiment (ELN, phase 2). Optional overlay. + # ------------------------------------------------------------------ + + def create_experiment( + self, + title: str, + campaign_ref: str | None = None, + arms: list | None = None, + controls: list | None = None, + replicate_of: str | None = None, + export: dict | None = None, + author: str | None = None, + experiment_id: str | None = None, + ) -> str: + eid = experiment_id or self._gen_id() + now = self._now() + norm_arms = [] + for a in arms or []: + norm_arms.append( + { + "name": a.get("name"), + "strain_ref": a.get("strain_ref") or a.get("strain"), + "condition": a.get("condition"), + "session_ids": list(a.get("session_ids") or []), + } + ) + data = { + "id": eid, + "title": title, + "campaign_ref": campaign_ref, + "hypothesis_refs": [], + "arms": norm_arms, + "controls": list(controls or []), + "replicate_of": replicate_of, + "export": export or {}, + "notes": None, + "status": "active", + "author": author, + "created_at": now, + "updated_at": now, + } + self._write_yaml(self.agent_dir / "experiments" / f"{eid}.yaml", data) + logger.info(f"Created experiment {eid} [{title[:50]}]") + return eid + + def get_experiment(self, experiment_id: str) -> dict | None: + p = self.agent_dir / "experiments" / f"{experiment_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_experiments(self) -> list[dict]: + d = self.agent_dir / "experiments" + if not d.exists(): + return [] + return [e for e in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if e] + + def link_session_to_arm(self, experiment_id: str, arm_name: str, session_id: str) -> bool: + e = self.get_experiment(experiment_id) + if not e: + return False + for arm in e.get("arms", []): + if arm.get("name") == arm_name: + if session_id not in arm.setdefault("session_ids", []): + arm["session_ids"].append(session_id) + e["updated_at"] = self._now() + self._write_yaml(self.agent_dir / "experiments" / f"{experiment_id}.yaml", e) + return True + return False + + # ------------------------------------------------------------------ + # Scientific spine — Hypothesis (ELN, phase 3). Optional overlay. + # ------------------------------------------------------------------ + _HYP_STATUSES = ("proposed", "supported", "refuted", "inconclusive") + + def create_hypothesis( + self, + statement: str, + predictions: list | None = None, + experiment_refs: list | None = None, + author: str | None = None, + hypothesis_id: str | None = None, + ) -> str: + hid = hypothesis_id or self._gen_id() + data = { + "id": hid, + "statement": statement, + "status": "proposed", + "predictions": list(predictions or []), + "experiment_refs": list(experiment_refs or []), + "author": author, + "basis": [], + "created_at": self._now(), + } + self._write_yaml(self.agent_dir / "hypotheses" / f"{hid}.yaml", data) + return hid + + def get_hypothesis(self, hypothesis_id: str) -> dict | None: + p = self.agent_dir / "hypotheses" / f"{hypothesis_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_hypotheses(self) -> list[dict]: + d = self.agent_dir / "hypotheses" + if not d.exists(): + return [] + return [h for h in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if h] + + def set_hypothesis_status(self, hypothesis_id: str, status: str) -> bool: + if status not in self._HYP_STATUSES: + raise ValueError(f"status must be one of {self._HYP_STATUSES}") + h = self.get_hypothesis(hypothesis_id) + if not h: + return False + h["status"] = status + self._write_yaml(self.agent_dir / "hypotheses" / f"{hypothesis_id}.yaml", h) + return True + + # ------------------------------------------------------------------ + # Scientific spine — Result (ELN, phase 3). Optional overlay. + # ------------------------------------------------------------------ + + def save_result(self, result: dict, result_id: str | None = None) -> str: + rid = result_id or result.get("id") or self._gen_id() + data = dict(result) + data["id"] = rid + data.setdefault("created_at", self._now()) + self._write_yaml(self.agent_dir / "results" / f"{rid}.yaml", data) + return rid + + def get_result(self, result_id: str) -> dict | None: + p = self.agent_dir / "results" / f"{result_id}.yaml" + return self._read_yaml(p) if p.exists() else None + + def list_results(self, experiment_ref: str | None = None) -> list[dict]: + d = self.agent_dir / "results" + if not d.exists(): + return [] + out = [r for r in (self._read_yaml(f) for f in sorted(d.glob("*.yaml"))) if r] + if experiment_ref is not None: + out = [r for r in out if r.get("experiment_ref") == experiment_ref] + return out + # ------------------------------------------------------------------ # Helpers # ------------------------------------------------------------------ diff --git a/gently/ui/web/routes/campaigns.py b/gently/ui/web/routes/campaigns.py index 7d5f6e62..1d9dd52e 100644 --- a/gently/ui/web/routes/campaigns.py +++ b/gently/ui/web/routes/campaigns.py @@ -105,6 +105,40 @@ async def get_campaign_tree(campaign_id: str): raise HTTPException(status_code=404, detail="Campaign not found") return tree + @router.get("/api/campaigns/{campaign_id}/eln") + async def get_campaign_eln(campaign_id: str): + """ELN scientific spine for a campaign: its experiments (arms × strain × + condition + controls), the strains they use, and the hypotheses they test.""" + cs = _get_store() + campaign = _resolve(cs, campaign_id) + cid = campaign.id + experiments = ( + [e for e in cs.list_experiments() if e.get("campaign_ref") == cid] + if hasattr(cs, "list_experiments") + else [] + ) + exp_ids = {e["id"] for e in experiments} + all_hyps = cs.list_hypotheses() if hasattr(cs, "list_hypotheses") else [] + hyp_by_exp: dict = {} + for h in all_hyps: + for er in h.get("experiment_refs") or []: + if er in exp_ids: + hyp_by_exp.setdefault(er, []).append(h) + strains: list = [] + seen: set = set() + for e in experiments: + for arm in e.get("arms") or []: + ref = arm.get("strain_ref") + rec = cs.resolve_strain(ref) if ref else None + arm["strain_name"] = rec["name"] if rec else ref + if rec and rec["id"] not in seen: + seen.add(rec["id"]) + strains.append(rec) + e["hypotheses"] = hyp_by_exp.get(e["id"], []) + # deduped campaign-level hypotheses (seed links each to all its experiments) + hypotheses = [h for h in all_hyps if exp_ids & set(h.get("experiment_refs") or [])] + return {"experiments": experiments, "strains": strains, "hypotheses": hypotheses} + @router.get("/api/campaigns/{campaign_id}/document") async def get_campaign_document(campaign_id: str): """Full plan as a structured document for the review page.""" diff --git a/gently/ui/web/routes/data.py b/gently/ui/web/routes/data.py index 01d334ac..fbecd013 100644 --- a/gently/ui/web/routes/data.py +++ b/gently/ui/web/routes/data.py @@ -100,6 +100,23 @@ def _require_agent_with_experiment(): raise HTTPException(status_code=503, detail="Agent not ready") return agent + def _gt_store(): + """Resolve a FileStore (with set_ground_truth) + the agent, if any. + + Annotation/export don't need the full agent — fall back to the viz's own + store so they work in benchmark / no-agent mode too. + """ + bridge = getattr(server, "agent_bridge", None) + agent = getattr(bridge, "agent", None) if bridge is not None else None + for cand in ( + getattr(agent, "store", None), + getattr(server, "gently_store", None), + getattr(server, "store", None), + ): + if cand is not None and hasattr(cand, "set_ground_truth"): + return cand, agent + return None, agent + @router.put("/api/embryos/{embryo_id}/position", dependencies=[Depends(require_control)]) async def update_embryo_position( embryo_id: str, @@ -150,6 +167,108 @@ async def update_embryo_position( logger.exception("Failed to publish OPERATOR_EDITED_EMBRYO") return emb.to_dict() + @router.post("/api/embryos/{embryo_id}/ground_truth", dependencies=[Depends(require_control)]) + async def set_embryo_ground_truth( + embryo_id: str, + body: dict = Body(...), # noqa: B008 + ): + """Author a ground-truth stage annotation over a timepoint range. + + The ELN annotation flywheel's write path: the human (or agent-assisted + batch-confirm) corrects/confirms the model's stage call, persisted via + FileStore.set_ground_truth with annotator provenance — replacing the old + localStorage-only Agree/Disagree dead-end. Range-based (start/end + timepoint) so a stretch can be confirmed in one write. + """ + store, agent = _gt_store() + if store is None: + raise HTTPException(status_code=503, detail="No ground-truth store available") + session_id = body.get("session_id") or getattr(agent, "session_id", None) + if not session_id: + raise HTTPException(status_code=400, detail="No active session for ground truth") + stage = body.get("stage") + if not stage: + raise HTTPException(status_code=400, detail="Body needs a stage") + try: + start_tp = int(body["start_timepoint"]) + except (KeyError, TypeError, ValueError): + raise HTTPException( + status_code=400, detail="Body needs an integer start_timepoint" + ) from None + end_raw = body.get("end_timepoint") + try: + end_tp = int(end_raw) if end_raw is not None else None + except (TypeError, ValueError): + raise HTTPException( + status_code=400, detail="end_timepoint must be an integer or null" + ) from None + store.set_ground_truth( + session_id, embryo_id, stage, start_tp, end_tp, body.get("annotator"), body.get("notes") + ) + return { + "ok": True, + "session_id": session_id, + "embryo_id": embryo_id, + "stage": stage, + "start_timepoint": start_tp, + "end_timepoint": end_tp, + "annotator": body.get("annotator"), + } + + @router.get("/api/embryos/{embryo_id}/annotation-summary") + async def embryo_annotation_summary(embryo_id: str, session_id: str | None = None): + """{n_predictions, n_annotated} for the per-embryo Push-to-HuggingFace affordance.""" + from gently.eln.export_service import annotation_summary + + store, agent = _gt_store() + sid = session_id or getattr(agent, "session_id", None) + if store is None or not sid: + return {"n_predictions": 0, "n_annotated": 0} + return annotation_summary(store, sid, embryo_id) + + @router.post("/api/embryos/{embryo_id}/export", dependencies=[Depends(require_control)]) + async def export_embryo_to_hf( + embryo_id: str, + body: dict = Body(...), # noqa: B008 + ): + """Approval-gated push of a per-embryo annotated timelapse to HuggingFace. + + The flywheel's terminal step: {prediction, human ground truth, provenance, + strain} rows -> the shared benchmark dataset. Gated three ways (explicit + confirm + require_control + HF_TOKEN); never automatic. + """ + from gently.eln import export_service, hf_connector + + if not body.get("confirm"): + raise HTTPException( + status_code=400, detail="export requires confirm=true (approval gate)" + ) + store, agent = _gt_store() + if store is None: + raise HTTPException(status_code=503, detail="No store available") + session_id = body.get("session_id") or getattr(agent, "session_id", None) + if not session_id: + raise HTTPException(status_code=400, detail="No active session") + if not hf_connector.token_present(): + raise HTTPException( + status_code=503, detail="HF_TOKEN not set — set it to enable HuggingFace export" + ) + rows = export_service.collect_embryo_annotations( + store, session_id, embryo_id, strain=body.get("strain") + ) + if not rows: + raise HTTPException(status_code=400, detail="no annotated timepoints to export") + records = hf_connector.build_records(rows) + try: + summary = hf_connector.push_dataset( + records, + repo=body.get("repo") or hf_connector.DEFAULT_REPO, + split=body.get("split", "train"), + ) + except hf_connector.HFExportError as e: + raise HTTPException(status_code=502, detail=str(e)) from e + return {"ok": True, "embryo_id": embryo_id, **summary} + @router.delete("/api/embryos/{embryo_id}", dependencies=[Depends(require_control)]) async def delete_embryo(embryo_id: str): """Remove an embryo from the experiment. diff --git a/gently/ui/web/static/css/campaigns.css b/gently/ui/web/static/css/campaigns.css index b874ab44..c6f2fc1a 100644 --- a/gently/ui/web/static/css/campaigns.css +++ b/gently/ui/web/static/css/campaigns.css @@ -1687,3 +1687,49 @@ } a { color: inherit; } } + +/* ── ELN scientific spine (experiments × arms × strains × hypotheses) ── */ +.eln-section { margin-top: 28px; } +.eln-block-title { + font-size: 0.72rem; font-weight: 700; text-transform: uppercase; letter-spacing: 0.05em; + color: var(--text-muted); margin: 20px 0 10px; display: flex; align-items: center; gap: 8px; +} +.eln-count { + font-size: 0.7rem; font-weight: 600; color: var(--accent); + background: rgba(96,165,250,0.12); border-radius: 999px; padding: 1px 8px; +} +.eln-strains { display: flex; flex-wrap: wrap; gap: 8px; } +.eln-strain-chip { + display: inline-flex; flex-direction: column; gap: 1px; + padding: 6px 12px; border: 1px solid var(--border); border-radius: 8px; background: var(--bg-card); +} +.eln-strain-name { font-weight: 600; font-size: 0.82rem; color: var(--text); } +.eln-strain-geno { font-size: 0.72rem; color: var(--text-muted); font-family: ui-monospace, monospace; } +.eln-experiment { + border: 1px solid var(--border); border-radius: 10px; padding: 14px 16px; + margin-bottom: 12px; background: var(--bg-card); +} +.eln-exp-title { font-weight: 600; font-size: 0.9rem; color: var(--text); margin-bottom: 10px; } +.eln-arms { display: flex; flex-direction: column; gap: 6px; margin-bottom: 10px; } +.eln-arm { + display: flex; align-items: baseline; gap: 10px; flex-wrap: wrap; + padding: 6px 10px; border-radius: 6px; background: var(--bg-hover); border-left: 3px solid var(--accent); +} +.eln-arm-control { border-left-color: var(--text-muted); } +.eln-arm-name { font-weight: 600; font-size: 0.8rem; color: var(--text); min-width: 90px; } +.eln-arm-strain { + font-size: 0.74rem; color: var(--accent); font-weight: 600; + background: rgba(96,165,250,0.1); border-radius: 4px; padding: 1px 6px; +} +.eln-arm-cond { font-size: 0.76rem; color: var(--text-muted); flex: 1; } +.eln-hyps { display: flex; flex-direction: column; gap: 6px; } +.eln-hyp { display: flex; align-items: baseline; gap: 8px; } +.eln-hyp-status { + font-size: 0.66rem; font-weight: 700; text-transform: uppercase; + border-radius: 4px; padding: 2px 6px; white-space: nowrap; +} +.eln-hyp-proposed { color: var(--accent); background: rgba(96,165,250,0.12); } +.eln-hyp-supported { color: var(--accent-green); background: rgba(74,222,128,0.12); } +.eln-hyp-refuted { color: #f87171; background: rgba(248,113,113,0.12); } +.eln-hyp-inconclusive { color: var(--accent-orange); background: rgba(251,146,60,0.12); } +.eln-hyp-text { font-size: 0.8rem; color: var(--text); line-height: 1.4; } diff --git a/gently/ui/web/static/css/main.css b/gently/ui/web/static/css/main.css index 2c7751aa..e4d3a908 100644 --- a/gently/ui/web/static/css/main.css +++ b/gently/ui/web/static/css/main.css @@ -4084,6 +4084,28 @@ kbd { border-right: 1px solid var(--border); } +.vitals-hf { + display: flex; + flex-direction: column; + align-items: center; + gap: 3px; + margin-top: 2px; +} +.vitals-annic { font-size: 0.62rem; color: var(--text-muted); text-align: center; } +.vitals-hf-btn { + font-size: 0.62rem; + font-weight: 600; + padding: 2px 8px; + border-radius: 5px; + border: 1px solid var(--accent); + color: var(--accent); + background: rgba(96, 165, 250, 0.1); + cursor: pointer; + white-space: nowrap; +} +.vitals-hf-btn:hover:not(:disabled) { background: rgba(96, 165, 250, 0.2); } +.vitals-hf-btn:disabled { opacity: 0.5; cursor: default; } + .vitals-name { font-weight: 700; font-size: 1rem; @@ -6765,6 +6787,66 @@ kbd { border-top: 1px solid var(--border); } +/* Ground-truth stage authoring (ELN annotation flywheel write path) */ +.gt-actions { + display: flex; + align-items: center; + gap: 0.5rem; + flex-wrap: wrap; + padding: 0.75rem 1rem; + background: var(--bg-hover); + border-top: 1px solid var(--border); + transition: background 0.3s ease; +} +.gt-label { + font-size: 0.72rem; + font-weight: 600; + text-transform: uppercase; + letter-spacing: 0.03em; + color: var(--text-muted); +} +.gt-stage-select { + padding: 0.4rem 0.55rem; + font-size: 0.8rem; + border-radius: 6px; + border: 1px solid var(--border); + background: var(--bg-card); + color: var(--text); + cursor: pointer; +} +.gt-stage-select:hover { border-color: var(--accent); } +.gt-confirm-btn { + padding: 0.45rem 0.85rem; + font-size: 0.8rem; + font-weight: 600; + border-radius: 6px; + border: 1px solid var(--accent-green); + background: rgba(74, 222, 128, 0.12); + color: var(--accent-green); + cursor: pointer; + transition: all 0.15s ease; +} +.gt-confirm-btn:hover { background: rgba(74, 222, 128, 0.22); } +.gt-badge { + display: inline-flex; + align-items: center; + gap: 0.25rem; + margin-left: auto; + padding: 0.3rem 0.6rem; + font-size: 0.76rem; + font-weight: 600; + border-radius: 999px; + background: rgba(74, 222, 128, 0.15); + color: var(--accent-green); + border: 1px solid var(--accent-green); +} +.gt-actions.gt-saved-flash { background: rgba(74, 222, 128, 0.16); } +.gt-actions.gt-need-control .gt-confirm-btn { + border-color: var(--accent-orange); + color: var(--accent-orange); + background: rgba(251, 146, 60, 0.12); +} + .vlm-action-btn { padding: 0.5rem 0.85rem; font-size: 0.8rem; diff --git a/gently/ui/web/static/js/campaigns.js b/gently/ui/web/static/js/campaigns.js index d42cdbd4..37d3e64c 100644 --- a/gently/ui/web/static/js/campaigns.js +++ b/gently/ui/web/static/js/campaigns.js @@ -278,6 +278,7 @@ async function openCampaign(campaignId) { await Promise.all([ loadDocument(campaignId), loadVersions(campaignId), + loadEln(campaignId), ]); // If we don't have campaigns list yet (direct URL), load in background @@ -458,6 +459,9 @@ function renderPlanDoc() { html += renderPhaseBlock(child, idx + 1); }); + // ELN scientific spine — experiments × arms × strains × hypotheses + html += renderElnSection(); + // Bibliography html += renderBibliography(); @@ -468,6 +472,82 @@ function renderPlanDoc() { $canvasContent.innerHTML = html; } +// ── ELN scientific spine (Strain / Experiment / Hypothesis) ────────────── +async function loadEln(campaignId) { + try { + const res = await fetch(`/api/campaigns/${encodeURIComponent(campaignId)}/eln`); + state.elnData = res.ok ? await res.json() : null; + } catch (e) { + state.elnData = null; + } +} + +function _clip(s, n) { + s = s || ''; + return s.length > n ? esc(s.slice(0, n)) + '…' : esc(s); +} + +function renderElnSection() { + const eln = state.elnData; + if (!eln) return ''; + const experiments = eln.experiments || []; + const strains = eln.strains || []; + if (experiments.length === 0 && strains.length === 0) return ''; + + let html = '
'; + + if (strains.length) { + html += `
Strains ${strains.length}
`; + html += '
'; + for (const s of strains) { + html += `
+ ${esc(s.name || s.id)} + ${s.genotype ? `${_clip(s.genotype, 48)}` : ''} +
`; + } + html += '
'; + } + + if (experiments.length) { + html += `
Experiments ${experiments.length}
`; + for (const e of experiments) { + html += `
${esc(e.title || e.id)}
`; + const arms = e.arms || []; + const controls = e.controls || []; + if (arms.length) { + html += '
'; + for (const a of arms) { + const isCtl = controls.includes(a.name); + html += `
+ ${esc(a.name || '')} + ${a.strain_name ? `${esc(a.strain_name)}` : ''} + ${a.condition ? `${_clip(a.condition, 140)}` : ''} +
`; + } + html += '
'; + } + html += '
'; + } + } + + const hyps = eln.hypotheses || []; + if (hyps.length) { + html += `
Hypotheses ${hyps.length}
`; + html += '
'; + for (const h of hyps) { + const st = h.status || 'proposed'; + html += `
+ ${esc(st)} + ${_clip(h.statement, 220)} +
`; + } + html += '
'; + } + + html += '
'; + return html; +} + function renderPhaseBlock(child, phaseNum) { const campaign = child.campaign; const items = child.items || []; diff --git a/gently/ui/web/static/js/devices.js b/gently/ui/web/static/js/devices.js index 47bf1605..56af7e34 100644 --- a/gently/ui/web/static/js/devices.js +++ b/gently/ui/web/static/js/devices.js @@ -2066,6 +2066,13 @@ const DevicesManager = (function () { return; } } + // Specimen-safety confirm: turning the laser ON delivers excitation + // light to live specimens (phototoxicity). Turning OFF never confirms + // — the safe direction needs no friction. + if (!window.confirm( + `Turn the laser ON (${config})?\n` + + `This delivers excitation light to live specimens.` + )) return; await setLaserPreset(config); } } @@ -2153,6 +2160,11 @@ const DevicesManager = (function () { if (!_lsTempInput) return; const target = parseFloat(_lsTempInput.value); if (isNaN(target) || target < 0 || target > 99.9) return; + // Specimen-safety confirm (see setTemperature): perturbs live embryos. + if (!window.confirm( + `Set water temperature to ${target.toFixed(1)} °C?\n` + + `This changes the thermal environment around live embryos.` + )) return; try { await fetch('/api/devices/temperature/set', { method: 'POST', @@ -2348,6 +2360,16 @@ const DevicesManager = (function () { setTimeout(loadTemperatureStatus, 1500); return; } + // Specimen-safety confirm: a setpoint change perturbs the thermal + // environment around live embryos — unlike the reversible map edits, + // this affects the biology, so gate it behind an explicit confirm. + if (!window.confirm( + `Set water temperature to ${target.toFixed(1)} °C?\n` + + `This changes the thermal environment around live embryos.` + )) { + loadTemperatureStatus(); + return; + } _tempBusy = true; _tempEl.classList.add('is-busy'); if (_tempSet) _tempSet.disabled = true; diff --git a/gently/ui/web/static/js/embryos.js b/gently/ui/web/static/js/embryos.js index 29340cf5..e38bc8da 100644 --- a/gently/ui/web/static/js/embryos.js +++ b/gently/ui/web/static/js/embryos.js @@ -800,6 +800,9 @@ const EmbryosManager = { html += '
'; container.innerHTML = html; + // Lazy-load per-embryo annotation counts for the Push-to-HF affordance. + this._loadHfSummaries(container); + // Click handlers on SVG data points container.querySelectorAll('.vitals-point').forEach(pt => { pt.addEventListener('click', (e) => { @@ -952,12 +955,62 @@ const EmbryosManager = { ${rate} ${eta} ${statusBadge} + + … + +
${svg}
`; }, + // ── Per-embryo HuggingFace push (annotation flywheel terminal step) ────── + async _loadHfSummaries(container) { + const sid = this.currentSessionId || ''; + const q = sid ? `?session_id=${encodeURIComponent(sid)}` : ''; + for (const span of container.querySelectorAll('.vitals-annic')) { + const eid = span.id.replace('hf-count-', ''); + try { + const res = await fetch(`/api/embryos/${eid}/annotation-summary${q}`); + if (!res.ok) continue; + const d = await res.json(); + span.textContent = `${d.n_annotated}/${d.n_predictions} annotated`; + const btn = span.parentElement.querySelector('.vitals-hf-btn'); + if (btn && !d.n_annotated) { btn.disabled = true; btn.title = 'No ground-truth annotations yet'; } + } catch (e) { /* leave placeholder */ } + } + }, + + async pushEmbryoToHf(embryoId) { + if (!window.confirm( + `Push ${embryoId}'s annotated timelapse to HuggingFace?\n` + + `(pskeshu/gently-perception-benchmark — this publishes the human ground truth + predictions.)` + )) return; + const span = document.getElementById(`hf-count-${embryoId}`); + const btn = span ? span.parentElement.querySelector('.vitals-hf-btn') : null; + const reset = (msg) => { + if (!btn) return; + btn.textContent = msg; + setTimeout(() => { btn.textContent = 'Push to HF'; btn.disabled = false; }, 2600); + }; + if (btn) { btn.disabled = true; btn.textContent = 'Pushing…'; } + try { + const res = await fetch(`/api/embryos/${embryoId}/export`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ confirm: true, session_id: this.currentSessionId }), + }); + if (res.status === 401 || res.status === 403) return reset('Log in'); + if (res.status === 503) return reset('Set HF_TOKEN'); + if (!res.ok) { console.debug('hf export failed:', await res.text()); return reset('Failed'); } + const d = await res.json(); + reset(`Pushed ${d.n} ✓`); + } catch (e) { console.debug('hf export error:', e); reset('Failed'); } + }, + // Header panel collapse state // ========================================== // State Persistence @@ -3262,7 +3315,7 @@ const EmbryosManager = { ` : ''} - ${detection.detected ? this.renderAgreeDisagreeButtons(detection, index) : ''} + ${detection.detected ? this.renderGroundTruthControl(detection, index) : ''} `; }, @@ -3293,6 +3346,92 @@ const EmbryosManager = { `; }, + // Ground-truth stage authoring (ELN annotation flywheel write path). + // Replaces the localStorage-only Agree/Disagree dead-end: the human confirms + // or corrects the model's stage call, persisted via POST /api/embryos/{id}/ + // ground_truth so it feeds accuracy + the HuggingFace export. + GT_STAGES: ['early', 'bean', 'comma', '1.5fold', '2fold', '3fold', 'pretzel', 'hatching', 'hatched'], + + renderGroundTruthControl(detection, index) { + const key = `${this.selectedEmbryoId}-${detection.timepoint}`; + const gt = this.groundTruths[key]; + const predicted = detection.stage || detection.predicted_stage + || (detection.findings && detection.findings.stage) || ''; + const chosen = gt ? gt.stage : predicted; + const opts = this.GT_STAGES.map(s => + `` + ).join(''); + const badge = gt + ? `✔ GT: ${gt.stage}` + : ''; + return ` +
+ Ground truth + + + ${badge} +
+ `; + }, + + async markGroundTruth(index, timepoint, stage) { + const embryoId = this.selectedEmbryoId; + if (!embryoId || timepoint == null || !stage) return; + const wrap = document.getElementById(`gt-sel-${index}`); + const container = wrap ? wrap.closest('.gt-actions') : null; + try { + const res = await fetch(`/api/embryos/${embryoId}/ground_truth`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + stage, + start_timepoint: timepoint, + session_id: this.currentSessionId, + }), + }); + if (res.status === 401 || res.status === 403) { + if (container) container.classList.add('gt-need-control'); + return; + } + if (!res.ok) { + console.debug('ground truth failed:', await res.text()); + if (container) { + container.classList.add('gt-need-control'); + const btn = container.querySelector('.gt-confirm-btn'); + if (btn) { + const prev = btn.textContent; + btn.textContent = 'Save failed'; + setTimeout(() => { + btn.textContent = prev; + container.classList.remove('gt-need-control'); + }, 1800); + } + } + return; + } + const data = await res.json(); + this.groundTruths[`${embryoId}-${timepoint}`] = { stage: data.stage, annotator: data.annotator }; + if (container) { + container.classList.remove('gt-need-control'); + let badge = container.querySelector('.gt-badge'); + if (!badge) { + badge = document.createElement('span'); + badge.className = 'gt-badge'; + container.appendChild(badge); + } + badge.innerHTML = `✔ GT: ${data.stage}`; + badge.title = 'Human ground truth' + (data.annotator ? ' by ' + data.annotator : ''); + const btn = container.querySelector('.gt-confirm-btn'); + if (btn) btn.textContent = 'Update'; + container.classList.add('gt-saved-flash'); + setTimeout(() => container.classList.remove('gt-saved-flash'), 900); + } + } catch (err) { console.debug('ground truth failed:', err); } + }, + // Render a verification event card for the reasoning panel renderVerificationCard(verification, index) { const isCompleted = verification.type === 'verification_completed'; @@ -3383,6 +3522,7 @@ const EmbryosManager = { // Track user agreement/disagreement with detections detectionAgreements: {}, // key: "{detector}-{timepoint}" -> true/false + groundTruths: {}, // key: "{embryo}-{timepoint}" -> {stage, annotator} (persisted GT) markAgreement(detectorName, timepoint, agrees) { const key = `${detectorName}-${timepoint}`; diff --git a/launch_gently.py b/launch_gently.py index 6afcbe86..a2fff1d2 100644 --- a/launch_gently.py +++ b/launch_gently.py @@ -232,6 +232,7 @@ async def main( no_browser: bool = False, no_api: bool = False, no_auth: bool = False, + seed: bool = False, ): # Set up log file in storage directory # Unified with FileStore: logs live under the same root as data @@ -546,6 +547,20 @@ def _status_provider(): agent_dir = storage_dir / "agent" context_store = FileContextStore(agent_dir) + # Optionally seed real research programs into an empty store so a fresh boot + # has real notebook content (Campaign/Strain/Experiment/Hypothesis) to work with. + if seed: + try: + from gently.eln.seed_loader import load_seed_dir + + if not context_store.get_root_campaigns(status=None): + seed_dir = Path(__file__).resolve().parent / "seed" / "programs" + loaded = load_seed_dir(context_store, seed_dir) + logger.info("Seeded %d research programs from %s", len(loaded), seed_dir) + else: + logger.info("--seed: store already has campaigns; skipping seed load") + except Exception: + logger.exception("seed load failed") agent.set_context_store(context_store) bridge.init_wizard(context_store=context_store, claude_client=agent.claude) @@ -665,6 +680,11 @@ def cli_main(): action="store_true", help="Do not auto-open the web UI in a browser", ) + parser.add_argument( + "--seed", + action="store_true", + help="Load the bundled seed research programs into the store if it has no campaigns", + ) args = parser.parse_args() # An API key is required unless running in UI-only mode. @@ -698,6 +718,7 @@ def cli_main(): no_browser=args.no_browser, no_api=args.no_api, no_auth=args.no_auth, + seed=args.seed, ) ) except (KeyboardInterrupt, RuntimeError, SystemExit): diff --git a/scripts/load_seed_programs.py b/scripts/load_seed_programs.py new file mode 100644 index 00000000..d7f344a0 --- /dev/null +++ b/scripts/load_seed_programs.py @@ -0,0 +1,54 @@ +"""Load the shipped seed research programs into a gently store. + +Loads ``seed/programs/*.yaml`` as real Campaign/Strain/Experiment/Hypothesis records +into ``{storage}/agent`` (the FileContextStore) so a fresh gently boots with real +notebook content instead of stubs. + +Usage: + python scripts/load_seed_programs.py # into the default storage + python scripts/load_seed_programs.py --storage /path # into a specific store +""" + +import argparse +import sys +from pathlib import Path + +_REPO = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(_REPO)) + +from gently.eln.seed_loader import load_seed_dir # noqa: E402 +from gently.harness.memory.file_store import FileContextStore # noqa: E402 + + +def main(): + ap = argparse.ArgumentParser(description="Load seed research programs into a gently store") + ap.add_argument("--storage", default=None, help="Storage base path (default: settings)") + args = ap.parse_args() + + if args.storage: + base = Path(args.storage) + else: + from gently.settings import settings + + base = settings.storage.base_path + + agent_dir = base / "agent" + store = FileContextStore(agent_dir) + loaded = load_seed_dir(store, _REPO / "seed" / "programs") + print(f"Loaded {len(loaded)} seed programs into {agent_dir}") + for rec in loaded: + camp = store.get_campaign(rec["campaign_id"]) + title = "" + if camp is not None: + title = (getattr(camp, "shorthand", None) or getattr(camp, "description", "") or "")[ + :60 + ] + print( + f" {rec['campaign_id']} {title} " + f"({len(rec['strain_ids'])} strains, {len(rec['experiment_ids'])} experiments, " + f"{len(rec['hypothesis_ids'])} hypotheses)" + ) + + +if __name__ == "__main__": + main() diff --git a/seed/programs/README.md b/seed/programs/README.md new file mode 100644 index 00000000..1786b584 --- /dev/null +++ b/seed/programs/README.md @@ -0,0 +1,14 @@ +# Seed research programs + +Real, version-controlled research programs (not stubs) that instantiate the ELN +scientific spine (Campaign → Strain → Experiment → Hypothesis). They (a) shape a +reusable gently by building against actual cases, (b) give real notebook data to work +with, (c) anchor development to the agentic-microscopy paper. From Hari Shroff's paper +plan + collaborator notes (Richard Ikegami, danienella). + +- **`stage-classification.yaml`** — Embryo developmental-stage classification — nuclear + bright-field fusion, with hatching detection +- **`temperature-hatching.yaml`** — Temperature → hatching-time distributions (K-pump thermo strain) +- **`mutant-screen.yaml`** — Mutant screen — blind genotype calling from live embryo imaging +- **`dopaminergic-outgrowth.yaml`** — Catching Outgrowth — Dopaminergic Neuronal Outgrowth in C. elegans + +A loader that imports these into a fresh gently instance is part of the ELN build (FileContextStore.create_strain/experiment/hypothesis). diff --git a/seed/programs/dopaminergic-outgrowth.yaml b/seed/programs/dopaminergic-outgrowth.yaml new file mode 100644 index 00000000..d52751da --- /dev/null +++ b/seed/programs/dopaminergic-outgrowth.yaml @@ -0,0 +1,186 @@ +# Seed research program: Catching Outgrowth — Dopaminergic Neuronal Outgrowth in C. elegans +# Real, version-controlled Campaign/Strain/Experiment/Hypothesis (ELN spine dogfood). +slug: dopaminergic-outgrowth +campaign: + title: Catching Outgrowth — Dopaminergic Neuronal Outgrowth in C. elegans + goal: Quantify the outgrowth of the dat-1 dopaminergic neuron in the developing embryo — onset time, + extension rate, duration, morphology — by using expression onset as the trigger to switch from low-cadence + monitoring to high-cadence capture, and contrast it against an early-born, slow ventral-nerve-cord + neuron, building toward a dependency-map of how neural architecture develops. + description: 'Outgrowth has never been caught — the literature images neurons at their final, wired-and-still + stage because the growing neuroblast does not hold still. This program images the DEVELOPING dopaminergic + neuron instead of its endpoint. The dat-1 dopamine-transporter reporter (dat-1p::GFP, strain BZ555) + turns on around the 3-fold stage, marking the CEP/ADE dopaminergic neurons and opening the window + in which their dendrite extension can be imaged. gently sits in a low-cadence, low-dose monitor mode + until the dopaminergic_signal.py two-stage detector (Perceiver->Classifier) reports the signal appearing + (intensity_level >= MEDIUM -> speed_up; structure_quality = GOOD -> burst), then switches to high + cadence to capture the outgrowth trajectory at the time resolution its dynamics demand. The test case + that shapes the design is a timing contrast: an early, slow, long-window ventral-nerve-cord (DD/GABAergic) + neuron versus the later dat-1 dopaminergic neuron — reconstructing the ordering (scaffold-then-follower) + that is the unit of a developmental dependency map. A real obstacle is built into the biology: the + 3-fold embryo twitches, which is exactly why nobody catches outgrowth at fine cadence; Richard''s + potassium-pump thermo-paralytic background (twitching stops at 25 C) is folded in as the motion-free + enabler, cross-linking the temperature seed program. This is the C. elegans first rung of Richard''s + ladder (worm -> mutant screen -> EM correlation -> zebrafish -> Danionella); the single deliverable + — outgrowth timing nobody has captured — is the proof the hammer builds houses.' +strains: +- name: BZ555 + what: Canonical, CGC-available dopaminergic reporter and the primary dat-1 strain for this program. + Cytoplasmic GFP driven by the dat-1 (dopamine transporter) promoter labels all 8 dopaminergic neurons + (4 CEP, 2 ADE, 2 PDE). Its onset around the 3-fold stage marks the neuron and opens the outgrowth + window — the appearance of this signal (via dopaminergic_signal.py) is the trigger that switches acquisition + from low cadence to high cadence. CEP dendrite extension along the nose is the outgrowth event captured. + genotype: egIs1 [dat-1p::GFP] + markers: + - 'dat-1p::GFP (dopaminergic; 8 neurons: CEP/ADE/PDE)' + organism_ref: c_elegans +- name: juIs76 + what: Early-born, slow-outgrowth ventral-nerve-cord reference (Jin lab, CGC-available). GFP driven by + the unc-25 (GAD) promoter labels the GABAergic DD/VD motor neurons; the embryonic DD neurons extend + commissures over a long developmental window, providing the 'early slow VNC neuron' contrast to the + later dat-1 dopaminergic neuron in the timing/ordering comparison. + genotype: juIs76 [unc-25p::GFP; lin-15(+)] + markers: + - unc-25p::GFP (GABAergic DD/VD ventral-nerve-cord motor neurons) + organism_ref: c_elegans +- name: dat-1p::GFP; unc-25p::mCherry dual-label (Richard — in construction) + what: 'Two-color line placing the early VNC (DD) neuron and the late dat-1 dopaminergic neuron in the + SAME embryo, so their outgrowth can be temporally ordered within one animal without cross-animal registration. + Being built by Richard (per notes: ''ventral nerve cord — Richard making the strain''). Until it is + available, the early-vs-late comparison runs the two single-color strains (juIs76, BZ555) as separate + arms staged to a common time-to-hatch clock.' + genotype: dat-1p::GFP; unc-25p::mCherry (target; cross of BZ555 x a red-VNC/GABAergic line) + markers: + - dat-1p::GFP (dopaminergic) + - unc-25p::mCherry (GABAergic VNC) + organism_ref: c_elegans +- name: dat-1p::GFP; K-pump thermo-paralytic (Richard — in construction) + what: dat-1 reporter crossed into Richard's potassium-pump thermo-paralytic background, which stops + embryonic twitching at 25 C (the 3-fold stage). This makes motion-free, high-cadence imaging of dopaminergic + dendrite outgrowth possible through the twitching-prone 3-fold window that otherwise defeats fine-cadence + capture. In construction; baseline experiments substitute BZ555 held at 25 C until the cross is available. + genotype: egIs1 [dat-1p::GFP]; (Richard/Janelia) + markers: + - dat-1p::GFP (dopaminergic) + - K-pump thermo-paralysis (25 C twitch suppression at 3-fold) + organism_ref: c_elegans +experiments: +- title: Exp-1 — Expression-triggered capture of dat-1 dopaminergic outgrowth + note: 'Core method demonstration: monitor-until-signal then high-cadence, driven by gently/app/detectors/dopaminergic_signal.py. + Low-cadence low-dose watch until the detector reports the signal, then speed_up/burst to catch CEP + dendrite extension; ramp_down_power on SATURATING; stop on has_hatched. The point is to catch the + outgrowth trajectory while spending photodose only once it matters.' + arms: + - name: Adaptive (monitor->burst) + strain: BZ555 + condition: 'Low-cadence monitor from 2-fold: 10 min interval, 488 nm @ 5%, ~40 slices. dopaminergic_signal.py + runs every timepoint; on intensity_level >= MEDIUM (speed_up) drop to 2 min cadence; on structure_quality + = GOOD (burst) fire 1 min bursts; ramp_down_power on SATURATING; stop on has_hatched. Captures dendrite + length-vs-time through hatch.' + - name: Fixed high-cadence (naive) + strain: BZ555 + condition: Constant 2 min cadence, 488 nm @ 10%, ~40 slices, from 2-fold to hatch, no trigger. Reference + trajectory and the cumulative photodose the adaptive arm must beat. + controls: + - 'Low-cadence-only (never switches): constant 10 min cadence — temporal-undersampling control; expected + to FAIL to resolve outgrowth rate, proving the trigger is necessary.' + - 'Minimal-imaging developmental control: one volume at 2-fold and one at hatch, otherwise dark — establishes + unperturbed hatch rate/timing to compare imaged arms against.' + - 'Detector false-onset control: autofluorescence-only / pre-onset frames plus a no-embryo field scored + by dopaminergic_signal.py — measures the false MEDIUM/GOOD rate (photodose wasted on noise).' + export_repo: pskeshu/gently-dopaminergic-outgrowth +- title: 'Exp-2 — Early-slow VNC vs late dat-1: outgrowth ordering (dependency map)' + note: 'Builds the ordering that a dependency map is made of: does the early VNC scaffold precede dopaminergic + extension? Two single-color arms now, staged to a common time-to-hatch clock; the dual-label same-embryo + arm activates when Richard''s strain lands, giving within-animal ordering.' + arms: + - name: Early VNC (DD) + strain: juIs76 + condition: Adaptive monitor->burst tuned to unc-25p::GFP; capture DD commissural outgrowth onset, + extension rate, and duration (early-born, slow, long window). + - name: Late dat-1 DA + strain: BZ555 + condition: Adaptive monitor->burst (as Exp-1 adaptive arm); capture dopaminergic outgrowth onset, + rate, and duration. + - name: Dual-label same-embryo + strain: dat-1p::GFP; unc-25p::mCherry dual-label (Richard — in construction) + condition: Two-color acquisition (488 + 561 nm) in one embryo for direct within-animal temporal ordering + of VNC vs DA outgrowth. Runs when the strain is available; until then ordering is inferred across + the two single-color arms via the common landmark clock. + controls: + - 'Common developmental clock: stage every embryo to time-before-hatch (plus the 2-fold landmark) so + cross-strain outgrowth times are comparable without the dual strain.' + - Per-strain minimal-imaging developmental control (hatch rate/timing) — confirm imaging does not shift + either neuron's schedule. +- title: Exp-3 — Paralysis-assisted outgrowth capture (thermo-immobilization) + note: '3-fold twitching is the specific reason outgrowth has never been caught at fine cadence. Cross-links + the temperature seed program: Richard''s K-pump strain stops twitching at 25 C, enabling motion-free + high-cadence trajectories. Separates temperature''s developmental speedup from any imaging perturbation + via hatching-time distributions.' + arms: + - name: 25 C thermo-paralysis + strain: dat-1p::GFP; K-pump thermo-paralytic (Richard — in construction) + condition: 'Hold 25 C across the outgrowth window to suppress 3-fold twitching; high-cadence (1-2 + min) motion-free capture of dopaminergic dendrite extension. Baseline until strain lands: BZ555 + held at 25 C.' + - name: 20 C twitching (control) + strain: BZ555 + condition: Standard 20 C, identical high-cadence — the twitch-corrupted baseline; quantifies the frame-to-frame + motion the paralysis arm removes. + controls: + - 'Thermal-only developmental control: unimaged embryos at 25 C vs 20 C — measure hatching-time distributions + to separate temperature''s developmental speedup from imaging perturbation.' + - 'Registration-error control: the same volume acquired back-to-back to bound the non-biological jitter + floor for tip-tracking.' +hypotheses: +- statement: dat-1p::GFP becomes detectable (dopaminergic_signal.py intensity_level >= MEDIUM) around + the 3-fold stage, and this expression onset reliably precedes and brackets the phase of maximal CEP + dendrite extension — making onset a usable trigger to open a high-cadence outgrowth-capture window. + predictions: + - In BZ555, the detector first reports MEDIUM intensity at ~3-fold, on the order of ~1-2 h before hatch + at 20 C, before structure_quality reaches GOOD. + - The adaptive (monitor->burst) arm captures the CEP dendrite length-vs-time trajectory at temporal + fidelity >= the fixed high-cadence arm. + - The low-cadence-only control fails to resolve the outgrowth rate (undersampled slope), confirming + the trigger is necessary to catch the dynamics. + note: Falsified if onset lags peak extension (dat-1 comes on only after outgrowth completes, consistent + with it being a terminal-differentiation gene) or if adaptive acquisition misses the trajectory — + either would force an earlier reporter. +- statement: The early ventral-nerve-cord (DD) neuron begins and completes its outgrowth earlier and over + a longer, slower window than the dat-1 dopaminergic neuron, placing the VNC scaffold before dopaminergic + process extension in the developmental dependency map. + predictions: + - DD (unc-25p::GFP) outgrowth onset precedes detectable dat-1 dopaminergic outgrowth, measured against + a common time-to-hatch clock. + - DD outgrowth duration > dat-1 dopaminergic outgrowth duration, and DD mean extension rate < dat-1 + rate (slow-and-long vs fast-and-late). + - In the dual-label same-embryo arm, VNC-before-DA ordering holds within individual animals, not only + across the population. + note: Refuted if dat-1 outgrowth precedes or co-occurs with DD outgrowth, or if durations/rates do not + separate — that would break the scaffold-then-follower assumption underlying the dependency map. +- statement: Expression-triggered differential acquisition (monitor->burst) captures dopaminergic outgrowth + dynamics as completely as constant high-cadence imaging while delivering substantially less pre-onset + photodose and less perturbation. + predictions: + - Cumulative 488 nm dose (integral of power x exposure x frames) in the adaptive arm is materially lower + than the fixed high-cadence arm, with most savings accrued before onset. + - Outgrowth rate and duration measured from the adaptive arm are statistically indistinguishable from + the fixed high-cadence arm (no dynamics lost). + - Hatch rate and outgrowth completion in the adaptive arm are >= the fixed high-cadence arm (less phototoxic + perturbation). + note: This is the method's core claim. Refuted if the adaptive arm under-samples the dynamics OR fails + to reduce dose/perturbation — either outcome sinks the differential-acquisition premise. +- statement: Thermo-paralysis at 25 C (K-pump background) suppresses 3-fold twitching enough to yield + motion-free, trackable dopaminergic outgrowth trajectories, without altering outgrowth kinematics + beyond the expected temperature-driven developmental speedup. + predictions: + - Frame-to-frame embryo / dendrite-tip displacement at 25 C paralysis is far below the 20 C twitching + baseline, approaching the registration-error floor. + - Dendrite-length-vs-time curves are recoverable (trackable tip) in the paralysis arm but corrupted + or untrackable in the twitching arm. + - Hatching-time distributions shift faster at 25 C, but outgrowth duration expressed in developmental + time (fraction of embryogenesis) is preserved vs 20 C — temperature rescales the clock without distorting + the process. + note: Refuted if 25 C changes outgrowth duration in developmental time (a thermal artifact, not just + a speedup) or if paralysis does not reduce motion enough to track the tip. Cross-links the temperature + -> hatching-time-distribution seed program. diff --git a/seed/programs/mutant-screen.yaml b/seed/programs/mutant-screen.yaml new file mode 100644 index 00000000..95acc991 --- /dev/null +++ b/seed/programs/mutant-screen.yaml @@ -0,0 +1,160 @@ +# Seed research program: Mutant screen — blind genotype calling from live embryo imaging +# Real, version-controlled Campaign/Strain/Experiment/Hypothesis (ELN spine dogfood). +slug: mutant-screen +campaign: + title: Mutant screen — blind genotype calling from live embryo imaging + goal: Test whether the agent can image a mixed dish of two C. elegans genotypes and report, per embryo, + which is which — first by fluorescent marker (validation) and then by neuroanatomical phenotype alone + in a shared reporter (the real screen), with post-hatch locomotion as independent ground truth. The + classification is the readout. + description: 'Hari Shroff''s agentic-microscopy ''mutant screen'' case study. Embryos of two genotypes + are co-cultured on one dish; the agent detects every embryo (bright-field localization), assigns each + a genotype call with confidence, and reports the mixed-population breakdown — the per-embryo call, + scored against ground truth, is the experiment''s output. Difficulty is graded across two experiments. + Experiment 1 (marker-based) mixes unlabeled wild type (N2) with a pan-neuronal TagRFP reporter (otIs355) + and calls genotype from marker presence in one volume — this validates the multi-embryo role/classification/reporting + machinery and the ''differential acquisition by expression'' path from Hari''s plan. Experiment 2 + (phenotype-based, the novel screen) mixes two strains carrying the SAME TagRFP marker — unc-6(+) reporter + vs unc-6(ev400); reporter — so marker presence is uninformative and genotype must be read from the + axon-guidance phenotype: unc-6/netrin loss displaces the nerve ring and misroutes commissures (danienella + outgrowth/guidance axis). Ground truth is obtained without a blinding co-marker by continuing acquisition + through hatch and confirming each call against the post-hatch Unc locomotion phenotype (ties in Hari''s + hatching-detection subproblem). This exercises multi-embryo roles, per-embryo classification + population + reporting, and the assisted scoring loop: an accuracy Result (predicted genotype vs behavior-confirmed + genotype) proposes a Hypothesis verdict for human confirmation, then exports a genotype-classification + benchmark to HuggingFace.' +strains: +- name: N2 + what: Wild-type Bristol reference. The unlabeled 'other' genotype in the Experiment-1 mixed dish and + the marker-negative control that sets the false-positive rate for marker-based genotype calling. Carries + no transgene, so it is TagRFP-dark under 561 nm. + genotype: wild type (Bristol isolate) + markers: [] + organism_ref: celegans +- name: otIs355 + what: 'Pan-neuronal nuclear-TagRFP reporter (Hobert lab, integrated array) in an unc-6(+) wild-type + background. Serves two roles: the TagRFP+ arm in Experiment 1 (marker-detectable), and the unc-6(+) + reporter control arm in Experiment 2 (same marker as the mutant, so it is separable from the mutant + only by neuroanatomy). Nuclear-localized RFP in every neuron gives a clean per-embryo detectable signal + and resolves the nerve-ring/commissure pattern used for phenotypic calling. FLAG: confirm the exact + CGC strain designation carrying otIs355 before ordering.' + genotype: otIs355 [rab-3p(prom1)::2xNLS::TagRFP] + markers: + - rab-3p::2xNLS::TagRFP (pan-neuronal, nuclear) + organism_ref: celegans +- name: unc-6(ev400); otIs355 + what: 'The mutant arm — the unc-6 null (ev400, canonical Hedgecock/Culotti X-linked deletion allele + of UNC-6/netrin) crossed into the otIs355 pan-neuronal reporter, so it carries the SAME TagRFP marker + as the otIs355 control and cannot be told apart by marker presence. UNC-6/netrin loss causes dorsoventral + axon-guidance defects: a displaced/misshapen nerve ring and misrouted commissures visible in the reporter + during embryogenesis, and an uncoordinated (Unc) locomotion phenotype after hatch that provides ground + truth. Built in-lab by crossing otIs355 to unc-6(ev400); ~10 days to a homozygous GFP+/Unc line (per + PLAN_MODE genetics item).' + genotype: unc-6(ev400) X; otIs355 [rab-3p(prom1)::2xNLS::TagRFP] + markers: + - rab-3p::2xNLS::TagRFP (pan-neuronal, nuclear) + organism_ref: celegans +experiments: +- title: Experiment 1 — Marker-based genotype call (validation of the screen machinery) + arms: + - name: WT (N2) + strain: N2 + condition: Co-cultured on a shared dish with otIs355; bright-field localization of every embryo, then + a single 488/561 nm marker-check volume. Expected TagRFP-negative → called wild type. + - name: Reporter (otIs355) + strain: otIs355 + condition: Co-cultured on the same shared dish with N2; same bright-field localization + single 561 + nm marker volume. Expected pan-neuronal nuclear TagRFP-positive → called reporter. + controls: + - Pure-N2 dish imaged blind with identical settings — measures the marker false-positive rate (autofluorescence + / bleed-through called as reporter). + - Pure-otIs355 dish imaged blind with identical settings — measures the marker false-negative rate (dim/early + embryos missed as WT) and detection sensitivity vs stage. + - 561 nm laser-off (bright-field only) acquisition — confirms calls are driven by marker fluorescence, + not a morphology confound. + - Known mix ratio (e.g. 50:50 plated by count) — the reported population breakdown is scored against + the plated ratio. + export_repo: 'pskeshu/gently-genotype-screen (FLAG: create/verify HF repo; falls back to pskeshu/gently-perception-benchmark)' + note: 'Trivial-by-design: this is the ''differential acquisition by expression'' path and validates + multi-embryo roles, per-embryo classification, and population reporting before Experiment 2 asks the + hard question. Ground truth = the marker channel itself (unambiguous). A candidate Result of kind=accuracy + scores predicted vs marker-ground-truth genotype per embryo; needs no timelapse (single decision volume).' +- title: Experiment 2 — Phenotype-based genotype call (the mutant screen; marker is uninformative) + arms: + - name: unc-6(+) reporter + strain: otIs355 + condition: 'Co-cultured on a shared dish with unc-6(ev400); otIs355 — BOTH TagRFP+, so marker presence + does not distinguish them. Timelapse through embryogenesis (bean → 3-fold) then continued to hatch. + Genotype called from pan-neuronal nerve-ring position + commissure guidance in the reporter. Expected: + wild-type nerve-ring/commissure pattern; normal post-hatch locomotion.' + - name: unc-6(ev400) reporter + strain: unc-6(ev400); otIs355 + condition: 'Same shared dish, same acquisition. Genotype called from the reporter phenotype. Expected: + displaced/misshapen nerve ring + misrouted (dorsoventral) commissures during elongation; post-hatch + uncoordinated (Unc) locomotion confirming the call.' + controls: + - Pure-otIs355 dish imaged blind — establishes the unc-6(+) phenotype prior and the classifier's per-arm + base rate. + - Pure-unc-6(ev400); otIs355 dish imaged blind — establishes the mutant phenotype prior; confirms Unc + penetrance/expressivity for ground-truth reliability. + - Stage-matched scoring — calls are compared only within matched developmental stage so genotype is + not read off a developmental-timing difference (guards H2's stage-dependence claim). + - Imaged-vs-unimaged hatch-timing band — a sibling cohort monitored at low cadence (no timelapse dose) + so continuing acquisition to hatch is shown not to create or shift the phenotype/hatch timing beyond + the control band. + - Post-hatch locomotion assay — short movement observation per hatched larva (Unc vs WT) is the definitive + independent ground truth against which embryo-morphology calls are scored. + export_repo: 'pskeshu/gently-genotype-screen (FLAG: create/verify HF repo; falls back to pskeshu/gently-perception-benchmark)' + note: The real screen. No blinding co-marker is used; ground truth comes from the post-hatch Unc phenotype + (H3 loop). unc-6/netrin is the guidance gene from the danienella outgrowth axis, so the phenotype + the classifier reads is genuine developmental-connectomics signal. Candidate Result kind=accuracy + scores embryo-morphology genotype call vs behavior-confirmed genotype, stratified by stage; a second + Result scores concordance between the two independent readouts. Needs the unc-6(ev400); otIs355 cross + complete (dependency). +hypotheses: +- statement: In a mixed dish of unlabeled N2 and pan-neuronal-TagRFP otIs355, per-embryo genotype is separable + at near-ceiling accuracy from marker presence in a single volume — validating the multi-embryo classify-and-report + machinery. + predictions: + - Per-embryo genotype-call accuracy >=0.98 vs marker ground truth on the mixed dish. + - Pure-N2 control yields <2% false-positive (marker-called) rate; pure-otIs355 control yields <2% false-negative + rate. + - A single 488/561 nm marker volume suffices — no timelapse needed; decision latency < one acquisition + cycle per embryo. + - The reported population breakdown matches the plated mix ratio within counting error. + note: 'Falsifier: accuracy near 50% (chance) or FP/FN >10% refutes — would indicate autofluorescence/bleed-through + or an exposure confound rather than a working marker channel. This is the cheap validation gate before + Experiment 2.' +- statement: unc-6(ev400) genotype is separable from unc-6(+) using embryonic neuroanatomy alone (a shared + TagRFP reporter, marker-uninformative), above chance, and the separability increases monotonically + with developmental stage as the nerve ring and commissures form. + predictions: + - Per-embryo genotype accuracy from reporter morphology is significantly above the 50% chance rate (target + >=0.75) vs behavior-confirmed ground truth. + - 'Accuracy is stage-dependent: near chance before the nerve ring resolves (pre-comma/bean), rising + after the 1.5-fold stage once commissures are laid down; a monotone increase across bean -> comma + -> 1.5-fold -> 2-fold -> 3-fold.' + - 'Errors are systematic, not random: mutant->WT misses dominate early (phenotype not yet manifest), + and confident correct calls concentrate at late elongation.' + - 'The signal is neuroanatomical, not timing: calls survive stage-matched scoring (Experiment-2 control).' + note: 'This is the novel result — genotype read from a developing axon-guidance phenotype, tied to the + UNC-6/netrin outgrowth axis. Falsifier: accuracy indistinguishable from 50% at ALL stages -> the phenotype + is not resolvable at gently''s marker/resolution, refuting the phenotype-based screen (would fall + back to requiring a marker, i.e. Experiment 1 only).' +- statement: 'Embryonic morphology-based genotype calls are confirmed by the post-hatch locomotion phenotype, + so the ground-truth loop closes unattended: the genotype the agent assigns from the embryo predicts + whether the hatched larva is Unc.' + predictions: + - Concordance between the embryo-morphology call and the post-hatch Unc/WT locomotion call is >= the + Experiment-2 morphology accuracy (both read the same latent genotype); disagreements localize to embryos + the agent scored low-confidence. + - Hatch is detectable from imaging alone (field-clearing / vermiform escape from the eggshell) with + recall >=0.9, so the classify-then-confirm loop runs without a human. + - Continuing acquisition to hatch does not shift per-genotype hatch timing beyond the imaged-vs-unimaged + control band — imaging reveals the phenotype, it does not create it. + - unc-6(ev400) Unc penetrance is high enough (from the pure-mutant control) that behavior is a reliable + per-embryo ground-truth label. + note: 'Falsifier: embryo calls and post-hatch behavior calls are uncorrelated -> the ''phenotype'' the + classifier reads in the embryo is an imaging artifact, not genotype, and Experiment 2''s accuracy + is spurious. This hypothesis also exercises Hari''s hatching-detection subproblem as the trigger that + ends each embryo''s track.' diff --git a/seed/programs/stage-classification.yaml b/seed/programs/stage-classification.yaml new file mode 100644 index 00000000..1502b4dd --- /dev/null +++ b/seed/programs/stage-classification.yaml @@ -0,0 +1,179 @@ +# Seed research program: Embryo developmental-stage classification — nuclear + bright-field fusion, with hatching detection +# Real, version-controlled Campaign/Strain/Experiment/Hypothesis (ELN spine dogfood). +slug: stage-classification +campaign: + title: Embryo developmental-stage classification — nuclear + bright-field fusion, with hatching detection + goal: 'Build an accurate, calibrated stage classifier for live C. elegans embryos on the diSPIM using + gently-perception, fusing label-free bright-field morphology with a nuclear fluorescence channel, + and detect the hatching event in real time by switching to high cadence. Run this program as the annotation->HuggingFace + flywheel engine: gently-perception predictions -> in-ELN human ground-truth correction -> an accuracy + Result per run -> approval-gated export to pskeshu/gently-perception-benchmark.' + description: 'gently-perception emits per-timepoint stage predictions (comma, 1.5-fold, 2-fold, 3-fold, + pretzel) on live embryos. Bright-field alone captures gross morphology (elongation, eggshell, movement) + but saturates as a staging cue once the body is fully folded: 2-fold and 3-fold look nearly identical + by outline. A ubiquitous nuclear reporter carries an orthogonal signal — nuclear count, packing, and + the ordered lattice of the elongating body — that keeps changing across exactly those late transitions + and collapses abruptly at hatch. This campaign tests whether fusing the two modalities beats either + alone, whether an adaptive monitor-until-3-fold -> high-cadence switch localizes the fast, stochastic + hatch event, and whether the human-corrected benchmark generalizes across nuclear-labelling density. + Ground truth is authored in-ELN on the Vitals stage strip (agent pre-screens high-agreement stretches + for one-range confirm; human decides the transitional frames), scored as a Result of kind:accuracy + (predicted vs ground truth per run), and exported to the shared benchmark on human approval. This + is seed program #1 from Hari Shroff''s agentic-microscopy paper plan (bright-field detection is DONE; + staging + hatching detection is next), and it is the primary near-term driver of the ELN scientific + spine.' +strains: +- name: RW10226 + what: 'Ubiquitous nuclear-GFP lineaging strain — every nucleus in the embryo is labelled through embryogenesis. + The gold-standard substrate for nuclear-based stage classification (StarryNite/AceTree lineaging use + it) and the primary strain of this campaign: the ordered nuclear lattice reports on stage across the + late transitions where bright-field outline saturates, and its collapse plus larval translation marks + hatch. Imaged BF-only, nuclear-only, or fused across the arms of Experiment 1.' + genotype: zuIs178 [his-72p::HIS-72::GFP; unc-119(+)] V; stIs10024 [pie-1p::H2B::GFP; unc-119(+)] + markers: + - his-72::GFP (ubiquitous somatic nuclear, 488 nm) + - pie-1p::H2B::GFP (early/germline nuclear, 488 nm) + organism_ref: celegans +- name: OH-otIs355 + what: 'Pan-neuronal but nuclear-localized (2xNLS) TagRFP reporter (Stefanakis, Carrera & Hobert 2015) + — a SPARSE nuclear pattern: only neuronal nuclei, not the whole embryo. This is the contrast strain + for Experiment 3: it tests whether stage classification needs the dense whole-embryo nuclear lattice + or whether a sparse nuclear subset suffices, and quantifies how far a benchmark calibrated on ubiquitous + nuclei transfers to a sparse pattern. Its red channel (561 nm) also leaves 488 free for future co-imaging.' + genotype: otIs355 [rab-3(prom1)::2xNLS::TagRFP] + markers: + - rab-3p::2xNLS::TagRFP (pan-neuronal nuclear, 561 nm) + organism_ref: celegans +- name: N2 + what: 'Wild-type Bristol — no reporter. Provides the label-free bright-field-only condition: the negative + arm that isolates exactly how much the nuclear channel adds to staging accuracy, and the cleanest + strain for scoring hatch morphology without any phototoxic fluorescence load. Also the over-sampled + BF hatch-witness strain in Experiment 2.' + genotype: wild-type (Bristol N2) + markers: [] + organism_ref: celegans +experiments: +- title: Fusion vs single-modality stage classification (within-embryo paired) + arms: + - name: bf-only + strain: RW10226 + condition: Bright-field transmitted-light max-projection only fed to gently-perception; nuclear GFP + channel withheld. Per-timepoint stage prediction over comma->1.5-fold->2-fold->3-fold->pretzel. + - name: nuclear-only + strain: RW10226 + condition: 488 nm nuclear-GFP max-projection only fed to gently-perception; bright-field withheld. + Same timepoints. + - name: fusion + strain: RW10226 + condition: Bright-field + nuclear-GFP fused (paired side-by-side projection) fed to gently-perception + in one call. Same timepoints, same embryos as the other two arms. + controls: + - Expert manual ground-truth staging on the full 3D volumes, authored in-ELN on the Vitals stage strip + (agent bulk-proposes high-agreement stretches; human confirms transitional frames) — the accuracy + reference for all three arms. + - 'Within-embryo paired design: the identical embryos and identical acquired volumes are scored under + all three input conditions, so input modality is the only variable (no acquisition, mounting, or embryo-to-embryo + confound).' + - N2 wild-type bright-field-only condition as the label-free floor — how far can you stage with zero + fluorescence and zero photodose. + export_repo: pskeshu/gently-perception-benchmark + note: Scored per run as a Result of kind:accuracy (top-1 stage accuracy + full confusion matrix, predicted + vs in-ELN ground truth). Because all three arms read the same volumes, the fusion-vs-single comparison + is a paired test. Corrected {projection ref, prediction, ground truth, provenance, strain} rows are + the export payload; the push to pskeshu/gently-perception-benchmark is approval-gated (require_control + + explicit confirm; HF_TOKEN from env). +- title: Hatching detection — adaptive high-cadence vs fixed low-cadence + arms: + - name: adaptive-high-fusion + strain: RW10226 + condition: Monitor at 10 min baseline until the 3-fold stage is detected, then switch to 30 s cadence + with BF+nuclear fusion until hatch is called. Hatch = collapse of the ordered nuclear lattice + + larva translating out of the eggshell (BF). + - name: adaptive-high-bf + strain: N2 + condition: Same monitor-until-3-fold -> 30 s high-cadence switch, bright-field only (no fluorescence). + Tests hatch detectability from morphology + movement alone. + - name: fixed-low + strain: RW10226 + condition: Fixed 10 min cadence throughout (no adaptive switch), BF+nuclear — the undersampled comparator + that cannot localize a sub-interval event. + controls: + - 'Over-sampled bright-field hatch witness: a subset of N2 embryos imaged at 5 s BF-only through the + peri-hatch window to establish the true hatch timestamp as ground truth (BF-only + minimal exposure + to avoid perturbing the event being measured).' + - Endpoint-only (near-unmonitored) embryos scored for hatch success/timing to bound how much the high-cadence + imaging load itself delays or perturbs hatching (photodose control). + export_repo: pskeshu/gently-perception-benchmark + note: 'Hatch-timing Result of kind:accuracy = |predicted hatch frame - witness ground-truth hatch time|, + per arm. The hatching subproblem is why staging needs a real-time cadence controller, not just a post-hoc + classifier: hatch is short and stochastic and only recoverable above a cadence threshold. Exported + as a hatching split of the benchmark.' +- title: Nuclear-pattern generalization — ubiquitous vs sparse-neuronal nuclei + arms: + - name: ubiquitous + strain: RW10226 + condition: Fusion staging (BF + nuclear GFP) on the dense whole-embryo nuclear lattice. Same acquisition + geometry and cadence as the sparse arm. + - name: sparse-neuronal + strain: OH-otIs355 + condition: Fusion staging (BF + nuclear TagRFP, 561 nm) on the sparse pan-neuronal nuclear subset + only. Matched geometry/cadence. + controls: + - Expert in-ELN ground-truth staging authored separately per strain (the sparse and dense patterns are + annotated independently). + - Matched acquisition geometry, cadence, and equal annotation budget across strains so the only variable + is nuclear-labelling density (channel wavelength noted as a covariate). + export_repo: pskeshu/gently-perception-benchmark + note: 'Tests strain-transfer of the exported benchmark: does a classifier / few-shot pool calibrated + on ubiquitous nuclei (RW10226) hold up on a sparse neuronal pattern (otIs355)? Quantifies a nuclear-density + dependence that the benchmark should record as a per-strain slice.' +hypotheses: +- statement: Fusing bright-field morphology with a nuclear fluorescence channel yields higher developmental-stage + classification accuracy than either modality alone, with the largest gain concentrated at the late + transitions (2-fold<->3-fold, 3-fold<->pretzel) where bright-field outline saturates but nuclear count + and lattice arrangement keep changing. + predictions: + - 'On the same paired embryos, top-1 stage accuracy: fusion > nuclear-only > bf-only.' + - The fusion gain over the best single modality is concentrated at the 2-fold->3-fold and 3-fold->pretzel + boundaries — the largest reduction in off-diagonal confusion-matrix mass sits there. + - bf-only systematically confuses 2-fold and 3-fold (elongation outline nearly identical); adding the + nuclear channel resolves that specific confusion. + - 'REFUTED IF: fusion does not beat the best single modality by a meaningful margin on the paired embryos, + or the gain is uniform across stages rather than concentrated at late transitions.' + note: 'The core claim of program #1 and the reason a single-channel classifier is not enough. Tested + by Experiment 1 as a within-embryo paired comparison.' +- statement: Hatching is a fast, stochastic event whose timestamp is only recoverable above a cadence + threshold; an adaptive monitor-until-3-fold -> high-cadence switch localizes hatch to within one interval, + while a fixed low-cadence schedule cannot — and the nuclear channel flags hatch more reliably than + bright-field alone. + predictions: + - The adaptive high-cadence (30 s) arm localizes hatch to within +/-1 frame of the 5 s over-sampled + bright-field witness; the fixed 10 min arm has error on the order of its interval. + - Fusion hatch-detection precision > bf-only precision, because the ordered nuclear lattice collapses + at hatch as an additional, earlier cue alongside larval movement. + - Hatch is signalled by simultaneous (within 1-2 frames) collapse of the nuclear lattice and larval + translation out of the eggshell in bright-field. + - 'REFUTED IF: the fixed low-cadence schedule recovers hatch time as accurately as the adaptive high-cadence + switch, or bf-only matches fusion hatch precision.' + note: The hatching-detection subproblem. Tested by Experiment 2; the photodose control bounds whether + the imaging needed to catch hatch perturbs the hatch itself. +- statement: 'The in-ELN annotation flywheel converges: human correction of gently-perception predictions + raises measured accuracy over successive export rounds, and a benchmark calibrated on ubiquitous nuclei + transfers imperfectly to a sparse neuronal nuclear pattern — a quantifiable strain-specificity gap + the benchmark must record.' + predictions: + - The accuracy Result computed per export round rises as corrected frames accumulate in pskeshu/gently-perception-benchmark + and the few-shot pool grows. + - At equal annotation budget, model accuracy on sparse-neuronal nuclei (otIs355) is lower than on ubiquitous + nuclei (RW10226), revealing a nuclear-density dependence. + - The agent's pre-screened high-agreement stretches (bulk range-confirm) match human ground truth at + a high rate, so human effort concentrates on transitional frames rather than the full timeline. + - 'REFUTED IF: accumulated corrected annotations do not improve accuracy across rounds, or accuracy + is strain-independent (no ubiquitous-vs-sparse gap).' + note: 'Validates the flywheel as an engine, not just a pipe. Tested across Experiment 1 (round-over-round + accuracy) and Experiment 3 (cross-strain transfer). Identifier flags: RW10226 is the Bao-lab lineaging + strain — verify the exact zuIs178/stIs10024 transgene strings against CGC before the first benchmark + push. otIs355 [rab-3(prom1)::2xNLS::TagRFP] is real (Stefanakis/Hobert 2015); confirm the CGC strain + designation that carries it (placeholder name OH-otIs355 used here). OH904/rab-3p::GFP is cited in + PLAN_MODE_DESIGN as pan-neuronal but is not used in this program — otIs355 is the nuclear-localized + pan-neuronal reporter this program needs.' diff --git a/seed/programs/temperature-hatching.yaml b/seed/programs/temperature-hatching.yaml new file mode 100644 index 00000000..e88e0509 --- /dev/null +++ b/seed/programs/temperature-hatching.yaml @@ -0,0 +1,182 @@ +# Seed research program: Temperature → hatching-time distributions (K-pump thermo strain) +# Real, version-controlled Campaign/Strain/Experiment/Hypothesis (ELN spine dogfood). +slug: temperature-hatching +campaign: + title: Temperature → hatching-time distributions (K-pump thermo strain) + goal: Quantify the distribution of C. elegans embryonic hatching times and measure how the two moves + the smart-microscope makes on its temperature setpoint — thermo-cycling to 25 °C for paralysis-free + imaging windows, and the imaging (photodose) itself — perturb that distribution, using Richard Ikegami's + temperature-sensitive potassium-channel strain that stops twitching at 25 °C from the 3-fold stage. + description: 'The late C. elegans embryo (3-fold → hatch) twitches vigorously, which motion-blurs 3D + light-sheet acquisition exactly in the developmental window we most want to image. Richard''s strain + carries a temperature-sensitive gain-of-function two-pore potassium channel (twk-18(cn110gf) class): + at the restrictive temperature (25 °C) the body-wall muscle hyperpolarizes and the embryo goes still + from the 3-fold stage, giving a paralysis-free imaging window; at 20 °C it twitches normally. This + turns the diSPIM''s temperature setpoint (ACUITYnano thermal controller, exposed as a settable axis) + into a scientific instrument, not just an environmental knob — the agent thermo-cycles: hold 20 °C, + ramp to 25 °C to silence twitching for a clean high-cadence window, return to 20 °C. Every such move + is a perturbation. The scientific question is the DISTRIBUTION of hatching times and how it shifts: + hatching time is a stochastic per-embryo quantity with a mode and a right-skewed tail, and we want + to know whether (a) thermalization alone perturbs it, (b) imaging/photodose perturbs it, and (c) whether + intermittent thermo-cycling is gentler than continuous 25 °C paralysis. Hatch is detected by bright-field + eggshell breach (an existing, validated case study); staging and hatch-timing are anchored to a ubiquitous + nuclear reporter. This campaign is the primary driver for a future typed Actuator axis (setpoint → + actuator generalization) and exports its hatch-detection frames as perception training data.' +strains: +- name: RK-KP1 + what: 'Richard Ikegami''s K-pump thermo strain: temperature-sensitive gain-of-function two-pore potassium + channel that paralyzes body-wall muscle (stops embryonic twitching) at the restrictive temperature + 25 °C from the 3-fold stage, while twitching normally at the permissive 20 °C. Carries a ubiquitous + nuclear reporter for staging and hatch-timing. The instrument for paralysis-free late-embryo light-sheet + imaging.' + genotype: twk-18(cn110gf); stIs10026[his-72p::HIS-24::mCherry] (N2 background) + markers: + - his-72p::HIS-24::mCherry (ubiquitous nuclear — staging + hatch timing) + - bright-field eggshell / hatch morphology + - body-wall muscle twitch (motion index readout) + organism_ref: celegans +- name: WT-HIS + what: 'Matched-marker wild-type control: identical ubiquitous nuclear reporter but wild-type twk-18, + so it twitches normally at BOTH 20 °C and 25 °C. Establishes that paralysis is K-channel-specific + (not a generic heat artifact) and, unimaged at 20 °C, provides the wild-type reference hatching-time + distribution against which the K-pump strain''s is compared.' + genotype: stIs10026[his-72p::HIS-24::mCherry] (N2 background, wild-type twk-18) + markers: + - his-72p::HIS-24::mCherry (ubiquitous nuclear — staging + hatch timing) + - bright-field eggshell / hatch morphology + - body-wall muscle twitch (motion index readout) + organism_ref: celegans +experiments: +- title: 'Exp A — Paralysis validation: does 25 °C silence twitching, and is it K-channel-specific?' + arms: + - name: KP restrictive + strain: RK-KP1 + condition: 'Constant 25 °C from comma stage. At 3-fold onset run a 10-min motion-assay window: bright-field + + nuclear frames every 30 s; compute a motion index (frame-to-frame difference of the segmented + embryo). Expect twitching abolished (paralysis).' + - name: KP permissive + strain: RK-KP1 + condition: Constant 20 °C. Identical 3-fold motion-assay window. Expect normal twitching — the permissive-temperature + control for the same genotype. + - name: WT restrictive + strain: WT-HIS + condition: Constant 25 °C. Identical 3-fold motion-assay window. Expect normal twitching — shows paralysis + is specific to the mutant K-channel, not a heat effect on any embryo. + controls: + - WT-HIS constant 20 °C motion-assay window — wild-type twitch baseline + - 'Reversibility check: hold RK-KP1 at 25 °C (paralyzed), ramp back to 20 °C, confirm twitching resumes + within one thermal time-constant — rules out heat damage vs reversible channel gating' + - n ≥ 8 embryos per arm; motion index blinded to arm during scoring + note: 'This is the tool-axis validator: it certifies that the temperature setpoint buys a genuine paralysis-free + window before any hatching-distribution claim rests on that protocol. Exercises the ACUITYnano setpoint + end-to-end (ramp, stabilize, hold, reverse).' +- title: 'Exp B — Does the imaging protocol perturb the hatching-time distribution? (2×2: temperature + protocol × imaging)' + arms: + - name: Cycled + imaged + strain: RK-KP1 + condition: Base 20 °C; agent ramps to 25 °C to open paralysis-free windows and images high-cadence + 3D light-sheet (~1-min interval, 488 nm for the nuclear marker) through 3-fold → hatch, returning + to 20 °C between windows. The full working protocol — both perturbations present (thermalization + + photodose). + - name: Cycled + unimaged + strain: RK-KP1 + condition: Identical 20↔25 °C thermo-cycling schedule but shutter CLOSED (no light). Isolates the + thermalization perturbation alone. Hatch detected by periodic low-dose bright-field only. + - name: Constant-20 + imaged + strain: RK-KP1 + condition: Constant 20 °C (embryo twitches — motion-blurred 3D) with the same imaging cadence and + cumulative photodose as Cycled+imaged. Isolates the imaging/photodose perturbation alone, thermally + unperturbed. + - name: Constant-20 + unimaged + strain: RK-KP1 + condition: Constant 20 °C, hatch detected by periodic low-dose bright-field only. The reference hatching-time + distribution — neither perturbation applied. + controls: + - Off-instrument incubator population of RK-KP1 at 20 °C, never mounted — off-instrument reference distribution + controlling for mounting/handling stress + - n ≥ 20 embryos per arm for distribution statistics (median, CV, 90th-percentile tail, arrest fraction) + - Match cumulative photodose between the two imaged arms so the imaged-vs-unimaged contrast is clean + export_repo: pskeshu/gently-hatching-time + note: Hatch detected by bright-field eggshell breach (validated case study). Hatching time measured + as minutes from a fixed nuclear-marker developmental landmark (3-fold onset) to eggshell breach, normalizing + out per-embryo mounting-time differences. The 2×2 factorial cleanly separates thermalization from + photodose. Exports bright-field hatch ± nuclear-stage frames + human ground truth to HuggingFace as + hatch-detection training data (approval-gated push). +- title: 'Exp C — Protocol optimization: is intermittent thermo-cycling gentler than continuous 25 °C + paralysis?' + arms: + - name: Constant-25 imaged + strain: RK-KP1 + condition: Constant 25 °C — continuous paralysis, development ~30% faster (~10 h to hatch). High-cadence + 3D through 3-fold → hatch. Simplest paralysis strategy, but sustained elevated temperature throughout. + - name: Thermo-cycled imaged + strain: RK-KP1 + condition: Base 20 °C with brief 25 °C windows for imaging only (the Exp B Cycled+imaged protocol). + Embryo spends most of development at 20 °C; elevated only during acquisition windows. + - name: Constant-20 unimaged baseline + strain: RK-KP1 + condition: Constant 20 °C, no imaging (bright-field hatch detection only). Baseline distribution (~14 + h) for shape comparison after regressing out the developmental-rate shift. + controls: + - WT-HIS constant 25 °C unimaged — temperature-only effect on the wild-type hatch distribution, separating + heat effects from K-channel-gating effects + - Match cumulative photodose between the two imaged arms so the comparison isolates temperature protocol, + not light dose + - n ≥ 20 embryos per arm + note: 'Outcome = perturbation of hatch-time DISTRIBUTION SHAPE (skew, CV, tail, arrest fraction) after + regressing out the mean developmental-rate shift that 25 °C trivially causes, PLUS hatch-success rate + and mean motion-blur during imaging windows. If thermo-cycling gives equal image quality with less + distribution perturbation it is Pareto-better. Directly motivates the typed Actuator axis: the agent + choosing a temperature schedule to minimize a perturbation objective.' +hypotheses: +- statement: In the K-pump strain, raising temperature to 25 °C abolishes embryonic twitching from the + 3-fold stage onward (a reversible, K-channel-gated paralysis), whereas wild-type embryos twitch normally + at both 20 °C and 25 °C. + predictions: + - RK-KP1 motion index at 3-fold drops ≥ 90% relative to its own 20 °C value within one thermal stabilization + time-constant of reaching 25 °C. + - WT-HIS motion index at 25 °C is statistically indistinguishable from WT-HIS at 20 °C (twitching persists) + — paralysis is mutant-specific, not a heat artifact. + - 'Paralysis is reversible: returning RK-KP1 from 25 °C to 20 °C restores twitching, ruling out thermal + damage.' + note: Falsified if RK-KP1 still twitches at 25 °C, if WT is silenced at 25 °C, or if paralysis does + not reverse on cooling. This is the premise the whole protocol rests on (Exp A). +- statement: 'Thermalization is gentle but light is not: thermo-cycling alone (unimaged) leaves the hatching-time + distribution statistically indistinguishable from the constant-20 °C unimaged baseline, while imaging + (photodose) shifts the distribution — delaying median hatch and/or inflating its spread and late tail.' + predictions: + - Cycled+unimaged median hatch time lies within a pre-registered equivalence margin of Constant-20+unimaged, + and the two distributions do not differ by a two-sample Kolmogorov–Smirnov test. + - Both imaged arms show a rightward median shift and/or larger CV and heavier 90th-percentile tail versus + their unimaged counterparts. + - The imaging effect scales with cumulative photodose (more/faster windows → larger shift), a dose–response + within the imaged arms. + note: Falsified if Cycled+unimaged differs from baseline as much as the imaged arms do (thermalization, + not light, dominates), or if imaging leaves the distribution unchanged. Tested by the Exp B 2×2 factorial. +- statement: Intermittent thermo-cycling perturbs the hatching-time distribution less than continuous + 25 °C paralysis, at equal imaging quality — so thermo-cycling is Pareto-preferable despite constant-25 + °C giving simpler continuous paralysis. + predictions: + - Constant-25 °C shifts the median hatch earlier (~10 h vs ~14 h, the expected developmental-rate effect) + but, after regressing out that mean shift, its residual distribution is broader (higher CV / heavier + tail) and its hatch-success rate lower than thermo-cycled. + - Motion-blur during the 25 °C imaging windows is comparable between constant-25 and thermo-cycled (both + paralyzed while imaging), so image quality is not sacrificed by cycling. + - WT-HIS at constant 25 °C unimaged shows a smaller distribution-shape perturbation than RK-KP1 at constant + 25 °C, indicating part of the constant-25 penalty is K-channel-gating load, not temperature alone. + note: Falsified if constant-25 °C hatch distribution is no broader than thermo-cycled with equal hatch + success (constant-25 would then be preferable for its simplicity). Tested by Exp C; directly motivates + a temperature-schedule Actuator objective. +- statement: Baseline hatching times are right-skewed (approximately log-normal), and perturbations act + primarily on the right tail — increasing the fraction of late-hatchers and arrests — rather than translating + the mode. + predictions: + - The Constant-20+unimaged hatch-time histogram fits a log-normal better than a normal distribution + (lower AIC) with positive skew. + - Under imaging and under sustained heat, the distribution mode is approximately preserved while the + 90th-percentile hatch time and the arrest fraction rise. + - The coefficient of variation increases monotonically with photodose across the imaged conditions. + note: Falsified if perturbation shifts the whole distribution (mode moves) without tail inflation, or + if the baseline is symmetric. This is the distribution-shape hypothesis — the reason the study measures + the full distribution, not just a mean hatch time. diff --git a/tests/test_eln_accuracy.py b/tests/test_eln_accuracy.py new file mode 100644 index 00000000..1ee2ea74 --- /dev/null +++ b/tests/test_eln_accuracy.py @@ -0,0 +1,49 @@ +"""Tests for ELN perception accuracy scoring.""" + +from gently.eln import accuracy as acc + + +def test_stage_at_timepoint_covers_range(): + gt = [ + {"stage": "bean", "start_timepoint": 0, "end_timepoint": 10}, + {"stage": "comma", "start_timepoint": 11, "end_timepoint": None}, # open-ended + ] + assert acc.stage_at_timepoint(gt, 5) == "bean" + assert acc.stage_at_timepoint(gt, 10) == "bean" + assert acc.stage_at_timepoint(gt, 11) == "comma" + assert acc.stage_at_timepoint(gt, 999) == "comma" # open-ended reaches on + + +def test_stage_at_timepoint_no_coverage(): + gt = [{"stage": "bean", "start_timepoint": 5, "end_timepoint": 10}] + assert acc.stage_at_timepoint(gt, 2) is None + + +def test_score_accuracy_mix(): + preds = [ + {"timepoint": 1, "predicted_stage": "bean"}, # correct + {"timepoint": 6, "predicted_stage": "comma"}, # wrong (gt bean) + {"timepoint": 12, "predicted_stage": "comma"}, # correct + {"timepoint": 50, "predicted_stage": "pretzel"}, # unscored (no gt) + ] + gt = [ + {"stage": "bean", "start_timepoint": 0, "end_timepoint": 10}, + {"stage": "comma", "start_timepoint": 11, "end_timepoint": 20}, + ] + s = acc.score_accuracy(preds, gt) + assert s["n_scored"] == 3 + assert s["n_correct"] == 2 + assert abs(s["accuracy"] - 2 / 3) < 1e-9 + assert s["unscored"] == 1 + assert s["confusion"]["bean|comma"] == 1 + + +def test_accuracy_result_shape_and_draft_when_unscorable(): + preds = [{"timepoint": 100, "predicted_stage": "comma"}] + gt = [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 10}] + r = acc.accuracy_result("s1", "e1", preds, gt, run_id="run7", annotator="kesh") + assert r["kind"] == "accuracy" + assert r["value"] is None # nothing scorable + assert r["status"] == "draft" + assert r["inputs"]["run_id"] == "run7" + assert r["provenance"]["author"] == "kesh" diff --git a/tests/test_eln_export_route.py b/tests/test_eln_export_route.py new file mode 100644 index 00000000..1e0315d2 --- /dev/null +++ b/tests/test_eln_export_route.py @@ -0,0 +1,87 @@ +"""Tests for the per-embryo HuggingFace export route + annotation-summary.""" + +from unittest.mock import MagicMock + +from fastapi import FastAPI +from fastapi.testclient import TestClient + + +def _client(store): + from gently.ui.web.auth import require_control + from gently.ui.web.routes.data import create_router + + app = FastAPI() + server = MagicMock() + agent = MagicMock() + agent.experiment = MagicMock() + agent.store = store + agent.session_id = "s1" + server.agent_bridge.agent = agent + app.include_router(create_router(server)) + app.dependency_overrides[require_control] = lambda: None + return TestClient(app) + + +def _store_with(preds, gt): + store = MagicMock() + store.set_ground_truth = MagicMock() # identifies it as the GT/FileStore + store.get_predictions.return_value = preds + store.get_ground_truth.return_value = gt + return store + + +_PREDS = [{"timepoint": 1, "predicted_stage": "bean"}, {"timepoint": 6, "predicted_stage": "comma"}] +_GT = [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 10, "annotator": "kesh"}] + + +def test_export_requires_confirm(): + r = _client(_store_with(_PREDS, _GT)).post("/api/embryos/e1/export", json={"session_id": "s1"}) + assert r.status_code == 400 + + +def test_export_requires_token(monkeypatch): + monkeypatch.delenv("HF_TOKEN", raising=False) + monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False) + r = _client(_store_with(_PREDS, _GT)).post( + "/api/embryos/e1/export", json={"confirm": True, "session_id": "s1"} + ) + assert r.status_code == 503 + + +def test_export_no_annotated_timepoints(monkeypatch): + monkeypatch.setenv("HF_TOKEN", "x") + store = _store_with([{"timepoint": 99, "predicted_stage": "comma"}], _GT) # tp 99 uncovered + r = _client(store).post("/api/embryos/e1/export", json={"confirm": True, "session_id": "s1"}) + assert r.status_code == 400 + + +def test_export_success_pushes(monkeypatch): + monkeypatch.setenv("HF_TOKEN", "x") + from gently.eln import hf_connector + + seen = {} + + def fake_push(records, **kw): + seen.update(n=len(records), repo=kw.get("repo")) + return { + "repo": kw.get("repo"), + "split": kw.get("split", "train"), + "n": len(records), + "revision": None, + } + + monkeypatch.setattr(hf_connector, "push_dataset", fake_push) + r = _client(_store_with(_PREDS, _GT)).post( + "/api/embryos/e1/export", json={"confirm": True, "session_id": "s1", "strain": "OH904"} + ) + assert r.status_code == 200 + assert r.json()["n"] == 2 + assert seen["n"] == 2 + assert seen["repo"] == "pskeshu/gently-perception-benchmark" + + +def test_annotation_summary_route(): + store = _store_with([{"timepoint": t} for t in (1, 2, 40)], _GT) + r = _client(store).get("/api/embryos/e1/annotation-summary?session_id=s1") + assert r.status_code == 200 + assert r.json() == {"n_predictions": 3, "n_annotated": 2} diff --git a/tests/test_eln_export_service.py b/tests/test_eln_export_service.py new file mode 100644 index 00000000..21d70681 --- /dev/null +++ b/tests/test_eln_export_service.py @@ -0,0 +1,50 @@ +"""Tests for the per-embryo annotation export service.""" + +from unittest.mock import MagicMock + +from gently.eln import export_service as es + + +def _store(preds, gt): + s = MagicMock() + s.get_predictions.return_value = preds + s.get_ground_truth.return_value = gt + return s + + +def test_collect_pairs_predictions_with_gt_ranges(): + preds = [ + {"timepoint": 1, "predicted_stage": "bean"}, + {"timepoint": 6, "predicted_stage": "comma"}, + {"timepoint": 50, "predicted_stage": "pretzel"}, # no GT → dropped + ] + gt = [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 10, "annotator": "kesh"}] + rows = es.collect_embryo_annotations(_store(preds, gt), "s1", "e1", strain="OH904") + assert len(rows) == 2 # only annotated timepoints + assert rows[0]["ground_truth_stage"] == "bean" + assert rows[0]["predicted_stage"] == "bean" + assert rows[1]["predicted_stage"] == "comma" # tp 6, still in [0,10] + assert rows[0]["annotator"] == "kesh" + assert rows[0]["strain"] == "OH904" + assert rows[0]["stage"] == "bean" # build_records fallback key + + +def test_collect_include_unannotated(): + preds = [{"timepoint": 99, "predicted_stage": "comma"}] + gt = [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 10}] + assert es.collect_embryo_annotations(_store(preds, gt), "s", "e", annotated_only=False) + assert es.collect_embryo_annotations(_store(preds, gt), "s", "e", annotated_only=True) == [] + + +def test_pred_stage_from_findings(): + preds = [{"timepoint": 2, "findings": {"stage": "1.5fold"}}] + gt = [{"stage": "1.5fold", "start_timepoint": 0, "end_timepoint": 10}] + rows = es.collect_embryo_annotations(_store(preds, gt), "s", "e") + assert rows[0]["predicted_stage"] == "1.5fold" + + +def test_annotation_summary(): + preds = [{"timepoint": t, "predicted_stage": "x"} for t in (1, 2, 3, 40)] + gt = [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 5}] + s = es.annotation_summary(_store(preds, gt), "s", "e") + assert s == {"n_predictions": 4, "n_annotated": 3} diff --git a/tests/test_eln_ground_truth_route.py b/tests/test_eln_ground_truth_route.py new file mode 100644 index 00000000..1de07f93 --- /dev/null +++ b/tests/test_eln_ground_truth_route.py @@ -0,0 +1,83 @@ +"""Tests for the ELN ground-truth authoring route (annotation flywheel write path).""" + +from unittest.mock import MagicMock + +from fastapi import FastAPI +from fastapi.testclient import TestClient + + +def _build_client(store): + from gently.ui.web.auth import require_control + from gently.ui.web.routes.data import create_router + + app = FastAPI() + server = MagicMock() + agent = MagicMock() + agent.experiment = MagicMock() # hasattr(agent, "experiment") → True + agent.store = store + agent.session_id = "s1" + server.agent_bridge.agent = agent + app.include_router(create_router(server)) + app.dependency_overrides[require_control] = lambda: None + return TestClient(app) + + +def test_ground_truth_route_persists_range(): + store = MagicMock() + client = _build_client(store) + resp = client.post( + "/api/embryos/embryo_1/ground_truth", + json={"stage": "comma", "start_timepoint": 40, "end_timepoint": 58, "annotator": "kesh"}, + ) + assert resp.status_code == 200 + store.set_ground_truth.assert_called_once_with("s1", "embryo_1", "comma", 40, 58, "kesh", None) + assert resp.json()["stage"] == "comma" + + +def test_ground_truth_route_open_ended_end(): + store = MagicMock() + client = _build_client(store) + resp = client.post( + "/api/embryos/e1/ground_truth", + json={"stage": "pretzel", "start_timepoint": 90}, + ) + assert resp.status_code == 200 + args = store.set_ground_truth.call_args[0] + assert args[3] == 90 and args[4] is None # end_timepoint None = open-ended + + +def test_ground_truth_route_requires_stage(): + store = MagicMock() + client = _build_client(store) + resp = client.post("/api/embryos/e1/ground_truth", json={"start_timepoint": 1}) + assert resp.status_code == 400 + + +def test_ground_truth_route_requires_int_start(): + store = MagicMock() + client = _build_client(store) + resp = client.post( + "/api/embryos/e1/ground_truth", json={"stage": "bean", "start_timepoint": "x"} + ) + assert resp.status_code == 400 + + +def test_ground_truth_route_benchmark_no_agent_uses_server_store(): + """No agent (launch_viz_server mode) → falls back to server.gently_store.""" + from gently.ui.web.auth import require_control + from gently.ui.web.routes.data import create_router + + store = MagicMock() + server = MagicMock() + server.agent_bridge = None # benchmark mode: no agent bridge + server.gently_store = store + app = FastAPI() + app.include_router(create_router(server)) + app.dependency_overrides[require_control] = lambda: None + client = TestClient(app) + resp = client.post( + "/api/embryos/e1/ground_truth", + json={"stage": "comma", "start_timepoint": 5, "session_id": "demo"}, + ) + assert resp.status_code == 200 + store.set_ground_truth.assert_called_once_with("demo", "e1", "comma", 5, None, None, None) diff --git a/tests/test_experiment_store.py b/tests/test_experiment_store.py new file mode 100644 index 00000000..7b146bd5 --- /dev/null +++ b/tests/test_experiment_store.py @@ -0,0 +1,41 @@ +"""Tests for the ELN Experiment store (FileContextStore, spine phase 2).""" + +from gently.harness.memory.file_store import FileContextStore + + +def test_create_and_get_experiment(tmp_path): + s = FileContextStore(tmp_path / "agent") + eid = s.create_experiment( + "32C vs control", + arms=[{"name": "hot", "strain_ref": "OH904", "condition": "25C"}], + controls=["control"], + ) + e = s.get_experiment(eid) + assert e["title"] == "32C vs control" + assert e["arms"][0]["name"] == "hot" + assert e["arms"][0]["strain_ref"] == "OH904" + assert e["controls"] == ["control"] + + +def test_arm_strain_alias(tmp_path): + s = FileContextStore(tmp_path / "agent") + eid = s.create_experiment("e", arms=[{"name": "a", "strain": "N2", "condition": "c"}]) + assert s.get_experiment(eid)["arms"][0]["strain_ref"] == "N2" # `strain` → strain_ref + + +def test_link_session_to_arm(tmp_path): + s = FileContextStore(tmp_path / "agent") + eid = s.create_experiment("e", arms=[{"name": "hot", "strain_ref": "x", "condition": "c"}]) + assert s.link_session_to_arm(eid, "hot", "sess1") is True + assert "sess1" in s.get_experiment(eid)["arms"][0]["session_ids"] + assert s.link_session_to_arm(eid, "missing", "sess1") is False + # idempotent + s.link_session_to_arm(eid, "hot", "sess1") + assert s.get_experiment(eid)["arms"][0]["session_ids"].count("sess1") == 1 + + +def test_list_experiments(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_experiment("a") + s.create_experiment("b") + assert len(s.list_experiments()) == 2 diff --git a/tests/test_hf_connector.py b/tests/test_hf_connector.py new file mode 100644 index 00000000..6f4d4a74 --- /dev/null +++ b/tests/test_hf_connector.py @@ -0,0 +1,58 @@ +"""Tests for the approval-gated HuggingFace export connector.""" + +import pytest + +from gently.eln import hf_connector as hc + + +def test_build_records_maps_prediction_and_ground_truth(): + rows = hc.build_records( + [ + { + "session_id": "s", + "embryo_id": "e", + "predicted_stage": "comma", + "stage": "bean", + "start_timepoint": 10, + "end_timepoint": 20, + "annotator": "kesh", + "strain": "OH904", + } + ] + ) + assert rows[0]["ground_truth_stage"] == "bean" # falls back to `stage` + assert rows[0]["predicted_stage"] == "comma" + assert rows[0]["start_timepoint"] == 10 + assert rows[0]["strain"] == "OH904" + + +def test_push_requires_token(monkeypatch): + monkeypatch.delenv("HF_TOKEN", raising=False) + monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False) + with pytest.raises(hc.HFExportError): + hc.push_dataset([{"a": 1}], token=None) + + +def test_push_empty_records_raises(): + with pytest.raises(hc.HFExportError): + hc.push_dataset([], token="x") + + +def test_push_uses_injected_fn_and_default_repo(): + seen = {} + + def fake(records, repo, split, revision, token): + seen.update(repo=repo, n=len(records), split=split, token=token) + + out = hc.push_dataset([{"a": 1}, {"a": 2}], split="train", token="tok", _push_fn=fake) + assert out["n"] == 2 + assert out["repo"] == "pskeshu/gently-perception-benchmark" + assert seen["n"] == 2 and seen["token"] == "tok" + + +def test_token_present(monkeypatch): + monkeypatch.setenv("HF_TOKEN", "abc") + assert hc.token_present() is True + monkeypatch.delenv("HF_TOKEN") + monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False) + assert hc.token_present() is False diff --git a/tests/test_hypothesis_store.py b/tests/test_hypothesis_store.py new file mode 100644 index 00000000..f67dd4d7 --- /dev/null +++ b/tests/test_hypothesis_store.py @@ -0,0 +1,36 @@ +"""Tests for the ELN Hypothesis store (FileContextStore, spine phase 3).""" + +import pytest + +from gently.harness.memory.file_store import FileContextStore + + +def test_create_get_hypothesis(tmp_path): + s = FileContextStore(tmp_path / "agent") + hid = s.create_hypothesis( + "25C shifts hatching-time distribution later", + predictions=[{"target": "median_hatch", "expected": "+30min"}], + author="human", + ) + h = s.get_hypothesis(hid) + assert h["statement"].startswith("25C") + assert h["status"] == "proposed" + assert h["predictions"][0]["target"] == "median_hatch" + assert h["author"] == "human" + + +def test_set_status(tmp_path): + s = FileContextStore(tmp_path / "agent") + hid = s.create_hypothesis("h") + assert s.set_hypothesis_status(hid, "supported") is True + assert s.get_hypothesis(hid)["status"] == "supported" + with pytest.raises(ValueError): + s.set_hypothesis_status(hid, "bogus") + assert s.set_hypothesis_status("nope", "refuted") is False + + +def test_list_hypotheses(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_hypothesis("a") + s.create_hypothesis("b") + assert len(s.list_hypotheses()) == 2 diff --git a/tests/test_result_store.py b/tests/test_result_store.py new file mode 100644 index 00000000..a765eb39 --- /dev/null +++ b/tests/test_result_store.py @@ -0,0 +1,22 @@ +"""Tests for the ELN Result store (FileContextStore, spine phase 3).""" + +from gently.eln.accuracy import accuracy_result +from gently.harness.memory.file_store import FileContextStore + + +def test_save_and_filter_results(tmp_path): + s = FileContextStore(tmp_path / "agent") + r = accuracy_result( + "sess", + "emb", + [{"timepoint": 1, "predicted_stage": "bean"}], + [{"stage": "bean", "start_timepoint": 0, "end_timepoint": 5}], + ) + r["experiment_ref"] = "exp1" + rid = s.save_result(r) + got = s.get_result(rid) + assert got["kind"] == "accuracy" + assert got["experiment_ref"] == "exp1" + assert len(s.list_results(experiment_ref="exp1")) == 1 + assert len(s.list_results(experiment_ref="other")) == 0 + assert len(s.list_results()) == 1 diff --git a/tests/test_seed_loader.py b/tests/test_seed_loader.py new file mode 100644 index 00000000..60b137ae --- /dev/null +++ b/tests/test_seed_loader.py @@ -0,0 +1,40 @@ +"""Tests for the ELN seed-program loader.""" + +from pathlib import Path + +from gently.eln.seed_loader import load_program, load_seed_dir +from gently.harness.memory.file_store import FileContextStore + + +def test_load_program(tmp_path): + s = FileContextStore(tmp_path / "agent") + program = { + "slug": "demo", + "campaign": {"title": "T", "goal": "G", "description": "D"}, + "strains": [{"name": "OH904", "genotype": "otIs355"}], + "experiments": [ + {"title": "E1", "arms": [{"name": "a", "strain": "OH904", "condition": "c"}]} + ], + "hypotheses": [{"statement": "H1", "predictions": ["p"]}], + } + out = load_program(s, program) + assert s.get_campaign(out["campaign_id"]) is not None + assert len(out["strain_ids"]) == 1 + assert len(out["experiment_ids"]) == 1 + e = s.get_experiment(out["experiment_ids"][0]) + assert e["arms"][0]["strain_ref"] == "OH904" + h = s.get_hypothesis(out["hypothesis_ids"][0]) + assert h["predictions"][0]["target"] == "p" + + +def test_load_real_seed_dir(): + """The shipped seed/programs/ load without error into a fresh store.""" + import tempfile + + seed_dir = Path(__file__).resolve().parents[1] / "seed" / "programs" + with tempfile.TemporaryDirectory() as td: + s = FileContextStore(Path(td) / "agent") + loaded = load_seed_dir(s, seed_dir) + assert len(loaded) == 4 # the 4 real programs + assert len(s.list_experiments()) >= 4 + assert len(s.list_strains()) >= 4 diff --git a/tests/test_strain_resolution.py b/tests/test_strain_resolution.py new file mode 100644 index 00000000..3ddbe233 --- /dev/null +++ b/tests/test_strain_resolution.py @@ -0,0 +1,22 @@ +"""Tests for display-side strain resolution (non-mutating, bare-string safe).""" + +from gently.eln.strain_resolution import enrich_imaging_spec, resolve_ref +from gently.harness.memory.file_store import FileContextStore + + +def test_resolve_ref_record_and_shim(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_strain("OH904", genotype="otIs355") + assert resolve_ref(s, "OH904")["genotype"] == "otIs355" # real record + assert resolve_ref(s, "unknown-str") == {"name": "unknown-str"} # bare-string shim + assert resolve_ref(s, None) is None + + +def test_enrich_imaging_spec_non_mutating(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_strain("N2") + spec = {"strain": "N2", "num_slices": 80} + out = enrich_imaging_spec(s, spec) + assert out["strain"] == "N2" # original preserved + assert out["strain_record"]["name"] == "N2" + assert "strain_record" not in spec # did not mutate input diff --git a/tests/test_strain_store.py b/tests/test_strain_store.py new file mode 100644 index 00000000..2e080c19 --- /dev/null +++ b/tests/test_strain_store.py @@ -0,0 +1,36 @@ +"""Tests for the ELN Strain store (FileContextStore, spine phase 1). + +Strain is an optional overlay with bare-string back-compat: a strain reference +that matches no record resolves to None so callers keep the display string. +""" + +from gently.harness.memory.file_store import FileContextStore + + +def test_create_and_get_strain(tmp_path): + s = FileContextStore(tmp_path / "agent") + sid = s.create_strain( + "OH904", genotype="otIs355", markers=["rab-3p::GFP"], organism_ref="celegans" + ) + rec = s.get_strain(sid) + assert rec is not None + assert rec["name"] == "OH904" + assert rec["genotype"] == "otIs355" + assert rec["markers"] == ["rab-3p::GFP"] + assert rec["organism_ref"] == "celegans" + + +def test_list_strains(tmp_path): + s = FileContextStore(tmp_path / "agent") + s.create_strain("A") + s.create_strain("B") + assert len(s.list_strains()) == 2 + + +def test_resolve_strain_by_id_and_name(tmp_path): + s = FileContextStore(tmp_path / "agent") + sid = s.create_strain("N2") + assert s.resolve_strain(sid)["name"] == "N2" # by id + assert s.resolve_strain("n2")["id"] == sid # case-insensitive name + assert s.resolve_strain("nonexistent") is None # bare string, no match → None + assert s.resolve_strain(None) is None