KonfAI Studio: a truthful chat over konfai-mcp, apps and runtime hardened - #71
Merged
Conversation
is_dataset_exist answered from a directory listing memoised at first use, so a group produced during the run through another Dataset object -- or, with loader workers, another process -- stayed invisible: the auto-mask path of a synthesis bundle read the mask its own run had just written and found nothing, in 4 runs out of 10. Membership now asks the disk for the entry itself, and a whole-volume write publishes by rename (a dotted staging name, so extension-based writer selection survives). On h5 the pooled read handle kept serving the view it opened on; reopening alone would not help, since HDF5 shares a file's metadata state across the handles one process holds. The pool now closes a handle whose store changed underneath it, and retries the reopen that lands while the writer is still mid-transaction. The transforms doc says what a Save-written group promises on each backend. Measured: membership 568 us -> 6.3 us (directory) / 1.8 us (h5); the stat probe costs 0.7% of a pooled read.
The evaluator placed nothing on the device it was given, so every metric ran on CPU -- 1659 s for an evaluation the GPU does in a fraction of that. And PSNR and SSIM carried no maximize flag, so any consumer ranking runs by them put the worst first.
The mirror replayed every raw redraw frame of a progress bar to what it believes is the console. Inside an MCP job or under nohup/SLURM that target is a file, where nothing overwrites: the animation alone reached 1.6 MB per smoke run (6369 frames), uncapped. Off a terminal the mirror now sends the folded line, at most one frame per 2 s window per bar; withheld frames land before the next message and at exit, so a bar's final state is never lost -- including with interleaved train/validation bars. A CRLF-terminated line is a message, not an animation frame: classified as one it vanished from the mirror and the log file both. On a real console nothing changes.
Resolving an app re-checked every file of the HuggingFace snapshot against the hub before each job; the check is network-bound and the answer never changes within a session. The snapshot is trusted once resolved.
Apps run as normal experiments again: import_app brings a bundle into the
session and run_prediction / run_resume / run_evaluation drive it, with the
app-execution tools and their tests restored. The experiment's state is
derived from the workspace on disk -- the same answer for every client --
and each stage names the step it is waiting on.
The wait_for_job freeze is sealed at fd level. redirect_stdout is
Python-level: fds 1/2 stayed on the inherited descriptors -- the JSON-RPC
stream itself -- so anything writing natively (KonfAI's console mirror in a
DDP grandchild, pip during app resolution) destroyed the response it
followed. Jobs dup2 their log onto fds 1/2 and devnull onto fd 0 before the
target runs; the isolated API child does the same into a sink whose tail is
attached to any failure, including a dup2 that itself fails. Verified on
the real transport: pre-fix the stream wedges at the fifth poll, post-fix
26/26 frames parse and the bytes sit in the job log.
Payloads say what an agent needs: inspect_dataset is one object per group;
a leaderboard metric ranks one way for every run; a weightless model runs
without a checkpoint; a job ending on SIGTERM with cancellation requested
is 'killed' and diagnosed as a deliberate stop, never a failure to
investigate. The fixed index cost drops ~2300 tokens per turn ('Inputs:'
sentences restated the Field descriptions; duplicated wordings unified);
the committed tool reference is regenerated.
The chat is driven by the experiment state the MCP server derives, and every turn ends in buttons that are the reply's real next moves -- its answers when it asks, the corrections when a run fails, never a padded count. The live feed tells the truth: a run tab opens at launch, curves and progress stream in, Stop resolves, and a stopped run is acknowledged as the user's deliberate act. The viewer follows what the assistant names, two volumes side by side with linked crosshairs. The transcript is served by the server: it records every turn it streams (interrupted ones included), and a browser whose localStorage has fewer turns adopts the copy -- an experiment driven from one machine no longer looks empty from another. The rail discovers experiments created elsewhere through the status poll it already pays for. Run tabs scroll instead of clipping, carry their status, and close without deleting anything; the mic says why it cannot work on an insecure origin; --ssl-certfile/--ssl-keyfile serve TLS directly (REMOTE.md shows the self-signed one-liner), with an empty flag refused rather than silently downgraded to plain http.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR adds app execution and experiment-state workflows, changes dataset and runtime I/O behavior, and integrates server-derived workflow state into Studio chat, live feeds, telemetry, and workspace controls. It also adds regression tests and refreshed documentation. ChangesMCP app workflows and experiment state
Core runtime and dataset I/O
Studio workflow and streaming interface
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 16
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
konfai-mcp/konfai_mcp/dataset_inspection.py (1)
314-316: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winUpdate the stale
is_input_meaningreference.The removed payload field is still named in the adjacent comment, so the guidance points at a key that no longer exists. Point the comment to
design_config_strategyor group-role clarification instead.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/dataset_inspection.py` around lines 314 - 316, Update the adjacent comment near the payload containing dataset_entry and suggested_groups_src to remove the stale is_input_meaning reference, and instead direct guidance toward design_config_strategy or clarifying group roles.konfai-mcp/konfai_mcp/server.py (1)
1035-1056: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
inspect_dataset's advertised outputs still describe the removed siblingstatisticsmap.Statistics now live under
payload["groups"][group], and the top-levelstatistics/statistics_groups/statistics_sample_limitkeys are gone, butkonfai-mcp/konfai_mcp/guide.pyLine 49 still says"Outputs: groups, statistics, dataset_entry hints, warnings, and next_actions."An agent reading the tool index will look for a key that no longer exists.📝 Suggested description fix (konfai-mcp/konfai_mcp/guide.py)
- "Outputs: groups, statistics, dataset_entry hints, warnings, and next_actions. " + "Outputs: groups (each carrying its own sampled statistics), dataset_entry hints, warnings, " + "statistics_missing_groups/reasons, and next_actions. "🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/server.py` around lines 1035 - 1056, Update the inspect_dataset output description in guide.py to remove the obsolete top-level “statistics” output and describe statistics as being included within “groups”. Keep the remaining advertised outputs, including dataset_entry hints, warnings, and next_actions, unchanged.konfai-mcp/konfai_mcp/runner.py (1)
101-161: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winTimeout path leaks the temp dir and drops the child's captured output.
The
TimeoutErrorat Lines 116-119 returns before_drain(output)at Line 142, soPath(tempfile.mkdtemp(...))is never removed (one leaked dir + log per timed-out job), and the tail — the only evidence of what a hung child was doing — is discarded exactly where it is most useful.🛡️ Proposed fix: drain in a finally / include the tail in the timeout
- process = context.Process(target=entry, args=(queue, target, kwargs, str(output)), daemon=True) - process.start() - result: dict[str, Any] | None = None - while result is None: - try: - result = queue.get(timeout=0.5) - except Empty: - if deadline is not None and time.monotonic() > deadline and process.is_alive(): - process.terminate() - process.join(5) - if process.is_alive(): - process.kill() - process.join(5) - raise TimeoutError( - f"Isolated subprocess '{target}' exceeded {timeout_s:.0f}s and was terminated. " - "Raise KONFAI_MCP_SUBPROCESS_TIMEOUT if this is a large model, or simplify the config." - ) from None + process = context.Process(target=entry, args=(queue, target, kwargs, str(output)), daemon=True) + process.start() + result: dict[str, Any] | None = None + while result is None: + try: + result = queue.get(timeout=0.5) + except Empty: + if deadline is not None and time.monotonic() > deadline and process.is_alive(): + process.terminate() + process.join(5) + if process.is_alive(): + process.kill() + process.join(5) + printed = _drain(output) + message = ( + f"Isolated subprocess '{target}' exceeded {timeout_s:.0f}s and was terminated. " + "Raise KONFAI_MCP_SUBPROCESS_TIMEOUT if this is a large model, or simplify the config." + ) + raise TimeoutError( + f"{message}\n\nSubprocess output (tail):\n{printed}" if printed else message + ) from None🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/runner.py` around lines 101 - 161, Update the subprocess timeout handling around the process loop and _drain so timed-out jobs always drain and remove the captured output directory before raising. Preserve the existing termination escalation, capture the returned output tail, and include it in the TimeoutError when present so diagnostics are retained.
🧹 Nitpick comments (9)
konfai/evaluator.py (1)
444-448: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value
non_blockingon a CPU→GPU copy of unpinned memory is a no-op; and D2D copies lose it.
tensor.device.type == "cpu"gates async copies to the H2D direction, but the evaluation tensors come straight from the DataLoader and are only asynchronous if pinned. Harmless, just note it buys nothing unless the loader pins memory.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai/evaluator.py` around lines 444 - 448, Update the _on helper’s tensor.to call so non_blocking is enabled only when the source CPU tensor is pinned, preserving synchronous behavior for unpinned DataLoader tensors and device-to-device copies.konfai/utils/runtime.py (2)
469-489: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value
_mirror_pendingis mutated from every thread that writes to stdout.
MinimalLogreplaces the process-widesys.stdout, so bar frames can arrive from more than one thread (tqdm in a background validation/prediction thread)._mirror_take_pendingiterates_mirror_pendingand then clears it, so a concurrentwritecan have its frame dropped or hit a mid-iteration mutation. A small lock around the pending dict (ordict.copy()+swap) would make this deterministic.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai/utils/runtime.py` around lines 469 - 489, The _mirror_pending dictionary in MinimalLog is accessed concurrently by stdout-writing threads, allowing _mirror_take_pending and write to race during iteration and clearing. Add a dedicated lock initialized alongside _mirror_pending, and use it to protect every pending-dictionary read, update, iteration, and clear in the related mirror methods, preserving the existing throttling and final-frame behavior.
519-534: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe throttle timestamp is global, so interleaved bars share one window.
_mirror_last_redrawis a single instant for all bars: with a train and a validation bar redrawing together, only one bar's frame is emitted per 2 s window and the other's live state stays pending until the window rolls (or__exit__). A tailed log then shows one bar advancing and the other frozen. Keying the last-emit time per_bar_keyrestores a truthful live feed for both at the same cost.Also worth noting: the mirror is folded, but
Log.write(Line 606-610) still appends_buffered_linefor every frame tolog_<rank>.txt, so the on-disk run log keeps the per-frame growth this change set out to remove. Please confirm that is intended.♻️ Proposed per-bar throttle
- self._mirror_last_redraw = 0.0 + self._mirror_last_redraw: dict[str, float] = {}if redraw: now = time.monotonic() - if now - self._mirror_last_redraw < self._MIRROR_REDRAW_EVERY: - self._mirror_pending[_bar_key(self._buffered_line)] = self._buffered_line + key = _bar_key(self._buffered_line) + if now - self._mirror_last_redraw.get(key, 0.0) < self._MIRROR_REDRAW_EVERY: + self._mirror_pending[key] = self._buffered_line return - self._mirror_last_redraw = now - held = self._mirror_take_pending(exclude=_bar_key(self._buffered_line)) + self._mirror_last_redraw[key] = now + held = self._mirror_take_pending(exclude=key) msg = f"{held}{self._buffered_line}\n"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai/utils/runtime.py` around lines 519 - 534, Update _mirror so redraw throttling uses a per-bar timestamp keyed by _bar_key(self._buffered_line), allowing interleaved bars to emit independently while preserving pending-frame handling. Also update Log.write so log_<rank>.txt does not append every redraw frame through _buffered_line; ensure the on-disk log follows the intended folded, throttled output behavior.konfai/utils/dataset.py (1)
160-173: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valuePutting an evicted handle back bypasses the LRU bound.
_close_idlere-inserts undersetdefaultwithout re-checking_MAX, so a file whose lock is repeatedly busy keeps the pool above the bound (and the reinserted entry lands at the tail, making it the last eviction candidate). Not harmful at_MAX = 8, but the invariant the comment relies on ("LRU-bounded") no longer holds strictly.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai/utils/dataset.py` around lines 160 - 173, The _close_idle method must preserve the pool’s _MAX LRU bound when reinserting a handle whose file lock is busy. Re-check capacity while holding _guard, retain the handle only when space is available, and otherwise evict/close the appropriate existing entry so reinsertion cannot exceed the limit or undermine LRU ordering.konfai-mcp/README.md (1)
231-232: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
set_parametersis a tool argument, not a tool.Listing it beside
list_app_parametersreads as a second tool name; the surrounding bullets are all tool names. Elsewhere in this doc/registry it is the per-run parameter ofrun_app_*/export_app.📝 Suggested wording
- - `list_app_parameters` / `set_parameters` — read tunable parameters (with - their constraints) and override them per run + - `list_app_parameters` — read tunable parameters (with their constraints); + override them per run via the `set_parameters` argument of `run_app_*`🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/README.md` around lines 231 - 232, Update the README entry containing `list_app_parameters` / `set_parameters` so `set_parameters` is described as a per-run argument for `run_app_*` and `export_app`, not as a separate tool name; keep `list_app_parameters` identified as the tool.konfai-mcp/konfai_mcp/runner.py (1)
361-373: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low valueConsider an explicit action allowlist before the
getattrdispatch.
actionis trusted to come from_prepare_action(onlyevaluate/uncertainty/pipeline), so nothing is exploitable today, but this is a public runner entrypoint invoked with job-persisted kwargs — a recovered/hand-edited job spec would call an arbitraryKonfAIAppattribute. A one-line membership check makes the contract local and turns a mismatch into a clear error instead of aTypeErrordeep in the child.♻️ Proposed guard
with _runtime_context(cwd=Path(cwd).resolve() if cwd is not None else None): _ensure_local_imports() + if action not in ("evaluate", "uncertainty", "pipeline"): + raise ValueError(f"Unsupported app action: {action!r}") from konfai_apps.app import KonfAIApp🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/runner.py` around lines 361 - 373, Update the public runner dispatch around getattr(KonfAIApp(...), action) to validate action against the explicit supported allowlist: evaluate, uncertainty, and pipeline. Reject any other value with a clear error before attribute lookup or invocation, while preserving the existing argument construction and dispatch behavior for allowed actions.studio/konfai_studio/agent.py (1)
618-634: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winAn always-empty
mcpServerslist costs the full 15 s on every session open.
if servers and all(...)never returns early when the SDK build reports an empty list (or omits the key), so__aenter__blocks for_MCP_READY_TIMEOUTbefore the first turn instead of the ~2 s handshake this is meant to cover. Treating "no servers reported" as "nothing to wait for" keeps the guard for the real case and drops the pathological one.♻️ Suggested tweak
- servers = (status or {}).get("mcpServers") or [] - if servers and all(s.get("status") != "pending" for s in servers): - return + servers = (status or {}).get("mcpServers") + if not servers: # nothing reported: this build has no status to wait on + return + if all(s.get("status") != "pending" for s in servers): + return🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@studio/konfai_studio/agent.py` around lines 618 - 634, Update _await_mcp_tools so an empty or missing mcpServers collection returns immediately as “nothing to wait for,” while retaining the existing wait until all reported servers are no longer pending.studio/frontend/src/RightPanel.tsx (1)
1453-1463: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueThe tab-close affordance is mouse-only.
A
<span>withonClickis not focusable and has no keyboard activation, so a keyboard user cannot close a run tab (nesting a real<button>inside the tab button is invalid, so the usual fix isrole="button" tabIndex={0}plus anonKeyDownfor Enter/Space, or lifting the ✕ out of the tab button). The "+N closed" chip is a button, so this is the only unreachable control here.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@studio/frontend/src/RightPanel.tsx` around lines 1453 - 1463, The tab-close span in the tab rendering should be keyboard-accessible. Update the rtab-close element with button semantics, make it focusable, and handle Enter and Space in onKeyDown by invoking the same hide/show behavior as onClick while preventing unintended tab activation; preserve event propagation prevention.studio/frontend/src/Chat.tsx (1)
516-524: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueTwo moves sharing a label collide on
key.The LLM writes these labels freely and
_NextMoves.moves()instudio/konfai_studio/agent.pydoes not dedupe them (unlikeworkflow.moves), so a repeated 2-4 word label yields duplicate React keys and one chip can drop or mis-update.key={\${c.label}-${i}`}(or deduping inmoves()`) settles it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@studio/frontend/src/Chat.tsx` around lines 516 - 524, Update the `shown.map` rendering in `Chat.tsx` to give each prompt chip a unique React key even when multiple moves share the same `label`; use the map index together with `c.label` (or deduplicate moves in `_NextMoves.moves()`). Preserve the existing prompt labels, titles, and click behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/usage/mcp.md`:
- Around line 45-49: The published-app guidance around run_app_infer and
run_app_pipeline and the corresponding tool table must distinguish local/Hugging
Face apps from remote host:port:name servers. State that remote apps should be
run directly with konfai-apps, while retaining the MCP-tool workflow for
supported local and Hugging Face apps.
In `@konfai-mcp/konfai_mcp/dataset_inspection.py`:
- Around line 57-58: Update the ignored-files assignment in the dataset
inspection layout-building logic to slice summary["ignored_files"] using a
dedicated cap before storing it in entry["ignored_files"]. Preserve the existing
conditional and avoid modifying entries that already define ignored_files.
In `@konfai-mcp/konfai_mcp/experiment_state.py`:
- Around line 104-116: Update the validation or registry-drift checks to
explicitly inspect konfai_mcp.experiment_state.STAGE_ACTIONS, verifying every
listed action is registered as a tool and failing with a clear diagnostic for
unknown names. Reuse the existing job-payload guard’s registry source and
validation behavior, while preserving the current stage-to-action defaults.
In `@konfai-mcp/konfai_mcp/metrics_service.py`:
- Around line 303-315: Extract the metric-direction resolution currently used in
compare_runs into a shared helper, including direction_source determination and
stale-direction warnings. Update compare_runs_payload to use this normalized
current direction for both runs instead of choosing declared_a.get(name) or
declared_b.get(name), so swapping run_a and run_b cannot change the winner.
Preserve explicit direction overrides and the existing leaderboard behavior.
In `@konfai-mcp/konfai_mcp/server_jobs.py`:
- Around line 40-60: Update _detach_inherited_stdio so opening devnull and
detaching file descriptor 0 use a separate try/except from the stdout/stderr
dup2 calls. Ensure any stdin-related OSError is reported without returning
early, then always attempt os.dup2(handle.fileno(), 1) and
os.dup2(handle.fileno(), 2), while preserving the existing error reporting
behavior.
In `@konfai-mcp/konfai_mcp/server_support.py`:
- Around line 499-503: Update the “Uncertainty” guidance in the authoring_checks
configuration to clarify that run_app_infer(uncertainty=True) retains
multi-channel inference stacks, which are then consumed by run_app_uncertainty
to produce uncertainty maps; do not describe inference completion itself as the
uncertainty result.
In `@konfai-mcp/tests/test_mcp_server_apps.py`:
- Around line 252-419: Add an end-to-end async fastmcp.Client test covering the
new app-execution tools, using a stubbed launch path to avoid starting a real
job. Invoke run_app_infer through mcp_server.mcp and assert its validated
parameters, completed output and metrics, plus the returned output,
set_parameters, and next_actions payload fields. Extend coverage as needed for
run_app_evaluate, run_app_uncertainty, run_app_pipeline, and fine_tune_app so
all five client-exposed tools are exercised.
In `@konfai/utils/dataset.py`:
- Around line 1232-1239: Update the staging-name handling in the image write
path around sitk.WriteImage and os.replace so temporary files retain a suffix
recognized by the existing _resolve_data_path and file_to_data .tmp checks,
while preserving the format extension required by SimpleITK. Also update
Dataset.get_group to ignore leading-dot staging names so in-progress or orphaned
files cannot produce an empty group name.
- Around line 1906-1927: Update is_dataset_exist in the directory-backed branch
so it returns True only when the HDF5 leaf for the requested name exists under
entry_group, rather than relying solely on file.is_exist(entry_group). Preserve
the existing directory probing and return False behavior when the requested
child is absent.
In `@studio/frontend/src/App.tsx`:
- Around line 351-355: Update the setGpuVramHist callback to skip appending a
VRAM sample when the GPU has no valid memory usage or total value, matching the
existing gpuHist guard for null utilization. Only append pct(g.used_gb,
g.total_gb) for valid reports, while preserving existing history entries and cap
behavior.
In `@studio/frontend/src/Chat.tsx`:
- Around line 190-210: Update the history-loading useEffect and its setMessages
merge logic to detect whether a turn is currently in flight, and skip replacing
the local message list while that turn is active. Preserve the existing
stale-session guard and server-history merge behavior when no send/streaming
operation is running, ensuring the assistant placeholder used by send and
patchAssistant remains available.
In `@studio/konfai_studio/agent.py`:
- Around line 643-657: Reset _interrupted at the start of every new turn, before
processing that turn’s result or error state, so a late interrupt cannot
suppress a subsequent turn’s genuine failure. Update the turn-start logic
associated with interrupt handling and preserve the existing
reset-on-interrupt-exception behavior.
In `@studio/konfai_studio/jobs.py`:
- Around line 276-290: Update the duplicate-run naming expression in the list
comprehension around _discover_run_logs so the disambiguating suffix is derived
from the actual `-<hash>` component of the output directory, regardless of
whether the log is nested under Statistics, Predictions, Evaluations, or
Uncertainties. Preserve the existing run name for non-duplicates and continue
using the suffix only when `(run, kind)` is duplicated and `base` is present.
In `@studio/konfai_studio/server.py`:
- Around line 227-235: Guard the post-turn state and next-prompts generation
within the stream’s existing protected flow so failures from `_state` or
`_turn_moves` do not escape `gen()` and prevent the final events; preserve the
intended state/move completion behavior. In the `record_turn` call, offload the
synchronous file I/O to a worker thread using the same `asyncio.to_thread`
pattern as `_system_snapshot`, awaiting it before emitting the post-turn events.
In `@studio/tests/test_live_feed.py`:
- Around line 45-69: Update the collect function in feed_events to replace
asyncio.timeout with a timeout mechanism compatible with Python 3.10, while
preserving the existing two-second limit, event collection, cancellation
handling, and stream cleanup behavior.
In `@tests/unit/test_dataset.py`:
- Around line 557-595: Update
test_an_evicted_h5_handle_goes_back_with_the_view_it_had so the child process is
always cleaned up after join(120). Wrap the join and exit-code assertion in a
try/finally block, and call child.terminate() in finally only when
child.is_alive() remains true.
---
Outside diff comments:
In `@konfai-mcp/konfai_mcp/dataset_inspection.py`:
- Around line 314-316: Update the adjacent comment near the payload containing
dataset_entry and suggested_groups_src to remove the stale is_input_meaning
reference, and instead direct guidance toward design_config_strategy or
clarifying group roles.
In `@konfai-mcp/konfai_mcp/runner.py`:
- Around line 101-161: Update the subprocess timeout handling around the process
loop and _drain so timed-out jobs always drain and remove the captured output
directory before raising. Preserve the existing termination escalation, capture
the returned output tail, and include it in the TimeoutError when present so
diagnostics are retained.
In `@konfai-mcp/konfai_mcp/server.py`:
- Around line 1035-1056: Update the inspect_dataset output description in
guide.py to remove the obsolete top-level “statistics” output and describe
statistics as being included within “groups”. Keep the remaining advertised
outputs, including dataset_entry hints, warnings, and next_actions, unchanged.
---
Nitpick comments:
In `@konfai-mcp/konfai_mcp/runner.py`:
- Around line 361-373: Update the public runner dispatch around
getattr(KonfAIApp(...), action) to validate action against the explicit
supported allowlist: evaluate, uncertainty, and pipeline. Reject any other value
with a clear error before attribute lookup or invocation, while preserving the
existing argument construction and dispatch behavior for allowed actions.
In `@konfai-mcp/README.md`:
- Around line 231-232: Update the README entry containing `list_app_parameters`
/ `set_parameters` so `set_parameters` is described as a per-run argument for
`run_app_*` and `export_app`, not as a separate tool name; keep
`list_app_parameters` identified as the tool.
In `@konfai/evaluator.py`:
- Around line 444-448: Update the _on helper’s tensor.to call so non_blocking is
enabled only when the source CPU tensor is pinned, preserving synchronous
behavior for unpinned DataLoader tensors and device-to-device copies.
In `@konfai/utils/dataset.py`:
- Around line 160-173: The _close_idle method must preserve the pool’s _MAX LRU
bound when reinserting a handle whose file lock is busy. Re-check capacity while
holding _guard, retain the handle only when space is available, and otherwise
evict/close the appropriate existing entry so reinsertion cannot exceed the
limit or undermine LRU ordering.
In `@konfai/utils/runtime.py`:
- Around line 469-489: The _mirror_pending dictionary in MinimalLog is accessed
concurrently by stdout-writing threads, allowing _mirror_take_pending and write
to race during iteration and clearing. Add a dedicated lock initialized
alongside _mirror_pending, and use it to protect every pending-dictionary read,
update, iteration, and clear in the related mirror methods, preserving the
existing throttling and final-frame behavior.
- Around line 519-534: Update _mirror so redraw throttling uses a per-bar
timestamp keyed by _bar_key(self._buffered_line), allowing interleaved bars to
emit independently while preserving pending-frame handling. Also update
Log.write so log_<rank>.txt does not append every redraw frame through
_buffered_line; ensure the on-disk log follows the intended folded, throttled
output behavior.
In `@studio/frontend/src/Chat.tsx`:
- Around line 516-524: Update the `shown.map` rendering in `Chat.tsx` to give
each prompt chip a unique React key even when multiple moves share the same
`label`; use the map index together with `c.label` (or deduplicate moves in
`_NextMoves.moves()`). Preserve the existing prompt labels, titles, and click
behavior.
In `@studio/frontend/src/RightPanel.tsx`:
- Around line 1453-1463: The tab-close span in the tab rendering should be
keyboard-accessible. Update the rtab-close element with button semantics, make
it focusable, and handle Enter and Space in onKeyDown by invoking the same
hide/show behavior as onClick while preventing unintended tab activation;
preserve event propagation prevention.
In `@studio/konfai_studio/agent.py`:
- Around line 618-634: Update _await_mcp_tools so an empty or missing mcpServers
collection returns immediately as “nothing to wait for,” while retaining the
existing wait until all reported servers are no longer pending.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4d132351-7991-485b-a2ea-155cfc84e508
📒 Files selected for processing (57)
.claude/skills/konfai-experiments/SKILL.md.claude/skills/konfai-experiments/references/tool-reference.mddocs/source/reference/components/transforms.mddocs/source/usage/mcp.mdkonfai-apps/konfai_apps/app.pykonfai-mcp/README.mdkonfai-mcp/konfai_mcp/capabilities.pykonfai-mcp/konfai_mcp/dataset_inspection.pykonfai-mcp/konfai_mcp/experiment_state.pykonfai-mcp/konfai_mcp/guide.pykonfai-mcp/konfai_mcp/metrics_service.pykonfai-mcp/konfai_mcp/runner.pykonfai-mcp/konfai_mcp/server.pykonfai-mcp/konfai_mcp/server_apps.pykonfai-mcp/konfai_mcp/server_experiments.pykonfai-mcp/konfai_mcp/server_jobs.pykonfai-mcp/konfai_mcp/server_support.pykonfai-mcp/konfai_mcp/workflows.pykonfai-mcp/tests/mcp_test_helpers.pykonfai-mcp/tests/test_experiment_state.pykonfai-mcp/tests/test_mcp_server_apps.pykonfai-mcp/tests/test_mcp_server_dataset_tools.pykonfai-mcp/tests/test_mcp_server_fixes.pykonfai-mcp/tests/test_mcp_server_jobs.pykonfai-mcp/tests/test_mcp_server_lot5.pykonfai-mcp/tests/test_mcp_server_refine_loop.pykonfai-mcp/tests/test_mcp_server_tool_index.pykonfai-mcp/tests/test_review_mcp_apps_misc.pykonfai-mcp/tests/test_workflow_registry.pykonfai/evaluator.pykonfai/metric/measure.pykonfai/utils/dataset.pykonfai/utils/runtime.pystudio/docs/REMOTE.mdstudio/frontend/.gitignorestudio/frontend/src/App.tsxstudio/frontend/src/Chat.tsxstudio/frontend/src/RightPanel.tsxstudio/frontend/src/sessionState.tsstudio/frontend/src/styles.cssstudio/frontend/src/useJobStream.tsstudio/konfai_studio/agent.pystudio/konfai_studio/cli.pystudio/konfai_studio/jobs.pystudio/konfai_studio/paths.pystudio/konfai_studio/registry.pystudio/konfai_studio/server.pystudio/konfai_studio/tensorboard.pystudio/konfai_studio/transcript.pystudio/konfai_studio/workflow.pystudio/tests/test_chat_flow.pystudio/tests/test_live_feed.pystudio/tests/test_smoke.pystudio/tests/test_workflow.pytests/unit/test_dataset.pytests/unit/test_dataset_streaming.pytests/unit/test_runtime.py
Comment on lines
+557
to
+595
| def test_an_evicted_h5_handle_goes_back_with_the_view_it_had(tmp_path: Path) -> None: | ||
| """A handle evicted while its file lock is busy returns to the pool instead of being closed. Re-stamping | ||
| it on the way back would hand it the store as it is now and launder a stale view into a fresh-looking | ||
| one, so the write that arrived meanwhile would stay invisible for the rest of the process.""" | ||
| pytest.importorskip("h5py") | ||
| from konfai.utils.dataset import _h5_read_pool | ||
|
|
||
| root = str(tmp_path / "ds") + "/" | ||
| Path(root).mkdir() | ||
| dataset = Dataset(root, "h5") | ||
| dataset.write("MASK", "P000", np.ones((4, 4, 4), dtype=np.uint8), Attribute()) | ||
| dataset.is_dataset_exist("MASK", "P000") # pool a handle on the store | ||
| store = dataset.filename + ".h5" | ||
| pooled = _h5_read_pool._handles.pop(store) | ||
|
|
||
| child = multiprocessing.get_context("spawn").Process(target=_write_mask_in_child_h5, args=(root, "P001")) | ||
| child.start() | ||
| child.join(120) | ||
| assert child.exitcode == 0, "the writer process failed; the assertions below would prove nothing" | ||
|
|
||
| # The file lock is reentrant, so only another thread can make it look busy to the evicting one. | ||
| held, release = threading.Event(), threading.Event() | ||
|
|
||
| def hold_the_file_lock() -> None: | ||
| with _get_h5_file_lock(store): | ||
| held.set() | ||
| release.wait(120) | ||
|
|
||
| holder = threading.Thread(target=hold_the_file_lock) | ||
| holder.start() | ||
| try: | ||
| assert held.wait(120) | ||
| _h5_read_pool._close_idle(store, pooled) | ||
| assert _h5_read_pool._handles[store].opened_on == pooled.opened_on, "the view it had, not the store now" | ||
| finally: | ||
| release.set() | ||
| holder.join(120) | ||
|
|
||
| assert dataset.is_dataset_exist("MASK", "P001") |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
sed -n '1,40p' tests/unit/test_dataset.py
rg -n '_get_h5_file_lock|^import threading|threading\.' tests/unit/test_dataset.pyRepository: fideus-labs/KonfAI
Length of output: 2175
🏁 Script executed:
python3 - <<'PY'
import multiprocessing as mp
import time
def sleeper():
time.sleep(2)
p = mp.get_context("spawn").Process(target=sleeper)
p.start()
p.join(0.2)
print("alive_after_timeout=", p.is_alive())
print("exitcode_after_timeout=", p.exitcode)
# leave it running? terminate to avoid orphaning in this probe
p.terminate()
p.join(2)
print("alive_after_terminate=", p.is_alive())
print("exitcode_after_terminate=", p.exitcode)
PYRepository: fideus-labs/KonfAI
Length of output: 1156
🏁 Script executed:
#!/bin/bash
python3 - <<'PY'
import multiprocessing as mp
import time
def sleeper():
time.sleep(2)
p = mp.get_context("spawn").Process(target=sleeper)
p.start()
p.join(0.2)
print("alive_after_timeout=", p.is_alive())
print("exitcode_after_timeout=", p.exitcode)
p.terminate()
p.join(2)
print("alive_after_terminate=", p.is_alive())
print("exitcode_after_terminate=", p.exitcode)
PYRepository: fideus-labs/KonfAI
Length of output: 1156
🏁 Script executed:
cat >/tmp/mp_timeout_probe.py <<'PY'
import multiprocessing as mp
import time
def sleeper():
time.sleep(2)
if __name__ == "__main__":
p = mp.get_context("spawn").Process(target=sleeper)
p.start()
p.join(0.2)
print("alive_after_timeout=", p.is_alive())
print("exitcode_after_timeout=", p.exitcode)
if p.is_alive():
p.terminate()
p.join(2)
print("alive_after_cleanup=", p.is_alive())
print("exitcode_after_cleanup=", p.exitcode)
PY
python3 /tmp/mp_timeout_probe.py
rm -f /tmp/mp_timeout_probe.pyRepository: fideus-labs/KonfAI
Length of output: 266
Terminate the child on timeout
Wrap the join(120) path in finally and call child.terminate() if it is still alive; otherwise a slow CI run can leave the spawned process running.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unit/test_dataset.py` around lines 557 - 595, Update
test_an_evicted_h5_handle_goes_back_with_the_view_it_had so the child process is
always cleaned up after join(120). Wrap the join and exit-code assertion in a
try/finally block, and call child.terminate() in finally only when
child.is_alive() remains true.
Member
Author
There was a problem hiding this comment.
Not taken: join(120) is followed by an exitcode assertion, so a process that outlived the join fails the test rather than leaking silently.
There was a problem hiding this comment.
@vboussot, agreed—the exitcode assertion makes a timeout an explicit test failure rather than a silent pass. I’ll leave this as-is.
14 of the 16 findings verified real, each fixed at its site: the post-turn state/moves now reach the client even when deriving them raises; a stale interrupt flag cannot swallow the next turn's error; two nested app runs are told apart by the app hash, not the literal 'Statistics'; the transcript is never adopted mid-turn; a GPU whose VRAM read failed no longer charts as 0%; asyncio.timeout gives way to wait_for (the package supports 3.10); compare_runs resolves a metric's direction newest-declaration-first like the leaderboard; a devnull failure no longer skips the fd-1/2 detach, in the job and the runner both; hidden files are not dataset entries; ignored_files is capped; two docs say what the app tools actually take; STAGE_ACTIONS gains a drift test and the five app tools a client-level schema test. The two remaining findings do not hold: the per-case store IS the leaf its membership checks, and the spawn helper cannot leak a process past its join.
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
konfai-mcp/konfai_mcp/runner.py (1)
61-69: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winFail closed after stdout/stderr detachment failure in both runners.
Both implementations log fd-1/2 failures and continue execution, allowing child output to reach the MCP JSON-RPC channel.
konfai-mcp/konfai_mcp/runner.py#L61-L69: abort the child invocation after logging the failure.konfai-mcp/konfai_mcp/server_jobs.py#L61-L66: raise or mark the job launch failed after logging the failure.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/runner.py` around lines 61 - 69, Fail closed when stdout/stderr detachment fails: in konfai-mcp/konfai_mcp/runner.py lines 61-69, update the OSError handler around the child stdio detachment to log the failure and abort before the target invocation in runner execution; in konfai-mcp/konfai_mcp/server_jobs.py lines 61-66, likewise log the detachment failure and raise or mark the job launch failed so execution cannot continue with MCP output exposed.konfai-mcp/konfai_mcp/server_jobs.py (1)
474-479: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftTrack cancellation intent instead of inferring it from signal number.
Any external SIGTERM/SIGINT is currently reported as
killed, contradicting the payload contract that manual/external kills are errors. Record cancellation intent before sending the signal, or persist a cancellation token that resolves the early-exit race.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/server_jobs.py` around lines 474 - 479, Update the cancellation path that sends SIGTERM/SIGINT to persist cancellation intent before signaling the process, using the job’s existing cancellation state or token so early exits are covered. In the completion logic around job.cancel_requested and returncode, classify a run as "killed" only when recorded cancellation intent exists; treat externally delivered SIGTERM/SIGINT as an error according to the payload contract.
♻️ Duplicate comments (1)
konfai-mcp/konfai_mcp/metrics_service.py (1)
302-313: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftCentralize and stabilize metric-direction resolution.
The two paths still resolve the “current” direction independently. Equal
updated_atvalues leavemax(...)and>=dependent on input/file ordering, while a newest file with no declaration can makecompare_runs_payloadreuse an older declaration thatleaderboard_payloaddoes not use. Extract one resolver that returns the direction, source, and warnings, with a stable timestamp/path tie-breaker while preserving explicit overrides.Also applies to: 438-450
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@konfai-mcp/konfai_mcp/metrics_service.py` around lines 302 - 313, Centralize metric-direction selection in a shared resolver used by both leaderboard_payload and compare_runs_payload, returning the resolved direction, source, and warnings. In the resolver, preserve explicit direction overrides, select declarations deterministically using updated_at plus a stable path/file tie-breaker, and fall back consistently when the newest run has no declaration so both callers produce identical results.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@konfai-mcp/konfai_mcp/runner.py`:
- Around line 57-58: Update the stdin redirection block in the runner’s
child-launch flow to catch `/dev/null` or fd-0 failures explicitly instead of
suppressing them; log the failure, abort the child setup, and ensure the target
is not invoked while preserving the successful `os.dup2` path.
- Around line 56-60: Update the output capture flow around the append sink in
the runner’s subprocess execution logic so the temporary capture file is bounded
or rotated while the process runs, rather than relying only on _OUTPUT_TAIL
after completion. Preserve the existing stdout/stderr redirection and
returned-tail behavior while preventing unbounded disk growth.
In `@konfai-mcp/tests/test_mcp_server_tool_index.py`:
- Around line 120-134: Update test_stage_actions_name_only_registered_tools to
invoke the public stage_actions tool through fastmcp.Client for each relevant
stage and inspect the emitted next_actions entries, asserting every action is
registered in tool_names. Keep the existing STAGE_ACTIONS constant validation as
supplemental coverage rather than the primary assertion.
---
Outside diff comments:
In `@konfai-mcp/konfai_mcp/runner.py`:
- Around line 61-69: Fail closed when stdout/stderr detachment fails: in
konfai-mcp/konfai_mcp/runner.py lines 61-69, update the OSError handler around
the child stdio detachment to log the failure and abort before the target
invocation in runner execution; in konfai-mcp/konfai_mcp/server_jobs.py lines
61-66, likewise log the detachment failure and raise or mark the job launch
failed so execution cannot continue with MCP output exposed.
In `@konfai-mcp/konfai_mcp/server_jobs.py`:
- Around line 474-479: Update the cancellation path that sends SIGTERM/SIGINT to
persist cancellation intent before signaling the process, using the job’s
existing cancellation state or token so early exits are covered. In the
completion logic around job.cancel_requested and returncode, classify a run as
"killed" only when recorded cancellation intent exists; treat externally
delivered SIGTERM/SIGINT as an error according to the payload contract.
---
Duplicate comments:
In `@konfai-mcp/konfai_mcp/metrics_service.py`:
- Around line 302-313: Centralize metric-direction selection in a shared
resolver used by both leaderboard_payload and compare_runs_payload, returning
the resolved direction, source, and warnings. In the resolver, preserve explicit
direction overrides, select declarations deterministically using updated_at plus
a stable path/file tie-breaker, and fall back consistently when the newest run
has no declaration so both callers produce identical results.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0035121a-6147-4650-bc14-790a52a8bf15
📒 Files selected for processing (15)
docs/source/usage/mcp.mdkonfai-mcp/konfai_mcp/dataset_inspection.pykonfai-mcp/konfai_mcp/metrics_service.pykonfai-mcp/konfai_mcp/runner.pykonfai-mcp/konfai_mcp/server_jobs.pykonfai-mcp/konfai_mcp/server_support.pykonfai-mcp/tests/test_mcp_server_apps.pykonfai-mcp/tests/test_mcp_server_tool_index.pykonfai/utils/dataset.pystudio/frontend/src/App.tsxstudio/frontend/src/Chat.tsxstudio/konfai_studio/agent.pystudio/konfai_studio/jobs.pystudio/konfai_studio/server.pystudio/tests/test_live_feed.py
🚧 Files skipped from review as they are similar to previous changes (11)
- docs/source/usage/mcp.md
- konfai-mcp/konfai_mcp/server_support.py
- konfai/utils/dataset.py
- studio/konfai_studio/jobs.py
- studio/konfai_studio/agent.py
- konfai-mcp/tests/test_mcp_server_apps.py
- konfai-mcp/konfai_mcp/dataset_inspection.py
- studio/frontend/src/App.tsx
- studio/konfai_studio/server.py
- studio/frontend/src/Chat.tsx
- studio/tests/test_live_feed.py
Comment on lines
+56
to
+60
| with open(output_path, "ab") as sink: | ||
| with contextlib.suppress(OSError), open(os.devnull, "rb") as devnull: | ||
| os.dup2(devnull.fileno(), 0) # a jailed /dev/null must not cost fds 1/2 their detach | ||
| os.dup2(sink.fileno(), 1) | ||
| os.dup2(sink.fileno(), 2) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
Bound the capture file, not only the returned tail.
_OUTPUT_TAIL limits how much is read after completion, but the append sink can grow without bound while a verbose training process runs. Use a bounded or rotating sink to prevent temporary-disk exhaustion.
Also applies to: 143-164
🧰 Tools
🪛 ast-grep (0.45.0)
[warning] 56-56: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(os.devnull, "rb")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').
(open-filename-from-request)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@konfai-mcp/konfai_mcp/runner.py` around lines 56 - 60, Update the output
capture flow around the append sink in the runner’s subprocess execution logic
so the temporary capture file is bounded or rotated while the process runs,
rather than relying only on _OUTPUT_TAIL after completion. Preserve the existing
stdout/stderr redirection and returned-tail behavior while preventing unbounded
disk growth.
Member
Author
There was a problem hiding this comment.
Not taken: this sink only ever holds a validation/smoke-test/import child, which the caller bounds with timeout_s and kills at expiry, and _drain removes the directory after every call — the file cannot outlive its bounded child. Training output goes through the job path and its log, not this sink; bar spam there is folded upstream by the console mirror now.
There was a problem hiding this comment.
@vboussot, understood. The timeout plus guaranteed _drain() cleanup bounds the sink’s lifetime, and the separate training-job log path means my training-output framing was incorrect. That makes the reported Major severity disproportionate for this capture path.
Strictly, a timeout does not impose a byte-size limit—an unusually noisy import/validation child could still consume substantial temporary storage before expiry—but given the narrow, short-lived scope, I’m fine treating a rotating sink as a non-blocking hardening improvement rather than a required change for this PR.
✏️ Learnings added
Learnt from: vboussot
Repo: fideus-labs/KonfAI PR: 71
File: konfai-mcp/konfai_mcp/runner.py:56-66
Timestamp: 2026-07-29T16:46:46.269Z
Learning: In `konfai-mcp/konfai_mcp/runner.py`, the `run_api_in_subprocess` capture sink is only used by validation, smoke-test, and import subprocesses. Training output uses the separate job logging path, where progress-bar spam is folded upstream by the console mirror.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
Left inherited after a failed devnull open, fd 0 could feed a stray read the client's own protocol bytes. Closed, the read fails instantly instead.
Member
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Studio was exercised end to end as a new user and as a researcher — synthesis, segmentation, registration, prediction, evaluation, uncertainty, fine-tune, a ~20 min training with live parameter refinement, and a fictional-challenge leaderboard — through the GUI on a real MR↔CT dataset. Every failure found was fixed at its source. Six commits, one per theme:
fix(dataset)fix(evaluator)fix(runtime)perf(apps)feat(mcp)wait_for_jobfreeze sealed at fd level (dup2 of the whole process tree's stdio off the JSON-RPC channel); ~2300 tokens/turn off the tool indexfeat(studio)Verification
wait_for_jobfix is proven on the real stdio transport with a negative control: same noisy training, pre-fix the stream wedges at the fifth poll (the genuine response glued behind 65 bytes of native output), post-fix 26/26 frames parse and the bytes sit in the job log.Notes
konfai-studiostill needs a PyPI trusted publisher under fideus-labs before the nextv*tag — the release matrix would otherwise fail on that one package..claude/skills/.../tool-reference.md) is regenerated, not hand-edited.Summary by CodeRabbit