Plain python export - #672
Conversation
A fourth mode next to FlowFrame/Polars/Project that rewrites a flow with no dataframe library: every table is a list[dict] and every node is an explicit loop. Meant for learning the patterns a dataframe hides - the accumulator dict behind a group by, the hash index behind a join. Structural bits worth knowing: - FlowGraphToPlainPythonConverter subclasses the existing converter, so the Polars path is untouched. PLAIN_PYTHON_NODE_TYPES is derived from the mixin's _handle_* methods, never hand-written, so an inherited Polars emitter can't leak pl. into the output. - Chain fusion is off; you can't pipe two for-loops together. - Nodes with no loop equivalent (formula, polars_code, sql, pivot) become exercise stubs that raise NotImplementedError instead of failing the whole export. Also adds GET /editor/code_to_plain_python and /editor/explain_node, plus the "How would I write this myself?" panel in the settings drawer, which shows the loop for that node's own settings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ports the teaching flavour to the WASM build as an opt-in second mode in
the Code panel (Polars stays the default, choice persists), plus the same
per-node "How would I write this myself?" panel. Embedders can hide both
with teachingMode={false}.
The reason this belongs in Lite more than anywhere else: the browser
already has a Python runtime, so the generated script can run where you
read it. The run button execs it in its own namespace - no execute_*
bridge, no engine state - stages the Read File contents into Pyodide's FS
and prints the rows underneath. An exercise stub becomes something you
fill in and immediately re-run.
Semantics are matched against src/pyodide/engine, not flowfile_core; the
two differ on the aggregate set, join suffixing and unique strategies.
tests/unit/plain-python-parity.test.ts pins that by running 37 fixtures
through the real engine.build_* calls and through the generated script
and diffing the rows. It reports skipped, not passed, when there's no
CPython with Polars around.
Two fixes fell out along the way: toPythonValue emitted JSON null/true/
false into generated Polars code (invalid Python for any manual_input
holding a null or bool), and FlowToPolarsConverter's members are now
protected so the plain flavour can subclass it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
✅ Deploy Preview for flowfile-wasm ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Claude finished @Edwardvaneechoud's task in 1m 22s —— View job Claude is reviewing this PR
|
The inline # comments were passive - something you read past on the way to the code, and they cannot show you anything. This adds the layer that can. Learning mode is a user setting (graduation cap in the icon rail). With it on, the Code panel opens on a third mode that steps through the flow one node at a time and shows three things per step: - the pattern behind it, in prose. Not what the line does, but why the shape is what it is and where else you will write it: the accumulator dict, the hash index, why sort takes a key, why stability matters. - that node's own loop, for your columns and your operators. - the real rows going in and coming out, from one instrumented run. The data comes from buildTraceCode(), the same pipeline with a capture after each block. __steps__ is module-level on purpose: an unfilled exercise raises, and the steps before it still have to be readable. The panel docks beside the canvas rather than covering it, and highlights the node it is describing - that is the canvas-to-code link. Below 1240px it falls back to a modal. The Plain Python editor is now writable too, so an exercise stub is something you fill in and re-run, and edits survive a trip to another tab. Validated by a fleet of inspection agents driving the live app (beginner comprehension, layout across four viewports, light mode, the no-auto-run contract, the edit/run loop, and fact-checking every concept claim against real Python). The explicit-run-only contract came back clean. Fixes that came out of it: the pivot exercise omitted the index column and the aggregation, so it could not be solved; a group by reported that it "dropped" rows when it had folded them; three concept claims were wrong; the docked panel buried the header's Run button; and edits were silently discarded on a tab switch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @Edwardvaneechoud's task in 2m 54s —— View job Claude's reviewTodo list
SummaryThis is a large but well-scoped addition: a fourth code-generation flavour (
The Backend correctness spot-checks against the documented Polars semantics (nulls sort first in both directions, null join keys never match, sum-over-all-null is 0 vs other aggs are null) all check out in New routes ( Frontend/wasm: no Test coverage looks proportionate: backend has dedicated unit + edge-case test files; wasm has unit tests plus a differential parity test ( Minor findings
Not verifiedI wasn't able to execute |
Three things, all pulling the same way: stop showing code in isolation. The walkthrough showed each node as a detached fragment, which told you nothing about where the step sits or what it is handed. It now shows the entire script with the current step's lines lit up, scrolling to them as you step. The converter resolves each step's line range against the finished script, accounting for the helpers above the body. Hovering any name in any of the three editors now explains it - the builtins the generated code leans on, the methods, and the helpers this generator writes for you. A test fails if the glossary drifts from the vocabulary the codegen actually emits. Pivot is no longer an exercise. It generates the two-level accumulator it really is: one pass to collect the column set, one to fill the grid. Matched against execute_pivot by probing the real engine - the value_agg naming rule, nulls-first labels, and the empty-cell-versus-absent-cell distinction that decides sum 0 from None. Seven parity fixtures, mutation tested. Also rewrote the sort background. "Missing values come first because that is simply what the engine does" explained nothing; it now shows that None < 5 raises rather than returning False, which is the actual reason the key carries a boolean flag, and reframes stability as something you use rather than a fact you are told. Fixed along the way: a tooltip anchored to the top visible line rendered and was then clipped invisible, because the panel's backdrop-filter makes it a containing block for the fixed-position tooltip. Bounding tooltipSpace at the editor makes CodeMirror flip it below instead. Known gap: the parity harness sorts object keys before comparing, so pivot's column ORDER is not differentially proven - only the column set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @Edwardvaneechoud's task in 5m 42s —— View job Claude's reviewTodo list
SummaryThree new commits (
I traced the line-range/offset arithmetic in Tests are substantive, not superficial: the glossary-drift test ( Minor findings
No Alembic migration needed (no |
Two tabs showed the same script, and the walkthrough lived on a screen of its own - so reading the code and reading about it were different places. Now there are two tabs, Polars and Python walkthrough, and the walkthrough is the plain-Python view rather than a parallel one. The editor is mounted once and never remounted. It sits in the middle of the panel, keeps ~478px at 1600x1000, and stepping never resizes it: the chips move a highlight through the script in place. Below it a margin carries Data and Output, split by a draggable divider that remembers where you put it. At modal widths the two flip to side-by-side columns. The background prose is now OPT-IN and collapsed by default. An embedded editor is a plain code view unless someone presses "Why does it look like this?", which adds a Why tab that stays open as you step and is remembered next session. That was the maintainer's call, and it also fixes the thing that made the old layout wrong: four paragraphs and an ASCII sketch filled the viewport and pushed the code below the fold. The highlight now rides the text. Ranges are kept as document offsets that map through edits (start assoc -1, end +1, so typing inside a block grows it) instead of being dropped on any change, and the header label re-derives its line numbers from the painted decoration - so label and paint cannot disagree, which is how the old "lines 75-78" pointed at a viewport showing line 48. scrollIntoView is verified against real geometry and retried once, because line heights are estimated until measured and lineWrapping makes the estimate worse. The layout came out of a design workflow: four proposals from different starting biases, adversarial judging that caught real arithmetic errors in them, then a synthesis resolving the fatal flaws. Known: uiStore.codePanelMode is now written but unread (Canvas moved to codePanelDocked). Re-seeding after a tab round-trip drifts against your own edits, and trace staleness is disclosed rather than fixed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @Edwardvaneechoud's task in 2m 56s —— View job Claude is reviewing this PR
|
It was pinned at 620px docked right, with a modal fallback below 1240px and no way to change either. Now it is a DraggableItem, the same overlay the node-settings panel already uses: drag it, resize it from any edge, take it fullscreen, minimise it, and it remembers where you left it. Reusing that system rather than hand-rolling a second one means the bespoke geometry goes away entirely - the .docked CSS, the 1240px media query, the click-through backdrop, Z_INDEX.DOCKED_PANEL, and the codePanelDocked/codePanelMode store fields (the latter was write-only). Persistence rides the existing intent record; no new localStorage key and no derived geometry stored. Two opt-in additions to DraggableItem, following the precedent of the existing wasm-only extras: an `actions` header slot (run/reset/refresh/ export live there now, and mousedown.stop keeps a button press from starting a drag) and a `flushContent` prop for a full-bleed body. Plus a `topOffset` prop so embed mode clears the in-canvas toolbar, replacing a hardcoded 100px of padding. The panel joins the normal panel stack rather than keeping its own 30000, so last-clicked wins between Code and Settings instead of Code permanently burying it, and the layout-controls widget stays reachable - which now matters more, since Reset Panel Layout is the way back from a panel dragged somewhere awkward. Because the panel is freely resizable, the inner two-column flip is now reachable at real widths; CodeMirror re-measures on container change so it does not render stale geometry. The outer edge drag and the inner bench/margin divider do not fight. Also fixed a console error this uncovered in showStep: the retry dispatch ran inside CodeMirror's measure cycle, which refuses updates. It hops out via rAF now, guarded against a detached view. Verified gone. Known: below ~1240px the panel no longer becomes a modal, so it docks at 620 over a narrow canvas - draggable and minimisable, which the modal was not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @Edwardvaneechoud's task in 2m 14s —— View job Claude's reviewTodo list
SummaryCommit
Two correctness details I traced by hand:
Status of prior findings
Not verifiedSame limitation as previous rounds: this sandbox's approval gate blocks |
|
Claude finished @Edwardvaneechoud's task in 2m 3s —— View job Claude's reviewTodo list
SummaryTwo commits landed since the last review round:
I traced the new tracing/comparison logic by hand:
No Status of prior findings
Not verifiedSame limitation as previous rounds: this sandbox's approval gate blocks |
|
Claude finished @Edwardvaneechoud's task in 2m 22s —— View job Claude's reviewTodo list
SummaryCommit
No behavioral/logic changes, no new security surface, nothing here touches Status of prior findings
Not verifiedSame limitation as previous rounds: this sandbox's approval gate blocks · Branch: |
|
Claude finished @Edwardvaneechoud's task in 1m 15s —— View job Claude's reviewTodo list
SummaryCommit Specifically,
Between
Also finishes the helper-extraction started in The Status of prior open findings
Not verifiedSame sandbox limitation as previous rounds: |
1 participant
Adding an old idea. Making flowfile a tool for teaching basic python and how you can work with plain python and data