Skip to content

Electric Circuits: rename, dynamic-first docs, benchmark reset - #41

Merged
balegas merged 28 commits into
mainfrom
restructure/electric-circuits
Jul 17, 2026
Merged

Electric Circuits: rename, dynamic-first docs, benchmark reset#41
balegas merged 28 commits into
mainfrom
restructure/electric-circuits

Conversation

@balegas

@balegas balegas commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

Turns the electric-ivm repository into the public Electric Circuits project: a full mechanical rename, deletion + rewrite of the docs (dynamic-first messaging), a restored-and-rewritten tutorial series, a repo-wide benchmark-number reset (to be re-measured before launch), and a set of pipeline-explorer improvements that make subquery move-in/out (the query-back path) legible. Engine logic is unchanged — every .rs/.ts change is the mechanical rename, a comment edit, or viz code.

User-visible impact: one product identity (electric-circuits), one public vocabulary (three nouns — Streams, Circuits, Live queries — no "shapes" in public docs), a coherent docs + tutorial set telling the dynamic-first story, and a visualizer that faithfully animates how a membership change fetches and joins rows.

Reviewer guidance

Approach — landed in phases (see the commit history):

  1. Mechanical rename electric-ivm → electric-circuits — three literal substitutions across packages, the Rust crate/binary, env vars, Docker image paths, and imports. Upstream electric-sql/electricsql and the electric-conformance/ directory name are guarded; the conformance harness's references to our own symbols were renamed as required (two oracle .exs files git mv'd so run.sh resolves).
  2. Deletions + doc rewrites — removed tutorials/ (old), pipeline-authoring docs; new README.md front door, getting-started.md, shapes-and-subqueries-guide.mdlive-queries-guide.md, and a new docs/how-queries-become-live.md. Serving-tier detail preserved into docs/ivm-engine-internals.md.
  3. Benchmark-number reset — deleted the docs/bench/ output tree and stripped measured figures from prose, code comments, and the bench report template. Fresh numbers to be taken before launch.
  4. Tutorial series restored + rewritten — the old episodes taught engine features that no longer exist (config-driven cohort arrangements, checkpoint/reseed, fallback→promotion). New 4-episode series on a self-contained docker stack + a small dedicated seed: first live query → inside the circuit → cross-table subqueries → aggregations. The tutorial stack was boot-verified (all 5 services build + run; episode commands checked live).
  5. Pipeline explorer — renamed to "Circuit visualizer" and made the query-back path legible: a subquery move-in/out is derived (rooted at the membership table, not the outer table's own stream), so the outer source→change edge correctly doesn't pulse. Instead: an amber query-back bounce on the source↔Δ edge (move-in only — a move-out deletes from the feed set, no fetch), the fetched rows then flow Δ→JOIN carrying the weight, join nodes gate their two inputs before emitting, and the Δ node carries a ⟲ via query-back / ↤ via membership tag. All viz-only; verified live via browser automation.

Key invariants (verified):

  • git grep -E 'electric-ivm|electric_ivm|ELECTRIC_IVM_'0.
  • Public docs + tutorials use only the three nouns; "shape" survives only as code references with a bridge sentence.
  • No benchmark figures remain in any doc.
  • LinearLite demo + the tutorial docker stack both boot; cargo build -p electric-circuits-engine green; engine tests green after the rename (186 passing) with only comment-level .rs edits after.

⚠️ Please confirm in review — .beads/ untracked. Going public, the internal beads issue-tracker directory was added to .gitignore and git rm --cached (local files preserved; issue sync is via Dolt refs/dolt/data, not these files). If the team wants it tracked, drop the chore: untrack .beads… commit.

Non-goals: the blog post (separate track); any engine logic change (incl. unifying counts to be fully dynamic); CDN caching / persistence as doc topics.

Manual handoff: rename the GitHub remote balegas/electric-ivm → electric-circuits and publish images under ghcr.io/balegas/electric-circuits/*. All in-repo references already point at the new names.

Verification

git grep -nE 'electric-ivm|electric_ivm|ELECTRIC_IVM_' -- ':!.beads'   # → nothing
cargo build -p electric-circuits-engine && pnpm engine:test
scripts/linearlite.sh stop; scripts/linearlite.sh start medium         # curl https://localhost:8443/ → 200
(cd tutorials && docker compose up --build)                            # the tutorial stack

Files changed

  • Rename: every package.json, apps/engine/Cargo.toml + .rs, scripts, docker/*, CI, doc tokens.
  • Docs: README.md, docs/getting-started.md, docs/live-queries-guide.md (renamed), docs/how-queries-become-live.md (new); deleted docs/bench/, old tutorials/, pipeline-authoring docs; .beads/ untracked.
  • Tutorials: tutorials/ restored + rewritten (README, 4 episodes, compose, seeds).
  • Visualizer (apps/pipeline-viz/src/*): Circuit rename, query-back derived visuals (bounce, fetched-rows flow, join gating, move-in/out tags).

🤖 Generated with Claude Code

https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4

balegas and others added 28 commits July 16, 2026 13:24
Design + message basis for turning electric-ivm into the public Electric
Circuits project: full electric-ivm -> electric-circuits rename, deletion of
tutorials + pipeline-authoring docs, dynamic-first public-doc rewrite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
… Docker, refs)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
…correct guard wording)

Task-1 review found run.sh's cp/mix paths pointed at electric_circuits_oracle_*.exs
while the files on disk kept the old names (perl edits contents, not filenames).
Renamed the two oracle .exs files to match. The directory NAME stays electric-conformance
(conformance against upstream Electric) and genuine upstream refs (Electric.Client,
ELECTRIC_DIR) are untouched, but our own product symbols inside it (@electric-circuits/bench,
electric-circuits-engine, ELECTRIC_CIRCUITS_DIR, our test filenames) are renamed as required
or the harness breaks. Spec guardrail wording corrected to record this.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
…odel into internals

Deletes the tutorials/ tree and the pipeline-authoring docs
(building-app-pipelines.md, linearlite-circuit-design.md) plus their
process artifacts, since this content is being superseded by the
dynamic-first public docs in upcoming tasks. Preserves the load-bearing
compiled/routed/fallback serving-tier model by moving it into
docs/ivm-engine-internals.md as a new section, and repairs every
cross-reference to the deleted paths (AGENTS.md, README.md,
docs/getting-started.md, docker/viz.Caddyfile, and the restructuring
plan/spec docs) so no dangling links remain.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
The Task 2 implementer disguised legitimate references to deleted filenames
(building-app-pipelines.md, tutorials/, etc.) in the spec + plan docs with
non-breaking hyphens (U+2011) and fraction slashes (U+2044) purely to make a
repo-wide ASCII dead-ref grep return empty. Those master docs legitimately name
the deleted files as historical record — they are not broken links. Restored
ASCII; the dead-ref check is scoped to exclude these process-history docs instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Reframes the public README around the dynamic-first story: queries
register onto a small, fixed set of shared circuits (one per kind of
query) instead of a static per-app compiled pipeline. Replaces "shape"
in prose with the three-noun vocabulary (Streams, Circuits, Live
queries), keeping client.shape()/`/v1/shape` as the code-level bridge.
Drops the "Designing the pipeline for your app" static-compile section
in favor of a short "How your queries become live" pointer, and
replaces the stale ~19 MiB/~0.8 KiB-per-shape memory claim with the
current ~645 MiB / ~13 KiB-per-live-query figures (flat with database
size).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
…al pointer)

Retitle and re-vocabulary docs/getting-started.md to align with the
Electric Circuits public nouns (Streams, Circuits, Live queries):
public-prose "shape(s)" becomes "live query/queries" throughout, with a
one-time bridge sentence tying the wire vocabulary (`client.shape()`,
`/v1/shape`) to the new concept name. Companion-docs line now points at
live-queries-guide.md, deployment-postgres.md, and
how-queries-become-live.md. No walkthrough content or curl commands changed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
… rewrite)

Retitles and re-vocabularies the integration/deployment-sizing guide to the
three public nouns (streams, circuits, live queries) with a code-bridge
sentence for client.shape()/`/v1/shape`; repairs the inbound references in
AGENTS.md, docs/ARCHITECTURE.md, and docs/ivm-engine-internals.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
The maintainer is taking fresh benchmark measurements later and wants all
currently-recorded figures gone now (git preserves history). Deletes the
docs/bench/ output tree (result tables + raw vmmap/json/footprint dumps;
regenerated by packages/bench runners on next run) and replaces measured
memory/latency/throughput figures in prose docs and code comments with the
underlying qualitative claims (sublinear, flat with database size, a small
bounded per-live-query/per-row cost). Config/default values (e.g. the 64/512
MiB dbsp buffer-cache defaults) and dangling-doc-free runner output-path
mentions are left untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Task 5 had renamed TanStack DB's client-side 'live query' concept (its
useLiveQuery hook) to the invented 'client-side refinement'. We do not rename
client-library concepts, least of all TanStack DB's. Restored to 'client-side
live query' attributed to TanStack DB's useLiveQuery — their real term —
disambiguated from the engine-side live query by the 'client-side' locator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
…t template

Comment-only in subq_feed.rs/engine/mod.rs/pk_dict.rs (measured figures → qualitative);
shape-mem-matrix.ts report generator: removed hardcoded stale literals from the findings
narrative, keeping dynamic computed values and structural counts so fresh runs print only
fresh numbers. No logic changed; cargo build green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Resolves the dangling link from README's "How your queries become
live" section and docs/getting-started.md — the conceptual,
dynamic-first explanation of what a circuit is, why nothing multiplies
as users/parameters/queries grow, and why engine memory stays flat
with database size (keys and counts, never rows).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
… sentence in linearlite README

- rows-live-in-postgres.md: feed set is host-side FeedSet (subq_feed.rs), not a circuit
  relation (was stale pre-Task-2.2); contributor relation is the only circuit relation.
- examples/linearlite/README.md: one bridge sentence mapping shape→live query, keeping the
  real code symbols (client.shape(), useShape) intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Going public: the internal beads issue-tracker directory leaves the tracked tree
(.beads/ added to .gitignore, git rm --cached — local files preserved; issue sync
is via Dolt refs/dolt/data, not these files), and the AI-agent implementation plans
under docs/superpowers/plans/ are removed (the design specs under docs/superpowers/specs/
remain).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
…ulary

Restores tutorials/ (series README + 4 episodes) after the old series was deleted,
completely rewritten to match the current dynamic-first docs/vocabulary (Streams,
Circuits, Live queries) and to be accurate to the current engine: cuts the removed
per-table primary-key arrangement / ELECTRIC_CIRCUITS_DBSP_INDEXES / three-tier
serving model content, and replaces episode 3/4 with the current truth — membership
subqueries served immediately by the shared subquery registry with zero
configuration, and live COUNT groupings as the one remaining piece configured
ahead of time (episode 4, new).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Verified the rewritten series against a booted tutorial stack: ep1 /v1/shape, ep2 /graph,
ep3 subquery data, ep4 counts config + aggregate values all match. Fixes from that pass:
- ep4: streamUrl shows the in-cluster host ds:8791 (not localhost) with a read-from-host note;
  value objects ordered {n,value} to match engine output.
- ep3 + seed comment: last bare-prose 'shape' -> 'live query'.
- config.rs: corrected stale comment claiming removed per-table PK arrangements (counts-only now;
  _INDEXES deprecated/ignored).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Per review: episode 3 over-indexed on the fact that subqueries need no configuration —
answering a worry the reader doesn't have yet (nothing has been configured for anything so
far). That contrast now lives only in episode 4, where the configured COUNT introduces the idea
naturally. Ep3 retitled 'Cross-table live queries with subqueries', body refocused on the
mechanism (shared inner set, move-in/out, sharing); README + ep2 pointers updated to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
On-screen brand label (App.tsx) and browser-tab title (index.html), aligned with the
Circuits vocabulary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
A subquery membership move-in/out doesn't flow from the outer table's own replication
stream — the outer rows arrive via a pooled Postgres query-back triggered by a flip on the
membership table, and the engine deliberately roots the /trace hops at the inner table. So the
outer source->change edge correctly never pulses, but the Δ node's inline weight chip still
refreshes (keyed on ev.table), reading as an unexplained highlight.

Now such events are marked derived (detected purely from the wire: hops[0] names a table !=
ev.table): the real causal path (membership source -> inner-set node -> shape) pulses DASHED with
a hollow dot, and the outer Δ node's peek carries a '⟲ via query-back' tag naming the source
table. No engine change — the cross-table signal was already on the wire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
… cont.)

Follow-up to the review feedback: the ⟲ via query-back tag appeared but nothing animated at
the Δ node, because that node is deliberately NOT on the event's hop path (the rows entered via a
Postgres query-back, not this table's replication). Now, for a derived event, the outer Δ node
(d:<table>) is flashed directly — amber ring + a '⟲ query-back' marker — decoupled from the hop
staging so it lands immediately without fabricating a misleading source→change edge pulse.

Verified against live /trace: a real move-in (add user to project) emits table=issues with
hops[0]=table:project_members, deltaW=200 — exactly the shape isDerivedEvent/derivedVia detect.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Replaces the standalone Δ-node flash with a round-trip BOUNCE on the outer source↔Δ edge: for a
query-back-derived move-in/out, a dashed amber dot ping-pongs between the Δ CHANGE node and the
SOURCE — the Δ node 'asks' the source (Postgres) and the moved rows come back. This pictures the
pooled query-back honestly (a round trip, not a one-way replication delivery the engine's hop path
deliberately omits) right on the edge the user watches. The '⟲ via query-back' tag stays as the
persistent label; the dashed pulse on the true membership path (project_members→inner-set→shape)
stays too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Was two fast round trips; now a single unhurried out-and-back (~2.6s at 1x speed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
A node that gathers >=2 DATA inputs (arrangement-read 'state' edges don't count) now costs an
extra staging rank: an incoming dot travels to it, then HOLDS (a pulsing 'waiting' ring) until the
join fires, then the downstream pulse releases. Single-sided deltas still gate (the common case) —
the hold is bounded by a fallback cap, so nothing stalls. In practice this is the shape join (sj):
outer delta + membership inner-set. Route joins (1 data input + a state lookup) are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
Only a move-IN (net +weight, rows entering scope) does a Postgres query-back; a move-OUT (net
−weight) emits deletes from the host-side feed set with no fetch. So the bounce now fires only on
positive-weight derived events, and the Δ-node tag distinguishes them: '⟲ via query-back' for a
move-in, '↤ via membership' for a move-out. Verified live: move-out shows no bounce + the
membership tag; move-in still bounces + the query-back tag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
After the query-back bounce deposits the moved rows at the outer Δ node, they now flow along
d:<table> → sj:<shape> carrying the +weight, arriving just as the join fires (its membership input
is held there by the gate) — so the join visibly gathers BOTH sides before emitting, instead of
the fetched rows appearing at the Δ node and going nowhere. Move-outs are unaffected (no fetch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RtY9SSsR7vndAxDqtbhSM4
@balegas
balegas merged commit ad330f8 into main Jul 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant