Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,14 @@ The [fractal-studio-example](examples/fractal-studio/README.md) is an interactiv
Type a prompt — *"an animated Julia set, c orbiting the main cardioid, with a glowing sunset palette"* — and the agent implements `fn shade(x: f64, y: f64, t: f64) -> u32`; the live animation morphs in place, no restart:

<p align="center" width="100%">
<video src="https://github.com/user-attachments/assets/1527ea1d-decd-46a4-9687-5189cac16bd9" width="80%" controls></video>
<video src="https://github.com/user-attachments/assets/31363d70-1fed-4811-b59b-160d769a928d" width="80%" controls></video>
</p>

- `shade` is called once per pixel (~0.5M calls/frame at 960×540), parallelized over all cores with rayon — an interpreted agent-code loop would be orders of magnitude too slow to animate.
- `shade` is called once per pixel (~0.5M calls/frame at 960×540, and the canvas re-renders at the window's size and aspect ratio), parallelized over all cores with rayon — an interpreted agent-code loop would be orders of magnitude too slow to animate.
- The user is the evaluator: the runtime keeps the chat history, so follow-up prompts refine the current shader.
- The canvas never freezes while the agent works: a single-lane `evolve_batch` compiles and registers the
candidate without touching the dispatch pointers, so rendering continues on the current revision until
`activate_revision` commits the new one in a few atomic stores at a frame boundary.
- Agent code panics are caught inside the dylib (rendered as black pixels) and fed back into the next evolution prompt.

```bash
Expand Down
11 changes: 4 additions & 7 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,7 @@
- Show multi-threading support with example.
- Show multi function evolution with example.
- Show example of using external dependency in generated dylibs, if configured.
- Support disallowing methods, crates or unsafe, enforced by the harness.
- Proper eval pipeline to compare model performance across tasks. My own benchmark suite so to say, aka `symbiont-eval`
- Compare examples with SOTA equivalent search strategies, see if it beats any already.
- Run Harness for my symbolic regression evaluation comparison, to see if it beats SOTA for ~150 optimization targets.
- Bidirectionality, like evolving a fractal rendering function using `evolvable` and a UI in the main harness binary shows the results.
- Capture the number of evolution failures by category, e.g how many compile errors, how many parse errors, how many HTTP errors, etc.
- Capture the inference cost in the responses, if available.
- Prefix-cache visibility is provider-dependent: `LLM_TOKENS{kind="cached_input"}` comes from
`usage.prompt_tokens_details.cached_tokens`, which vLLM never populates (verified: identical
Expand All @@ -18,12 +13,14 @@
means paying dependency compilation per lane. Only worth it once
`symbiont_build_slot_wait_seconds` says so — see [CAVEATS.md](CAVEATS.md).
- Track the context length of the prompt (system + user) and make it available to query.
- Cap the runtime of agent code to a user-specified maximum to prevent infine loops in agent code.
This is not really possible, unless the function signature has cooperative cancellation code passed in like an `stop: AtomicBool` and the agent must ensure to check it in each loop round.
- Provide a way to call `info`, `debug` and `trace` like logging functions in the code and have them feed into the context in a smart way.
Maybe its possible to re-use `tracing` here, depending on if its safe to do across dylib boundaries.
It would need to be its own buffer though.
- Natively support storing the correctly generated rust code in a DB.
Maybe rig has some native DB support?
- Support passing in images if the LLM supports multi-modality.
Giving Agents image context might help improve the reasoning in certain problem cases.
- example of evolving a CUDA kernel.
- example for an interactive background daemon that creates dynamic wallpapers based on user prompts / revision.
* Similar to fractal studio
* Could be CPU native or CUDA accelerated
36 changes: 29 additions & 7 deletions examples/fractal-studio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ hot-swaps the dylib. The live animation morphs in place, no restart.
## Why this showcases symbiont

- **Bare-metal performance where it matters**: `shade` is called once per pixel
(~0.5M calls/frame at 960×540), parallelized over all cores with rayon, with
(~0.5M calls/frame at 960×540, more on a larger window), parallelized over all cores with rayon, with
fractal workloads running hundreds of iterations per pixel. The ~1.6 ns
dispatch overhead makes the hot-swap abstraction effectively free — an
interpreted agent-code loop would be orders of magnitude too slow to animate.
Expand All @@ -32,17 +32,39 @@ hot-swaps the dylib. The live animation morphs in place, no restart.
## Architecture

Three threads, coordinated around the feedback-loop contract
(*no evolvable call may be in flight while the dylib is swapped*):
(*no evolvable call may be in flight while the dispatch pointers are swapped*):

- **egui UI** (main thread): canvas, prompt box, telemetry (ms/frame, Mpix/s),
and a syntax-highlighted view of the live agent code.
- **render thread**: tight frame loop calling `shade` for every pixel via
rayon. Parks at a frame boundary when an evolution is requested.
- **evolution worker**: drains the render gate, runs `Runtime::evolve` on a
tokio runtime, publishes the new code, resumes rendering.
rayon, at whatever size the UI last reported (capped at ~1080p worth of
pixels, then upscaled). Parks at a frame boundary only for the revision
swap.

The animation freezes (showing the last frame) while the agent generates and
compiles, then resumes with the new shader — that pause *is* the contract.
The canvas is re-rendered at the window's aspect ratio instead of being fitted
into it, so resizing never letterboxes — which means `aspect` is not a
constant, and the evolution prompt tells the agent as much.
- **evolution worker**: runs a single-lane `Runtime::evolve_batch` on a tokio
runtime, then drains the render gate and calls
`Runtime::activate_revision`.

### The animation keeps running while the agent works

`evolve_batch` compiles and *registers* the candidate without touching the
dispatch pointers, which is why it is exempt from the feedback-loop contract:
a retained-but-inactive revision is invisible to running calls. The render
thread therefore keeps calling the **current** revision's function pointer for
the whole generate → validate → compile round — seconds of LLM inference plus a
`cargo build --release` — and the canvas never freezes.

Only the commit is gated. `activate_revision` republishes function pointers
that were resolved when the dylib was loaded, so it is a handful of atomic
stores: the render thread parks at a frame boundary, the swap happens, and
rendering resumes. The side panel reports that park time in microseconds next
to the multi-second evolution time.

The one visible effect during an evolution is a frame-rate dip: the nested
`cargo build` competes for the same cores rayon renders on.

## Running

Expand Down
Loading
Loading