Skip to content

Commit 767c840

Browse files
rsasaki0109claude
rsasaki0109
and
claude
authored
Reframe README as "tiny robot failure lab" + add 10-lesson tour (#6)
* Reframe README around "tiny robot failure lab" and add a 10-lesson tour Sharpen the project's entry point so the failure-aware closed-loop idea lands in the first screen instead of being buried under 39 examples. - README: lead with "tutorials assume actions succeed; this teaches what happens when they don't", a single hero GIF (pick_and_retry), an 8-line annotated core loop, a browser-run link, and three starter loops. Move the 39-example status, Colab list, and star CTA below the fold. Drop the duplicate core-loop snippet further down. - lessons/README.md: a failure-first, 10-lesson guided tour that curates existing tested examples (bare loop -> fail/retry -> replan -> recovery -> belief -> active perception -> safety -> uncertainty -> clarify -> capstone), each with what-fails / what-it-observes / how-belief-changes notes. Linked from the README. Curates rather than copies so the lessons stay in sync with the tested examples. - docs/release_notes_v0.1.0.md: draft body + pre-publish checklist for the first public "Tiny Robot Failure Lab" release. All ten lesson commands verified headless; markdown/playground asset tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add naive-vs-failure-aware hero GIF and a Pyodide playground design memo - scripts/make_hero_compare_gif.py: generate a side-by-side hero GIF on the same Tabletop2D task with the same seed. Left runs a "no belief update" baseline that locks onto its first guess and keeps grabbing the same spot; right runs the repo's PickAndRetryAgent, which looks, updates its belief, and recovers. The baseline fails on ~63% of seeds (not a cherry-picked fluke); seed 3 shows the common case: naive gives up after 8 misses, failure-aware recovers in 3. A --search mode scans seeds. - docs/assets/gifs/naive_vs_failure_aware.gif: the rendered hero (passes the existing --check-gifs frame/nonblank checks). - README: lead with the comparison hero instead of the single pick GIF, with a caption framing the gap as the point of the repo. - docs/pyodide_playground_strategy.md: design memo for running the real Python loops in the browser. Notes that today's playground.js is a JS reimplementation (drift risk) and proposes a "Python computes, JS draws" split so Pyodide reduces drift instead of adding a surface; phased plan + risks. asset tests and --check-gifs green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 97e08a3 commit 767c840

6 files changed

Lines changed: 762 additions & 50 deletions

File tree

‎README.md‎

Lines changed: 54 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -5,27 +5,42 @@
55
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
66
![Core dependencies](https://img.shields.io/badge/core-numpy%20%2B%20matplotlib-orange)
77

8-
**Robots observe, act, fail, retry, update beliefs, and replan.**
9-
This repo shows that loop in small, readable Python — no ROS, no GPU, no
10-
simulator. Just `numpy + matplotlib`.
11-
12-
[Open the example gallery](https://rsasaki0109.github.io/PythonInteractiveRobotics/),
13-
[try the live playground](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html), or open a
14-
[shareable live trace](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html?scenario=household&answer=red&compare=1&autoplay=1),
15-
or jump straight into the first runnable loop below. You can also run the
16-
flagship loops directly in Colab:
17-
[pick and retry](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/pick_and_retry.ipynb),
18-
[safety filter](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/safety_filter_cbf.ipynb), and
19-
[human correction replanning](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/human_correction_replanning.ipynb).
20-
For language ambiguity, try
21-
[clarifying question](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/clarifying_question.ipynb), or run the integrated
22-
[household task agent](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/household_task_agent.ipynb).
23-
If the project helps you teach, prototype, or explain robotics loops, a GitHub
24-
star helps others find it.
8+
### A tiny robot failure lab
259

26-
| Avoiding | Reaching under occlusion | Mapping while uncertain |
27-
| --- | --- | --- |
28-
| ![A point robot's naive go-to-goal velocity is projected onto a CBF safe set at every step. The policy itself never knows the obstacles exist - a separate runtime safety filter slides it around them.](docs/assets/gifs/safety_filter_cbf.gif) | ![A 2-link arm predicts a briefly occluded moving target, keeps servoing through the occlusion, and reaches the intercept point when the target reappears.](docs/assets/gifs/moving_target_reaching.gif) | ![A toy active-SLAM agent shrinks pose belief and occupancy belief at the same time, by picking moves that maximize expected entropy drop.](docs/assets/gifs/active_slam_toy.gif) |
10+
**Robotics tutorials often assume actions succeed. This repo teaches what
11+
happens when they do not.**
12+
13+
Watch a robot miss a grasp, update its belief, and recover — in pure Python.
14+
No ROS. No GPU. No simulator. Just `numpy + matplotlib`.
15+
16+
![Side by side on the same tabletop task. Left, a naive picker locks onto its first guess and keeps grabbing the same empty spot until it gives up after eight misses. Right, a failure-aware agent looks from a better viewpoint, updates its belief about where the object is, and recovers the grasp in three tries.](docs/assets/gifs/naive_vs_failure_aware.gif)
17+
18+
*Same task, same seed. Left: a naive picker that never updates its guess keeps
19+
missing and gives up. Right: the failure-aware agent looks, updates its belief,
20+
and recovers. That gap is the whole repo.*
21+
22+
[▶ Run in your browser](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html?scenario=household&answer=red&compare=1&autoplay=1)
23+
· [Start with `01_pick_and_retry.py`](#try-it)
24+
· [Take the 10-lesson tour](lessons/README.md)
25+
· [Browse all loops](https://rsasaki0109.github.io/PythonInteractiveRobotics/)
26+
27+
The whole repo is one loop, written out small enough to read:
28+
29+
```python
30+
obs = env.reset(seed=0)
31+
agent.reset()
32+
33+
for t in range(max_steps):
34+
action = agent.act(obs) # think
35+
obs, reward, done, info = env.step(action) # act
36+
agent.update(obs, reward, info) # observe failure, update belief
37+
if done:
38+
break # ...else replan and retry
39+
```
40+
41+
`info["failure"]` is a first-class part of that loop — grasp misses, occlusion,
42+
localization drift, blocked paths. The interesting behaviour is what the robot
43+
does *after* it fails.
2944

3045
## Try it
3146

@@ -39,23 +54,26 @@ python3 examples/manipulation/01_pick_and_retry.py
3954
A tiny tabletop robot misses a grasp, updates its belief, and retries — in
4055
under 5 seconds. Core dependencies are `numpy` and `matplotlib` only.
4156

42-
For an even smaller first loop:
43-
44-
```bash
45-
python3 examples/runtime/01_sense_act_loop.py
46-
```
47-
48-
## Start Here
57+
## Three loops to start with
4958

5059
| If you want to see | Run | What it teaches |
5160
| --- | --- | --- |
5261
| Failure recovery | `python3 examples/manipulation/01_pick_and_retry.py` | grasp miss -> belief update -> retry |
53-
| Runtime safety | `python3 examples/navigation/29_safety_filter_cbf.py` | nominal controller -> CBF projection -> safe motion |
54-
| Active perception | `python3 examples/navigation/07_active_slam_toy.py` | map and pose uncertainty -> information-seeking action |
55-
| Shareable live trace | [Try live](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html?scenario=household&answer=red&compare=1&autoplay=1) | belief entropy, compare mode, and failure timeline |
56-
| Human correction | [Open in Colab](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/human_correction_replanning.ipynb) | shortcut -> human correction -> cost update -> replan |
57-
| Language ambiguity | [Open in Colab](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/clarifying_question.ipynb) | ambiguous command -> ask question -> answer -> act |
58-
| Integrated household task | [Open in Colab](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/household_task_agent.ipynb) | clarify -> plan -> safety check -> retry -> human replan |
62+
| Online replanning | `python3 examples/navigation/04_online_replanning_astar.py` | plan -> hit a hidden wall -> replan |
63+
| Asking for help | `python3 examples/embodied_ai/35_clarifying_question.py "pick the block" --answer red` | ambiguous command -> ask -> act |
64+
65+
Prefer the browser? Try the [live playground](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html)
66+
(belief entropy, compare mode, failure timeline) — every run is a
67+
[Shareable live trace](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html?scenario=household&answer=red&compare=1&autoplay=1)
68+
you can link to — or open the flagship loops in Colab:
69+
[pick and retry](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/pick_and_retry.ipynb),
70+
[safety filter](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/safety_filter_cbf.ipynb),
71+
[human correction replanning](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/human_correction_replanning.ipynb),
72+
[clarifying question](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/clarifying_question.ipynb), or the integrated
73+
[household task agent](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/household_task_agent.ipynb).
74+
75+
If the project helps you teach, prototype, or explain robotics loops, a GitHub
76+
star helps others find it.
5977

6078
## Status
6179

@@ -202,24 +220,10 @@ python scripts/run_all_smoke_tests.py --gifs --check-gifs
202220

203221
CI runs the same smoke suite and GIF checks on Python 3.10, 3.11, and 3.12.
204222

205-
## Core idea
206-
207-
```python
208-
obs = env.reset(seed=0)
209-
agent.reset()
210-
211-
for t in range(max_steps):
212-
action = agent.act(obs)
213-
obs, reward, done, info = env.step(action)
214-
agent.update(obs, reward, info)
215-
env.render()
216-
217-
if done:
218-
break
219-
```
223+
## Inspecting a run
220224

221-
The goal is not photorealism.
222-
The goal is to understand the perception-action loop.
225+
The goal is not photorealism. It is to understand the perception-action loop
226+
shown at the top of this README — and to see the internal state that drives it.
223227

224228
Every example returns a `Trace`, so headless runs can be inspected without
225229
rendering. See `docs/trace.md` for the full trace contract.

‎docs/assets/gifs/naive_vs_failure_aware.gif‎

64.6 KB
Loading

‎docs/pyodide_playground_strategy.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Design memo: running the real Python loops in the browser (Pyodide)
2+
3+
Status: **proposal / not yet implemented.** This memo scopes follow-up item ②
4+
from the launch plan — the highest-leverage growth hook GPT Pro flagged
5+
("install-free / readable / no heavy stack" is what lands on Hacker News).
6+
7+
## Why now
8+
9+
The current [`docs/playground.js`](playground.js) is a ~1200-line **JavaScript
10+
reimplementation** of just two scenarios (`clarifying`, `household`). It does not
11+
run any of the repo's Python. Two consequences:
12+
13+
1. **Drift risk.** The JS dynamics can silently disagree with the tested Python
14+
examples; nothing keeps them in sync.
15+
2. **Weak headline.** "An interactive web demo" is ordinary. "**The actual
16+
`numpy` example you'd run locally, running in your browser, no install**" is
17+
the share-worthy version — and it's true to the repo's whole premise.
18+
19+
Pyodide ([pyodide.org](https://pyodide.org)) runs CPython + NumPy in WebAssembly
20+
in the browser, so we can execute the real example code with zero install.
21+
22+
## The key architecture decision: who renders?
23+
24+
The examples use `numpy` for the loop and `matplotlib` for rendering. Matplotlib
25+
*is* available in Pyodide, but it is a large download and slow to first paint.
26+
We do not need it in the browser, because **we already have a renderer**: the
27+
existing playground draws scenes and belief panels from plain data.
28+
29+
**Recommended split — Python computes, JS draws:**
30+
31+
```
32+
Pyodide (Python, numpy only) Browser (existing JS)
33+
run(seed, render=False) -> Trace ──▶ trace JSON ──▶ current scene /
34+
trace.summary() / per-step records (postMessage) belief / timeline
35+
renderer
36+
```
37+
38+
- The Python side runs the **real** `examples/.../*.py` loop headless
39+
(`render=False`) and returns the `Trace` (already a first-class, tested object
40+
— see [`docs/trace.md`](trace.md)).
41+
- A thin serializer turns the `Trace` (obs/action/info/reward per step) into the
42+
JSON shape the current `playground.js` renderer already consumes.
43+
- The JS reimplementation of dynamics gets **deleted**; JS keeps only drawing.
44+
45+
This keeps the browser bundle small (no matplotlib), makes the playground a true
46+
mirror of the tested code, and removes the drift problem instead of adding to it.
47+
48+
Fallback option (heavier): import `matplotlib` in Pyodide and blit Agg PNG frames
49+
to a `<canvas>`. Simpler to wire (reuses `env.render`), but a multi-MB download
50+
and janky first paint. Keep this only as a stopgap for an example whose JSON
51+
renderer is not ready yet.
52+
53+
## Packaging
54+
55+
The loops only need `pir` + `numpy`. Options, simplest first:
56+
57+
1. **Load `pir` as source over the network.** `micropip` or Pyodide's
58+
`loadPackage("numpy")` for numpy, then fetch the handful of `pir/**.py` files
59+
(or a generated single-file bundle) and write them into Pyodide's virtual FS.
60+
No build step; works from GitHub Pages.
61+
2. **Ship a wheel.** `python -m build`, host `pir-0.1.0-py3-none-any.whl` under
62+
`docs/`, `micropip.install("./pir-...whl")`. Cleaner import story; adds a
63+
release artifact to keep current.
64+
65+
Start with (1) for the first flagship, move to (2) if import wiring gets noisy.
66+
`numpy` has a prebuilt Pyodide package, so no compilation is needed.
67+
68+
## The 5 flagship loops (and their render shape)
69+
70+
| Example | Renderer needed | Notes |
71+
| --- | --- | --- |
72+
| `manipulation/01_pick_and_retry` | tabletop (continuous) | best first target; the hero story, small state |
73+
| `navigation/04_online_replanning_astar` | grid + path | reuses grid renderer; shows replanning |
74+
| `navigation/29_safety_filter_cbf` | continuous + obstacles | needs vector overlay (nominal vs safe u) |
75+
| `navigation/07_active_slam_toy` | grid + belief heatmap | reuses belief panel |
76+
| `embodied_ai/35_clarifying_question` | already in JS today | swap JS dynamics for real Python first |
77+
78+
Two render families cover all five: **grid** (already drawn today) and
79+
**tabletop/continuous** (small addition). Build those two renderers once.
80+
81+
## Phased plan
82+
83+
**Phase 0 — proof of concept (½ day).** A standalone `docs/pyodide_poc.html`
84+
that loads Pyodide, installs numpy, fetches `pir` + `pick_and_retry.py`, runs
85+
`run(seed=0, render=False)`, and `console.log`s `trace.summary()`. Goal: confirm
86+
load time and that the real loop runs unmodified. Decide packaging (1) vs (2).
87+
88+
**Phase 1 — one real loop on the page (1–2 days).** Add a "Run real Python"
89+
toggle to the existing playground for `clarifying_question` (its renderer already
90+
exists). Python produces the trace; JS draws it; delete the JS dynamics for that
91+
scenario. This is the first honest "real Python in your browser" claim.
92+
93+
**Phase 2 — tabletop renderer + hero loop (1–2 days).** Add the continuous
94+
tabletop renderer and wire `pick_and_retry`. Now the README hero GIF has a
95+
"run it yourself" twin.
96+
97+
**Phase 3 — editable code cell (1–2 days).** Expose the agent's `act()` in a
98+
small editor so visitors can tweak the retry/belief logic and re-run. This is
99+
the "wow, I can edit the robot's brain in the browser" moment that converts to
100+
stars.
101+
102+
Ship Phase 0–1 behind the existing playground before any Hacker News launch;
103+
Phases 2–3 can follow the launch.
104+
105+
## Risks / watch-list
106+
107+
- **First-load latency.** Pyodide core is a few MB. Lazy-load it only when the
108+
user clicks "Run real Python"; keep the instant JS-rendered preview as the
109+
default first paint. Cache aggressively.
110+
- **No silent matplotlib import.** If any example imports matplotlib at module
111+
top level, the headless path drags it in. Keep example imports of matplotlib
112+
lazy (inside `render`/`main`), as `01_pick_and_retry.py` already does.
113+
- **Trace serialization is the contract.** Add a `tests/` check that the JSON
114+
serializer covers every field the JS renderer reads, so Python and browser
115+
cannot drift — this is the guard that makes Pyodide *reduce* drift rather than
116+
add a new surface.
117+
- **Keep it optional.** Pyodide is a `docs/` concern only. It must never become
118+
a core dependency or touch the 5-second local first-run.
119+
120+
## Definition of done (item ②)
121+
122+
- One flagship loop runs its **unmodified** Python `run(...)` in the browser.
123+
- The JS reimplementation of that scenario's dynamics is deleted.
124+
- First paint stays instant (Pyodide lazy-loaded on demand).
125+
- A test pins the trace-JSON contract shared by Python and JS.

‎docs/release_notes_v0.1.0.md‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Release draft — v0.1.0 "Tiny Robot Failure Lab"
2+
3+
> Draft for the first public GitHub Release. Paste the body below into the
4+
> release on the `v0.1.0` tag. Keep it a story, not a changelog. Cut anything
5+
> that reads like an internal status report.
6+
7+
**Tag:** `v0.1.0` &nbsp;·&nbsp; **Title:** `v0.1.0 — Tiny Robot Failure Lab`
8+
9+
---
10+
11+
## Release body (copy from here)
12+
13+
### Robotics tutorials usually assume actions succeed. This one is about what happens when they don't.
14+
15+
Real robots miss grasps, drive into walls they couldn't see, lose track of where
16+
they are, and misread ambiguous commands. PythonInteractiveRobotics is a tiny
17+
lab for exactly that part of robotics — **observe, act, fail, update your
18+
belief, replan, retry** — in readable Python with no ROS, no GPU, and no
19+
simulator. Just `numpy + matplotlib`.
20+
21+
**What's in this first release**
22+
23+
A failure-first course of **10 short, runnable loops**, in order, plus 39 total
24+
examples to branch into. Start here:
25+
26+
```bash
27+
git clone https://github.com/rsasaki0109/PythonInteractiveRobotics.git
28+
cd PythonInteractiveRobotics
29+
python3 -m pip install -e .
30+
python3 examples/manipulation/01_pick_and_retry.py
31+
```
32+
33+
A tabletop robot misses a grasp, updates its belief, and retries — in under 5
34+
seconds.
35+
36+
**Try it without installing**
37+
38+
- ▶ **Run in your browser:** the [live playground](https://rsasaki0109.github.io/PythonInteractiveRobotics/playground.html)
39+
with belief entropy, compare mode, and a failure timeline.
40+
- 📓 **Open in Colab:** [pick and retry](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/pick_and_retry.ipynb),
41+
[safety filter](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/safety_filter_cbf.ipynb),
42+
[household task agent](https://colab.research.google.com/github/rsasaki0109/PythonInteractiveRobotics/blob/main/notebooks/household_task_agent.ipynb).
43+
- 🎓 **Take the tour:** the [10-lesson failure-first course](https://github.com/rsasaki0109/PythonInteractiveRobotics/blob/main/lessons/README.md).
44+
45+
**Highlights**
46+
47+
- **Fail and retry** — grasp miss → belief update → retry (`manipulation/01`)
48+
- **Replan around a hidden wall** — plan → see new obstacle → replan (`navigation/04`)
49+
- **Act to learn** — toy active SLAM that moves to shrink uncertainty (`navigation/07`)
50+
- **Stay safe at runtime** — a CBF safety filter the policy never knows about (`navigation/29`)
51+
- **Ask before acting** — ambiguous command → clarify → act (`embodied_ai/35`)
52+
- **Put it together** — a household agent that clarifies, plans, stays safe,
53+
retries, and replans in one run (`embodied_ai/36`)
54+
55+
Every example exposes failure through `info["failure"]` and returns an
56+
inspectable `Trace`, so you can study a run headless without rendering.
57+
58+
**Under the hood**
59+
60+
- 39 runnable examples · 38 generated GIFs · 5 Colab notebooks
61+
- 111 smoke / regression tests · CI green on Python 3.10, 3.11, and 3.12
62+
- Core deps: `numpy` + `matplotlib` only; optional Gymnasium-style adapters and
63+
ROS2 / simulator bridge docs for when you outgrow the toy worlds
64+
65+
**Where this sits**
66+
67+
Not a replacement for ROS2, MoveIt, MuJoCo, Isaac Sim, or LeRobot — and not a
68+
benchmark. Think of it as the missing closed-loop chapter you read *after*
69+
algorithm textbooks like PythonRobotics and *before* a heavy stack: a small,
70+
debuggable model of failure, belief, recovery, and replanning.
71+
72+
**Contribute**
73+
74+
Good first contributions are deliberately small:
75+
76+
- add a new **failure mode** to an existing world
77+
- add a **one-file lesson** in the failure-first style
78+
- improve a **trace story / GIF**
79+
80+
See [`CONTRIBUTING.md`](https://github.com/rsasaki0109/PythonInteractiveRobotics/blob/main/CONTRIBUTING.md).
81+
If this helped you learn, teach, or prototype, a ⭐ helps others find it.
82+
83+
---
84+
85+
## Pre-publish checklist
86+
87+
Before tagging `v0.1.0`, confirm the launch surface is ready (see also
88+
`docs/public_launch.md`):
89+
90+
- [ ] README top renders the hero GIF, the 8-line loop, and the browser link in
91+
the first screen
92+
- [ ] `lessons/README.md` links resolve on GitHub and all 10 commands run
93+
- [ ] CI green on `main` for 3.10 / 3.11 / 3.12
94+
- [ ] GitHub repo "social preview" image set (Settings → General → Social preview)
95+
- [ ] Playground page loads from GitHub Pages
96+
- [ ] Tag the release: `git tag -a v0.1.0 -m "Tiny Robot Failure Lab" && git push origin v0.1.0`
97+
98+
## How to cut the release
99+
100+
```bash
101+
# from a clean main with everything pushed
102+
git tag -a v0.1.0 -m "Tiny Robot Failure Lab"
103+
git push origin v0.1.0
104+
# then create the GitHub Release on that tag and paste the body above,
105+
# or: gh release create v0.1.0 --title "v0.1.0 — Tiny Robot Failure Lab" --notes-file <(...)
106+
```

0 commit comments

Comments
 (0)