Skip to content

Commit 1f516af

Browse files
committed
add object permanence toy example
1 parent 7a26277 commit 1f516af

10 files changed

Lines changed: 501 additions & 20 deletions

File tree

README.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,10 @@ robotics loops rather than standalone algorithms.
1111

1212
## Current Status
1313

14-
- 25 runnable examples
14+
- 26 runnable examples
1515
- 20 learning-path roadmap examples
16-
- 24 README GIFs generated from runnable examples
17-
- 67 smoke and regression tests
16+
- 25 README GIFs generated from runnable examples
17+
- 68 smoke and regression tests
1818
- Core dependencies only: `numpy` and `matplotlib`
1919

2020
See `docs/status.md` for the implementation snapshot and `docs/plan.md` for
@@ -136,6 +136,10 @@ These GIFs are generated from the runnable examples, not separate animations.
136136
| --- | --- |
137137
| ![A kitchen agent parses a bring goal, searches containers, handles a closed cabinet, picks a mug, and places it on the table.](docs/assets/gifs/goal_conditioned_minikitchen.gif) | ![A toy VLA loop parses a language goal, reads visual tokens, picks from low confidence, recovers with a close view, and places the block.](docs/assets/gifs/tiny_vla_loop.gif) |
138138

139+
| Object permanence toy |
140+
| --- |
141+
| ![An embodied agent sees an object, watches it go behind an occluder, persists its memory, walks to the remembered position, and peeks behind the occluder to recover the object.](docs/assets/gifs/object_permanence_toy.gif) |
142+
139143
### World models
140144

141145
| Tiny world-model planning |
34.1 KB
Loading

docs/learning_paths.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,11 @@
3636
2. `examples/embodied_ai/10_door_search_pomdp.py`
3737
3. `examples/embodied_ai/18_goal_conditioned_minikitchen.py`
3838
4. `examples/embodied_ai/19_tiny_vla_loop.py`
39-
5. `examples/world_models/20_tiny_world_model_planning.py`
39+
5. `examples/embodied_ai/21_object_permanence_toy.py`
40+
6. `examples/world_models/20_tiny_world_model_planning.py`
4041

4142
## Future Embodied AI Extensions
4243

43-
1. object permanence toy
44-
2. curiosity grid exploration
44+
1. curiosity grid exploration
4545

4646
See `docs/goal_commands.md` for the first controlled-language command set.

docs/plan.md

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ first-run experience small, fast, and failure-aware.
88

99
The repository currently has:
1010

11-
- 25 runnable examples
11+
- 26 runnable examples
1212
- 20 numbered learning-path examples
13-
- 5 extra examples outside the original learning-path roadmap
14-
- 24 generated README GIFs
15-
- 67 smoke and regression tests
13+
- 6 extra examples outside the original learning-path roadmap
14+
- 25 generated README GIFs
15+
- 68 smoke and regression tests
1616
- GitHub Actions CI for Python 3.10, 3.11, and 3.12
1717
- core dependencies limited to `numpy` and `matplotlib`
1818
- optional Gymnasium-style adapters for `GridWorld2D`,
@@ -94,6 +94,10 @@ Recent completed work:
9494
its pose belief, runs a Bayes update from each observation, then grasps
9595
with the type that maximizes expected success. GIF and smoke test cover
9696
the loop.
97+
- `examples/embodied_ai/21_object_permanence_toy.py` was added. The agent
98+
sees an object once, watches it go behind an occluder, persists the last
99+
known position across the disappearance, walks to it, and peeks behind
100+
the occluder to recover the object. GIF and smoke test cover the loop.
97101

98102
The next agent should not redo those items. If any of them seem missing, first
99103
check the current branch and latest pulled commit.
@@ -250,17 +254,18 @@ Already done from the previous tier:
250254
grasp choice -> failure update -> retry.
251255
- `examples/manipulation/09_active_viewpoint_for_grasp.py` — pose belief ->
252256
active viewpoint -> Bayes update -> grasp.
257+
- `examples/embodied_ai/21_object_permanence_toy.py` — see object ->
258+
memory persists across occlusion -> peek to recover.
253259

254260
Recommended next examples:
255261

256262
| Priority | Example | Area | Loop |
257263
| --- | --- | --- | --- |
258-
| 1 | `examples/embodied_ai/21_object_permanence_toy.py` | embodied AI | object disappears -> memory persists -> search |
259-
| 2 | `examples/embodied_ai/22_where_did_i_see_it.py` | embodied AI | memory query -> revisit place -> act |
260-
| 3 | `examples/world_models/23_model_error_recovery.py` | world model | prediction failure -> update model -> recover |
261-
| 4 | `examples/navigation/24_information_gain_navigation.py` | navigation | goal progress vs information gain |
262-
| 5 | `examples/manipulation/25_clear_path_before_pick.py` | manipulation | precondition failure -> clear obstacle -> pick |
263-
| 6 | `examples/navigation/27_multi_agent_avoidance.py` | navigation | observe agents -> avoid -> replan |
264+
| 1 | `examples/embodied_ai/22_where_did_i_see_it.py` | embodied AI | memory query -> revisit place -> act |
265+
| 2 | `examples/world_models/23_model_error_recovery.py` | world model | prediction failure -> update model -> recover |
266+
| 3 | `examples/navigation/24_information_gain_navigation.py` | navigation | goal progress vs information gain |
267+
| 4 | `examples/manipulation/25_clear_path_before_pick.py` | manipulation | precondition failure -> clear obstacle -> pick |
268+
| 5 | `examples/navigation/27_multi_agent_avoidance.py` | navigation | observe agents -> avoid -> replan |
264269

265270
Selection rule:
266271

docs/status.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@ see what exists, what is verified, and what should come next.
55

66
## Snapshot
77

8-
- Runnable examples: 25
8+
- Runnable examples: 26
99
- Learning-path roadmap examples: 20
10-
- README GIFs: 24
11-
- Smoke and regression tests: 67
10+
- README GIFs: 25
11+
- Smoke and regression tests: 68
1212
- Core dependencies: `numpy`, `matplotlib`
1313
- Contributor extra: `pip install -e ".[dev]"`
1414
- CI: Python 3.10, 3.11, 3.12
@@ -31,7 +31,7 @@ goal-command example. The numbered 20-example roadmap is tracked in
3131
| Runtime | 2 | smallest observe-act-observe loop, trace replay |
3232
| Navigation | 9 | reactive avoidance, dynamic obstacles, replanning, exploration, belief, active SLAM, MPC, recovery, localization recovery |
3333
| Manipulation | 9 | retry, reactive grasping, IK servo, moving target reaching, search, push recovery, suction sorting, belief grasp selection, active viewpoint grasp |
34-
| Embodied AI | 4 | controlled goals, memory, POMDP search, tiny VLA loop |
34+
| Embodied AI | 5 | controlled goals, memory, POMDP search, tiny VLA loop, object permanence |
3535
| World models | 1 | action-conditioned dynamics, prediction error, model update, replanning |
3636

3737
## Verification

examples/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ Run any example headless with its `--no-render` flag when available.
4848
| `embodied_ai/10_door_search_pomdp.py` | `python examples/embodied_ai/10_door_search_pomdp.py` | room belief -> door/container action -> belief update |
4949
| `embodied_ai/18_goal_conditioned_minikitchen.py` | `python examples/embodied_ai/18_goal_conditioned_minikitchen.py "bring mug to table"` | goal -> container search -> pick -> place |
5050
| `embodied_ai/19_tiny_vla_loop.py` | `python examples/embodied_ai/19_tiny_vla_loop.py "place red block in blue bin"` | language -> visual tokens -> skill -> retry |
51+
| `embodied_ai/21_object_permanence_toy.py` | `python examples/embodied_ai/21_object_permanence_toy.py` | see object -> memory persists across occlusion -> peek |
5152

5253
## World Models
5354

0 commit comments

Comments
 (0)