add inverse_reward_from_demo example

A new embodied-AI example that learns linear reward weights from a
single demonstration trajectory and reuses them to plan to a brand
new goal in the same world.

The demonstrator walks from a start cell to a goal cell along a path
that detours through two hidden "scenic" zones the engineer never
told the agent about. The agent extracts a per-cell feature vector
phi(s) = (scenic, wall_adjacent, interior), computes the demo path's
feature expectation mu_demo, the uniform-walkable-cell expectation
mu_uniform, and clips their difference to learn linear reward
weights `w = clip(mu_demo - mu_uniform)`.

It then plans to a *different* (start, goal) pair with shaped A*
whose edge cost is `1 - lambda * (w . phi(target_cell))`. On the
default grid the learned-reward path is the same length as the
Manhattan-baseline path (21 cells) but visits four scenic cells
versus one incidentally for the baseline. The agent never saw a
label saying scenic mattered - it inferred that preference from one
trajectory.

This is structurally different from 28_curiosity_grid_exploration.py
and 32_empowerment_navigation.py, where the intrinsic signal is
hand-designed by the engineer. Here the agent has no built-in
preference for scenic zones; it learns the preference from the demo.

Exposes demo_path_length, learned_path_length, baseline_path_length,
demo_scenic_step_count, learned_scenic_step_count,
baseline_scenic_step_count, mu_demo, mu_uniform, learned_weights.
Structured failures: timeout (terminal), no_demo_path (recoverable -
the demo trajectory cannot be synthesized), no_learned_path
(terminal - shaped A* could not reach the new goal),
out_of_bounds/collision (recoverable, only on rejected actions).

Tests: smoke test pins seed=0 and asserts success, the demo touches
multiple scenic cells, the learned path's scenic step count strictly
exceeds the Manhattan-baseline path's, and the scenic-feature weight
is positive. A second test pins the IRL math: `mu_demo[scenic] >
mu_uniform[scenic]` and the resulting scenic weight is positive.

Counts: 36 examples / 35 GIFs / 88 tests / 8 embodied-AI examples.

Author: rsasaki0109

README.md

@@ -11,10 +11,10 @@ robotics loops rather than standalone algorithms.
 
 ## Current Status
 
-- 35 runnable examples
+- 36 runnable examples
 - 20 learning-path roadmap examples
-- 34 README GIFs generated from runnable examples
-- 86 smoke and regression tests
+- 35 README GIFs generated from runnable examples
+- 88 smoke and regression tests
 - Core dependencies only: `numpy` and `matplotlib`
 
 See `docs/status.md` for the implementation snapshot and `docs/plan.md` for

docs/assets/gifs/inverse_reward_from_demo.gif

@@ -6,10 +6,10 @@ continuous-control Gymnasium adapter, and the curiosity exploration example.
 
 ## Current Stable Surface
 
-- 35 runnable examples
-- 20 numbered learning-path examples plus 15 extras
-- 34 generated README GIFs with nonblank checks
-- 86 smoke, adapter, and regression tests
+- 36 runnable examples
+- 20 numbered learning-path examples plus 16 extras
+- 35 generated README GIFs with nonblank checks
+- 88 smoke, adapter, and regression tests
 - CI green on Python 3.10, 3.11, and 3.12
 - Gymnasium-style adapters for `GridWorld2D`, `DynamicObstacleGridWorld`,
   `BlockedPathWorld`, `MovingObstacleWorld`, and `Tabletop2D`
@@ -39,6 +39,7 @@ continuous-control Gymnasium adapter, and the curiosity exploration example.
 | `30_conformal_ask_for_help.py` | Offline conformal calibration; place when the prediction set is a singleton, defer to a toy oracle when not. Counters: `q_hat`, `help_request_count`, `coverage_violation_count`. |
 | `31_options_with_interrupts.py` | Sutton-style options framework on a battery-aware navigation task; `go_to_goal` and `dock_and_charge` options with explicit `β` and a meta-policy interrupt rule. Counters: `option_start_count`, `option_interrupt_count`, `interrupts_due_to_battery_count`, `dock_count`, `recharge_step_count`. |
 | `32_empowerment_navigation.py` | k-step empowerment as a state-intrinsic shaping signal; shaped A* prefers open cells with many reachable successors. Geometry-only intrinsic motivation contrasting with the visit-count signal in `28_curiosity_grid_exploration.py`. |
+| `33_inverse_reward_from_demo.py` | Linear inverse reward learning from one demo trajectory; `w = clip(mu_demo - mu_uniform)` then shaped A* to a new goal. The agent reproduces the demo's preference for hidden scenic zones without being told what scenic means. |
 
 ## Ranked Next Work
 

docs/implementation_gap_audit.md

@@ -45,7 +45,8 @@
 5. `examples/embodied_ai/21_object_permanence_toy.py`
 6. `examples/embodied_ai/28_curiosity_grid_exploration.py`
 7. `examples/embodied_ai/32_empowerment_navigation.py`
-8. `examples/world_models/20_tiny_world_model_planning.py`
-9. `examples/world_models/23_model_error_recovery.py`
+8. `examples/embodied_ai/33_inverse_reward_from_demo.py`
+9. `examples/world_models/20_tiny_world_model_planning.py`
+10. `examples/world_models/23_model_error_recovery.py`
 
 See `docs/goal_commands.md` for the first controlled-language command set.

docs/learning_paths.md

@@ -8,11 +8,11 @@ first-run experience small, fast, and failure-aware.
 
 The repository currently has:
 
-- 35 runnable examples
+- 36 runnable examples
 - 20 numbered learning-path examples
-- 15 extra examples outside the original learning-path roadmap
-- 34 generated README GIFs
-- 86 smoke and regression tests
+- 16 extra examples outside the original learning-path roadmap
+- 35 generated README GIFs
+- 88 smoke and regression tests
 - GitHub Actions CI for Python 3.10, 3.11, and 3.12
 - core dependencies limited to `numpy` and `matplotlib`
 - optional Gymnasium-style adapters for `GridWorld2D`,

docs/plan.md

@@ -5,10 +5,10 @@ see what exists, what is verified, and what should come next.
 
 ## Snapshot
 
-- Runnable examples: 35
+- Runnable examples: 36
 - Learning-path roadmap examples: 20
-- README GIFs: 34
-- Smoke and regression tests: 86
+- README GIFs: 35
+- Smoke and regression tests: 88
 - Core dependencies: `numpy`, `matplotlib`
 - Contributor extra: `pip install -e ".[dev]"`
 - CI: Python 3.10, 3.11, 3.12
@@ -31,7 +31,7 @@ goal-command example. The numbered 20-example roadmap is tracked in
 | Runtime | 2 | smallest observe-act-observe loop, trace replay |
 | Navigation | 13 | reactive avoidance, dynamic obstacles, replanning, exploration, belief, active SLAM, MPC, recovery, localization recovery, information-gain detour, multi-agent avoidance, CBF safety filter, options with interrupts |
 | Manipulation | 11 | retry, reactive grasping, IK servo, moving target reaching, search, push recovery, suction sorting, belief grasp selection, active viewpoint grasp, clear path before pick, conformal ask-for-help |
-| Embodied AI | 7 | controlled goals, memory, POMDP search, tiny VLA loop, object permanence, curiosity exploration, empowerment shaping |
+| Embodied AI | 8 | controlled goals, memory, POMDP search, tiny VLA loop, object permanence, curiosity exploration, empowerment shaping, inverse reward from demo |
 | World models | 2 | action-conditioned dynamics, prediction error, model update, replanning, regime-shift detection, system identification |
 
 ## Verification

docs/status.md

@@ -57,6 +57,7 @@ Run any example headless with its `--no-render` flag when available.
 | `embodied_ai/21_object_permanence_toy.py` | `python examples/embodied_ai/21_object_permanence_toy.py` | see object -> memory persists across occlusion -> peek |
 | `embodied_ai/28_curiosity_grid_exploration.py` | `python examples/embodied_ai/28_curiosity_grid_exploration.py` | visit counts -> novelty score -> A* to novel cell -> coverage |
 | `embodied_ai/32_empowerment_navigation.py` | `python examples/embodied_ai/32_empowerment_navigation.py` | k-step empowerment -> shaped A* -> prefer open routes |
+| `embodied_ai/33_inverse_reward_from_demo.py` | `python examples/embodied_ai/33_inverse_reward_from_demo.py` | demo feature expectation -> learned weights -> shaped A* to new goal |
 
 ## World Models