Skip to content

fix(rollout): persist post-action frames - #209

Open
Vedangalle wants to merge 5 commits into
robocurve:mainfrom
Vedangalle:agent/persist-post-action-frames
Open

fix(rollout): persist post-action frames#209
Vedangalle wants to merge 5 commits into
robocurve:mainfrom
Vedangalle:agent/persist-post-action-frames

Conversation

@Vedangalle

@Vedangalle Vedangalle commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Persist every distinct rollout observation through FrameStore, including the terminal post-action state.

Previously, the rollout stored only each step's pre-action observation. That left full camera arrays inside StepResult.observation and gave the terminal observation no sidecar reference, because it never becomes a later step's pre-action input.

Changes

  • store the reset observation at index 0 and each step result at t + 1;
  • add StepRecord.result_image_refs for post-action frame handles;
  • strip camera arrays from both recorded pre-action and post-action observations;
  • reuse each stored post-action view on the next step, preventing duplicate writes;
  • document the storage and compatibility contract in plans/0032-post-action-frame-persistence.md;
  • add regression coverage for contiguous indexing, exact write count, and terminal-frame recovery.

A one-camera rollout with n steps now writes exactly n + 1 frame files.

Compatibility

  • result_image_refs is appended with a default of None.
  • Runs without FrameStore preserve the original observation and result objects.
  • StepRecord is not part of the schema-versioned EvalLog, so no log migration is required.
  • Core remains NumPy-only and no exported symbol changes are required.

Verification

  • Tests added/updated, including terminal-state and exact-write-count coverage
  • Full local suite on current main: 876 passed, 3 skipped
  • Coverage: 100% line and branch
  • ruff check . passes
  • ruff format --check . passes
  • Strict mypy passes
  • CHANGELOG.md updated under "Unreleased"
  • Core remains NumPy-only

The gap surfaced while building a process-aware evaluator on top of Inspect Robots: https://github.com/Vedangalle/inspect-robots-verifier

@Vedangalle
Vedangalle marked this pull request as ready for review July 28, 2026 21:32
@jeqcho
jeqcho force-pushed the agent/persist-post-action-frames branch from ca7340d to 25c30bc Compare July 29, 2026 06:15
@jeqcho

jeqcho commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Heads up: main's history was rewritten today to purge 3 GB of accidentally committed eval logs (#212). Your branch was based on one of the rewritten commits, so I rebased it onto the new main and force-pushed (maintainer edits are enabled on this PR). No changes to your commits themselves; CI will re-run. If you have local work on this branch, run git fetch and rebase onto the updated remote branch before pushing.

@Vedangalle
Vedangalle force-pushed the agent/persist-post-action-frames branch from 25c30bc to c98fd9c Compare July 29, 2026 06:50
@Vedangalle

Copy link
Copy Markdown
Contributor Author

@jeqcho When you have a chance, I’d appreciate your (any) feedback on the FrameStore contract here - particularly whether result_image_refs is the right interface for post-action and terminal observations. Happy to adjust quickly! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants