Skip to content

feat: add seconds-based task horizons - #161

Merged
jeqcho merged 3 commits into
robocurve:mainfrom
Vedangalle:codex/max-seconds-horizon
Jul 27, 2026
Merged

feat: add seconds-based task horizons#161
jeqcho merged 3 commits into
robocurve:mainfrom
Vedangalle:codex/max-seconds-horizon

Conversation

@Vedangalle

Copy link
Copy Markdown
Contributor

Summary

Add seconds-based task horizons as an alternative to fixed step counts. This lets benchmark authors express a physical duration without coupling the task definition to one embodiment's control rate, while preserving the integer TaskEnvelope.max_steps contract used by adapters.

Changes

  • Require each Task to declare exactly one of max_steps or keyword-only max_seconds.
  • Validate seconds horizons and embodiment control_hz values as positive and finite.
  • Resolve ceil(max_seconds * control_hz) after compatibility checking and before bind_task() or rollout, with a one-step minimum for positive floating-point underflow.
  • Record the declared max_seconds and resolved max_steps in EvalLog schema v1 while preserving old-log compatibility.
  • Show declared and resolved horizons in CLI notices, inspect, eval-set, and HTML reports.
  • Document the design and migration behavior; update the changelog and edge-case coverage.

Rate-unknown and event-driven embodiments continue to use max_steps. Scorer interfaces, ad-hoc --max-steps, rollout pacing, and dependencies are unchanged.

Checklist

  • pre-commit run --all-files and the pre-push hook stage pass.
  • Tests added/updated, including mock-world orchestration and edge cases.
  • Coverage remains at 100% line and branch (769 passed, 2 skipped).
  • ruff check . and ruff format --check . pass.
  • Strict mypy passes.
  • CHANGELOG.md updated under "Unreleased".
  • Public API behavior is documented and API snapshot tests pass; no new exported symbol is required.
  • Core remains NumPy-only with no new dependencies.

Additional documentation checks:

  • API reference generation passes.
  • Docusaurus production build passes.

Related

Closes #160

@Vedangalle
Vedangalle marked this pull request as ready for review July 22, 2026 07:55
@Adityakk9031

Copy link
Copy Markdown
Contributor

please close this maintainer assigned this issue to me

@Adityakk9031

Copy link
Copy Markdown
Contributor

check this robocurve/kitchenbench#29

@Vedangalle

Vedangalle commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for flagging this, and sorry - I missed Jay’s note on KitchenBench #29 that #160 was yours first. I saw the framework issue unassigned and implemented it from there; I don’t want to duplicate or cut across your work.

Since #161 now contains the core framework implementation and validation, could we coordinate by keeping this PR for the inspect-robots hook while you continue the KitchenBench task and version migration in #29 against it? @jeqcho, I’m happy to follow your preference if you want the ownership structured differently.

@Adityakk9031

Adityakk9031 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

please close this

@Adityakk9031

Copy link
Copy Markdown
Contributor

@Vedangalle please close this pr

@jeqcho

jeqcho commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Hi @Vedangalle and @Adityakk9031, thanks both for taking #160 on.

I'm going with #161. To be clear about why, since the sequencing is messy: I offered this to @Adityakk9031 on kitchenbench#29 on 07-21, and I should have assigned the issue then. That one's on me. But the reply came 21 hours later, and by that point #161 by @Vedangalle had been up and ready for review (07-22 07:55:34 UTC (#161 (comment)), while @Adityakk9031 claimed it 11 hours after #161 at 19:30:21 (robocurve/kitchenbench#29 (comment))). I'd rather not discard finished work over a claim that arrived after it.

To @Vedangalle: the design doc helped, and two calls in particular: making max_seconds keyword-only so the positional Task(...) signature doesn't shift under existing benchmarks, and moving assert_compatible ahead of bind_task so a bad control_hz is caught before an adapter sees the envelope.

To @Adityakk9031: your PR #162 does one thing #161 doesn't: it rejects bool for max_steps, so Task(max_steps=True) can't slip through. This is good, and I would love to see a PR from you on this. There are plenty other issues open across github.com/robocurve if you want the next one.

@jeqcho
jeqcho merged commit 7530a59 into robocurve:main Jul 27, 2026
18 checks passed
@jeqcho

jeqcho commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Merged. The underflow floor is the detail I liked most here: max(1, ceil(...)) means a tiny max_seconds on a slow embodiment still gets one step instead of a trial that ends before it starts. I checked it holds even at 5e-324 where the product itself underflows to zero.

I resolved the CHANGELOG conflict against main and pushed it to your branch rather than making you rebase, since main had moved a fair way.

@Vedangalle

Copy link
Copy Markdown
Contributor Author

Thanks @jeqcho - I appreciate the clear rationale and the merge. Glad the API compatibility, validation ordering, and underflow handling proved useful. I’ll keep an eye on the downstream KitchenBench integration and any follow-up gaps that surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[design] Seconds-based task horizons resolved against the embodiment's control rate at eval() time

3 participants