Skip to content

feat: enforce per-task execution budgets - #59

Open
carbongotfound wants to merge 4 commits into
milind-soni:mainfrom
carbongotfound:feat/task-budget-guards
Open

feat: enforce per-task execution budgets#59
carbongotfound wants to merge 4 commits into
milind-soni:mainfrom
carbongotfound:feat/task-budget-guards

Conversation

@carbongotfound

Copy link
Copy Markdown
Contributor

Summary

Adds optional, per-task execution budgets to the harness. Existing turns remain unlimited unless a caller explicitly supplies a budget on POST /api/bots/:id/messages.

  • centrally observes canonical runtime events instead of adding provider-specific counters
  • supports input, output, and total token limits when providers report usage
  • supports tool-call, observable computer-action, failed-tool retry, peer-delegation, and wall-clock limits
  • emits structured task.budget.updated (approaching) and task.budget.exhausted events
  • appends a clear activity entry and safely interrupts the running adapter when a hard limit is reached
  • disposes guards on normal completion, dispatch failures, and user cancellation

Architecture

TaskBudgetGuard is a small erasable-TypeScript module compatible with the repository's native Node type-strip execution. It has explicit fields and constructor assignments—no parameter properties and no new transpiler/runtime dependency.

The guard consumes only the existing canonical event stream. Token usage uses monotonic provider-reported snapshots; absent token dimensions stay absent, so a token limit is never enforced from a fabricated estimate. Computer and delegation budgets are applied only where tool titles make those actions observable.

Overlap avoided

Open PR #57 records usage and improves replay compaction. This PR does not duplicate its storage/UI accounting: it uses the canonical events to enforce explicit task limits and introduces separate budget-status events. It does touch the shared contracts/index lifecycle, so it should be merged or rebased thoughtfully with #57.

Validation

  • pnpm typecheck
  • pnpm test — 69 passed, 39 skipped
  • pnpm build
  • focused native Node strip-mode harness test (server/index.test.ts) — passed; this is the path that previously surfaced the parameter-property failure
  • native Node MCP protocol smoke: node --experimental-strip-types server/computer-proxy.ts

Limitations

  • No token savings are claimed; this slice establishes bounded execution, not an optimization benchmark.
  • The UI currently shows hard-limit exhaustion through the activity timeline; detailed budget configuration/status remains API/runtime-event based until it can be added coherently with PR feat: compact replay context and track task usage #57's task usage UI.
  • Retry and computer-action budgets are enforced only for actions observable through normalized tool events. Providers that omit those events are not guessed at or forcibly limited on that dimension.

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@carbongotfound, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 52 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1dd68fc0-329c-4aeb-9916-d9b71fa9dca8

📥 Commits

Reviewing files that changed from the base of the PR and between 8490f3a and decb3a3.

📒 Files selected for processing (5)
  • server/contracts.ts
  • server/index.test.ts
  • server/index.ts
  • server/task-budget.test.ts
  • server/task-budget.ts

Comment @coderabbitai help to get the list of available commands.

@milind-soni milind-soni left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Execution budgets need a few semantic fixes before they can safely enforce limits. thread.token-usage.updated is cumulative provider or thread telemetry for some drivers, so treating its absolute snapshot as current-task usage can exhaust a new task because of earlier turns; establish a per-turn baseline or consume turn-scoped deltas. A duration limit can also fire during provisioning before sendTurn exists, after which the code still dispatches the turn and the already-exhausted guard cannot stop it again. Finally, a failed tool is not necessarily a retry. Please make the counters truthful, prevent dispatch after pre-turn exhaustion, and add integration tests for resumed sessions, provisioning timeout, user interruption, and exact one-time exhaustion.

@carbongotfound

Copy link
Copy Markdown
Contributor Author

Addressed the requested budget semantics in b29f8fe. The guard now snapshots known per-thread provider telemetry at task start and counts only the task-local delta, so resumed/cumulative sessions do not consume a new task's token budget. A failed tool becomes a retry only when that same tool is invoked again. Deadline exhaustion during provisioning is checked immediately before sendTurn, preventing any provider dispatch after the task has already stopped; exhaustion remains one-time. Added coverage for cumulative baselines, retry classification, and exact one-time provisioning-time deadline exhaustion. Verified: corepack pnpm exec vitest run server/task-budget.test.ts server/index.test.ts (19 passed) and corepack pnpm typecheck.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants