fix(routing): define hard cost-cap behavior when cost evidence is unknown

[abhisheksharma2411]

Fixes #1181.

Summary

limits.maxEstimatedCostUsd is documented as a hard per-request ceiling, but it never fires on the live routing path.

evaluatePolicyProfile only excludes a candidate when the estimate is a finite number (src/routing/evaluator.ts):

const overCostLimit = costLimit !== undefined
  && typeof estimatedCost === "number"   // an unknown estimate passes silently
  && Number.isFinite(estimatedCost)
  && estimatedCost > costLimit;

…while routeModel assembles cost evidence without usage (src/router.ts):

cost: costEvidenceForCandidate({
  provider: candidate.provider,
  model: candidate.model,
  limitUsd: profile.limits.maxEstimatedCostUsd,   // no `usage`
}),

costEvidenceForCandidate correctly returns { limitUsd, incomplete: true } with no estimatedUsd, so on the live path the estimate is always unknown and any candidate passes a cap the operator configured as hard. The existing coverage passes only because it supplies usage directly — exercising a path production does not take.

As #1181 notes, failing closed unconditionally is not safe either: with usage unwired it would reject every live candidate whenever a cap is set, and it would silently change the documented dry-run contract.

This adds an explicit, opt-in policy instead:

{
  "limits": {
    "maxEstimatedCostUsd": 0.25,
    "onUnknownCost": "exclude"   // "allow" (default) | "exclude"
  }
}

• "allow" (default) — preserves current behaviour and the documented contract exactly. No existing deployment changes on upgrade.
• "exclude" — the ceiling becomes genuinely hard: a candidate whose cost cannot be proven under the cap is ineligible.

Unknown-cost exclusions emit a distinct cost-limit-unknown code, so a trace distinguishes known above the cap from cost is unknown — the operator-facing distinction the issue asks for.

Kept deliberately separate from unknownEvidence.cost, which governs how an unknown-cost candidate is scored rather than whether the ceiling applies. A test asserts the two mechanisms stay distinguishable.

File	Change
src/types.ts	OcxRoutingUnknownCostCapMode; onUnknownCost on OcxRoutingProfileLimits
src/routing/profile.ts	Validation ("allow" / "exclude") + normalization
src/routing/evaluator.ts	Cap policy under unknown evidence; cost-limit-unknown exclusion
tests/cost-cap-unknown-evidence.test.ts	New — 5 cases

Verification

The defect was reproduced with a failing test before the fix: with a $0.000001 cap and live-path evidence (no usage), the candidate was still eligible and selected.

Commands run against the dev base:

bun test tests/cost-cap-unknown-evidence.test.ts     → 5 pass, 0 fail
bun test tests/cost-scoring.test.ts \
         tests/routing-profile.test.ts               → 27 pass, 0 fail
bun x tsc --noEmit                                   → clean
bun scripts/test.ts                                  → 10134 pass, 7 skip, 0 fail

New test cases:

1. Repro — live-path evidence carries no estimate, so the hard cap never fires (documents current behaviour)
2. Fail-closed — onUnknownCost: "exclude" excludes the candidate and emits cost-limit-unknown
3. Default unchanged — with no onUnknownCost, behaviour is identical to today
4. Mechanism distinctness — unknownEvidence.cost: "exclude" emits unknown-price, not cost-limit-unknown
5. Inert without a cap — onUnknownCost has no effect when maxEstimatedCostUsd is unset

Coverage on touched files: src/routing/profile.ts 89.8% lines; every added line in src/routing/evaluator.ts covered. No GUI changes, so no screenshot applies.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed. — The new field is documented via JSDoc on OcxRoutingProfileLimits and its validation message. Happy to add a docs entry if you point me at the right file.
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. — No secrets or auth touched. The default is deliberately "allow" so an upgrade cannot silently make every live candidate ineligible; the stricter behaviour is opt-in.

Notes for review

• Default is unchanged, so this is additive and non-breaking.
• Wiring real usage into the live routing path is deliberately not attempted here — that is the larger change Routing: define hard cost-cap behavior when runtime cost evidence is unknown #1181 warns about, and it is separable from defining the policy.
• Happy to rename onUnknownCost, or fold it into unknownEvidence, if you would prefer a single mechanism. I kept them separate because they answer different questions.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[github-actions]

⏳ DRAFT

• wrong target branch (main); retarget to dev.

What to do

• Retarget this PR to dev — all contributions go to dev.
• Tick all four boxes in the PR description once you're done (currently 0/4).

Review readiness checklist

• ⬜ All CI tests are green on my local testing.
• ⬜ I pushed my PR to the latest dev commit.
• ⬜ I resolved all correct Codex and CodeRabbit findings.
• ⬜ My PR is ready for review.

0/4 boxes ticked.

Its title has been prefixed with [WRONG BRANCH].
This pull request was already a draft. Its draft status will be preserved after every issue above is resolved.
@abhisheksharma2411 Tick the boxes once your local CI is green, your branch is on the latest dev commit, and every correct Codex and CodeRabbit finding is resolved.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f05f0179-bc9f-4f58-9145-3dd564a13923

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Changes

Unknown cost cap handling

Layer / File(s)	Summary
Cost policy contract and normalization src/types.ts, src/routing/profile.ts	Adds OcxRoutingUnknownCostCapMode, validates "allow" and "exclude", and preserves onUnknownCost in normalized profiles and revisions.
Unknown estimate evaluation src/routing/evaluator.ts	Keeps known over-limit estimates under cost-limit. Unknown estimates use cost-limit-unknown and become ineligible only when the policy is "exclude".
Regression coverage tests/cost-cap-unknown-evidence.test.ts	Tests default allow behavior, fail-closed behavior, separate scoring policy handling, and behavior without a configured maximum cost.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RoutingProfile
  participant RoutingEvaluator
  participant Candidate
  RoutingProfile->>RoutingEvaluator: Provide normalized cost limits
  RoutingEvaluator->>Candidate: Read estimated cost
  alt Known estimate exceeds cap
    RoutingEvaluator->>Candidate: Exclude with cost-limit
  else Unknown estimate and policy is exclude
    RoutingEvaluator->>Candidate: Exclude with cost-limit-unknown
  else Unknown estimate and policy is allow
    RoutingEvaluator->>Candidate: Keep eligible
  end

Loading

Possibly related PRs

• lidge-jun/opencodex#1011: Introduced the routing profile limits and evaluator extended by this change.
• lidge-jun/opencodex#1015: Added related cost-limit and unknown-cost handling in the routing evaluator.
• lidge-jun/opencodex#1018: Added related routing-profile cost-limit and unknown-evidence policy logic.

Suggested reviewers: ingwannu, lidge-jun, wibias

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The code and tests implement the policy, but no documentation changes show consistent behavior across dry-run, live routing, and logs [#1181].	Update routing documentation and verify that the policy, explanation, and trace code are consistent across dry-run, live routing, and logs.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the routing change for unknown runtime cost evidence and hard cost-cap behavior.
Out of Scope Changes check	✅ Passed	Changes in src/routing/evaluator.ts, src/routing/profile.ts, src/types.ts, and the regression test directly support issue #1181.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/cost-cap-unknown-evidence.test.ts`:
- Around line 1-21: Update the documentation and test names in the cost-cap
behavior tests to reflect the evaluator’s current behavior: unknown cost
evidence is allowed by default, while exclusion is opt-in. Remove obsolete
reproduce-before-fixing text that says the explicit exclusion test fails until a
fix lands, including the header and the statement near the second test, and
rename both tests to clearly describe their default-allow and opt-in-exclusion
behavior.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 72e55f8e-291a-41d9-aa70-52a02f1ae7c9

📥 Commits

Reviewing files that changed from the base of the PR and between 121f1ad and 1862f1c.

📒 Files selected for processing (4)

• src/routing/evaluator.ts
• src/routing/profile.ts
• src/types.ts
• tests/cost-cap-unknown-evidence.test.ts

[coderabbitai]

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Update the obsolete reproduce-before-fixing text.

Lines 19-21 and Line 105 state that the explicit exclusion test fails until a fix lands. The evaluator now implements this behavior, so the test passes. Rename the tests and update the header to describe default allow behavior and opt-in exclusion behavior.

Proposed documentation update

- * Reproduction for issue `#1181` — "Routing: define hard cost-cap behavior when
- * runtime cost evidence is unknown".
+ * Regression coverage for issue `#1181` — "Routing: define hard cost-cap behavior
+ * when runtime cost evidence is unknown".
...
- * Test 1 documents current (fail-open) behaviour and passes today.
- * Test 2 asserts the behaviour the issue asks for and FAILS until the fix
- * lands, per the reproduce-before-fixing rule.
+ * Test 1 verifies the default fail-open policy.
+ * Test 2 verifies opt-in fail-closed behavior.
...
-  test("REPRO: live-path evidence carries no estimate, so the hard cap never fires", async () => {
+  test("default allow keeps an unknown-cost candidate eligible", async () => {
...
-  test("EXPECTED (fails until fixed): fail-closed cap excludes unknown-cost candidates", async () => {
+  test("opt-in fail-closed cap excludes unknown-cost candidates", async () => {

Also applies to: 73-73, 105-105

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/cost-cap-unknown-evidence.test.ts` around lines 1 - 21, Update the
documentation and test names in the cost-cap behavior tests to reflect the
evaluator’s current behavior: unknown cost evidence is allowed by default, while
exclusion is opt-in. Remove obsolete reproduce-before-fixing text that says the
explicit exclusion test fails until a fix lands, including the header and the
statement near the second test, and rename both tests to clearly describe their
default-allow and opt-in-exclusion behavior.

[Wibias]

Malfunctioned PR a swell as to broad. Please open a new smaller PR.