Skip to content

fix(context): compute require_citations gate after the max_chars budget#175

Open
seekmistar01 wants to merge 1 commit into
vouchdev:mainfrom
seekmistar01:fix/context-citation-gate-after-budget
Open

fix(context): compute require_citations gate after the max_chars budget#175
seekmistar01 wants to merge 1 commit into
vouchdev:mainfrom
seekmistar01:fix/context-citation-gate-after-budget

Conversation

@seekmistar01

Copy link
Copy Markdown

Summary

Closes #174

build_context_pack computed the uncited claim list (and thus the require_citations gate failure and the uncited_items reported in quality) over the full item list, then the max_chars budget popped tail items. So a ContextPack could come back ok=False with uncited_items=[...] naming claims that aren't present in pack["items"] — failing the pack for citation reasons driven by items the consumer never received.

Change

  • src/vouch/context.py — move the require_citations / uncited computation to after the max_chars budget step, so it only considers the items actually returned.
  • tests/test_context.pytest_require_citations_only_considers_returned_items: with max_chars + require_citations, asserts every id in quality.uncited_items is in the returned items.

Why it's safe

  • Behavior is unchanged when there is no max_chars budget (nothing is popped, so pre- vs post-budget computation is identical).
  • When a budget drops items, the gate now correctly reflects only the returned pack.

Verification

  • pytest tests/test_context.py6 passed (existing + the new regression test). ruff check clean.

`build_context_pack` computed the `uncited` claim list (and therefore the
`require_citations` gate failure and the `uncited_items` reported in
`quality`) over the full item list, then the `max_chars` budget popped tail
items. So a ContextPack could be returned `ok=False` with
`uncited_items=[...]` naming claims that are not present in `pack["items"]`
at all — the consumer is told the pack failed citation requirements because
of items it never received.

Move the citation-gate computation to after the budget step so it only
considers the items actually returned. Add a regression test asserting
`uncited_items` is a subset of the returned items when `max_chars` and
`require_citations` are combined.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seekmistar01, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 55 minutes and 4 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4035a033-67a4-47b9-8d48-0cf27feb8ad7

📥 Commits

Reviewing files that changed from the base of the PR and between 3beb821 and 4b5f8cc.

📒 Files selected for processing (2)
  • src/vouch/context.py
  • tests/test_context.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@plind-junior

Copy link
Copy Markdown
Collaborator

Review

Summary: The PR correctly moves the require_citations gate to after the max_chars budget step, fixing the root cause described in issue #174, and adds a focused regression test. The change is minimal, safe, and well-reasoned.

What works

  • src/vouch/context.py:119–121 — Moving the uncited computation to a list-comprehension after items.pop() is the right fix and the comment explains the "why" clearly.
  • src/vouch/context.py:130–133 — The downstream if uncited: / failed.append("require_citations") block is untouched and works correctly with the new placement; uncited is now always populated after budget trimming.
  • tests/test_context.py:41–58 — The regression test directly asserts the invariant (uncited_items ⊆ returned), pins a max_chars=80 budget tight enough to force drops, and guards against a vacuous pass with the assert pack["quality"]["uncited_items"] pre-condition check.

Suggestions

  • [non-blocking] src/vouch/context.py:104 — The uncited: list[str] = [] initialisation on line 104 is now dead (it gets replaced unconditionally on line 121 whenever require_citations is True, and the variable is never read before that point). Consider removing the initialiser or inlining it as uncited: list[str] to avoid the misleading empty-list default. It doesn't affect correctness, but it leaves a stale artefact from the old flow.
  • [non-blocking] tests/test_context.py:53–54max_chars=80 with 20 claims of ~45-char summaries reliably forces drops today, but it depends on the MockEmbedder returning all 20 claims in the result set. If the embedding mock or the default limit=10 changes, the test could silently become vacuous (the pre-condition assert on line 57 would catch it, but only at runtime). Consider passing limit=20 explicitly so the intent is self-documenting and the test stays robust to future limit default changes.

Verdict

Approve — Minimal, correct fix for the exact issue described; regression test is in place; no out-of-scope changes.

@plind-junior

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 4b5f8ccd7e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ContextPack require_citations gate fails on / reports items dropped by the max_chars budget

2 participants