feat(consider): a non-recoverable refusal still frames the decision from frozen facts (closes #674) - #697
Merged
Merged
Conversation
…rom frozen facts (closes #674) When a recorded book exists but consider cannot produce a consequence for a genuinely non-recoverable reason (structural corruption, an integrity warning this route cannot scope, or no usable holding left -- the residue after #673's per-holding exclusions), the CLI now attaches a bounded usable_facts packet to the error payload: the last finalized review's own concentration reading and committed rule, copied verbatim from last_state.json, never recomputed. The recoverable single-holding refusal (#673) is untouched -- this leaf only wraps the three genuinely whole-book-fatal raise sites in _consider_rows. references/trade-consequence.md gains "When the whole book refuses", the agent-facing contract: lead with a decision tension (never the raw error or a request to restart), cite only usable_facts fields, frame at least two of the user's own nominated options, never name one to sell, and fall back to decision-framing.md's no-book contract when the packet is null. evals/run_episodes.py gains a usable_facts_grounding check (narrower than number_provenance -- it grades against the bounded packet, not the whole plan) and EP-010 replays the scene end to end. Both mutation-tested by hand (cp backup/restore): forcing _usable_facts_snapshot to return None turns two test_consider.py assertions red; neutering the new checker turns EP-010's recorded_miss from an expected fail into a false pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
considercannot produce a consequence for a genuinely non-recoverable reason (structural corruption, an integrity warning this route cannot scope to one holding, or no usable holding left once [defect·P1] One unmatched sell anywhere in the history disables consider for the whole account — the review lane discloses the same condition and continues #673's exclusions run), the CLI now attaches a boundedusable_factspacket to the error payload instead of leaving the agent with only a bare message to narrate.usable_factsis never computed by this call. It is copied, filtered, from the last finalized review's frozenlast_state.json:concentration(max_pos_pct/max_pos_ticker/ai_pct/max_sector_pct/top3_pct) andcommitment(rule/metric_key/metric_value/goal). Either half is omitted when the last review never froze it; the whole field isnullwhen nothing was ever finalized in this root, or nothing usable survived — the caller then falls back toreferences/decision-framing.md's no-book contract rather than inventing a portfolio claim.references/trade-consequence.mdgains "When the whole book refuses ([design·P1] When consider cannot compute, no contract owns the answer — so the agent narrates the gap instead of framing the decision #674)": the agent-facing contract. First visible sentence is a decision tension (never the raw error, never a restart request); frame at least two of the user's own nominated options; cite only fields present inusable_facts; never name which security to sell; state once that the computed consequence itself is unavailable.evals/run_episodes.pygains ausable_facts_groundingcheck — deliberately narrower thannumber_provenance(it grades against the bounded packet, not the whole plan) — andEP-010-non-recoverable-refusal-narrated-instead-of-framed.jsonreplays the scene end to end (recorded-miss fails on an ungrounded number + zero framed options; repaired answer passes).#673's now-recoverable single-holding refusal is untouched: this leaf wraps only the three genuinely whole-book-fatal raise sites inside_consider_rows, never the per-holding refusalconsequence.consequenceraises later.Files changed
skills/fomo-kernel/engine/review.py—ReviewErrorgains an optionalpayload_extrakwarg merged intomain()'s emitted JSON; newCONSIDER_REFUSAL_CONCENTRATION_KEYS/CONSIDER_REFUSAL_COMMITMENT_KEYSconstants; new_usable_facts_snapshot(root); the ledger-basis resolution block in_consider_rowsis wrapped so everyReviewErrorraised inside it (structural corruption / unscopable warning / no usable holding) carries the packet.skills/fomo-kernel/references/trade-consequence.md— new section "When the whole book refuses ([design·P1] When consider cannot compute, no contract owns the answer — so the agent narrates the gap instead of framing the decision #674)".evals/run_episodes.py— newusable_facts_groundingcheck, its own bounded numbers allow-set derived per-episode from the replayed plan'smetrics, mirrored constant (locked by a drift test, not a shared import —review.pypulls in the rest of the engine by bare sibling import).evals/episodes/episode.schema.json—usable_facts_groundingadded to thechecksenum.evals/episodes/EP-010-non-recoverable-refusal-narrated-instead-of-framed.json— new episode.tests/test_consider.py— five new tests: the packet is attached on the "no usable holding" and the "structural corruption" refusals when a prior review froze one;usable_facts: nullwhen nothing was ever finalized, and when a finalized state froze nothing usable; the recoverable single-holding refusal carries nousable_factskey at all.tests/test_episode_checkers.py— unit probes for the new checker (grounding accepts both the record's own scale and its ×100 percent form; catches a number the engine computed elsewhere in the plan that is not part of this refusal's packet; catches fewer than two framed options) plus a drift test locking the mirrored constant toreview.py's.docs/maintainer-guide.md— new mirrored-surfaces row naming every partner file.Proof (per the maintainer's disposition)
tests/test_consider.py::test_a_no_usable_holding_refusal_carries_the_last_reviews_usable_factsand::test_a_structural_refusal_also_carries_usable_factsseedlast_state.json(the same flat shapesession._project_legacy_lockedactually writes) then force each of two CLI-reachable non-recoverable shapes via a realledger.jsonl, asserting the exactusable_factspacket.references/trade-consequence.md's new section;EP-010'srepairedanswer is a worked example (checked mechanically for grounding + option count) and itsrecorded_missdemonstrates the failure mode.cpbackup + restore (nevergit checkout),__pycache__cleared, import-checked before and after:_usable_facts_snapshottoreturn Noneunconditionally:python3 tests/test_consider.py→FAIL test_a_no_usable_holding_refusal_carries_the_last_reviews_usable_facts: None/FAIL test_a_structural_refusal_also_carries_usable_facts: None,125/127 passed(the three tests expectingnull/absence correctly stayed green). Restored, re-verified127/127 passed.check_usable_facts_groundingtoreturn []unconditionally:python3 evals/run_episodes.py EP-010→FAIL EP-010/recorded_miss: expected fail, got pass — no findings;python3 tests/test_episode_checkers.py→AssertionError: []ontest_usable_facts_grounding_catches_a_number_the_engine_computed_elsewhere_in_the_plan. Restored, re-verified full green.python3 tests/run_all.py→PASS: all 49 suites passed.Adjacent findings (not folded in)
_canonical_consider_beforehas its own defensive-floor whole-book refusal ("no holding left to size this answer against once [...] is excluded") — a fourth non-recoverable shape this leaf does not wrap. Likely unreachable in production (it fires only once_consider_rowsalready succeeded), but a candidate for a follow-up if it ever proves reachable.evals/EVALS.md's changelog table was not extended with a new dated entry — it reads as historical narrative rather than an enforced mirror, and nothing in the suite reads it.b0bd0a8(this branch's base) toc217228(fix(qa): the preview receipt is the settled pre-commitment card, not an intermediate render (closes #663) #696) during this session — no file overlap with this change.🤖 Generated with Claude Code