fix(engine): load_slot rolls back session logs (F08-1) + lookup_lore reachable for natural queries (F10-1) - #849
Conversation
… natural queries (F08-1, F10-1) F08-1 (issue #779): load_slot rolled back only snapshot.json, leaving a discarded timeline (an undone TPK, a post-slot orphan session) permanently canon in read_log_all / recap — the DM's lean-beat memory replayed events the player undid. save_slot now captures a SESSION-LOG manifest (each sessions/*.jsonl byte length at slot time) into a sidecar (slots/.manifests/<slot>.json, invisible to list_slots' non-recursive slots/*.json glob). load_slot reconciles the logs to that manifest AFTER restoring the snapshot, under the same campaign_lock: post-slot orphan sessions are archived whole, sessions that grew past their slot-time length are truncated back (archiving the discarded tail) under sessions/rolled-back-<ts>/ — invisible to read_log_all's non-recursive *.jsonl glob, so recap/lean-memory match the rolled-back snapshot. The discarded timeline is ARCHIVED (recoverable), never deleted. Manifest-less (legacy) slots degrade to today (snapshot restored, logs untouched); a session SHORTER than its manifest length is left as-is (never padded/raised). Truncation is at a newline boundary (append_log writes entry+"\n"), so neither the head nor the archived tail is ever a torn JSONL line. Additive: no model field added; old snapshots/slots round-trip; engine stays sole writer (atomic temp+os.replace). F10-1 (issue #782): lookup_lore buried the dedicated lore page for natural queries. (a) _safe_match dropped sub-2-char tokens (unless that empties the match) — the 1-char "s" of a possessive query ("Wyrm's Crossing") matched every page's apostrophe-s and buried the dedicated page. (b) Tier-0 authored-canon precedence gained a NOISE FLOOR: an authored page keeps absolute precedence over the 351-page wiki tier only if its FTS |rank| is within a small fraction (0.01) of the best match overall — so a stopword-only match ("the Counting House" → authored pages matching only "the" at |rank|≈2e-6, six orders of magnitude below the genuine wiki hit) is demoted below the wiki tier (kept as a fallback, never dropped) while genuine authored matches still lead. The floor is relative (not a fixed epsilon) because bm25 magnitudes are corpus-size-dependent. Clean queries leave the weak set empty → ordering reduces to today's tier-0-then-tier-1 (byte-identical; verified 0/18 clean-query outputs changed on the shipped corpus). Tests (red-first): - test_slots.py +12: grown-session truncate, orphan-session archive, tail archive, manifest-less degrade, shorter-than-manifest leave-as-is, snapshot byte-identical, manifest-not-listed, no-sessions-dir safe, end-to-end tool rollback. - test_lorebook.py +7: possessive token-drop, stopword noise-floor (synthetic + real corpus repros), genuine-tie still wins authored, clean-query byte-identity. Full engine suite 2097 passed; fast_gate Tier-0 PASS. Closes #779 Closes #782 Source: docs/audits/ENGINE-AUDIT-2026-06-11.md (F08-1, F10-1)
|
Warning Review limit reached
More reviews will be available in 57 minutes and 46 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 665370930d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if cur_len <= kept_len: | ||
| continue # unchanged or shorter — leave as-is (never pad/raise) |
There was a problem hiding this comment.
When a player saves slot A, keeps playing and saves slot B, then loads A, this rollback truncates the live session file and archives B's tail. If they later load B, the file is now shorter than B's manifest, and this branch silently leaves the shortened log instead of restoring the saved bytes. The snapshot returns to B, but read_log_all/session recap lose all narration between A and B, so named save slots are not actually restorable after an earlier rollback; the slot needs to store/restore log content (or revive the archived tail) rather than only byte lengths.
Useful? React with 👍 / 👎.
1 participant
Root cause
F08-1 (#779) — load_slot rolls back the snapshot but NOT the session logs.
save_slotcopied onlysnapshot.json;load_slotwrote only the snapshot back. The append-onlysessions/*.jsonllogs were never reconciled, so a discarded timeline — an undone TPK appended to the same session, or a whole post-slot orphan session — stayed canon inread_log_allandrecap_from_store, the DM's lean-beat memory. The #749/#763 decontamination filtered only exact-match wrapper-heartbeat lines; rolled-back story prose passed straight through. Permanent, silent corruption (two timelines interleaved under the stablet-sort).F10-1 (#782) — lookup_lore buries the dedicated lore page for natural queries.
Two classes of natural query couldn't reach the dedicated page in the 356-page corpus:
"Wyrm's Crossing"tokenized to["Wyrm","s","Crossing"]; the bare 1-char"s"matched every page's apostrophe-s and dragged unrelated pages abovewyrm-s-crossing.md."the Counting House"— three authored (tier-0) pages matched ONLY the stopword"the"at noise rank (|rank| ≈ 2e-6), yet tier-0's absolute precedence let them fill the cap and bury the genuinely-matchingcounting-house-baldur-s-gate.md(a tier-1 wiki page at |rank| ≈ 8.97).Fix
F08-1 (
store.py):save_slotwrites a session-log manifest ({filename: byte_length}) to a sidecarslots/.manifests/<slot>.json(a dedicated subdir, invisible tolist_slots' non-recursiveslots/*.jsonglob).load_slotreconciles the logs to that manifest AFTER restoring the snapshot, under the samecampaign_lock:Discarded entries are archived under
sessions/rolled-back-<ts>/(recoverable, and invisible toread_log_all's non-recursive*.jsonlglob), never deleted. Truncation lands on a newline boundary (append_logwritesentry+"\n"), so neither head nor archived tail is ever a torn JSONL line.F10-1 (
lorebook.py):_safe_matchdrops sub-2-char tokens (unless that empties the match → keeps them so a degenerate query still searches).0.01 × best_match_overall. Noise-rank stopword matches (≈6 orders of magnitude below a genuine hit) are DEMOTED below the wiki tier (kept as a fallback, never dropped); genuine authored matches still lead. The floor is relative, not a fixed epsilon, because bm25 magnitudes are corpus-size-dependent (a tiny corpus ranks every match ≈2e-6).Invariants
_atomic_writeor temp+fsync+os.replace, under the caller'scampaign_lock.clawdnd-*ids, orCLAWDND_*env touched.Tests (red-first, all green)
test_slots.py+12: grown-session truncate, orphan-session archive, tail archive, manifest-less degrade, shorter-than-manifest leave-as-is, snapshot byte-identical, manifest-not-listed, no-sessions-dir safe, end-to-end tool rollback.test_lorebook.py+7:_safe_matchtoken-drop, possessive + stopword reachability (synthetic + real-corpus repros), genuine-tie still wins authored, clean-query byte-identity (0/18 shipped clean queries changed).Verification
qa/fast_gate.shTier-0: PASS (188 deterministic).read_log_all/recap; after → neither.wyrm-s-crossing.mdat Epic 2: Characters & progression #3; stopword →counting-house-baldur-s-gate.mdat Epic 2: Characters & progression #3 with genuinebaldurs-gate.md/factions.mdstill leading.Closes #779
Closes #782
Source: docs/audits/ENGINE-AUDIT-2026-06-11.md (F08-1, F10-1)