Add scratch-file offload for elided tool output (ports OpenDev's truncation.rs pattern)#6
Merged
Merged
Conversation
Ports OpenDev's truncation.rs recovery pattern (crates/opendev-tools-impl): before compress() elides the middle of oversized output, save the full original to ~/.claude/comb/tool-output/ and embed the path in the marker, so elision is recoverable instead of permanent data loss. - saveOverflow(): writes full text, never throws, caps at 1MB with head75/tail25 split (not a flat cutoff) if exceeded, matching OpenDev's save_overflow behavior - cleanupOldFiles(): 7-day retention sweep, exported but not auto-invoked on the hot path (same reasoning as OpenDev's cleanup_old_files) - COMB_COMPRESS_SAVE_FULL=0 opt-out (defaults on, since silently losing data is worse than a stray file on disk) - 9 new tests in test/overflow.test.js covering the write path, the 1MB cap, the opt-out, and cleanup; all 19 pre-existing tests still pass unmodified Found while reviewing OpenDev's compaction/truncation source for comb's design - see crates/opendev-tools-impl/src/truncation.rs upstream.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ports a pattern from OpenDev's
truncation.rs(opendev-to/opendev,crates/opendev-tools-impl/src/truncation.rs) into comb: before eliding the middle of oversized tool output, save the full original to disk so it's recoverable.Why
comb's
compress()currently elides the middle of oversized output permanently — there was no way to get it back if it turned out to matter. OpenDev solves the same problem in its own tool-output truncation path by always writing the full output to a scratch file first and returning a path alongside the truncated preview.What changed
saveOverflow(text): writes the full untouched text to~/.claude/comb/tool-output/tool_<timestamp>_<8-hex>, mode0o600. Never throws — on any FS error it returnsnullandcompress()just omits the recovery hint; the file write is best-effort, the compression itself is not.MAX_OVERFLOW_BYTES), matching OpenDev's constant. Past the cap, the saved file is itself capped head 75% / tail 25%, not a flat cutoff — so both how the output started and how it ended survive even in the worst case.compress()'s marker now includes, full output: <path>when a save succeeded.COMB_COMPRESS_SAVE_FULL=0opt-out. Defaults on (opt-out rather than opt-in) since silently losing tool output seems worse than an occasional stray file on disk.cleanupOldFiles(): 7-day retention sweep. Exported but not auto-invoked by the hook itself — PostToolUse runs in the hot path on every matched tool call, so a directory scan there is wasted work every time. Meant to be wired into aSessionStarthook or cron if disk growth inOVERFLOW_DIRbecomes a problem.Testing
test/compress.test.jspass unmodified — nothing about the existing elision/salvage/gate/rule-store behavior changed.test/overflow.test.js:saveOverflowwrites the exact full text and returns a pathcompress()'s marker contains a working path, and the saved file matches the original byte-for-byteCOMB_COMPRESS_SAVE_FULL=0writes nothing and omits the path from the markercleanupOldFiles()keeps recent files, removes files past retention (verified viafs.utimesSync, not a real 7-day wait), leaves non-tool_*files alone, and returns0gracefully when the directory doesn't exist yetBashtool_response, confirmed the returned JSON'supdatedToolOutput.outputcontains a working path and the file on disk matches the pre-elision original exactly.What I deliberately did NOT port from OpenDev
HEAD_CHARS/TAIL_CHARS/THRESHOLDare already purely character-based, and OpenDev's dual line+byte trigger wasn't something comb's actual design was missing; would've been scope creep.COMB_COMPRESS_SAVE_FULLopt-out (vs. no toggle at all in OpenDev) since comb already has a convention of explicit env-var toggles for anything that touches the filesystem/network (seeCOMB_RULE_STORE_URL).