recent.md and archive.md grow unbounded — hit 6.4GB/1.8GB, froze sessions

[gsa1571]

What happened

state/recent.md and state/archive.md in a project's memory folder grew unbounded over months of daily use:

• recent.md — 6.4 GB
• archive.md — 1.8 GB (present in two separate state folders for the same logical project — see note below)

The SessionStart hook cats both files into every new session's context. Once they hit multi-GB size, every claude launch in that project froze. On macOS this also caused iTerm2's memory usage to balloon to ~56GB, forcing a full system restart to recover.

Root cause (as far as I could tell)

Unlike now.md / today-*.md / core-memories.md, which appear to get rotated per the plugin's own tiered-history scheme, recent.md and archive.md have no size cap or rotation — they only ever get appended to. Over enough sessions/days, that's unbounded growth with no ceiling.

Fix I applied (workaround, not a real fix)

For each bloated file:

mv state/recent.md state/recent.md.bak
touch state/recent.md
# confirmed `claude -p 'ping'` worked again
rm state/recent.md.bak   # once confirmed working

This recovers the session but discards all history that was in the file — there's no way to salvage/re-compress the old content back down to a reasonable size with the tools I had available.

Secondary issue noticed while debugging

Two separate state/ folders exist for what is logically the same project, because a __ in the real directory name (__XX) gets encoded differently across sessions/worktrees:

• ~/.claude/projects/-Users-sergii-Documents-WORK-PHP---XX-autoparts/state/
• ~/.claude/projects/-Users-sergii-Documents-WORK-PHP-__XX-autoparts-worktrees-ai-helper/state/

Both had their own independently-bloated archive.md. Not sure if that's expected (different physical worktree paths) or a slug-encoding bug, flagging it in case it's related.

Suggested fix

Some form of size cap / rotation on recent.md and archive.md, similar to what already exists for the tiered daily logs — e.g. roll over to an archive-of-archives once a size threshold is hit, rather than appending forever.

Environment

• macOS (Darwin 25.5.0)
• Claude Code 2.1.221 → 2.1.228 (updated during troubleshooting, bug present on both)
• Plugin: remember@claude-plugins-official

[fdaviddpt]

Fixed — merged to main in #347. v0.20.0 is cut and waiting on the default-branch CI run; I will post here again when the release is tagged.

First: thank you. This was an unusually good report — you gave the sizes, the platform, both plugin versions you reproduced on, the recovery you used, and a secondary observation you weren't sure about. That is more than enough to work from, and the fact that you'd already lost the contents of those files and still took the time to write it up is appreciated.

Your observation was right and the mechanism was inside out

You wrote that recent.md and archive.md "only ever get appended to". Nothing appends to them. The only writer in the tree is cp "$RECENT_OUT" "$RECENT_FILE" in run-consolidation.sh, which replaces the file wholesale with the consolidation model's output, and it has done that since the first commit.

But what you observed — a file that only grows and never shrinks — was exactly correct, and here is how it gets there:

1. Consolidation refuses to send a prompt over consolidate_max_bytes (default 600000). It never counted the bytes it wrote back. capture_output=True bounds that subprocess by a wall clock, not by a size, so the length of a response was not a quantity anything downstream measured. On a stub we reproduced one round writing 52 MB — 87× the cap the same function had refused to send seconds earlier.
2. recent.md is itself part of the input that cap is measured on. So the very next round assembles an oversized prompt, raises, and skips — and so does every round after it, forever. Rotating archive.md is no escape when the bulk is recent.md.

Never grows again, never shrinks. From the outside that is indistinguishable from append-only, which is why your reading was reasonable — and why nobody had caught it: the thing that broke the store also disabled the only mechanism that could repair it.

What shipped

• The same number now caps both directions: a response over consolidate_max_bytes is refused and the round skips, leaving staging and memory intact so the next one retries.
• The store is sized before it is read. Previously a 6.4 GB recent.md had to be read into memory and a prompt built around it before the pipeline was allowed to notice it was too large to send — from a script that runs disowned next to your live session. That is the allocation that took your machine down.
• A memory file over thresholds.memory_inject_max_bytes (new, default 200000) is named with its size instead of cat'd at session start. None of the above helps a file that is already oversized on disk, and that file is what froze every launch. A session that starts and tells you the store is broken beats one that hangs.

Still open, and it is the part that affects you specifically

#348 — if you still had that store, v0.20.0 would give you working sessions and a consolidation that never runs again. The bytes stay safe, but nothing heals it, and mv recent.md && touch remains the only cure. archive.md already has a rotate-to-a-dated-sibling path; recent.md needs the same, and there are real design questions in it (where does an unconsolidated span belong, and should it self-repair or wait to be asked). Comments welcome there.

Your secondary observation — two state/ folders because __XX slugs differently across sessions and worktrees — is a real question and I have not touched it. If it is still reproducible for you, a separate issue with both paths would be useful; slug encoding has bitten this plugin before (#157/#158/#174/#263).

Unrelated, in case it is useful

The investigation behind this was done with supertool — our other open-source plugin. It batches file reads, greps, git and GitHub state into single calls, and validates edits with rollback on a syntax failure. It is what made tracing one writer across scripts/, pipeline/ and hooks.d/ cheap enough to do properly rather than guessing. Same marketplace, same license. No obligation — just adjacent.

[fdaviddpt]

Released: v0.20.0.

One caveat on getting it, since a tag is not the same as delivery. If you installed from the Anthropic plugin marketplace, the catalogue pins this plugin by commit sha and that pin still points at the 0.19.0 commit right now — it moves when their bumper next runs, which I have seen land anywhere from about an hour to fourteen hours after a release. FORCE_AUTOUPDATE_PLUGINS=1 on a session start will not shorten that: nothing on your side is stale, the input is. Installing directly from this repo gets it immediately.

Worth checking once it does land — the version a plugin reports and the directory it lives in have disagreed before, so read .claude-plugin/plugin.json inside the install rather than trusting the path.