A store already over the consolidation cap is never recovered — it just skips forever

[fdaviddpt]

Follow-up to #346 / #347. That PR stops a store from getting into this state and stops an oversized file freezing the session. It does not get anybody out of it, and the reporter of #346 is already in it.

The state

Once recent.md alone exceeds consolidate_max_bytes (default 600000), every consolidation round assembles an oversized prompt and skips. _rotate_archive is the existing escape hatch and it only ever touches archive.md (pipeline/shell.py:313), so it cannot help when recent.md is the bulk — which is exactly #346's shape.

After #347 that store: starts sessions fine, names recent.md as too large to inject, and consolidates never again. Nothing shrinks it. Staging today-*.md files are never retired either, because retirement happens after a successful round — so they accumulate for as long as the condition lasts.

The only recovery available to the #346 reporter was:

mv state/recent.md state/recent.md.bak && touch state/recent.md

which discards every byte of history in the file. That is the current answer, and it is not an acceptable one for a plugin whose whole job is not losing memory.

The shape of a fix

archive.md already has the pattern: rotate to a dated sibling, start a fresh empty one, keep the bytes on disk and reachable by grep (#123/#124 named the rotated files at session start so that slice stops being invisible). The same move for recent.md — rotate to recent-YYYY-MM-DD.md, start fresh — would heal the store, lose nothing, and let consolidation resume the next round.

Open questions worth settling before building it, not assumptions to code around:

• Rotating recent.md retires a span that was never consolidated into archive.md. Is a dated recent-*.md sibling the right resting place for it, or should it be re-fed as staging so a later round can compress it properly?
• Should this fire automatically on the skip path, or only when a human asks (a /remember:doctor repair, say)? Automatic is what actually helps someone who has no idea the plugin has stopped working; manual is the conservative direction for a file the user's memory lives in.
• The session-start notice from fix: cap the consolidation response, and stop reading a store too large to read (#346) #347 currently says "run /remember:doctor". doctor.sh should have something to say about an oversized memory file, and today it does not.

Not this issue

now.md / today-*.md unbounded growth is separate and filed on its own.