fix(tracking): reclaim history.db pages after retention cleanup - #3479
Open
imkp1 wants to merge 1 commit into
Open
fix(tracking): reclaim history.db pages after retention cleanup#3479imkp1 wants to merge 1 commit into
imkp1 wants to merge 1 commit into
Conversation
cleanup_old() deleted rows past the 90-day window but nothing returned the freed pages, so history.db stayed at its high-water mark for good. Retention bounded the row count, not the file size. New databases are now created with auto_vacuum=INCREMENTAL and give the pages back after every cleanup that removed rows. A database created before this change is converted once, by the first cleanup that frees pages, and reclaims its accumulated slack at the same time. Reclamation is best-effort: a database that cannot be shrunk right now must never fail the command being recorded.
imkp1
marked this pull request as ready for review
August 8, 2026 09:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cleanup_old()deleted rows past the 90-day window but nothing returned the freed pages, sohistory.dbstayed at its high-water mark for good. Retention bounded the row count, not the file size.auto_vacuum=INCREMENTALand hand the pages back after every cleanup that actually removed rows. The pragma has to be set beforejournal_mode=WAL, because switching to WAL writes the header and a database can no longer change its vacuum mode after that.needs_full_vacuumis what keeps that a one-time event.reset_all()reclaims too. Resetting stats to zero should return the space, not just empty the tables.There is no "vacuum once the freelist passes X%" rule here. Reclaiming is driven by whether a delete removed rows, and the work
incremental_vacuumdoes is proportional to the pages that delete freed.The two secondary query items at the end of the issue (SQL-side
SUM/COUNTinget_summary_filtered, and the parameter-dependentORthat stops the project-scope filter using its index) are left out on purpose. They are a separate change and this repo asks for one focused fix per PR. Happy to follow up with them.Test plan
cargo fmt --all --check && cargo clippy --all-targets && cargo testFour new unit tests in
src/core/tracking.rs: a new database reportsauto_vacuum=2; cleanup leaves an empty freelist and a smaller page count; a database built without auto_vacuum is converted and shrunk on its first cleanup;reset_allreclaims. They assert onauto_vacuum,freelist_countand before/afterpage_countrather than on bytes or milliseconds, since in WAL mode the on-disk size only follows a checkpoint.Manual, against a database with 200k rows all past the retention window:
Manual worst case for the one-time conversion, 100k rows still inside the window so the vacuum has to rewrite all of them: the run that converts takes 145 ms, the runs after it 32 ms. It happens once per database.
Steady state, same 100k-row database, 60 interleaved runs of each binary: 8.31 ms median before this change, 8.16 ms after. When nothing expires,
cleanup_olddeletes 0 rows and does no extra work at all.Steady state for a user past the window, where rows expire on almost every insert: 100k rows spread across the last 91 days, 50 interleaved runs each, 8.07 ms median before and 7.88 ms after.
incremental_vacuumonly touches the pages that delete just freed, so this stays a handful of pages per command.Closes #3393
🤖 Opened with Superhuman, an open-source contribution agent.