Skip to content

test(queue): the concurrency item #117 left open on issue 103 - #123

Merged
pitimon merged 1 commit into
mainfrom
test/103-compaction-concurrency
Jul 25, 2026
Merged

test(queue): the concurrency item #117 left open on issue 103#123
pitimon merged 1 commit into
mainfrom
test/103-compaction-concurrency

Conversation

@pitimon

@pitimon pitimon commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

The last unticked box on #103: "Compaction is safe against a concurrent append (test with the sync lock held)." #117 shipped the feature and named this as the honest gap.

It is safe because of the lock, not on its own. A test that only asserted the happy path would be claiming a property the code does not have, so both halves are demonstrated.

Four tests

1. The hazard is real, and shown to be real. Patching writeFileSync to append to the queue in the window between the temp write and the rename, the appended row is lost — it was on the old inode and went with it.

If that test ever starts failing because the append survives, compaction grew a merge step and the lock requirement should be re-examined. That is a finding, not a broken test, and the test says so.

2. The lock genuinely excludes a second holder, and releases. Without that, "compaction runs inside the lock" would guarantee nothing.

3. The positive case in the arrangement sync actually uses — lock, compact, release — with a would-be concurrent writer refused mid-operation.

4. A source fact pinning the order in sync.js. Reordering those lines would leave the other three green while reopening the window, which is exactly the regression the other three cannot see.

Test plan

  • ci:local exit 0 — 956 root tests (+4), 302 dashboard
  • The loss test fails loudly if compaction ever starts merging, with the reason in the assertion message

After this

#103's definition of done is complete, with two deviations already recorded on the issue: automatic compaction is deliberately not shipped (it rewrote an 11 MB production queue during npm test), and the doctor check is a warn rather than a fail because the rows are already on disk and being rendered.

The last unticked box: "Compaction is safe against a concurrent append (test
with the sync lock held)."

It is safe BECAUSE of the lock, not on its own, so a test that only asserted the
happy path would be claiming a property the code does not have. Both halves are
demonstrated instead.

First, the hazard is real and shown to be real: patching writeFileSync to append
to the queue in the window between the temp write and the rename, the appended
row IS lost. It was on the old inode and went with it. If that test ever starts
failing because the append survives, compaction grew a merge step and the lock
requirement should be re-examined — a finding, not a broken test.

Second, the lock genuinely excludes a second holder, and releases. Without that
"compaction runs inside the lock" would guarantee nothing.

Third, the positive case in the arrangement sync actually uses — lock, compact,
release — with a would-be concurrent writer refused mid-operation.

Fourth, a source fact pinning the ORDER in sync.js. Reordering those lines would
leave the other three green while reopening the window, which is exactly the
kind of regression the other three cannot see.

ci:local exit 0: 956 root tests, 302 dashboard.
@pitimon
pitimon merged commit f4621fe into main Jul 25, 2026
1 check passed
@pitimon
pitimon deleted the test/103-compaction-concurrency branch July 25, 2026 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant