Skip to content

[ISSUE #10756] Fix duplicate dispatch ConsumeQueueExt leak - #10759

Open
ai-yang wants to merge 1 commit into
apache:developfrom
ai-yang:agent/fix-cqext-duplicate-dispatch
Open

[ISSUE #10756] Fix duplicate dispatch ConsumeQueueExt leak#10759
ai-yang wants to merge 1 commit into
apache:developfrom
ai-yang:agent/fix-cqext-duplicate-dispatch

Conversation

@ai-yang

@ai-yang ai-yang commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Which Issue(s) This PR Fixes

Fixes #10756

Brief Description

ConsumeQueue.putMessagePositionInfoWrapper() allocated a ConsumeQueueExt unit before the main CQ idempotency check. Replaying a duplicate dispatch therefore skipped the CQ append but still left an unreferenced Ext unit, which a later valid dispatch could seal inside the live Ext range.

This change reuses the main CQ physical-end idempotency predicate before allocating an Ext unit. Duplicate replay still follows the existing successful wrapper path, so checkpoint advancement and multi-dispatch behavior are preserved while the orphan allocation is avoided.

How Did You Test This Change?

  • Unmodified develop: the deterministic duplicate-dispatch regression failed in 5/5 isolated JDK 8 Maven processes.
  • Fixed targeted regression: 20 isolated Maven/JVM processes, 1/1 each (20/20 total).
  • Complete ConsumeQueueTest: 11/11.
  • Full store -am test: common 241/241, remoting 174/174, and store 314 tests with 4 skips, 0 failures, and 0 errors.
  • Maven validate and Checkstyle: 0 violations.
  • SpotBugs: 0 bugs/errors across all four reactor modules.
  • git diff --check: passed.

Scope and Concurrency

The new guard reuses the physical-end idempotency predicate already applied by putMessagePositionInfo() on the same dispatch attempt. It introduces no new shared state or locking and leaves supported writes, checkpoint advancement, retries, and multi-dispatch handling unchanged. This PR is scoped to duplicate re-dispatch; cleanup after a genuine Ext-append/CQ-append failure remains a separate failure mode.

@ai-yang
ai-yang marked this pull request as ready for review August 2, 2026 14:23

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by github-manager-bot

Summary

Prevents ConsumeQueueExt orphan entries when duplicate dispatch requests arrive by extracting isDispatchAlreadyApplied() and applying it consistently in both the ext-write guard and the main dispatch path.

Findings

  • [Info] ConsumeQueue.java:730 — The new isDispatchAlreadyApplied() extraction is clean and ensures both putMessagePositionInfoWrapper and putMessagePositionInfo use the same duplicate check. Good consolidation.
  • [Info] ConsumeQueue.java:901 — The helper method is well-named and the semantics are clear (offset + size <= maxPhysicOffset means the dispatch was already applied).
  • [Info] ConsumeQueueTest.java — Test testDuplicateDispatchDoesNotLeaveConsumeQueueExtOrphan directly verifies the fix scenario with ext address validation after reload. Solid coverage.

Suggestions

  • Consider adding a brief Javadoc on isDispatchAlreadyApplied() explaining that it detects duplicate dispatch after broker crash recovery, to help future readers understand why the check matters in both call sites.

Automated review by github-manager-bot

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Fix for ConsumeQueueExt orphan entries during duplicate dispatch — extracts isDispatchAlreadyApplied() for consistent use in both the ext-write guard and the main dispatch path. Test coverage is solid.

The previous suggestion about Javadoc was informational only. The code is correct and the fix is well-targeted. LGTM.


Automated review by github-manager-bot

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Defensive fix with proper validation and test coverage. LGTM.


Automated review by github-manager-bot

@ai-yang

ai-yang commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

@RongtongJin @guyinyou, could you please take a human review when convenient? This is a focused Store/ConsumeQueueExt fix that prevents duplicate dispatch from allocating an orphan Ext unit. The regression reproduces the old behavior deterministically; the complete Store reactor, Checkstyle, and SpotBugs pass. The remaining GitHub Actions runs are currently awaiting maintainer approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Duplicate dispatch leaves orphaned ConsumeQueueExt entries

2 participants