[RIP-81] Master-slave metadata incremental synchronization

[Crazylychee]

Which Issue(s) This PR Fixes

Fixes #9616

Brief Description

Master Node:

• MetadataChangeObserver serves as a monitoring interface, extending two subclasses, NoopMetadataChangeObserver and SyncMetadataChangeObserver, for implementing no-op and synchronous message sending, respectively. Configuration map operations in configuration classes such as TimerMetrics, MessageRequestModeManager, ScheduleMessageService, ConsumerOffsetManager, SubscriptionGroupManager, and TopicConfigManager have been modified: Compute methods are used to ensure the atomicity of message sending and in-memory configuration updates, ensuring simultaneous success and failure to prevent data inconsistencies.

• For the frequently updated consumerOffset, a dual buffering approach is implemented, with scheduled and buffer-full synchronous message sending strategies. This reduces the synchronous message sending burden caused by frequent consumerOffset updates. The scheduled strategy also mitigates the long inconsistency window caused by the buffer-full strategy in low-frequency updates.

• A scheduled task is used to store the latest metadata snapshot, which contains the current metadata and the maximum offset of the corresponding system topic.

• Methods for obtaining snapshots from the master node are defined in the AdminBrokerProcessor and BrokerOuterAPIs.

Slave nodes:

• Use DefaultLitePullConsumer to set the starting consumption point for consuming synchronous messages

How Did You Test This Change?

broker config

allowMetadataIncrementalSync = true

Relevant work is already in progress. Proposal documentation is available here:
English version:
https://docs.google.com/document/d/1Q4dckYs5ALt_BM5aId9-3N9-kaGL-cLYvAVhv9Mc7bs/edit?usp=sharing
Chinese version:
https://m0sgzrp00oj.feishu.cn/docx/CvbedEJXto7A2BxfYVXcLXATnbd?from=from_copylink

[codecov-commenter]

Codecov Report

❌ Patch coverage is 18.73706% with 785 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.80%. Comparing base (0e72809) to head (9a0cac9).

Files with missing lines	Patch %	Lines
...apache/rocketmq/broker/slave/SlaveSynchronize.java	8.61%	191 Missing ⚠️
...org/apache/rocketmq/broker/out/BrokerOuterAPI.java	0.00%	148 Missing ⚠️
...a/org/apache/rocketmq/broker/BrokerController.java	3.17%	119 Missing and 3 partials ⚠️
...ocketmq/broker/processor/AdminBrokerProcessor.java	0.00%	97 Missing ⚠️
...pache/rocketmq/common/sync/MetadataChangeInfo.java	0.00%	40 Missing ⚠️
.../broker/loadbalance/MessageRequestModeManager.java	22.85%	25 Missing and 2 partials ⚠️
...ache/rocketmq/broker/sync/SyncMessageProducer.java	0.00%	25 Missing ⚠️
.../rocketmq/broker/offset/ConsumerOffsetManager.java	48.93%	22 Missing and 2 partials ⚠️
...cketmq/broker/schedule/ScheduleMessageService.java	0.00%	16 Missing and 1 partial ⚠️
.../broker/subscription/SubscriptionGroupManager.java	64.58%	16 Missing and 1 partial ⚠️
... and 11 more

Additional details and impacted files

@@              Coverage Diff              @@
##             develop    #9617      +/-   ##
=============================================
- Coverage      48.11%   47.80%   -0.31%     
- Complexity     12138    12168      +30     
=============================================
  Files           1313     1317       +4     
  Lines          93233    94144     +911     
  Branches       11944    12031      +87     
=============================================
+ Hits           44859    45009     +150     
- Misses         42802    43557     +755     
- Partials        5572     5578       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.