Skip to content

Pipe: add ConfigNode memory control for log reducer#18069

Merged
jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:codex/confignode-pipe-log-reducer-memory
Jul 1, 2026
Merged

Pipe: add ConfigNode memory control for log reducer#18069
jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:codex/confignode-pipe-log-reducer-memory

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add ConfigNodeMemoryConfig and confignode_memory_proportion for ConfigNode Pipe/Free memory split.
  • Wire PipePeriodicalLogReducer to DataNode and ConfigNode memory resize hooks.
  • Keep ConfigNode Pipe memory exposed only as logReducerMemory, so no other CN module is attached to the new memory framework.

Tests

  • PASS: mvn spotless:apply -pl iotdb-core/node-commons,iotdb-core/confignode,iotdb-core/datanode
  • PASS: mvn test -pl iotdb-core/node-commons,iotdb-core/confignode "-Dtest=PipePeriodicalLogReducerTest,ConfigNodeMemoryConfigTest" -DfailIfNoTests=false
  • PASS: mvn compile -pl iotdb-core/node-commons,iotdb-core/confignode
  • NOTE: mvn compile -pl iotdb-core/node-commons,iotdb-core/datanode reaches javac but fails on existing calc MemoryReservationManager#setHighestPriority mismatch unrelated to this Pipe change.

jt2594838
jt2594838 reviewed Jun 30, 2026
View reviewed changes
Comment thread ...b-core/confignode/src/main/java/org/apache/iotdb/confignode/conf/ConfigNodeMemoryConfig.java Outdated
Comment thread ...b-core/confignode/src/main/java/org/apache/iotdb/confignode/conf/ConfigNodeMemoryConfig.java Outdated
@jt2594838 jt2594838 merged commit 9554e5d into apache:master Jul 1, 2026
39 of 42 checks passed
@jt2594838 jt2594838 deleted the codex/confignode-pipe-log-reducer-memory branch July 1, 2026 10:32
jt2594838 pushed a commit that referenced this pull request Jul 3, 2026
@Caideyipi
Pipe: add ConfigNode memory control for log reducer (#18069) (#18087)
8e213dd 
* Pipe: add ConfigNode memory control for log reducer

* Address ConfigNode memory config review

(cherry picked from commit 9554e5d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jt2594838 jt2594838 jt2594838 left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Caideyipi @jt2594838