Skip to content

[Pipe] Improve pipe control-flow lock responsiveness#18076

Merged
jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:pipe-control-flow-acceleration
Jul 1, 2026
Merged

[Pipe] Improve pipe control-flow lock responsiveness#18076
jt2594838 merged 2 commits into
apache:masterfrom
Caideyipi:pipe-control-flow-acceleration

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Motivation

Pipe control operations currently sit behind coarse-grained ConfigNode/DataNode locks. When a transient per-pipe failure happens, the old retry path slept for 3 seconds while still holding the pipe/task-coordinator/node procedure locks, so one problematic pipe could make other pipe operations feel stuck. On the DataNode side, pushMultiPipeMeta also stopped at the first per-pipe failure, which made one bad pipe prevent the rest of the batch from being applied.

This PR focuses on low-risk acceleration instead of timeout tuning, so users can observe real progress via show pipes rather than getting a quick timeout with unclear server-side state.

Changes

  • ConfigNode pipe procedure retry path
    • Remove the 3s lock-held sleep before retrying the same pipe procedure state.
    • Mark only retry executions as isYieldAfterExecution=true, so the procedure executor releases the global locks and requeues the procedure before retrying.
    • Keep normal successful pipe states serialized; this avoids interleaving create/alter/drop calculate/write/push phases and avoids stale meta push risks.
  • ConfigNode release side effects
    • Update PipeMetaSyncProcedure last-synced version only after successful completion.
    • Update PipeProcedureMetrics timer only when the procedure is actually finished, not on retry-yield lock releases.
  • DataNode multi-pipe meta push
    • Continue processing the remaining pipe metas/drop requests after a per-pipe failure message.
    • Collect only non-null per-pipe exception messages.
    • Return success iff no per-pipe exception messages were collected.
    • Preserve already collected per-pipe messages if a later unexpected exception aborts the batch.
    • Extract a package-visible helper/handler to make the batch behavior unit-testable without mocking the global PipeDataNodeAgent.

Tests

Added focused UT coverage:

  • AbstractOperatePipeProcedureV2Test
    • Successful state execution does not yield.
    • Retry state returns quickly without the previous 3s lock-held sleep.
    • Retry yield flag is reset after the next successful execution.
    • State-machine execution yields only before the retry threshold and does not keep yielding after failure.
  • DataNodeInternalRPCServiceImplPushMultiPipeMetaTest
    • Multi-pipe push continues after one per-pipe failure.
    • Multi-pipe drop continues after one per-pipe failure.
    • Successful multi-pipe push/drop responses do not carry exception messages.
    • Already collected per-pipe messages are retained when a later pipe throws unexpectedly.
    • Invalid request and unexpected exception paths return pipe-push-meta error without invoking unrelated handlers.

Verification

  • git diff --check passed.
  • git diff --cached --check passed before commit.

Local Maven verification was attempted but blocked by local environment issues unrelated to this patch:

  • mvn -o test -pl iotdb-core/confignode -Dtest=AbstractOperatePipeProcedureV2Test -DfailIfNoTests=false fails during main compile because the local cached dependencies do not match current source APIs (PipeMetaKeeper, SystemConstant, TopicConstant, etc.).
  • mvn -o test -pl iotdb-core/datanode -Dtest=DataNodeInternalRPCServiceImplPushMultiPipeMetaTest -DfailIfNoTests=false and later Spotless/test-compile attempts hit JVM native-memory/pagefile crashes on this Windows environment.

@jt2594838 jt2594838 merged commit fa194d3 into apache:master Jul 1, 2026
39 of 41 checks passed
@jt2594838 jt2594838 deleted the pipe-control-flow-acceleration branch July 1, 2026 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants