Skip to content

[dev/1.3] Pipe: preserve TsFile parse OOM cause#18105

Open
Caideyipi wants to merge 2 commits into
apache:dev/1.3from
Caideyipi:codex/preserve-tsfile-oom-cause-dev-1.3
Open

[dev/1.3] Pipe: preserve TsFile parse OOM cause#18105
Caideyipi wants to merge 2 commits into
apache:dev/1.3from
Caideyipi:codex/preserve-tsfile-oom-cause-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

This PR cherry-picks f8e393e6f0c0f7c940942eadc2b10171b69d78f6 to dev/1.3.

origin/dev/1.3 already has 8657782, the dev/1.3 counterpart of b871f1e42a8, but it was still wrapping TsFile parse/read failures with PipeException without preserving the cause. That hides PipeRuntimeOutOfMemoryCriticalException from PipeProcessorSubtask's root-cause check and can stop the pipe instead of treating the OOM as a temporary retryable failure.

Changes:

  • Preserve the original exception as the PipeException cause in PipeTsFileInsertionEvent.
  • Add unit coverage for cause preservation and processor retry behavior.
  • Add an integration test using throwing-exception-processor to keep a pipe RUNNING through temporary TsFile parse OOM.

Validation:

  • Attempted: mvn -pl iotdb-core/datanode -Dtest=PipeProcessorSubtaskExecutorTest -DskipITs test. This failed before running the target test because the datanode module compiled against stale local SNAPSHOT dependencies missing PipePeriodicalLogReducer and FileUtils.createLink.
  • Attempted: mvn -pl iotdb-core/datanode -am -Dtest=PipeProcessorSubtaskExecutorTest -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false -DskipITs test. This failed in the existing thrift-commons code generation/compile step because generated thrift code references TProtocol.incrementRecursionDepth/decrementRecursionDepth, before reaching datanode tests.

Caideyipi added 2 commits July 3, 2026 12:28
(cherry picked from commit f8e393e6f0c0f7c940942eadc2b10171b69d78f6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant