Skip to content

[VL] Validate cuDF whole stage even when GPU table scan is enabled#12383

Open
ReemaAlzaid wants to merge 1 commit into
apache:mainfrom
ReemaAlzaid:gpu-cudf-g1-tablescan-validation
Open

[VL] Validate cuDF whole stage even when GPU table scan is enabled#12383
ReemaAlzaid wants to merge 1 commit into
apache:mainfrom
ReemaAlzaid:gpu-cudf-g1-tablescan-validation

Conversation

@ReemaAlzaid

Copy link
Copy Markdown
Contributor

What changes are proposed in this pull request?

CudfNodeValidationRule decides which whole stage transformers can run on the cuDF/GPU backend. That tag is used for stage level GPU offload, GPU shuffle, and node naming.

Before this change, when spark.gluten.sql.columnar.backend.velox.cudf.enableTableScan=true, we were tagging every stage for GPU offload without running native validation, even when spark.gluten.sql.columnar.backend.velox.cudf.enableValidation=true. That was different from the enableTableScan=false path, where validation is still applied.

This PR makes the validation behavior consistent. We now validate the stage regardless of enableTableScan. The native validator already handles TableScan, so stages with scans can still offload, but only if the rest of the operators are supported by cuDF.

I also moved the decision logic into a small decideOffload(...) helper so it can be unit tested without needing a GPU. No new configs are added.

How was this patch tested?

  • Added CudfNodeValidationRuleSuite, with 4 passing tests covering the offload decision matrix.
  • Added a regression test for enableTableScan=true where a cuDF-unsupported stage should not be force-offloaded.
  • Manually tested on GPU with TPC-H using cudf=true and enableTableScan=true. Supported stages keep the same behavior

@github-actions github-actions Bot added the VELOX label Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant