[AURON #2437] Support inner residual join conditions in native broadcast hash join

[weimingdiit]

Which issue does this PR close?

Closes #2437

Rationale for this change

Auron currently cannot execute Spark broadcast hash joins natively when an inner join contains a residual non-equi condition in addition to its equi-join keys.

Supporting these conditions improves native execution coverage for common broadcast join queries and keeps the condition evaluation inside the join operator.

What changes are included in this PR?

• Passes the BroadcastHashJoinExec residual condition to NativeBroadcastJoinExec.
• Converts the residual condition to the existing native JoinFilter representation.
• Adds the JoinFilter to the native broadcast join protobuf plan.
• Parses and passes the filter through the native planner.
• Evaluates the residual condition against candidate rows produced by the broadcast hash lookup.
• Supports both BuildLeft and BuildRight.
• Limits residual-condition support to inner broadcast hash joins.
• Adds Scala and Rust test coverage for both broadcast build sides.

Outer, semi, anti, null-aware anti, and broadcast nested-loop join condition support are not included.

Are there any user-facing changes?

No user-facing API changes. More inner broadcast hash joins with residual conditions can now be executed natively by Auron.

How was this patch tested?

UT.

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No

Generated-by: OpenAI Codex (GPT-5)

[Copilot]

Pull request overview

This PR extends Auron’s native Broadcast Hash Join (BHJ) support for inner joins to handle Spark’s residual (non-equi) join conditions by converting the condition into the existing JoinFilter representation, serializing it into the native protobuf plan, and evaluating it inside the native join operator against candidate matches produced by the hash lookup.

Changes:

• Plumbs BroadcastHashJoinExec.condition into NativeBroadcastJoinExec / NativeBroadcastJoinBase, converting it to a native JoinFilter and embedding it in the BHJ protobuf node (inner joins only).
• Updates the native planner and BHJ implementation to parse and apply the join filter during native execution (including both build sides).
• Adds Scala + Rust test coverage validating BHJ residual-condition execution for both BuildLeft and BuildRight.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
spark-extension/src/main/scala/org/apache/spark/sql/execution/auron/plan/NativeBroadcastJoinBase.scala	Adds residual-condition → JoinFilter conversion and attaches it to the BHJ protobuf node (inner-only assertion).
spark-extension/src/main/scala/org/apache/spark/sql/auron/Shims.scala	Extends the shim API to pass an optional join condition into native broadcast join construction.
spark-extension/src/main/scala/org/apache/spark/sql/auron/AuronConverters.scala	Validates (config + inner-only) and forwards residual conditions when converting BroadcastHashJoinExec to native.
spark-extension/src/main/java/org/apache/auron/spark/configuration/SparkAuronConfiguration.java	Updates config description to include BHJ residual-condition evaluation.
spark-extension-shims-spark/src/test/scala/org/apache/auron/AuronQuerySuite.scala	Adds a Spark-side test asserting native BHJ is chosen and that the residual condition is present for both build sides.
spark-extension-shims-spark/src/main/scala/org/apache/spark/sql/execution/joins/auron/plan/NativeBroadcastJoinExec.scala	Carries Spark’s condition through the native BHJ exec node instead of forcing None.
spark-extension-shims-spark/src/main/scala/org/apache/spark/sql/auron/ShimsImpl.scala	Wires the new condition parameter into the Spark shim implementation.
native-engine/datafusion-ext-plans/src/joins/test.rs	Adds a native-engine unit test for BHJ inner join with a residual filter on both probe/build configurations.
native-engine/datafusion-ext-plans/src/joins/bhj/full_join.rs	Enables join-filter evaluation in BHJ inner join path (and updates the error message for non-inner/outer usage).
native-engine/datafusion-ext-plans/src/broadcast_join_exec.rs	Allows join_filter for inner BHJ (removes the prior restriction tied to is_built).
native-engine/auron-planner/src/planner.rs	Parses the new BHJ filter field from protobuf and enforces inner-only semantics in the planner.
native-engine/auron-planner/proto/auron.proto	Adds JoinFilter filter to BroadcastJoinExecNode protobuf schema.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.