Skip to content

Fix cost model problem where useless INs were not consistently rejected#4000

Draft
alecgrieser wants to merge 19 commits intoFoundationDB:mainfrom
alecgrieser:make-in-operator-antisymmetric
Draft

Fix cost model problem where useless INs were not consistently rejected#4000
alecgrieser wants to merge 19 commits intoFoundationDB:mainfrom
alecgrieser:make-in-operator-antisymmetric

Conversation

@alecgrieser
Copy link
Collaborator

The idea of this tiebreaker in the cost model is that we don't want to use an IN plan (IN-join or IN-union) if the underlying IN predicate does not participate in some kind of index scan. As written, the previous implementation would only do that if: (1) one of the two plans was not an IN plan or (2) the useless IN plan was on the left. If there was a useful IN plan on the left, then it would consider the two plans equal (moving on to other tiebreakers); if both of the plans were useless INs, then it would always favor the right, despite the fact that this tiebreaker really shouldn't have distinguished the two plans. The new logic now determines if the plan is useless or not, and if it is, it favors the other plan, unless they're both useless, in which case, it moves on to the next tiebreaker.

This fixes #3998.

normen662 and others added 18 commits November 25, 2025 14:57
Merge remote-tracking branch 'upstream/main' into pre-pruning
One thing that was subtle was that the choice of PickLeft (always returning -1) meant that the comparison in the PlanGenerator was always updating to the new plans when one was available. This flips the comparison so that we now only update the plan if the new plan is strictly better than the old plan. Because the as-yet-best plan is the right operand in the comparator, ties will result in us sticking with that plan.
@alecgrieser
Copy link
Collaborator Author

This is built on top of #3993, and it should be rebased once that one is in.

The idea of this tiebreaker in the cost model is that we don't want to use an IN plan (IN-join or IN-union) if the underlying IN predicate does not participate in some kind of index scan. As written, the previous implementation would only do that if: (1) one of the two plans was not an IN plan or (2) the useless IN plan was on the left. If there was a useful IN plan on the left, then it would consider the two plans equal (moving on to other tiebreakers); if both of the plans were useless INs, then it would always favor the right, despite the fact that this tiebreaker really shouldn't have distinguished the two plans. The new logic now determines if the plan is useless or not, and if it is, it favors the other plan, unless they're both useless, in which case, it moves on to the next tiebreaker.

This fixes FoundationDB#3998.
@alecgrieser alecgrieser force-pushed the make-in-operator-antisymmetric branch from 031e8dd to 7825b85 Compare March 12, 2026 15:22
@alecgrieser alecgrieser added the bug fix Change that fixes a bug label Mar 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix Change that fixes a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

In-operator comparison in PlanningCostModel has antisymmetry violation

2 participants