Skip to content

Fix ordering detection and placement - #109

Merged
SimonCropp merged 7 commits into
mainfrom
fix/ordering-detection-and-placement
Jul 24, 2026
Merged

Fix ordering detection and placement#109
SimonCropp merged 7 commits into
mainfrom
fix/ordering-detection-and-placement

Conversation

@SimonCropp

Copy link
Copy Markdown
Owner

Four correctness fixes and two performance changes, one commit each.

Ordering nested in a lambda suppressed the default ordering

QueryAnalyzer was an ExpressionVisitor that descended into every node, including the lambdas passed to operators. Any OrderBy inside a subquery lambda set HasOrdering, so the query was treated as explicitly ordered and got no default ordering at all:

context.Employees.Where(_ => _.Tasks.OrderBy(task => task.Title).Count() >= 0)
-- before: no ORDER BY at all
select e.Id, ... from Employees as e where (select COUNT(*) ...) >= 0

Include and Select lambdas already had hand rolled guards against this. Every other operator did not, so Where, Any, SelectMany and GroupBy all triggered it, as did an ordering nested inside a filtered Include.

Replaced the visitor with a walk of the chain itself, which is the only place top level ordering and Include can appear.

Ordering was applied after the operators that choose rows

Ordering was only pushed beneath Select and Include, and appended to the end of the chain otherwise. So Take produced:

select * from (select top(3) * from Employees) order by HireDate desc

Three arbitrary rows, sorted after the fact. Paging could skip and repeat rows across pages while looking correctly sorted, and EF Core still raised RowLimitingOperationWithoutOrderByWarning.

First, FirstOrDefault, Last, LastOrDefault and ElementAt were missed entirely: their result type is the entity rather than IQueryable<T>, so element type detection returned null and nothing was applied, leaving EF Core to raise FirstWithoutOrderByAndFilterWarning.

The two placement walks are now one predicate naming the operators the ordering has to precede, used both to find the source entity type and to insert the ordering. Aggregates and Single are excluded, since ordering them is pointless work, and snapshots assert they stay unordered.

Redundant order detection walked into combined sequences

FindSource followed the first argument of any static method whose source looked enumerable, which includes Concat, Union, Join and friends. Ordering applied to one side was reported as redundant even though the default ordering is never applied to the combined result, so the advice to remove it was wrong.

Inherited ordering could not see non public properties

CreateForDerivedType looked the property up with the default binding flags, which only match public members. Ordering configured on a mapped non public property failed to inherit with Property 'SortOrder' not found on derived type 'Derived', even though the derived type does have it.

Performance

Both are compile time only, since QueryCompilationStarting sits behind EF Core's compiled query cache.

  • IncludeOrderingApplicator rebuilt the entire query tree on every compilation even when there was no Include to touch. The chain analysis already reports whether one is present, so it now runs first and gates the visit. The options extension is also resolved once instead of once each by RequiredOrder and RedundantOrder.
  • Dropped the two element type caches. Both guard a fast path of a few type comparisons that is cheaper than the dictionary probe, and they pinned every type they were handed for the life of the process, which stops a collectible load context unloading. The Include method cache stays, since MakeGenericMethod is genuinely expensive. OrderByClause also no longer holds the Queryable and Enumerable method tables in static fields after resolving its eight methods.

Notes

The second fix changes behaviour: Take now returns different rows than before, deterministic ones, and queries that previously tripped EF Core's row limiting and first without ordering warnings no longer do. That may warrant a major version bump rather than a patch.

Two things left alone deliberately. Configuration.cache still pins entity types for the process lifetime, since it is the config lookup on every compilation and has to be a real cache. RequiredOrder.validated still keys by DbContext type, so the same context type used with different UseDefaultOrderBy options would share a validation result; the tests already declare a separate context type per option combination, so the design was left as is.

Verification

122 tests pass. Each fix has tests, and for the last two I confirmed they are regression tests by reverting the source fix and watching them fail.

QueryAnalyzer was an ExpressionVisitor that descended into every node,
including the lambdas passed to operators. Any OrderBy inside a subquery
lambda, for example

    context.Employees.Where(_ => _.Tasks.OrderBy(task => task.Title).Count() >= 0)

set HasOrdering, so the interceptor treated the query as explicitly
ordered and applied no default ordering at all. The generated SQL had no
ORDER BY. Include and Select lambdas were already guarded against this,
but every other operator was not.

The same false positive hit filtered Include, where an ordering nested in
the filter suppressed the default ordering of the included collection.

Replace the visitor with a walk of the chain itself, which is the only
place top level ordering and Include can appear. Ordering nested in a
lambda now orders its own subquery and leaves the outer query alone.
Ordering was only pushed beneath Select and Include, and appended to the
end of the chain otherwise. So Take produced

    select * from (select top(3) * from Employees) order by HireDate desc

which takes three arbitrary rows and only then sorts them. Paging could
skip and repeat rows across pages while looking correctly sorted, and EF
Core still raised RowLimitingOperationWithoutOrderByWarning.

First, FirstOrDefault, Last, LastOrDefault and ElementAt were missed
entirely. Their result type is the entity rather than IQueryable<T>, so
element type detection returned null and no ordering was applied at all,
leaving EF Core to raise FirstWithoutOrderByAndFilterWarning.

Replace the two placement walks with one predicate naming the operators
the ordering has to precede, used both to find the source entity type and
to insert the ordering. Aggregates and Single are deliberately excluded,
since ordering them is pointless work.
FindSource followed the first argument of any static method whose source
looked enumerable, which includes Concat, Union, Join and the other
operators that combine two sequences. Ordering applied to one side was
then reported as redundant, even though the default ordering is never
applied to the combined result, so the advice to remove it was wrong.

Skip those operators when walking to the source.
CreateForDerivedType looked the property up with the default binding
flags, which only match public members. An entity can map a non public
property, and ordering configured on one failed to inherit with

    Property 'SortOrder' not found on derived type 'Derived'.

even though the derived type does have it. Include non public instance
properties in the lookup.
IncludeOrderingApplicator is an ExpressionVisitor, so it rebuilt the
entire query tree on every compilation even though most queries have no
Include for it to touch. The chain analysis already reports whether one
is present, so run the analysis first and only visit when it is.

The rewrite only replaces the inside of Include lambdas, so neither the
ordering nor the Include flag can change as a result of it.

Also resolve the options extension once and pass it to both callers,
instead of RequiredOrder and RedundantOrder each resolving
IDbContextOptions from the context on every compilation.
OrderByClause held the full method tables of Queryable and Enumerable in
static fields, used only to resolve eight methods during static init. The
MethodInfo instances are cached by the runtime either way, so what the
fields actually kept alive was the two arrays. Resolve per method and let
them go.

The element type lookups were cached in static dictionaries keyed by
Type. A query root is already IQueryable<T> and a navigation is declared
as one of a handful of collection types, so both lookups hit a fast path
of a few type comparisons, which is cheaper than the dictionary probe
guarding it. The dictionaries also pinned every type they were handed for
the life of the process, which stops a collectible load context
unloading. Removed. The Include method cache stays, since MakeGenericMethod
is genuinely expensive.

Also append and reverse instead of repeatedly inserting at the head of the
clause list, and mark the remaining mutable statics readonly.
The placement of the default ordering relative to Skip, Take and First is
now user visible, so say where it goes and which operators are left alone.
@SimonCropp SimonCropp added this to the 1.1.0 milestone Jul 24, 2026
@SimonCropp
SimonCropp merged commit bbb4d98 into main Jul 24, 2026
4 of 6 checks passed
@SimonCropp
SimonCropp deleted the fix/ordering-detection-and-placement branch July 24, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant