Skip to content

Support transform-based sort orders in ReplaceSortOrderAction (only identity is supported today) #2764

Description

@viirya

Is your feature request related to a problem or challenge?

ReplaceSortOrderAction (transaction/sort_order.rs) only supports sorting by a column's raw value. asc/desc hardcode Transform::Identity internally and only accept a column name — there is no way to declare a sort order using a transform (bucket[N], year, truncate[W], etc.).

Java's SortOrderBuilder.asc/desc accept a Term, which can be a plain column reference or a transform expression (Expressions.bucket(name, n), Expressions.year(name), ...), so this is a gap relative to the Java reference.

The underlying spec-level type is already ready for this: SortField has a general transform: Transform field, and SortOrderBuilder::check_compatibility already validates arbitrary transforms against the source column's type (including a bucket[4] round-trip in its own tests). The gap is only in the transaction-layer API surface.

Describe the solution you'd like

Add asc_with_transform / desc_with_transform to ReplaceSortOrderAction, alongside the existing asc/desc (unchanged, still implying identity):

tx.replace_sort_order()
    .asc_with_transform("event_time", Transform::Year, NullOrder::First)
    .desc_with_transform("id", Transform::Bucket(16), NullOrder::Last);

Transform-compatibility with the source column's type should be checked at commit time (once the table schema is available), matching the timing of Java's SortOrder.Builder.build().

Scope note

This is a metadata-declaration gap only — which sort order (and transform) can be recorded on a table. Whether the write path actually sorts data to match a table's declared sort order is a separate, pre-existing gap unaffected by this: no writer in crates/iceberg/src/writer/ reads SortOrder or sorts data today, for any sort order including the existing identity case.

Willingness to contribute

I can contribute this independently.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions