Is your feature request related to a problem? Please describe.
Auron Iceberg native changelog scan supports insert-only changelog tasks and materializes changelog metadata columns as per-task constant values.
However, simple predicates on changelog metadata columns are not used to prune changelog scan tasks during native scan planning. For example, a query filtering by _commit_snapshot_id may still plan all insert-only changelog tasks in the selected changelog range.
Describe the solution you'd like
Use simple changelog metadata predicates to prune AddedRowsScanTasks before building native Iceberg scan tasks.
The scope should be limited to task-level constants:
_change_type_commit_snapshot_id_change_ordinal
Only simple predicate forms should be supported:
Unsupported expressions such as OR, NOT, complex expressions, or mixed data-column predicates should not be used for task-level pruning.
Describe alternatives you've considered
One alternative is to rely only on the existing post-scan filter path.
That keeps correctness, but it can require native scan to read changelog tasks that are known not to match based on task-level metadata.
Additional context
This change should not add support for delete, update, position delete, or equality delete changelog operations.
Is your feature request related to a problem? Please describe.
Auron Iceberg native changelog scan supports insert-only changelog tasks and materializes changelog metadata columns as per-task constant values.
However, simple predicates on changelog metadata columns are not used to prune changelog scan tasks during native scan planning. For example, a query filtering by
_commit_snapshot_idmay still plan all insert-only changelog tasks in the selected changelog range.Describe the solution you'd like
Use simple changelog metadata predicates to prune
AddedRowsScanTasks before building native Iceberg scan tasks.The scope should be limited to task-level constants:
_change_type_commit_snapshot_id_change_ordinalOnly simple predicate forms should be supported:
=INANDUnsupported expressions such as
OR,NOT, complex expressions, or mixed data-column predicates should not be used for task-level pruning.Describe alternatives you've considered
One alternative is to rely only on the existing post-scan filter path.
That keeps correctness, but it can require native scan to read changelog tasks that are known not to match based on task-level metadata.
Additional context
This change should not add support for delete, update, position delete, or equality delete changelog operations.