Apache Iceberg Rust version
None
Describe the bug
When using iceberg-catalog-glue, every call to update_table pushes ALL historical schema versions' fields to AWS Glue's column definition. This causes Glue Console to display duplicate columns — one entry per schema version for every field that existed in any past schema.
To Reproduce
- Create an Iceberg table via the Glue catalog
- Insert data with new fields, triggering schema evolution (AddSchema + SetCurrentSchema)
- Observe the table in AWS Glue Console → Table → Schema
Expected behavior
Only the current schema's fields should be pushed to Glue's column definition. Historical schema versions are already preserved in the Iceberg metadata file on S3.
https://github.com/apache/iceberg-rust/blob/main/crates/catalog/glue/src/schema.rs#L57-L63
for schema in metadata.schemas_iter() {
if schema.schema_id() == current_schema.schema_id() {
continue;
}
visit_schema(schema, &mut builder)?;
}
The from_iceberg function visits the current schema (marked as current=true), then iterates over all historical schemas and visits them too (marked as current=false). All of these are included in the Glue TableInput.storage_descriptor.columns. Glue's API does not filter by iceberg.field.current, so the UI displays all of them.
Willingness to contribute
None
Apache Iceberg Rust version
None
Describe the bug
When using
iceberg-catalog-glue, every call toupdate_tablepushes ALL historical schema versions' fields to AWS Glue's column definition. This causes Glue Console to display duplicate columns — one entry per schema version for every field that existed in any past schema.To Reproduce
Expected behavior
Only the current schema's fields should be pushed to Glue's column definition. Historical schema versions are already preserved in the Iceberg metadata file on S3.
https://github.com/apache/iceberg-rust/blob/main/crates/catalog/glue/src/schema.rs#L57-L63
The from_iceberg function visits the current schema (marked as current=true), then iterates over all historical schemas and visits them too (marked as current=false). All of these are included in the Glue TableInput.storage_descriptor.columns. Glue's API does not filter by iceberg.field.current, so the UI displays all of them.
Willingness to contribute
None