Skip to content

fix(writer): validate equality delete field ids#2723

Open
u70b3 wants to merge 1 commit into
apache:mainfrom
u70b3:fix/equality-delete-id-validation
Open

fix(writer): validate equality delete field ids#2723
u70b3 wants to merge 1 commit into
apache:mainfrom
u70b3:fix/equality-delete-id-validation

Conversation

@u70b3

@u70b3 u70b3 commented Jun 27, 2026

Copy link
Copy Markdown

Which issue does this PR close?

What changes are included in this PR?

  • Add writer config validation for equality delete field IDs.
  • Reject empty equality ID lists, duplicate equality field IDs, and field IDs that are missing from the table schema.
  • Keep the existing RecordBatchProjector behavior for unsupported field types and map/list reachability.
  • Fix a stale comment: equality delete fields may be optional, unlike identifier fields.

Are these changes tested?

  • cargo test -p iceberg test_equality_delete_rejects_empty_equality_ids --lib
  • cargo test -p iceberg test_equality_delete_rejects_duplicate_equality_ids --lib
  • cargo test -p iceberg test_equality_delete_rejects_missing_equality_id --lib
  • cargo test -p iceberg test_equality_delete_unreachable_column --lib
  • cargo test -p iceberg test_equality_delete_with_nullable_field --lib
  • cargo test -p iceberg equality_delete --lib
  • cargo fmt --check
  • cargo test -p iceberg --lib
  • cargo clippy -p iceberg --all-features --lib --tests -- -D warnings
  • make check
  • make unit-test
  • Docker-backed integration path:
    • docker compose -f dev/docker-compose.yaml up -d --build --wait
    • make nextest (1832 passed, 0 skipped)

Note: the Docker-backed run used local mirror substitutions for image and Python package downloads because direct pulls from Docker Hub timed out in my environment. No compose-file changes are included in this PR.

@u70b3 u70b3 force-pushed the fix/equality-delete-id-validation branch from ef457cc to a770760 Compare June 27, 2026 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Equality delete writer should reject invalid equality field IDs

1 participant