Skip to content

Add SkipIfReferenced flag to removeSnapshotsUpdate#722

Open
dhananjaykrutika wants to merge 1 commit intoapache:mainfrom
dhananjaykrutika:remove-snapshot-flag
Open

Add SkipIfReferenced flag to removeSnapshotsUpdate#722
dhananjaykrutika wants to merge 1 commit intoapache:mainfrom
dhananjaykrutika:remove-snapshot-flag

Conversation

@dhananjaykrutika
Copy link
Contributor

ExpireSnapshots computes candidate snapshots for deletion at time T1, but by the time CommitTable applies the update at T2, a concurrent client might link one of these snapshots to a new branch or tag. Without protection, the snapshot is deleted and its new ref is silently cleaned up.

This change adds a SkipIfReferenced flag to removeSnapshotsUpdate. When set, Apply() filters out any snapshot that is still referenced before calling RemoveSnapshots(). ExpireSnapshots sets this flag via SetSkipIfReferenced().

The default behavior (SkipIfReferenced=false) is unchanged: snapshots are removed and orphaned refs are silently cleaned up, preserving backward compatibility with explicit client-driven remove-snapshot updates.

ExpireSnapshots computes candidate snapshots for deletion at time T1,
but by the time CommitTable applies the update at T2, a concurrent
client might link one of these snapshots to a new branch or tag.
Without protection, the snapshot is deleted and its new ref is silently
cleaned up.

This change adds a SkipIfReferenced flag to removeSnapshotsUpdate.
When set, Apply() filters out any snapshot that is still referenced
before calling RemoveSnapshots(). ExpireSnapshots sets this flag
via SetSkipIfReferenced().

The default behavior (SkipIfReferenced=false) is unchanged: snapshots
are removed and orphaned refs are silently cleaned up, preserving
backward compatibility with explicit client-driven remove-snapshot
updates.
@zeroshade
Copy link
Member

does skip-if-referenced exist in the iceberg rest spec?

@dhananjaykrutika
Copy link
Contributor Author

does skip-if-referenced exist in the iceberg rest spec?

No, it doesn't. And that's precisely where the problem lies. I started a discussion thread at https://lists.apache.org/thread/bx4vfclvpjh1osj3g1jlsr0zfowkb6xz to explore possible solutions. The goal was to gather community input on whether introducing skip-if-referenced would improve the spec and align better with real-world usage patterns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants