Skip to content

concurrency: stale Mutex.Unlock retry can delete a newer same-session lock - #22115

Open
AkashKumar7902 wants to merge 1 commit into
etcd-io:mainfrom
AkashKumar7902:agent/fence-stale-mutex-unlock
Open

concurrency: stale Mutex.Unlock retry can delete a newer same-session lock#22115
AkashKumar7902 wants to merge 1 commit into
etcd-io:mainfrom
AkashKumar7902:agent/fence-stale-mutex-unlock

Conversation

@AkashKumar7902

@AkashKumar7902 AkashKumar7902 commented Jul 18, 2026

Copy link
Copy Markdown

Fixes #22082

Problem

If an Unlock delete commits but its response is lost, the old Mutex
retains its key and create revision. A later lock using the same session and
prefix reuses the key at a newer create revision. Retrying the stale Unlock
currently deletes that newer lock.

Root cause

Mutex.Unlock deletes by key name only, without verifying that the key is
still the incarnation acquired by that Mutex.

Change

Fence the delete in one transaction with Mutex.IsOwner(), which compares the
key's current create revision with the revision recorded at acquisition. A
stale comparison performs no delete; after a successful RPC, the old local
mutex handle is retired as before.

The regression test adapts the reproducer from #22082. It simulates a committed
delete whose response is replaced by DeadlineExceeded, reacquires the
identical key with the same session, and verifies that retrying the old unlock
preserves the newer create revision.

Verification

The new test failed before the production change because the newer key was
deleted. It passes after the change.

  • go test ./integration/clientv3/concurrency -run '^TestMutexStaleUnlockDoesNotDeleteSameSessionRelock$' -count=10
  • go test -race ./integration/clientv3/concurrency -run '^TestMutexStaleUnlockDoesNotDeleteSameSessionRelock$' -count=1
  • go test ./integration/clientv3/concurrency -count=1
  • go test ./concurrency -count=1
  • go vet ./concurrency
  • make verify-lint
  • make verify-mod-tidy
  • git diff --check

The aggregate make verify reached verify-bom but cannot complete faithfully
on this Darwin/arm64 host: the local BOM generator excludes the amd64-only
Antithesis SDK and therefore differs from the checked-in Linux BOM. Prow's
supported Linux/amd64 checks remain authoritative.

Compatibility

This changes one DeleteRange RPC into one transactional compare-and-delete
RPC. There is no public API, documentation, or main-branch changelog change.
Maintainers can decide whether the fix should be backported.

… lock

Signed-off-by: Akash Kumar <meakash7902@gmail.com>
@kubernetes-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AkashKumar7902
Once this PR has been reviewed and has the lgtm label, please assign siyuanfoundation for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow

Copy link
Copy Markdown

Hi @AkashKumar7902. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@AkashKumar7902
AkashKumar7902 marked this pull request as ready for review July 18, 2026 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

concurrency: stale Mutex.Unlock retry can delete a newer same-session lock

1 participant