*: support shared lock upgrade

[wfxr]

What problem does this PR solve?

Issue Number: ref #68815

Problem Summary:

The TiDB Next-gen shared-lock rollout needs TiDB-side wiring for allowing a pessimistic shared lock to upgrade to an exclusive lock only when the rollout gate is enabled. This rollout is intentionally limited to the TiDB Next-gen kernel; the Classic kernel must reject attempts to enable the gate. It also needs stable TiDB error semantics for typed lock-upgrade conflicts so they abort the current transaction as a non-retryable deadlock instead of entering statement retry. If TiKV reports that the original shared lock was lost during an upgrade, TiDB must expose a stable error and roll back the transaction before subsequent statements can continue using it.

What changed and how does it work?

• Add tidb_enable_shared_lock_upgrade as a global/session boolean system variable, defaulting to OFF. It can be enabled only on the TiDB Next-gen kernel; the Classic kernel rejects ON.
• Propagate the session gate into LockCtx.AllowSharedLockUpgrade when TiDB builds the pessimistic lock context.
• Map client-go ErrLockUpgradeConflict to a non-retryable deadlock at the TiDB transaction driver boundary. The client marks the original conflict as rollback-only, so later lock and commit operations cannot continue while an explicit rollback remains available for cleanup.
• Map client-go ErrSharedLockLost to TiDB error 9015 with the transaction start TS and redacted key, and roll back the affected transaction even if it is not pessimistic.
• Expand TiDB Next-gen RealTiKV shared-lock rollout coverage for successful S-to-X upgrade waits, second-upgrader deadlock handling, transaction cleanup, and foreign-key update/delete cases.
• Replace the local Go workspace with temporary pseudo-versioned wfxr/kvproto and wfxr/client-go module replacements, remove go.work/go.work.sum, and regenerate standalone Go and Bazel dependency metadata.

TODO / Blocker:

• Unblock the dependency update across pingcap/kvproto#1458, pingcap/kvproto#1495, and tikv/client-go#2014.
  • The latest shared-lock-upgrade client-go branch imports github.com/pingcap/kvproto/pkg/apipb, which was introduced by kvproto#1458, but the CSE-compatible kvproto fork pinned by this PR does not include that package.
  • Updating directly to a kvproto revision that contains *: Add switch for newplanner #1458 preserves the legacy V1/V2 wire fields, but changes their generated Go representation to protobuf oneof wrappers. Existing TiDB composite literals such as AutoIDRequest.KeyspaceID, RequestHeader.KeyspaceId, kvrpcpb.Context.KeyspaceId, and keyspacepb.KeyspaceMeta.Id therefore no longer compile and must be migrated or covered by a compatibility strategy.
  • After the companion changes and compatibility work are ready, update kvproto and client-go to upstream pseudo-versions and remove the temporary replace directives.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Validation:

./tools/check/failpoint-go-test.sh pkg/sessionctx/variable -run 'TestNewSessionVars|TestTiDBEnableSharedLockUpgradeGate' -count=1
./tools/check/failpoint-go-test.sh pkg/sessionctx/variable -run TestTiDBEnableSharedLockUpgradeGate -tags=intest,deadlock,nextgen -count=1
./tools/check/failpoint-go-test.sh pkg/executor -run TestImportIntoShouldHaveSameFlagsAsInsert -count=1
./tools/check/failpoint-go-test.sh pkg/store/driver/txn -run 'TestLockNotFoundPrint|TestLockUpgradeConflictMapsToNonRetryableDeadlock' -count=1
GOWORK=off ./tools/check/failpoint-go-test.sh pkg/store/driver/txn -run 'TestSharedLockLostErrorMapping|TestLockUpgradeConflictMapsToNonRetryableDeadlock' -count=1
GOWORK=off ./tools/check/failpoint-go-test.sh pkg/session -run TestSharedLockLostRollsBackTransaction -count=1
GOWORK=off go mod verify
GOWORK=off go mod tidy -diff

# With a local Next-gen cluster containing PD, three TiKV nodes, TiKV Worker, and MinIO:
NEXT_GEN=1 go test ./tests/realtikvtest/txntest \
  -tags=intest,nextgen \
  -with-real-tikv \
  -count=1 \
  -timeout=15m \
  -run '^TestSharedLockBlockExclusiveLock$/(shared_lock_upgrade_waits_for_last_holder|second_upgrader_returns_deadlock)$' \
  -v
NEXT_GEN=1 go test ./tests/realtikvtest/txntest \
  -tags=intest,nextgen \
  -with-real-tikv \
  -count=1 \
  -timeout=15m \
  -run '^TestSharedLockCascadeUpdateExplicitPessimisticTxn$/(insert_child_then_update_parent|insert_child_then_delete_parent_restrict|insert_child_then_delete_parent_cascade)$' \
  -v

make bazel_prepare
make lint

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Add the experimental `tidb_enable_shared_lock_upgrade` global/session variable, defaulting to `OFF`, to gate shared-lock to exclusive-lock upgrades on the TiDB Next-gen kernel. The variable cannot be enabled on the Classic kernel.

Summary by CodeRabbit

• New Features
  • Added tidb_enable_shared_lock_upgrade to control shared-lock upgrades during pessimistic transactions.
  • Shared-lock upgrades now support improved waiting, deadlock handling, and transaction state management on supported next-generation kernels.
• Bug Fixes
  • Added clearer shared-lock-loss and lock-upgrade conflict errors.
  • Transactions now roll back appropriately after shared-lock loss or pessimistic deadlocks.
  • Sensitive lock-key details are redacted from error output.
• Tests
  • Expanded coverage for lock upgrades, error mapping, rollback behavior, and cascading transaction scenarios.

[ti-chi-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

📝 Walkthrough

Walkthrough

TiDB adds a NextGen-only shared-lock upgrade variable, propagates it to TiKV, maps shared-lock loss and upgrade conflicts, rolls back affected transactions, and adds unit and RealTiKV coverage. TiKV dependency replacements and versions are also updated.

Changes

Shared-lock upgrade support

Layer / File(s)	Summary
Upgrade gate and dependency contract go.mod, DEPS.bzl, pkg/sessionctx/vardef/tidb_vars.go, pkg/sessionctx/variable/*	Updates TiKV dependencies and adds the NextGen-only tidb_enable_shared_lock_upgrade variable with default initialization and validation.
Shared-lock error contract pkg/errno/*, pkg/kv/*	Adds error code 9015, its SQL mapping, and the exported KV error.
Lock-context propagation pkg/executor/select.go, pkg/executor/select_test.go	Passes the session upgrade setting and ForUpdateTS into LockCtx.
Transaction error mapping and rollback pkg/store/driver/txn/*, pkg/session/tidb.go, pkg/session/tidb_test.go	Maps shared-lock loss and lock-upgrade conflicts, redacts lock keys, and rolls back valid transactions for the affected errors.
NextGen shared-lock integration coverage tests/realtikvtest/txntest/shared_lock_test.go	Tests lock waiting, competing upgrader deadlocks, and foreign-key update and delete behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: 🔵 Low · up to 83d19

The shared-lock upgrade feature is gated off by default and the focused error-handling and rollback behavior is covered by tests. Merge readiness is low risk, but the PR should remain held until its temporary dependency replacements are switched back to durable upstream versions.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Sysvar
  participant LockContext
  participant TiKV
  participant Transaction
  Client->>Sysvar: Enable shared-lock upgrades
  Sysvar->>LockContext: Store EnableSharedLockUpgrade
  LockContext->>TiKV: Request shared-lock upgrade
  TiKV-->>Transaction: Return lock loss or upgrade conflict
  Transaction->>Transaction: Map error and roll back

Loading

Possibly related issues

• Add typed errors for shared-lock upgrade conflicts and ownership loss to kvrpcpb kvproto#1519 — Adds the typed SharedLockLost error consumed by this change.

Possibly related PRs

• pingcap/tidb#70301 — Updates the same TiKV client dependency declarations.
• pingcap/tidb#70434 — Modifies the same RealTiKV shared-lock test file.

Suggested reviewers: d3hunter

Poem

A rabbit guards the shared-lock gate,
While TiKV errors map their fate.
Keys hide softly in redacted snow,
Lost locks send transactions below.
Tests leap through waits and deadlocks bright—
Upgrade paths now hop just right.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely summarizes the primary change: support for shared-lock upgrades.
Description check	✅ Passed	The description includes the required issue reference, problem, implementation details, tests, side effects, documentation impact, and release note.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cfzjywxk, terry1purcell for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS
• pkg/session/OWNERS
• pkg/sessionctx/vardef/OWNERS
• pkg/sessionctx/variable/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[wfxr]

/hold

[coderabbitai]

🧹 Nitpick comments (1)

go.mod (1)

366-368: 🔒 Security & Privacy | 🔵 Trivial

Temporary fork dependency is consistent with the PR's tracked /hold status.

The replace directives point github.com/pingcap/kvproto and github.com/tikv/client-go/v2 to github.com/wfxr/kvproto and github.com/wfxr/client-go/v2. This matches the PR description: the PR is /hold, and the TODO is to switch to upstream pseudo-versions after the companion kvproto and client-go changes merge. Confirm CI blocks merge until the replace directives point back to upstream modules, since a personal-namespace fork is not a durable dependency source.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@go.mod` around lines 366 - 368, Keep the temporary fork replacements for
github.com/pingcap/kvproto and github.com/tikv/client-go/v2 while the PR remains
on hold, and add or update CI merge gating to block merging until both
replacements are restored to upstream modules at the required pseudo-versions
after the companion changes merge.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@go.mod`:
- Around line 366-368: Keep the temporary fork replacements for
github.com/pingcap/kvproto and github.com/tikv/client-go/v2 while the PR remains
on hold, and add or update CI merge gating to block merging until both
replacements are restored to upstream modules at the required pseudo-versions
after the companion changes merge.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5247c999-efee-4b73-b21b-b1828b7056d7

📥 Commits

Reviewing files that changed from the base of the PR and between d5f9ca5 and 83d198b.

⛔ Files ignored due to path filters (1)

• go.sum is excluded by !**/*.sum

📒 Files selected for processing (20)

• DEPS.bzl
• go.mod
• pkg/errno/errcode.go
• pkg/errno/errname.go
• pkg/executor/select.go
• pkg/executor/select_test.go
• pkg/kv/error.go
• pkg/kv/error_test.go
• pkg/session/BUILD.bazel
• pkg/session/tidb.go
• pkg/session/tidb_test.go
• pkg/sessionctx/vardef/tidb_vars.go
• pkg/sessionctx/variable/session.go
• pkg/sessionctx/variable/sysvar.go
• pkg/sessionctx/variable/sysvar_test.go
• pkg/sessionctx/variable/varsutil_test.go
• pkg/store/driver/txn/BUILD.bazel
• pkg/store/driver/txn/driver_test.go
• pkg/store/driver/txn/error.go
• tests/realtikvtest/txntest/shared_lock_test.go

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.3276%. Comparing base (4b2a5bb) to head (83d198b).
⚠️ Report is 13 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #69559        +/-   ##
================================================
- Coverage   76.3277%   76.3276%   -0.0001%     
================================================
  Files          2041       2041                
  Lines        558363     558361         -2     
================================================
- Hits         426186     426184         -2     
  Misses       131277     131277                
  Partials        900        900                

Components	Coverage Δ
dumpling	59.8974% <ø> (ø)
parser	∅ <ø> (∅)
br	62.7090% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[wfxr]

/retest

[wfxr]

/retest

[wfxr]

/retest

[ti-chi-bot]

@wfxr: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/check_dev	83d198b	link	true	/test check-dev
pull-build-next-gen	83d198b	link	true	/test pull-build-next-gen
idc-jenkins-ci-tidb/build	83d198b	link	true	/test build
pull-integration-realcluster-test-next-gen	83d198b	link	true	/test pull-integration-realcluster-test-next-gen

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.