kafka: decouple sarama from tests to make it library independent

[3AceShowHand]

What problem does this PR solve?

Issue Number: close #5884

Kafka Sink tests are coupled to Sarama implementation details, making them difficult to reuse when introducing another Kafka client.

What is changed and how it works?

This PR makes Kafka Sink tests library-independent by testing TiCDC-owned behavior instead of Sarama internals. It covers Kafka Sink verification, topic management, configuration, OAuth2, and producer error handling through common interfaces and test doubles, providing a reusable behavior baseline for future Kafka client implementations.

The production Kafka client and runtime behavior remain unchanged.

Check List

Tests

• Unit test
• Integration test

Questions

Will it cause performance regression or break compatibility?

No.

Do you need to update user documentation, design documentation or monitoring documentation?

No.

Release note

None

Summary by CodeRabbit

• Bug Fixes

  • Improved Kafka producer error reporting with clearer message context and error details.
  • Improved handling and validation of Kafka SASL, TLS, OAuth2, replication, and topic configuration errors.
  • Kafka OAuth2 token requests now provide more reliable endpoint and response error handling.

• Refactor

  • Streamlined Kafka administration interfaces and topic-management integration.
  • Improved Kafka test infrastructure for more consistent configuration and administration behavior.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Kafka administration now uses the AdminClient abstraction. Kafka tests replace Sarama fixtures with direct mocks and HTTP servers. Sink verification supports factory injection, and producer error handling accepts explicit error context.

Changes

Kafka admin abstraction and wiring

Layer / File(s)	Summary
AdminClient contract and wiring pkg/sink/kafka/..., downstreamadapter/sink/helper.go, downstreamadapter/sink/topicmanager/...	Renames ClusterAdminClient to AdminClient and updates factories, options, metrics, helpers, and topic management.
Mock-based Kafka tests downstreamadapter/sink/topicmanager/..., pkg/sink/kafka/options_test.go, downstreamadapter/sink/kafka/sink_test.go, scripts/generate-mock.sh	Replaces Sarama fixtures with MockAdminClient, explicit expectations, and a no-op metrics collector.
Sink factory injection downstreamadapter/sink/kafka/sink.go, downstreamadapter/sink/kafka/sink_test.go	Adds injectable Kafka factory creation for sink verification tests.
Producer and OAuth2 coverage pkg/sink/kafka/sarama_async_producer.go, pkg/sink/kafka/oauth2_token_provider_test.go, pkg/sink/kafka/sarama_config_test.go, pkg/sink/kafka/sarama_sync_producer_test.go	Refines producer error handling, adds OAuth2 request and endpoint-error coverage, and removes obsolete configuration tests.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

• pingcap/ticdc#5368: Modernizes Kafka tests and mock infrastructure in overlapping files.
• pingcap/ticdc#5617: Modifies Kafka sink verification and factory handling.
• pingcap/ticdc#5818: Modifies Kafka option validation and related tests.

Suggested reviewers: lidezhu, wk989898, asddongmen

Poem

A rabbit checks the Kafka gate,
While mock clients wait in state.
Admin paths use one clear name,
OAuth tests verify the frame.
Errors carry context bright—
The sink tests hop through the night.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: decoupling Kafka Sink tests from Sarama.
Description check	✅ Passed	The description includes the issue, problem, implementation, tests, compatibility answers, documentation impact, and release note.
Linked Issues check	✅ Passed	The changes satisfy issue #5884 by replacing Sarama-dependent test fixtures with common interfaces and test doubles.
Out of Scope Changes check	✅ Passed	The production changes support the required test decoupling and do not introduce unrelated functionality.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/sink/kafka/oauth2_token_provider_test.go`:
- Around line 90-93: Update the test around newTokenProvider and provider.Token
to assert that the returned Sarama access token has the expected value
"access-token", while retaining the existing no-error assertions and focused
deterministic setup.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 77fa725d-1c04-481d-8a76-cdc909b849df

📥 Commits

Reviewing files that changed from the base of the PR and between 39ee207 and 97f1ec5.

📒 Files selected for processing (6)

• pkg/sink/kafka/oauth2_token_provider_test.go
• pkg/sink/kafka/options_test.go
• pkg/sink/kafka/sarama_config_test.go
• pkg/sink/kafka/sarama_factory.go
• pkg/sink/kafka/sarama_sync_producer_test.go
• scripts/generate-mock.sh

💤 Files with no reviewable changes (1)

• pkg/sink/kafka/sarama_config_test.go

🚧 Files skipped from review as they are similar to previous changes (2)

• scripts/generate-mock.sh
• pkg/sink/kafka/sarama_factory.go

[3AceShowHand]

/test all

[3AceShowHand]

/test all

[3AceShowHand]

/test all

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lidezhu, wk989898

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [lidezhu,wk989898]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-08-06 06:42:43.032210782 +0000 UTC m=+2683149.068305848: ☑️ agreed by wk989898.
• 2026-08-10 06:53:06.555108068 +0000 UTC m=+3029372.591203125: ☑️ agreed by lidezhu.