kafka: create and wait for topic during verification (#5898)

[ti-chi-bot]

This is an automated cherry-pick of #5898

What problem does this PR solve?

Issue Number: close #5896

What is changed and how it works?

When a Kafka consumer starts immediately after a changefeed is created, it can auto-create the topic with the broker defaults before TiCDC initializes the Kafka sink. Kafka sink verification previously sent a validate-only CreateTopics request, so the topic was not actually created. This can leave the topic with a partition count different from the changefeed configuration, causing message delivery failures and increasing replication lag.

What is changed and how it works?

• Make Kafka sink verification create a missing topic and wait until it is visible in Kafka metadata before returning.
• Keep verification lightweight by using a one-shot topic initialization path that does not spawn unncessary resource.
• Start the runtime topic manager's metadata refresh only after the initial topic is ready.
• Make TiCDC's Kafka admin CreateTopic operation always create the topic. The Sarama adapter now owns the fixed validateOnly=false argument.

Check List

Tests

• Unit test

  go test --tags=intest ./downstreamadapter/sink/topicmanager \
    ./downstreamadapter/sink/kafka ./pkg/sink/kafka -count=1

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

Summary by CodeRabbit

Summary by CodeRabbit

• New Features

  • Kafka topics now use a streamlined creation and verification process.
  • Topic visibility is confirmed before use, improving setup reliability.
  • Metadata refresh begins only after a topic is created or detected.

• Bug Fixes

  • Improved handling of existing topics and topic-creation errors.
  • Topic configuration validation, including replication requirements, remains enforced.
  • Simplified topic creation behavior provides more consistent Kafka administration.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign asddongmen for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8e902368-b1c1-4cf8-a7c1-489deaee5ce3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[3AceShowHand]

duplicate of https://github.com/pingcap/ticdc/pull/5897，close it