kafka: unify sink errors and replace failpoint tests (#5786)

[ti-chi-bot]

This is an automated cherry-pick of #5786

What problem does this PR solve?

Kafka sink errors are overly fine-grained and inconsistently wrapped. Some propagation paths add multiple TiCDC error codes, lose the original cause, or use an error category owned by another sink. This makes retry classification and troubleshooting harder.

Kafka producer failure tests also rely on failpoints even when the behavior can be covered by focused unit tests and real Kafka failures.

Issue Number: close #5790

What is changed and how it works?

• Consolidate configuration errors into ErrKafkaInvalidConfig.
• Consolidate Admin API errors into ErrKafkaAdminAPI.
• Consolidate producer send errors into ErrKafkaSendMessage.
• Add ErrNewKafkaSink and ErrKafkaSinkClosed, and delete superseded Kafka error definitions.
• Wrap third-party errors once where their semantic category is first known; upper layers return them directly.
• Preserve original causes and propagate context cancellation without extra wrapping.
• Classify external storage and marshal failures at their owning boundaries.
• Correct the Pulsar producer creation error category.

The Kafka failpoints are removed without dropping their behavioral coverage:

• KafkaSinkAsyncSendError: replaced by a real broker-rejection scenario in kafka_big_messages. The producer starts with a large topic limit, the test lowers the broker-side topic limit, and Kafka rejects an already-enqueued message through Sarama Errors(). The test checks ErrKafkaSendMessage, warning-to-normal recovery, replay, and final data consistency.
• Kafka sink unit coverage verifies that an AsyncRunCallback error makes sink.Run return the error and mark the sink abnormal without depending on Sarama.
• KafkaSinkSyncSendMessageError: single-message failure, cause preservation, and single ErrKafkaSendMessage wrapping are covered by TestSyncProducerErrorWrappedOnce.
• KafkaSinkSyncSendMessagesError: batch failure, cause preservation, and single ErrKafkaSendMessage wrapping are covered by TestSyncProducerErrorWrappedOnce.
• DML, DDL, and checkpoint log context previously checked by the failpoint integration test is covered by focused BuildEventLogContext unit tests.

Check List

Tests

• Unit test
• Integration test

The Kafka integration scenario requires the repository Linux test environment and is left for CI; local unit tests, shell validation, and static checks pass on macOS/ARM.

Questions

Will it cause performance regression or break compatibility?

No performance regression is expected. This PR intentionally removes obsolete exported Kafka error variables and changes affected RFC codes and messages; compatibility aliases are not retained.

Do you need to update user documentation, design documentation or monitoring documentation?

No user-facing documentation or monitoring changes are required.

Release note

None

Summary by CodeRabbit

• Bug Fixes

  • Improved Kafka and Pulsar sink error reporting for clearer configuration, connection, administration, and message-delivery failures.
  • Invalid Kafka settings and topic expressions now consistently report configuration errors.
  • Kafka topic and broker settings now distinguish missing configuration from administration failures and use safe fallback behavior.
  • Sending messages after a sink closes now returns a consistent closed-sink error.
  • External storage and Pulsar producer failures now use more accurate error classifications.

• Tests

  • Expanded coverage for asynchronous delivery failures, closed sinks, configuration errors, and topic administration scenarios.
  • Updated integration coverage for asynchronous message-size failures and Pulsar sink recovery.

[ti-chi-bot]

This cherry pick PR is for a release branch and has not yet been approved by triage owners.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick:

1. It must be LGTMed and approved by the reviewers firstly.
2. For pull requests to TiDB-x branches, it must have no failed tests.
3. AFTER it has lgtm and approved labels, please wait for the cherry-pick merging approval from triage owners.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ti-chi-bot]

@3AceShowHand This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign 3aceshowhand for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

@ti-chi-bot: ## If you want to know how to resolve it, please read the guide in TiDB Dev Guide.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bd794593-0c6c-4f0d-8e9e-5cc09204d939

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[3AceShowHand]

close by #5811