Skip to content

test: stabilize sync_diff_inspector checkpoint test#12559

Open
joechenrh wants to merge 2 commits intopingcap:masterfrom
joechenrh:fix/stabilize-checkpoint-test
Open

test: stabilize sync_diff_inspector checkpoint test#12559
joechenrh wants to merge 2 commits intopingcap:masterfrom
joechenrh:fix/stabilize-checkpoint-test

Conversation

@joechenrh
Copy link
Contributor

@joechenrh joechenrh commented Mar 13, 2026

What problem does this PR solve?

Issue Number: close #12553

What is changed and how it works?

The checkpoint integration test (checkpoint/run.sh) was flaky because it pre-computed a bucket_index_right (or bucket_index_left-bucket_index_right) from the last run and then used check_contains_regex to match a full indexCode pattern in the resumed run's first chunk. When chunks are created in parallel, multiple resumed chunks can share the same lower bound, so the selected first chunk may carry a different—but equally valid—indexCode, causing the regex match to fail nondeterministically.

Changes (bucket splitter, cases 1 & 2):

  • Removed the pre-computed bucket_index_right/bucket_index_left variables that were derived from the previous run's index.
  • Instead of matching the entire indexCode string with a regex, we now parse the resumed first chunk's indexCode into an array and assert only the structurally stable fields:
    • Case 1 (last chunk in bucket): after resuming, the next chunk must start a new bucket, so we assert chunkIndex == 0.
    • Case 2 (middle of bucket): the next chunk must be the immediate successor within the same bucket, so we assert chunkIndex == lastChunkIndex + 1 and chunkCount == lastChunkCount.

Changes (random splitter, case 1):

  • Removed the check_contains_regex assertion on indexCode entirely, because the random splitter may re-split the remaining range after resuming, making the indexCode unstable across runs.
  • We still assert that the resumed chunk's bound matches the last chunk's upper bound, which is the meaningful invariant.

Check List

Tests

  • Integration test

Questions

Will it cause performance regression or break compatibility?

No. This is a test-only change.

Do you need to update user documentation, design documentation or monitoring documentation?

No.

Release note

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-triage-completed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Mar 13, 2026
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 13, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tammyxia for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 13, 2026
@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request stabilizes the sync_diff_inspector checkpoint test by refactoring how chunk indices are validated. It moves away from brittle regex-based assertions to more explicit shell array parsing and direct comparisons, making the test more resilient to variations in chunk generation and resumption logic, particularly for random splitters.

Highlights

  • Improved Chunk Index Validation: Replaced regex-based checks for first_chunk_index with more robust shell array parsing and direct index comparisons, enhancing test stability.
  • Removed Redundant Bucket Index Logic: Eliminated unnecessary calculations and echoes for bucket_index_right and bucket_index_left that were previously used for regex assertions.
  • Adjusted Random Splitter Test: Removed a strict regex assertion for indexCode in the continuous checkpoint test, acknowledging that the random splitter's indexCode may not be stable across runs.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • sync_diff_inspector/tests/sync_diff_inspector/checkpoint/run.sh
    • Removed unused bucket_index_right and bucket_index_left variable assignments and echo statements.
    • Refactored first_chunk_index parsing from regex to shell array manipulation for more precise validation.
    • Updated chunk index validation logic to use direct array element comparisons instead of regex patterns.
    • Removed a specific regex check for first_chunk_index in the continuous checkpoint test, adding a comment about the instability of indexCode for random splitters.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to stabilize a checkpoint test by replacing brittle regex checks with more robust parsing and assertions in a shell script. The changes are a good improvement. I've suggested a couple of refactorings to simplify the new parsing logic and remove leftover debugging code, making the script cleaner and more idiomatic.

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Mar 13, 2026
@joechenrh
Copy link
Contributor Author

/severity minor
/type bug

@ti-chi-bot ti-chi-bot bot added severity/minor type/bug The issue is confirmed as a bug. labels Mar 13, 2026
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ti-chi-bot ti-chi-bot bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed do-not-merge/needs-triage-completed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 13, 2026
@joechenrh
Copy link
Contributor Author

Makes sense, simplified both blocks in joechenrh@9b0ed66 . Replaced with a single IFS=: read -r -a and dropped the debug loops.

@codecov
Copy link

codecov bot commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 53.3562%. Comparing base (d16456b) to head (9b0ed66).
⚠️ Report is 2 commits behind head on master.
✅ All tests successful. No failed tests found.

❌ Your project check has failed because the head coverage (53.3562%) is below the target coverage (60.0000%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
Components Coverage Δ
cdc 57.3045% <ø> (∅)
dm 49.0888% <ø> (+0.0207%) ⬆️
engine 50.6941% <ø> (ø)
Flag Coverage Δ
cdc 57.3045% <ø> (?)
unit 53.3562% <ø> (+3.8804%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #12559        +/-   ##
================================================
+ Coverage   49.4757%   53.3562%   +3.8804%     
================================================
  Files           486       1010       +524     
  Lines         70677     139860     +69183     
================================================
+ Hits          34968      74624     +39656     
- Misses        32951      59624     +26673     
- Partials       2758       5612      +2854     
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 13, 2026

@joechenrh: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-check 9b0ed66 link true /test pull-check
pull-dm-integration-test 9b0ed66 link true /test pull-dm-integration-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note-none Denotes a PR that doesn't merit a release note. severity/minor size/S Denotes a PR that changes 10-29 lines, ignoring generated files. type/bug The issue is confirmed as a bug.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sync_diff_inspector checkpoint integration test flakes on resumed chunk selection

1 participant