Add stable,flaky syntax to --flaky_test_attempts - #18
Merged
Conversation
Allow CI to set different retry counts for non-flaky and flaky-marked tests via comma-separated values while preserving backward compatibility for bare integer and regex@N overrides.
alexeagle
requested review from
bllanesfigma,
dmiller-figma and
tbarrella-figma
August 7, 2026 18:41
dmiller-figma
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--flaky_test_attemptsto accept two comma-separated integers (<stable>,<flaky>) so CI can retry only flaky-marked targets without changing BUILD files.3) still applies to all tests;defaultstill means stable=1 / flaky=3.regex@Noverrides remain supported with a single integer only (regex@1,3is rejected).Upstream PR: bazelbuild#30638
Tracking issue: bazelbuild#30108
Semantics
flaky=Falseflaky=Truedefault31,31,5//pkg/@3(matched targets)If upstream never merges, are we stuck carrying this patch forever?
No — but we would need to keep carrying it as long as we want
--flaky_test_attempts=1,3in.bazelrc.This patch is small and isolated (~300 lines, mostly tests; core logic is
FlakyTestAttempts+ converter +TestStrategylookup). It does not fork broad Bazel subsystems, and it rebases cleanly onto upstream releases (the upstream PR is the same diff againstmaster).If upstream merges: drop the fork patch on the next Bazel version bump / rebase from
bazelbuild/bazel. No migration needed — behavior and flag syntax are identical.If upstream does not merge: we have three options, none of which block upgrading Bazel:
.bazelrcback to stock semantics.--flaky_test_attempts=3still works on vanilla Bazel (retries all tests). We lose the stable/flaky split until we find another approach (e.g. integerflakyattribute, orregex@Nlists).1,3, behavior matches stock Bazel.The patch is a capability opt-in, not a lock-in. Stock Bazel continues to work; we only depend on this fork delta for the specific
.bazelrcvalue1,3(or other<stable>,<flaky>pairs).Test plan
FlakyTestAttemptsTest— parsing and validationTestAttemptsConverterTest— global, regex, and rejection ofregex@1,3test_flaky_test_attempts_stable_flaky_splitshell test — verifies1,2split and bare integer backward compat