Fix GPU CI: disable codecov safe.directory write (.gitconfig lock race)#67
Merged
ChrisRackauckas merged 1 commit intoJun 23, 2026
Conversation
The two GPU matrix jobs (CUDA Core / CUDA Applications) run concurrently on the same self-hosted runner host sharing a single ~/.gitconfig. The codecov-action@v7 setup runs `git config --global --add safe.directory`, and the two jobs race on git's config lock, failing one job with `error: could not lock config file .../.gitconfig: File exists` (exit 255) even though the tests themselves pass. This is the dominant cause of the intermittent CUDA Core / CUDA Applications reds on main. Setting `disable_safe_directory: true` skips that unnecessary global-git write (the workspace is already owned by the runner user), removing the race without masking any failure. The tests still fully gate the job and coverage still uploads. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
GPU Testsworkflow onmainis intermittently red. Its two matrix jobs —CUDA CoreandCUDA Applications— run concurrently on the same self-hosted GPU runner host, sharing a single/home/chrisrackauckas/.gitconfig.The
codecov/codecov-action@v7post-test step runsgit config --global --add safe.directory .... When both matrix jobs reach that step at overlapping times, they race on git's config-file lock and one job dies with:This happens after
ComplementaritySolve tests passedis printed, so the tests themselves are fine — it is purely the codecov action's git-config setup racing. Which of the two jobs loses the race varies run-to-run (it hitCUDA Applicationson the latestmainrun andCUDA Coreon a recent PR run), confirming a concurrency race rather than a code/test issue.fail_ci_if_error: falseis already set, but it does not help here: the failure is in the action'ssafe.directorysetup step, which exits 255 before that flag is consulted.Fix
Set
disable_safe_directory: trueon the codecov step. This skips thegit config --global --add safe.directorywrite, which is the only operation racing on the shared~/.gitconfig. The write is unnecessary on these self-hosted runners (the workspace is already owned by the runner user), so disabling it removes the race without masking any failure. The test step (julia-runtest) still fully gates the job, and coverage still uploads.Notes
.jlfiles touched (no Runic formatting needed).safe.directorysetup and that the tests pass before it; this is a self-hosted-runner-environment race, so it cannot be reproduced on a non-GPU dev box.Please ignore until reviewed by @ChrisRackauckas