Reduce configure times when regenerating clients#3765
Conversation
|
@apache-hb Thanks for bringing this PR! It looks good, i'm going to run it through our CI and once it passes I will merge the PR |
|
@apache-hb Hey, I ran it through our CI and im seeing an |
|
I think that git gc error is transient, I don't think my git commits are causing it. |
|
@apache-hb There is a git gc timing issue. Git's background auto-gc gets triggered during codegen. With the new parallel formatting, the build phase completes while auto-gc is still running — its gc.pid lock file gets captured in the artifact zip. When PrepareArtifacts unpacks the zip and runs git gc, it sees the stale lock and fails. We have a fix out for it in release next week. |
|
@apache-hb PR has been merged in commit |
Description of changes:
Runs clang-format in parallel after generating client code. On my epyc 9654 this reduces the time to configure a new build directory by about ~75%. The implementation is a bit naive and only splits the generated file set into equal sizes, when regenerating a single client some clang-format processess will only format a single file.
I think the only mild downside is the logs look a bit uglier as they now contain interleaved invocations of clang-format
Check all that applies:
Check which platforms you have built SDK on to verify the correctness of this PR.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.