Remove 50-shard limit on test shard_count - #20
Draft
dmiller-figma wants to merge 6455 commits into
Draft
Conversation
…ocumentation referencing it. (bazelbuild#27747) PiperOrigin-RevId: 834716100 Change-Id: Ie5b720ea288789a70a1ac2977b8a23081db06884
…azelbuild#27720) We previously tried to ensure that `BulkTransferException#getLostArtifacts` was never called with any exceptions mentioning non-inputs, but that still resulted in crashes. As the retrier logic in the remote spawn runner makes this error-prone and it is very unlikely that the `InputMetadataProvider` is actually missing metadata for an input, this change simplifies the logic by treating every `ActionInput` missing from it as an output and raising an error (possibly resulting in a local fallback or retry) instead of crashing. Speculatively fixes bazelbuild#27229 Closes bazelbuild#27500. PiperOrigin-RevId: 834106361 Change-Id: Ie002690cf41513d679d653f33ee96ab011f89f88 Commit bazelbuild@09115bb Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…dule (bazelbuild#27805) Adding support for `flag_alias` . Design https://docs.google.com/document/d/1yOvi4hVV7Ja32ocwVb4lsEUnijftk8nilXPncYm-BH8/edit?tab=t.0#heading=h.qn3unswby87l Closes bazelbuild#27658. PiperOrigin-RevId: 836843577 Change-Id: I83b22b3c7d4ff560821930389985ca061e8d11a2 Commit bazelbuild@d7f6d29 Co-authored-by: Ara Nguyen <aranguyen@google.com>
Otherwise facts corresponding to extensions that aren't up-to-date in Skyframe after the command (for example, because they haven't been requested) will be dropped. Fixes bazelbuild#27730 Closes bazelbuild#27744. PiperOrigin-RevId: 834878573 Change-Id: I828af6b59dc61c42f1e8971843be3a6ed1ac9fe0 Commit bazelbuild@ade92cf Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
Previously bazelbuild@f60a22e replaced `--experimental_starlark_types` with two separate flags, `--experimental_starlark_type_syntax` and `--experimental_starlark_type_checking`. However, it failed to wire the latter through to the actual type checking logic. This makes it impossible to process bzl files that contain type annotations without also attempting to check those types -- a no-no for Bazel 9.0, which should tolerate but ignore type annotations. - Clarify type checking flag's description. - Moved key constant for type checking flag from `BuildLanguageOptions` into `StarlarkSemantics`, since it is referenced within the Starlark interpreter itself. - Rename `FileOptions#allowArbitraryTypeExpressions` -> `tolerateInvalidTypeExpressions` to make the intent clearer to the reader. - Guard on flag in `Eval.java` and `StarlarkFunction.java`. - Rename `TypeCheckTest` -> `DynamicTypeCheckTest` in anticipation of upcoming static type checking. Fixes bazelbuild#27716. PiperOrigin-RevId: 839269449 Change-Id: Ieb01c00a30758d7d486ef5ffdb03198fcff233ac
This is a fork of bazelbuild#27498. From the comments/open items on that PR, I've: 1. Removed the tests that were called out as not needed. 2. Added manual tag to `focus_test` to avoid pickup in larger runs until issue with `--experimental_enable_skyfocus` is resolved. 3. Switched failing test cases from using `from bazel_tools.tools.python.runfiles import runfiles` to use `from python.runfiles import runfiles` There are still a couple failing tests, looking for some guidance on if these can be removed/ignored: ``` test_source_file_does_not_override_standard_library FAILED: Expected regexp 'I am lib!' not found. ``` and ``` 1) explicitInitPy_CanBeSelectivelyDisabled(com.google.devtools.build.lib.bazel.rules.python.BazelPyBinaryConfiguredTargetTest) value of : getEmptyFilenames() unexpected (4): pkg/_foo.venv/lib/python3.11/__init__.py, pkg/_foo.venv/__init__.py, pkg/_foo.venv/lib/python3.11/site-packages/__init__.py, pkg/_foo.venv/lib/__init__.py --- expected : [pkg/__init__.py] but was : [pkg/_foo.venv/lib/python3.11/__init__.py, pkg/_foo.venv/__init__.py, pkg/_foo.venv/lib/python3.11/site-packages/__init__.py, pkg/__init__.py, pkg/_foo.venv/lib/__init__.py] at com.google.devtools.build.lib.bazel.rules.python.BazelPyBinaryConfiguredTargetTest.explicitInitPy_CanBeSelectivelyDisabled(BazelPyBinaryConfiguredTargetTest.java:235) 2) explicitInitPy_CanBeGloballyDisabled(com.google.devtools.build.lib.bazel.rules.python.BazelPyBinaryConfiguredTargetTest) value of : getEmptyFilenames() unexpected (4): pkg/_foo.venv/lib/python3.11/__init__.py, pkg/_foo.venv/__init__.py, pkg/_foo.venv/lib/python3.11/site-packages/__init__.py, pkg/_foo.venv/lib/__init__.py --- expected : [pkg/__init__.py] but was : [pkg/_foo.venv/lib/python3.11/__init__.py, pkg/_foo.venv/__init__.py, pkg/_foo.venv/lib/python3.11/site-packages/__init__.py, pkg/__init__.py, pkg/_foo.venv/lib/__init__.py] at com.google.devtools.build.lib.bazel.rules.python.BazelPyBinaryConfiguredTargetTest.explicitInitPy_CanBeGloballyDisabled(BazelPyBinaryConfiguredTargetTest.java:250) ``` Closes bazelbuild#27792. PiperOrigin-RevId: 839448559 Change-Id: I888614dd92fcbc4deda63a86a452bbd94e8f1523 Commit bazelbuild@fbe3009 --------- Co-authored-by: Damian Banki <ddaniel.banki@gmail.com>
…azelbuild#27821) Ensures that the returned `Path` is still in the overlay file system. Also make the error message emitted by `Path#checkSameFileSystem` more informative. This is motivated by and helped discover the above as the fix for the following crash observed when using the remote repo contents cache with an explicit `--sandbox_base`: ``` Caused by: java.lang.IllegalArgumentException: Files are on different filesystems: /dev/shm/bazel-sandbox.b10976335efa519b0184f3091ac8e21f7beefb92142303f9ab2c3341f45a2f28/linux-sandbox/18/execroot/_main/external/c-ares+/configs/ares_build.h (on com.google.devtools.build.lib.unix.UnixFileSystem@5e0a8154), /home/ubuntu/.cache/bazel/_bazel_ubuntu/123/execroot/_main/external/c-ares+/configs/ares_build.h (on com.google.devtools.build.lib.remote.RemoteExternalOverlayFileSystem@6cd9bfda) at com.google.devtools.build.lib.vfs.Path.checkSameFileSystem(Path.java:964) at com.google.devtools.build.lib.vfs.Path.createSymbolicLink(Path.java:523) at com.google.devtools.build.lib.vfs.Path.createSymbolicLink(Path.java:535) at com.google.devtools.build.lib.sandbox.SymlinkedSandboxedSpawn.copyFile(SymlinkedSandboxedSpawn.java:129) ``` Alternative to bazelbuild#27721 Closes bazelbuild#27802. PiperOrigin-RevId: 837832265 Change-Id: I3b73167496b011aef66954d59ca3804b4b64996f Commit bazelbuild@8eaf6a9 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
) Bazel already supports decompressing tar files compressed with bz2, gz, xz, and zst. This change adds support for decompressing individual files (not in a tar bundle) that are compressed with those compression algorithms. In other words, support for decompression: before: .tar.bz2, .tar.gz, .tar.xz, .tar.zst after : .tar.bz2, .bz2, .tar.gz, .gz, .tar.xz, .xz, .tar.zst, .zst Fixes: bazelbuild#20125 Closes bazelbuild#27413. PiperOrigin-RevId: 839740791 Change-Id: I82595436f6ecab23374db7c50a5027a4bb279578 Commit bazelbuild@80fe2d9 Co-authored-by: Will Stranton <2659963+willstranton@users.noreply.github.com>
Default behavior changed at bazelbuild@0dd0da9. Also space out some paragraphs more to de-densify content. Closes bazelbuild#27750. PiperOrigin-RevId: 840133157 Change-Id: Icf233c4617da7c06a5f6256f14571f703b726f24 Commit bazelbuild@4b98a9d Co-authored-by: Greg Estren <gregce@google.com>
…azelbuild#27881) ModuleExtensionId is not comparable by itself and needs a comparator. Before this fix, I was able to crash Bazel with this message: ``` FATAL: bazel crashed due to an internal error. Printing stack trace: java.lang.ClassCastException: class com.google.devtools.build.lib.bazel.bzlmod.ModuleExtensionId cannot be cast to class java.lang.Comparable (com.google.devtools.build.lib.bazel.bzlmod.ModuleExtensionId is in unnamed module of loader 'app'; java.lang.Comparable is in module java.base of loader 'bootstrap') at com.google.common.collect.NaturalOrdering.compare(NaturalOrdering.java:29) at com.google.common.collect.ImmutableSortedMap.lambda$fromEntries$0(ImmutableSortedMap.java:536) at java.base/java.util.TimSort.countRunAndMakeAscending(Unknown Source) at java.base/java.util.TimSort.sort(Unknown Source) at java.base/java.util.Arrays.sort(Unknown Source) at com.google.common.collect.ImmutableSortedMap.fromEntries(ImmutableSortedMap.java:528) at com.google.common.collect.ImmutableSortedMap.fromEntries(ImmutableSortedMap.java:495) at com.google.common.collect.ImmutableSortedMap.copyOfInternal(ImmutableSortedMap.java:474) at com.google.common.collect.ImmutableSortedMap.copyOf(ImmutableSortedMap.java:372) at com.google.devtools.build.lib.bazel.bzlmod.BazelLockFileModule.afterCommand(BazelLockFileModule.java:135) at com.google.devtools.build.lib.runtime.BlazeRuntime.afterCommand(BlazeRuntime.java:734) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:711) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:257) at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:607) at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$0(GrpcServerImpl.java:677) at io.grpc.Context$1.run(Context.java:566) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) ``` Fixes bazelbuild#27522 Closes bazelbuild#27816. PiperOrigin-RevId: 840671327 Change-Id: Ia7eb05be4fd099f0e2711065faf4c344d45265b5 Commit bazelbuild@b2587f6 Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>
…azelbuild#27875) Fixes bazelbuild#25260 Implements the request made [here](bazelbuild#25205 (comment)) and moves the default output root on macOS from `/private/var/tmp` to `$HOME/Library/Caches`. May constitute an incompatible change, though on a cursory glance I didn't see any precedent for incompatible startup options in source. Closes bazelbuild#25262. PiperOrigin-RevId: 840604399 Change-Id: I391b805b2c8e99d3ffe44a76e916d1c9839225e5 Commit bazelbuild@13da9a1 Co-authored-by: Aaron Sky <aaronsky@skyaaron.com>
…elbuild#27887) In `.bazelrc` files, you can now conditionally import .rc files based on the current bazel version. For example: ``` # Use new feature flags available in 6.0: try-import-if-bazel-version >=6.0.0 %workspace%/configs/features.rc # Use legacy flags in older versions: try-import-if-bazel-version <7.0.0 %workspace%/configs/legacy.rc ``` The format is: `try-import-if-bazel-version <conditional-operator><version> <config-file>` `conditional-operator` can take one of 7 different operators: 1) `>` 2) `>=` 3) `<` 4) `<=` 5) `==` 6) `!=` 7) `~` The tilde (~) operator follows the semantics of npm's tilde ranges: https://docs.npmjs.com/cli/v6/using-npm/semver#tilde-ranges-123-12-1 The implementation follows the proposal by meteorcloudy@ in bazelbuild#24043 (comment) with the exception that the comparison operator and the version string are joined together (no space inbetween). Fixes bazelbuild#24043 Closes bazelbuild#27675. PiperOrigin-RevId: 840844695 Change-Id: Ibd10cb8c3fc1b2a23faa90fa4a94cba862e5d7a9 Commit bazelbuild@d61a021 Co-authored-by: Will Stranton <2659963+willstranton@users.noreply.github.com>
Better clarifies the difference between configuration-based options (`FragmentOptions`) and others. PiperOrigin-RevId: 840742919 Change-Id: I3c85cb3d5400b3277dda44e78f0bc73001aa697d Commit bazelbuild@9997810 Co-authored-by: gregce <gregce@google.com>
…ld#27895) Document counter-intuitive behavior with `exec` selects on toolchain config rules. Based on the discussion in the issue: bazelbuild#27623 Closes bazelbuild#27709. PiperOrigin-RevId: 834965287 Change-Id: I8b1edb2b4438907be0f00dd53d7d0e0f9629f5f1 Commit bazelbuild@3e02568 Co-authored-by: Fabian Brandstetter <21087362+FaBrand@users.noreply.github.com>
…ild#27903) This fixes an issue where repositories injected with `--inject_repository` can invalidate `MODULE.bazel.lock` by changing the order innate `use_repo_rule` extensions are registered. For example; ```starlark # //:MODULE.bazel local_repository = use_repo_rule("@//:defs.bzl", "local_repository") local_repository(name = "repo") ``` ```shell bazel mod dump_repo_mapping '' --inject_repository=my_repo=%workspace%/other_repo # Bazel 9+ # {"my_repo":"+local_repository+my_repo","repo":"+local_repository2+repo",...} # Bazel 8 # {"my_repo":"+_repo_rules+my_repo","repo":"+_repo_rules2+repo",...} ``` Handling repository injections later changes how collisions with apparent repository names from innate extensions (`use_repo_rule(...)(...)`) and module extensions (`use_extension(...).__(...)`) are reported. To permit debugging (and improve collision investigation in general) the error message now refers to the incoming and existing repo name definition. For example; ```diff -Error in use_repo: The repo name 'my_repo' is already being used by --inject_repository at <builtin> +ERROR: The repo name 'my_repo' cannot be defined by --inject_repository at <builtin> as it is already defined by a use_repo() call at /___/MODULE.bazel:2:9 ``` Closes bazelbuild#27795. PiperOrigin-RevId: 841889374 Change-Id: I01a4500ae5580338f96d593323c1d680fdbb7672 Commit bazelbuild@d564fc9 Co-authored-by: Jordan Mele <mele@canva.com>
This change fixes: 1. NullPointerException that occurred because the graph traversal logic only examined dependencies, failing to catch cases where the root module itself was the target. The fix handles "zero-length path" by adding the starting node to the result immediately, preventing the empty output that triggered the crash. 2. An infinite recursion issue in TextOutputFormatter when the dependency graph contains cycles formed by merging different paths. It adds a parentStack to track visited nodes and detect cycles dynamically. Fixes bazelbuild#27839 Closes bazelbuild#27879 PiperOrigin-RevId: 842137415 Change-Id: I131d3310b1fa939164379a50419f16ebb071ec55 --------- Co-authored-by: Yun Peng <pcloudy@google.com>
…zelbuild#27898) The implementation has to handle command line overrides on module repos subject to non-registry overrides specially to avoid a cycle when requesting the main repo mapping. Fixes bazelbuild#24617 Fixes bazelbuild#17128 RELNOTES: `--override_repository` now supports apparent repository names from the point of view of the main repository. An unknown apparent repo name will result in an error. Closes bazelbuild#27706. PiperOrigin-RevId: 841652194 Change-Id: Ic8d53e5c7beec0cf14a293c2d589ab08a5ba4f9d Commit bazelbuild@82c06ca Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…uild#27899) Per bazelbuild#27413 (review) Internally, Apache Commons Compress is at 1.28.0 already. This allows a workaround in GzFunctionTest to be removed. Closes bazelbuild#27851. PiperOrigin-RevId: 840705541 Change-Id: I44801214254560a917e63125d2345fb75cb22cb2 Co-authored-by: Will Stranton <2659963+willstranton@users.noreply.github.com>
By repurposing AbstractFileSystem as DiskBackedFileSystem, to be extended only by disk-backed subclasses. This encourages other subclasses to make an explicit decision on how to open files instead of inheriting a potentially incorrect implementation. As a bonus, we can also dispense with the duplicate profiling logic in UnixFileSystem. PiperOrigin-RevId: 837473162 Change-Id: I970d0a9711ecc583b1ecae650703d14d4a5db2a4 Commit bazelbuild@3673564
) - Split some paragraphs into smaller chunks. - Add more background context. - Re-order around common use frist (as users of existing definitions), writing your own definitions after. Current docs at https://bazel.build/extending/platforms. Closes bazelbuild#27766. PiperOrigin-RevId: 840819072 Change-Id: I426aa1891f15d54056392a39f8dd69b2fdb1b642 Commit bazelbuild@73608c7 Co-authored-by: Greg Estren <gregce@google.com>
…ng to handle ClosedByInterruptException correctly. (bazelbuild#27916) The comment in DiskBackedFileSystem explains why we're doing this and why this particular implementation was chosen. PiperOrigin-RevId: 842191458 Change-Id: I5f722f388dbe747442beeb34bc7a9458e05d2ddf
Cherry-picks the following commits: * bazelbuild@1bf36f0 * bazelbuild@6b57b60 * bazelbuild@586d38d * bazelbuild@d1cf1f0 * bazelbuild@718aab6
… needed (bazelbuild#27919) Shared Merkle subtree computations are now subject to a refcount-based cancelation scheme to ensure that they can continue through dynamic branch cancelations if another execution reuses them. Fixes bazelbuild#27777 Closes bazelbuild#27819. PiperOrigin-RevId: 842627529 Change-Id: I75139af0550fc97b22de13635e061e91b93f6e90 Commit bazelbuild@db2f824 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
It has been deprecated in favor of --remote_default_exec_properties since 2019. PiperOrigin-RevId: 837080711 Change-Id: I30d3350cd425711093dcec5c38b0e75cd7d75956
Get the target path from the `InputMetadataProvider` instead and avoid I/O that way. Closes bazelbuild#27886. PiperOrigin-RevId: 842394317 Change-Id: Idaa039ebbc0298bd8366e438358ccdc82370468f Commit bazelbuild@cb7dc52 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…azelbuild#27927) When multiple `module_interfaces` are specified on a single `cc_library`, the individual compilation actions form a DAG based on `import`s between these modules. Consider the following situation: * `a.cppm` imports `b.cppm`, both of which are in the `module_interfaces` of a single `cc_library`. * Building the target populates the action cache with an entry for `a.pcm` that stores `b.pcm` as a discovered input. * Now edit `a.cppm` and `b.cppm` so that `b.cppm` imports `a.cppm` and `a.cppm` no longer imports `b.cppm`. * Build again (optionally after a shutdown). Before this commit, this resulted in an action cycle since during action cache checking, Bazel would reuse or look up the inputs discovered in the previous build, thus introducing an edge from `a.pcm` to `b.pcm`. Together with the newly discovered edge from `b.pcm` to `a.pcm`, this resulted in a cycle. This is fixed by not requesting the previously discovered inputs (either retained in memory or in the action cache) if the mandatory inputs changed. In the case of C++20 modules, this is sufficient since the modmap file, which lists all transitive `.pcm` files required for compilation, is a mandatory input. As part of this change, `MetadataDigestUtils.fromMetadata` had to be modified to always return a byte array of proper digest length, even if called with an empty map, to match the assumptions of the action cache. This change is pretty much Fabian's PR bazelbuild#27492 with a tiny fix added on top (not returning from computeMandatoryInputsDigest() early on valuesMissing() if inErrorBubbling() is true) Closes bazelbuild#27492. PiperOrigin-RevId: 842733471 Change-Id: I48fa2c0bceb888dcb58db29d50c30719b2122c5d (cherry picked from commit cb9bd86) Closes bazelbuild#27544
…#27931) This removes the ExperimentalGrpcRemoteExecutor class entirely to replace it with GrpcRemoteExecutor. Mark the `--experimental_remote_execution_keepalive` flag as no-op. Fix bazelbuild#26055 Closes bazelbuild#26231. PiperOrigin-RevId: 842678454 Change-Id: I0a1a36668f193b9b6e6b3bf4f39740a1463c43fd Commit bazelbuild@e0f94dd Co-authored-by: Son Luong Ngoc <sluongng@gmail.com>
This PR contains 2 commit(s). 1)This attribute is no longer available by default and is breaking this rule at HEAD Found via debugging Bazel 9 with https://github.com/bazeltools/bazel_jar_jar/blob/660b5517e56562af9e7d6168b45886b7d786da5f/test/deps.bzl#L4 RELNOTES: None PiperOrigin-RevId: 842722721 Change-Id: I1959531b821121283fb476714e78fa3370d0a05a Commit bazelbuild@3f2295c 2)RELNOTES: None PiperOrigin-RevId: 843143069 Change-Id: Id849ec7e6490409781ccae9a0eccca2650028fee Commit bazelbuild@d9a9ffe --------- Co-authored-by: pcloudy <pcloudy@google.com>
This PR implements part of the [Execution Platform Scoped Spawn Strategies](https://github.com/bazelbuild/proposals/blob/2b717b19fe805c405576c4feb9ffc6b772068898/designs/2023-06-04-exec-platform-scoped-spawn-strategies.md) proposal. It adds a new flag `--allowed_strategies_by_exec_platform` which permits filtering spawn strategies for spawns execution platform. ## Example ```ini # //.bazelrc # Default strategies (order sensitive) build --spawn_strategy=remote,worker,sandboxed,local # Mnemonic targeted strategy override (order sensitive) build --strategy=BAR=remote,sandboxed # Host platform allowed strategies build --allowed_strategies_by_exec_platform=@platforms//host:host=local,sandboxed,worker # Remote platform allowed strategies build --allowed_strategies_by_exec_platform=//:remote_platform=remote ``` For an action with mnemonic `FOO` configured for the host platform (`@platforms//host:host`), it will resolve `worker,sandboxed,local` as it's spawn strategy candidates. * `remote` was eliminated as a candidate (not in allow list for platform). * Order from `--spawn_strategy` was preserved. For an action with mnemonic `BAR` configured for the host platform (`@platforms//host:host`), it will resolve `sandboxed` as it's spawn strategy candidate. * `remote` was eliminated as a candidate (not in allow list for platform). * Mnemonic override applied, leaving `sandboxed` as the final candidate. For an action with mnemonic `BAR` configured for the remote platform (`//:remote_platform`), it will resolve `remote` as it's spawn strategy candidate. * `sandboxed` was eliminated as a candidate (not in allow list for platform). * Mnemonic override applied, leaving `remote` as the final candidate. If no spawn strategy candidate remains after filtering, the standard error will be logged. ``` ERROR: /workspaces/___/BUILD.bazel:3:22: _description_ [for tool] failed: _mnemonic_ spawn \ cannot be executed with any of the available strategies: []. Your --spawn_strategy, \ --genrule_strategy, --strategy and/or --allowed_strategies_by_exec_platform flags are probably \ too strict. Visit bazelbuild#7480 for advice ``` Closes bazelbuild#27667.
) Always use the HTTP downloader in this case as it supports these URLs. Fixes bazelbuild#26810 Closes bazelbuild#27799. PiperOrigin-RevId: 843590344 Change-Id: Ia28da102b29d828edd95ea0c1fb9f50ff70d3d85 Commit bazelbuild@65574f3 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…bazelbuild#29701) (bazelbuild#29889) ### Description When checking external repos for external modifications, don't use ctime as it is sensitive to hardlink count changes (e.g. with `--experimental_use_hermetic_linux_sandbox`) and thus causes false positives. ### Motivation Fixes bazelbuild#29590 ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: External repos are no longer refetched whenever they contribute inputs to an action using the hermetic Linux sandbox. Closes bazelbuild#29701. PiperOrigin-RevId: 934309229 Change-Id: I22aa96959047496f5b71234cd2363ca98619a903 Commit bazelbuild@57b14c4 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…mp (bazelbuild#29876) (bazelbuild#29915) ### Description ### Motivation A repo fetched into the repo contents cache is materialized in the output base as a symlink into the cache. When the cache lives under /tmp but outside the output base (e.g. when relocated via `--repository_cache` or `--repo_contents_cache`), the symlink target wasn't made available inside a sandbox that uses a hermetic `/tmp`. Fixes bazelbuild#29649 ### Build API Changes No ### Checklist - [ ] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: None Closes bazelbuild#29876. PiperOrigin-RevId: 935046365 Change-Id: Idbc024b147665b764582b6a15606508bd9bbf428 Commit bazelbuild@66f3856 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…uild#29869) This change extracts setup logic for remote repo content caching from the gRPC path so it can be shared with the HTTP/disk path. Remote repo content caching is currently only initialized for gRPC cache backends. The `--experimental_remote_repo_contents_cache` flag is a no-op when using a cache backend like GCS. No - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). RELNOTES: Fixed repo content caching for non-gRPC cache backends. Closes bazelbuild#29744. PiperOrigin-RevId: 928545624 Change-Id: Ib811c85d0c2462f91a8a8114640d58edf5b4c0a3 (cherry picked from commit d1e894e) Fixes bazelbuild#29747 Co-authored-by: Bryce Lampe <brycelampe@gmail.com>
…azelbuild#29811) (bazelbuild#29947) ### Description Also simplify is by making it a record. ### Motivation `ParsedFlagValue#equals` and `#hashCode` did not consider `flagAliasMapping`, possibly resulting in incorrect change pruning. ### Build API Changes No ### Checklist - [ ] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: None Closes bazelbuild#29811. PiperOrigin-RevId: 931256807 Change-Id: I9d1efac44149121d0a3d0ca9d74a6fe01d006ef3 Commit bazelbuild@cc071f9 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
) When a remotely cached repo is partially materialized since it contains action inputs, Skyframe provides the metadata of these inputs after following all symlinks as it does for all source files. Since (chains of) symlinks need to be preserved and materialized in the same way as if the repo rule ran locally or was fully materialized as a dependency of another repo rule, `AbstractActionInputPrefetcher` is changed to walk the full chain of symlinks behind a prefetched external repo path replant every link verbatim on the host file system. This proves very difficult to get right for repos with symlinks pointing into the main repo as such symlinks would cross over into the host filesystem. For this reason such symlinks now exclude a repo from being cached remotely. Note that the local repo contents cache already doesn't cache such repos. Since main repo files are readily available, this is considered a minor loss in usefulness only. Fixes bazelbuild#29656 Fixes bazelbuild#29515 RELNOTES: The remote repo contents cache now correctly materializes chains of symlinks as action inputs, but no longer supports symlinks into the main repository. Closes bazelbuild#29767. PiperOrigin-RevId: 936532134 Change-Id: I48b63f6fd2d5f57e139175e3ed578d65f6abbd08 (cherry picked from commit c310e3e) Fixes bazelbuild#29816
…lbuild/b… (bazelbuild#29944) …azel/pull/29902) On Linux, gRPC defaults to epoll which uses a different set of options. bazelbuild#29879 worked on Mac but was a no-op on Linux. Closes bazelbuild#29902. PiperOrigin-RevId: 936173912 Change-Id: I3796427c6f9fe887b0557bb56466e176af25f160 <!-- Thank you for contributing to Bazel! Please read the contribution guidelines: https://bazel.build/contribute.html --> ### Description <!-- Please provide a brief summary of the changes in this PR. --> ### Motivation <!-- Why is this change important? Does it fix a specific bug or add a new feature? If this PR fixes an existing issue, please link it here (e.g. "Fixes bazelbuild#1234"). --> ### Build API Changes <!-- Does this PR affect the Build API? (e.g. Starlark API, providers, command-line flags, native rules) If yes, please answer the following: 1. Has this been discussed in a design doc or issue? (Please link it) 2. Is the change backward compatible? 3. If it's a breaking change, what is the migration plan? --> No ### Checklist - [ ] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes <!-- If this is a new feature, please add 'RELNOTES[NEW]: <description>' here. If this is a breaking change, please add 'RELNOTES[INC]: <reason>' here. If this change should be mentioned in release notes, please add 'RELNOTES: <reason>' here. --> RELNOTES: None Commit bazelbuild@46c5e78 Co-authored-by: vadim <vadim@buildbuddy.io>
…azelbuild#29986) This change introduces measured peak memory usage (peak RSS) to the SpawnMetrics and exposes it in the execution log. Fixes bazelbuild#29872 Specifically: - Updates spawn.proto to add measured_memory_peak_bytes to SpawnMetrics. - Updates SpawnMetrics (Java) to store and propagate measuredMemoryPeakBytes. - Updates SpawnResult to populate this metric from the parsed execution statistics (memoryInKb). - Updates SpawnLogContext to write this metric to the execution log proto. - Updates linux-sandbox (C++) to read peak memory from cgroups (if available) and override the rusage maxrss value, providing high-accuracy aggregate memory usage for sandboxed actions on Linux. - Updates SpawnLogContextTestBase to verify the metric is correctly logged. PiperOrigin-RevId: 937171286 Change-Id: I4f563c7b8a361bb778f97e20cb3cd2ce9266f469 Commit bazelbuild@b3126b5 Co-authored-by: twerth <twerth@google.com>
…e dirs (bazelbuild#29692) (bazelbuild#29998) ### Description Checking for package traversals of source directories did not take the repo name into account, leading to false positives. ### Motivation Fixes bazelbuild#29688 ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: None Closes bazelbuild#29692. PiperOrigin-RevId: 937911172 Change-Id: I9405df2e845ec22a4fd9b10182cf9f0749b0931d Commit bazelbuild@b46d244 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…zelbuild#30005) Modified from the original via: * using a distinct version for the action cache that can't collide with future versions at HEAD * including bazelbuild@741ee01 (see bazelbuild#29976 for context) Original commit message follows: In-memory outputs (--experimental_inmemory_{dotd,jdeps}_files) are kept only in memory and never written to disk. shouldTrustMetadata() gates trust of a remote output's metadata on shouldDownloadOutput(), which under --remote_download_outputs=all is unconditionally true, so an in-memory output's still-alive remote metadata is distrusted and its action is re-executed on every incremental build (e.g. every CppCompile, which emits a .d, and every Java compile, which emits a .jdeps): the action re-executes, hits the remote cache, and re-downloads its outputs, turning a no-op build into a full re-validation pass. This is a regression from bazelbuild#27291, which moved these outputs from on-disk (local, always trusted) to in-memory (remote) metadata. An in-memory output is indistinguishable from an ordinary remote output that an earlier build left unmaterialized (e.g. under --remote_download_outputs=minimal): both are remote with no contents proxy. So mark in-memory outputs when their metadata is injected, carry the bit through the action cache (bumping its version), and trust a marked output via its TTL in shouldTrustMetadata(). This is read from the metadata alone, so it holds in both ActionCacheChecker and FilesystemValueChecker, and on a cold server. Ordinary unmaterialized remote outputs stay unmarked and are still re-fetched, so switching to --remote_download_outputs=all correctly materializes them. The regression test covers a warm rebuild, a rebuild after a server restart, and the minimal -> all transition. Also manually tested with a full Chromium build - the second build with a warm server went from re-executing 72k compile actions in ~216s down to ~0.9s with 0 actions. Verified again with a cold server, which also re-ran 0 actions. Fixes bazelbuild#29313 RELNOTES: None Closes bazelbuild#29993. PiperOrigin-RevId: 937976000 Change-Id: I1fb24ae4db407101ebb4f4c203f258bd7c15f51f (cherry picked from commit 4c92958) Fixes bazelbuild#29995 Co-authored-by: Philipp Wollermann <philwo@google.com>
…ndows (bazelbuild#29921) (bazelbuild#29984) ### Description `AsExecutablePathForCreateProcess` shortens an executable's path with `GetShortPathNameW` so it fits `CreateProcessW`'s `MAX_PATH`-bound command line, and currently fails outright when shortening can't bring it under `MAX_PATH`. This change adds a fallback: it launches the executable through `CreateProcessW`'s _lpApplicationName_, which is not subject to the `MAX_PATH` limit, in extended-length (`\\?\`) form. Supplying _lpApplicationName_ also lifts the `MAX_PATH` limit from the executable part of _lpCommandLine_, so the original path is kept there as `argv[0]`. Shortening is still attempted first, so the change is a pure superset of the current behavior: paths that shorten today are launched unchanged, and only the currently-failing ones take the fallback. The fallback is limited to absolute, normalized paths to plain executables: - a batch file is not a valid _lpApplicationName_ as it must be run through a command interpreter, - neither is a non-normalized path because its `.`/`..` components would survive unresolved in the extended-length path. The fix relies on `\\?\` bypassing `MAX_PATH` at the path-parsing layer, **independently** of the _LongPathsEnabled_ registry value and the _longPathAware_ manifest. ### Motivation Fixes bazelbuild#19710. Avoid errors like: ``` ==================== Test output for //some/configuration/issues/invalidconfig:invalidconfig_test: ERROR(tools/test/windows/tw.cc:1307) ERROR: src/main/native/windows/process.cc(82): WaitableProcess::Create(C:\user\execroot\_main\bazel-out\x64_windows-fastbuild-ST-000000000001\bin\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe.runfiles\_main\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe): ERROR: src/main/native/windows/util.cc(292): AsExecutablePathForCreateProcess(C:\user\execroot\_main\bazel-out\x64_windows-fastbuild-ST-000000000001\bin\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe.runfiles\_main\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe): ERROR: src/main/native/windows/util.cc(262): GetShortPathNameW(\\?\C:\user\execroot\_main\bazel-out\x64_windows-fastbuild-ST-000000000001\bin\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe.runfiles\_main\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe): cannot shorten the path enough ERROR(tools/test/windows/tw.cc:1517) Failed to start test process (arg: C:\user\execroot\_main\bazel-out\x64_windows-fastbuild-ST-000000000001\bin\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe.runfiles\_main\some\configuration\issues\invalidconfig\invalidconfig_test_\invalidconfig_test.exe) ================================================================================ ``` Shortening fails in two situations, both handled here: - 8dot3 short-name creation is disabled on the volume (common in containers, microsoft/Windows-Containers#507), so `GetShortPathNameW` is a no-op and a long runfiles executable path stays over `MAX_PATH`, - the path is nested so deeply that even fully 8dot3-shortened it still overflows `MAX_PATH`. Prior art: Rust's `std::process::Command` passes a verbatim `\\?\` path as _lpApplicationName_ for a >`MAX_PATH` executable and special-cases batch files the same way: - rust-lang/rust#87704, - rust-lang/rust#92519. ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: Fixed Bazel being unable to run a Windows executable whose path is too long to shorten under `MAX_PATH`, e.g. when 8dot3 short names are disabled (microsoft/Windows-Containers#507). Closes bazelbuild#29921. PiperOrigin-RevId: 937426905 Change-Id: Ia48d41e000622fc122d1b258ad15daa5018be75e Commit bazelbuild@4a80d43 Co-authored-by: Rgis Desgroppes <rdesgroppes@gmail.com>
…nts cache (bazelbuild#30031) (bazelbuild#30037) ### Description When a repo is restored from the remote repo contents cache, its files are injected into the in-memory overlay file system and `.bzl`/`REPO.bazel` files are additionally prefetched to the native file system as regular files. `RemoteExternalFileSystem#getInputStream` relies on this by redirecting reads of such files to the native file system, but didn't follow symlinks before doing so. ### Motivation Fixes bazelbuild#29656 (comment) ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: Fixed spurious "No such file" errors with the remote repo contents cache when a repo contains a symlink to a `.bzl` file. Closes bazelbuild#30031. PiperOrigin-RevId: 938614073 Change-Id: I1decc39029a973cf1984d7518242d0ffc56c0711 Commit bazelbuild@ad1793f Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…a flag (bazelbuild#30019) (bazelbuild#30040) ### Description bazelbuild#29692 made the package-boundary check for source directories in `RecursiveFilesystemTraversalFunction` repo-aware. Besides fixing the false positives reported in bazelbuild#29688, it also made the check effective for source directories in external repos, which is a breaking change. This PR gates that new enforcement behind `--incompatible_check_external_repo_source_dir_package_boundary` (default off). When the flag is off, package boundary crossings by source directories in external repositories are skipped entirely rather than reverting to the old main-repo lookup. As a result, bazelbuild#29688 stays fixed regardless of the flag value. ### Motivation bazelbuild#29688 (comment) ### Build API Changes Yes: adds the command-line flag `--incompatible_check_external_repo_source_dir_package_boundary`. 1. Has this been discussed in a design doc or issue? N/A 2. Is the change backward compatible? Yes. The flag defaults to off, preserving the pre-bazelbuild#29692 (9.1.x) behavior for source directories in external repositories. 3. If it's a breaking change, what is the migration plan? Enabling the flag (eventually flipping the default to on in a future major release) restores the strict, repo-aware enforcement. ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: Source directories in external repositories are temporarily allowed to cross package boundaries. This will be disallowed in the future, gated by the new `--incompatible_check_external_repo_source_dir_package_boundary` flag. Closes bazelbuild#30019. PiperOrigin-RevId: 938703055 Change-Id: I5bf779d6ea0c10579564d77fc9eb0adf6aaacd03 (original commit 3ad37ef) Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…elbuild#30045) (bazelbuild#30060) ### Description `RemoteExternalOverlayFileSystem` now drops all in-memory repo state when running a command without a cache configured. ### Motivation Fixes a crash in the remote repo contents cache (`--experimental_remote_repo_contents_cache`) when a repo injected by a cachedbuild is accessed by a later command that runs without a remote cache: ``` java.lang.RuntimeException: Unrecoverable error while evaluating node 'SINGLE_EXTENSION_EVAL:@@gazelle+//:extensions.bzl%go_deps' (requested by nodes 'SINGLE_EXTENSION:@@gazelle+//:extensions.bzl%go_deps') at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:552) at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:435) at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source) at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) Caused by: net.starlark.java.eval.Starlark$UncheckedEvalException: NullPointerException thrown during Starlark evaluation (module extension @@gazelle+//:extensions.bzl%go_deps) at <starlark>.path(<builtin>:0) at <starlark>.deps_from_go_mod(/root/.cache/bazel/_bazel_root/f3c1d9cbc2a31229f3cc0e3922038c0e/external/gazelle+/internal/bzlmod/go_mod.bzl:179) at <starlark>._go_deps_impl(/root/.cache/bazel/_bazel_root/f3c1d9cbc2a31229f3cc0e3922038c0e/external/gazelle+/internal/bzlmod/go_deps.bzl:448) Caused by: java.lang.NullPointerException: Cannot invoke "java.util.concurrent.ExecutorService.submit(java.util.concurrent.Callable)" because "this.materializationExecutor" is null at com.google.devtools.build.lib.remote.RemoteExternalOverlayFileSystem.lambda$ensureMaterialized$0(RemoteExternalOverlayFileSystem.java:301) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source) at com.google.devtools.build.lib.remote.RemoteExternalOverlayFileSystem.ensureMaterialized(RemoteExternalOverlayFileSystem.java:298) at com.google.devtools.build.lib.bazel.repository.starlark.StarlarkBaseExternalContext.getPathFromLabel(StarlarkBaseExternalContext.java:2363) at com.google.devtools.build.lib.bazel.repository.starlark.StarlarkBaseExternalContext.getPath(StarlarkBaseExternalContext.java:1573) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at net.starlark.java.eval.MethodDescriptor.call(MethodDescriptor.java:261) at net.starlark.java.eval.BuiltinFunction.positionalOnlyCall(BuiltinFunction.java:74) at net.starlark.java.eval.Starlark.positionalOnlyCall(Starlark.java:878) at net.starlark.java.eval.Eval.evalPositionalOnlyCall(Eval.java:789) at net.starlark.java.eval.Eval.evalCall(Eval.java:706) at net.starlark.java.eval.Eval.eval(Eval.java:566) at net.starlark.java.eval.Eval.execAssignment(Eval.java:120) at net.starlark.java.eval.Eval.exec(Eval.java:300) at net.starlark.java.eval.Eval.execStatements(Eval.java:82) at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:66) at net.starlark.java.eval.StarlarkFunction$ArgumentProcessor.call(StarlarkFunction.java:577) at net.starlark.java.eval.StarlarkCallable.positionalOnlyCall(StarlarkCallable.java:112) at net.starlark.java.eval.Starlark.positionalOnlyCall(Starlark.java:878) at net.starlark.java.eval.Eval.evalPositionalOnlyCall(Eval.java:789) at net.starlark.java.eval.Eval.evalCall(Eval.java:706) at net.starlark.java.eval.Eval.eval(Eval.java:566) at net.starlark.java.eval.Eval.execAssignment(Eval.java:120) at net.starlark.java.eval.Eval.exec(Eval.java:300) at net.starlark.java.eval.Eval.execStatements(Eval.java:82) at net.starlark.java.eval.Eval.execFor(Eval.java:137) at net.starlark.java.eval.Eval.exec(Eval.java:308) at net.starlark.java.eval.Eval.execStatements(Eval.java:82) at net.starlark.java.eval.Eval.execFor(Eval.java:137) at net.starlark.java.eval.Eval.exec(Eval.java:308) at net.starlark.java.eval.Eval.execStatements(Eval.java:82) at net.starlark.java.eval.Eval.execFunctionBody(Eval.java:66) at net.starlark.java.eval.StarlarkFunction$ArgumentProcessor.call(StarlarkFunction.java:577) at net.starlark.java.eval.StarlarkCallable.positionalOnlyCall(StarlarkCallable.java:112) at net.starlark.java.eval.Starlark.positionalOnlyCall(Starlark.java:878) at com.google.devtools.build.lib.bazel.bzlmod.RegularRunnableExtension.runInternal(RegularRunnableExtension.java:293) at com.google.devtools.build.lib.bazel.bzlmod.RegularRunnableExtension.lambda$run$0(RegularRunnableExtension.java:236) at com.google.devtools.build.skyframe.WorkerSkyKeyComputeState.lambda$getOrStartWorker$0(WorkerSkyKeyComputeState.java:172) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:128) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:80) at java.base/java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run(Unknown Source) at java.base/java.lang.VirtualThread.run(Unknown Source) ``` ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: Fixed a crash with `--experimental_remote_repo_contents_cache` when running without a remote or disk cache. Closes bazelbuild#30045. PiperOrigin-RevId: 939694605 Change-Id: Ie6fa660d0d623e586abd5016a0aa3d0b66c64136 Commit bazelbuild@0464e54 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…bazelbuild#29951) OutputArtifactConflictTest flakes under Skymeld + nokeep_going because evaluation can abort with a TopLevelAspectsKey as the only processed analysis error. A local stress run on origin/master reproduced the flake with: ```text bazel test --nocache_test_results --runs_per_test=50 \ --test_filter=testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution \ //src/test/java/com/google/devtools/build/lib/buildtool:OutputArtifactConflictTest ``` The failing run looked like this: ```text FAIL: ...OutputArtifactConflictTest (shard 3 of 3, run 32 of 50) testConflictErrorAndUnfinishedAspectAnalysis_mergedAnalysisExecution [skymeld=true,minimizeMemory=true,keepGoing=false] expected to contain any of: [//x:y, //x/y:y, //x:fail_analysis] but was : [] ``` The existing error processor skipped non-configured-target keys in this path, so BEP consumers could miss the AnalysisFailureEvent even though the analysis error was reported to the event handler. Report failures against the top-level aspect's base configured target so the failure is emitted on the same path used for configured target keys. Update the BEP integration test expectation for the additional aborted event that is now reported. Local validation: - Reproduced the baseline flake on origin/master (a300dd5): failed on run 32/50 with empty `analysisFailures`. - Verified the CopyOnWriteArrayList-only test-listener change still failed on run 10/50 with the same empty-list assertion. - Verified this patch without the listener-list change passed 50/50 and 100/100 local stress runs of the same filtered test. Closes bazelbuild#29850. PiperOrigin-RevId: 936417007 Change-Id: If2ea9a4b157fb6d646608a8adfb2c5f4d12de5c4 Commit bazelbuild@8d8826c Co-authored-by: Son Luong Ngoc <sluongng@gmail.com>
…nts proxy (bazelbuild#30004) (bazelbuild#30127) ### Description Adds an integration test for the `FileContentsProxy` optimization of the remote repo contents cache (PR bazelbuild#28654) and proves that the part of the PR reverted in 741ee01 isn't relevant - it only applies to outputs, but external repo files are always source files. ### Motivation PR bazelbuild#28654 added the contents proxy optimization for the remote repo contents cache but landed without a test. This adds coverage so the injection-site behavior can't silently regress. Closes bazelbuild#29976 ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: None Closes bazelbuild#30004. PiperOrigin-RevId: 941950605 Change-Id: Iaf770c416d7496d002db44d77105b36975dded6a Commit bazelbuild@a4191ed Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…ows (bazelbuild#30159) This test is failing due to a clang update. PiperOrigin-RevId: 943497618 Change-Id: Ib2993df0e60e39c7563280c3393ce69f0c3f0dcf (Original commit 7460f6d)
…azelbuild#30150) Redo logic for bfs of VendorCommand.java ### Description Adding the nodes of the graph into visited when we're dequeing while checking if they exist while enqueing means that we might accidentally add the same nodes multiple time, leading to OOM on big production graphs. Example: ``` target keys = [A, B] nodes = [A, B] A -> C B -> C ``` We visit A, then add C to nodes. Then we visit B, also adding C to nodes under the old logic (duplicated). This patch changes the logic so that we add a node to visited whenever we enqueue so this doesn't happen. See https://en.wikipedia.org/wiki/Breadth-first_search#Pseudocode for pseudocode. ### Motivation Fix oom bug ### Build API Changes No ### Checklist - [NA] I have added tests for the new use cases (if any). - [NA] I have updated the documentation (if applicable). ### Release Notes RELNOTES: None Closes bazelbuild#30088. PiperOrigin-RevId: 943722164 Change-Id: I153afd4281a3c138fe401a145b0d6465fa198e50 Commit bazelbuild@01407e4 Co-authored-by: Jasmine Tang <jjasmine@igalia.com>
Cherry-picks fbbdfad onto release-9.2.0. Do not cache (parts of) the CommandContext in BuildEventServiceGrpcClient. The client may outlive the command that started it. Caching it in a StreamContext, on the other hand, is fine because these are command-scoped. PiperOrigin-RevId: 836649514 Change-Id: Id21577f480597dc085fdc73aa102870fdfabd268 Co-authored-by: tjgq <tjgq@google.com>
…ath mapping (bazelbuild#30193) Merkle subtrees for tree artifacts are deduplicated across actions by content. When such a shared subtree is uploaded for remote execution and one of its blobs is missing from the CAS, the resulting `CacheNotFoundException` is annotated with the exec path of whichever action happened to compute the shared subtree. With path mapping, other actions that join the deduplicated computation can then no longer map the lost input back to their own inputs, so the `BulkTransferException` surfaces as a fatal `EXEC_IO_EXCEPTION` instead of a `LostInputsExecException` and action rewinding never runs. This change adds the unmapped exec path to the cache key of a tree artifact to avoid a shared upload in this case. Other artifact types are not affected and the fast path of a remote cache check without uploads is unaffected. Fixes bazelbuild#30065. RELNOTES: Fixed a spurious remote-execution failure with `--experimental_output_paths=strip` where a lost input in a tree artifact shared between actions could not be recovered by action rewinding. Closes bazelbuild#30085. PiperOrigin-RevId: 944321975 Change-Id: If36408a56ab8137b915970827ebbd7a91d9ad9c1 (cherry picked from commit 412de86) Closes bazelbuild#30103
…azelbuild#30194) `RepoEnvironmentFunction` no longer reads the entire repository environment from a single whole-map `PrecomputedValue.REPO_ENV` node. This is crucial for incremental efficiency: changing a single variable must not invalidate consumers of the others. Reading the whole environment as one Skyframe node instead would result in widespread invalidation of all dependent nodes whenever any variable changes. Crucially, change pruning is not fully effective here as it doesn't resurrect nodes that aren't part of the current evaluation, but every evaluation marks all transitive dependents as dirty. When an IDE/BSP server runs `bazel mod` with a different value for an incidental variable such as `TERM` than the user's interactive `bazel build`, the next build reloads targets in repositories — despite nothing depending on `TERM`: ```console $ TERM=xterm bazel build //src:bazel # up-to-date, 1 internal action $ TERM= bazel mod show_repo @@grpc+ # interleaved command, different TERM $ TERM=xterm bazel build //src:bazel # 522 packages reloaded, 25107 targets reconfigured ``` This happens because `bazel mod` requests all module extensions, which depend on the repo env and thus see the modified repo env in their transitive closure, but does *not* request the repos generated by these module extensions, so they are just marked as dirty and not resurrected by change pruning. Note that repos are *not* refetched, but their Skyfunctions rerun and thus invalidate all dependents (e.g. package loading). Fixes bazelbuild#29956, which was introduced by 01407ce RELNOTES: Modifying environment variables no longer causes spurious reloading of packages in external repositories. Closes bazelbuild#29946. PiperOrigin-RevId: 944163161 Change-Id: Ie2e3d29a564331321a5fdabcdc0f08806f778123 (cherry picked from commit 93867c2) Closes bazelbuild#29957
…r stub" (bazelbuild#30043) (bazelbuild#30182) ### Description This reverts commit f5aeb71, relanding bazelbuild#29822. Additional fix: the change makes `//src/tools/launcher:launcher` depend on a `windows_resources` target, but its embedded `@bazel_tools` copy had no rc toolchain for `toolchain_type`, so the postsubmit `bazel_bootstrap_distfile_test` (which rebuilds `@bazel_tools` from source, unlike presubmit) could not bootstrap Bazel on Windows. The resource-compiler package lived at `src/main/res`, outside `@bazel_tools`, so embedding it there required `@bazel_tools`-qualified labels and hand-staging the package in the lockfile test. Move it to `tools/res` instead, beside the other embedded toolchains, so it is embedded and staged like its siblings and labels resolve relative to the loading repo. Each module must then register its own rc toolchain, so this registers `@local_config_winsdk//:all` and `empty_rc_toolchain` in `src/MODULE.tools` (as in the root `MODULE.bazel`) and drops the now redundant `empty_rc_toolchain` registration from `testBuildLaunchersWithClangClOnWindows`. Verified with `bazel_bootstrap_distfile_test` on Windows: ``` bazel.exe test //src/test/shell/bazel:bazel_bootstrap_distfile_test [...] INFO: Build completed successfully, 2 total actions //src/test/shell/bazel:bazel_bootstrap_distfile_test PASSED in 732.8s Executed 1 out of 1 test: 1 test passes. ``` ### Motivation Same as bazelbuild#29822: > On Windows, manifest-less PE executables whose name contains keywords like "install", "setup", "update", or "patch" trigger the "[UAC installer-detection heuristic](https://www.advancedinstaller.com/user-guide/vista-uac.html)" and demand elevation, even when the target does nothing privileged. > > When running such executables without admin privileges, one indeed face errors resembling: > ``` > FATAL: ExecuteProgram(C:\[...]\install.exe) failed: ERROR: src/main/native/windows/process.cc(189): CreateProcessW("C:\[...]\install.exe"): The requested operation requires elevation. > (error: 740) > ``` > > Declaring `asInvoker` in the launcher stub bypasses the heuristic, similar to rust-lang/cargo#393 (`cargo update` triggered the same heuristic): cargo now ships a `windows.manifest.xml` embedded via its build script. > > Because the manifest lives in the PE resource section, `launcher_maker`'s byte-copy propagates it to every generated binary automatically. Fixes bazelbuild#29819. (for real) ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES[NEW]: Windows launcher stubs now embed an `asInvoker` UAC manifest, preventing "The requested operation requires elevation. (error: 740)" for targets whose name matches the installer-detection heuristic. Closes bazelbuild#30043. PiperOrigin-RevId: 944416012 Change-Id: I37a388ad42ea8192af454a978de39dbe4ff2d100 Commit bazelbuild@e37255a --------- Co-authored-by: Rgis Desgroppes <rdesgroppes@gmail.com> Co-authored-by: iancha1992 <heec@google.com>
…ontents cache (bazelbuild#30161) (bazelbuild#30190) ### Description This is meant as a temporary stopgap for the many symlink-related issues found with the remote repo contents cache and includes tests found while debugging bazelbuild#30149. The new and old tests are parameterized in a test suite variable to keep the diff small and enable future experimentation with improved symlink support. As part of bazelbuild#30160, I'll investigate a more conceptual approach to symlink resolution that unifies the logic with that in RemoteActionFileSystem. ### Motivation Fixes bazelbuild#30149 ### Build API Changes No ### Checklist - [x] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: The remote repo contents cache no longer caches or restores repos with cross-repo symlinks, thus avoiding a large surface area for bugs. The effort to reenable this support is tracked by bazelbuild#30160. Closes bazelbuild#30161. PiperOrigin-RevId: 944468477 Change-Id: I4a435beaadb7c2df3e11acd910814736d2fb32c0 Commit bazelbuild@e65fe06 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…ld#30047) (bazelbuild#30167) ### Description ### Motivation This is necessary to support C++20 modules with path mapping enabled. Further rules_cc fixes are necessary before this can be tested end-to-end. ### Build API Changes No ### Checklist - [ ] I have added tests for the new use cases (if any). - [ ] I have updated the documentation (if applicable). ### Release Notes RELNOTES: None Closes bazelbuild#30047. PiperOrigin-RevId: 944024894 Change-Id: I4e2c41b74fbce5157463a6710a6ea9846e4f0766 Commit bazelbuild@416e9b3 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
…meException (bazelbuild#28305) (bazelbuild#30212) When attempting to run a Bazel command, Bazel server will shutdown and exit with an unexpected error when the [downloader config](https://bazel.build/reference/command-line-reference#common_options-flag--downloader_config) is invalid: ``` $ echo "#test\nfoo" > bazel_downloader.cfg $ cat bazel_downloader.cfg #test foo $ bazel-dev build //... --downloader_config=bazel_downloader.cfg < exits immediately with the Bazel server killed > $ echo $? 37 ``` `37` corresponds to `Unhandled Exception / Internal Bazel Error` as per the [docs](https://bazel.build/run/scripts#exit-codes). This PR introduces changes that will let catch and rethrow `UrlRewriterParseException` before `Closer.rethrow()` to prevent it from being wrapped in `RuntimeException`. ``` $ bazel-dev-fixed query //... --downloader_config=bazel_downloader.cfg ... ERROR: Failed to parse downloader config at bazel_downloader.cfg:2: Unable to parse: foo $ echo $? 2 ``` Closes bazelbuild#28305. PiperOrigin-RevId: 891501161 Change-Id: I3cbfe2e2cdcad14c6b5e8210e2e1b8e999eba0ae Commit bazelbuild@45ea1f0 Co-authored-by: Alexey Tereshenkov <50622389+AlexTereshenkov@users.noreply.github.com>
Release Notes:
dmiller-figma
force-pushed
the
graphite-base/20
branch
from
August 15, 2026 05:39
a6781ba to
8220c61
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Base the Figma fork on upstream Bazel 9.2.0 and carry forward the existing shard-count compatibility change. Some fork consumers intentionally use more than 50 test shards.
Changes
Test Plan
USE_BAZEL_VERSION=9.2.0 bazel test //src/test/java/com/google/devtools/build/lib/analysis:RuleConfiguredTargetTest