Don't reference an action filesystem from build events - #30695
Draft
fmeum wants to merge 4 commits into
Draft
Conversation
fmeum
force-pushed
the
bep-test-outputs-off-action-fs
branch
2 times, most recently
from
August 12, 2026 19:49
7edcfeb to
8463435
Compare
`TestAttempt` reported the test outputs as `Path`s on the test action's `RemoteActionFileSystem`. Build events are uploaded asynchronously, so that kept the filesystem - and with it the metadata of every input of the test action, including all of its runfiles - reachable until the transport had drained. Report them on the local filesystem instead. `TestRunnerAction` keeps resolving declared outputs through the path resolver, since an action filesystem may be the only place they exist; the conversion happens in `StandaloneTestStrategy`, which both of Bazel's paths funnel through - `processTestAttempt` for an executed test and `newCachedTestResult` for a locally cached one. Coverage data is always downloaded, but the test log is not with `--remote_download_minimal`, so `TestAttempt` now also carries its metadata; the build event artifact uploader prefers that over reading the file and reports the blob without downloading it. Progress towards bazelbuild#24527.
`CompletionContext` asks the `OutputService` for an `ArtifactPathResolver` so that `TargetCompleteEvent` and `NamedArtifactGroup` can turn artifacts into `Path`s. `RemoteOutputService` answered with a `RemoteActionFileSystem` wrapping the target's `ActionInputMap`. Since build events are uploaded asynchronously, that filesystem stayed reachable until the last transport had drained. It is no longer needed. Both events supply the artifact's `FileArtifactValue` alongside the path, and since 9a849d9 `ByteStreamBuildEventArtifactUploader` uses that metadata instead of stat'ing the file, so it never consults the filesystem to learn a digest, size or type. Drop the override and let the default `ArtifactPathResolver.IDENTITY` produce the same paths. The `OutputService` hook itself is left in place: other implementations, in particular those whose action filesystem takes full control of the output base, may still need to hand the build event stream a filesystem of their own. Also let `NamedArtifactGroup` retain just the `ArtifactPathResolver` instead of the whole `CompletionContext`. It stores eagerly expanded (artifact, metadata) pairs and consulted the context for nothing else, but there is one such event per nested set node of every target's output groups, so it was keeping the target's `ActionInputMap` alive along with it. Progress towards bazelbuild#24527.
The primary output path and the stdout/stderr paths reported by `ActionExecutedEvent` came from the action's `ArtifactPathResolver`, so a published event kept the action filesystem alive until the transport had drained it. The stdout/stderr paths additionally made the action result upload read the action filesystem from a background thread with `--remote_cache_async`. Report them on the local filesystem instead, but only where that is known to work: an output service whose action filesystem takes full control of the output base has no local action log directory - `ExecutionTool` skips creating it when `supportsLocalActions()` is false - and its outputs may not exist locally either. For those, keep reporting the action filesystem paths. Progress towards bazelbuild#24527.
`ByteStreamBuildEventArtifactUploader` is only used by Bazel, whose action filesystem is always `RemoteActionFileSystem`, so the invariant can be checked unconditionally there rather than having each filesystem opt in. The check fails hard rather than reporting a bug: `BugReport` returns early for a binary that isn't a released Blaze, and only logs otherwise, so a violation would go unnoticed in Bazel. With the invariant in place, the uploader no longer needs to ask an action filesystem whether a file is stored remotely: a build event that references such a file supplies its metadata, which already carries that information. Progress towards bazelbuild#24527.
fmeum
force-pushed
the
bep-test-outputs-off-action-fs
branch
from
August 12, 2026 20:08
8463435 to
b85d86d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A build event outlives the action or target it refers to: it is uploaded asynchronously, so anything it references stays on the heap until the last transport has delivered it. Several events referenced an action filesystem, which in turn retains everything needed to serve that action's inputs. This removes those references and then makes the property self-enforcing.
Four commits, each reviewable on its own:
Test build events.
TestAttemptreported test outputs on the test action'sRemoteActionFileSystem, retaining the metadata of every input of the test action — for a test, its whole runfiles tree.TestRunnerActionis shared with Blaze and keeps resolving declared outputs through the path resolver, since an action filesystem may be the only place they exist; the conversion to local paths happens inStandaloneTestStrategy, which is Bazel-only and which both paths funnel through —processTestAttemptfor an executed test, andnewCachedTestResult, whichTestRunnerAction#actionCacheHitcalls throughTestActionContext. Coverage data is always downloaded, but the test log is not under--remote_download_minimal, soTestAttemptalso carries its metadata; the uploader prefers that over reading the file and reports a remote-only log as a bytestream URI. This resolvesTODO(b/199940216)for the one file that needed it.Completion events.
CompletionContextasked theOutputServicefor a path resolver purely soTargetCompleteEventandNamedArtifactGroupcould turn artifacts intoPaths, andRemoteOutputServiceanswered with aRemoteActionFileSystemover the target'sActionInputMap. All six call sites already have the artifact'sFileArtifactValue, which the uploader prefers, so Bazel now falls back toArtifactPathResolver.IDENTITY.NamedArtifactGroupadditionally drops itsCompletionContextfield — it stores eagerly expanded(artifact, metadata)pairs and used the context for nothing else, and there is one such event per nested set node of every target's output groups.ActionExecutedEvent. The primary output and stdout/stderr paths came from the action's path resolver. Reporting stdout/stderr locally also stops the action result upload from reading the action filesystem on a background thread under--remote_cache_async.Enforcement.
ByteStreamBuildEventArtifactUploaderis Bazel-only, and Bazel's action filesystem is alwaysRemoteActionFileSystem, so it checks every path it is handed and reports a non-fatal bug otherwise — no opt-in mechanism needed, and nothing in a shared class. With the invariant established, the uploader no longer has to ask an action filesystem whether a file is stored remotely: an event that references such a file supplies its metadata, which already says so.isRemoteFilegoes away with it.On other
OutputServiceimplementationsThe
OutputServicepath resolver hook is deliberately not removed, and commit 3 is conditional onActionFileSystemType#supportsLocalActions. An action filesystem that takes full control of the output base is the only place its outputs exist, andExecutionTooldoes not even create a local action log directory for it, so its stdout/stderr cannot be reported locally. The enforcement in commit 4 lives in the Bazel-only ByteStream uploader, so it does not constrain other implementations at all.Validation
The invariant check in commit 4 was run across
remote/...,buildtool/...,runtime/...,analysis/...,exec/...,skyframe/...andactions/...— 320 tests, all passing, with no production code path tripping it. The single initial violation wasByteStreamBuildEventArtifactUploaderTest#remoteFileShouldNotBeUploaded_actionFs, which handed the uploader an action filesystem path directly; it now expresses remoteness through metadata, the way production does.Commit 1 has a regression test in
StandaloneTestStrategyTest: it feedsnewCachedTestResulta test log on a stand-in action filesystem and asserts the resultingTestAttemptreports the same path on the real one.Note that
TrimTestConfigurationTest#flagOffDifferentTestOptions_ResultsInDifferentCTsis flaky independently of this change — it fails roughly 3 in 10 runs on unmodified master.Motivation
Progress towards #24527. Supersedes #30691 (commit 2 here) and the closed #30692.
Build API Changes
No
Checklist
Release Notes
RELNOTES: None