Skip to content

Don't mutate an action's input map after it has been executed - #30693

Draft
fmeum wants to merge 1 commit into
bazelbuild:masterfrom
fmeum:no-input-map-mutation-after-execution
Draft

Don't mutate an action's input map after it has been executed#30693
fmeum wants to merge 1 commit into
bazelbuild:masterfrom
fmeum:no-input-map-mutation-after-execution

Conversation

@fmeum

@fmeum fmeum commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Description

ActionInputMap is thread-compatible, but the action filesystem reads the map belonging to the action being executed, and that map is read from other threads after the action has finished:

  • build events referencing the filesystem's Paths are uploaded asynchronously by ByteStreamBuildEventArtifactUploader;
  • with --remote_cache_async (on by default), RemoteExecutionService#doUploadOutputs runs on a background thread and digests the action's stdout/stderr, which live under the exec root and are resolved through the action filesystem.

ActionExecutionFunction#addDiscoveredInputs mutated that map from the Skyframe thread after execution, for actions that discover new inputs while running (CppCompileAction via its .d file, LtoBackendAction, StarlarkAction with an unused inputs list). Since putIfAbsent links a new entry into its hash bucket before filling in the corresponding slot, a concurrent reader can observe a half-initialized entry:

java.lang.NullPointerException: Cannot invoke "String.hashCode()" because "this.paths[index]" is null
	at com.google.devtools.build.lib.actions.ActionInputMap.getIndex(ActionInputMap.java:196)
	at com.google.devtools.build.lib.actions.ActionInputMap.getInput(ActionInputMap.java:343)
	at com.google.devtools.build.lib.skyframe.ActionInputMetadataProvider.getInput(ActionInputMetadataProvider.java:128)
	at com.google.devtools.build.lib.actions.DelegatingPairInputMetadataProvider.getInput(DelegatingPairInputMetadataProvider.java:103)
	at com.google.devtools.build.lib.remote.RemoteActionFileSystem.statInternal(RemoteActionFileSystem.java:686)
	at com.google.devtools.build.lib.remote.RemoteActionFileSystem.getFastDigest(RemoteActionFileSystem.java:467)
	...
	at com.google.devtools.build.lib.remote.ByteStreamBuildEventArtifactUploader.readPathMetadata(ByteStreamBuildEventArtifactUploader.java:203)

Inputs that are only discovered after execution now go into a separate map, so the map the action filesystem reads is immutable from the moment the action starts executing. The pre-execution call is unchanged — nothing else can observe the filesystem yet at that point. The action cache update, the only consumer of the late-discovered metadata, receives a provider that covers both maps.

Motivation

Fixes a flaky internal-error crash. The new test in TimestampBuilderTest fails without the change (the input metadata provider handed to the action reports the late-discovered input afterwards) and additionally asserts that the input is still tracked by the action cache.

Fixes #30683.

Build API Changes

No

Checklist

  • I have added tests for the new use cases (if any).
  • I have updated the documentation (if applicable).

Release Notes

RELNOTES: Fixed a rare crash with NullPointerException in ActionInputMap.getIndex when an action discovers additional inputs while executing.

`ActionInputMap` is thread-compatible, but the action filesystem reads the one
belonging to the action being executed, and it is read from other threads after
the action has finished: build events referencing its `Path`s are uploaded
asynchronously, and with `--remote_cache_async` (on by default) the action
result is uploaded on a background thread.

`ActionExecutionFunction#addDiscoveredInputs` mutated that map from the Skyframe
thread after execution. Since `putIfAbsent` links an entry into its hash bucket
before filling in the corresponding slot, a concurrent reader can observe a
half-initialized entry:

    java.lang.NullPointerException: Cannot invoke "String.hashCode()" because "this.paths[index]" is null
    	at com.google.devtools.build.lib.actions.ActionInputMap.getIndex(ActionInputMap.java:196)
    	at com.google.devtools.build.lib.actions.ActionInputMap.getInput(ActionInputMap.java:343)
    	...
    	at com.google.devtools.build.lib.remote.RemoteActionFileSystem.statInternal(RemoteActionFileSystem.java:686)
    	at com.google.devtools.build.lib.remote.RemoteActionFileSystem.getFastDigest(RemoteActionFileSystem.java:467)
    	...
    	at com.google.devtools.build.lib.remote.ByteStreamBuildEventArtifactUploader.readPathMetadata(ByteStreamBuildEventArtifactUploader.java:203)

Keep inputs that are only discovered after execution in a separate map, so that
the map the action filesystem reads is immutable from the moment the action
starts executing. The action cache update, the only consumer of those inputs,
gets a metadata provider that covers both maps.

Fixes bazelbuild#30683.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NullPointer: Bazel fails with non-descriptive error

1 participant