Add a _runfiles_enabled marker file to runfiles trees

[fmeum]

Description

Adds an empty _runfiles_enabled file to every runfiles tree, which is present in a runfiles directory if and only if that directory has been fully materialized.

Since the marker is an ordinary runfiles entry, it is created by exactly those mechanisms that also materialize all other runfiles: the symlink tree, the sandbox and remote execution. A runfiles directory that only contains the manifest and the workspace subdirectory, which is what RunfileSymlinksMode.SKIP leaves behind, does not contain it.

Bazel's own consumers are switched over to it:

• tools/test/test-setup.sh and tools/test/windows/tw.cc derive RUNFILES_MANIFEST_ONLY from it instead of passing through the value set by Bazel, which in particular means that they now unset it if the sandbox or remote execution materialized the runfiles directory even though --enable_runfiles is off.
• The Windows launcher checks for the marker file at runtime instead of reading the symlink_runfiles_enabled launch info key, which is baked in at analysis time and thus suffers from the same imprecision. The key is no longer read (rules may keep emitting it).

RUNFILES_MANIFEST_ONLY continues to be set by BazelRuleClassProvider and TestRunnerAction for runfiles libraries that don't recognize the marker file yet.

SourceManifestAction's GUID is bumped because the action's output changes without its key otherwise doing so.

The second commit removes the only case in which a fully materialized runfiles directory can hold stale contents. The marker file cannot detect that case, so it would otherwise be pushed onto every consumer that starts trusting the directory. RunfilesTreeUpdater skipped recreating a tree if the output manifest in it matched the input manifest. The manifest only records the set of runfiles, so this implies that the tree is up to date only if it consists of symbolic links, whose targets are the authoritative files. It does not on a file system that materializes symlinks as copies (Windows without --windows_enable_symlinks), where a runfile that is modified without the manifest changing leaves a stale copy behind. Since linkManifest() makes the output manifest a symbolic link on every file system that supports them, the check would never both apply and be sound, so it is dropped. To keep recreating the tree cheap where it used to fire, SymlinkTreeHelper now keeps an existing copy that is still up to date instead of deleting and rewriting it, which also stops a single modified runfile from causing the entire tree to be copied again.

Motivation

Fixes #24929.

Runfiles libraries have to prefer the manifest over the runfiles directory because they can't tell whether the directory is usable. The information they get instead, RUNFILES_MANIFEST_ONLY, is derived from the value of --enable_runfiles alone, so it is wrong whenever a spawn's inputs are materialized by something other than the local symlink tree.

With the marker file, a runfiles library can reliably detect that the directory is usable and prefer it, which makes runfiles resolve to paths inside $RUNFILES_DIR rather than to their sources — the behavior requested in #24929 — and is also faster than parsing the manifest.

This is the Bazel-side prerequisite. The follow-up in rules_cc, rules_java and rules_python is for Runfiles.Create to apply the following order, which preserves today's behavior exactly when the marker file is absent, i.e. on older Bazel versions:

1. if RUNFILES_DIR is set and contains _runfiles_enabled, use the directory;
2. otherwise, if RUNFILES_MANIFEST_ONLY=1 and a manifest is available, use the manifest;
3. otherwise, if RUNFILES_DIR is set, use the directory;
4. otherwise, use the manifest.

This is based on #20964, adapted to the current code base.

Build API Changes

No

Checklist

• I have added tests for the new use cases (if any).
• I have updated the documentation (if applicable).

Release Notes

RELNOTES: Runfiles trees now contain an empty _runfiles_enabled file, which is present in a runfiles directory if and only if that directory has been fully materialized. Runfiles libraries can use it to decide whether to resolve runfiles paths against the directory instead of the manifest.