Avoid running tests on platforms without all target constraints

[moroten]

Description of the feature request:

Background

The example below will allow running a test built for wasm+gpu on a platform without any GPU.

Consider the following test toolchain:

toolchain(
    name = "wasm_on_linux",
    exec_compatible_with = ["@platforms//os:linux", "@platforms//cpu:x86_64"],
    target_compatible_with = ["@platforms//cpu:wasm64"],
    toolchain = "@bazel_tools//tools/test:empty_toolchain",
    toolchain_type = "@bazel_tools//tools/test:default_test_toolchain_type",
)

With this toolchain, tests can run on a Linux amd64 machine when being built for wasm. The problem is that if the target platform has the constraints ["@platforms//cpu:wasm64", "//gpu:small"], it will also match.

One can consider adding another toolchain:

toolchain(
    name = "wasm_on_linux",
    exec_compatible_with = ["@platforms//os:linux", "@platforms//cpu:x86_64", "//gpu:small"],
    target_compatible_with = ["@platforms//cpu:wasm64", "//gpu:small"],
    toolchain = "@bazel_tools//tools/test:empty_toolchain",
    toolchain_type = "@bazel_tools//tools/test:default_test_toolchain_type",
)

If that toolchain is registered earlier, things will work, as long as no-one tries to build with the constraint //gpu:large in their target platform.

Suggestion

Extend the meaning of use_target_platform_constrains in the toolchain rule to the following algorithm:

exec_constraints = toolchain.exec_compatible_with
if toolchain.use_target_platform_constrains:
    unmatched_constraints = target_platform.constraints - toolchain.target_compatible_with
    exec_constraints += unmatched_constraints

Existing rules with use_target_platform_constrains=True already have empty target_compatible_with and exec_compatible_with, so that will behave the same. In the example above, the GPU constraint will be added to the exec_compatible_with constraints as it is not matched by the target_compatible_with constraints.

Which category does this issue belong to?

Configurability

What underlying problem are you trying to solve with this feature?

Avoiding running tests on platforms that do not fulfill all the target constraints.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 9.1.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

Have you found anything relevant by searching the web?

Implementation tracking issue #25160 for Execution platform selection for test actions in multi-platform builds.

The issue was found in buildbarn/bb-storage#357 where the @rules_go//go/toolchain:cgo_off constraint is problematic.

Any other information, logs, or outputs that you want to share?

No response

[gregestren]

If that toolchain is registered earlier, things will work, as long as no-one tries to build with the constraint //gpu:large in their target platform.

Could #29655 help, so a platform that sets //gpu:small or //gpu:large must implicitly set some other //gpu:enabled that the toolchain can declare compatibility with?

[moroten]

If that toolchain is registered earlier, things will work, as long as no-one tries to build with the constraint //gpu:large in their target platform.

Could #29655 help, so a platform that sets //gpu:small or //gpu:large must implicitly set some other //gpu:enabled that the toolchain can declare compatibility with?

No, #29655 is unrelated.

If one looks at the my_sh_test example in the design document, a Linux target platform with //gpu:small set, or any other constraint value set, may execute on a platform with just a //platforms/os:maxos set. One can't expect the rule author of my_sh_test to know about all platforms and constraint values other people create.

[fmeum]

I think this algorithm makes a lot of sense. I actually considered exactly this behavior in a first version of the design, but then decided to be more cautious and limit it to empty target/exec constraints for now. Happy to review a PR (or I can send one next week).