Run gfxstream end to end on macOS, guest to host over kumquat - #168
Open
utzcoz wants to merge 7 commits into
Open
Run gfxstream end to end on macOS, guest to host over kumquat#168utzcoz wants to merge 7 commits into
utzcoz wants to merge 7 commits into
Conversation
ANGLE's macOS sources are Objective-C++, which cc_library cannot compile, so they move into a dedicated angle_apple objc_library that the main libraries depend on when targeting macOS. The Vulkan back end is built against the Metal and IOSurface window system integration, and the platform selects gain a macOS arm so that the target graph resolves on Darwin. This produces libEGL.dylib and libGLESv2.dylib, which the GLES tests need in order to run on macOS at all.
macOS System Integrity Protection strips DYLD_LIBRARY_PATH from the environment, so a bare dlopen cannot find a library that was staged into a directory named there. Fall back to trying each LD_LIBRARY_PATH entry as a full path, which is not subject to that stripping. Needed so that tests can point the loader at a driver directory they staged themselves.
The external driver selected through VK_DRIVER_FILES on macOS is normally MoltenVK. With Metal argument buffers enabled, pipelines that bind an immutable YCbCr sampler, which is what the YUV compositor pipelines do, fail SPIR-V-to-MSL conversion and are never built. Compositing a planar-YUV layer then aborts the renderer. MVK_CONFIG_* variables are ignored by drivers that are not MoltenVK, so this is safe to set unconditionally on macOS.
CreateOSWindow on macOS is written in Objective-C++, which cc_library cannot compile. Move it into a dedicated objc_library that the test support library depends on when targeting macOS, and split the headers out so that both can include them.
Add gles_angle_vulkan_moltenvk, which uses ANGLE for GLES on top of the system MoltenVK for Vulkan. MoltenVK is provided at runtime rather than as a build artifact, so only the ANGLE libraries are staged as data. ANGLE looks for the Vulkan loader in its own module directory, and DYLD_LIBRARY_PATH is unavailable under System Integrity Protection, so the environment copies the ANGLE libraries and the loader into a writable directory and loads ANGLE from there. The staged copies inherit the read-only permissions of their build outputs, so each destination is removed before it is written; that also keeps staging idempotent, since it runs once per test in a process. The driver basenames differ by platform, and the filesystem calls now report their error_code instead of discarding it.
A portability-subset implementation such as MoltenVK requires the extension to be enabled when the portability enumeration flag is set, but requesting it on a device that does not report it fails device creation. Ask the device first.
The macOS Bazel job only built until now. Give it the host driver environment and run the Vulkan host tests against MoltenVK, and add a second job that runs the GLES framebuffer tests through ANGLE on MoltenVK. The reusable workflow grows the inputs those need: which driver environment to select, which test targets to run, and extra flags for the test step only. They are passed through the environment rather than expanded into the script, the way the existing inputs are, and globbing is disabled around the test command so that a value such as --test_arg=--gtest_filter=*Foo* survives verbatim. macOS runners now also install molten-vk and vulkan-loader, which together give the tests a Vulkan implementation to run against.
Contributor
Author
|
For dependencies like molten-vk and vulkan-loader, what's the best practice of gfxstream of using them? Installing outer dependency or submodule integration under third_party. @jmacnak @gurchetansingh PTAL. |
Collaborator
|
I don't see any virtgpu_kumquat changes in this MR, perhaps you missed them. We probably want the changes to land in Mesa3D first, and then figure out a way to bring-them back here for the end2end tests. I think you should try to get Kumquat working without Bazel build first, just with meson (it's simpler). For example, on Linux, we are able to run gfxstream-vulkan (not gfxstream GLES, which is being phased out anyways) over Kumquat just using the meson build: https://github.com/magma-gpu/rutabaga_gfx/#build-gfxstream-guest For MacOS, you will have fun challenges around:
That said, I do think there's some emulated copy mode gfxstream-vulkan uses on iOS using VkFlushMemory ranges: you'll have to check. But zero copy is the ideal. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the macOS host and GLES test PR. That one made the host side testable on macOS; this one adds the guest side, so the full guest-to-host path runs: the gfxstream guest driver talks to a kumquat server over the macOS IPC layer, and the host renders with ANGLE on MoltenVK.
Unlike the previous PR, this one does carry third_party/mesa patches.