where does this sit with respect to gbm

[robclark]

1. gbm has some extra stuff, like surfaces, for gluing together with egl.. but gbm_surface perhaps doesn't make as much sense for other APIs like v4l
2. gbm is (mostly) using fourcc, but if "liballoc" uses dataformat this might not fit so nicely as a (backward compatible) extension to gbm

Possibly this new thing just ends up sitting on the side as a separate API which only deals with allocation, with some small extension to gbm to get from a alloc_bo_t to a struct gbm_bo? You could also maybe just do that via importing the dma-buf fd into gbm, but it would be convenient for buffers allocated via the gl/vk driver's "liballoc" backend, to be able to recover the original pipe_resource or equivalent.

The other possibility is new set of EGL extension, plus maybe some optional (ie. not supported by all backends) features like surfaces, so this new thing replaces gbm. Although there are existing users who use gbm to do gl apps on "bare metal" so this would mean that mesa has to support both gbm and new thing as potential args to eglGetDevice()..

[cubanismo]

This gets into some topics I had hoped to postpone, but I do have some thoughts. Allocating the surfaces is just the first step towards solving the set of problems I listed in the XDC presentation. After that, you need to figure out how to use them. Part of that is beyond the scope of an allocation API that doesn't do any sort of rendering on its own, but part of it relies on the allocation API to be the arbiter of certain properties. Ultimately, I think the below will be split into multiple separate issues, but since they're all relevant to your questions, listing them here for now:

• How do we access these images in GL/Vulkan/EGL - In the simplest sense, I believe a single allocation should be bound to a single Vulkan image or EGLImage. What people making window systems are going to want to do is assemble swapchains (in Vulkan terms) out of those images. In Vulkan, that's easy: Swapchains already are collections of specially allocated images. However, one use case I like to keep in mind is the vendor-agnostic EGL platform driver mechanism Miguel presented on at XDC. This relies on having a lower-level EGLSurface you can wrap with another EGLSurface exposed to the user. When the user calls CreatePlatformSurface(myNativePlatform), the vendor-agnostic platform layer creates one of these lower-level EGLSurface objects instead, backing it with some surfaces it allocates. In our initial sample code, we used EGLStream + Stream producer surfaces. I'd propose introducing a simpler surface type: EGLImageSurface. This would be an EGLSurface made up of previously allocated EGLImage objects. Basically, a vehicle to bind an EGLImage to framebuffer 0. Calling SwapBuffers on these surfaces would simply pump some buffers within the GL/GLES drivers. The idea is basically borrowed from GBM surfaces, but moves the construct into EGL, since EGL is the only API that really needs the concept.
• How are surface state transitions managed - Once you start using images with non-trivial capabilities, you need to make sure they're in the right state for the engine you're using. On some hardware, the same allocation can be used as a render target, a texture, and a display surface, and a video encode source surface, but not at the same time. Since SwapBuffers() may not be used, and doesn't know which usages it is transitioning between, a separate mechanism is needed. This roughly corresponds to the issue Vulkan image layout transitions solve. My proposal for a while has been something along the lines of an "eglFlushImage()" call. The questions is what parameters such a call takes. I would propose that the allocator library exposes a set of states and their corresponding usage bits. The states would be opaque, but the application could pick one based on the desired usage. For example, if the application requested display, render target, and texture usage, then queried for the available states, it might get back two states, one with the display usage associated with it, and one with the render target and texture usage associated with it.
• How are surface accesses synchronized? - This is arguably out of the allocator's scope entirely. The app could choose various methods to perform synchronization. For example, if all the usage occurs in EGL, EGLSync objects would be sufficient. The obvious choice here would be android/linux fence fds. However, those are only available on Linux, and only on very modern kernels from my understanding.

Coming back around to the broader question, given the above, my view is that GBM could be implemented as a wrapper on top of the allocator library. However, I don't see current GBM sitting side-by-side with the allocator library, and I don't view the current GBM<->EGL paths as correct solutions. Even with a GBM implementation sitting on top of the allocator API defined here, there wouldn't be sufficient information available to implement an EGL window surface on top of a GBM surface, nor to import a foreign allocator surface into GBM and begin displaying it, for the reasons outlined above in the state transitions bullet.

[robclark]

This does end up sounding more like a "buffer manager" than simply an allocator. Although possibly the only thing really needed that isn't already covered elsewhere (gbm, fence-fd's and the related EGL extensions, etc) is surface state transition.. I guess we have half of that in the form of usage_t. We just need an API to do something like: int buffer_transition_to(alloc_bo_t *bo, alloc_dev_t *dev, usage_t *new_state).. plus maybe some api to get/set fence fd's??

[cubanismo]

Not sure if you were implying this, but I don't think the allocator library shouldn't actually perform the transitions. That requires a graphics command buffer in most cases, and I don't want the allocator to have to get in the business of setting one of those up and handling all the synchronization with other command buffers it entails. It should just describe the transitions in a format other APIs can consume. For example, Vulkan already has extensive infrastructure for doing state transitions. The allocator could just export some data that can be passed into a Vulkan pipeline barrier as an extension.

[robclark]

I was assuming that allocator backend was just a shim that calls into driver to do whatever. But yeah, I guess if that was exposed as a "liballoc" API then we'd need some sort of context object that maps to an EGLContext/etc, and that probably isn't a direction we want to go..

Originally was thinking this way because something like a v4l camera/decoder/etc doesn't have some userspace API.. but I guess practically it is only the gpu (gl/vk/cl) drivers that would need state transitions. So I guess if we only care about this when the buffer transitions between a gpu driver and something else (other gpu driver, or something that isn't a gpu driver) then handling it within the gpu api makes sense.

[cubanismo]

Yes, currently we manage to get away with only the GPU APIs performing transitions. That means compositors tends to require a GPU API if they want to re-purpose buffers behind the client's back (E.g., take a buffer intended for scanout and start using it as a video encode source), but that hasn't been an onerous requirement for anyone thus far. For us, the GPU is the only engine that can perform such a transition fast enough to be useful anyway.

I'd be interested to hear whether this is a problematic assumption for other vendors.

[lynxeye-dev]

Agree that we want to keep layout transitions out of the liballoc API.

A more generic wording for shunting this to other parts of the stack is to say the native API of the buffer producer should provide the means to do the layout transitions. If the producer is advanced enough to require layout transitions it should provide a way to do them. Currently this matches only GPUs, but may extend to video DSPs or whatever in the future.