How does the allocator know what back-ends to load?

[gfxstrand]

Because our strategy for allocation involves going to the different back-ends and asking them to allocate, we need some strategy for knowing which back-ends care to be involved at any given point. I really would like to avoid the Vulkan loader solution of "just load everything". Some companies are known to ship rather large drivers and are liable to make libdrm2_vendor.so a symlink to their libGL.so. I don't want doing something with the allocator to mean you always pay the cost of loading ~1GiB worth of shared libraries into your address space just because you have lots of big heavy GL drivers installed.

This means that you need some way of determining when a particular back-end will be needed. We have a few different options:

1. The client specifically asks us to load back-ends
2. We figure out, based on capabilities, which back-ends are relevant.

[gfxstrand]

I know @robclark made some comments in the direction of (1) but I think I'd prefer (2) if we can make it work. One option would be to provide, with your capbailities, some sort of tag saying "You may need to load libdrm2_foo.so in order to allocate"

[robclark]

I think if we don't do (1) then that means knowing how map capabilities (by which I think you mean use-cases??) to a set of devices. Which gets us back to the gralloc problem, and not being suitable for a non-device-specific distro. Unless I misunderstand what you mean (since in "liballoc" parlance a set of capabilities are what you get from the backend/driver in response to assertion_t (set of generic parameters) plus usage_t)

My reasoning behind (1) is that the client already knows what set of APIs and/or devices are involved in whatever it is that it is trying to do.

[cubanismo]

If we go with the json-style backend description files GLVND and Vulkan are using, we could ask vendors to include some vendor->backend filename mapping in the json file.

However, I think the harder part is going from -> <backend .so name>. How does a client enumerate without first loading <backend .so name>? Presumably it could get them from some other API. However, if it has to ask the allocator library for them, the allocator library probably needs to open the backends to list them. That's the cache 22 that caused Vulkan and GLVND+EGL to load all available backends. FWIW, as a vendor with a giant monolithic lib[E]GL thing that does Vulkan too, we're aware it's not ideal for load times. There's some re-factoring planned. I view it as an implementation problem rather than Vulkan/GLVND+EGL interface design problem, but I'm still a bit wary of the "load everything" design anyway.

[robclark]

I guess I was hoping to avoid the bootstrap problem by declaring that the client somehow automagically figured out which APIs (via glvnd/vk-loader/etc), and v4l devices, and whatever else, since it would have needed to figure that out somehow already.. and now all it needs to do is figure out optimal buffer format to use across the various devices.

Partly that was based on my thinking that this was something that supplements gbm rather than replaces gbm. I'm still thinking about that part.. if you needed a liballoc instance before you had an EGLDisplay that maybe changes things.. I'm still thinking about your comments in #1.

[gfxstrand]

I don't think that magically loading things requires that we have a particularly complicated mapping. You get a list of capability/restriction tags from each driver/API in the negotiation. With that mapping, they can also provide an identifier specifying which liballoc back-end they're associated with. Then liballoc just loads those back-ends only.

We could possibly also go off of something where the json files have a list of vendor IDs. If there are caps with one of those vendor ids, then you should load that allocator. For the "common" cases such as ION, the back-end could be builtin and wouldn't need loading.

[cubanismo]

If the libraries loaded are based on those the capabilities require, then you have a chicken-and-egg problem: At least one instance of the allocator would have had to load some set of libraries to generate the capability set in the first place, and how does that bootstrap instance know which backends to load?

[robclark]

@jekstrand I punted on trying to define exactly how loader works (json or whatever).. that is kinda a boring detail once we figure out whether liballoc backend is the thing that bootstraps the API or just something that comes later once API is already bootstrapped.. I guess important question is whether you have the API (EGLDisplay or gbm_device or whatever) or fd first, or whether "liballoc" is your entry point into everything. I'd been assuming the former.. but depending on the conclusion of #1 that might change.

@cubanismo yeah, loading lib based on cap-set you get from the alloc_dev_t doesn't work.. when you need to get the cap-set from the alloc_dev_t in the first place.. I'm a bit in limbo on this until we figure out whether "liballoc" (or whatever it ends up getting called) replaces gbm or sits beside gbm. I have some vague conflicting half-baked thoughts on that. But my feeling until now is that for most devices, we actually have to open the device to query it to figure out caps-set so there isn't much we can do without already having the device fd (for things that are purely kernel interface) or some sort of userspace device handle (gbm or egl or whatever) first. And if that is the case "liballoc" loader doesn't have to open everything since whatever you want is already opened.

tl;dr: I think we solve #1 first, and then how the "liballoc" loader works falls out as a conclusion of that.

PS. whether chicken or egg came first, I think the scientific answer is neither :-P

[gfxstrand]

@robclark, I think the question of where you get the caps plays into this as well (#6). I would ideally like to see you get the caps from your GL or Vulkan driver most of the time and only call into the allocator to do the allocation. I don't want every GL user to have to go find their device node. For scanout, you may want some sort of open("/dev/dri/card0") interface but you definitely know the actual device since you're going to use it for KMS. However, I think this is the exception rather than the rule.

[robclark]

@jekstrand right, I mentioned as much in USAGE.md: "In cases where the underlying device file is not exposed (such as OpenMAX or OpenCL, or OpenGL when not running on top of gbm) an extension would be required to retrieve the alloc_dev_t from that particular API.".. so I think we are on the same page there.

[cubanismo]

FWIW, right now the prototype loads backends like GLVND+EGL and libvulkan do based on json config files. When it gets a device file as proposed in USAGE.md, it asks each available backend if it supports that file, and inits a device instance on the first one that returns true.

I ran into some problems implementing this: The NVIDIA driver doesn't allocate out of a single FD. It allocates out of two or more depending on the type of allocation being made. Only one of those FDs is device-specific. The other ones can dispatch to all devices, so they aren't useful in identifying a device. However, we still need them to init our driver. This is sort of the fundamental problem with trying to integrate our driver into mechanisms like systemd that hand us an FD and expect us to init graphics drivers from it, so I probably should have anticipated it.

Given that, I'm kind of coming back to the idea of exposing at least one init path that works off of device UUIDs or some other stateless identifier.

Of course, there could always be the "extract an alloc_device_t from EGL/Vulkan/etc." path that @robclark proposed, but I think that would cause problems for something like gralloc using the allocation library as a backend, and you'd also get the circular dependency of how does EGL/Vulkan/etc. get an alloc_device_t in the first place? I'd prefer the answer to that not be "Some vendor-specific back door."

@jekstrand As far as getting caps from Vulkan, that sounds fine, but EGL/GL mostly lack the mechanism to describe caps well. I think you'd end up with "Everything for graphics" or "nothing", plus maybe a special case for OES_external_texture. Maybe that's OK, but if we're really going to expect people to build Vulkan drivers ON TOP of the allocator library, I think it has to be able to define usage itself that is at a minimum as expressive as what Vulkan can do. I know for our HW, we would still prefer even slightly finger-grained usage than what Vulkan exposes now, but that may be something addressable as a future Vulkan extension as well. Then of course, there are all the non-graphics devices where the allocator library will likely be the only game in town for capability queries, at least at first. And maybe it's not bad to try to centralize all of this logic in one library + driver module per vendor, even if it is some upfront work.