Can I help out with improving the state of the Mesa driver?

[astillich]

As the title says. What needs to be done and how could I help out on the development side? I have experience in C and GL, but find it hard to figure out where to start. Also: the compile-time dependency to the grate version of libdrm was removed, but I still need to have it around at runtime, correct?

[kusma]

Hey. The major blocker right now is s proper fragment shader IR. I've actually carved out some time at work to take a crack at getting something working, but it's probably not going to get us all the way there.

If you have experience in writing compiler backends or are interested in learning about it, I would love some help here.

After that's done, the next major thing is wiring up texturing. This kinda depends on compiler work first.

Once those things are in place, we should actually be pretty good, and from there it's mostly upstreaming the driver, and the endless cycle of testing and fixing bugs. We might also want to set up some CI runners if possible, and do some hardware enablement for later versions of the GPU, and other missing features.

[digetx]

@astillich Hello, please tell us which Tegra device do you have.

The https://github.com/grate-driver/grate should be a good place for the start, compile it and run the tests. If everything appear to work fine, then take a look at https://github.com/grate-driver/grate/wiki and start hacking the code to get familiar with the basics. Be prepared that this may be not that easy as it may sound if you haven't touched Tegra hardware before, hence it's normal if it will take some time and effort.

The #tegra IRC channel should be a good place for chatting. You could ask anything in this thread as well, whatever works best for you.

The dependency on a customized libdrm was indeed removed, but the Tegra/Grate DRM code itself was moved inside of the mesa / opentegra / libvdpau drivers. This eased the development process a lot and removed maintenance burden. It's enough to have any generic libdrm installed, i.e. libdrm which is shipped with the Linux distro you're using will be good.

@kusma currently working on refreshing the mesa driver, adding fragment code generation and textures support. He has https://gitlab.freedesktop.org/kusma/mesa. I assume that the latest work should appear here in the master branch once it will be ready.

I rebased the older code base on a more recent version of mesa https://github.com/grate-driver/mesa/tree/21.0.3 The code is a proof-a-concept which can draw glxgears, it's a good place for starting the mesa development. Pull the code, compile it and check that glxgears work.

Mesa has a good documentation https://gallium.readthedocs.io/en/latest/, you should take a look at it, look at the code of other gallium drivers. Download the GLES spec and start wiring up the code generation and etc. Eventually you'll may need to set up a working environment to get command stream traces from the proprietary driver, we could discuss it later on.

[kusma]

I assume that the latest work should appear here in the master branch once it will be ready.

Yeah, or in some other branch in that repo. I'll let the relevant people know ;)

I rebased the older code base on a more recent version of mesa

I actually also rebased on an even more recent version, but I haven't tested that beyond seeing that it still compiles. I suppose I can share that as well. (edit: here)

[astillich]

I have a Nexus 7 2012 which has a Tegra30 SoC. Actually, I have several and the reason I'm offering help is because I want to get HW accelerated rendering running on these devices which should support a custom Qt/QML application for my self-built home automation system. The tablets will be used as stationary control panels. Reading your answers it seems that it's quite some way to go. I have limited time, so I'd like to invest it into getting things working and not in upstreaming code to Mesa or getting other devices to work.

I will run a custom postmarketOS build on the devices, I have already built the 21.0.3 grate fork of Mesa as a package. I wondered if I'd also need to package the custom libdrm for this to work, that's why I asked about it. Then I ran out of time (vacation was over), and I haven't gotten around to testing the package on the device yet.

I actually skimmed some of the Mesa documentation, but I'm still lost on what exactly needs to be done and how the differnt components work together.

I do have some experience with compiler backends, but in a less formal way from some experiments with my own scripting languages where it was all put together as needed. I don't have any experience doing that in a GPU context. For clarification: we are talking about translating NIR (or TGSI?) produced by Mesa from GLSL ES to something which can be sent to the GPU using libdrm to get it to the device via the kernel driver?

How can I capture command streams? Can I do that on a rooted android device or do I need to get a Linux system with the old NV tegra support working on them?

I guess the best way to move on would be to look at the VS IR and inspecting Kusma's fork? Some concrete pointers along the lines of "do X now, so that we can do Y" would be very much appreciated.

[digetx]

I have a Nexus 7 2012 which has a Tegra30 SoC. Actually, I have several and the reason I'm offering help is because I want to get HW accelerated rendering running on these devices which should support a custom Qt/QML application for my self-built home automation system. The tablets will be used as stationary control panels. Reading your answers it seems that it's quite some way to go. I have limited time, so I'd like to invest it into getting things working and not in upstreaming code to Mesa or getting other devices to work.

Nexus 7 2012 has a great upstream support status, you're good with it.

I will run a custom postmarketOS build on the devices, I have already built the 21.0.3 grate fork of Mesa as a package. I wondered if I'd also need to package the custom libdrm for this to work, that's why I asked about it. Then I ran out of time (vacation was over), and I haven't gotten around to testing the package on the device yet.

Having extra motivation is always helpful, it's great that you have it.

Time is a universal problem. You may have noticed that grate drivers are progressing staidly, but not very fast.

I actually skimmed some of the Mesa documentation, but I'm still lost on what exactly needs to be done and how the differnt components work together.

Mesa is a very complex software, it's normal to feel lost in it, especially in the beginning. In general it's more productive to focus on easier things first and then move iteratively to a more difficult problems.

I do have some experience with compiler backends, but in a less formal way from some experiments with my own scripting languages where it was all put together as needed. I don't have any experience doing that in a GPU context. For clarification: we are talking about translating NIR (or TGSI?) produced by Mesa from GLSL ES to something which can be sent to the GPU using libdrm to get it to the device via the kernel driver?

Yes, we need to take Mesa's IR and state, translate it to the HW state and HW opcodes, and send it all to hardware via libdrm and kernel DRM driver.

For vertex programs TGSI should be more appropriate since it maps well to hardware. For the fragment NIR should be more suitable.

How can I capture command streams? Can I do that on a rooted android device or do I need to get a Linux system with the old NV tegra support working on them?

You will need to prepare rootfs with the old NV tegra driver and then LD_PRELOAD https://github.com/grate-driver/grate/tree/master/src/libwrap to capture streams.

I guess the best way to move on would be to look at the VS IR and inspecting Kusma's fork? Some concrete pointers along the lines of "do X now, so that we can do Y" would be very much appreciated.

• Choose a simple problem that you want solve.
• Enable debug prints of the mesa's grate gallium driver
• Run your application.
• See which code paths need to be implemented.
• Add more debug messages and stubs if necessary.
• Start to implement missing parts.
• Move to next problem.

Kusma's fork is a good variant. You may start with the VS IR.

[astillich]

Thank you for the helpful answer. To get a better feeling of things, and to get some reference data, I decided to set up an old Linux distribution on one of the tablets and managed to install a build of Ubuntu 13.04 which contains the proprietary NV drivers. I wasn't sure how to get the rootfs from NV onto the android device without disabling the android bootloader (?), which I don't really want to do. Also, the rootfs is for an NV tablet which makes me suspicious if there would be problems with it.

Running es2_info confirms that I'm running on the proper driver (EGL_VENDOR = NVIDIA) and es2gears works fine. However, running es2tri fails with a fragment shader compile error, which I haven't investigated yet (there's no output about what's wrong). The tools were installed from the distro Mesa packages.

I tried to compile grate tools repository, which mostly worked, but it seems to have a dependency on the custom libdrm, as it cannot find drmSetClientCap which seems to come from it. So I guess to get that working, I need the custom libdrm and overwrite the distro version with it?

I'll have a closer look at Mesa stuff and the grate driver code in the coming days.

[digetx]

Thank you for the helpful answer. To get a better feeling of things, and to get some reference data, I decided to set up an old Linux distribution on one of the tablets and managed to install a build of Ubuntu 13.04 which contains the proprietary NV drivers. I wasn't sure how to get the rootfs from NV onto the android device without disabling the android bootloader (?), which I don't really want to do. Also, the rootfs is for an NV tablet which makes me suspicious if there would be problems with it.

Having rootfs over NFS is very useful for such cases, you won't need to write anything to device and could switch to another rootfs instantly.

You could write rootfs to /data partition. You could also write it to a subdirectory, but then you'll need to chroot there. I used https://gist.github.com/digetx/698af095d432d597d0835ab9f7b95f55 for chrooting.

Running es2_info confirms that I'm running on the proper driver (EGL_VENDOR = NVIDIA) and es2gears works fine. However, running es2tri fails with a fragment shader compile error, which I haven't investigated yet (there's no output about what's wrong). The tools were installed from the distro Mesa packages.

I don't know what's wrong with es2tri. You could also run the GL tests of grate.

I tried to compile grate tools repository, which mostly worked, but it seems to have a dependency on the custom libdrm, as it cannot find drmSetClientCap which seems to come from it. So I guess to get that working, I need the custom libdrm and overwrite the distro version with it?

I'll have a closer look at Mesa stuff and the grate driver code in the coming days.

The very old libdrm doesn't have drmSetClientCap(), I pushed fix for that.

[astillich]

Thanks for the info about mounting rootfs via NFS, I think it will come in handy when I have a cross compile setup. Right now, compiling on the device is enough for me.

I'll hack es2tri to output shader errors, I'd like to know why it fails with the NV drivers, might be useful info and it's not a lot of effort.

The grate tools now compile with the fix. I think I'll continue tomorrow.

[astillich]

I managed to capture the command stream of some of the ES2 examples in grate, so that's working for me now. A newer version os es2_tri runs fine, so I won't bother with it anymore. Time to get comfortable with mesa and the IL stuff.

[digetx]

Great, you could also install https://github.com/grate-driver/envytools and then recompile libwrap in order to get a meaningful output from the traces.

[tamburro92]

Hi,
I started to look at code in gallium driver, but before start I want to know what features are missing that need to implement at this moment.
Hope I can bring some contributes...

[digetx]

You may start with rebasing driver onto the recent Mesa version. It should compile without errors and glxgears should work.