support direct access to nvbuffer

[maddanio]

I am murky on the details but would probably try this myself and then provide a PR.

What I would like is, like the nvenc encoder for ffmpeg provides, provide direct access to the underlying hardware buffer, so we can avoid memcopies, especially between different kinds of memory. in this case I suppose that would be the nvbuffer.

I suppose that also entails hooking into the ffmpeg mechanisms for transfer from/to hwaccel devices.

[maddanio]

Ya, found it. basically one would have to hook into the api provided in the hwcontext header in ffmpeg and basically implkement a hwcontext_nvmpi with associated functions and structs.

[maddanio]

then the transfer from/to the hwaccel is actually triggered by ffmpeg and can then be opted out of, to do things directly. thats how you can convince libavcodec to directly encode cuda memory rendered to by other code.

[maddanio]

ok, so after some research, I think the path would be to use DMA and the egl stuff to basically get a cuda pointer for each encoder plane (probably similar for the decoder path) and then basically reuse the code that ffmpeg uses for hwaccel nvenc/cuvid

[mguzzina]

Hi, I believe I'm possibly trying to do something similar.
I'm trying to add Jetson hardware support for Jellyfin (jellyfin/jellyfin-ffmpeg#653) and I've been told I would need to add hwcontext_nvmpi (plus filters support).
I don't fully know what this entails yet as I'm not too familiar with ffmpeg, so if you have any suggestions on where to start or what to do they'd be very welcome.

Could you elaborate a bit on what

use DMA and the egl stuff to basically get a cuda pointer for each encoder plane

and

reuse the code that ffmpeg uses for hwaccel nvenc/cuvid

mean?

[maddanio]

Well basically the jetson encoder gives you a set of device pointers. Ideally you put your data directly there, but not sure ffmpeg api works like that

[maddanio]

And ya, sorry, moved on to some code that can be found in NVIDIAs examples. I can post my encoder class if you want

[mguzzina]

Please do, it might be helpful