Hardware decoding not working on Wayland

[vista-]

Hey folks,

Thanks for putting this Flatpak together. I've been trying to get hardware decoding working on my setup, without much success. I'm running Fedora 34 (although it shouldn't matter that much in case of a Flatpak-packaged software) on a laptop with an 8th gen Intel processor, running sway (Wayland-based DE) -- ordinarily, I have no issues with hardware decoding, both when using media players (say, mpv) or when playing videos in a browser, but I couldn't get it working with the Steam Link app.

I also tried building a Flatpak of this PR: #20, which bumps the libva version up to 2.11.0 (the version of the lib that's installed on my system) without any luck. Any further troubleshooting/debugging suggestions are more than welcome!

You can find some (hopefully) helpful output down here:

$ vainfo
libva info: VA-API version 1.11.0
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_11
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.11 (libva 2.11.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.3 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointFEI
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointFEI
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointFEI
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
$ flatpak run com.valvesoftware.SteamLink
[...]
INFO: ffmpeg verbose: Opened VA display via X11 display :99.0.
INFO: ffmpeg verbose: libva: VA-API version 1.11.0
INFO: ffmpeg error: libva: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
INFO: ffmpeg error: Failed to initialise VAAPI connection: -1 (unknown libva error).
INFO: CVAAPIAccel: av_hwdevice_ctx_create() failed
INFO: libavcodec software decoding with 4 threads
INFO: ffmpeg verbose: Reinit context to 1920x1088, pix_fmt: yuv420p
[...]

[slouken]

Can you try this build?
flatpak install --user https://dl.flathub.org/build-repo/66473/com.valvesoftware.SteamLink.flatpakref

[vista-]

What I'm experiencing:

The initial menu loads fine, and after clicking connect, the window disappears, I get the loglines about ffmpeg opening the libva library for hardware decoding, but no window actually appears.

steamlink keeps running in the background, I have to manually send a kill signal to it to end the process.

$ flatpak run --user com.valvesoftware.SteamLink
INFO: Hardware: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
qt.qpa.wayland: Setting cursor position is not possible on wayland
INFO: Connected to Remote Client service with session 8093617778759811482
INFO: Starting Streaming
INFO: Received authorization proof request, responding with encrypted challenge
INFO: Received start streaming response k_ERemoteDeviceStreamingInProgress
INFO: Received start streaming response k_ERemoteDeviceStreamingInProgress
INFO: Received start streaming response k_ERemoteDeviceStreamingSuccess
qt.qpa.wayland: Setting cursor position is not possible on wayland
INFO: Initializing player
INFO: Refresh rate: 60.00
INFO: Effective rate: 59.75
INFO: Video rect: 0x0 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Initialized player
INFO: Server available at address: 192.168.10.43:27031
INFO: Connecting to server at 192.168.10.43:27031 using direct UDP
INFO: Session state Idle -> Connecting
INFO: CStreamTransportClient stopping Remote Client service connection
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Session state Connecting -> Handshake
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Session state Handshake -> Authenticating
INFO: Got control packet k_EStreamControlServerHandshake
INFO: Session state Authenticating -> Negotiating
INFO: Got control packet k_EStreamControlAuthenticationResponse
INFO: Got control packet k_EStreamControlNegotiationInit
INFO: Session state Negotiating -> StreamStarting
INFO: Got control packet k_EStreamControlNegotiationSetConfig
INFO: Session state StreamStarting -> Streaming
INFO: Got control packet k_EStreamControlSetQoS
INFO: Got control packet k_EStreamControlSetTargetBitrate
INFO: Starting audio stream: frequency=48000, channels=2
INFO: Opened audio device: format=2, channels=2, frequency=48000
INFO: Initialized audio decoder: codec=3, sample rate=48000, channels=2
INFO: Got control packet k_EStreamControlStartAudioData
INFO: Got control packet k_EStreamControlSetSpectatorMode
INFO: Got control packet k_EStreamControlSetCaptureSize
INFO: Got control packet k_EStreamControlSetTitle
INFO: Got control packet k_EStreamControlSetIcon
INFO: Got control packet k_EStreamControlHideCursor
INFO: Got control packet k_EStreamControlSetActivity
INFO: Got control packet k_EStreamControlStartVideoData
INFO: Got control packet k_EStreamControlVideoEncoderInfo
INFO: CVDPAUAccel: GL_NV_vdpau_interop not available on current context
INFO: ffmpeg verbose: Trying to use DRM render node for device 0.
INFO: ffmpeg verbose: libva: VA-API version 1.12.0
INFO: ffmpeg verbose: libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
INFO: ffmpeg verbose: libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so
INFO: ffmpeg verbose: libva: Found init function __vaDriverInit_1_12
INFO: ffmpeg verbose: libva: va_openDriver() returns 0
INFO: ffmpeg verbose: Initialised VAAPI connection: version 1.12
INFO: ffmpeg verbose: VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 21.4.1 ().
INFO: ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour.
INFO: VAAPI DRM hardware decoding
INFO: ffmpeg verbose: Reinit context to 1920x1088, pix_fmt: vaapi_vld
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
^C

(keeps running)

After sending pkill to the steamlink process:

$ pkill steamlink
$ INFO: Stopped audio decoder
INFO: Session state Streaming -> StreamStopping
INFO: Streaming session complete, average network time: 2.56ms, stddev: 7.70ms, 0.00 percent frame loss, result: excellent
INFO: Session state StreamStopping -> Idle
qt.qpa.wayland: Setting cursor position is not possible on wayland
INFO: Connected to Remote Client service with session 8201859150277012981

I get back to the main menu of the application.

vainfo:

$ vainfo
libva info: VA-API version 1.11.0
libva info: Trying to open /usr/lib64/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_11
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.11 (libva 2.11.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.3 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileNone                   :	VAEntrypointStats
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointFEI
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointFEI
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointFEI
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointFEI
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD

I'll try to update to Fedora 35 (currently running 34), maybe there's a new version of libva in that. I'll report back once I have results on F35.

[vista-]

Can confirm that I experience the same thing on F35.

I also tried updating Steam Link to the newest version and running that (branch=stable), I get the following output running that (the window also disappears after connecting):

 $ flatpak run --branch=stable com.valvesoftware.SteamLink
INFO: Hardware: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
qt.qpa.wayland: Setting cursor position is not possible on wayland
INFO: Connected to Remote Client service with session 3915169177402801914
INFO: Starting Streaming
INFO: Received authorization proof request, responding with encrypted challenge
INFO: Received start streaming response k_ERemoteDeviceStreamingInProgress
INFO: Received start streaming response k_ERemoteDeviceStreamingSuccess
INFO: Initializing player
INFO: Refresh rate: 60.00
INFO: Effective rate: 59.75
INFO: Video rect: 0x0 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Initialized player
INFO: Server available at address: 192.168.10.43:27031
INFO: Connecting to server at 192.168.10.43:27031 using direct UDP
INFO: Session state Idle -> Connecting
INFO: CStreamTransportClient stopping Remote Client service connection
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Session state Connecting -> Handshake
INFO: Session state Handshake -> Authenticating
INFO: Got control packet k_EStreamControlServerHandshake
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Session state Authenticating -> Negotiating
INFO: Got control packet k_EStreamControlAuthenticationResponse
INFO: Got control packet k_EStreamControlNegotiationInit
INFO: Session state Negotiating -> StreamStarting
INFO: Got control packet k_EStreamControlNegotiationSetConfig
INFO: Session state StreamStarting -> Streaming
INFO: Got control packet k_EStreamControlSetQoS
INFO: Got control packet k_EStreamControlSetTargetBitrate
INFO: Starting audio stream: frequency=48000, channels=2
INFO: Opened audio device: format=2, channels=2, frequency=48000
INFO: Initialized audio decoder: codec=3, sample rate=48000, channels=2
INFO: Got control packet k_EStreamControlStartAudioData
INFO: Got control packet k_EStreamControlSetSpectatorMode
INFO: Got control packet k_EStreamControlSetCaptureSize
INFO: Got control packet k_EStreamControlSetTitle
INFO: Got control packet k_EStreamControlSetIcon
INFO: Got control packet k_EStreamControlHideCursor
INFO: Got control packet k_EStreamControlSetActivity
INFO: Got control packet k_EStreamControlStartVideoData
INFO: Got control packet k_EStreamControlVideoEncoderInfo
INFO: CVDPAUAccel: GL_NV_vdpau_interop not available on current context
INFO: CVAAPIAccelX11: Couldn't find X11 display and window
INFO: ffmpeg verbose: Trying to use DRM render node for device 0.
INFO: ffmpeg verbose: libva: VA-API version 1.8.0
INFO: ffmpeg verbose: libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
INFO: ffmpeg verbose: libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so
INFO: ffmpeg verbose: libva: Found init function __vaDriverInit_1_8
INFO: ffmpeg verbose: libva: va_openDriver() returns 0
INFO: ffmpeg verbose: Initialised VAAPI connection: version 1.8
INFO: ffmpeg verbose: VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 21.3.1 ().
INFO: ffmpeg verbose: Driver not found in known nonstandard list, using standard behaviour.
INFO: VAAPI DRM hardware decoding
INFO: ffmpeg verbose: Reinit context to 1920x1088, pix_fmt: vaapi_vld
INFO: Marking frame 0 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 2 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 4 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 5 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 7 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 8 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 11 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 16 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 18 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 20 complete with status k_EStreamFrameResultDroppedLate
INFO: Targeting 30.00 FPS
INFO: Got control packet k_EStreamControlSetTargetFramerate
INFO: Marking frame 32 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 35 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 40 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 45 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 50 complete with status k_EStreamFrameResultDroppedLate
INFO: Targeting 60.00 FPS
INFO: Got control packet k_EStreamControlSetTargetFramerate
INFO: Marking frame 117 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 138 complete with status k_EStreamFrameResultDroppedLate
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video size: 1920x1080, output size: 1920x1080, overlay size: 1920x1080
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Video rect: 1920x1080 at 0,0
INFO: Texture rect: 1920x1080 at 0,0
INFO: Marking frame 482 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 483 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 485 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 532 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 533 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 575 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 617 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 866 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 913 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 914 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 1121 complete with status k_EStreamFrameResultDroppedLate
INFO: Marking frame 1123 complete with status k_EStreamFrameResultDroppedLate
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: Marking frame 1170 complete with status k_EStreamFrameResultDroppedLate
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
INFO: CVAAPIAccelDRM: vaExportSurfaceHandle() failed: operation failed
... keeps spamming the console ...

[vista-]

Just tested with hardware rendering disabled -- this does not seem to be a hardware rendering issue, it looks like the streaming window appears on a different screen/display than the one I originally started Steam Link on / that has focus when starting streaming.

However, I feel like there's a regression in how the streamed content is rendered -- it only works in fullscreen, and cannot be windowed any more -- if I try to window it, then the streamed video is no longer stretched across my display and is rendered in the original resolution, however, the rest of the display is black (and the video stream plays with the content aligned to the right side of the display, not centered).

If I switch back to an earlier version of the flatpak (commit 10caec0bc183df7c2d9cf2bc165f0907ee1c58fe795383c983865ba55884c082), the stream can be windowed again without an issue.