linux-pipewire: Support MJPEG and H264 devices

[dimtpap]

Description

With #9771, support was added for the Camera portal. However, code to support MJPEG and H264 devices didn't make it. This PR is basically the code that didn't land + some more to finish it up.

Similarly to the v4l2 plugin, it uses ffmpeg to decode the encoded streams.

Motivation and Context

MJPEG and H264 devices exist

Fixes #10180
Fixes #10994

How Has This Been Tested?

Connected an MJPEG webcam, added the source, video was coming through.
Raw formats worked when converting the MJPEG stream to a raw format using GStreamer.
I couldn't get H264 working with GStreamer, I'd appreciate if someone with an H264 device tested this.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[tytan652]

After fighting with the non-specified flow of the source as it is (the format change is done only if the resolution and framerate is changed).
It looks functional as is at least for MJPEG, I can't test H264.

[mercuriete]

tested this patch
https://patch-diff.githubusercontent.com/raw/obsproject/obs-studio/pull/11371.patch

on top of gentoo obs-studio-9999 package and it worked for me so far.

One little thing:
after open a webcam using MJPEG format like mine: Logitech HD Pro Webcam C920...
if you open Kamoso, Kamoso hangs.
But it works perfectly with chromium using google meets.

My use case is I can't change resolutions on pipewire-v4l2 so I use obs to change resolution and now video formats.

Now I can select 1080p@30fps.

I would recommend to test this patch using other software running at the same time.
At this point I only know "kamoso", "chromium" and "google-chrome"

Thank you, now I can go to work meetings using 1080p before it defaults to 640x480.

PS:
I would like to have someone porting qv4l2 to pipewire because after migrating to pipewire I lose all control of my webcam.
pw-v4l2 qv4l2 dont work.

[ChrisLane]

Just tested this for my use case in #10180 and it seems to be working very well for MJPEG!

Turns out my stream from gphoto2 was raw jpeg and not MJPEG though so I still needed to do some encoding to test

sink_name="DSLR"
pipewire_sink_props="p,node.description=$sink_name,node.name=$sink_name,media.role=Camera,media.class=Video/Source"

gphoto2 --stdout --capture-movie \
  | gst-launch-1.0 \
      fdsrc \
      ! jpegdec \
      ! jpegenc \
      ! jpegparse \
      ! queue \
      ! pipewiresink stream-properties="$pipewire_sink_props"

[ChrisLane]

@dimtpap would you mind resolving the merge conflicts please?

[dimtpap]

For the record I'm waiting for #11741 so I can finish and undraft this

[ChrisLane]

@dimtpap I've discovered that I sometimes get a segfault when switching between scene collections when a PipeWire MJPEG source is used.
I was unable to reproduce this with the video format set to YUYV422 on the same build.

Steps to reproduce:

1. Install obs-studio-git modified with adding patch -Np1 < "$srcdir/11371.patch" to prepare() to apply this patch.
2. From fresh OBS configs, create a second scene collection
3. Add 1080p 30fps MJPEG Pipewire Video Capture Device to a scene
4. Switch between the scenes repeatedly until a segfault occurs

I ran OBS in GDB and reproduced the segfault, here's the log for that:
https://gist.github.com/ChrisLane/547ec9570bb9c6f87b8ff5ce837d4ab6

Usually it says 'loop->recurse > 0' failed at ../pipewire/src/pipewire/thread-loop.c:425 pw_thread_loop_wait() just before the segfault but that doesn't seem to have happened in this case.

[ChrisLane]

@dimtpap there seems to be an issue with this when compiling and running against latest PipeWire.
The issue is described in a thread in this now closed PipeWire issue: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/4643#note_2877800

Do you think this needs to be resolved in this PR?

[stephematician]

I ran OBS in GDB and reproduced the segfault, here's the log for that: https://gist.github.com/ChrisLane/547ec9570bb9c6f87b8ff5ce837d4ab6

@ChrisLane - can you provide the results of where from GDB when the crash occurs?

[ChrisLane]

I ran OBS in GDB and reproduced the segfault, here's the log for that: gist.github.com/ChrisLane/547ec9570bb9c6f87b8ff5ce837d4ab6

@ChrisLane - can you provide the results of where from GDB when the crash occurs?

I don't think I'll have the time to provide that I'm afraid, I don't use the same set up any more.

[dimtpap]

I've discovered that I sometimes get a segfault when switching between scene collections when a PipeWire MJPEG source is used. I was unable to reproduce this with the video format set to YUYV422 on the same build.

I got segfaults like this and in generally when destroying the source because the decoder was destroyed before the stream using it. This is now fixed.

[Stealcase]

I am very happy to see activity on this PR, because it would solve my Logitech HD Pro Webcam C920 being limited to 5fps and YUYV422 when trying to record in OBS through Video Capture Device (Pipewire) (BETA).

My impression is that C920s are very common in the wild after the pandemic, and the current experience of using them with OBS on Wayland has a ton of hiccups.

So I think this will increase the perceived stability/functionality of Pipewire and OBS on linux for a lot of the people who don't ever open Github.

[ManuLinares]

LGTM

---

Thanks, this patch works perfectly.

I'm using a Google Pixel 6a as s webcam via usb which now lets me pick

I'm on archlinux, obs-studio 32.0.2-1 plus this patch.
pipewire 1:1.4.9-1

[Novantric]

Let's get this reviewed!

[phileas0408]

can confirm it works perfectly with my cheap webcam and capture card
tested on flatpak

[dimtpap]

Also note that while working on this I ran into the same problem described in #12779 and had to include that for this to work

[RytoEX]

Also note that while working on this I ran into the same problem described in #12779 and had to include that for this to work

#12779 has been merged.

[rmader]

Gentle ping here, just in case it fell through the cracks - otherwise sorry for the noise! :)

AFAICS all requested changes in the last review have been addressed and the MR has been rebased on top of #12779

@RytoEX @GeorgesStavracas when you have time, any chance you can have another look?