Skip to content

Fix/linux desktop capturer x11 abi#236

Open
Isacco-B wants to merge 2 commits intodevopvoid:mainfrom
Isacco-B:fix/linux-desktop-capturer-x11-abi
Open

Fix/linux desktop capturer x11 abi#236
Isacco-B wants to merge 2 commits intodevopvoid:mainfrom
Isacco-B:fix/linux-desktop-capturer-x11-abi

Conversation

@Isacco-B
Copy link
Copy Markdown

@Isacco-B Isacco-B commented Apr 5, 2026

Problem

On Linux with WebRTC m140 (branch 7339), constructing DesktopAndCursorComposer
crashes with SIGSEGV. The root cause is an ABI mismatch in DesktopCaptureOptions:

The prebuilt WebRTC library is compiled with use_x11=true, which adds
X11-specific fields to DesktopCaptureOptions (conditional on WEBRTC_USE_X11).
The JNI CMake build already links against X11 Xfixes Xrandr Xcomposite but did
not define WEBRTC_USE_X11 as a preprocessor symbol. This caused the struct
layout seen by JNI code to differ from the one in the library → reading out of
bounds when passing options to DesktopAndCursorComposer → SIGSEGV.

This affects all Linux architectures (x86-64, arm, arm64) and all distributions.
It was introduced as a regression with the m140 update.

Fix

  1. Root cause (CMakeLists.txt): add WEBRTC_USE_X11 to Linux compile
    definitions so the JNI code and the WebRTC library agree on the struct layout.

  2. JNI safety (DesktopCapturer.cpp, JNI_ScreenCapturer.cpp,
    JNI_WindowCapturer.cpp): replace the throw in the constructor (which would
    cross the JNI boundary as undefined behavior) with a log + early return, and
    add null-guards on all methods. The JNI entry points now catch any remaining
    C++ exceptions and convert them to Java exceptions.

Testing

Verified on Ubuntu 24.04 (X11 session), linux_x86-64.
Screen and window capture work correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant