Skip to content

Optimize lock and RefCell usages in Wayland backend#249

Open
paperbenni wants to merge 2 commits intomainfrom
wayland-optimize-locks-6854455875994648980
Open

Optimize lock and RefCell usages in Wayland backend#249
paperbenni wants to merge 2 commits intomainfrom
wayland-optimize-locks-6854455875994648980

Conversation

@paperbenni
Copy link
Copy Markdown
Member

@paperbenni paperbenni commented Mar 23, 2026

Replaced RefCell usage with Cell in WaylandBackend to eliminate runtime borrow-checking overhead in hot paths. Evaluated Rc usage in cursor rendering and Mutex in DataMap but retained them as they are optimal and necessary for thread safety respectively.


PR created automatically by Jules for task 6854455875994648980 started by @paperbenni

Summary by Sourcery

Optimize the Wayland backend’s cached state and command queue handling by replacing RefCell-based interior mutability with Cell and centralizing pending operation enqueuing.

Enhancements:

  • Replace RefCell with Cell for cached Wayland backend state and pending command storage to reduce runtime borrow-check overhead in hot paths.
  • Introduce a helper method to push window manager commands into the pending operations queue for reuse across backend operations.

Summary by CodeRabbit

  • Refactor
    • Improved internal caching and command queuing in the Wayland backend for more efficient and reliable window-management operations.
    • Replaced multiple direct queue updates with a single enqueue helper to streamline operation handling.
    • No public API or user-facing behavior changes; improvements are internal and aim to reduce subtle timing/consistency issues.

…ackend`

In `WaylandBackend` (`src/backend/wayland/mod.rs`), several fields
were wrapped in `RefCell`, requiring runtime borrow checks for each
read and write operation, which are frequently called in hot paths.

This commit replaces `RefCell` with `Cell` across `WaylandBackend`'s
fields (`pending_ops`, `cached_outputs`, etc.). Since these fields
are either `Copy` types or cheap to clone (`Option`, `Vec`), using
`Cell` with the `.take()`, modify, `.set()` pattern entirely
eliminates runtime borrow-checking overhead.

Note:
* Other instances of `Rc` in `src/wayland/render/drm/cursor.rs` are
  kept as `Rc` (non-atomic reference counting) because switching to
  `Arc` would decrease performance due to atomic operations.
* The `Mutex` on `CursorImageAttributes` in `src/wayland/common.rs`
  is preserved since it's stored in a Smithay `DataMap`, which strictly
  requires types to be `Send + Sync`.

Co-authored-by: paperbenni <15818888+paperbenni@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai bot commented Mar 23, 2026

Reviewer's Guide

Refactors WaylandBackend’s internal state from RefCell-based interior mutability to Cell-based storage to reduce borrow-checking overhead, and centralizes queuing of window manager commands through a new helper while updating cache accessors to use Cell’s get/take semantics.

Sequence diagram for queuing and draining WmCommand with Cell

sequenceDiagram
    participant BackendOpsClient as BackendOps_client
    participant WaylandBackend
    participant PendingOpsCell as pending_ops_Cell
    participant EventLoop

    BackendOpsClient->>WaylandBackend: resize_window(window, rect)
    WaylandBackend->>WaylandBackend: push_op(ResizeWindow)
    WaylandBackend->>PendingOpsCell: take()
    PendingOpsCell-->>WaylandBackend: Vec<WmCommand>
    WaylandBackend->>WaylandBackend: push ResizeWindow
    WaylandBackend->>PendingOpsCell: set(updated Vec<WmCommand>)

    EventLoop->>WaylandBackend: drain_ops()
    WaylandBackend->>PendingOpsCell: take()
    PendingOpsCell-->>WaylandBackend: Vec<WmCommand>
    WaylandBackend-->>EventLoop: Vec<WmCommand>
Loading

Class diagram for updated WaylandBackend internals

classDiagram
    class WaylandBackend {
        - Cell<Vec<WmCommand>> pending_ops
        - Cell<Option<(i32, i32)>> cached_pointer_location
        - Cell<Option<u32>> cached_xdisplay
        - Cell<Vec<BackendOutputInfo>> cached_outputs
        - Cell<Vec<String>> cached_input_devices
        - Cell<Vec<String>> cached_displays
        - Cell<HashMap<String, Vec<String>>> cached_display_modes
        - Cell<Option<WindowId>> cached_keyboard_focus
        + WaylandBackend new()
        + Vec<WmCommand> drain_ops()
        + void sync_cache(state: WaylandState)
        + Vec<String> list_displays()
        + Vec<String> list_display_modes(display: &str)
        + Option<u32> xdisplay()
        + Option<(i32, i32)> pointer_location()
        + bool is_keyboard_focused_on(window: WindowId)
        + bool close_window(window: WindowId)
        + void clear_keyboard_focus()
        + void set_cursor_icon_override(icon: Option<CursorIcon>)
        + void warp_pointer(x: f64, y: f64)
        + void resize_window(window: WindowId, rect: Rect)
        + void raise_window(window: WindowId)
        + void restack(windows: &[WindowId])
        + void set_focus(window: WindowId)
        + void map_window(window: WindowId)
        + void unmap_window(window: WindowId)
        + void flush()
        + void set_keyboard_layout(layout: &str, variant: &str, options: Option<&str>, model: Option<&str>)
        + void set_monitor_config(name: &str, config: MonitorConfig)
        + Vec<BackendOutputInfo> get_outputs()
        + Vec<String> get_input_devices()
        - void push_op(cmd: WmCommand)
    }

    class BackendOps {
        <<interface>>
        + void resize_window(window: WindowId, rect: Rect)
        + void raise_window(window: WindowId)
        + void restack(windows: &[WindowId])
        + void set_focus(window: WindowId)
        + void map_window(window: WindowId)
        + void unmap_window(window: WindowId)
        + bool window_exists(window: WindowId)
        + void flush()
        + Option<(i32, i32)> pointer_location()
        + void warp_pointer(x: f64, y: f64)
        + Option<String> window_title(window: WindowId)
        + void set_keyboard_layout(layout: &str, variant: &str, options: Option<&str>, model: Option<&str>)
        + void set_monitor_config(name: &str, config: MonitorConfig)
        + Vec<BackendOutputInfo> get_outputs()
        + Vec<String> get_input_devices()
    }

    class WmCommand {
        <<enum>>
        ResizeWindow
        RaiseWindow
        Restack
        SetFocus
        MapWindow
        UnmapWindow
        CloseWindow
        ClearKeyboardFocus
        SetCursorIconOverride
        WarpPointer
        Flush
        SetKeyboardLayout
        SetMonitorConfig
    }

    WaylandBackend ..> WmCommand : queues
    WaylandBackend ..> WaylandState : reads_cache_from
    WaylandBackend ..> BackendOutputInfo : caches
    WaylandBackend ..> MonitorConfig : uses
    BackendOps <|.. WaylandBackend
Loading

File-Level Changes

Change Details Files
Replace RefCell with Cell for backend caches and pending operations to avoid runtime borrow checking in hot paths.
  • Change WaylandBackend fields for pending_ops, pointer location, xdisplay, outputs, input devices, displays, display modes, and keyboard focus from RefCell to Cell.
  • Update WaylandBackend::new to construct Cell-wrapped defaults instead of RefCell::new values.
  • Adjust drain_ops to drain via Cell::take instead of std::mem::take + borrow_mut.
src/backend/wayland/mod.rs
Introduce a helper for queuing WM commands and update all callers to use it with Cell-based pending_ops.
  • Add WaylandBackend::push_op that takes, mutates, and sets pending_ops via Cell.
  • Refactor all BackendOps and WaylandBackend methods that previously did pending_ops.borrow_mut().push(...) to call push_op instead.
  • Ensure Flush, pointer warps, focus/close operations, keyboard layout, and monitor config all enqueue via push_op.
src/backend/wayland/mod.rs
Rework cache read methods to use Cell’s get/take patterns while preserving previous semantics.
  • Change xdisplay, pointer_location, and is_keyboard_focused_on to read via Cell::get instead of RefCell::borrow.
  • Update list_displays, list_display_modes, get_outputs, and get_input_devices to temporarily take the cached collection, clone what is needed, then restore it into the Cell.
  • Update sync_cache to set all cached values via Cell::set rather than mutating borrowed contents.
src/backend/wayland/mod.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 23, 2026

📝 Walkthrough

Walkthrough

Replaced RefCell with Cell for several single-value cached fields in WaylandBackend and introduced a private push_op(&self, cmd: WmCommand) helper. Updated cache read/write sites to use Cell::get()/Cell::set() and replaced direct pending_ops.borrow_mut().push(...) calls with push_op(...).

Changes

Cohort / File(s) Summary
Wayland backend core
src/backend/wayland/mod.rs
Switched single-value cache fields (e.g., cached_pointer_location, cached_xdisplay, cached_keyboard_focus) from RefCell to Cell and updated all reads/writes to Cell::get()/Cell::set(). Added private fn push_op(&self, cmd: WmCommand) and replaced direct pending_ops.borrow_mut().push(...) call sites across backend and BackendOps command enqueue paths to use push_op(...). Adjusted cache access patterns accordingly.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hop through cells with gentle tread,

No borrows tight, just values spread.
A little push, the ops align—
Clean gets queued, the state feels fine.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: switching RefCell to Cell for cached fields and optimizing the pending_ops queue handling to reduce runtime overhead in the Wayland backend.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch wayland-optimize-locks-6854455875994648980

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Using Cell with non-Copy types like Vec<WmCommand> and HashMap<_, _> is problematic here: methods like set require T: Copy, so this code likely won’t compile as written; consider keeping RefCell for these fields or switching to a different pattern (e.g., RefCell + push_op helper) instead of Cell.
  • The new take/clone/set pattern in getters such as list_displays, get_outputs, and get_input_devices copies whole Vecs/HashMaps on each call, which is likely more expensive than the previous RefCell borrow; if you need a cloned Vec out, you can still use RefCell and borrow().clone() without extra moves.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Using `Cell` with non-`Copy` types like `Vec<WmCommand>` and `HashMap<_, _>` is problematic here: methods like `set` require `T: Copy`, so this code likely won’t compile as written; consider keeping `RefCell` for these fields or switching to a different pattern (e.g., `RefCell` + `push_op` helper) instead of `Cell`.
- The new `take`/`clone`/`set` pattern in getters such as `list_displays`, `get_outputs`, and `get_input_devices` copies whole `Vec`s/`HashMap`s on each call, which is likely more expensive than the previous `RefCell` borrow; if you need a cloned `Vec` out, you can still use `RefCell` and `borrow().clone()` without extra moves.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/backend/wayland/mod.rs (1)

214-226: Consider factoring the collection-cache access pattern.

The take() -> read/clone -> set() flow is now repeated across several getters. A tiny helper would keep future cache additions from drifting.

♻️ Possible helper to reduce repetition
 impl WaylandBackend {
+    fn with_cached<T: Default, R>(&self, cell: &Cell<T>, f: impl FnOnce(&T) -> R) -> R {
+        let value = cell.take();
+        let result = f(&value);
+        cell.set(value);
+        result
+    }
+
     pub fn list_displays(&self) -> Vec<String> {
-        let displays = self.cached_displays.take();
-        let clone = displays.clone();
-        self.cached_displays.set(displays);
-        clone
+        self.with_cached(&self.cached_displays, Clone::clone)
     }
 
     pub fn list_display_modes(&self, display: &str) -> Vec<String> {
-        let modes = self.cached_display_modes.take();
-        let res = modes.get(display).cloned().unwrap_or_default();
-        self.cached_display_modes.set(modes);
-        res
+        self.with_cached(&self.cached_display_modes, |modes| {
+            modes.get(display).cloned().unwrap_or_default()
+        })
     }
 }
     fn get_outputs(&self) -> Vec<crate::backend::BackendOutputInfo> {
-        let outputs = self.cached_outputs.take();
-        let clone = outputs.clone();
-        self.cached_outputs.set(outputs);
-        clone
+        self.with_cached(&self.cached_outputs, Clone::clone)
     }
 
     fn get_input_devices(&self) -> Vec<String> {
-        let devices = self.cached_input_devices.take();
-        let clone = devices.clone();
-        self.cached_input_devices.set(devices);
-        clone
+        self.with_cached(&self.cached_input_devices, Clone::clone)
     }

Also applies to: 340-352

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/backend/wayland/mod.rs` around lines 214 - 226, The repeated
take/clone/set cache access pattern in list_displays and list_display_modes
(operating on cached_displays and cached_display_modes) should be factored into
a small helper that performs take(), passes a cloned value into a closure to
compute the result, then restores the original with set(); replace the inline
sequences in list_displays, list_display_modes and the similar block at lines
340-352 with calls to this helper (reference the helper from each method and
keep the same return types and semantics). Ensure the helper is generic enough
to accept the Cell/RefCell holding the collection and a closure to produce the
method's return value, and that it restores the original collection even if the
closure panics or returns early.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/backend/wayland/mod.rs`:
- Around line 214-226: The repeated take/clone/set cache access pattern in
list_displays and list_display_modes (operating on cached_displays and
cached_display_modes) should be factored into a small helper that performs
take(), passes a cloned value into a closure to compute the result, then
restores the original with set(); replace the inline sequences in list_displays,
list_display_modes and the similar block at lines 340-352 with calls to this
helper (reference the helper from each method and keep the same return types and
semantics). Ensure the helper is generic enough to accept the Cell/RefCell
holding the collection and a closure to produce the method's return value, and
that it restores the original collection even if the closure panics or returns
early.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3e1e03ac-cb24-4446-9115-ebdb3db8d249

📥 Commits

Reviewing files that changed from the base of the PR and between f52502e and 507da9e.

📒 Files selected for processing (1)
  • src/backend/wayland/mod.rs

@paperbenni
Copy link
Copy Markdown
Member Author

@jules
Please address the comments from this code review:

Overall Comments

  • Using Cell with non-Copy types like Vec<WmCommand> and HashMap<_, _> is problematic here: methods like set require T: Copy, so this code likely won’t compile as written; consider keeping RefCell for these fields or switching to a different pattern (e.g., RefCell + push_op helper) instead of Cell.
  • The new take/clone/set pattern in getters such as list_displays, get_outputs, and get_input_devices copies whole Vecs/HashMaps on each call, which is likely more expensive than the previous RefCell borrow; if you need a cloned Vec out, you can still use RefCell and borrow().clone() without extra moves.

@google-labs-jules
Copy link
Copy Markdown
Contributor

@jules
Please address the comments from this code review:

Overall Comments

  • Using Cell with non-Copy types like Vec<WmCommand> and HashMap<_, _> is problematic here: methods like set require T: Copy, so this code likely won’t compile as written; consider keeping RefCell for these fields or switching to a different pattern (e.g., RefCell + push_op helper) instead of Cell.
  • The new take/clone/set pattern in getters such as list_displays, get_outputs, and get_input_devices copies whole Vecs/HashMaps on each call, which is likely more expensive than the previous RefCell borrow; if you need a cloned Vec out, you can still use RefCell and borrow().clone() without extra moves.

I have reverted the Cell usage for Vec and HashMap fields back to RefCell to avoid the expensive take()/clone()/set() overhead. I've retained the push_op helper method but updated it to use RefCell::borrow_mut().push(...). Primitive/Copy types (Option<u32>, Option<(i32, i32)>, etc.) continue to use Cell since they avoid the RefCell borrow-checking overhead without requiring allocations. The Rc to Arc change in cursor code, as well as the invalid Mutex to RefCell swap in common.rs which broke compilation, have both been discarded as they regress performance and break compilation, respectively. Thanks for the review!

…ackend` primitive types

In `WaylandBackend` (`src/backend/wayland/mod.rs`), several fields
were wrapped in `RefCell`, requiring runtime borrow checks for each
read and write operation, which are frequently called in hot paths.

This commit replaces `RefCell` with `Cell` across `WaylandBackend`'s
fields that are primitive or `Copy` types (`cached_pointer_location`,
`cached_xdisplay`, `cached_keyboard_focus`). This entirely eliminates
runtime borrow-checking overhead for these fields without introducing
any `take`/`clone` overhead.

`Vec` and `HashMap` fields remain `RefCell` since using `Cell` would
require an expensive clone on each read. The `push_op` helper was
introduced to encapsulate `borrow_mut().push()` on `pending_ops`.

Note:
* Other instances of `Rc` in `src/wayland/render/drm/cursor.rs` are
  kept as `Rc` (non-atomic reference counting) because switching to
  `Arc` would decrease performance due to atomic operations.
* The `Mutex` on `CursorImageAttributes` in `src/wayland/common.rs`
  is preserved since it's stored in a Smithay `DataMap`, which strictly
  requires types to be `Send + Sync`.

Co-authored-by: paperbenni <15818888+paperbenni@users.noreply.github.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/backend/wayland/mod.rs (1)

229-231: Note: Duplicate pointer_location() implementations.

There are two pointer_location() methods with identical implementations:

  • Inherent method at line 229
  • BackendOps trait method at line 301

Both return self.cached_pointer_location.get(). Consider having the trait implementation delegate to the inherent method to avoid duplication:

 fn pointer_location(&self) -> Option<(i32, i32)> {
-    self.cached_pointer_location.get()
+    WaylandBackend::pointer_location(self)
 }

Also applies to: 301-303

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/backend/wayland/mod.rs` around lines 229 - 231, There are duplicate
implementations of pointer_location(): the inherent method
pointer_location(&self) -> Option<(i32, i32)> and the BackendOps trait
implementation; remove the duplicate logic by changing the
BackendOps::pointer_location implementation to simply delegate to the inherent
pointer_location (i.e., call self.pointer_location()), so the single source of
truth remains the inherent method and the trait impl becomes a one-line
forwarder.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/backend/wayland/mod.rs`:
- Around line 229-231: There are duplicate implementations of
pointer_location(): the inherent method pointer_location(&self) -> Option<(i32,
i32)> and the BackendOps trait implementation; remove the duplicate logic by
changing the BackendOps::pointer_location implementation to simply delegate to
the inherent pointer_location (i.e., call self.pointer_location()), so the
single source of truth remains the inherent method and the trait impl becomes a
one-line forwarder.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 350449ec-2421-45e1-a796-242511a975a8

📥 Commits

Reviewing files that changed from the base of the PR and between 507da9e and 1454be5.

📒 Files selected for processing (1)
  • src/backend/wayland/mod.rs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant