Skip to content

linux-computer-use.md

Latest commit

 

History

History
122 lines (87 loc) · 3.61 KB

linux-computer-use.md

File metadata and controls

122 lines (87 loc) · 3.61 KB
Raw

Linux Computer Use

Linux Computer Use is an opt-in UI surface backed by a native Rust MCP backend, codex-computer-use-linux. The backend is bundled and registered by default; the in-app Computer Use controls are disabled until you opt in.

It supports:

  • app listing and accessibility trees through AT-SPI
  • screenshots through GNOME Shell DBus, the Codex GNOME Shell extension, or XDG Desktop Portal
  • window listing and focusing on GNOME, KWin/Plasma, Hyprland, Niri, COSMIC, and i3
  • keyboard, text, click, scroll, and drag input through /dev/uinput, XDG RemoteDesktop portal, or ydotool

Runtime Dependencies

Install ydotool 1.0 or newer when you need the fallback input path. Some Debian and Ubuntu releases still package the incompatible pre-1.0 CLI; the Computer Use readiness report detects and rejects it instead of sending unsafe input commands.

# Debian / Ubuntu
sudo apt install ydotool
sudo apt install ydotoold   # on Ubuntu releases that split the daemon

# Fedora
sudo dnf install ydotool

# Arch / Manjaro
sudo pacman -S ydotool

# openSUSE
sudo zypper install ydotool

The preferred coordinate input path opens /dev/uinput directly. The XDG RemoteDesktop portal can also provide input on desktops that expose it.

For ydotool, run a daemon and make sure your user can access the socket:

sudo systemctl enable --now ydotoold
sudo usermod -a -G input "$USER"

Then log out and back in.

Some distros name the unit ydotool.service instead of ydotoold.service, and some install /usr/bin/ydotoold without a service unit. If the system unit path is awkward, a user-session service that binds %t/.ydotool_socket is also valid.

Portal packages are needed when your desktop relies on XDG Desktop Portal input or screenshots:

  • KDE Plasma: xdg-desktop-portal-kde
  • sway/wlroots: xdg-desktop-portal-wlr
  • Hyprland: xdg-desktop-portal-hyprland
  • GNOME: usually available by default

Niri window listing and exact focus use the niri command and the active session's NIRI_SOCKET. The Computer Use backend hydrates NIRI_SOCKET for GUI starts, but the socket must still belong to the active Niri session and be reachable by the desktop user.

Verify Readiness

Once Computer Use is visible in the Codex UI, ask Codex:

Check whether Linux Computer Use is ready

You can also run the backend directly:

./codex-app/resources/plugins/openai-bundled/plugins/computer-use/bin/codex-computer-use-linux doctor
./codex-app/resources/plugins/openai-bundled/plugins/computer-use/bin/codex-computer-use-linux setup
./codex-app/resources/plugins/openai-bundled/plugins/computer-use/bin/codex-computer-use-linux apps
./codex-app/resources/plugins/openai-bundled/plugins/computer-use/bin/codex-computer-use-linux windows
./codex-app/resources/plugins/openai-bundled/plugins/computer-use/bin/codex-computer-use-linux screenshot

Enable The In-App UI

Ad hoc, for one build:

CODEX_LINUX_ENABLE_COMPUTER_USE_UI=1 make build-app

Persistent, including future auto-updater rebuilds:

mkdir -p ~/.config/codex-desktop
echo '{"codex-linux-computer-use-ui-enabled": true}' > ~/.config/codex-desktop/settings.json

To opt back out, unset the env var and remove the settings flag or set it to false.

Nix:

nix run github:ilysenko/codex-desktop-linux#codex-desktop-computer-use-ui

Combined with a Linux feature output:

nix run github:ilysenko/codex-desktop-linux#computer-use-ui-remote-mobile-control

Side-By-Side Dev Variant

make build-dev-app
make run-dev-app

Override the dev identity with DEV_APP_ID, DEV_APP_NAME, and CODEX_WEBVIEW_PORT if needed.