CDP Mode leaks event loops, file descriptors and temp profile dirs in long-running processes

[montovaneli]

Setup

• SeleniumBase 4.50.1
• Python 3.12.0
• Windows 11/Ubuntu 24.04

The use case

I run a process that stays alive for a long time and starts/stops many CDP sessions in a loop (a worker that never restarts Python). After a while the process keeps growing in memory, I eventually hit "Too many open files" on Linux, and my temp folder fills up with uc_* profile directories.

For now I work around this by isolating each session in its own spawned child process, so when the child exits the OS reclaims everything (FDs, loops, sockets, Chrome, temp dirs). It works, but it never felt right to me, and it made me think there was probably room to improve cleanup in UC/CDP Mode itself, so I decided to dig in and track down where the leaks were actually coming from.

After looking into it, the main issue is that the real cleanup only runs at atexit, so if the process never exits nothing actually gets freed. Calling stop() (or quit()) ends the Chrome process but leaves the rest behind.

How to reproduce

import glob, os, tempfile, gc
from seleniumbase import sb_cdp

tmp = tempfile.gettempdir()
def uc_count():
    return len(glob.glob(os.path.join(tmp, "uc_*")))

for i in range(1, 11):
    sb = sb_cdp.Chrome("about:blank")
    sb.quit()
    gc.collect()
    print(f"after run {i}: uc_* temp dirs = {uc_count()}")

The uc_* count goes up by one every run and never comes back down (until the process exits). If you also watch open file descriptors (e.g. ls /proc/self/fd| wc -l on Linux), they keep climbing too. The same happens with Driver(uc=True).

What I found

A few separate things add up:

1. Every Browser is added to the module-global __registered__instances__ set in start() and never removed. Because of that the browser (and its connection, websockets and subprocess pipes) stays referenced forever and can't be garbage-collected, even after the caller drops all references. (seleniumbase/undetected/cdp_driver/browser.py)

2. Browser.stop() ends Chrome but, on the success path, the break skips the line that resets self._process, so the three subprocess pipes (stdin/stdout/stderr) stay open.

3. The auto-generated temp profile dir (uc_*, created by temp_profile_dir() in config.py) is only deleted inside deconstruct_browser(), which only runs via atexit.register(...). stop() never removes it. Note this dir is created even in the connect-existing case, so it's an empty leftover folder every run.

4. sb_cdp.Chrome creates two event loops: one inside cdp_util.start_sync and another in Chrome.__init__. Neither is closed, so each run leaks the loop's selector FD and self-pipe.

5. The CDP event loop is only closed inside the with SB() context manager teardown (sb_manager.py). The Driver(uc=True) path and the pure-CDP path never close it.

So each cycle permanently pins roughly 5-8 file descriptors (selector FDs, self-pipes, the DevTools websocket, the subprocess pipes) plus a temp dir.

What I think could change

The fix that worked for me was to make stop() actually release things instead of leaving it all to atexit:

• Remove the instance from __registered__instances__ in stop() so it can be collected.
• Always clear self._process in stop().
• Remove the auto-generated temp dir in stop() (with a small retry for Windows file locks).
• Have sb_cdp.Chrome reuse a single event loop and remember it on the browser so stop() can close it.
• Close the loop and websockets on the Driver(uc=True) and pure-CDP paths too, not just under with SB(). A small idempotent teardown helper called from quit()/stop() covers all three paths.

I tested this with real Chrome across pure CDP, Driver(uc=True), and with SB(uc=True), looping many start/stop cycles in one process: the uc_* count and the registry size stay flat, the loops get closed, and there's no leftover noise.

I know the project is mostly closed to PRs, so I'm filing this as an issue instead, but I already have a working patch I've validated locally (master...montovaneli:SeleniumBase:leak-fix). Feel free to analyze/use it if you want to.

Thanks a lot for your project!

[mdmintz]

I tried the patch both locally and in GitHub Actions on MacOS, and it didn't seem to work:

https://github.com/mdmintz/seleniumbase-examples/actions/runs/28056645567/job/83060560419#step:9:13

Run python examples/check_for_leaks.py --debug
/var/folders/8j/sfr9qqcj73j4p6nhwcfpr0th0000gn/T
after run 1: uc_* temp dirs = 0
after run 2: uc_* temp dirs = 2
after run 3: uc_* temp dirs = 2
after run 4: uc_* temp dirs = 3
after run 5: uc_* temp dirs = 4

And here's the script that was run: https://github.com/mdmintz/seleniumbase-examples/blob/00f39463cd409263bc8bbe39c9185663eac1cc08/examples/check_for_leaks.py

Your branch was installed with:
pip install --force-reinstall --no-cache-dir git+https://github.com/montovaneli/SeleniumBase.git@leak-fix

Something may be missing from your patch. Try updating it, and run an example from a GitHub Actions job from your repo so that your proposed fix can be confirmed. You can reuse the YML file from https://github.com/mdmintz/seleniumbase-examples/actions/runs/28056645567/workflow to help you confirm the fix.

[montovaneli]

Ah, the fix works on Windows, but it didn’t really work on POSIX, possibly because of SIGTERM. I’ll test it on my Mac Mini and on the Ubuntu server where I run some processes, and I’ll also confirm the fix using the yml file. Thanks!