UC in multiprocessing not working

[bersena]

Hi Everyone, I am trying to run SB in parallel. in this code I am expecting output to be:

URL: https://google.com/ncr - Title: Google
URL: https://www.python.org - Title: Welcome to Python.org
URL: https://www.example.com - Title: Example Domain
URL: https://www.wikipedia.org - Title: Wikipedia
URL: https://www.github.com - Title: GitHub · Build and ship software on a single, collaborative platform · GitHub

but as soon as I activate uc=True or even with uc_subprocess=True. I even tried passing user_data_dir also, it gives out all sorts of problems like this:

URL: https://google.com/ncr - Title: Google
URL: https://www.python.org - Title: Wikipedia
URL: https://www.example.com - Title: Example Domain
URL: https://www.wikipedia.org - Title: None
URL: https://www.github.com - Title: GitHub · Build and ship software on a single, collaborative platform · GitHub

or this

URL: https://google.com/ncr - Title: GitHub · Build and ship software on a single, collaborative platform · GitHub
URL: https://www.python.org - Title: GitHub · Build and ship software on a single, collaborative platform · GitHub
URL: https://www.example.com - Title: Example Domain
URL: https://www.wikipedia.org - Title: GitHub · Build and ship software on a single, collaborative platform · GitHub
URL: https://www.github.com - Title: GitHub · Build and ship software on a single, collaborative platform · GitHub

Or even sometimes only one window is opening all urls, and others getting stalled. and does not even quit.

from seleniumbase import SB
import concurrent.futures


def run_instance(url):
    with SB(uc=True, uc_subprocess=True) as sb:
        sb.open(url)
        sb.sleep(5)
        title = sb.get_page_title()
        return f"URL: {url} - Title: {title}"


if __name__ == '__main__':
    # List of URLs to process
    urls = [
        "https://google.com/ncr",
        "https://www.python.org",
        "https://www.example.com",
        "https://www.wikipedia.org",
        "https://www.github.com",
        # Add more URLs as needed
    ]

    # Run the instances with 5 parallel workers
    with concurrent.futures.ProcessPoolExecutor(max_workers=5) as executor:
        results = executor.map(run_instance, urls)

    # Print the title retrieved from each URL
    for result in results:
        print(result)

so I suppose UC mode does not support multithreading or multiprocessing.

is there a way I can parallelize opening urls in different windows using UC mode?

[mdmintz]

See the example for multithreading with CDP Mode:
SeleniumBase/examples/cdp_mode/raw_multi_cdp.py

Also see SeleniumBase/examples/raw_multi_drivers.py

You'll need to use ThreadPoolExecutor.