Web scraping issue (server side?)

[TsunamicBug]

Trying to fetch a fandom page returns a client error most of the time (At least in docker-compose PostgreSQL)

Steps:

1. Create a new project in the character section
2. When the project is created, find any URL from any popular fandom page
3. Add a context source and fetch

Console output:

app-1        | INFO - 2026-02-07 05:15:13,639 - worker - worker - Worker: Submitting job a7f8255d-5584-4906-9b52-579507834753 (Task: fetch_source_content, Active: 1, Limit: 1)
app-1        | INFO - 2026-02-07 05:15:13,718 - httpx - _client - HTTP Request: GET https://elderscrolls.fandom.com/wiki/Lydia_(Skyrim) "HTTP/1.1 403 Forbidden"
app-1        | ERROR - 2026-02-07 05:15:13,720 - services.background_jobs - background_jobs - [a7f8255d-5584-4906-9b52-579507834753] Failed to fetch content for source dfeec9fe-f0eb-4e79-8e27-f21cd651ecf9: Client error '403 Forbidden' for url 'https://elderscrolls.fandom.com/wiki/Lydia_(Skyrim)'
app-1        | For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403
app-1        | Traceback (most recent call last):
app-1        |   File "/app/server/src/services/background_jobs.py", line 189, in fetch_source_content
app-1        |     content = await scraper.get_content(
app-1        |   File "/app/server/src/services/scraper.py", line 135, in get_content
app-1        |     response.raise_for_status()
app-1        |   File "/usr/local/lib/python3.10/site-packages/httpx/_models.py", line 829, in raise_for_status
app-1        |     raise HTTPStatusError(message, request=request, response=self)
app-1        | httpx.HTTPStatusError: Client error '403 Forbidden' for url 'https://elderscrolls.fandom.com/wiki/Lydia_(Skyrim)'
app-1        | For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

System information:
Operating System: Fedora Linux 43
KDE Plasma Version: 6.5.5
KDE Frameworks Version: 6.22.0
Qt Version: 6.10.1
Kernel Version: 6.18.8-200.fc43.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 7 7800X3D 8-Core Processor
Memory: 16 GiB of RAM (15.4 GiB usable)
Graphics Processor: AMD Radeon RX 6900 XT
Manufacturer: Micro-Star International Co., Ltd.
Product Name: MS-7D74
System Version: 1.0

Edit: Upon further testing this seems to be an issue on Fandom's end as other websites work. On the site itself there was a message about how they were upgrading "platform security".

Edit 2 on 22/3/26: For anyone reading this, I tested this again in a Debian virtual machine, ran this application in Docker and it is the EXACT same issue, so as far as I know, it is Fandom's fault, not a system issue. The solution I found for now is to use a wayback machine link instead. You can test this yourself: https://web.archive.org/web/20260129171639/https://elderscrolls.fandom.com/wiki/Lydia_(Skyrim)