[Bug]: Coudnt get any new Listings

[strech345]

Bug Description

Normally i got every day some new listings. A week ago i hat the problem that after some time running, it woudnt found any new listings. Frontend was still running and reacting. I update to the newest version at that time (19.3.4). Than it found new listings. But again after some days it is still silent.

Im using docker with portainer on an an small linux server.
logs:

TROUBLESHOOTING: https://pptr.dev/troubleshooting
    at ChildProcess.onClose (/fredy/node_modules/@puppeteer/browsers/lib/cjs/launch.js:340:24)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
[2026-02-09 13:59:20] WARN: Error executing with puppeteer executor Error: Failed to launch the browser process:  spawn /usr/bin/chromium-browser EAGAIN
stderr:
TROUBLESHOOTING: https://pptr.dev/troubleshooting
    at ChildProcess.onClose (/fredy/node_modules/@puppeteer/browsers/lib/cjs/launch.js:340:24)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
[2026-02-09 13:59:20] WARN: Error executing with puppeteer executor Error: Failed to launch the browser process:  Code: 2
stderr:
/usr/bin/chromium-browser: line 12: can't fork: Resource temporarily unavailable
TROUBLESHOOTING: https://pptr.dev/troubleshooting
    at ChildProcess.onClose (/fredy/node_modules/@puppeteer/browsers/lib/cjs/launch.js:340:24)
    at ChildProcess.emit (node:events:531:35)
    at ChildProcess._handle.onexit (node:internal/child_process:293:12)
[2026-02-09 13:59:21] WARN: Error executing with puppeteer executor Error: Failed to launch the browser process:  Code: null
stderr:
[204360:204360:0209/135920.836021:FATAL:third_party/crashpad/crashpad/util/posix/spawn_subprocess.cc:237] posix_spawn /usr/lib/chromium/chrome_crashpad_handler: Resource temporarily unavailable (11)

Steps to Reproduce

_

Expected Behavior

_

Actual Behavior

_

Screenshots / Logs

No response

Environment

_

Additional Context

container version: 19.3.4
images run with 700mb ram
docker version: Standalone 29.1.5
linux kernel: Linux 6.12.57+deb12-amd64

[strech345]

answer of gemini:

The reported issue is caused by resource exhaustion (EAGAIN: Resource temporarily unavailable) when the application attempts to launch multiple Chromium instances concurrently.

• Environment: Docker with 700MB RAM.
• Root Cause: lib/services/jobs/jobExecutionService.js launches all jobs and all their providers concurrently.
  • runAll() uses forEach without awaiting executeJob, starting all jobs at once.
  • executeJob() uses Promise.allSettled() to run all providers of a job at once.
• Result: In a setup with many jobs or providers, the system runs out of PIDs or memory, causing Chromium to fail to fork.

Proposed Changes

lib/services/jobs/jobExecutionService.js

• Modify runAll() to process jobs sequentially or with a small concurrency limit.
• Modify executeJob() to process providers sequentially instead of using Promise.allSettled() for all at once.

Note

For resource-constrained environments like 700MB RAM, sequential execution is the safest default.

maybe i can create this image with this changes and test it on my server. will see ;-)

[orangecoding]

tbh, I've never seen / heard about this.
What is the interval you run your job? How many jobs in parallel and how many provider per job?

Fredy is built in a way, it re-uses a given Puppeteer instance. it is not spawning a new one for every new job actually

[strech345]

1 Job with 12 Providers every 15mins

[orangecoding]

Ok. Can you 'reliably' reproduce it? Or does it happen randomly?

[strech345]

i can check again. i now restart the container. this are the first logs

[2026-02-14 14:21:46] INFO: Could not find config file. Will create one with default values now
[2026-02-14 14:21:48] WARN: No sqlitepath configured. Using default db/listings.db
[2026-02-14 14:21:48] INFO: Started Fredy successfully. Ui can be accessed via http://localhost:9998
[2026-02-14 14:22:36] WARN: Error executing with puppeteer executor TimeoutError: Navigation timeout of 30000 ms exceeded
    at new Deferred (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:60:34)
    at Deferred.create (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:21:16)
    at new LifecycleWatcher (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:73:60)
    at CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:149:29)
    at CdpFrame.<anonymous> (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)
    at CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:43)
    at execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:78:33)
    at runNextTicks (node:internal/process/task_queues:65:5)
    at process.processImmediate (node:internal/timers:453:9)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-14 14:22:36] WARN: Error executing with puppeteer executor TimeoutError: Navigation timeout of 30000 ms exceeded
    at new Deferred (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:60:34)
    at Deferred.create (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:21:16)
    at new LifecycleWatcher (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:73:60)
    at CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:149:29)
    at CdpFrame.<anonymous> (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)
    at CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:43)
    at execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:78:33)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-14 14:22:36] WARN: Error executing with puppeteer executor TimeoutError: Navigation timeout of 30000 ms exceeded
    at new Deferred (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:60:34)
    at Deferred.create (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:21:16)
    at new LifecycleWatcher (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:73:60)
    at CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:149:29)
    at CdpFrame.<anonymous> (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)
    at CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:43)
    at execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:78:33)
    at runNextTicks (node:internal/process/task_queues:65:5)
    at process.processImmediate (node:internal/timers:453:9)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-14 14:22:37] WARN: Error executing with puppeteer executor TimeoutError: Navigation timeout of 30000 ms exceeded
    at new Deferred (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:60:34)
    at Deferred.create (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:21:16)
    at new LifecycleWatcher (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:73:60)
    at CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:149:29)
    at CdpFrame.<anonymous> (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)
    at CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:43)
    at execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:78:33)
    at runNextTicks (node:internal/process/task_queues:65:5)
    at process.processImmediate (node:internal/timers:453:9)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-14 14:22:37] WARN: Error executing with puppeteer executor TimeoutError: Navigation timeout of 30000 ms exceeded
    at new Deferred (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:60:34)
    at Deferred.create (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:21:16)
    at new LifecycleWatcher (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:73:60)
    at CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:149:29)
    at CdpFrame.<anonymous> (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)
    at CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:43)
    at execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:78:33)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-14 14:22:48] WARN: We have been detected as a bot :-/ Tried url: =>  https://www.mcmakler.de/immobilien/results?page=0&placeId=4251865&propertyTypes=APARTMENT&search=XXXXXXX&sortBy=DATE&sortOn=DESC
[2026-02-14 14:22:50] ERROR: Error sending photo to Telegram and use a fallback: API error for 'XXXXX'. 'sendPhoto' returned {"ok":false,"error_code":400,"description":"Bad Request: failed to get HTTP URL content"}

[strech345]

at least i found out that it did not found my config. fixed it and will now see...

[orangecoding]

The config log is not an error. If you start Fredy for the first time, it creates the config for you.

[orangecoding]

See #270

[strech345]

hey thanks. do you will build a new version, or should i first build and test it on my system?

[orangecoding]

Im currently testing it on my system with about 150 jobs on a pi and a couple of other low powered machines. Give it 1 more day plz

[orangecoding]

Fixed here:
https://github.com/orangecoding/fredy/releases/tag/19.5.0

[strech345]

@orangecoding thanks for solving it. Looks good, its still running :-).

still get some errors, not shure if this is important.

[2026-02-18 20:53:59] INFO: Started Fredy successfully. Ui can be accessed via http://localhost:9998
[2026-02-18 21:24:11] WARN: Error executing with puppeteer executor Error: net::ERR_CONNECTION_REFUSED at https://www.1a-immobilienmarkt.de/suchen/XXXX/wohnung-mieten.html?data_hash=25bc3491498c5cb2d17c90cd4d25fc7a&search=yes&sort_type=newest
    at navigate (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:184:27)
    at async Deferred.race (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:36:20)
    at async CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:150:25)
    at async CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:20)
    at async execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:106:22)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-18 23:39:11] WARN: Error executing with puppeteer executor Error: net::ERR_CONNECTION_REFUSED at https://www.1a-immobilienmarkt.de/suchen/XXXX/wohnung-mieten.html?data_hash=25bc3491498c5cb2d17c90cd4d25fc7a&search=yes&sort_type=newest
    at navigate (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:184:27)
    at async Deferred.race (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:36:20)
    at async CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:150:25)
    at async CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:20)
    at async execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:106:22)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-19 03:39:11] WARN: Error executing with puppeteer executor Error: net::ERR_CONNECTION_REFUSED at https://www.1a-immobilienmarkt.de/suchen/XXXX/wohnung-mieten.html?data_hash=25bc3491498c5cb2d17c90cd4d25fc7a&search=yes&sort_type=newest
    at navigate (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:184:27)
    at async Deferred.race (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:36:20)
    at async CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:150:25)
    at async CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:20)
    at async execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:106:22)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)
[2026-02-19 06:09:00] ERROR: Error: Failed to launch the browser process:  Code: null

stderr:
[10211:10228:0219/060900.500153:ERROR:dbus/bus.cc:406] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[10211:10228:0219/060900.520125:ERROR:dbus/bus.cc:406] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
[0219/060900.549832:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.550670:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.550867:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.551343:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.551811:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.552378:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.552592:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.552735:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.553136:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.553725:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.553895:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.553994:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.554078:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.554383:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.554938:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.555276:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.555619:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.555906:WARNING:third_party/crashpad/crashpad/snapshot/linux/process_reader_linux.cc:95] sched_getscheduler: Function not implemented (38)
[0219/060900.557950:ERROR:third_party/crashpad/crashpad/util/file/file_io_posix.cc:145] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)
[0219/060900.558014:ERROR:third_party/crashpad/crashpad/util/file/file_io_posix.cc:145] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq: No such file or directory (2)

TROUBLESHOOTING: https://pptr.dev/troubleshooting

    at ChildProcess.onClose (/fredy/node_modules/@puppeteer/browsers/lib/cjs/launch.js:350:24)
    at ChildProcess.emit (node:events:531:35)
    at ChildProcess._handle.onexit (node:internal/child_process:293:12)
[2026-02-19 07:39:11] WARN: Error executing with puppeteer executor Error: net::ERR_CONNECTION_REFUSED at https://www.1a-immobilienmarkt.de/suchen/XXXX/wohnung-mieten.html?data_hash=25bc3491498c5cb2d17c90cd4d25fc7a&search=yes&sort_type=newest
    at navigate (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:184:27)
    at async Deferred.race (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:36:20)
    at async CdpFrame.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:150:25)
    at async CdpPage.goto (/fredy/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:580:20)
    at async execute (file:///fredy/lib/services/extractor/puppeteerExtractor.js:106:22)
    at async Extractor.execute (file:///fredy/lib/services/extractor/extractor.js:36:27)

[orangecoding]

Here are 2 errors.

net::ERR_CONNECTION_REFUSED
This indicates either a Network issue on your side or the provider is unreachable.

Failed to launch the browser process
That indicates Puppeteer can't start chrome. Tbh as you are rly the only one that keeps reporting issues, I'd think it's something on your side. Maybe you limited the resources for the container or something like that.