Too many calls to redis database when queue is empty or not frequently utilized

[denishsharma]

Hi, I have noticed that there are frequent calls made to the redis server when queue is empty, I did redis-cli MONITOR and there were more than 100 calls made per seconds and I havent checked with knex but there might be the same.

To reproduce the results, just follow the basic setup mentioned in the readme of the repo and then do redis-cli MONITOR in another terminal and check the calls it make.

[RomainLanz]

Hey @denishsharma! 👋🏻

What's the idleDelay you have used in your configuration?

[denishsharma]

@RomainLanz idleDelay is the default one which is set to 2s

[RomainLanz]

And what's the concurrency setup you are using?

[denishsharma]

All the values are default as shown in the readme of the repo. I believe

queue/src/worker.ts

Line 161 in 59556a0

debug('worker %s is idle, waiting for %dms', this.#id, delay)

there should be some sort of sleep or setTimeout here

[denishsharma]

await new Promise<void>(resolve => setTimeout(() => resolve(), delay)) I added this line in my fork and it is now waiting for 2s

[RomainLanz]

We do wait at line 164.

queue/src/worker.ts

Line 164 in 59556a0

await setTimeout(delay)

Can you run the worker using the environment variable NODE_DEBUG=boringnode:* node ....

[denishsharma]

I am trying to use it in nestjs and this was giving me error TypeError [ERR_INVALID_ARG_TYPE]: The "callback" argument must be of type function. Received type number (2000) so I updated to await setTimeout(() => {}, delay) which was causing the issue and then replaced with await new Promise(resolve => setTimeout(resolve, delay)) which fixed the issue.

Also i noticed in the code that Worker is always using the default driver and is not configurable and do you have example of multi worker setup with different drivers?

[RomainLanz]

We use setTimeout from node:timers/promises, so this error shouldn't happen unless something in your runtime swaps that symbol to the callback API (global/polyfill/fake timers or a build transform).

If you can share Node version and Nest build setup (ESM vs CJS, tsconfig module, Jest/fake timers), I can help identify where it's being replaced. The best would be to share a repository with the minimum amount of code to reproduce your error.

Note that the package is ESM‑only, so NestJS must be configured for NodeNext/ESM.

For drivers: the worker currently uses the default adapter because it calls QueueManager.use() without a name. This has to be changed.