Error downloading packages

[dappelt]

Description

Downloading some packages causes an error.

This error happens with the latest version of vuln-reach-cli and with the config file at https://gitlab.com/-/snippets/3623569.

The error is intermittent. When rerunning vuln-reach-cli, packages that failed to download before might be downloaded successfully.

I verified that packages that vuln-reach-cli failed to download are available and can be downloaded manually. For example, @aashutoshrathi/word-wrap can be downloaded with $ wget $(npm info @aashutoshrathi/word-wrap dist.tarball).

Expected Behavior

All packages are downloaded and reachability analysis is performed.

Actual Behavior

$ vuln-reach-cli config.toml
Downloading ...
Error: error sending request for url (https://registry.npmjs.org/@aashutoshrathi/word-wrap): error trying to connect: dns error: failed to lookup address information: nodename nor servname provided, or not known

Caused by:
    0: error trying to connect: dns error: failed to lookup address information: nodename nor servname provided, or not known
    1: dns error: failed to lookup address information: nodename nor servname provided, or not known
    2: failed to lookup address information: nodename nor servname provided, or not known

[andreaphylum]

Thank you for reporting this!

It looks like a few connections are failing. The reason is most likely that the CLI is starting all connections concurrently and thus runs into a rate limit of sorts when there is a high number of them.

The tarballs are cached and aren't redownloaded, so as a stopgap it should be enough to re-run the CLI to download the missing packages without incurring in the connection failures from the high concurrency.

On our end, buffering the connections instead of awaiting all of them together should be sufficient to solve this.