diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index 9868312..36c6a25 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -4,15 +4,19 @@ title: Network Errors import { Aside } from "@astrojs/starlight/components"; -If you run lychee and some links fail then it could be certificate-related issue. -Certificates are used to verify the identity of a website and to establish a secure connection. -Different operating systems and tools have different ways to handle certificates. +export const Yes = () => ; +export const No = () => ; + +If you run lychee and some links fail, it could be a transient network issue or +a certificate-related problem. Certificates are used to verify the identity of a +website and to establish a secure connection. Different operating systems and +tools have different ways to handle certificates. If lychee is unable to verify the certificate of a website, it will show a network error. This could be due to an expired certificate, an invalid certificate, or a certificate from an unknown authority. -Examples of network errors that could be certificate-related: +Examples of network errors: ```bash Failed: Network error: error sending request for url @@ -22,6 +26,86 @@ Failed: Network error: error sending request for url Network error: Forbidden ``` +```bash +Network error: Connection reset by server. Server forcibly closed connection +``` + +## Retry Policy + +lychee automatically retries failed requests before declaring a link broken. +Not every error is worth retrying — some are permanent, while others are +transient and may succeed on a second attempt. + +The table below summarises which error types trigger a retry: + +| Error Type | Retried? | Examples | +| ------------------------------- | -------------- | --------------------------------------------- | +| 5xx Server Errors | | 500, 502, 503, 504 | +| 408 Request Timeout | | Request took too long | +| 429 Too Many Requests | | Rate limit exceeded | +| Connection Timeout | | Server accepted connection but never replied | +| Connection Reset | | Connection dropped unexpectedly | +| Connection Aborted | | Connection terminated mid-request | +| Incomplete Message | | Response cut off before completion | +| 4xx Client Errors | | 400, 401, 403, 404 (except 408, 429) | +| 2xx Success | | 200, 201, 204 | +| 3xx Redirects | | 301, 302 | +| Connection Refused / Failure | | Connection refused, DNS failure, blocked port | +| Certificate Issues | | SSL/TLS errors | +| Invalid Request Body | | Malformed data | +| Decoding Errors | | Can't parse response | +| Redirect Errors | | Too many redirects, etc. | + +The distinction between the two connection rows matters: a *Connection Timeout* +means lychee reached the server but gave up waiting for a reply, so it's worth +retrying. A *Connection Refused / Failure* means lychee never established a +connection in the first place — for example a refused port, a firewall +blocking the connection, or a DNS lookup that returned no result — so a +retry won't help. lychee reports these as `Connection refused - server may be +down or port blocked` or `Connection failed. Check network connectivity and +firewall settings`. + +**As a rule of thumb:** +- **Retries:** Temporary problems (server down, network hiccup, timeout) +- **No Retry:** Permanent problems (bad request, auth failure, not found) + +### Configuring retries + +By default, lychee retries failed requests up to **3 times** with an +exponential backoff starting at **1 second** (i.e. 1 s, 2 s, 4 s). + +You can tune this behaviour with two options: + +| Option | CLI flag | Config key | Default | +|---|---|---|---| +| Maximum retries | `--max-retries ` | `max_retries` | `3` | +| Initial wait between retries | `--retry-wait-time ` | `retry_wait_time` | `1` | + +For example, to retry up to 5 times with a 3-second initial wait: + +```bash +lychee --max-retries 5 --retry-wait-time 3 https://example.com +``` + +Or in `lychee.toml`: + +```toml +max_retries = 5 +retry_wait_time = 3 +``` + +To disable retries entirely, set `--max-retries 0`: + +```bash +lychee --max-retries 0 https://example.com +``` + + + ## What now? ### Double-check with a different tool @@ -36,7 +120,8 @@ If this works, then the issue is with lychee and not the website. In that case, please open an issue on the lychee GitHub repository. If it doesn't work, then the issue is with the website or your system. -It might be related to the certificate or the user agent. The site might also use a bot detection such as Cloudflare Bot Management. Read on to find out more. +It might be related to the certificate or the user agent. The site might also +use bot detection such as Cloudflare Bot Management. Read on to find out more. ### Use the `--insecure` flag @@ -82,7 +167,7 @@ use a service like [BuiltWith](https://builtwith.com/). Just enter the website URL and check if it uses any bot detection services like Cloudflare Bot Management or Cloudflare Challenge. -If the website uses a bot detection service, which is blocking lychee, there's +If the website uses a bot detection service which is blocking lychee, there's little you can do. You can try contacting the website administrator and ask them to whitelist lychee. @@ -113,4 +198,5 @@ Also, see this [Stack Overflow discussion](https://stackoverflow.com/a/24618403/270334) for additional information. -After that, try running lychee again and see if the issue is resolved. If not, please open an issue on the lychee GitHub repository. +After that, try running lychee again and see if the issue is resolved. If not, +please open an issue on the lychee GitHub repository. \ No newline at end of file