From 7a51218ea0d36190be144f368b97abf7f9e99d4d Mon Sep 17 00:00:00 2001 From: Matthias Date: Fri, 13 Mar 2026 18:25:55 +0100 Subject: [PATCH 1/7] docs: document retry policy for network errors Closes #149 - Add a 'Retry Policy' section to the Network Errors troubleshooting page - Include an SVG-icon table listing which error types are retried and which are not (based on the logic in lychee-lib/src/retry.rs) - Document the --max-retries and --retry-wait-time options with their defaults (3 and 1s) - Add 'Connection reset by server' as an example error message - Minor wording improvements throughout --- .../docs/troubleshooting/network-errors.mdx | 168 +++++++++++++++++- 1 file changed, 161 insertions(+), 7 deletions(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index 9868312..8ecc241 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -4,15 +4,16 @@ title: Network Errors import { Aside } from "@astrojs/starlight/components"; -If you run lychee and some links fail then it could be certificate-related issue. -Certificates are used to verify the identity of a website and to establish a secure connection. -Different operating systems and tools have different ways to handle certificates. +If you run lychee and some links fail, it could be a transient network issue or +a certificate-related problem. Certificates are used to verify the identity of a +website and to establish a secure connection. Different operating systems and +tools have different ways to handle certificates. If lychee is unable to verify the certificate of a website, it will show a network error. This could be due to an expired certificate, an invalid certificate, or a certificate from an unknown authority. -Examples of network errors that could be certificate-related: +Examples of network errors: ```bash Failed: Network error: error sending request for url @@ -22,6 +23,157 @@ Failed: Network error: error sending request for url Network error: Forbidden ``` +```bash +Network error: Connection reset by server. Server forcibly closed connection +``` + +## Retry Policy + +Lychee automatically retries failed requests before declaring a link broken. +Not every error is worth retrying — some are permanent, while others are +transient and may succeed on a second attempt. + +The table below summarises which error types trigger a retry: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Error typeExampleRetried?
TimeoutNetwork error: operation timed out + + + +
Too Many Requests (429)Network error: too many requests + + + +
Server error (5xx)Response: 503 Service Unavailable + + + +
Request timeout (408)Response: 408 Request Timeout + + + +
Incomplete responseNetwork error: connection closed before message completed + + + +
Connection reset / abortedNetwork error: Connection reset by server + + + +
Connection refused / errorNetwork error: connection refused + + + +
Certificate / TLS errorNetwork error: invalid certificate + + + +
Redirect errorFailed: Too many redirects + + + +
Client error (4xx, except 408/429)Response: 404 Not Found + + + +
+ +### Configuring retries + +By default, lychee retries failed requests up to **3 times** with an +exponential backoff starting at **1 second** (i.e. 1 s, 2 s, 4 s). + +You can tune this behaviour with two options: + +| Option | CLI flag | Config key | Default | +|---|---|---|---| +| Maximum retries | `--max-retries ` | `max_retries` | `3` | +| Initial wait between retries | `--retry-wait-time ` | `retry_wait_time` | `1` | + +For example, to retry up to 5 times with a 5-second initial wait: + +```bash +lychee --max-retries 5 --retry-wait-time 3 https://example.com +``` + +Or in `lychee.toml`: + +```toml +max_retries = 5 +retry_wait_time = 3 +``` + +To disable retries entirely, set `--max-retries 0`: + +```bash +lychee --max-retries 0 https://example.com +``` + + + ## What now? ### Double-check with a different tool @@ -36,7 +188,8 @@ If this works, then the issue is with lychee and not the website. In that case, please open an issue on the lychee GitHub repository. If it doesn't work, then the issue is with the website or your system. -It might be related to the certificate or the user agent. The site might also use a bot detection such as Cloudflare Bot Management. Read on to find out more. +It might be related to the certificate or the user agent. The site might also +use bot detection such as Cloudflare Bot Management. Read on to find out more. ### Use the `--insecure` flag @@ -82,7 +235,7 @@ use a service like [BuiltWith](https://builtwith.com/). Just enter the website URL and check if it uses any bot detection services like Cloudflare Bot Management or Cloudflare Challenge. -If the website uses a bot detection service, which is blocking lychee, there's +If the website uses a bot detection service which is blocking lychee, there's little you can do. You can try contacting the website administrator and ask them to whitelist lychee. @@ -113,4 +266,5 @@ Also, see this [Stack Overflow discussion](https://stackoverflow.com/a/24618403/270334) for additional information. -After that, try running lychee again and see if the issue is resolved. If not, please open an issue on the lychee GitHub repository. +After that, try running lychee again and see if the issue is resolved. If not, +please open an issue on the lychee GitHub repository. \ No newline at end of file From 910d58b255d8e99857bb4dfc5e1c224d25c07dff Mon Sep 17 00:00:00 2001 From: Matthias Date: Fri, 13 Mar 2026 21:47:36 +0100 Subject: [PATCH 2/7] refactor: replace HTML table + SVG icons with plain Markdown table --- .../docs/troubleshooting/network-errors.mdx | 113 ++---------------- 1 file changed, 12 insertions(+), 101 deletions(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index 8ecc241..ace8724 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -35,107 +35,18 @@ transient and may succeed on a second attempt. The table below summarises which error types trigger a retry: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Error typeExampleRetried?
TimeoutNetwork error: operation timed out - - - -
Too Many Requests (429)Network error: too many requests - - - -
Server error (5xx)Response: 503 Service Unavailable - - - -
Request timeout (408)Response: 408 Request Timeout - - - -
Incomplete responseNetwork error: connection closed before message completed - - - -
Connection reset / abortedNetwork error: Connection reset by server - - - -
Connection refused / errorNetwork error: connection refused - - - -
Certificate / TLS errorNetwork error: invalid certificate - - - -
Redirect errorFailed: Too many redirects - - - -
Client error (4xx, except 408/429)Response: 404 Not Found - - - -
+| Error type | Example | Retried? | +| ---------------------------------- | -------------------------------------------------------------- | -------- | +| Timeout | `Network error: operation timed out` | ✅ | +| Too Many Requests (429) | `Network error: too many requests` | ✅ | +| Server error (5xx) | `Response: 503 Service Unavailable` | ✅ | +| Request timeout (408) | `Response: 408 Request Timeout` | ✅ | +| Incomplete response | `Network error: connection closed before message completed` | ✅ | +| Connection reset / aborted | `Network error: Connection reset by server` | ✅ | +| Connection refused / error | `Network error: connection refused` | ❌ | +| Certificate / TLS error | `Network error: invalid certificate` | ❌ | +| Redirect error | `Failed: Too many redirects` | ❌ | +| Client error (4xx, except 408/429) | `Response: 404 Not Found` | ❌ | ### Configuring retries From dfd44e867091c98118792af7b65c44d7d01290cf Mon Sep 17 00:00:00 2001 From: Matthias Date: Fri, 13 Mar 2026 21:50:56 +0100 Subject: [PATCH 3/7] refactor: restore original table from issue #149 --- .../docs/troubleshooting/network-errors.mdx | 33 ++++++++++++------- 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index ace8724..bb92189 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -35,18 +35,27 @@ transient and may succeed on a second attempt. The table below summarises which error types trigger a retry: -| Error type | Example | Retried? | -| ---------------------------------- | -------------------------------------------------------------- | -------- | -| Timeout | `Network error: operation timed out` | ✅ | -| Too Many Requests (429) | `Network error: too many requests` | ✅ | -| Server error (5xx) | `Response: 503 Service Unavailable` | ✅ | -| Request timeout (408) | `Response: 408 Request Timeout` | ✅ | -| Incomplete response | `Network error: connection closed before message completed` | ✅ | -| Connection reset / aborted | `Network error: Connection reset by server` | ✅ | -| Connection refused / error | `Network error: connection refused` | ❌ | -| Certificate / TLS error | `Network error: invalid certificate` | ❌ | -| Redirect error | `Failed: Too many redirects` | ❌ | -| Client error (4xx, except 408/429) | `Response: 404 Not Found` | ❌ | +| Error Type | Retried? | Examples | +| ------------------------------- | -------- | --------------------------------------------- | +| 5xx Server Errors | ✅ Yes | 500, 502, 503, 504 | +| 408 Request Timeout | ✅ Yes | Request took too long | +| 429 Too Many Requests | ✅ Yes | Rate limit exceeded | +| Connection Timeout | ✅ Yes | Server didn't respond in time | +| Connection Reset | ✅ Yes | Connection dropped unexpectedly | +| Connection Aborted | ✅ Yes | Connection terminated mid-request | +| Incomplete Message | ✅ Yes | Response cut off before completion | +| 4xx Client Errors | ❌ No | 400, 401, 403, 404 (except 408, 429) | +| 2xx Success | ❌ No | 200, 201, 204 | +| 3xx Redirects | ❌ No | 301, 302 | +| Initial Connection Failure | ❌ No | Can't reach server at all | +| Certificate Issues | ❌ No | SSL/TLS errors | +| Invalid Request Body | ❌ No | Malformed data | +| Decoding Errors | ❌ No | Can't parse response | +| Redirect Errors | ❌ No | Too many redirects, etc. | + +**As a rule of thumb:** +- **Retries:** Temporary problems (server down, network hiccup, timeout) +- **No Retry:** Permanent problems (bad request, auth failure, not found) ### Configuring retries From fda8362d90c4aa4dab6e9f4504bd95de628c14a5 Mon Sep 17 00:00:00 2001 From: Matthias Date: Fri, 13 Mar 2026 21:52:20 +0100 Subject: [PATCH 4/7] refactor: use SVG icons in table without text labels --- .../docs/troubleshooting/network-errors.mdx | 37 ++++++++++--------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index bb92189..00de037 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -4,6 +4,9 @@ title: Network Errors import { Aside } from "@astrojs/starlight/components"; +export const Yes = () => ; +export const No = () => ; + If you run lychee and some links fail, it could be a transient network issue or a certificate-related problem. Certificates are used to verify the identity of a website and to establish a secure connection. Different operating systems and @@ -35,23 +38,23 @@ transient and may succeed on a second attempt. The table below summarises which error types trigger a retry: -| Error Type | Retried? | Examples | -| ------------------------------- | -------- | --------------------------------------------- | -| 5xx Server Errors | ✅ Yes | 500, 502, 503, 504 | -| 408 Request Timeout | ✅ Yes | Request took too long | -| 429 Too Many Requests | ✅ Yes | Rate limit exceeded | -| Connection Timeout | ✅ Yes | Server didn't respond in time | -| Connection Reset | ✅ Yes | Connection dropped unexpectedly | -| Connection Aborted | ✅ Yes | Connection terminated mid-request | -| Incomplete Message | ✅ Yes | Response cut off before completion | -| 4xx Client Errors | ❌ No | 400, 401, 403, 404 (except 408, 429) | -| 2xx Success | ❌ No | 200, 201, 204 | -| 3xx Redirects | ❌ No | 301, 302 | -| Initial Connection Failure | ❌ No | Can't reach server at all | -| Certificate Issues | ❌ No | SSL/TLS errors | -| Invalid Request Body | ❌ No | Malformed data | -| Decoding Errors | ❌ No | Can't parse response | -| Redirect Errors | ❌ No | Too many redirects, etc. | +| Error Type | Retried? | Examples | +| ------------------------------- | -------------- | --------------------------------------------- | +| 5xx Server Errors | | 500, 502, 503, 504 | +| 408 Request Timeout | | Request took too long | +| 429 Too Many Requests | | Rate limit exceeded | +| Connection Timeout | | Server didn't respond in time | +| Connection Reset | | Connection dropped unexpectedly | +| Connection Aborted | | Connection terminated mid-request | +| Incomplete Message | | Response cut off before completion | +| 4xx Client Errors | | 400, 401, 403, 404 (except 408, 429) | +| 2xx Success | | 200, 201, 204 | +| 3xx Redirects | | 301, 302 | +| Initial Connection Failure | | Can't reach server at all | +| Certificate Issues | | SSL/TLS errors | +| Invalid Request Body | | Malformed data | +| Decoding Errors | | Can't parse response | +| Redirect Errors | | Too many redirects, etc. | **As a rule of thumb:** - **Retries:** Temporary problems (server down, network hiccup, timeout) From 161469d948dd1f6324c1106ac5b429110bc4a3da Mon Sep 17 00:00:00 2001 From: Matthias Endler Date: Sat, 14 Mar 2026 11:22:38 +0100 Subject: [PATCH 5/7] Update src/content/docs/troubleshooting/network-errors.mdx Co-authored-by: Kait <39479354+katrinafyi@users.noreply.github.com> --- src/content/docs/troubleshooting/network-errors.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index 00de037..4326427 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -72,7 +72,7 @@ You can tune this behaviour with two options: | Maximum retries | `--max-retries ` | `max_retries` | `3` | | Initial wait between retries | `--retry-wait-time ` | `retry_wait_time` | `1` | -For example, to retry up to 5 times with a 5-second initial wait: +For example, to retry up to 5 times with a 3-second initial wait: ```bash lychee --max-retries 5 --retry-wait-time 3 https://example.com From 335aac3fafa277ba0d5c7130a4cdf84768e722af Mon Sep 17 00:00:00 2001 From: Matthias Date: Sun, 7 Jun 2026 12:29:55 +0200 Subject: [PATCH 6/7] Lowercase lychee in retry policy intro --- src/content/docs/troubleshooting/network-errors.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index 4326427..659708f 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -32,7 +32,7 @@ Network error: Connection reset by server. Server forcibly closed connection ## Retry Policy -Lychee automatically retries failed requests before declaring a link broken. +lychee automatically retries failed requests before declaring a link broken. Not every error is worth retrying — some are permanent, while others are transient and may succeed on a second attempt. From 56a15f609981d89f9915ab3e0b802b7f4dfd4221 Mon Sep 17 00:00:00 2001 From: Matthias Date: Sun, 7 Jun 2026 12:30:05 +0200 Subject: [PATCH 7/7] Clarify connection timeout vs. failure in retry table --- src/content/docs/troubleshooting/network-errors.mdx | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/src/content/docs/troubleshooting/network-errors.mdx b/src/content/docs/troubleshooting/network-errors.mdx index 659708f..36c6a25 100644 --- a/src/content/docs/troubleshooting/network-errors.mdx +++ b/src/content/docs/troubleshooting/network-errors.mdx @@ -43,19 +43,28 @@ The table below summarises which error types trigger a retry: | 5xx Server Errors | | 500, 502, 503, 504 | | 408 Request Timeout | | Request took too long | | 429 Too Many Requests | | Rate limit exceeded | -| Connection Timeout | | Server didn't respond in time | +| Connection Timeout | | Server accepted connection but never replied | | Connection Reset | | Connection dropped unexpectedly | | Connection Aborted | | Connection terminated mid-request | | Incomplete Message | | Response cut off before completion | | 4xx Client Errors | | 400, 401, 403, 404 (except 408, 429) | | 2xx Success | | 200, 201, 204 | | 3xx Redirects | | 301, 302 | -| Initial Connection Failure | | Can't reach server at all | +| Connection Refused / Failure | | Connection refused, DNS failure, blocked port | | Certificate Issues | | SSL/TLS errors | | Invalid Request Body | | Malformed data | | Decoding Errors | | Can't parse response | | Redirect Errors | | Too many redirects, etc. | +The distinction between the two connection rows matters: a *Connection Timeout* +means lychee reached the server but gave up waiting for a reply, so it's worth +retrying. A *Connection Refused / Failure* means lychee never established a +connection in the first place — for example a refused port, a firewall +blocking the connection, or a DNS lookup that returned no result — so a +retry won't help. lychee reports these as `Connection refused - server may be +down or port blocked` or `Connection failed. Check network connectivity and +firewall settings`. + **As a rule of thumb:** - **Retries:** Temporary problems (server down, network hiccup, timeout) - **No Retry:** Permanent problems (bad request, auth failure, not found)