Skip to content

docs: document retry policy for network errors #150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mre merged 7 commits into from
Jun 7, 2026
Merged

docs: document retry policy for network errors #150

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7a51218
docs: document retry policy for network errors
mre Mar 13, 2026
910d58b
refactor: replace HTML table + SVG icons with plain Markdown table
mre Mar 13, 2026
dfd44e8
refactor: restore original table from issue #149
mre Mar 13, 2026
fda8362
refactor: use SVG icons in table without text labels
mre Mar 13, 2026
161469d
Update src/content/docs/troubleshooting/network-errors.mdx
mre Mar 14, 2026
335aac3
Lowercase lychee in retry policy intro
mre Jun 7, 2026
56a15f6
Clarify connection timeout vs. failure in retry table
mre Jun 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 93 additions & 7 deletions src/content/docs/troubleshooting/network-errors.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,19 @@ title: Network Errors

import { Aside } from "@astrojs/starlight/components";

If you run lychee and some links fail then it could be certificate-related issue.
Certificates are used to verify the identity of a website and to establish a secure connection.
Different operating systems and tools have different ways to handle certificates.
export const Yes = () => <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="#22c55e" strokeWidth="2.5" strokeLinecap="round" strokeLinejoin="round" aria-label="Yes"><polyline points="20 6 9 17 4 12"/></svg>;
export const No = () => <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="#ef4444" strokeWidth="2.5" strokeLinecap="round" strokeLinejoin="round" aria-label="No"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>;

If you run lychee and some links fail, it could be a transient network issue or
a certificate-related problem. Certificates are used to verify the identity of a
website and to establish a secure connection. Different operating systems and
tools have different ways to handle certificates.

If lychee is unable to verify the certificate of a website, it will show a
network error. This could be due to an expired certificate, an invalid
certificate, or a certificate from an unknown authority.

Examples of network errors that could be certificate-related:
Examples of network errors:

```bash
Failed: Network error: error sending request for url
Expand All @@ -22,6 +26,86 @@ Failed: Network error: error sending request for url
Network error: Forbidden
```

```bash
Network error: Connection reset by server. Server forcibly closed connection
```

## Retry Policy

lychee automatically retries failed requests before declaring a link broken.
Not every error is worth retrying &mdash; some are permanent, while others are
transient and may succeed on a second attempt.

The table below summarises which error types trigger a retry:

| Error Type | Retried? | Examples |
| ------------------------------- | -------------- | --------------------------------------------- |
| 5xx Server Errors | <Yes /> | 500, 502, 503, 504 |
| 408 Request Timeout | <Yes /> | Request took too long |
| 429 Too Many Requests | <Yes /> | Rate limit exceeded |
| Connection Timeout | <Yes /> | Server accepted connection but never replied |
| Connection Reset | <Yes /> | Connection dropped unexpectedly |
| Connection Aborted | <Yes /> | Connection terminated mid-request |
| Incomplete Message | <Yes /> | Response cut off before completion |
| 4xx Client Errors | <No /> | 400, 401, 403, 404 (except 408, 429) |
| 2xx Success | <No /> | 200, 201, 204 |
| 3xx Redirects | <No /> | 301, 302 |
| Connection Refused / Failure | <No /> | Connection refused, DNS failure, blocked port |
| Certificate Issues | <No /> | SSL/TLS errors |
| Invalid Request Body | <No /> | Malformed data |
| Decoding Errors | <No /> | Can't parse response |
| Redirect Errors | <No /> | Too many redirects, etc. |

The distinction between the two connection rows matters: a *Connection Timeout*
means lychee reached the server but gave up waiting for a reply, so it's worth
retrying. A *Connection Refused / Failure* means lychee never established a
connection in the first place &mdash; for example a refused port, a firewall
blocking the connection, or a DNS lookup that returned no result &mdash; so a
retry won't help. lychee reports these as `Connection refused - server may be
down or port blocked` or `Connection failed. Check network connectivity and
firewall settings`.

**As a rule of thumb:**
- **Retries:** Temporary problems (server down, network hiccup, timeout)
- **No Retry:** Permanent problems (bad request, auth failure, not found)

### Configuring retries

By default, lychee retries failed requests up to **3 times** with an
exponential backoff starting at **1 second** (i.e. 1 s, 2 s, 4 s).

You can tune this behaviour with two options:

| Option | CLI flag | Config key | Default |
|---|---|---|---|
| Maximum retries | `--max-retries <NUM>` | `max_retries` | `3` |
| Initial wait between retries | `--retry-wait-time <SECS>` | `retry_wait_time` | `1` |

For example, to retry up to 5 times with a 3-second initial wait:

```bash
lychee --max-retries 5 --retry-wait-time 3 https://example.com
```

Or in `lychee.toml`:

```toml
max_retries = 5
retry_wait_time = 3
```

To disable retries entirely, set `--max-retries 0`:

```bash
lychee --max-retries 0 https://example.com
```

<Aside>
Retries only apply to requests where lychee judges the failure to be
transient (see the table above). Permanent errors such as a `404 Not Found`
or a certificate problem are reported immediately without any retry.
</Aside>

## What now?

### Double-check with a different tool
Expand All @@ -36,7 +120,8 @@ If this works, then the issue is with lychee and not the website.
In that case, please open an issue on the lychee GitHub repository.

If it doesn't work, then the issue is with the website or your system.
It might be related to the certificate or the user agent. The site might also use a bot detection such as Cloudflare Bot Management. Read on to find out more.
It might be related to the certificate or the user agent. The site might also
use bot detection such as Cloudflare Bot Management. Read on to find out more.

### Use the `--insecure` flag

Expand Down Expand Up @@ -82,7 +167,7 @@ use a service like [BuiltWith](https://builtwith.com/). Just enter the website
URL and check if it uses any bot detection services like Cloudflare Bot Management or
Cloudflare Challenge.

If the website uses a bot detection service, which is blocking lychee, there's
If the website uses a bot detection service which is blocking lychee, there's
little you can do. You can try contacting the website administrator and ask them
to whitelist lychee.

Expand Down Expand Up @@ -113,4 +198,5 @@ Also, see this [Stack Overflow
discussion](https://stackoverflow.com/a/24618403/270334) for additional
information.

After that, try running lychee again and see if the issue is resolved. If not, please open an issue on the lychee GitHub repository.
After that, try running lychee again and see if the issue is resolved. If not,
please open an issue on the lychee GitHub repository.
Loading