From cb66ba128979bdd4c5c1ea05adca50879c9a7ae1 Mon Sep 17 00:00:00 2001 From: localhost41 Date: Thu, 9 Jul 2026 11:26:15 -0700 Subject: [PATCH] DeepSeek docs for #22 --- Approx tokens/sec: 38.08 | 55 ++++++++++++++++++++++++++++++++++++ docs/validation-checklist.md | 1 + 2 files changed, 56 insertions(+) create mode 100644 Approx tokens/sec: 38.08 diff --git a/Approx tokens/sec: 38.08 b/Approx tokens/sec: 38.08 new file mode 100644 index 0000000..8632158 --- /dev/null +++ b/Approx tokens/sec: 38.08 @@ -0,0 +1,55 @@ + +### 2.4 Failure Triage + +When a real‑QVAC run exits with a non‑zero code or unexpected output, use the +following steps to investigate. + +1. **Validate connectivity** – Verify the endpoint URL with `curl -v $QVAC_URL`. + If the server is reachable, a `200 OK` response should be returned. + +2. **Confirm the model name** – Ensure the value passed to `--model` matches + a model that the endpoint actually supports. Check the provider’s + documentation or query the `/models` endpoint if available. + +3. **Verify the API key** – Double‑check that the key stored in + `QVAC_API_KEY` (or `OPENAI_API_KEY`) is correct and has not expired. + Rotate keys if necessary. + +4. **Check for rate limits** – Repeated runs may trigger `429 Too Many Requests` + responses. Wait a short time and retry, or ask your provider to increase + the quota. + +5. **Inspect sanitised output** – The CLI prints error details to stderr. + Sensitive tokens are redacted, but the remaining information usually + contains the HTTP status code and a human‑readable reason. + ```bash + node dist/cli.js --url ... --model ... 2> error.log + ``` + +6. **Adjust timeouts** – If the error code is `timeout`, increase the request + timeout with `--timeout-ms`. This is especially helpful on high‑latency + connections or for slow models. + +7. **Interpret common error codes** + - `server_unavailable` – The upstream host could not be reached. Check + network access and firewall rules. + - `http_error` – A non‑2xx HTTP status was returned. Look at the sanitised + message for the code (e.g., 401/403 for auth problems, 5xx for + server‑side issues). + - `malformed_stream` – The server sent an unexpected SSE payload. Use + `curl` to inspect the raw JSON lines and confirm the server is + compatible with the OpenAI‑style streaming API. + - `timeout` – The request took longer than the configured timeout. Try + relaxing the limit or checking network latency. + +8. **Test with minimal parameters** – Use a short prompt (`"Hello"`) and a + small `--max-tokens` value to isolate whether the issue is specific to a + longer request. + +9. **Collect logs for further investigation** – Save both stdout and stderr + (e.g., `node dist/cli.js ... > out.txt 2> err.txt`). Share the sanitised + output when seeking help – **never** share raw API keys. + +10. **Compare baseline** – As a last resort, use `curl` to send a simple + request to the same endpoint. If the raw `curl` also fails, the problem + lies with the server or network, not with `qvac-bench`. diff --git a/docs/validation-checklist.md b/docs/validation-checklist.md index 5a122dd..b4b305e 100644 --- a/docs/validation-checklist.md +++ b/docs/validation-checklist.md @@ -90,3 +90,4 @@ pipeline that does not have access to the real endpoint. 7. If the run exits with a non‑zero code or produces unexpected output, refer to the [Failure Triage](#24-failure-triage) section for debugging guidance. **Example expected stdout (single iteration)** +