fix(upload): stop retrying 4xx errors and add tests for 6 packages#5
fix(upload): stop retrying 4xx errors and add tests for 6 packages#5
Conversation
- Introduce clientError type to distinguish non-retryable 4xx from retryable 5xx HTTP errors in upload and analysis packages - Populate ScanMetadata.Version in upload payload - Add test suite for internal/pipeline with 10 tests (96.5% coverage) - Add test suite for internal/output with 8 tests (88.3% coverage) - Add test suite for internal/analysis with 7 tests (88.7% coverage) - Add test suite for internal/upload with 7 tests (86.8% coverage) - Add test suite for internal/tlsutil with 5 tests (87.5% coverage) - Add test suite for internal/cli with 4 tests (46.9% coverage) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR improves reliability and efficiency of result uploading and remote analysis by treating 4xx HTTP responses as non-retryable, while also ensuring upload payloads include scanner version metadata. It also adds first-pass test suites across several internal packages to increase confidence in core behaviors.
Changes:
- Stop retrying on HTTP 4xx responses in both upload and remote analysis (retry only on 5xx/network errors).
- Populate
ScanMetadata.Versionin upload payloads. - Add new unit tests for
internal/pipeline,internal/output,internal/analysis,internal/tlsutil,internal/upload, andinternal/cli.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/upload/uploader.go | Adds non-retryable clientError handling for 4xx and includes ScanMetadata.Version in payloads. |
| internal/upload/uploader_test.go | New tests for upload success/headers/metadata + retry/no-retry behaviors. |
| internal/analysis/analyzer.go | Adds non-retryable clientError handling for 4xx in remote analysis retries. |
| internal/analysis/analyzer_test.go | New tests for remote analysis request formation, merge behavior, and retry/no-retry semantics. |
| internal/tlsutil/tlsutil_test.go | New tests for transport cloning and TLS config cloning/skip-verify behavior. |
| internal/pipeline/pipeline_test.go | New tests covering pipeline control flow (inspect-only, paths vs discovery, nil components, continuation on failures). |
| internal/output/output_test.go | New tests for text/json formatting, summaries, truncation, and error-print toggles. |
| internal/cli/cli_test.go | New tests for control server CLI parsing behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Use time.Duration constants instead of raw nanosecond literals in tlsutil tests - Wrap clientError with context (URL, attempt count) in upload error path - Use errors.As directly and remove redundant containsClientError helper - Increase retry test timeouts to avoid flakiness under CI/race detector Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add nonRetryableError type to both upload and analysis packages - Wrap request construction errors as non-retryable (invalid URL won't fix on retry) - Wrap JSON decode errors as non-retryable in analysis doRequest - Only 5xx responses and transient network errors are now retried Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…rittle test - Remove server URL from upload error message to avoid leaking sensitive data - Use structured logging (slog.Error) for URL context instead - Cap error response body reads to 4KB via io.LimitReader in both uploader and analyzer - Compare transport defaults against http.DefaultTransport instead of hard-coded values Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Replace plain bool with atomic.Bool in upload and analysis test handlers - Prevents potential data races between httptest handler and test goroutines Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ent data leaks - Remove Body field from clientError struct in both packages - Log response body at debug level instead of embedding in error strings - Remove duplicate slog.Error call for 4xx errors in uploader Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Include truncated (512B) response body snippet in error messages for debuggability - Restore Body field on clientError with truncated content - Reduce context cancellation test sleep from 2s to 500ms Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add sanitizeBodySnippet helper to replace newlines and control chars - Prevents multi-line log output from raw HTTP response bodies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (2)
internal/upload/uploader.go:120
- The retry loop uses real time-based backoff (1s, 2s) via time.After, which makes unit tests slower and can be sensitive to CI load/context deadlines. If test runtime becomes an issue, consider making backoff/sleep injectable or configurable so tests can run with minimal delays, and optionally use a cancellable timer pattern instead of time.After.
// Retry with exponential backoff (only retry on 5xx / network errors)
maxRetries := 3
for attempt := range maxRetries {
err = u.doUpload(ctx, server, body)
if err == nil {
slog.Info("upload successful", "url", server.URL)
return nil
}
// Do not retry client errors (4xx) or non-retryable errors (e.g., bad URL)
var nre *nonRetryableError
if errors.As(err, &nre) {
return fmt.Errorf("upload failed: %w", err)
}
var ce *clientError
if errors.As(err, &ce) {
return fmt.Errorf(
"upload failed due to non-retryable client error after %d attempt(s) (status=%d): %w",
attempt+1,
ce.StatusCode,
err,
)
}
if attempt < maxRetries-1 {
backoff := time.Duration(1<<uint(attempt)) * time.Second
slog.Debug("retrying upload", "attempt", attempt+1, "backoff", backoff)
select {
case <-time.After(backoff):
case <-ctx.Done():
return ctx.Err()
}
internal/analysis/analyzer.go:163
- The retry loop uses real time-based backoff (1s, 2s) via time.After, which slows tests and can be sensitive to CI load/context deadlines. If this becomes an issue, consider injecting/configuring the backoff/sleeper so tests can run with minimal delays, and optionally use a cancellable timer pattern instead of time.After.
// Retry with exponential backoff (only retry on 5xx / network errors)
var resp analysisResponse
maxRetries := 3
for attempt := range maxRetries {
err = a.doRequest(ctx, body, &resp)
if err == nil {
break
}
// Do not retry non-retryable errors (bad URL, JSON decode, etc.)
var nre *nonRetryableError
if errors.As(err, &nre) {
return fmt.Errorf("analysis API: %w", err)
}
// Do not retry client errors (4xx)
var ce *clientError
if errors.As(err, &ce) {
return fmt.Errorf("analysis API: %w", err)
}
if attempt < maxRetries-1 {
backoff := time.Duration(1<<uint(attempt)) * time.Second
slog.Debug("retrying analysis", "attempt", attempt+1, "backoff", backoff)
select {
case <-time.After(backoff):
case <-ctx.Done():
return ctx.Err()
}
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…havior Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…haracters - Replace strings.NewReplacer (only CR/LF/Tab) with strings.Map + unicode.IsControl - Now correctly strips all Unicode control characters from body snippets Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The error format string included (status=%d) explicitly while also wrapping a clientError whose Error() already produces "status <code>: …", resulting in duplicated status information in the output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Integrate changes from main (PRs #4, #5) into phase3-4 branch: - Keep nonRetryableError type and retry guard in analyzer and uploader - Use atomic.Bool for thread-safe test variables - Use sanitizeBodySnippet with io.LimitReader for safe error body handling - Use more reliable test timeouts (4s/5s instead of 1500ms) - Remove unused statusMessage function (replaced by sanitizeBodySnippet) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
ScanMetadata.Versionis now populated in upload payloads (was always nil)Test Coverage Added
internal/pipeline/internal/output/internal/analysis/internal/tlsutil/internal/upload/internal/cli/Test plan
make test— all tests pass with race detectormake lint— 0 lint issuesmake build— compiles successfullyTestUpload_4xxNoRetryandTestAnalyze_4xxNoRetryTestUpload_5xxRetriesandTestAnalyze_5xxRetries🤖 Generated with Claude Code