Add comprehensive test coverage for scanner, enrichment, and CLI

[beejak]

Summary

This PR adds extensive test coverage across the scanner, enrichment, report generation, and CLI components. The changes include unit tests, integration tests, and end-to-end tests that verify core functionality without requiring external services (via mocking).

Key Changes

New Test Files

• pkg/kev/kev_test.go: Unit tests for CISA KEV catalog loading, caching, HTTP error handling, concurrent safety, and case normalization
• pkg/remediate/enrich_integration_test.go: Integration tests for enrichment pipeline with mocked KEV server, verifying exploit flagging, severity upgrades, and ransomware detection
• pkg/report/report_test.go (additions): SARIF 2.1 structural validation tests covering required fields, tool driver metadata, result formatting, and rule deduplication
• pkg/scanner/parse_test.go: Fixture-based tests for Trivy JSON parsing, finding conversion, severity distribution, and remediation link population
• cmd/cli/main_test.go: CLI exit-code tests using fake Trivy binary, verifying policy enforcement (--fail-on-severity, --fail-on-count), and error handling
• cmd/server/main_test.go: HTTP handler tests for /health, /api/scan, and summary building logic
• cmd/baseline/main_test.go: Tests for image loading, CSV escaping, and markdown report generation
• cmd/mcp-server/main_test.go: Tests for severity parsing and summary formatting
• pkg/runc/runc_test.go (additions): Tests for HostVersion() with mocked Docker/runc binaries

Test Infrastructure

• pkg/scanner/testdata/trivy-fixture.json: Minimal Trivy JSON fixture (5 vulnerabilities across severity levels) for reproducible parsing tests
• Mock HTTP servers: KEV and Trivy endpoints mocked in tests to avoid network dependencies
• Fake binary helpers: Scripts for testing CLI integration with Trivy and runc version detection
• Cache reset utilities: Test helpers to isolate KEV and runc state between tests

Documentation Updates

• docs/testing.md: Reorganized test matrix with current status; added CLI exit-code, enrichment integration, and server tests
• docs/architecture-diagrams.md: Updated flowcharts to include rootfs/LXC scanning, runc advisory, KEV/OSV enrichment, and new output formats (CSV, SBOM)
• .github/workflows/ci.yml: New CI pipeline with go vet, race-detector unit tests, and conditional integration tests

Source Code Modifications

• pkg/kev/kev.go: Changed cisaKEVURL from const to var to allow test mocking; added SetURLForTest() and ResetForTest() helper functions
• cmd/cli/main.go: Fixed runScan() to return exit code (was discarded); now properly propagates policy violations to os.Exit()
• go.mod: Updated Go version to 1.25.0 and MCP SDK to v1.6.1

Notable Implementation Details

• Tests use httptest.NewServer() to mock external APIs without network calls
• KEV and runc tests include concurrent safety verification with -race detector
• CLI tests use shell scripts as fake Trivy binaries to simulate real execution paths
• SARIF tests validate GitHub/Azure import requirements (schema, version, tool metadata)
• Enrichment integration tests verify the full pipeline: mock KEV → exploit flag → severity upgrade → policy gate
• All tests are self-contained; no external fixtures or live API dependencies

Testing Strategy

• Unit tests cover individual components (parsing, enrichment, policy, KEV client)
• Integration tests wire together real functions with mocked HTTP servers
• CLI tests verify exit codes and policy enforcement with fake Trivy
• Race detector enabled on all unit tests to catch concurrency issues

https://claude.ai/code/session_0189QVFiKNFT5MEsskeEi19t

Greptile Summary

This PR adds comprehensive test coverage across all pkg/* and cmd/* packages, refactors runScan() to return an exit code instead of calling os.Exit() directly, and introduces a new GitHub Actions CI pipeline along with CI templates for CircleCI, AWS CodeBuild, Google Cloud Build, Bitbucket Pipelines, and Tekton. The Go module is bumped to 1.25.0 to satisfy the MCP SDK dependency.

• Test coverage: Unit, integration, and CLI exit-code tests added for pkg/kev, pkg/runc, pkg/scanner, pkg/remediate, pkg/report, cmd/cli, cmd/server, cmd/mcp-server, and cmd/baseline; all use mock HTTP servers or fake binaries and avoid live network dependencies.
• CLI fix: runScan() now returns an int instead of calling os.Exit(), making exit-code behavior fully testable and correctly propagating policy violations.
• CI templates: Two of the five new platform templates have defects \u2014 the CircleCI exit-code capture pattern is broken under set -e, and the CodeBuild spec pins Go 1.21 while go.mod requires 1.25.0.

Confidence Score: 4/5

Safe to merge for the Go source and test changes; two new CI templates for CircleCI and AWS CodeBuild have defects that would affect any team adopting them.

The core Go changes (runScan refactor, kev test helpers, test files) are correct and well-tested. The CodeBuild template fails to compile on the first run due to a Go version mismatch, and the CircleCI template silently loses the security gate if a user works around the failing step.

ci/circleci/config.example.yml (broken exit-code capture) and ci/aws-codebuild/buildspec.yml (Go version mismatch with go.mod).

Important Files Changed

Filename	Overview
cmd/cli/main.go	Fixed runScan() to return an int exit code instead of calling os.Exit() directly, making the function testable and correctly propagating policy violations.
pkg/kev/kev.go	Changed cisaKEVURL from const to var and added exported SetURLForTest/ResetForTest helpers for cross-package mock injection; HTTP status check is still absent (previously flagged).
.github/workflows/ci.yml	New CI pipeline adding go vet, race-detector tests for ./pkg/... ./cmd/..., and a conditional integration job; previously flagged bugs (cmd/... missing, changed_files integer) are both fixed here.
ci/circleci/config.example.yml	New CircleCI template with a broken exit-code capture pattern: echo $? after docker run is dead code under CircleCI's default set -e, so the propagation step always exits 0 if users work around the failing step.
ci/aws-codebuild/buildspec.yml	New CodeBuild template hardcodes golang: 1.21 but go.mod now requires 1.25.0; the from-source build path fails immediately with a toolchain version error.
pkg/kev/kev_test.go	13 tests covering cache TTL, concurrent safety, case normalization, and HTTP error paths using mock HTTP servers; thorough and race-detector clean.
cmd/cli/main_test.go	7 exit-code tests using fake Trivy binaries in PATH; covers policy enforcement, Trivy errors, offline KEV skip, and SBOM/Dockerfile merge paths.
pkg/remediate/enrich_integration_test.go	9 integration tests wiring real Enrich() to a mock KEV server; verifies exploit flagging, severity upgrades, ransomware detection, and policy gating end-to-end.
cmd/server/main_test.go	8 HTTP handler tests covering health endpoint, SSE content-type, input validation, concurrency guard, and summary counting.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
  participant Test as Test
  participant CLI as runScan()
  participant Scanner as pkg/scanner
  participant KEV as pkg/kev
  participant Enricher as pkg/remediate
  participant Policy as pkg/policy
  participant Report as pkg/report

  Test->>CLI: runScan(ctx, opts)
  CLI->>Scanner: Scan(ctx, ScanOptions)
  Scanner-->>CLI: []Finding
  alt "offline=false"
    CLI->>Enricher: Enrich(findings, false)
    Enricher->>KEV: Load() mock HTTP
    KEV-->>Enricher: CVE map
    Enricher-->>CLI: enriched
  else "offline=true"
    CLI->>Enricher: Enrich(findings, true)
    Enricher-->>CLI: enriched
  end
  CLI->>Report: Generate(enriched)
  Report-->>CLI: files written
  CLI->>Policy: EvaluateFailPolicy
  alt violated
    Policy-->>CLI: "shouldFail=true"
    CLI-->>Test: return 1
  else ok
    Policy-->>CLI: "shouldFail=false"
    CLI-->>Test: return 0
  end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
  participant Test as Test
  participant CLI as runScan()
  participant Scanner as pkg/scanner
  participant KEV as pkg/kev
  participant Enricher as pkg/remediate
  participant Policy as pkg/policy
  participant Report as pkg/report

  Test->>CLI: runScan(ctx, opts)
  CLI->>Scanner: Scan(ctx, ScanOptions)
  Scanner-->>CLI: []Finding
  alt "offline=false"
    CLI->>Enricher: Enrich(findings, false)
    Enricher->>KEV: Load() mock HTTP
    KEV-->>Enricher: CVE map
    Enricher-->>CLI: enriched
  else "offline=true"
    CLI->>Enricher: Enrich(findings, true)
    Enricher-->>CLI: enriched
  end
  CLI->>Report: Generate(enriched)
  Report-->>CLI: files written
  CLI->>Policy: EvaluateFailPolicy
  alt violated
    Policy-->>CLI: "shouldFail=true"
    CLI-->>Test: return 1
  else ok
    Policy-->>CLI: "shouldFail=false"
    CLI-->>Test: return 0
  end

Loading

Comments Outside Diff (3)

1. .github/workflows/ci.yml, line 49-51 (link)

   Trivy install script fetched from floating main branch

   The install script URL uses main/contrib/install.sh, which is a moving target — any future commit to that branch can change the script's behavior. Only the Trivy binary version (v0.69.1) is pinned. Pinning the script to a specific release tag or commit hash (e.g. https://raw.githubusercontent.com/aquasecurity/trivy/v0.69.1/contrib/install.sh) is a simple safeguard against unexpected upstream changes.

   

2. pkg/kev/kev.go, line 72-79 (link)

   Missing HTTP status check — silent empty-catalog caching

   Load() never inspects resp.StatusCode. If the CISA endpoint returns a 4xx/5xx with a parseable JSON body — e.g. a CDN that returns {} or {"vulnerabilities":[]} on error — json.Decode succeeds, knownExploited is set to a non-nil empty map, and lastFetch is stamped. On every subsequent call for the next 24 hours the cache-hit guard (knownExploited != nil && …) returns immediately, so all CVEs are silently treated as "not exploited" with no error surfaced to the caller.

   Add a status check immediately after resp.Body.Close deferral:

   if resp.StatusCode != http.StatusOK {
       return fmt.Errorf("KEV endpoint returned HTTP %d", resp.StatusCode)
   }

   

3. ci/aws-codebuild/buildspec.yml, line 183-186 (link)

   Go runtime version 1.21 is incompatible with go.mod requiring 1.25.0

   go.mod was bumped to go 1.25.0 in this PR (required by the MCP SDK). When CodeBuild provisions golang: 1.21 and then runs go build ./cmd/scanner, the toolchain rejects the module with an error like note: module requires Go >= 1.25.0. The build fails immediately. This line should be updated to match the minimum version in go.mod.

   

_{Reviews (6): Last reviewed commit: "Update COMPARISON and README to reflect ..." | Re-trigger Greptile}

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.