Skip to content

Add per-ecosystem record counts to scan_summary #52

Description

@fuleinist

Problem

scan_summary.counts only contains two keys today — package and finding — even when a scan produced thousands of records across a dozen ecosystems:

"counts": { "package": 5421, "finding": 3 }

Operators want to know how many records came from each ecosystem (npm vs pypi vs go modules vs MCP servers vs ...) at a glance. Today the only way to get that is to stream the full NDJSON through a downstream pipeline and run jq -c 'select(.record_type=="package")' | jq -r .ecosystem | sort | uniq -c — awkward in CI, impossible in a single scan_summary consumer that just wants the headline number per ecosystem for capacity planning, drift detection, or fleet-wide dashboards.

The OSV subsystem already tracks per-ecosystem counts internally (internal/osv/osv.go EcosystemCounts), so the building block is there — it's just not exposed in the main scan_summary record.

Proposed change

Populate scan_summary.counts with one key per ecosystem that emitted at least one package record, in addition to the existing package and finding keys. Example end state for a heterogeneous developer machine:

"counts": {
  "package": 5421,
  "finding": 3,
  "npm": 4210,
  "pypi": 873,
  "go": 234,
  "rubygems": 87,
  "packagist": 12,
  "mcp": 5
}

Scope:

  • Add a map[string]int to scanner.Result (e.g. RecordsByEcosystem) populated from the emit closure after ObservePackage returns written=true (so suppressed-dedup and --findings-only records are not double-counted, mirroring the existing RecordsEmitted semantics).
  • In cmd/bumblebee/runScan, merge the per-ecosystem map into the existing counts map before EmitSummary is called. Use the same model.supportedEcosystemOrder ordering for determinism in any sorted serialization.
  • No schema change required — docs/schema/v0.1.0/scan-summary.schema.json already declares "counts": { "type": "object", "additionalProperties": { "type": "integer" } }, so adding new keys is schema-legal today.
  • Document the per-ecosystem keys in README.md next to the existing package/finding description (a one-line "each emitted ecosystem is also added to counts" note).
  • Test in internal/scanner/scanner_test.go: a multi-ecosystem fixture scan asserts that result.RecordsByEcosystem contains the expected per-ecosystem counts and that the merged scan_summary.counts map carries the same keys.

Use case

A nightly fleet scan posts scan_summary to a central store. A dashboard team wants a stacked bar chart of "package records by ecosystem" over the last 30 days without re-streaming the full record NDJSON. Today the only answer is package_records_emitted (one global number); after this change they can read the same dashboard from scan_summary.counts directly.

Acceptance criteria

  • scanner.Result.RecordsByEcosystem is populated for every successful scan.
  • scan_summary.counts contains one key per ecosystem that emitted at least one package record, in addition to the existing package and finding keys.
  • --findings-only and dedup-suppressed records do not inflate per-ecosystem counts (same semantics as package_records_emitted).
  • The order of keys in the emitted counts map is deterministic across runs (stable sort by model.supportedEcosystemOrder).
  • Default bumblebee scan output for an existing single-ecosystem scan is byte-identical except for the new keys in scan_summary.counts.
  • README.md documents the new keys.
  • New test in internal/scanner/scanner_test.go asserts per-ecosystem counts on a multi-ecosystem fixture scan.
  • go test ./... passes; gofmt -l . prints nothing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions