Skip to content

fix: benchmark per-file metrics use wrong file count for native engine #925

@carlos-alm

Description

@carlos-alm

Problem

Reported by Greptile in PR #924: the native engine's perFile.* values in the benchmark data don't reconcile with the raw totals when divided by the reported file count (727).

Root cause: In scripts/benchmark.ts, each engine worker computes its own totalFiles from the DB after building. However, formatEngineResult() (lines 40-55) strips the per-engine files field, and the parent process uses primary.files (WASM's count) as the top-level files field for both engines.

The report script (scripts/update-benchmark-report.ts line 104) then uses h.files (the top-level count) in the table for both native and WASM rows, even though native may have processed fewer files (~663 vs 727).

Fix

  1. Include files in formatEngineResult() in scripts/benchmark.ts
  2. Update update-benchmark-report.ts to use each engine's own file count in the table row (falling back to top-level if not present for backward compat)

Impact

Cosmetic — the per-file values are internally consistent with the native engine's actual file count, but the displayed Files column is misleading. Extrapolated estimates (50K files) use the potentially-incorrect per-file values.

Ref: PR #924

Metadata

Metadata

Assignees

No one assigned

    Labels

    follow-upDeferred work from PR reviews that needs tracking

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions