Skip to content

fix(bench): dep-bump fixture should reflect post-ignore-filter content (#845)#850

Merged
gfargo merged 1 commit intomainfrom
fix/bench-dep-bump-respects-ignore-list
May 6, 2026
Merged

fix(bench): dep-bump fixture should reflect post-ignore-filter content (#845)#850
gfargo merged 1 commit intomainfrom
fix/bench-dep-bump-respects-ignore-list

Conversation

@gfargo
Copy link
Copy Markdown
Owner

@gfargo gfargo commented May 6, 2026

Context

The user pointed out that lockfiles should already be auto-skipped — and they're right. DEFAULT_IGNORED_FILES (src/lib/config/constants.ts:5-12) covers package-lock.json, yarn.lock, pnpm-lock.yaml, bun.lockb, bun.lock, plus node_modules; DEFAULT_IGNORED_EXTENSIONS covers .lock and .map. getChanges strips matching files before the diff-condensing pipeline ever sees them.

The dep-bump fixture from #849 included yarn.lock and reported 27 s / 1 LLM call / 20k prompt tokens for it. That was a bench-fixture artifact, not a real-world cost — and the original "lockfile fast-path" angle from the plan is moot because the existing filter already handles it.

Fix

  • Drop yarn.lock from DEP_BUMP_FILES; the realistic post-filter shape is just package.json + CHANGELOG.md.
  • Update the dep-bump-shape test to assert the post-filter invariant (no lockfile in the fixture) instead of asserting one is present.
  • Add a guard test that fails loudly if any future fixture accidentally drifts back into including a default-ignored file (file basename in DEFAULT_IGNORED_FILES or extension in DEFAULT_IGNORED_EXTENSIONS). Catches this class of mistake at CI time.

Re-captured baseline

fixture before after Δ
dep-bump 27,158 ms 0 ms -27,158 ms
dep-bump LLM calls 1 0 -1
dep-bump tokens 19,597 0 -19,597

Other fixtures unchanged (re-ran for completeness; numbers within ms of prior baseline). The new full baseline is committed at .bench/baseline.json.

Plan implication

The original #845 plan listed a "lockfile fast-path" optimization angle. Dropped — the existing ignore filter already handles it; any pipeline-level skip would be redundant. The remaining optimization plan stands as-is.

Test plan

  • npm run lint
  • npm run test:jest (1250 tests pass — added guard test + updated dep-bump shape assertion)
  • npm run build
  • npm run test:cli
  • npm run bench --update → dep-bump now reports 0ms / 0 calls

#845)

The dep-bump fixture from #849 included `yarn.lock` and reported
27 seconds of LLM work for it. That's a bench-fixture artifact,
not a real-world cost. Lockfiles live in `DEFAULT_IGNORED_FILES`
and the `.lock` extension lives in `DEFAULT_IGNORED_EXTENSIONS`
(see `src/lib/config/constants.ts`), so `getChanges` strips them
before the diff-condensing pipeline ever sees them on a real
`coco commit`.

- Drop `yarn.lock` from DEP_BUMP_FILES; the realistic shape is
  just `package.json` + `CHANGELOG.md`.
- Update the dep-bump-shape test to assert the post-filter
  invariant (no lockfiles in the fixture) instead of asserting a
  lockfile is present.
- Add a guard test that fails loudly if any future fixture
  accidentally drifts back into including a default-ignored file.
- Re-baseline. dep-bump now reports 0 ms / 0 LLM calls (early
  exit, total budget already under threshold), reflecting what a
  real dep bump costs the pipeline.

The "lockfile fast-path" optimization angle from the original
plan is dropped — the existing ignore filter already handles
that, and any pipeline-level skip would be redundant.
@gfargo gfargo merged commit 57620dd into main May 6, 2026
9 checks passed
@gfargo gfargo deleted the fix/bench-dep-bump-respects-ignore-list branch May 6, 2026 00:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant