Skip to content

[Test] Add regression detection tests across existing archetypes after hasKeyword boundary change #98

Description

@SaharPak

Problem

PR #97 introduced a new hasKeyword helper in packages/core/src/archetypes/index.ts that uses regex word boundaries instead of text.includes. This is a correctness fix (e.g. swift no longer matches swiftly, go no longer matches go-to-market), but it silently changes detection behavior for every existing archetype, not just the new mobile one.

Today there are no regression tests pinning down what each existing archetype should detect, so:

  • If a borderline resume was previously being scored correctly by a partial-match side effect, that signal is now gone and detection may flip to a different archetype.
  • If a future change touches hasKeyword or the detector loop, there's no safety net to catch a regression in any archetype other than mobile-engineer.

The mobile-engineer detection tests added in PR #97 are a good template; this issue asks for the same coverage across the rest.

Proposed solution

Add a describe block per existing archetype in packages/core/src/__tests__/archetypes.test.ts (or split into per-archetype files if the file gets unwieldy), each with at minimum:

  1. One realistic positive test — a 2-3 sentence resume that should detect as that archetype. Use the real mix of keywords the role would actually use.
  2. One realistic negative test — a resume from an adjacent role that should NOT detect as this archetype (mirrors the swift/swiftly case the PR added for mobile).
  3. One structural validation test — archetype is registered, has the documented keyword/verb/anti-pattern counts, weights sum to 1.0 (mirroring the assertions in the new mobile-engineer block).

Archetypes to cover:

  • ai-product-manager
  • ai-engineer
  • backend-engineer
  • frontend-engineer
  • qa-test-engineer
  • devops-sre
  • data-engineer

Mirror the structural assertions in packages/intelligence/src/__tests__/intelligence.test.ts for the intelligence registry so both packages stay aligned — that drift is the explicit pain point tracked in docs/ARCHETYPE_GAP_AUDIT.md.

Alternatives considered

  • Snapshot a single end-to-end fixture per archetype: cheaper but less precise; a future keyword list change would update the snapshot instead of failing it.
  • Defer entirely: leaves the boundary change unverified; the next detector bug will be caught by users, not tests.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions