fix(cursor-review): survive a rebranded model family in the drift check (BE-4852)

[mattmillerai]

STACKED — merging lands on matt/be-4819-cursor-review-catalog-drift (owned by @mattmillerai, PR #87), NOT main. The file this PR edits does not exist on main yet; #87 introduces it. Merge #87 first, then this.

ELI-5

The weekly checker looks at Cursor's model list and tells us if one of our four review models disappeared, or if a newer one showed up that we might want to switch to. It figures out "which lab is this?" by taking the first chunk of the model's name — gpt-5.6-sol-max → gpt. That works right up until a lab renames its family: when OpenAI ships o5-pro next to gpt-*, the checker reads the lab as o5, sees that nobody pins o5, and quietly drops it. So the one thing the check exists for — "a newer model shipped and nobody noticed" — is exactly the thing it could miss. This PR adds a second, quieter list for models from families we pin nothing from, so a renamed family shows up instead of vanishing. It also stops the parser treating a hyphenated English word at the start of a line (gpt-based models are …) as if it were a model name.

What changed

Both fixes were deferred out of #87 by the review panel (thread) — they change the design, not just a regex, so they were out of scope there.

1. Unpinned-families catch-all (analyze → report["unpinned_labs"]). Catalog ids whose family prefix matches no pin at all now get their own section instead of being dropped. This is the ticket's first proposed option, chosen over an explicit LAB_ALIASES = {"gpt": "openai", "o": "openai", …} table: the alias table is more precise but reintroduces exactly the hand-maintained list that deriving labs from the pins exists to avoid, and it needs a human edit before the rebrand it is supposed to catch. The catch-all costs one section and stays zero-maintenance.

Deliberate placement, per the ticket's "keep it out of urgent" note:

• Not urgent — it is a standing watchlist, not breakage, so it never reddens the weekly run.
• Counted in has_findings — it has to be, or a rebranded family renders into a body nobody ever sees (a clean has_findings closes the sticky issue). See the judgment call below on auto-close.
• Rendered collapsed, below the same-lab review-me list, so the labs Comfy will never pin cannot crowd out the list that actually needs a decision.

A lab whose only pin was just delisted still counts as pinned, so its successors stay in the same-lab list (with the delisted-pin context) rather than being demoted into the quiet fold — covered by a test.

2. Model ids must carry a digit (_is_model_id). gpt-based is lowercase, hyphenated and has letters, so it parsed as a catalog id and added noise to the review-me list — and, with change 1 in place, would have minted a phantom gpt-based family. Every id a lab has actually shipped carries a version number somewhere; a prose word does not. This is the ticket's first proposed option for (2); "require the token to appear in more than one catalog line" was rejected because a real catalog lists each id exactly once.

The rule lives in the single _is_model_id that the pin validator and the catalog parser both call, so the two cannot drift apart — relaxing it is one edit, not two. test_pin_and_catalog_id_shape_rules_agree (the tripwire the ticket names) is extended rather than weakened: it now also pins code-supernova/gpt-based as rejected and o5-pro as accepted, on both sides.

Verification

python3 -m unittest discover -s .github/cursor-review/tests -p 'test_*.py' — 103 passed (the exact command test-cursor-review-scripts.yml runs). check_agents_md.py --root . passes (one pre-existing CODEOWNERS warning). No shell changed, so shellcheck is unaffected.

New tests: the rebranded-family catch-all (o5-pro → family o5, invisible to the same-lab list), finding-but-never-urgent, a catalog of exactly the pins still having no unpinned families (the auto-close path is not unconditionally broken), a delisted pin's lab still counting as pinned, prose-word rejection, digit-less pin rejected loudly rather than read as delisted, the collapsed-section render, backtick-injection safety on the new section's notes, and a 4000-family catalog still fitting under GitHub's body cap.

Judgment calls

• The tightened id rule binds pins too, on purpose. _is_model_id gates extraction, so a digit-less pin (code-supernova is a real Cursor id shape) now fails extraction with an actionable message and a red run, rather than being silently reported as delisted every Monday. I checked this against the live pins rather than assuming: gpt-5.6-sol-max, claude-opus-4-8-thinking-max, gemini-3.1-pro, kimi-k2.7-code and the judge_model default all carry digits and extract cleanly from the real cursor-review.yml (asserted by test_extraction_works_against_the_real_cursor_review_workflow). The cost of the tightening is that a digit-less catalog id would no longer reach the review-me list — it is still reproduced verbatim in the raw-catalog fold, which is where a human reads it anyway.
• The sticky issue's auto-close, which the ticket asked me to consider. Folding unpinned_labs into has_findings does not newly pin the issue open: unpinned alone already makes has_findings true on any real catalog, since Cursor lists more tiers per pinned lab than the panel pins four of. The auto-close path was already reserved for a catalog trimmed to exactly the pins, and still is. I chose not to add a separate "worth an issue" gate — it would reintroduce the miss (a rebranded family with no other drift would file nothing), and splitting the two flags is a bigger design change than this deferred follow-up should carry. Flagging it in case you'd rather the catch-all be step-summary-only.
• Not done, deliberately: no LAB_ALIASES table (see above), and lab_of itself is unchanged — its docstring now says plainly that it returns a family prefix, not a vendor, and points at the catch-all.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 33657e78-e0ae-449a-999e-afce67748524

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch matt/be-4852-catalog-drift-lab-heuristics

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch matt/be-4852-catalog-drift-lab-heuristics

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 4 finding(s).

Severity	Count
🟡 Medium	3
⚪ Nit	1

Panel: 8/8 reviewers contributed findings.

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 3 finding(s).

Severity	Count
🟡 Medium	1
🟢 Low	2

Panel: 8/8 reviewers contributed findings.