feat(migrate): recommend Claude Sonnet 5 / Opus 4.8 as Anthropic defaults - #167
feat(migrate): recommend Claude Sonnet 5 / Opus 4.8 as Anthropic defaults#167herosjourney wants to merge 3 commits into
Conversation
|
Addressed both merge-blocking review items:
Ready for another look. |
Address Kiro review on awslabs#167: remove non-existent claude-opus-4-8-20250610-v1:0 STATIC_FALLBACK keys, and correct the gemini mapping guide context window from 200K (copied from Sonnet 4.6) to 1M. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Resolved the merge conflict with main (f064392). One conflicted hunk, in clarify-ai.md's Q17 feature table, and it was a semantic cross rather than a textual one: this PR bumps the agentic-workflows model (Sonnet 4.6 → Sonnet 5) while main independently moved the agentic runtime (Bedrock Agents → AgentCore Harness, from the Bedrock Agents EOL work this PR explicitly scoped out to #161). Resolution combines both intents: Claude Sonnet 5 with AgentCore (Harness). Post-merge verification on the combined tree: lint:md 0 errors (855 files), shared:check OK, agent-advisor scoring 230 passed, llm-to-bedrock scripts 160 passed, plugin tests 81 passed. Also confirmed the merge kept main's clarify changes intact around this PR's edits (Q15 provider-neutral wording, Q14/Q19 detect-confirm cards) with zero stray Sonnet 4.6 references left in clarify-ai.md. |
leon1418
left a comment
There was a problem hiding this comment.
[🤖 AI review 🤖]
Summary
This PR does a mechanical text-substitution of Sonnet 4.6 → Sonnet 5 and Opus 4.7 → Opus 4.8 across 32 files (documentation, fixtures, scoring code, and tests). The change is well-scoped and internally consistent in the places it touches. However:
-
Partial supersedence by PR #195: PR #195 (Model Recommend phase) removes the
model_recommendationfield and_MODEL_PRIORITY/_FEATURE_OVERRIDE/_MIGRATE_FAMILYdictionaries fromscoring.pyentirely (−93 lines), and removes the corresponding tests. 7 files overlap. If #195 merges first, allscoring.pyand related test changes in this PR become a merge conflict on dead code. Conversely if #167 merges first, #195 gets a trivial conflict (just different string values in the block it deletes). -
Independent value: 25 of 32 files are NOT touched by #195 — the pricing guides, mapping tables, reference docs, lifecycle table, and llm-to-bedrock scripts are independent contributions that have value regardless of #195's fate.
-
PR #169 (discover-preview fix + breadcrumbs) is OPEN but touches
discover-preview.md— this PR also touches that file. Textual conflict is likely but trivial (different hunks).
Findings
- 2 Nits (stale context-window claims after bumping to Sonnet 5 1M)
- 1 FYI (Opus 4.6 vs 4.8 inconsistency in clarify tables)
- 0 blocking issues
Merge Recommendation
MERGEABLE with the nits addressed, but coordinate with #195 author — whoever merges second gets conflicts in scoring.py. The cleanest path: merge #167 first (since it's simpler), then #195 rebases and just deletes the updated strings.
| 87-94% savings. Strong cost case at any volume. Nova Lite (300K context) covers most GPT-5.4 Mini use cases. | ||
|
|
||
| ### GPT-4/4 Turbo → Claude Sonnet 4.6 | ||
| ### GPT-4/4 Turbo → Claude Sonnet 5 |
There was a problem hiding this comment.
[🤖 AI review 🤖]
Nit: This sentence now says "GPT-5.4 → Claude Sonnet 5" but keeps "Both have ~200K+ context" — the Gemini guide table (this PR) updates Sonnet 5's context column to 1M. Should be "Both have 200K+ context" → "Sonnet 5 has 1M context" or "~1M+ context" to stay consistent with the Bedrock capabilities table change at ai-gemini-to-bedrock.md line 44.
| ### GPT-4/4 Turbo → Claude Sonnet 4.6 | ||
| ### GPT-4/4 Turbo → Claude Sonnet 5 | ||
|
|
||
| 70-90% savings, similar or better quality, longer context (200K vs 128K). Low risk. |
There was a problem hiding this comment.
[🤖 AI review 🤖]
Nit: Similarly, "longer context (200K vs 128K)" under the GPT-4/4 Turbo → Claude Sonnet 5 header should now reflect the 1M context window stated elsewhere in this PR for Sonnet 5. The 200K figure is Sonnet 4.6's context — Sonnet 5 is 1M per the capabilities table this PR introduces.
| | Gemini 2.5 Pro | Nova 2 Pro ($1.38/$11) — 9% cheaper; or Nova Pro ($0.80/$3.20) — 62% cheaper | Gemini 2.5 Pro is $1.25/$10; migration case is cost + AWS consolidation | | ||
| | Gemini 3 Pro / 3.1 Pro | Claude Sonnet 4.6 ($3/$15) — agentic reliability; or Nova 2 Pro ($1.38/$11) — cost | Gemini 3.1 Pro is $2/$12 — cheaper than Sonnet 4.6; migration case is agentic reliability and AWS ecosystem, NOT cost. Be honest: Gemini 3.1 Pro leads on general benchmarks. | | ||
| | Gemini 3 Pro / 3.1 Pro | Claude Sonnet 5 ($3/$15) — agentic reliability; or Nova 2 Pro ($1.38/$11) — cost | Gemini 3.1 Pro is $2/$12 — cheaper than Sonnet 5; migration case is agentic reliability and AWS ecosystem, NOT cost. Be honest: Gemini 3.1 Pro leads on general benchmarks. | | ||
| | Gemini Flash 1.5 / 2.0 (older) | Nova Lite ($0.06/$0.24) or Nova Micro ($0.035/$0.14) — **flag Gemini 1.5 Flash as EOL (Sep 2025); recommend upgrading source model to 3.5 Flash before or alongside migration** | Strong Bedrock cost savings; 1.5 Flash is past EOL so migration is doubly urgent | |
There was a problem hiding this comment.
[🤖 AI review 🤖]
FYI: Several rows in Q17/Q22 tables (clarify-ai.md, clarify.md) still say "Opus 4.6 for hardest" or "Opus 4.6 for most demanding" in lines this PR updates. The PR description explicitly bumps Opus 4.7 → 4.8 and the scoring code uses claude_opus_4_8, but the extended-thinking and complex-reasoning rows in the clarify tables reference Opus 4.6. This appears intentional per the anthropic-to-bedrock note ("Opus 4.6 remains a valid alternative where batch or regional availability matters"), but if the intent is to recommend 4.8 as the new default for hardest-reasoning, these rows should also say 4.8. No action required if the mixed references are deliberate.
…ults Catalog already priced Sonnet 5, Opus 4.8, and Fable 5 as Active, but clarify/design/scoring still defaulted to Sonnet 4.6 and Opus 4.7. Point new migrations at Sonnet 5 (flagship) and Opus 4.8 (hardest reasoning), keep Haiku 4.5 for cost/speed, and leave Fable 5 opt-in only. Co-authored-by: Cursor <cursoragent@cursor.com>
Address Kiro review on awslabs#167: remove non-existent claude-opus-4-8-20250610-v1:0 STATIC_FALLBACK keys, and correct the gemini mapping guide context window from 200K (copied from Sonnet 4.6) to 1M. Co-authored-by: Cursor <cursoragent@cursor.com>
Address review nits on ai-openai-to-bedrock.md: GPT-5.4 and GPT-4/4 Turbo mapping sections still said 200K after the Sonnet 5 bump. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Rebased onto current Merge conflicts with #195 (Model Recommend):
Review nits: FYI on Opus 4.6 in clarify tables: left as-is — intentional alternate where batch/regional availability matters; hardest-reasoning rows already say Opus 4.8. |
8fb8bc3 to
1b1af62
Compare
Summary
Why
The catalog already knew about Sonnet 5 / Opus 4.8 / Fable 5, but Q16/Q17, mapping guides, agent-advisor scoring, and report fixtures still called Sonnet 4.6 “latest” and recommended it by default. That is inconsistent for customers starting migrations now.
Test plan
anthropic.claude-sonnet-5,anthropic.claude-opus-4-8, preferus.prefix) and regional availabilitypython3 -m pytest skills/agent-advisor/scripts/test_scoring.py(57 passed locally)Out of scope