Skip to content

feat(migrate): recommend Claude Sonnet 5 / Opus 4.8 as Anthropic defaults - #167

Open
herosjourney wants to merge 3 commits into
awslabs:mainfrom
herosjourney:feat/recommend-sonnet-5-opus-4-8
Open

feat(migrate): recommend Claude Sonnet 5 / Opus 4.8 as Anthropic defaults#167
herosjourney wants to merge 3 commits into
awslabs:mainfrom
herosjourney:feat/recommend-sonnet-5-opus-4-8

Conversation

@herosjourney

@herosjourney herosjourney commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Bumps new-migration recommend defaults from Claude Sonnet 4.6 / Opus 4.7 → Claude Sonnet 5 / Claude Opus 4.8, aligning clarify/design/scoring with models already marked Active in the pricing cache.
  • Keeps Haiku 4.5 for cost/speed; Fable 5 stays opt-in only (frontier / Mythos pricing).
  • Sonnet 4.6 remains Active in the pricing cache and static fallback table for existing workloads.
  • Comparison tables keep steady-state $3/$15 for Sonnet 5 (post-intro), with callouts that intro pricing is $2/$10 through Aug 31, 2026.

Why

The catalog already knew about Sonnet 5 / Opus 4.8 / Fable 5, but Q16/Q17, mapping guides, agent-advisor scoring, and report fixtures still called Sonnet 4.6 “latest” and recommended it by default. That is inconsistent for customers starting migrations now.

Test plan

  • Confirm Bedrock inference-profile IDs (anthropic.claude-sonnet-5, anthropic.claude-opus-4-8, prefer us. prefix) and regional availability
  • Confirm agentic/tool-use claims still hold for Sonnet 5 (guides previously asserted Sonnet 4.6 GDPval leadership)
  • python3 -m pytest skills/agent-advisor/scripts/test_scoring.py (57 passed locally)
  • Spot-check a gcp-to-aws AI-only clarify → design path defaults to Sonnet 5
  • After Aug 31, 2026: refresh Sonnet 5 static fallback from intro $2/$10 → $3/$15

Out of scope

@herosjourney

Copy link
Copy Markdown
Contributor Author

Addressed both merge-blocking review items:

  1. Dropped anthropic.claude-opus-4-8-20250610-v1:0 / us.… from STATIC_FALLBACK — kept suffix-less Opus 4.8 IDs only.
  2. Fixed gemini mapping guide Sonnet 5 context 200K → 1M.

Ready for another look.

herosjourney pushed a commit to herosjourney/startups that referenced this pull request Jul 22, 2026
Address Kiro review on awslabs#167: remove non-existent
claude-opus-4-8-20250610-v1:0 STATIC_FALLBACK keys, and correct the
gemini mapping guide context window from 200K (copied from Sonnet 4.6)
to 1M.

Co-authored-by: Cursor <cursoragent@cursor.com>
@herosjourney

Copy link
Copy Markdown
Contributor Author

Resolved the merge conflict with main (f064392). One conflicted hunk, in clarify-ai.md's Q17 feature table, and it was a semantic cross rather than a textual one: this PR bumps the agentic-workflows model (Sonnet 4.6 → Sonnet 5) while main independently moved the agentic runtime (Bedrock Agents → AgentCore Harness, from the Bedrock Agents EOL work this PR explicitly scoped out to #161). Resolution combines both intents: Claude Sonnet 5 with AgentCore (Harness). Post-merge verification on the combined tree: lint:md 0 errors (855 files), shared:check OK, agent-advisor scoring 230 passed, llm-to-bedrock scripts 160 passed, plugin tests 81 passed. Also confirmed the merge kept main's clarify changes intact around this PR's edits (Q15 provider-neutral wording, Q14/Q19 detect-confirm cards) with zero stray Sonnet 4.6 references left in clarify-ai.md.

@herosjourney
herosjourney marked this pull request as ready for review July 23, 2026 00:30
@herosjourney
herosjourney requested a review from a team as a code owner July 23, 2026 00:30

@leon1418 leon1418 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Summary

This PR does a mechanical text-substitution of Sonnet 4.6 → Sonnet 5 and Opus 4.7 → Opus 4.8 across 32 files (documentation, fixtures, scoring code, and tests). The change is well-scoped and internally consistent in the places it touches. However:

  1. Partial supersedence by PR #195: PR #195 (Model Recommend phase) removes the model_recommendation field and _MODEL_PRIORITY / _FEATURE_OVERRIDE / _MIGRATE_FAMILY dictionaries from scoring.py entirely (−93 lines), and removes the corresponding tests. 7 files overlap. If #195 merges first, all scoring.py and related test changes in this PR become a merge conflict on dead code. Conversely if #167 merges first, #195 gets a trivial conflict (just different string values in the block it deletes).

  2. Independent value: 25 of 32 files are NOT touched by #195 — the pricing guides, mapping tables, reference docs, lifecycle table, and llm-to-bedrock scripts are independent contributions that have value regardless of #195's fate.

  3. PR #169 (discover-preview fix + breadcrumbs) is OPEN but touches discover-preview.md — this PR also touches that file. Textual conflict is likely but trivial (different hunks).

Findings

  • 2 Nits (stale context-window claims after bumping to Sonnet 5 1M)
  • 1 FYI (Opus 4.6 vs 4.8 inconsistency in clarify tables)
  • 0 blocking issues

Merge Recommendation

MERGEABLE with the nits addressed, but coordinate with #195 author — whoever merges second gets conflicts in scoring.py. The cleanest path: merge #167 first (since it's simpler), then #195 rebases and just deletes the updated strings.

87-94% savings. Strong cost case at any volume. Nova Lite (300K context) covers most GPT-5.4 Mini use cases.

### GPT-4/4 Turbo → Claude Sonnet 4.6
### GPT-4/4 Turbo → Claude Sonnet 5

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Nit: This sentence now says "GPT-5.4 → Claude Sonnet 5" but keeps "Both have ~200K+ context" — the Gemini guide table (this PR) updates Sonnet 5's context column to 1M. Should be "Both have 200K+ context" → "Sonnet 5 has 1M context" or "~1M+ context" to stay consistent with the Bedrock capabilities table change at ai-gemini-to-bedrock.md line 44.

### GPT-4/4 Turbo → Claude Sonnet 4.6
### GPT-4/4 Turbo → Claude Sonnet 5

70-90% savings, similar or better quality, longer context (200K vs 128K). Low risk.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

Nit: Similarly, "longer context (200K vs 128K)" under the GPT-4/4 Turbo → Claude Sonnet 5 header should now reflect the 1M context window stated elsewhere in this PR for Sonnet 5. The 200K figure is Sonnet 4.6's context — Sonnet 5 is 1M per the capabilities table this PR introduces.

| Gemini 2.5 Pro | Nova 2 Pro ($1.38/$11) — 9% cheaper; or Nova Pro ($0.80/$3.20) — 62% cheaper | Gemini 2.5 Pro is $1.25/$10; migration case is cost + AWS consolidation |
| Gemini 3 Pro / 3.1 Pro | Claude Sonnet 4.6 ($3/$15) — agentic reliability; or Nova 2 Pro ($1.38/$11) — cost | Gemini 3.1 Pro is $2/$12 — cheaper than Sonnet 4.6; migration case is agentic reliability and AWS ecosystem, NOT cost. Be honest: Gemini 3.1 Pro leads on general benchmarks. |
| Gemini 3 Pro / 3.1 Pro | Claude Sonnet 5 ($3/$15) — agentic reliability; or Nova 2 Pro ($1.38/$11) — cost | Gemini 3.1 Pro is $2/$12 — cheaper than Sonnet 5; migration case is agentic reliability and AWS ecosystem, NOT cost. Be honest: Gemini 3.1 Pro leads on general benchmarks. |
| Gemini Flash 1.5 / 2.0 (older) | Nova Lite ($0.06/$0.24) or Nova Micro ($0.035/$0.14) — **flag Gemini 1.5 Flash as EOL (Sep 2025); recommend upgrading source model to 3.5 Flash before or alongside migration** | Strong Bedrock cost savings; 1.5 Flash is past EOL so migration is doubly urgent |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🤖 AI review 🤖]

FYI: Several rows in Q17/Q22 tables (clarify-ai.md, clarify.md) still say "Opus 4.6 for hardest" or "Opus 4.6 for most demanding" in lines this PR updates. The PR description explicitly bumps Opus 4.7 → 4.8 and the scoring code uses claude_opus_4_8, but the extended-thinking and complex-reasoning rows in the clarify tables reference Opus 4.6. This appears intentional per the anthropic-to-bedrock note ("Opus 4.6 remains a valid alternative where batch or regional availability matters"), but if the intent is to recommend 4.8 as the new default for hardest-reasoning, these rows should also say 4.8. No action required if the mixed references are deliberate.

Logan Kleier and others added 3 commits August 13, 2026 07:09
…ults

Catalog already priced Sonnet 5, Opus 4.8, and Fable 5 as Active, but
clarify/design/scoring still defaulted to Sonnet 4.6 and Opus 4.7. Point
new migrations at Sonnet 5 (flagship) and Opus 4.8 (hardest reasoning),
keep Haiku 4.5 for cost/speed, and leave Fable 5 opt-in only.

Co-authored-by: Cursor <cursoragent@cursor.com>
Address Kiro review on awslabs#167: remove non-existent
claude-opus-4-8-20250610-v1:0 STATIC_FALLBACK keys, and correct the
gemini mapping guide context window from 200K (copied from Sonnet 4.6)
to 1M.

Co-authored-by: Cursor <cursoragent@cursor.com>
Address review nits on ai-openai-to-bedrock.md: GPT-5.4 and GPT-4/4 Turbo
mapping sections still said 200K after the Sonnet 5 bump.

Co-authored-by: Cursor <cursoragent@cursor.com>
@herosjourney

Copy link
Copy Markdown
Contributor Author

Rebased onto current main (post-#195) and addressed the review nits.

Merge conflicts with #195 (Model Recommend):

Review nits: ai-openai-to-bedrock.md GPT-5.4 and GPT-4/4 Turbo sections now say Sonnet 5 has 1M context (was stale 200K).

FYI on Opus 4.6 in clarify tables: left as-is — intentional alternate where batch/regional availability matters; hardest-reasoning rows already say Opus 4.8.

@herosjourney
herosjourney force-pushed the feat/recommend-sonnet-5-opus-4-8 branch from 8fb8bc3 to 1b1af62 Compare August 13, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants