feat(heroku-to-aws): live-captured prices for the Estimate baseline (stacked on #148) - #150
Conversation
…Discovery previously required Terraform with heroku_* resources, which most startups on Heroku don't have. Adds a consent-gated, read-only live discovery path: an interactive capture step (exact-command whitelist, config-var key names only, auth:token banned) writes raw CLI output to live-capture/, and a new 'live' fragment parses it into the inventory. When Terraform is also present, live wins for current state and disagreements surface as drift (unmanaged resources, plan changes, config conflicts) — never silently resolved. Includes replay fixtures with a drift-assertion set, validated by end-to-end fresh-agent runs of the Discover phase.
…heck_expected_drift.py (stdlib-only) machine-checks a scenario-B replay's heroku-resource-inventory.json against expected-drift.json: metadata, apps, merged resources, conflict fields, drift counts, forbidden clustering fields, and secret hygiene (no config-var value payloads, no raw config_vars, no secret-shaped strings). Validated PASS against a fresh fresh-agent replay of scenario B on this branch. Same pattern as the gcp-live-capture asserter (awslabs#149) so both live-discovery skills regress identically.
…ne Live discovery (awslabs#148) captures each add-on's actual billed plan price (monthly_price_usd), but Estimate could only build a current-spend baseline from a billing export -- and the migrate-or-stay comparison was gated on billing data, so live-discovered runs got projected AWS costs with no side-by-side at all. - New baseline rung between billing data and the rate-card cache: add-ons priced exactly from live-captured plan prices, dynos from the published-rate cache (the Platform API does not price formations). Distinct provenance (current_costs.source: live_prices_plus_cache) and a mandatory caveat: derived from plan prices, not an invoice; excludes usage-based charges. - The comparison (Part 3) and migration cost considerations (Part 4) now run for ANY determined baseline, labeled with baseline_source -- previously both were billing-only, which orphaned cache-derived baselines too. - Schema: add cached_stale to pricing_source.status (already mandated by both skills' estimate specs; the enum rejected it). - Fixtures: seed-estimate/ (the scenario-B inventory post-merge, plus clarify/design artifacts) + expected-estimate.json + check_expected_estimate.py asserting the exact $352 baseline math, provenance label, caveat, and unlocked comparison. Validated by a fresh-agent replay of the Estimate phase (asserter PASS).
… prices Review follow-up: 'live discovery ran' was ambiguous when the run captured zero priced add-ons (e.g. a dyno-only app) -- rung 2 would fire, price everything from the cache via fallthrough, and still label the baseline live_prices_plus_cache with 'exact for add-ons' accuracy. Right number, wrong provenance. The rung now requires at least one add-on resource carrying config.monthly_price_usd; otherwise fall to rung 3, whose pricing_cache label describes that baseline honestly.
|
Review follow-up pushed (17cf2f1): tightened the rung-2 gate. "Live discovery ran" was ambiguous when the capture contained zero priced add-ons (e.g. a dyno-only app) — the rung would fire, price everything from the cache via per-resource fallthrough, and still label the baseline |
…and the estimated rung Three review-observation cleanups from the estimate-phase replay: - EBS root volumes: new ebs block in the shared pricing file (gp3 $0.08/GB-mo, 8 GB AL2023 EB root default) + vendored sync. EB estimates now price root volumes instead of shipping a permanent 'known minor omission' warning. - Sign conventions: roi_analysis.monthly_difference_* (AWS minus Heroku, negative = AWS cheaper) and financial_summary.monthly_savings_* (Heroku minus AWS) are the same fact with opposite signs. Both specs now cross-reference the other, name their own convention, and require any presentation to label direction in words rather than print a bare signed value. - Pricing hierarchy: new row 4 'estimated' — the documented home of the services_by_source.estimated bucket the schema and assembler already carry. A rate stated verbatim by this file's own formulas may price a service (always with a warning naming the rate); only no-cache no-MCP no-formula is 'unavailable'.
|
Pushed 24092ac — three review-observation cleanups from the estimate replay, all in this PR's blast radius: 1. EB root volumes are now priced. New |
…g; docs/spec cleanups Review-observation batch, replay-validated: - Q7 confirm-first: when discovery captured the postgres store's sizeBytes, Clarify derives the band (decimal GB) and presents it for founder confirmation instead of asking cold -- one less question per run, and the founder's correction wins when given. The phase postcondition now states the rule precisely: no COLD question whose answer discovery determined; confirmations are the sanctioned form. Validated by a fresh-agent discover->clarify replay (Q7 presented as a confirmation of band B from the ~3 GB store; HANDOFF_OK both phases). - Case fix folded back from that replay: five files matched peripherals[] against 'Postgres' while the discover schema emits lowercase 'postgres' -- a literal match would wrongly skip Q7, the DB migration script, and the has_databases complexity input. All now lowercase + case-insensitive. - Pricing hierarchy row 4 'estimated' (mirrors the heroku change on awslabs#150): the documented home of the services_by_source.estimated bucket; Lambda's formula constants are the canonical example. - generate-docs: guide step numbers count existing scripts (the file set is conditional, so numbering gaps are normal and must be called out), and a Provision Infrastructure slot between scripts 01 and 02 (terraform apply + sst deploy) -- both folded back from the generate replay. - SKILL.md: estimate/generate are read-only consumers of assessment-state.json (no phase file specifies a write; none should be improvised). - clarify.md: repaired the malformed declined-answer error-table row.
Resolves add/add conflict in fixtures/heroku-live-capture/README.md after awslabs#148 merged to main: kept awslabs#150's version (adds Scenario C estimate-baseline replay); all other content byte-identical between sides.
Resolves conflicts after awslabs#148 + awslabs#150 merged to main: - SKILL.md description: merged both sides — kept main's live-CLI discovery phrasing (awslabs#148) AND awslabs#152's what-if-workshop trigger words + workshop sentence. Auto-merge already restored the live-first Philosophy bullet, file-tree discover-live entries, and Migration-mode line from main. - seed-estimate/.phase-status.json + estimate-assemble.md: kept awslabs#152's additive workshop content (workshop phase seed, optional workshop metadata block).
Resolves README conflicts by UNIONING both sides: kept main's newer content (agent-advisor sections from awslabs#147, the awslabs#148 live-Heroku-discovery corrections, tf-best-practices) AND re-applied vercel-to-aws's unique additions (intro block, assessment triggers, Vercel to AWS mapping tables, trigger-table row, requirement, and the DSL-architecture resumability-ledger note). dprint-formatted. Also fixes two pre-existing CI failures on the branch: - workshop fragment _ids (workshop-sheet/-refresh/-compare) now match workshop.md's declared _ids (sheet/refresh/compare) per the house convention (frontmatter validator). - shared:sync'd vercel's vendored estimate schema + pricing to canonical (picks up awslabs#150's cached_stale enum + arm64 rates that landed on main).
* feat(migrate): add vercel-to-aws migration assessment skill Adds a new migration-to-aws skill: an honest Vercel-to-AWS assessment for Next.js apps (not a full migration plan). Built on the phase DSL (same architecture as heroku-to-aws), 5 backbone phases + 1 optional scaffold checkpoint: - PreScan -> Discover -> Clarify -> Recommend -> Report, with an optional Scaffold checkpoint - Discover computes a Coupling Score and 10 named Pre-Flight Checks unconditionally, before Recommend has run - Recommend is a fixed-precedence, three-outcome engine (OpenNext/SST, ECS Fargate, or a Vercel+AWS Hybrid) - every recommendation traces to exactly one rule, never a judgment call - Report renders a validated HTML assessment (new scripts/validate-assessment-report.py + pytest suite + reference/stub fixtures), with outcome-filtered findings, a reader-vocabulary rule, and a cost-labeling rule enforced by the validator - Scaffold (optional) emits a thin IaC skeleton matching the recommended outcome - SST/Terraform for OpenNext, Terraform-only for Fargate or the Hybrid path - and defaults compute to Graviton (ARM64) where SST/Terraform expose the option - assessment-state.json: a skill-owned resumability ledger (independent of the vendored .phase-status.json) supporting incremental, effort-for-confidence input collection across multiple sessions Registers the skill in the plugin's README, the top-level migrate README, and the Claude/Codex/Cursor plugin manifests. Verified: frontmatter validator (0 problems), pytest (39/39), dprint check and markdownlint-cli2 clean, reference fixture passes the validator (REPORT_OK), stub fixture fails with actionable errors (REPORT_FAIL). * chore: remove .kiro/specs/vercel-to-aws (local working scratch, not shipped) Per the precedent set in 033669c (removed heroku-eks-support, heroku-to-aws-migration, and migration-telemetry-sharing spec dirs from a prior PR), .kiro/specs/ is local Kiro working material, not project documentation this repo ships - no .kiro/specs/ directory exists anywhere on main today. The requirements/design/tasks docs used to drive this skill's build don't belong in the PR. * feat(vercel-to-aws): add estimate phase + promote scaffold to full generate phase Add a mandatory estimate phase (three-tier AWS cost projection vs. Vercel spend) and promote the optional scaffold checkpoint into a mandatory generate phase producing production-ready Terraform, including: - baseline.tf (GuardDuty, CloudTrail, IMDSv2, EBS encryption, budget alerts, Access Analyzer, remote-state backend, compliance-conditional Config/SecurityHub) - Full VPC with public/private subnets, NAT gateway, route tables - Outcome-specific compute: SST/OpenNext (A), Fargate+ALB+CloudFront (B), API Gateway+Lambda (C-A), Fargate backend (C-B) - Peripheral Terraform: RDS (Postgres), ElastiCache (KV), S3 (Blob), EventBridge (Cron) - Numbered migration scripts (01-06) with dry-run defaults - MIGRATION_GUIDE.md with phased timeline, verification, rollback, Go/No-Go gates - generation-warnings.json for unpriced/unmapped services Backbone changes: prescan -> discover -> clarify -> recommend -> estimate -> generate -> report -> complete (was: prescan -> discover -> clarify -> recommend -> report -> complete + scaffold checkpoint) Clarify extended with Q6 (Vercel spend), Q7 (database size), Q8 (compliance) to feed the estimate and generate phases. Vendored shared files: aws-infra-pricing.json, estimation-infra.schema.json, complexity-tiers.json — registered in sync-vendored-shared.ts auto-discovery. Breaking change: existing .phase-status.json files from the old backbone are not compatible; re-run from prescan. * fix(vercel-to-aws): complete generate-phase integration (SKILL.md, report, validator, READMEs, scaffold deletion) * fix(vercel-to-aws): capture/parse split — align prescan/discover with the worker contract Three architecture contradictions made the skill unexecutable as written: prescan declared _interactive:false + _exec:rw while its collect fragment converses with the founder (token request), runs next build, and calls the Vercel API; four of discover's dispatched fragments needed shell or network the rw worker (Read/Grep/Glob/Write/Edit) does not have; and the token, collected interactively and never persisted, could not reach a dispatched worker at all. Retrofit (the pattern proven on heroku/gcp live discovery, #148/#149): - prescan runs INLINE (_interactive: true, _exec removed) — its work is conversational + shell + network by nature; scope boundary corrected (the Tier 1 build-health attempt belongs to prescan-collect per Requirements 1.1-1.2) - NEW discover-capture.md: main-window pre-work owning ALL shell/network — adapter/manifest build capture, a GET-only 8-endpoint API whitelist (Vercel tokens cannot be scoped read-only, so the whitelist IS the read-only guarantee), env keys projected to names before touching disk, headers-only probe capture, manifest.json index; VERCEL_TOKEN lives in an env var in the main window only - discover-adapter/manifests/api/probe become parse-only fragments reading capture/; triggers keyed to the capture manifest; new postconditions (no env values/token material anywhere; coupling items carry confidence) - prescan-collect token request rewritten per Vercel's token docs: read-only tokens do not exist — ask for project-scoped + shortest expiration, state the GET-whitelist enforcement honestly, give the revoke command - Vercel MCP positioned as optional read-only enrichment (runtime logs, access links) with the purchase-tool caution — not the discovery backbone Validated: fixtures/vercel-capture/ replay set (manifest-fallback path, ISR cross-check, unclassified Route Handlers, skipped crons/usage, names-only env keys) + fresh-agent Discover replay: HANDOFF_OK, zero network/build/token, check_expected_discovery.py PASS. Also: shared:sync refreshed the stale vendored INTERPRETER.md this branch carried. * fix(vercel-to-aws): verify capture whitelist against Vercel REST API docs - Domains endpoint corrected to the documented /v9 (was /v5) - Env row note strengthened: the documented response schema carries value/ vsmValue/legacyValue fields even with decrypt=false — the key-name pipeline projection is the REAL protection, not the query param - Crons and storage rows reworded honestly: neither path is in the public REST reference; discover the GET endpoints via 'vercel api list' (the CLI's beta OpenAPI-backed api command), else record skipped — vercel.json and env-name/ dependency signals remain the fallbacks - Transport note added: prefer 'vercel api' (session/token auth, --paginate for the 100-per-page list endpoints, OpenAPI endpoint discovery), raw curl with manual pagination.next handling as fallback - Rate limits documented as a non-issue (200-1000 reads/min per endpoint vs this capture's small volume) - Fixture manifest endpoint strings synced * fix(vercel-to-aws): sync fixture manifest endpoints with the doc-verified whitelist The prior commit's message claimed this sync but the edit had failed on formatting drift — domains now /v9, crons/stores rows use the OpenAPI-discovered wording. * chore(vercel-to-aws): sync vendored aws-infra-pricing.json from canonical The merge from main brought the July 2026 canonical pricing update; the vercel skill's vendored copy was stale and failing shared:check. * style: dprint formatting after merge * fix(vercel-to-aws): constrain the OpenAPI-discovered capture rows Review follow-up: rows 6-7 (crons/stores) stay unpinned because Vercel does not document those paths -- pinning guesses would be fake precision that breaks silently. But the discovery step was looser than the pinned rows. New constraints: the resolved operation must be GET in the spec, its path must contain the row's subject segment (crons / stores|storage), the exact resolved path is recorded verbatim in the manifest entry with discovered_via: 'vercel api list' (audit trail), and no match means skipped -- never a close-enough substitute. Fixture manifest updated to model the audit fields. * feat(vercel-to-aws): best-effort terraform validate in the Generate assembler Review follow-up + parity fix: heroku (generate-terraform.md Step 12) and gcp (validation-report degraded-offline convention) both validate their generated Terraform; vercel's generate had static postconditions only. New assembler Step 6: static HCL checks (syntax, reference integrity, variable completeness, output references), then terraform init -backend=false && terraform validate when the binary is available. Failures and skipped runs land in generation-warnings.json with a founder-facing action line -- never a generation halt, matching the degraded-offline convention of the other two skills. * fix(vercel-to-aws): commit the missing capture/build manifest fixtures The capture manifest's build.files names three .next build manifests (routes/prerender/app-path-routes) that were never committed -- the validated replay ran against scratch copies. Reconstructed exactly per expected-discovery.json and discover-manifests.md: /blog/[slug] in dynamicRoutes with two prerendered entries at initialRevalidateSeconds 3600 (genuinely ISR), / static, /dashboard absent from prerender-manifest (dynamic), /api/checkout only in app-path-routes-manifest (Route Handler, unclassified by page manifests -> LOW-confidence dynamic). Preview keys are labeled fixture placeholders, not secrets. * fix(vercel-to-aws): un-ignore the fixture capture/build manifests The repo-root .gitignore's global build/ rule silently dropped the three canned .next manifests from the previous commit. Scoped negation inside the fixture directory re-includes them — they are committed test data mirroring the real capture layout, not build output. * fix(vercel-to-aws): routes-manifest arrays classify path shape, not rendering Fresh-agent replay against the restored build-manifest fixtures surfaced the ambiguity: staticRoutes/dynamicRoutes distinguish fixed paths from [param] segments -- a fixed-path page that renders per-request still sits in staticRoutes, and the replay classified /dashboard as static because of it. The rendering disposition comes from prerender-manifest membership: numeric initialRevalidateSeconds = isr, false = static, absent = dynamic. Spelled out the resolution table and extended the never-infer rule to cover static-from-staticRoutes as well as isr-from-revalidate-exports. * test(vercel-to-aws): estimate+generate replay fixtures; price both tiebreak paths Review follow-up: Discover had replay fixtures and an asserter; Estimate and Generate were spec-only. Now the whole pipeline is machine-checkable. Spec gap found while seeding: estimate had no handling for the unresolved [A,B] tiebreak that recommend legitimately produces and generate explicitly asks the founder about. Estimate is non-interactive, so it now prices BOTH candidate outcomes: projected_costs (and everything downstream) = Outcome A, a tiebreak_alternative section = Outcome B with its own Property-16 check, and mandatory tiebreak honesty in financial_summary/path_label (both balanced totals + how it gets resolved). The A-vs-B cost delta is itself a resolving input. Fixtures: seed-estimate/ and seed-generate/ are harvested from a validated fresh-agent chain replay (discover->clarify->recommend under documented-default answers, Q6 spend $200-1000, genuine [A,B] tiebreak; discover asserter PASS). expected-estimate.json pins the $600 midpoint baseline, both-path pricing, and comparison arithmetic; expected-generate.json pins the artifact set, OpenNext/Fargate mutual exclusion, provider pin, no-placeholder scan, compliance-none baseline, estimated-service coverage, and the Step 6 validate contract (pass = no warning; fail/skip = exactly one entry with a founder action). Both asserters PASS against fresh-agent replays of their phases (the generate replay's terraform init -backend=false + validate genuinely passed via a local provider cache). seed/.phase-status.json gains the estimate/generate phases this branch declares. * fix(vercel-to-aws): seed phase list includes the declared scaffold phase Caught by the new fixtures-check tool: the skill declares 6 phases (incl. scaffold) but the replay seed's .phase-status.json listed 5 -- phase-status.schema.json requires one entry per declared phase. (On the stacked generate-phase branch the seed already tracks that branch's estimate/generate phase set.) * fix(vercel-to-aws): Q7 confirms the discovered DB size; estimated rung; docs/spec cleanups Review-observation batch, replay-validated: - Q7 confirm-first: when discovery captured the postgres store's sizeBytes, Clarify derives the band (decimal GB) and presents it for founder confirmation instead of asking cold -- one less question per run, and the founder's correction wins when given. The phase postcondition now states the rule precisely: no COLD question whose answer discovery determined; confirmations are the sanctioned form. Validated by a fresh-agent discover->clarify replay (Q7 presented as a confirmation of band B from the ~3 GB store; HANDOFF_OK both phases). - Case fix folded back from that replay: five files matched peripherals[] against 'Postgres' while the discover schema emits lowercase 'postgres' -- a literal match would wrongly skip Q7, the DB migration script, and the has_databases complexity input. All now lowercase + case-insensitive. - Pricing hierarchy row 4 'estimated' (mirrors the heroku change on #150): the documented home of the services_by_source.estimated bucket; Lambda's formula constants are the canonical example. - generate-docs: guide step numbers count existing scripts (the file set is conditional, so numbering gaps are normal and must be called out), and a Provision Infrastructure slot between scripts 01 and 02 (terraform apply + sst deploy) -- both folded back from the generate replay. - SKILL.md: estimate/generate are read-only consumers of assessment-state.json (no phase file specifies a write; none should be improvised). - clarify.md: repaired the malformed declined-answer error-table row. * feat(vercel-to-aws): add post-Estimate what-if workshop checkpoint Freeze discovery/coupling/preflight and reprice traffic/outcome/region/arch via Recommend+Estimate inner runs with scenario snapshots. Outcome overrides use fired_rule workshop_override (declared contract) with transcript workshop_note provenance. Stacked on the generate-phase skill (#144). Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vercel-to-aws): resume must not recompute Estimate mid-workshop Port INTERPRETER deferred-advance checkpoint resume and vercel SKILL workshop resume rule so warm starts never re-run Estimate while workshop is unresolved. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(vercel-to-aws): render what-if scenarios in assessment report Add conditional what-if-scenarios section (workshop compare table) and enforce it in validate-assessment-report.py when scenarios/index.json has ≥2 entries. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(vercel-to-aws): list scenarios/ in generated README artifacts Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vercel-to-aws): rename assessment-report.html to migration-report.html Unify the stakeholder HTML filename with GCP and Heroku. Validator script name unchanged (validate-assessment-report.py) to avoid colliding with GCP's validate-migration-report.py. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vercel-to-aws): make the workshop branch pass mise build Pre-existing CI failures on this branch, all fixed: (1) eight files committed with dprint drift (fmt:check); (2) markdownlint MD025/MD029/ MD031 — the fixtures README second H1, report-render's scenario table breaking an ordered list, workshop-assemble's fenced block splitting a list, workshop-refresh's unspaced fences; (3) shared:check — the workshop schema additions (estimation-infra workshop metadata) and the Fargate/EC2 arm64 rates were edited in the VENDORED copies instead of the canonical skills/shared files. Ported vendored -> canonical and ran shared:sync (heroku's vendored copies pick up the same additive changes). mise run build now exits 0. * feat(vercel-to-aws): shareable calculator.aws link per workshop scenario workshop-refresh 6b: after each snapshot, best-effort build_estimate via the aws-pricing-calculator MCP -> shareable calculator.aws URL stored as estimation_summary.calculator_url (PRIMARY outcome's Balanced services; tiebreak_alternative skipped; null + one chat note on failure; never blocks). Compare table and report scenario section render the link. AWS computes regional prices server-side when opened -- the honest region answer. Server registration (.mcp.json) rides the heroku workshop branch; degradation here is silent-null when unconfigured. --------- Co-authored-by: Logan Kleier <lkleier@amazon.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Karthik Balasubramanian <carthick@amazon.com>
Resolves conflicts after the workshop stack + awslabs#149/awslabs#151/awslabs#155/awslabs#162 landed: - heroku-to-aws SKILL.md + heroku-live-capture/README.md: took main (stale awslabs#148/awslabs#150 lineage; main has the merged workshop/what-if + Scenario C content). - gcp-live-capture/README.md: kept THIS branch's added 'Scenario C — derived GCP baseline' block (its genuine new content; main's side empty there). - migrate/README.md: took main (adds vercel + agent-advisor reqs; awslabs#149's gcloud GCP requirement lines already present above the conflict). New gcp-infra-pricing-cache.md auto-enrolls in pricing:staleness (6/6 fresh) and the seed-baseline fixtures pass fixtures:check (141 json / 10 asserters). Full build green.
Summary
config.monthly_price_usd) into the Estimate phase's current-spend baseline: a new rung between billing data and the rate-card cache, with exact add-on math + published dyno rates and a mandatory not-an-invoice caveat.baseline_source.Motivation: live-discovered runs (the common TF-less case after #148) had real plan prices sitting in the inventory that Estimate could not use without a billing export.
Changes
estimate-cost-engine.mdlive_prices_plus_cache: add-ons summed exactly frommonthly_price_usd; dynos fromheroku-pricing-cache.md× quantity; per-resource fallthrough for unpriced entries; gate requires ≥1 add-on with a live price (a dyno-only capture falls honestly to the cache rung).cost_comparison.baseline_source; derived-source per-app breakdown variant.baseline_available/baseline_sourcereplacebilling_data_available); dual-run window priced at ~baseline/month with the derived caveat.ebsblock in the shared pricing file (kills the permanent "known omission" warning); sign conventions documented and cross-referenced (roi_analysisdifference vsfinancial_summarysavings); pricing-hierarchy row 4estimateddocumented.Schema
estimation-infra.schema.json(+ vendored sync):cached_staleadded to thepricing_source.statusenum — both heroku and gcp specs already mandated it; the enum rejected it (caught by the validation replay).Relationship to other PRs
Test plan
fixtures/heroku-live-capture/seed-estimate/(13-resource merged inventory, no billing profile, hand-assembled preferences + design per the specs)python3 fixtures/heroku-live-capture/check_expected_estimate.py <run-dir>asserts the exact baseline math ($220 live add-ons + $132 cache dynos = $352.00), source label, caveat wording, unlocked comparison arithmetic, Property-16, no fabricated billing fields, secret hygieneHANDOFF_OK, baseline derived exactly as specced, asserter PASS on the first runmise run buildgreen +lint:frontmatterFollow-up
GCP twin shipped as #156. Pre-existing observations recorded in PR comments (RDS Proxy formula-row placement,
service_count >= 9complexity cliff).