ci(migrate): run committed fixture asserters in every build - #163
Merged
Conversation
4 tasks
Union-resolved mise.toml lint group: fixtures:assert (this PR) + fixtures:check + pricing:staleness (from awslabs#151/awslabs#155, now on main) all run in build. Registered the two gcp-live-capture asserters that landed via awslabs#149/awslabs#156 after this branch was cut (the PR's own registry-completeness guard correctly flagged them): - gcp-live-capture/check_expected_baseline.py (smoke — already exits 1 cleanly) - gcp-live-capture/check_expected_drift.py (smoke — added the same missing-artifact guard this PR applies to the other 4, replacing a FileNotFoundError traceback with a clean one-line FAIL + exit 1) fixtures:assert now 10/10 (3 golden, 7 smoke). Full build green.
icarthick
added a commit
to herosjourney/startups
that referenced
this pull request
Jul 22, 2026
Clean merge to current main (through awslabs#163). No conflicts. Full build green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The plugin's 8 committed fixture asserters (
check_expected_*.py) encode exact-math guarantees — workshop scenario deltas, reprice totals, drift detection, secret hygiene — but nothing ran them automatically: they were a run-it-by-hand convention. This PR adds afixtures:assertmise task, wired intomise run build(and therefore CI), that runs all of them on every build. Golden guarantees become permanent invariants, and asserter bitrot becomes a build failure instead of a silent gap.Changes
tools/run-asserters.py(new, stdlib-only): runs every asserter in two tiers.after-arm64-reprice,after-graviton-reprice,after-outcome-b-x86) — must PASS.check_expected_*.pyon disk that isn't registered fails the build, so future asserters can't ship un-run.mise.toml: newfixtures:asserttask, added to thelinttask group (runs inbuildand pre-existing CI).FileNotFoundErrortraceback when the run dir was missing its primary artifact (heroku-live-capture/check_expected_drift.py,heroku-live-capture/check_expected_estimate.py,vercel-capture/check_expected_estimate.py,vercel-capture/check_expected_generate.py): they now report a clean one-line FAIL and exit 1, matchingcheck_expected_discovery.py's existing guard style. This is also better replay UX — a failed replay gets "missing estimation-infra.json" instead of a stack trace.Relationship to other PRs
Independent — based on current
main(post-#162). Complements #151 (fixtures-check.tsvalidates fixture JSON shape; this PR executes the asserters). No file overlap with #151 or any other open PR.Test plan
mise run fixtures:assertgreen: 8/8 (3 golden PASS, 5 smoke clean-fail)mise run buildgreen (160/160 checks) including the new task and bandit with inline# nosecjustifications matching house convention