Skip to content

AI-review fixes: correctness, statistical honesty, and format currency - #82

Draft
LlamaAdam wants to merge 24 commits into
masterfrom
claude/ollama-code-analysis-ak77i1
Draft

AI-review fixes: correctness, statistical honesty, and format currency#82
LlamaAdam wants to merge 24 commits into
masterfrom
claude/ollama-code-analysis-ak77i1

Conversation

@LlamaAdam

@LlamaAdam LlamaAdam commented Aug 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Implements the findings of three AI review rounds plus a negative-mode critique of the whole program. Analysis documents live in mtga-advisor#3 (docs/ollama-analysis/). One commit per theme.

Full suite: 3,520 passed, 164 skipped, 0 failed (baseline was 3,230 passed / 1 failed — the one environment-dependent live-network test is now properly stubbed).

Round 1 — the joint top-5 roadmap

Commit Fix
aad2d0e --auto-propose never applied the LLM's proposal to disk: it proposed against the new file and simmed two pre-existing decks, so every knowledge-log row paired a manifest with an unrelated diff. Now proposes against the old deck, materializes --new via apply_proposal_to_deck, records applied adds/cuts, and fails fast before any LLM spend. The web writer also stored the absolute margin in a column documented as signed, making regressions read as improvements.
af29600 Verdicts are significance-based: decisive counts are head-to-head only (filler pod wins no longer inflate the gate), and kept/reverted requires an exact two-sided binomial test at α=0.05. The old fixed `
27bc2d5 Bracket estimator re-based on the Oct-2025/Feb-2026 official rules: combo floor defers to the speed rule, the repealed 4+-tutor auto-bump is demoted to a labeled heuristic, extra-turn floors key on chainability. Extra-turn/MLD lists are regenerable from oracle snapshots; legality reports and commander-doctor warn on stale card data.
0407c4b Commander's free first mulligan (CR 103.5c) is modelled — consistency.py claimed the rule doesn't exist while deck_builder_manabase depended on it, so every keep/on-curve stat was pessimistic. The dashboard also preflight-checks decks against the vendored Forge corpus instead of silently producing partial sim data.
b30cff0 EDHREC ingestion tries json.edhrec.com before the brittle HTML scrape; manabase essentials gain triomes and surveil duals; deck_health shares the builder's land band so the grader stops docking the app's own builds.

Round 2 + negative mode

Commit Fix
a12380f Critical: the bandit/search evaluator bypassed the significance discipline entirely — reward >= --sim-margin (default 1) permanently advanced a deck on a 23-22 split, and crashed sims entered arm statistics as measured ties. Now routes through the binomial verdict, records failures as skipped pulls, and normalizes rewards so UCB1/Thompson actually explore.
7cae564 Archetype classification derived from oracle signals instead of card names (~70-85% of decks previously defaulted to "midrange", which made pool diversity a no-op and meant the estimator's combo/stax weights never fired). Pool ranking moves to a Wilson lower bound and finally acts on suspected_inflated.
926be8f Per-profile Forge lockfiles so concurrent web and CLI sims can't collide on a profile's deck dir, cache and log. Windows-safe (O_EXCL, not fcntl), with mtime-based stale reclaim.
c6714ce The dashboard rebuilt a ~32k-file card index on every request for decks containing an MDFC; now memoized behind a lock keyed on corpus mtime. PUT /api/deck_text was the one deck writer skipping the Name= restamp — pasting deck A's text into deck B made later sims misattribute wins.
f15d31f Statistical honesty everywhere a user sees a verdict: meta_test no longer claims "the references BEAT your deck" on a 1-2 record, verdict floors are aligned, tooltips state decisive-basis noise, and a fenced backfill script can repair legacy margins.
e7fadf8 Polish: no-op sim guard, manifest padding fields, EDHREC double-sleep, JS error log out of vendor/, price-staleness warnings.
67eba38 Intrinsic ward is a threat, not a protection slot (the new pattern was filling the protection quota with Dragons that protect nothing); 30 oracle fixtures declared, caught by the repo's own tripwire.
13bdafb, 0c734c0 Docs: environment-variable reference (16 flags documented nowhere), corrected advisor source count, honest layering statement, dead doc references, un-pinned stale test counts.

Notes

  • Every finding was verified against source before being acted on; the review rounds also corrected each other (e.g. the domain analyst caught that reanimator combos were mis-priced by the round-1 fix itself, and the code reviewer disproved two claimed classifier gaps by executing them).
  • Owner decisions on the remaining product questions are recorded in docs/ollama-analysis/DECISIONS_FOR_REVIEW.md in the analysis PR; the follow-on work (replication for unattended runs, sim-default raise, local-model router, umbrella CLI, politics guard) is queued from there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou

claude added 17 commits August 13, 2026 21:25
@claude
fix: auto-propose applies the proposal to disk; web margin stored signed
aad2d0e 
Two knowledge-log poisoners from the 2026-08-13 AI review (roadmap #1):

- propose_then_iterate() proposed against the --new file and then simmed
  two pre-existing files, so the recorded manifest and the simmed diff
  were unrelated. It now proposes against the OLD deck, materializes
  --new via apply_proposal_to_deck (same legality invariants as
  auto-curate), records applied adds/cuts as the manifest with the LLM
  intent under requested_*, and fails fast if --new already exists
  (before any LLM spend) or --old is missing.

- web save_iteration stored the payload's margin verbatim, which carries
  ComparisonReport.margin = abs(new-old); the knowledge-log column is
  documented signed. Web-saved regressions read as improvements in any
  pooled analysis. Margin is now always recomputed as new_wins-old_wins.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
feat: EDHREC JSON-first ingestion + modern manabase tiers (roadmap #5)
b30cff0 
- edhrec_client: commander/tag/salt/average-deck fetchers now try
  json.edhrec.com first and fall back to the __NEXT_DATA__ HTML scrape;
  shared payload walkers, same cache-poisoning guards on both paths.
  De-brittles ingestion and gives the bot-challenge-blocked salt
  backfill a JSON route.
- staples/deck_builder_manabase: add the 10 triomes (3+ color
  identities, tiered below untapped duals) and the 10 MKM surveil
  duals (top budget-tier two-color duals); resolvable offline via
  _TIERED_LANDS.
- deck_health now imports the builder's LAND_COUNT_BAND (33-40) instead
  of a hand-kept (33,38) — the grader no longer docks ~24 points from
  the app's own 39-40-land builds.
- target_land_count learns optional MDFC spell-front discounting
  (0.5/land, matching deck_health's weighting); callers unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
feat: bracket estimator on Oct-2025 rules; generated lists; legality …
27bc2d5 
…staleness

Roadmap #3 from the 2026-08-13 AI review — the estimator encoded
pre-Oct-2025 Commander bracket beta rules:

- Two-card combo floor defers to combo_detection's speed rule (late-
  assembling pairs are B3-legal since the 2025-10-21 update); unknown
  speed still floors B4 (conservative on missing data).
- Tutor 4+ auto-bump demoted from 'official rules transcription' to a
  clearly-labeled power heuristic — the official tutor restriction was
  repealed 2025-10-21 (the GC list carries efficient tutors now).
- Extra-turn floor re-keyed on chainability: B4 at 3+ extra-turn cards,
  or 2 plus a curated recursion/copy enabler; 1-2 non-chaining cards
  are a weighted nudge (B3-legal in low quantities).
- _EXTRA_TURN_CARDS +7 and _MLD_CARDS +8 well-known misses; both lists
  now regenerable offline from oracle snapshots via
  scripts/refresh_card_lists.py --only extra-turns|mld (tutors stay
  curated — oracle text can't separate tutors from fetches/ramp).
- Legality freshness: LegalityReport carries data_age_days +
  data_warning when backing snapshots exceed 45 days (7 B&R windows/yr
  now); doctor gains an oracle-snapshot staleness check. No auto
  refetch storms — informational only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
feat: significance-based A/B verdicts; honest LLM sim summaries
af29600 
Roadmap #2 from the 2026-08-13 AI review:

- heuristic_verdict counts decisive as old_wins + new_wins (head-to-
  head), matching the repo's own knowledge-log convention — it
  previously counted filler-won pod games, so the min_decisive gate
  effectively always passed.
- Fixed absolute-margin thresholds replaced with an exact two-sided
  binomial test vs p=0.5 (stdlib math.comb) in both verdict sites
  (analyst.heuristic_verdict, _proposer_sim._verdict_from_ab). Under
  the old rule, P(|new-old| >= 4) at 20 decisive games is ~0.50 — half
  of neutral swaps earned a confident verdict. Strong verdicts now
  require p < 0.05 (>= 15-5 at n=20). margin_* knobs deprecated but
  accepted; --sim-margin retained as a pre-filter.
- LLM-facing sim summaries (claude/ollama verdict paths) now carry
  signed_margin, winner, draws, and h2h_decisive instead of the
  absolute margin — a small model could previously read margin:6 on a
  regression as an improvement. Verdict prompt rewritten to match.
- ComparisonReport.to_dict gains signed_margin + h2h_decisive
  (add-only; margin/winner keys untouched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: model Commander's free mulligan; surface Forge sim coverage
0407c4b 
Roadmap #4 from the 2026-08-13 AI review:

- consistency.py claimed 'Wizards' Commander rules don't grant a free
  mulligan' — they do (CR 103.5c), and deck_builder_manabase's Karsten
  table already depends on it. The Monte Carlo now gives the first
  mulligan a fresh 7 with no bottoming; paid mulligans follow normal
  London count-down. Every keep/on-curve stat was systematically
  pessimistic (measured: avg kept hand 6.12 -> 6.47 on a
  mulligan-prone deck).
- Dashboard now reports sim coverage: deck names are preflight-checked
  against the vendored Forge card-script corpus and unsupported cards
  get a warning pill + detail modal — previously a deck full of cards
  Forge can't simulate simmed as silently-partial data. Additive
  sim_coverage payload key; fail-quiet when no corpus is vendored.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
docs: changelog + architecture notes for the AI-review roadmap fixes
fa8ba39 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
docs: fix drift found by negative-mode review
0c734c0 
- architecture.md: advisor 'routes to 7 sources' -> 3 backends +
  filters (the other four _advisor_* modules are models/filters);
  layering paragraph now states web sits on top and names the three
  deliberate reporting-surface exceptions to 'lower never imports
  higher' (status/doctor -> knowledge_log, deck_dashboard ->
  archetype/staples).
- pyproject.toml: script-naming comment cited PROJECT.md and
  audit_workflow.md, consolidated away 2026-05-13.
- STATUS.md: un-pin the hand-maintained '3002 passed' suite counts
  (three copies, all stale) in favor of 'run pytest for the live
  number'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: reanimation-aware combo speed pricing; chain-enabler repeatabili…
4fbe7b1 
…ty split

Two directional errors in the freshly-rebased bracket floors (round-2
cross-examination findings):

- combo_detection priced two-card combos by summed printed MV, so
  reanimator pairs (Worldgorger Dragon 6 + Animate Dead 2 = 8 >= 7)
  read 'late-game B3-legal' despite a ~4-mana effective assembly.
  Creature pieces partnered with a reanimation spell are now priced at
  min(printed MV, cheapest reanimation MV) — pattern-matched against
  real oracle wording with a hard-tagged offline fallback set.
  Missing data stays conservative (B4).
- _EXTRA_TURN_CHAIN_ENABLERS mixed one-shot rebuys with repeatable
  engines; 2 extra turns + Eternal Witness hard-floored B4. Split by
  repeatability per the official 'chained or looped' language:
  repeatable engines (Reiterate, Mirari, Mystic Sanctuary, Timetwister,
  Underworld Breach, Past in Flames) keep the hard floor; one-shot
  rebuys (Eternal Witness, Regrowth, Snapcaster, Twincast, ...) become
  a capped 0.25 weight. Calibration pins added for both directions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: role classifier learns seven evergreen card shapes
177bff3 
Verified-by-execution gaps (round-2 finding): Negate, Swan Song, Light
Up the Stage, Prey Upon, Diabolic Edict, and Earthquake all classified
'other'. Added patterns with false-positive guards + real-oracle
fixtures: restricted counterspells, impulse draw, fight/bite removal,
edicts, X-damage wipes, ward (protection), Treasure plurals (ramp).
Zero expectation drift in role_target_report/deck_health consumers;
interaction.py inherits the new wipe shapes by design.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: finish the significance discipline everywhere a user sees a verdict
f15d31f 
Round-2 statistical-honesty batch:

- meta_test no longer prints 'references BEAT your deck' on records the
  binomial test can't support (CLI default is 2 games/ref; a 1-2 split
  now reads 'not statistically distinguishable' with the split needed
  for significance).
- Verdict floors aligned: AnalystConfig.min_decisive_games now
  defaults to the canonical MIN_DECISIVE_GAMES_FOR_VERDICT (20),
  defined beside the binomial primitive and imported by _proposer_sim
  (an 8-0 sweep no longer earns 'kept' @ 0.97 through the analyst
  while the proposer path calls identical data inconclusive).
- _proposer_sim docstring example corrected: 26-14 does NOT clear
  alpha=0.05 at n=40 (p=0.081); the true boundary is 27-13.
- Web sim tooltips restated on the decisive-game basis
  (±0.22/±0.11/±0.07); '40 (confident)' relabeled '40 (verdict floor)'.
- save_iteration stores margin=NULL (not a fabricated 0) when no
  head-to-head decisive games exist.
- New scripts/backfill_web_margins.py: dry-run-by-default recompute of
  legacy web-saved absolute margins from persisted sim reports, hard
  fenced to id >= 314 (pre-fix rows are seat-attribution-suspect;
  recomputing them would launder artifacts).
- test_meta_test's live-EDHREC fetch stubbed (was the suite's one
  environment-dependent failure).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: bandit/search path obeys the significance discipline
a12380f 
Negative-mode P03 (critical) + P16:

- improve.py advanced the base deck on `reward >= --sim-margin` with a
  default margin of 1 — a raw one-win margin, no significance test, no
  decisive gate. A 23-22 split over 45 games permanently advanced the
  deck and logged it as validated. The accept decision now routes
  through _proposer_sim._verdict_from_ab (exact binomial at
  VERDICT_ALPHA over the standard decisive floor); --sim-margin
  survives as a back-compat pre-filter.
- Sim/apply failures returned reward 0.0, entering arm statistics as
  measured ties; they are now skipped pulls that leave arm stats
  untouched, with the reason logged.
- Rewards normalized to the signed decisive margin in [-1, +1]: raw
  O(±20) margins dwarfed UCB1's c*sqrt(ln N / n) bonus and Thompson's
  unit variance, collapsing both policies to greedy-on-one-noisy-pull.
  decisive == 0 yields no reward rather than a fabricated tie.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
feat: archetype v2 from oracle signals; Wilson-bound pool ranking
7cae564 
Round-2 finding: the name-keyword content scan needed 3 hits against
tiny lists, so ~70-85% of real decks fell to the 'midrange' default —
which made pool archetype-diversity a de facto no-op (every
arrangement 'violated', WARN-and-ship fired routinely) and meant the
bracket estimator's combo/stax weights never fired.

- classify() now derives labels from oracle-backed signals the
  pipeline already computes: game-ending combos + tutor density ->
  combo; interaction report (stack/wipe/instant share) -> control; a
  new ~10-pattern stax oracle table -> stax; tribal + curve/creature
  share -> aggro; midrange stays the honest default. Public signature
  unchanged; lookups are disk-cache-only (pool curation classifies
  ~60 candidates per run); a cold cache degrades to the name scan
  rather than reporting a false 'no lock pieces' zero.
- The NotImplementedError LLM stubs are gone — an offline v2 makes
  them moot.
- pool_curator ranks survivors on the Wilson lower bound and finally
  acts on suspected_inflated (demoted at equal bound) instead of
  merely tagging it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: cross-invocation Forge profile locking
926be8f 
Negative-mode P09: _discover_profiles re-enumerated vendor/forge*
per call and the free-queue was a per-call local, so the module's own
'two chunks never share a profile' guarantee held only within one
invocation. With the web UI launching background sim jobs, a second
web job or a concurrent CLI run could double-book a profile and
collide on its deck dir, cache, and forge.log.

Each profile now carries an advisory .commander-builder.lock acquired
via os.open(O_CREAT|O_EXCL) — atomic and, unlike fcntl, present on
Windows (the primary desktop target). Locked profiles are skipped
during discovery; all-locked fails fast with an actionable message
instead of hanging; locks older than a generous max-sim duration are
reclaimed by mtime (a bricked profile is worse than a rare
double-book after a SIGKILLed JVM); release happens in a finally on
every exit path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: auto-propose no-op guard, manifest padding, EDHREC sleeps, log p…
e7fadf8 
…ath, price staleness

Polish batch from rounds 1-2 and negative mode:

- propose_then_iterate aborts before the sim when the applier drops
  every proposed pair (it used to sim a deck against itself for 10+
  games and record a noise row).
- _materialize_proposed_deck copies padded_count/padded_breakdown/
  dropped_* into the manifest, so the manifest-matches-diff invariant
  holds on short imports too — the exact mismatch class the fix was
  written to eliminate.
- EDHREC JSON-first no longer double-sleeps on fallback, and skips the
  JSON probe for pasted URLs that have no JSON twin.
- The JS error log moves out of vendor/ into the user config home
  (P21); the old file is left alone with a pointer note.
- Pricing payloads carry price_data_age_days + a stale flag past the
  same 45-day threshold legality already uses (P19) — prices came from
  undated snapshots that can be months stale after a reprint.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
docs: environment-variable reference; correct JS error-log path
13bdafb 
P22: 16 COMMANDER_BUILDER_*/MTG_CARDS_DIR flags existed in source with
no reference anywhere (several documented only in CHANGELOG
archaeology). Adds a single table under Persistence locations, and
updates the js-error-log row to its new ~/.commander-builder home.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: cache the Forge card index; restamp and atomically write deck_te…
c6714ce 
…xt saves

Round-2 web-integrity batch:

- _sim_coverage built a fresh CardsLoader per request, and any MDFC
  front-face or unsupported card fell through to a full ~32k-file
  corpus scan whose result was then discarded — so every dashboard
  load of exactly the decks the feature exists to flag paid a 1-2s
  rescan. The supported-card index is now memoized behind a lock and
  keyed on corpus mtime, so a Forge upgrade still invalidates it.
- PUT /api/deck_text was the one deck writer that skipped the
  Name=/filename restamp dck_meta exists to protect: pasting deck A's
  text into deck B's editor stored Name=A and made every later sim
  misattribute B's wins. Saves are now restamped, written via temp
  file + os.replace, and rejected with a 400 when the body has no
  [Main] section.
- The three log_error tests move with the sink to the config home
  (the polish batch relocated it out of vendor/).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: intrinsic ward is a threat, not a protection slot; declare new f…
67eba38 
…ixture roles

The full-suite run caught what per-batch runs could not:

- The round-2 ward pattern matched the bare keyword, so any creature
  that merely HAS ward (Miirym, Phyrexian Fleshgorger) classified as
  'protection'. That fills the ROLE_TARGETS protection quota — which
  exists to guarantee a deck can protect its commander — with bodies
  that protect nothing but themselves, suppressing the advisor's real
  protection recommendations. The pattern now requires a GRANT
  ('has/have/gains ward'), so equipment, auras and instants still
  count and resilient threats no longer do.
- Two concurrent batches added 30 oracle fixtures without declaring
  expected roles; the repo's own tripwire test caught it. All 30 are
  now declared, each annotated with the pattern it exercises.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@LlamaAdam LlamaAdam changed the title AI-review roadmap: five correctness and format-currency fixes AI-review fixes: correctness, statistical honesty, and format currency Aug 17, 2026
claude added 7 commits August 17, 2026 07:19
@claude
feat: owner-approved decisions batch 1 (host gate, [REF] fillers, reb…
05deaa7 
…uild opt-in, era stamps, sim caveat)

Five decisions from the 2026-08-17 owner review:

- The Flask before_request hook now rejects any request whose Host
  isn't loopback (127.0.0.1/localhost/::1, optional port), closing the
  DNS-rebinding path to deck reads and PUT /api/config. Note: serving
  with --host 0.0.0.0 will now 403 LAN Host headers, which is the
  intended trade.
- [REF] decks (Moxfield top-likes) leave filler seats, matching the
  [PREMADE] popularity rule; they remain pool candidates, since a
  ranked seat and an unranked filler seat are different jobs.
- The 30+30 rebuild tier is opt-in (COMMANDER_BUILDER_REBUILD_TIER,
  matching the card_score convention): auto-mode caps at overhaul and
  says why, because the health score gating that 6x cost multiplier
  has never been validated. --mode rebuild is unaffected.
- knowledge_log schema v3 stamps every row with its measurement era
  (1 pre-seat-attribution, 2 mixed denominators, 3 head-to-head
  margin verdicts, 4 significance verdicts) so the three incompatible
  eras stop contaminating pooled analysis. Date decides and id only
  breaks ties — an id-first draft retro-labelled fresh low-id rows.
  Unknowable rows stay NULL rather than guessed.
- README states plainly that a kept verdict certifies 'better against
  Forge's AI', not 'better at your table'.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
docs: record decisions batch 1 (changelog, env table, era supersedes …
8acff41 
…--min-id)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
feat: sim default is the verdict floor; unattended runs replicate bef…
6a7fb56 
…ore advancing

Owner decisions A3 and A2 (2026-08-17):

- --sim-games now defaults to min_sim_games_for_verdict() (40 pod
  games) instead of 5. At 5 games --run-sim could only ever record
  'inconclusive', so the flagship close-the-loop flag structurally
  could not close the loop. A cost line prints before the spend
  ('running 40 pod games (~67 min) -- the minimum for a statistically
  meaningful verdict'), and --smoke restores the 5-game sanity check
  while saying plainly that it will record inconclusive.

- Unattended improve runs now require a SECOND independent A/B in the
  same direction before advancing the base deck. Greedy round-chaining
  defaults ON (a false 'kept' becomes the base every later round is
  measured against); the interactive bandit explorer defaults OFF
  since UCB1 already re-pulls arms. Explicit --replicate/--no-replicate
  always wins.

  A failed replication invents no new verdict label: the row takes the
  second run's verdict with a 'replication_failed: run 1 kept, run 2
  neutral ... base NOT advanced' note, run 1's sim report intact. The
  confirm run is a bare A/B, never a round or a pull, so it cannot
  recurse. Confirmed swaps cost 2x sim time, stated up front.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
docs: changelog for the sim-default and replication decisions
23eb98d 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
feat: politics guard on by default; Archidekt as a fallback import lane
757da2b 
Owner decisions C2 and C3 (2026-08-17):

POLITICS GUARD. Forge's AI does not negotiate, pick an archenemy, or
model an opponent's incentive to pay a tax, so goad / monarch / vote /
tempting-offer / Rhystic-tax / pillow-fort cards read to the sim as
no-ops. A loop that cuts what doesn't move the margin will therefore
'empirically' cut exactly the cards that define multiplayer Commander:
a near-zero margin on one of these is evidence the instrument is
blind, not evidence against the card.

staples gains an oracle-pattern politics tagger (six tags, with
guards so 'unless you pay' upkeep costs and 'unless its controller
pays' soft counters don't match). Tagged cards are exempt from
margin-driven cuts in all three cut paths — the heuristic loop,
card_score's ranking, and now the orchestrator, which is the only
path Claude's and bracket-peers' cuts traverse. Skips are disclosed
via AdviceReport.skipped_for_politics rather than silently shortening
the list. Per-deck opt-out: [metadata] PoliticsGuard=off, carried
across re-imports, failing safe to ON on any unrecognized value.

ARCHIDEKT LANE. Acquisition rode Moxfield's undocumented private API
alone; one ToS or CDN change stranded imports, harvest, peers and
meta-test refs at once. Archidekt (public, documented) is now a real
single-deck import lane via import_deck(source=)/--archidekt, with
fallback on 5xx/403/URLError/parse failures but NOT on 404 (a working
API saying 'no such deck' must not silently import a different one).
No id translation is attempted. Moxfield-only capabilities (bulk
bracket harvest, top-likes search) now name the working alternative
when they fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
docs: changelog for the politics guard and Archidekt lane
6ac3f41 
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
@claude
fix: FP-013 gate counts only rows whose LABELS are trustworthy
3abc711 
The gate checked the training triple's shape -- manifest, decided
verdict, >=40-game sim -- but not whether the verdict meant anything.
A pre-e8777b6 row can satisfy all three and still be worthless: its
wins were credited to the wrong deck. The fine-tune learns the
VERDICT, so the era that matters is the one that produced the LABEL,
not the one that produced the raw counts.

Rows must now come from measurement era >= 4, the first era whose
kept/reverted comes from the significance test. Era 3 is reported
separately as 'relabelable' rather than dropped: its measurement is
sound (same decisive denominator), only its verdicts came from the
game-count-invariant |margin| >= 4, so re-scoring the stored sim
report promotes it. Eras 1-2 are unrecoverable and excluded, as is an
unstamped row -- unknown provenance is not evidence of good
provenance. --health discloses both buckets so the headline number
never reads as 'that's all the history there is'.

Worth noting alongside the same day's sim-default change: at the old
default of 5 games this gate was mathematically unreachable, since
every row the flagship flag produced fell under its own 40-game bar.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mdg9HA1ZkdKQ3wpZvra3ou
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@LlamaAdam @claude