Re-derive the campaign the README picture claims - #31
Merged
Conversation
`plates.json` is what the showcase frame is rendered from: every well of every round, what went into it, and what the colorimeter read. It is a published claim about what this code does, and nothing re-ran it. A change to the optimizer, the dye model, or the instrument's noise would have left a picture in the README describing a campaign the code no longer runs, with every other check still green. Examples are not on the default test path, so nothing was watching. Byte equality is not available and demanding it would be worse than nothing: a campaign mints a fresh id per run, so the file differs on every execution for the one reason that carries no meaning. Identifiers are stripped and everything else is compared — the recipes tried, the colours measured, the errors, where the search ended up. Verified failing as well as passing. Changing the simulator seed makes it report `.best.measured_rgb[0]: committed 72.31714298414094, regenerated 71.68308612059515`, which is the level of detail this needs: a bare "the campaign changed" would send somebody diffing a six-hundred-well document by eye. Its own CI job rather than a step in `test`, because it costs a minute and would otherwise add that minute to all three Python versions. In parallel it costs nothing on the wall clock. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018PcmK6wy24Fokk5hqtfH74
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
plates.jsonis what the showcase frame in the README is rendered from — every well of every round, what went into it, and what the colorimeter read. It is a published claim about what this code does, and nothing re-ran it.A change to the optimizer, the dye model, or the instrument's noise would have left a picture in the README describing a campaign the code no longer runs, with every other check still green. Examples are not on the default test path, so nothing was watching.
Why not a byte diff
A campaign mints a fresh
campaign_idand a freshrun_idper well on every execution, so the file differs on every run for the one reason that carries no meaning. Demanding byte equality would make this fail constantly and get deleted.Identifiers are stripped at any depth; everything else is compared — the recipes tried, the colours measured, the errors, where the search ended up.
Verified failing, not just passing
Changing the simulator seed produces:
That precision is the point. A bare "the campaign changed" would send somebody diffing a six-hundred-well JSON document by eye.
Good news from writing it: the committed showcase is currently honest. It reproduces exactly.
Its own CI job
It costs a minute, and as a step in
testit would add that minute to all three Python versions. In parallel it costs nothing on the wall clock. Same reasoning the scene rebuild uses, one tier cheaper.make showcaselocally; documented inAGENTS.mdalongside the other gates.Gates
make lint0 ·make test707 passed ·make showcasepasses in 59s🤖 Generated with Claude Code
https://claude.ai/code/session_018PcmK6wy24Fokk5hqtfH74