Experiment PR disposition + serialization policy#166
Open
danReynolds wants to merge 1 commit into
Open
Conversation
Codifies how scheduled experiment runs should close out their PRs so they stop stacking up open: - Disposition by diff, not verdict: PRs touching no runtime code (lib/native/hook) auto-merge on green; runtime-code PRs hold for review. - One experiment in flight at a time + atomic number reservation — the root cause of both the stacking and the three-way exp-168 number collision. - A recipe for resolving the shared derived-file (history.json/signals.json) conflicts when a PR does fall behind main. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds three sections to
resqlite-experiment/SKILL.mdso scheduled experiment runners stop leaving PRs stacked open.1. Closing out the PR: merge, auto-merge, or hold
Disposition is decided by the diff, not the verdict word:
lib//native//hook/clean) →gh pr merge --squash --auto. Covers recorded rejections, measurement lanes, tooling/CI guards. Close instead of merge if the record adds no reusable signal.2. One experiment in flight at a time
Every experiment PR rewrites the same shared files (
README.md,signals.json, generatedhistory.json), and the freshness CI gate fails any PR whosehistory.jsonpredates a merge — so concurrent PRs stale each other and pile up. Adds: atomic experiment-number reservation (check open PRs/branches, not just the README) and "don't open N+1 while a prior is unmerged." This is the root cause of the current pile and the three-way exp 168 number collision (#161/#162/#164 all claimed 168).3. Resolving a stale derived-file conflict
The mechanical recipe: merge main, regenerate
history.json/devices.json, hand-unionsignals.json, run the two CI checks locally, commit.Why now
Five experiment PRs were open at once; three of them (#161/#162/#164) collided on experiment number 168, which made their
signals.json/README impossible to union without a renumber. This policy prevents both failure modes going forward.🤖 Generated with Claude Code