Skip to content

tech-debt-backlog §§7.18, 7.19: harden scheduled-task prompts - #137

Merged
jascal merged 1 commit into
mainfrom
tech-debt-backlog-7-18-7-19
Jun 16, 2026
Merged

tech-debt-backlog §§7.18, 7.19: harden scheduled-task prompts#137
jascal merged 1 commit into
mainfrom
tech-debt-backlog-7-18-7-19

Conversation

@jascal

@jascal jascal commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Summary

Two [S]-sized prompt hardening items from PR #78's review log
(logs/pr-review-2026-05-28.log), both targeting the scheduled-
task prompts under scripts/.

  • §7.18 — scripts/pr-review-prompt.txt step 2.b. Replaced
    the body contains "Claude" substring check with a "starts
    with" match against the canonical header ## Code Review — Claude Sonnet 4.6. The header is already required by step 3.e
    of the same prompt, so anchoring on it is the lowest-friction
    discriminator and prevents a false-positive skip when a human
    reviewer happens to mention Claude in a normal review body.
    Chose option (b) of the three the task body enumerated:
    (a) couples the check to a fragile bot identity, (c) adds an
    out-of-band marker for a property the step-3.e header already
    conveys structurally.
  • §7.19 — scripts/nightly-prompt.txt Step 2. Restructured
    the cross-check into two ordered checks.
    1. Whole-change exact match. Compare <change-name> to
      every open PR's headRefName, stop on equality. Catches
      single-task changes (add-reset-syntax,
      fix-mps-encoding-non-factorizing, …) whose branch
      matches the change directory by convention and which the
      existing §N.M-anchor scan misses because they don't use
      anchors.
    2. Per-task anchor scan, kept as-is and labelled as the
      granular case for multi-task changes like
      tech-debt-backlog whose individual PRs branch off as
      tech-debt-backlog-7-18, tech-debt-backlog-7-16-7-17,
      etc.

Bundled because both items come from the same review log and
both target sibling files in scripts/, matching the §7.16/§7.17
(PR #76, → PR #135) and §7.21–§7.24 (PR #99, → PR #129) bundling
precedents.

Test plan

  • pytest -q: 1282 passed, 8 skipped — the two prompt
    files are not referenced by anything under tests/
    (verified by grep -ri 'pr-review-prompt\|nightly-prompt'
    over tests/), so the edits land as doc-style changes.
  • Both tasks marked [x] with closing notes in
    openspec/changes/tech-debt-backlog/tasks.md per the
    file's convention (§6.1).
  • §7.18 will be exercised on the next scheduled run of the
    PR-review automation; §7.19 on the next nightly. Both are
    pure prompt edits, so the behaviour change is observable
    only at the next automation invocation.

🤖 Generated with Claude Code

Both items from PR #78's review log; both target the
`scripts/*-prompt.txt` files the scheduled automations consume.

§7.18 — `scripts/pr-review-prompt.txt` step 2.b. Replaced the
`body contains "Claude"` substring check with a "starts with"
match against the canonical header `## Code Review — Claude
Sonnet 4.6`. The header is already required by step 3.e of the
same prompt, so anchoring on it is the lowest-friction
discriminator and prevents a false-positive skip when a human
reviewer happens to mention Claude in a normal review body
(option (b) of the three the task body enumerated).

§7.19 — `scripts/nightly-prompt.txt` Step 2. Restructured the
cross-check into two ordered checks. (1) Whole-change exact
match: compare `<change-name>` to every open PR's
`headRefName`, stop on equality — catches single-task changes
(`add-reset-syntax`, `fix-mps-encoding-non-factorizing`, …)
whose branch matches the change directory by convention and
which the existing `§N.M`-anchor scan misses. (2) Per-task
anchor scan, kept as-is and labelled as the granular case for
multi-task changes like `tech-debt-backlog` whose individual
PRs branch off as `tech-debt-backlog-7-18`,
`tech-debt-backlog-7-16-7-17`, etc.

Bundled in one PR because both tasks come from the same review
log (`logs/pr-review-2026-05-28.log`) and both target sibling
files in `scripts/`, matching the §7.16/§7.17 (PR #76) and
§7.21–§7.24 (PR #99) bundling precedents.

Tests: pytest -q is green at 1282 passed / 8 skipped — the
prompt files are not referenced by anything under tests/, so
the edits land as doc-style changes. The new behaviour will be
exercised on the next scheduled run of each automation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@jascal jascal left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — Claude Sonnet 4.6

Scope: automation-prompt-only. Two .txt files under scripts/ (pr-review-prompt.txt step 2.b, nightly-prompt.txt Step 2) plus the [x] closure notes for §7.18/§7.19 in openspec/changes/tech-debt-backlog/tasks.md. No production code, no test code.

Verdict

Looks good — ship it. Both edits are correct, internally consistent with the prompts they live in, and add zero lint or test debt. I was able to verify §7.18 live (this very review run exercised it), and §7.19's two-check ordering is sound. A few non-blocking observations below, the most useful being a version-coupling note on §7.18.

Correctness

§7.18 — header-anchored dedup check. The substring body contains "Claude"body **starts with** "## Code Review — Claude Sonnet 4.6" swap is the right fix for the stated false-positive (a human reviewer mentioning "Claude" in a normal body). I verified it end-to-end rather than trusting the prose, because this run is the consumer of the edited prompt:

  • PRs #135 and #136 each carry a prior review whose body begins with exactly ## Code Review — Claude Sonnet 4.6 → both correctly matched and skipped.
  • PR #137 has empty reviews/comments → correctly flagged for review.

So the new discriminator behaves correctly against real data. It works because gh pr review --comment --body preserves the body verbatim, so the canonical header really does sit at offset 0 (confirmed against #135/#136's stored bodies). The em-dash (, U+2014) in the step-2.b literal matches the one in the step-3.e header and in the posted bodies — important, since a hyphen/em-dash mismatch would silently break the match.

§7.19 — two ordered cross-checks. The restructure is correct, and the ordering matters in exactly the way the PR claims:

  • Check 1 (headRefName == <change-name> exact) fires only for single-task changes, whose branch equals the change name by the Step 3 (git checkout -b <change-name>) convention. For a multi-task change like tech-debt-backlog, the open branches are tech-debt-backlog-7-18-7-19, tech-debt-backlog-7-20, etc. — none equals tech-debt-backlog, so Check 1 correctly does not fire and does not wrongly abort the whole change.
  • Check 2 (the pre-existing per-task §N.M anchor scan) then handles the multi-task granularity.

Short-circuiting the entire nightly in Check 1 (stop … do not implement) is the right call given Step 1 has already committed the run to one change: if that whole change is in flight as one PR, there is no uncovered work. Good.

Non-blocking notes

  1. §7.18 hardcodes the model version in two places that must move in lockstep. The dedup literal in step 2.b (## Code Review — Claude Sonnet 4.6) and the required header in step 3.e are now coupled by an exact version string. If step 3.e's header is ever bumped (e.g. a future "Sonnet 4.7" or model rename) without bumping step 2.b in the same edit, the "starts with" check stops matching newly-posted reviews and duplicate reviews silently return — the exact failure mode this task set out to prevent, reintroduced one version bump later. The old "Claude" substring was looser but version-agnostic. The trade-off here is reasonable (false-positive skips are the more likely/annoying failure, and a version bump is a deliberate edit that should touch both lines), but it's worth a one-line comment in the prompt tying the two literals together, e.g. anchoring step 2.b on the model-agnostic prefix ## Code Review — Claude instead of the full version string would keep the false-positive fix while decoupling from the version. Optional.

  2. §7.19 leans on a branch-naming convention that Step 3 doesn't actually emit. Check 1's exactness and Check 2's tech-debt-backlog-7-18 examples both assume per-task branch names, but Step 3 line 51 literally only does git checkout -b <change-name> — it never produces the -7-18-7-19 suffix that the multi-task PRs (including this one) actually use. This is a pre-existing tension, not introduced here, and it doesn't affect §7.19's correctness for the single-task case Check 1 targets. Flagging only because the new text makes the convention load-bearing; a future cleanup task could reconcile Step 3's branch-naming instruction with the observed multi-task naming.

  3. §7.19 residual gap (acknowledged, narrowed not closed). A single-task change whose branch does not follow the branch == change-name convention and which lacks a §N.M anchor would still slip past both checks. Check 1 closes the common convention-following case, which is the right 80/20 — just noting the tail isn't fully covered, consistent with the PR body's own framing.

Lint (ruff)

.venv/bin/ruff check scripts/ → "No Python files found … All checks passed!" (the prompts are .txt, so ruff never touches them). Full-tree ruff check . reports 18 errors, all pre-existing and confined to test files (test_compiler.py ×9, test_noise_model_section.py ×5, test_verifier.py ×2, plus test_qpc_convergence.py, test_examples.py) — unused pytest imports and E402. Same baseline flagged in the #135/#136 reviews. This PR adds zero lint debt and remains a good candidate for a dedicated test-file cleanup task.

Tests

.venv/bin/pytest --tb=short -q on this branch: 1282 passed, 8 skipped, 0 failed (~23s). Matches the PR's test-plan claim exactly, and confirms #135's suspicion that the "78 failed on main" paragraph in that PR was an environment artifact, not a real baseline — the suite is fully green here.

Test coverage

N/A — neither prompt is referenced by anything under tests/ (verified). The flip side worth recording: the dedup and cross-check logic these prompts encode has no automated guard, so its correctness rests entirely on live automation runs. §7.18 is the lucky case — this run exercised and confirmed it. §7.19 will only be exercised on the next nightly; until then its correctness is by-inspection only.

Security / performance

None. Pure prompt text; no execution-path or data-flow change.

What's good

  • The change is self-dogfooding and verifiable — §7.18 edits the exact check the running automation uses to decide what to skip, and it demonstrably did the right thing on #135/#136/#137 in this run. That's the strongest kind of evidence a prompt edit can have.
  • Splitting the nightly cross-check into an explicitly-labelled "whole-change vs per-task" pair makes the single-task-vs-multi-task distinction legible to the next reader, instead of leaving it implicit in one dense paragraph.
  • The tasks.md closure notes are specific and honest about the options not taken (rejecting (a) bot-identity coupling and (c) an out-of-band marker, with reasons) — good documentation discipline, consistent with the §7.16/§7.20 closures.

This review was posted automatically by Claude Sonnet 4.6.

@jascal
jascal merged commit 1fbea27 into main Jun 16, 2026
6 checks passed
jascal added a commit that referenced this pull request Jun 16, 2026
#138)

Three [XS] follow-ups, all surface-only wording fixes from prior
PR review logs.

- §7.27 — fix the line citation in §7.22's resolution note.
  `examples/hybrid-bridge/README.md:60` → :59. `2·asin√0.85 ≈ 2.346`
  actually sits on line 59 of that file; the PR #129 reviewer flagged
  the off-by-one and it persisted in the merged backlog body.
- §7.28 — replace "class-level docstring" with "leading `#` comment
  on the class" in §7.12's resolution note. Verified against
  tests/test_mcp_server.py:181-188: the §7.12 rationale lives in a
  contiguous `#` comment block immediately above the
  `@pytest.mark.skip`, not a docstring (no `"""…"""` exists on the
  class). The prior wording would mislead a future reader using
  `pydoc` / `__doc__` to find it.
- §7.29 — rewrite the below-atol-guard test's spectrum-recovery
  comment. PR #134's "CNOT inverse-permutation is unitary, so
  singular values propagate unchanged" framing was loose — an entry
  permutation is a row-dependent column swap that does not in
  general preserve a *reshaped* matrix's spectrum. New comment
  references the actual chain that does: exact inverse swap of
  _apply_cnot's forward swap, then a `full_matrices=False` SVD with
  every singular value kept (no truncation), so the reconstruction
  is exact and _apply_cnot's downstream SVD sees the singular
  values of M itself.

Bundled because all three are XS doc-nits from review logs, matching
the §§7.21–7.24 (PR #129), §§7.16/7.17 (PR #135), and §§7.18/7.19
(PR #137) bundling precedent.

`pytest -q`: 1282 passed, 8 skipped (the §7.29 test fixture stays
green; the §7.27/§7.28 edits are tasks.md-only and not referenced
by any test).

Tasks marked `[x]` with closing notes per §6.1.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant