From 742c00517979b7e926f448701c429aec3f6d8357 Mon Sep 17 00:00:00 2001 From: Roberto Cano <3525807+robercano@users.noreply.github.com> Date: Fri, 24 Jul 2026 21:05:33 +0200 Subject: [PATCH 1/4] =?UTF-8?q?feat(loop):=20milestone-scoped=20census=20?= =?UTF-8?q?=E2=80=94=20drain=20current=20version=20before=20wandering=20(i?= =?UTF-8?q?ssue=20#174)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Milestones represent versions/sprints; the PR-loop census now scopes its ADVANCE candidate set to the lowest version-sorted OPEN milestone that has a qualifying planned+module candidate, recomputed fresh every tick (no persistent cursor), falling back to today's unscoped behavior when no open milestone qualifies — byte-identical output for repos that don't use milestones. Emits milestone=/milestone_open= census lines, and logs an idempotent milestone-complete event (the one deliberate exception to census's read-only/re-run-safe contract) when a milestone's open_issues count drains to zero. Co-Authored-By: Claude Sonnet 5 --- .claude/scripts/loop-census.sh | 190 +++++++++++++++- .claude/scripts/loop-census.test.sh | 324 +++++++++++++++++++++++++--- 2 files changed, 483 insertions(+), 31 deletions(-) diff --git a/.claude/scripts/loop-census.sh b/.claude/scripts/loop-census.sh index c426b5c..aef3dbf 100644 --- a/.claude/scripts/loop-census.sh +++ b/.claude/scripts/loop-census.sh @@ -63,6 +63,17 @@ # non-"none" advance_ready when plan.gate != # "off" (issue #100) — tells the tick which # driver prompt variant to build. +# milestone= the CURRENT open milestone in scope (issue +# milestone_open=<n> #174) — see MILESTONE SCOPING below. Emitted +# ONLY when a milestone is actually in scope +# (immediately after `planned_issues=`, before +# the per-issue `issue=`/detail lines — see +# MILESTONE SCOPING for why this position was +# chosen). Absent entirely on the fallback path +# (no qualifying open milestone, or no +# milestones at all), so census output for a +# repo that doesn't use milestones stays +# byte-identical to before this feature. # main_dirty=yes|no `git -C $root status --porcelain` is non-empty # AFTER excluding (a) sandbox-mask phantom paths # (device-node masks, see below) and (b) the @@ -118,6 +129,63 @@ # blocked candidate is skipped regardless of how high its priority is; edges # are semantics, priority is just iteration order among what's unblocked. # +# --- MILESTONE SCOPING (issue #174) ----------------------------------------- +# Milestones represent versions (SCRUM sprints): the loop must drain the +# CURRENT milestone before it wanders into a future one. "Current" is derived +# FRESH every run (no persistent cursor) as: among the repo's OPEN milestones +# (state=open), the one with the lowest version-ish title (natural/`sort -V` +# semantics — v1.0 < v1.1 < v2.0, "Sprint 1" < "Sprint 2") that has AT LEAST +# ONE open candidate — same predicate the census already uses (planned label +# + a module: label). When that milestone drains (0 qualifying candidates), +# THIS SAME re-derivation picks the next-lowest qualifying open milestone as +# current on the very next tick automatically — no bookkeeping. An idle gap +# after a milestone drains, before the owner labels the next milestone's +# issues `planned`, is intended: the owner controls phase boundaries by when +# they add that label. +# +# SCOPE: the candidate set for ADVANCE (planned_issues/issue=/in_flight=/ +# stalled=/blocked=/plan_wait=/advance_ready=) is filtered down to ONLY the +# current milestone's issues — applied as a FILTER on top of the existing +# (priority, number) ordering (issue #173), never disturbing it. Feedback/ +# merge-phase counts (open_prs/feedback_prs/ci_fix_prs/comment_fix_prs/ +# rebase_prs) are completely UNAFFECTED: open PRs are always serviced +# regardless of milestone. +# +# FALLBACK (graceful degradation): if NO open milestone has a qualifying +# candidate — including the common case of a repo that doesn't use +# milestones at all, where the REST fetch below returns an empty set — census +# falls back to TODAY'S unscoped behavior (every planned+module candidate, +# repo-wide). This is why `milestone=`/`milestone_open=` are emitted ONLY +# when a milestone is actually in scope: it keeps output for a +# milestone-less repo byte-identical to before this feature, rather than +# printing an empty/sentinel milestone line every tick. +# +# WHERE milestone=/milestone_open= SIT: immediately after `planned_issues=`, +# before the per-candidate `issue=` detail lines — grouped with the other +# scope-summary fact (planned_issues is already the SCOPED count once a +# milestone is in play) rather than interleaved among per-issue lines. +# +# gh 2.4.0 CONSTRAINT: this gh version has no `gh milestone` subcommand and no +# `gh api graphql` — milestone data is fetched with a plain REST `gh api` +# call (`repos/{owner}/{repo}/milestones?state=open`), always routed through +# bot-gh.sh like every other gh call here. Per-issue milestone association +# reuses the EXISTING `gh issue list --json ...` call that already fetches +# the planned+module candidates — a `milestone` field is simply added to its +# --json/--jq, rather than issuing a second per-milestone issue query. +# +# MILESTONE-COMPLETE EVENT: the milestones REST response already reports +# `open_issues`/`closed_issues` per milestone, so "the last issue in a +# milestone closes" is detected directly (open_issues==0 with closed_issues +# >= 1, i.e. genuinely drained rather than never-populated) without any extra +# gh call. This is the ONE deliberate exception to this script's read-only / +# re-run-safe contract (see the file-level comment below): it appends to +# events.jsonl via log-event.sh. To stay idempotent under re-runs (no +# duplicate event every idle tick), it FIRST scans events.jsonl for an +# existing milestone-complete event carrying that milestone's title, and +# only logs when none is found — approach (b) from the issue's acceptance +# criteria, chosen because it needs no change to loop-tick.sh's control flow +# and keeps the guard co-located with the detection logic that needs it. +# # --- BLOCKING-GRAPH GATE (issue #97) ---------------------------------------- # advance_ready additionally skips any otherwise-eligible candidate (branch= # none, open_prs=0, no active driver) whose body contains an explicit @@ -187,6 +255,9 @@ # Repo derived from the git remote; override with $1. Bot login via $BOT_LOGIN. # Invoke as `bash .claude/scripts/loop-census.sh` (pre-approve that exact # command). Read-only: advances no cursor, mutates nothing — safe to re-run. +# The SOLE exception is the milestone-complete event append (issue #174, see +# MILESTONE SCOPING above) — an idempotent, guarded events.jsonl write, never +# a stdout/behavior change; every other line above stays a pure read. set -euo pipefail # Two-root derivation (issue #63): script_dir = sibling scripts, root = consumer project. @@ -274,6 +345,67 @@ case "$plan_mode" in off|label|always) ;; *) plan_mode=off ;; esac events_file="${CLAUDE_EVENTS_FILE:-$root/.claude/state/events.jsonl}" +# --- milestone REST fetch (issue #174) --------------------------------------- +# gh 2.4.0 has no `gh milestone` subcommand and no `gh api graphql` — plain +# REST, always through the bot-gh.sh wrapper. `2>/dev/null || true` degrades +# gracefully (empty $milestones_tsv) on ANY failure — a repo with no +# milestones, a stubbed bot-gh.sh in tests that doesn't implement `api`, or a +# transient gh error — which is exactly the FALLBACK path (see MILESTONE +# SCOPING above): census must never abort, and an empty result here makes +# every downstream milestone check a no-op, falling back to today's unscoped +# behavior. +milestones_tsv=$(gh api "repos/$repo/milestones?state=open" \ + --jq '.[] | [.number, .title, .open_issues, .closed_issues] | @tsv' 2>/dev/null) || true + +# Version-sort ascending by title (2nd TSV field) — natural/`sort -V` +# semantics (v1.0 < v1.1 < v2.0, "Sprint 1" < "Sprint 2"). GNU coreutils sort +# supports "V" as a per-key modifier (`-k2,2V`), so this needs no manual +# swap-sort-swap dance. +milestones_sorted="" +if [ -n "$milestones_tsv" ]; then + milestones_sorted=$(printf '%s\n' "$milestones_tsv" | sort -t $'\t' -k2,2V) +fi + +# --- milestone-complete event (issue #174) — THE ONE read-only exception ---- +# A milestone is "complete" when the REST fetch's own open_issues/ +# closed_issues counters show it fully drained (open_issues==0) AND it +# genuinely had issues to drain (closed_issues>=1 — never fires for an empty/ +# never-populated milestone). Guarded against duplicate logging by scanning +# events_file FIRST for an existing milestone-complete event carrying this +# milestone's title before appending — safe to re-run every tick, exactly +# like every other check in this script, EXCEPT this one intentionally +# mutates events.jsonl (via log-event.sh) as its side effect. +if [ -n "$milestones_tsv" ]; then + while IFS=$'\t' read -r ms_num ms_title ms_open ms_closed; do + [ -z "${ms_title:-}" ] && continue + case "$ms_open" in ''|*[!0-9]*) continue ;; esac + case "$ms_closed" in ''|*[!0-9]*) continue ;; esac + if [ "$ms_open" -eq 0 ] && [ "$ms_closed" -ge 1 ]; then + already_logged=$(CLAUDE_MS_EVENTS_FILE="$events_file" CLAUDE_MS_TITLE="$ms_title" node -e ' + const fs = require("fs"); + const file = process.env.CLAUDE_MS_EVENTS_FILE; + const title = process.env.CLAUDE_MS_TITLE; + let found = false; + try { + const text = fs.readFileSync(file, "utf8"); + for (const line of text.split("\n")) { + if (!line.trim()) continue; + let o; + try { o = JSON.parse(line); } catch (e) { continue; } + if (o.phase === "milestone-complete" && o.task === title) { found = true; break; } + } + } catch (e) { /* no file yet -> not logged */ } + console.log(found ? "yes" : "no"); + ' 2>/dev/null) || already_logged="no" + if [ "$already_logged" != "yes" ]; then + CLAUDE_EVENTS_FILE="$events_file" bash "$script_dir/log-event.sh" \ + --role census --task "$ms_title" --phase milestone-complete \ + --detail "milestone drained (all issues closed)" >/dev/null 2>&1 || true + fi + fi + done <<< "$milestones_tsv" +fi + # --- stall detection helper (issue #98) -------------------------------------- # $1 = issue number. Prints the age in whole minutes of the NEWEST # events.jsonl line whose `task` field is either "$1" or "issue-$1" (both @@ -374,10 +506,13 @@ echo "rebase_prs=$rebase_prs" # #173). Rank is derived from the labels field already in the TSV (no extra # gh call): prepend a rank column, sort numerically on (rank, number), then # strip the rank column back off so the downstream `while IFS=$'\t' read -r -# num labels title` loop is unchanged. Titles are the LAST TSV field (may -# contain spaces) and are left untouched by this transform. -planned=$(gh issue list -R "$repo" --state open --label planned --json number,title,labels \ - --jq '.[] | [.number, ([.labels[].name]|join(",")), .title] | @tsv' \ +# num labels milestone_title title` loop is unchanged. The `milestone` field +# (issue #174) is fetched via the SAME --json/--jq as the rest of this TSV +# (never a separate per-milestone issue query — see the gh 2.4.0 note in +# MILESTONE SCOPING above), placed BEFORE title so title stays the LAST TSV +# field (may contain spaces) and is left untouched by this transform. +planned=$(gh issue list -R "$repo" --state open --label planned --json number,title,labels,milestone \ + --jq '.[] | [.number, ([.labels[].name]|join(",")), (.milestone.title // ""), .title] | @tsv' \ | awk -F'\t' 'BEGIN { OFS = "\t" } { labels = $2 @@ -391,6 +526,33 @@ planned=$(gh issue list -R "$repo" --state open --label planned --json number,ti | sort -t $'\t' -k1,1n -k2,2n \ | cut -f2-) +# --- current-milestone detection (issue #174) ------------------------------- +# Walk the version-sorted open milestones ascending; the FIRST one with at +# least one qualifying candidate (module-label hit, same predicate as the +# main loop below) becomes "current". Stays empty (fallback: unscoped, exactly +# today's behavior) when $milestones_sorted is empty (no milestones at all — +# byte-identical output) or when no open milestone has a qualifying candidate. +current_milestone="" +if [ -n "$milestones_sorted" ]; then + while IFS=$'\t' read -r ms_num ms_title ms_open ms_closed; do + [ -z "${ms_title:-}" ] && continue + qualifying=0 + while IFS=$'\t' read -r pnum plabels pmilestone ptitle; do + [ -z "${pnum:-}" ] && continue + [ "$pmilestone" = "$ms_title" ] || continue + hit=0 + while IFS= read -r ml; do + case ",$plabels," in *",$ml,"*) hit=1; break;; esac + done <<< "$module_labels" + [ "$hit" -eq 1 ] && qualifying=$((qualifying + 1)) + done <<< "$planned" + if [ "$qualifying" -ge 1 ]; then + current_milestone="$ms_title" + break + fi + done <<< "$milestones_sorted" +fi + planned_count=0 advance_ready="none" fallback_ready="none" @@ -401,8 +563,16 @@ blocked_lines="" plan_wait_lines="" advance_plan_state="" fallback_plan_state="" -while IFS=$'\t' read -r num labels title; do +while IFS=$'\t' read -r num labels milestone_title title; do [ -z "${num:-}" ] && continue + # Milestone scoping (issue #174): when a current milestone is in scope, + # the ADVANCE candidate set is filtered down to ONLY its issues — a plain + # skip here, applied on top of the (priority, number) ordering already + # established above, never disturbing it. A no-op when current_milestone + # is empty (fallback: unscoped, today's behavior). + if [ -n "$current_milestone" ] && [ "$milestone_title" != "$current_milestone" ]; then + continue + fi hit=0 while IFS= read -r ml; do case ",$labels," in *",$ml,"*) hit=1; break;; esac @@ -559,6 +729,16 @@ if [ "$advance_ready" = "none" ] && [ "$fallback_ready" != "none" ]; then fi echo "planned_issues=$planned_count" +# milestone=/milestone_open= (issue #174): only when a milestone is actually +# in scope — see the position rationale in MILESTONE SCOPING above. Absent +# entirely on the fallback path, preserving byte-identical output for repos +# that don't use milestones. milestone_open reuses $planned_count directly: +# once a milestone is in scope, planned_count IS that milestone's qualifying +# count (the filter above already scoped it), so no separate count is kept. +if [ -n "$current_milestone" ]; then + echo "milestone=$current_milestone" + echo "milestone_open=$planned_count" +fi [ -n "$detail" ] && printf '%s' "$detail" [ -n "$in_flight" ] && printf '%s' "$in_flight" [ -n "$stalled_lines" ] && printf '%s' "$stalled_lines" diff --git a/.claude/scripts/loop-census.test.sh b/.claude/scripts/loop-census.test.sh index a56e93c..9f85737 100644 --- a/.claude/scripts/loop-census.test.sh +++ b/.claude/scripts/loop-census.test.sh @@ -39,6 +39,7 @@ script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" census_src="$script_dir/loop-census.sh" resolve_roots_src="$script_dir/resolve-roots.sh" cockpit_src="$script_dir/cockpit.sh" +log_event_src="$script_dir/log-event.sh" work="$(mktemp -d "${TMPDIR:-/tmp}/loop-census-test.XXXXXX")" trap 'rm -rf "$work"' EXIT @@ -136,8 +137,10 @@ EOF # issues 43 and 100 already have one; issue 42 and 44 do not (issue 4 has # no branch, so it can't have a PR either). # - `pr list ... --json number ...`: open PR count (2, matching above). -# - `issue list ...`: TSV `num<TAB>labels<TAB>title` -# for the five planned+module:test issues. +# - `issue list ...`: TSV `num<TAB>labels<TAB>milestone<TAB>title` +# for the five planned+module:test issues (milestone field empty here — +# none of these issues are milestone-scoped; see issue #174's dedicated +# milestone-scoping fixtures further below). cat > "$scripts_dir/bot-gh.sh" <<'EOF' #!/usr/bin/env bash case "$1" in @@ -156,11 +159,11 @@ case "$1" in fi ;; issue) - printf '4\tplanned,module:test\tIssue four\n' - printf '42\tplanned,module:test\tIssue forty two\n' - printf '43\tplanned,module:test\tIssue forty three\n' - printf '44\tplanned,module:test\tIssue forty four\n' - printf '100\tplanned,module:test\tIssue one hundred\n' + printf '4\tplanned,module:test\t\tIssue four\n' + printf '42\tplanned,module:test\t\tIssue forty two\n' + printf '43\tplanned,module:test\t\tIssue forty three\n' + printf '44\tplanned,module:test\t\tIssue forty four\n' + printf '100\tplanned,module:test\t\tIssue one hundred\n' ;; *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; esac @@ -346,8 +349,8 @@ case "$1" in fi ;; issue) - printf '5\tplanned,module:test\tIssue five\n' - printf '6\tplanned,module:test\tIssue six\n' + printf '5\tplanned,module:test\t\tIssue five\n' + printf '6\tplanned,module:test\t\tIssue six\n' ;; *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; esac @@ -461,8 +464,8 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '20\tplanned,module:test\tCandidate twenty\n' - printf '21\tplanned,module:test\tCandidate twenty one\n' + printf '20\tplanned,module:test\t\tCandidate twenty\n' + printf '21\tplanned,module:test\t\tCandidate twenty one\n' else # all-open-issue-numbers fetch: 99 (the blocker) is still OPEN. printf '20\n21\n99\n' @@ -511,8 +514,8 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '20\tplanned,module:test\tCandidate twenty\n' - printf '21\tplanned,module:test\tCandidate twenty one\n' + printf '20\tplanned,module:test\t\tCandidate twenty\n' + printf '21\tplanned,module:test\t\tCandidate twenty one\n' else # all-open-issue-numbers fetch: 99 (the blocker) is now CLOSED — # absent from this list entirely. @@ -561,8 +564,8 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '30\tplanned,module:test\tCandidate thirty\n' - printf '31\tplanned,module:test\tCandidate thirty one\n' + printf '30\tplanned,module:test\t\tCandidate thirty\n' + printf '31\tplanned,module:test\t\tCandidate thirty one\n' else printf '30\n31\n' fi @@ -611,7 +614,7 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '40\tplanned,module:test\tTracker forty\n' + printf '40\tplanned,module:test\t\tTracker forty\n' else printf '40\n41\n' fi @@ -693,8 +696,8 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '3\tplanned,module:test\tLow-priority-in-number-order candidate\n' - printf '50\tplanned,module:test,priority:critical\tHigh-priority higher-numbered candidate\n' + printf '3\tplanned,module:test\t\tLow-priority-in-number-order candidate\n' + printf '50\tplanned,module:test,priority:critical\t\tHigh-priority higher-numbered candidate\n' else printf '3\n50\n' fi @@ -739,8 +742,8 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '5\tplanned,module:test\tUnlabeled lower-numbered candidate\n' - printf '6\tplanned,module:test,priority:low\tLabeled higher-numbered candidate\n' + printf '5\tplanned,module:test\t\tUnlabeled lower-numbered candidate\n' + printf '6\tplanned,module:test,priority:low\t\tLabeled higher-numbered candidate\n' else printf '5\n6\n' fi @@ -793,8 +796,8 @@ case "$1" in case "$2" in list) if printf '%s\n' "$*" | grep -q -- '--label'; then - printf '70\tplanned,module:test,priority:high\tBlocked high-priority candidate\n' - printf '8\tplanned,module:test\tUnblocked lower-priority candidate\n' + printf '70\tplanned,module:test,priority:high\t\tBlocked high-priority candidate\n' + printf '8\tplanned,module:test\t\tUnblocked lower-priority candidate\n' else # all-open-issue-numbers fetch: 99 (the blocker) is still OPEN. printf '70\n8\n99\n' @@ -886,10 +889,10 @@ case "$1" in fi ;; issue) - printf '80\tplanned,module:test\tStale issue eighty\n' - printf '81\tplanned,module:test\tFresh issue eighty one\n' - printf '82\tplanned,module:test\tNo-events issue eighty two\n' - printf '90\tplanned,module:test\tStale-but-has-PR issue ninety\n' + printf '80\tplanned,module:test\t\tStale issue eighty\n' + printf '81\tplanned,module:test\t\tFresh issue eighty one\n' + printf '82\tplanned,module:test\t\tNo-events issue eighty two\n' + printf '90\tplanned,module:test\t\tStale-but-has-PR issue ninety\n' ;; *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; esac @@ -979,6 +982,275 @@ check "fixture with HEAD detached: main_head=detached" bash -c 'printf "%s\n" "$ check "fixture with HEAD detached: main_dirty still no (working tree itself is clean)" bash -c 'printf "%s\n" "$1" | grep -qx "main_dirty=no"' _ "$out_detached" git -C "$fixture" checkout -q main +# --------------------------------------------------------------------------- +# Milestone scoping (issue #174): milestones represent versions (SCRUM +# sprints) -- the census's ADVANCE candidate set must scope to the CURRENT +# open milestone (lowest version-sorted open milestone with >=1 qualifying +# planned+module candidate), recomputed fresh every run, falling back to +# today's unscoped behavior when no open milestone qualifies (including the +# common case of a repo with no milestones at all, exercised implicitly by +# every fixture ABOVE this point in the file, none of which stub the `api` +# subcommand -- their bot-gh.sh falls through to the unhandled-args case, +# which degrades gracefully to an empty $milestones_tsv). +# +# build_milestone_fixture: same no-branch/no-open-PR shape as +# build_plain_fixture above, but also copies in log-event.sh (needed by the +# milestone-complete-event scenario) since these fixtures' own bot-gh.sh +# additionally answers the `api .../milestones?state=open` REST call. +# --------------------------------------------------------------------------- +build_milestone_fixture() { + local dir="$1" + local scripts="$dir/.claude/scripts" + mkdir -p "$scripts" + cp "$census_src" "$scripts/loop-census.sh" + cp "$resolve_roots_src" "$scripts/resolve-roots.sh" + cp "$log_event_src" "$scripts/log-event.sh" + cat > "$dir/.claude/gates.json" <<'EOF' +{ + "modules": [{ "name": "test", "path": ".", "description": "", "owner": "" }], + "merge": { "baseBranch": "main" } +} +EOF + for stub in pr-feedback pr-ci-fix pr-comment-fix pr-rebase; do + cat > "$scripts/$stub.sh" <<'EOF' +#!/usr/bin/env bash +exit 0 +EOF + done + chmod +x "$scripts"/*.sh + git -C "$dir" init -q -b main + git -C "$dir" -c user.email=t@e.st -c user.name=t commit -q --allow-empty -m init +} + +# --- SCOPING: two open milestones, both with a qualifying candidate -- only +# the LOWER-versioned milestone's issue may advance. Issue 5 (LOWER number) +# is deliberately assigned to the HIGHER-version milestone (v2.0), and issue +# 10 (HIGHER number) to the LOWER-version milestone (v1.0): a scoping-unaware +# revert (plain priority/number order, ignoring milestone) would pick issue 5 +# first, so this genuinely discriminates. --- +dirScoping="$work/milestone-scoping" +build_milestone_fixture "$dirScoping" +cat > "$dirScoping/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.0\t1\t0\n' + printf '2\tv2.0\t1\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) + if printf '%s\n' "$*" | grep -q -- '--label'; then + printf '5\tplanned,module:test\tv2.0\tHigher-milestone lower-numbered candidate\n' + printf '10\tplanned,module:test\tv1.0\tLower-milestone higher-numbered candidate\n' + else + printf '5\n10\n' + fi + ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirScoping/.claude/scripts/bot-gh.sh" +outScoping="$(env -u GATES_FILE bash "$dirScoping/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(scoping) current milestone is v1.0 (lowest version-sorted open milestone with a qualifying candidate)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "milestone=v1.0"' _ "$outScoping" +check "(scoping) milestone_open=1 (only v1.0's own candidate counted)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "milestone_open=1"' _ "$outScoping" +check "(scoping) advance_ready is issue 10 (v1.0-scoped), not issue 5 (lower-numbered but v2.0)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=10"' _ "$outScoping" +check "(scoping) issue 5 (out-of-scope v2.0 candidate) is never even detailed/counted" bash -c \ + '! printf "%s\n" "$1" | grep -q "^issue=5 "' _ "$outScoping" +check "(scoping) planned_issues=1 (scoped count, not the repo-wide 2)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "planned_issues=1"' _ "$outScoping" + +# --- FALLBACK: an open milestone exists, but NO candidate targets it -- must +# fall back to today's unscoped behavior (no milestone= line, the unassigned +# candidate still advances). --- +dirFallback="$work/milestone-fallback" +build_milestone_fixture "$dirFallback" +cat > "$dirFallback/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.0\t3\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) + if printf '%s\n' "$*" | grep -q -- '--label'; then + printf '20\tplanned,module:test\t\tUnassigned candidate\n' + else + printf '20\n' + fi + ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirFallback/.claude/scripts/bot-gh.sh" +outFallback="$(env -u GATES_FILE bash "$dirFallback/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(fallback) no milestone= line when no open milestone has a qualifying candidate" bash -c \ + '! printf "%s\n" "$1" | grep -q "^milestone="' _ "$outFallback" +check "(fallback) advance_ready falls back to the unscoped candidate (today's behavior)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=20"' _ "$outFallback" +check "(fallback) planned_issues=1 counted (unscoped)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "planned_issues=1"' _ "$outFallback" + +# --- RECOMPUTE-ON-DRAIN (a): v1.0 is fully drained (no qualifying candidate +# left) while v2.0 DOES have one -- v2.0 becomes current automatically, no +# persistent cursor needed. --- +dirDrainA="$work/milestone-drain-recompute" +build_milestone_fixture "$dirDrainA" +cat > "$dirDrainA/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.0\t0\t2\n' + printf '2\tv2.0\t1\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) + if printf '%s\n' "$*" | grep -q -- '--label'; then + printf '30\tplanned,module:test\tv2.0\tNext-milestone candidate\n' + else + printf '30\n' + fi + ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirDrainA/.claude/scripts/bot-gh.sh" +outDrainA="$(env -u GATES_FILE bash "$dirDrainA/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(recompute-on-drain) v1.0 drained -> v2.0 becomes current automatically" bash -c \ + 'printf "%s\n" "$1" | grep -qx "milestone=v2.0"' _ "$outDrainA" +check "(recompute-on-drain) advance_ready advances v2.0's candidate (issue 30)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=30"' _ "$outDrainA" + +# --- RECOMPUTE-ON-DRAIN (b): v1.0 drained, v2.0 open but its issues are NOT +# YET labeled `planned` -- the loop must IDLE (advance_ready=none), never +# advance into the next milestone early. --- +dirDrainB="$work/milestone-drain-idle" +build_milestone_fixture "$dirDrainB" +cat > "$dirDrainB/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.0\t0\t2\n' + printf '2\tv2.0\t1\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + # v2.0's issue exists but isn't labeled `planned` yet -- the real + # `--label planned` GitHub-side filter this call uses would exclude it + # regardless of --label being present, so the planned TSV is empty. + list) : ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirDrainB/.claude/scripts/bot-gh.sh" +outDrainB="$(env -u GATES_FILE bash "$dirDrainB/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(recompute-on-drain, idle) no qualifying milestone -- no milestone= line" bash -c \ + '! printf "%s\n" "$1" | grep -q "^milestone="' _ "$outDrainB" +check "(recompute-on-drain, idle) loop idles rather than advancing into the unlabeled milestone" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=none"' _ "$outDrainB" +check "(recompute-on-drain, idle) planned_issues=0" bash -c \ + 'printf "%s\n" "$1" | grep -qx "planned_issues=0"' _ "$outDrainB" + +# --- MILESTONE-COMPLETE event (issue #174): a fully-drained milestone +# (open_issues=0, closed_issues>=1) logs ONE milestone-complete event via +# log-event.sh -- and re-running census repeatedly must NOT append a +# duplicate, preserving the read-only/re-run-safe contract for every other +# line this script prints (this write is the single deliberate exception). --- +dirComplete="$work/milestone-complete" +build_milestone_fixture "$dirComplete" +cat > "$dirComplete/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.0\t0\t2\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) : ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirComplete/.claude/scripts/bot-gh.sh" +eventsComplete="$work/milestone-complete-events.jsonl" +: > "$eventsComplete" +env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsComplete" bash "$dirComplete/.claude/scripts/loop-census.sh" "acme/repo" >/dev/null +env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsComplete" bash "$dirComplete/.claude/scripts/loop-census.sh" "acme/repo" >/dev/null +env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsComplete" bash "$dirComplete/.claude/scripts/loop-census.sh" "acme/repo" >/dev/null + +check "(milestone-complete) event logged for the drained milestone" bash -c \ + 'grep -q "\"phase\":\"milestone-complete\"" "$1" && grep -q "\"task\":\"v1.0\"" "$1"' _ "$eventsComplete" +check "(milestone-complete) idempotent -- exactly one event after three census runs" bash -c \ + '[ "$(grep -c "\"phase\":\"milestone-complete\"" "$1")" -eq 1 ]' _ "$eventsComplete" + echo "" if [ "$fail" -eq 0 ]; then echo "loop-census.test.sh: PASS ($ok checks)" From 3d4d6dba87b20896889065379d84c4f608f6186f Mon Sep 17 00:00:00 2001 From: Roberto Cano <3525807+robercano@users.noreply.github.com> Date: Fri, 24 Jul 2026 21:18:14 +0200 Subject: [PATCH 2/4] test(loop): add milestone-scoping edge-case coverage (issue #174) Tests reviewer follow-up: sort -V regression lock (v1.9/v1.10 lexical vs version divergence), never-populated milestone-complete guard negative, un-drained-milestone negative, and a no-leak assertion for the milestone-less path. loop-census.sh source is unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --- .claude/scripts/loop-census.test.sh | 165 ++++++++++++++++++++++++++++ 1 file changed, 165 insertions(+) diff --git a/.claude/scripts/loop-census.test.sh b/.claude/scripts/loop-census.test.sh index 9f85737..538d4ae 100644 --- a/.claude/scripts/loop-census.test.sh +++ b/.claude/scripts/loop-census.test.sh @@ -1251,6 +1251,171 @@ check "(milestone-complete) event logged for the drained milestone" bash -c \ check "(milestone-complete) idempotent -- exactly one event after three census runs" bash -c \ '[ "$(grep -c "\"phase\":\"milestone-complete\"" "$1")" -eq 1 ]' _ "$eventsComplete" +# --------------------------------------------------------------------------- +# sort -V regression lock (issue #174 follow-up): two open milestones titled +# "v1.9" and "v1.10", where LEXICAL and VERSION order genuinely DIVERGE +# (lexically "v1.10" < "v1.9" since '1' < '9' at the third character; only +# `sort -V`'s numeric-aware comparison puts v1.9 first). Each has its own +# qualifying planned+module candidate -- issue 15 -> v1.9, issue 16 -> v1.10, +# fed to the fake `api` stub in v1.10-first order so a correct `sort -V` +# inside loop-census.sh is what has to reorder them, not accidental input +# order. If the "V" modifier were ever dropped from the `sort -k2,2V` call, +# plain lexical sort would pick v1.10 first and this whole block would flip. +# --------------------------------------------------------------------------- +dirSortV="$work/milestone-sort-v" +build_milestone_fixture "$dirSortV" +cat > "$dirSortV/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.10\t1\t0\n' + printf '2\tv1.9\t1\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) + if printf '%s\n' "$*" | grep -q -- '--label'; then + printf '15\tplanned,module:test\tv1.9\tLower-version candidate\n' + printf '16\tplanned,module:test\tv1.10\tHigher-version candidate\n' + else + printf '15\n16\n' + fi + ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirSortV/.claude/scripts/bot-gh.sh" +outSortV="$(env -u GATES_FILE bash "$dirSortV/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(sort -V regression) milestone=v1.9 (lower VERSION wins over lexically-earlier-looking v1.10)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "milestone=v1.9"' _ "$outSortV" +check "(sort -V regression) advance_ready=15 (v1.9's candidate), not issue 16 (v1.10)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=15"' _ "$outSortV" +check "(sort -V regression) issue 16 (out-of-scope v1.10 candidate) is never detailed" bash -c \ + '! printf "%s\n" "$1" | grep -q "^issue=16 "' _ "$outSortV" + +# --------------------------------------------------------------------------- +# never-populated guard negative test (issue #174 follow-up): a milestone +# REST stub reports open_issues==0 AND closed_issues==0 -- brand-new, never +# populated with any issues at all -- with no candidates targeting it. The +# milestone-complete guard requires closed_issues>=1 (genuinely drained) +# alongside open_issues==0, so this must log ZERO milestone-complete events; +# a guard that fired on open==0 alone (ignoring closed>=1) would wrongly +# treat "never populated" as "complete". +# --------------------------------------------------------------------------- +dirNeverPop="$work/milestone-never-populated" +build_milestone_fixture "$dirNeverPop" +cat > "$dirNeverPop/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv3.0\t0\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) : ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirNeverPop/.claude/scripts/bot-gh.sh" +eventsNeverPop="$work/milestone-never-populated-events.jsonl" +: > "$eventsNeverPop" +env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsNeverPop" bash "$dirNeverPop/.claude/scripts/loop-census.sh" "acme/repo" >/dev/null +env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsNeverPop" bash "$dirNeverPop/.claude/scripts/loop-census.sh" "acme/repo" >/dev/null + +check "(never-populated guard) never-populated milestone (open=0, closed=0) logs zero milestone-complete events" bash -c \ + '[ "$(grep -c "\"phase\":\"milestone-complete\"" "$1")" -eq 0 ]' _ "$eventsNeverPop" + +# --------------------------------------------------------------------------- +# un-drained milestone negative test (issue #174 follow-up): an OPEN +# milestone (open_issues>0) genuinely in scope -- it has its own qualifying +# planned+module candidate, so milestone= is non-vacuous (this isn't just an +# empty/fallback state) -- yet still un-drained. The milestone-complete guard +# must not fire: zero events logged while the milestone is still in play. +# --------------------------------------------------------------------------- +dirUndrained="$work/milestone-undrained" +build_milestone_fixture "$dirUndrained" +cat > "$dirUndrained/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tv1.0\t2\t1\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) + if printf '%s\n' "$*" | grep -q -- '--label'; then + printf '60\tplanned,module:test\tv1.0\tIn-play candidate\n' + else + printf '60\n' + fi + ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirUndrained/.claude/scripts/bot-gh.sh" +eventsUndrained="$work/milestone-undrained-events.jsonl" +: > "$eventsUndrained" +outUndrained="$(env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsUndrained" bash "$dirUndrained/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(un-drained) milestone=v1.0 genuinely in scope (non-vacuous -- has its own qualifying candidate)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "milestone=v1.0"' _ "$outUndrained" +check "(un-drained) advance_ready=60 (the in-scope milestone's qualifying candidate)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=60"' _ "$outUndrained" +check "(un-drained) zero milestone-complete events logged (milestone still open, not genuinely drained)" bash -c \ + '[ "$(grep -c "\"phase\":\"milestone-complete\"" "$1")" -eq 0 ]' _ "$eventsUndrained" + +# --------------------------------------------------------------------------- +# no-leak assertion for the truly-milestone-less path (issue #174 follow-up): +# reuses fixture1's ALREADY-captured $out (its bot-gh.sh has no `api` case at +# all -- an unhandled `gh api ...` call falls into the catch-all `exit 1`, +# exactly the "no api stub" shape this check calls for). The existing +# `grep -qx` positive checks above only assert specific lines are PRESENT; +# they would not catch an EXTRA leaked `milestone=`/`milestone_open=` line +# sitting elsewhere in the same output, so this asserts their absence +# directly. +# --------------------------------------------------------------------------- +check "(no-leak) pre-milestone fixture (no api stub) never leaks a milestone= line" bash -c \ + '! printf "%s\n" "$1" | grep -q "^milestone="' _ "$out" +check "(no-leak) pre-milestone fixture (no api stub) never leaks a milestone_open= line" bash -c \ + '! printf "%s\n" "$1" | grep -q "^milestone_open="' _ "$out" + echo "" if [ "$fail" -eq 0 ]; then echo "loop-census.test.sh: PASS ($ok checks)" From 80cb48dac7c6356b419f068e0780938465e5fa19 Mon Sep 17 00:00:00 2001 From: Roberto Cano <3525807+robercano@users.noreply.github.com> Date: Fri, 24 Jul 2026 22:42:51 +0200 Subject: [PATCH 3/4] fix(loop): milestone-complete idempotency must survive events.jsonl rotation (issue #174) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reviewer-found bug: the milestone-complete guard scanned events.jsonl for a prior marker line, but log-event.sh caps that file to the last 2000 lines — over the multi-day idle gap a drained milestone sits in, the marker eventually rotates out and census re-logs a duplicate on the next tick. Track completion instead in a dedicated, never-rotated sidecar ledger (.claude/state/milestone-complete-logged.json, keyed by milestone number), serialized with flock against concurrent census/cockpit invocations. Also adds --paginate to the milestones REST fetch and test coverage for rotation survival and space-containing milestone titles. --- .claude/scripts/loop-census.sh | 92 ++++++++++++++++++----------- .claude/scripts/loop-census.test.sh | 68 +++++++++++++++++++++ 2 files changed, 125 insertions(+), 35 deletions(-) diff --git a/.claude/scripts/loop-census.sh b/.claude/scripts/loop-census.sh index aef3dbf..0935fd5 100644 --- a/.claude/scripts/loop-census.sh +++ b/.claude/scripts/loop-census.sh @@ -179,12 +179,20 @@ # >= 1, i.e. genuinely drained rather than never-populated) without any extra # gh call. This is the ONE deliberate exception to this script's read-only / # re-run-safe contract (see the file-level comment below): it appends to -# events.jsonl via log-event.sh. To stay idempotent under re-runs (no -# duplicate event every idle tick), it FIRST scans events.jsonl for an -# existing milestone-complete event carrying that milestone's title, and -# only logs when none is found — approach (b) from the issue's acceptance -# criteria, chosen because it needs no change to loop-tick.sh's control flow -# and keeps the guard co-located with the detection logic that needs it. +# events.jsonl via log-event.sh. A drained milestone typically stays +# state=open for days (the idle gap after it drains IS the PO-feedback +# phase), so census re-observes the SAME drained milestone on every tick +# through that whole window — idempotency has to survive that. events.jsonl +# itself is NOT a valid ledger for that check: log-event.sh caps it to the +# last EVENTS_MAX_LINES (default 2000) lines, oldest-first, so a scan-based +# "is it already in events.jsonl" guard eventually loses its own marker line +# to rotation and re-logs a duplicate. Idempotency is instead tracked in a +# small, NEVER-rotated sidecar file (`.claude/state/milestone-complete- +# logged.json`, keyed by milestone NUMBER — stable, unlike a title an owner +# could edit), with the whole check-then-log critical section serialized by +# a real `flock` (same TOCTOU class loop-tick.sh's advance lock closes, +# issue #81) so two overlapping census/cockpit invocations can't both +# observe "not yet logged" and both append. # # --- BLOCKING-GRAPH GATE (issue #97) ---------------------------------------- # advance_ready additionally skips any otherwise-eligible candidate (branch= @@ -256,8 +264,9 @@ # Invoke as `bash .claude/scripts/loop-census.sh` (pre-approve that exact # command). Read-only: advances no cursor, mutates nothing — safe to re-run. # The SOLE exception is the milestone-complete event append (issue #174, see -# MILESTONE SCOPING above) — an idempotent, guarded events.jsonl write, never -# a stdout/behavior change; every other line above stays a pure read. +# MILESTONE SCOPING above) — an events.jsonl write guarded to fire once per +# milestone by a separate, never-rotated sidecar ledger, never a stdout/ +# behavior change; every other line above stays a pure read. set -euo pipefail # Two-root derivation (issue #63): script_dir = sibling scripts, root = consumer project. @@ -354,7 +363,7 @@ events_file="${CLAUDE_EVENTS_FILE:-$root/.claude/state/events.jsonl}" # SCOPING above): census must never abort, and an empty result here makes # every downstream milestone check a no-op, falling back to today's unscoped # behavior. -milestones_tsv=$(gh api "repos/$repo/milestones?state=open" \ +milestones_tsv=$(gh api --paginate "repos/$repo/milestones?state=open" \ --jq '.[] | [.number, .title, .open_issues, .closed_issues] | @tsv' 2>/dev/null) || true # Version-sort ascending by title (2nd TSV field) — natural/`sort -V` @@ -370,38 +379,51 @@ fi # A milestone is "complete" when the REST fetch's own open_issues/ # closed_issues counters show it fully drained (open_issues==0) AND it # genuinely had issues to drain (closed_issues>=1 — never fires for an empty/ -# never-populated milestone). Guarded against duplicate logging by scanning -# events_file FIRST for an existing milestone-complete event carrying this -# milestone's title before appending — safe to re-run every tick, exactly -# like every other check in this script, EXCEPT this one intentionally -# mutates events.jsonl (via log-event.sh) as its side effect. +# never-populated milestone). Idempotency is tracked in a dedicated, +# never-rotated sidecar file (NOT events.jsonl — see the MILESTONE-COMPLETE +# EVENT note above for why a rotation-subject log can't be the ledger), +# keyed by milestone number, with the check-then-log critical section +# serialized by flock against concurrent census/cockpit invocations. +milestone_state_file="${CLAUDE_MILESTONE_STATE_FILE:-$root/.claude/state/milestone-complete-logged.json}" +milestone_lock_file="${CLAUDE_MILESTONE_LOCK_FILE:-$root/.claude/state/milestone-complete.flock}" if [ -n "$milestones_tsv" ]; then + mkdir -p "$(dirname "$milestone_state_file")" 2>/dev/null || true while IFS=$'\t' read -r ms_num ms_title ms_open ms_closed; do [ -z "${ms_title:-}" ] && continue + case "$ms_num" in ''|*[!0-9]*) continue ;; esac case "$ms_open" in ''|*[!0-9]*) continue ;; esac case "$ms_closed" in ''|*[!0-9]*) continue ;; esac if [ "$ms_open" -eq 0 ] && [ "$ms_closed" -ge 1 ]; then - already_logged=$(CLAUDE_MS_EVENTS_FILE="$events_file" CLAUDE_MS_TITLE="$ms_title" node -e ' - const fs = require("fs"); - const file = process.env.CLAUDE_MS_EVENTS_FILE; - const title = process.env.CLAUDE_MS_TITLE; - let found = false; - try { - const text = fs.readFileSync(file, "utf8"); - for (const line of text.split("\n")) { - if (!line.trim()) continue; - let o; - try { o = JSON.parse(line); } catch (e) { continue; } - if (o.phase === "milestone-complete" && o.task === title) { found = true; break; } - } - } catch (e) { /* no file yet -> not logged */ } - console.log(found ? "yes" : "no"); - ' 2>/dev/null) || already_logged="no" - if [ "$already_logged" != "yes" ]; then - CLAUDE_EVENTS_FILE="$events_file" bash "$script_dir/log-event.sh" \ - --role census --task "$ms_title" --phase milestone-complete \ - --detail "milestone drained (all issues closed)" >/dev/null 2>&1 || true - fi + ( + exec 8>"$milestone_lock_file" + flock -x 8 + already_logged=$(CLAUDE_MS_STATE_FILE="$milestone_state_file" CLAUDE_MS_NUM="$ms_num" node -e ' + const fs = require("fs"); + const file = process.env.CLAUDE_MS_STATE_FILE; + const num = process.env.CLAUDE_MS_NUM; + let logged = []; + try { logged = JSON.parse(fs.readFileSync(file, "utf8")); } catch (e) { logged = []; } + if (!Array.isArray(logged)) logged = []; + console.log(logged.includes(num) ? "yes" : "no"); + ' 2>/dev/null) || already_logged="no" + if [ "$already_logged" != "yes" ]; then + CLAUDE_EVENTS_FILE="$events_file" bash "$script_dir/log-event.sh" \ + --role census --task "$ms_title" --phase milestone-complete \ + --detail "milestone drained (all issues closed)" >/dev/null 2>&1 || true + CLAUDE_MS_STATE_FILE="$milestone_state_file" CLAUDE_MS_NUM="$ms_num" node -e ' + const fs = require("fs"); + const file = process.env.CLAUDE_MS_STATE_FILE; + const num = process.env.CLAUDE_MS_NUM; + let logged = []; + try { logged = JSON.parse(fs.readFileSync(file, "utf8")); } catch (e) { logged = []; } + if (!Array.isArray(logged)) logged = []; + if (!logged.includes(num)) logged.push(num); + const tmp = file + ".tmp." + process.pid; + fs.writeFileSync(tmp, JSON.stringify(logged)); + fs.renameSync(tmp, file); + ' 2>/dev/null || true + fi + ) fi done <<< "$milestones_tsv" fi diff --git a/.claude/scripts/loop-census.test.sh b/.claude/scripts/loop-census.test.sh index 538d4ae..7255e67 100644 --- a/.claude/scripts/loop-census.test.sh +++ b/.claude/scripts/loop-census.test.sh @@ -1251,6 +1251,23 @@ check "(milestone-complete) event logged for the drained milestone" bash -c \ check "(milestone-complete) idempotent -- exactly one event after three census runs" bash -c \ '[ "$(grep -c "\"phase\":\"milestone-complete\"" "$1")" -eq 1 ]' _ "$eventsComplete" +# --- MILESTONE-COMPLETE ROTATION SURVIVAL (issue #174 review fix): a scan-of- +# events.jsonl idempotency guard breaks the moment log-event.sh's own +# retention cap (EVENTS_MAX_LINES, default 2000) rotates the marker line back +# out of the log -- inevitable over the multi-day idle gap this feature is +# built around. Simulate that by wiping the marker straight out of +# events.jsonl (strictly worse than real rotation, which only evicts it +# eventually) and confirm a 4th census run still does NOT re-log: idempotency +# must be tracked in the separate, never-rotated +# .claude/state/milestone-complete-logged.json sidecar, not events.jsonl. --- +: > "$eventsComplete" +env -u GATES_FILE CLAUDE_EVENTS_FILE="$eventsComplete" bash "$dirComplete/.claude/scripts/loop-census.sh" "acme/repo" >/dev/null + +check "(milestone-complete, rotation survival) no duplicate event once the events.jsonl marker line is gone (simulated rotation)" bash -c \ + '[ "$(grep -c "\"phase\":\"milestone-complete\"" "$1")" -eq 0 ]' _ "$eventsComplete" +check "(milestone-complete, rotation survival) sidecar ledger still remembers milestone 1 as already logged" bash -c \ + 'grep -q "1" "$1"' _ "$dirComplete/.claude/state/milestone-complete-logged.json" + # --------------------------------------------------------------------------- # sort -V regression lock (issue #174 follow-up): two open milestones titled # "v1.9" and "v1.10", where LEXICAL and VERSION order genuinely DIVERGE @@ -1306,6 +1323,57 @@ check "(sort -V regression) advance_ready=15 (v1.9's candidate), not issue 16 (v check "(sort -V regression) issue 16 (out-of-scope v1.10 candidate) is never detailed" bash -c \ '! printf "%s\n" "$1" | grep -q "^issue=16 "' _ "$outSortV" +# --------------------------------------------------------------------------- +# space-in-title coverage (issue #174 follow-up): milestone titles are +# free-form ("Sprint 1", not just "vX.Y" strings) -- the header comment +# explicitly cites "Sprint 1" < "Sprint 2" as supported. Confirm a spaced +# title round-trips correctly through both TSVs (jq's @tsv escapes embedded +# whitespace-adjacent characters safely; a title is still one field) and that +# exact-match milestone scoping works unchanged. --- +dirSpaceTitle="$work/milestone-space-title" +build_milestone_fixture "$dirSpaceTitle" +cat > "$dirSpaceTitle/.claude/scripts/bot-gh.sh" <<'EOF' +#!/usr/bin/env bash +case "$1" in + repo) echo "acme/repo" ;; + api) + printf '1\tSprint 1\t1\t0\n' + printf '2\tSprint 2\t1\t0\n' + ;; + pr) + if printf '%s\n' "$*" | grep -q 'headRefName'; then + : # no open PRs + else + echo 0 + fi + ;; + issue) + case "$2" in + list) + if printf '%s\n' "$*" | grep -q -- '--label'; then + printf '25\tplanned,module:test\tSprint 1\tSprint-1 candidate\n' + printf '26\tplanned,module:test\tSprint 2\tSprint-2 candidate\n' + else + printf '25\n26\n' + fi + ;; + view) echo '{"body":""}' ;; + *) echo "unhandled issue subcmd: $*" >&2; exit 1 ;; + esac + ;; + *) echo "fake-bot-gh.sh: unhandled args: $*" >&2; exit 1 ;; +esac +EOF +chmod +x "$dirSpaceTitle/.claude/scripts/bot-gh.sh" +outSpaceTitle="$(env -u GATES_FILE bash "$dirSpaceTitle/.claude/scripts/loop-census.sh" "acme/repo")" + +check "(space-in-title) milestone=Sprint 1 (spaced title survives both TSV round-trips intact)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "milestone=Sprint 1"' _ "$outSpaceTitle" +check "(space-in-title) advance_ready=25 (Sprint 1's candidate), not issue 26 (Sprint 2)" bash -c \ + 'printf "%s\n" "$1" | grep -qx "advance_ready=25"' _ "$outSpaceTitle" +check "(space-in-title) issue 26 (out-of-scope Sprint 2 candidate) is never detailed" bash -c \ + '! printf "%s\n" "$1" | grep -q "^issue=26 "' _ "$outSpaceTitle" + # --------------------------------------------------------------------------- # never-populated guard negative test (issue #174 follow-up): a milestone # REST stub reports open_issues==0 AND closed_issues==0 -- brand-new, never From afd9d2dccca31b8755f6d4b6d7ed53dc842739ce Mon Sep 17 00:00:00 2001 From: Roberto Cano <3525807+robercano@users.noreply.github.com> Date: Fri, 24 Jul 2026 22:50:32 +0200 Subject: [PATCH 4/4] fix(loop): milestone-complete lock acquisition must never abort census (issue #174) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reviewer-found regression: the flock guard added for the rotation fix opened and acquired its lock fd with no failure handling under this script's own set -euo pipefail, so a permission/missing-dir/disk-full failure on the lock file killed the entire census run before it printed advance_ready=/ planned_issues=/etc. — the exact invariant this script promises to never break. Guard both the exec/flock lines and the subshell invocation so a lock failure degrades to "skip logging this tick" instead of aborting. --- .claude/scripts/loop-census.sh | 14 ++++++++++---- .claude/scripts/loop-census.test.sh | 16 ++++++++++++++++ 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/.claude/scripts/loop-census.sh b/.claude/scripts/loop-census.sh index 0935fd5..e3946ec 100644 --- a/.claude/scripts/loop-census.sh +++ b/.claude/scripts/loop-census.sh @@ -383,7 +383,13 @@ fi # never-rotated sidecar file (NOT events.jsonl — see the MILESTONE-COMPLETE # EVENT note above for why a rotation-subject log can't be the ledger), # keyed by milestone number, with the check-then-log critical section -# serialized by flock against concurrent census/cockpit invocations. +# serialized by flock against concurrent census/cockpit invocations. The +# whole thing runs under this script's own `set -euo pipefail`, so lock +# acquisition (`exec 8>…`/`flock -x 8`) is explicitly guarded (`|| exit 0`) +# and the enclosing subshell invocation ends in `|| true`: a lock/fs failure +# (permission denied, missing dir, disk full) must degrade to "skip logging +# this tick" — never abort census before it prints advance_ready=/ +# planned_issues=/etc., which loop-tick.sh depends on for every run. milestone_state_file="${CLAUDE_MILESTONE_STATE_FILE:-$root/.claude/state/milestone-complete-logged.json}" milestone_lock_file="${CLAUDE_MILESTONE_LOCK_FILE:-$root/.claude/state/milestone-complete.flock}" if [ -n "$milestones_tsv" ]; then @@ -395,8 +401,8 @@ if [ -n "$milestones_tsv" ]; then case "$ms_closed" in ''|*[!0-9]*) continue ;; esac if [ "$ms_open" -eq 0 ] && [ "$ms_closed" -ge 1 ]; then ( - exec 8>"$milestone_lock_file" - flock -x 8 + exec 8>"$milestone_lock_file" 2>/dev/null || exit 0 + flock -x 8 2>/dev/null || exit 0 already_logged=$(CLAUDE_MS_STATE_FILE="$milestone_state_file" CLAUDE_MS_NUM="$ms_num" node -e ' const fs = require("fs"); const file = process.env.CLAUDE_MS_STATE_FILE; @@ -423,7 +429,7 @@ if [ -n "$milestones_tsv" ]; then fs.renameSync(tmp, file); ' 2>/dev/null || true fi - ) + ) || true fi done <<< "$milestones_tsv" fi diff --git a/.claude/scripts/loop-census.test.sh b/.claude/scripts/loop-census.test.sh index 7255e67..bd0a920 100644 --- a/.claude/scripts/loop-census.test.sh +++ b/.claude/scripts/loop-census.test.sh @@ -1268,6 +1268,22 @@ check "(milestone-complete, rotation survival) no duplicate event once the event check "(milestone-complete, rotation survival) sidecar ledger still remembers milestone 1 as already logged" bash -c \ 'grep -q "1" "$1"' _ "$dirComplete/.claude/state/milestone-complete-logged.json" +# --- MILESTONE-COMPLETE LOCK-FAILURE NEVER ABORTS CENSUS (issue #174 review +# fix): a prior version of the idempotency fix opened the flock fd +# (`exec 8>…`) and acquired it (`flock -x 8`) with no failure guard, under +# this script's own `set -euo pipefail` -- if the lock file can't be opened +# (unwritable state dir, missing parent, disk full) the subshell aborted and +# that nonzero exit propagated to census itself, killing the WHOLE run +# before it printed advance_ready=/planned_issues=/etc. Point the lock at an +# unwritable path and confirm census still completes normally instead of +# dying. --- +outLockFail="$(env -u GATES_FILE CLAUDE_MILESTONE_LOCK_FILE="/nonexistent-dir-$$/lock.flock" \ + bash "$dirComplete/.claude/scripts/loop-census.sh" "acme/repo")" +check "(milestone-complete, lock failure) census still completes (advance_ready= line present) when the lock file can't be opened" bash -c \ + 'printf "%s\n" "$1" | grep -q "^advance_ready="' _ "$outLockFail" +check "(milestone-complete, lock failure) census still completes (planned_issues= line present) when the lock file can't be opened" bash -c \ + 'printf "%s\n" "$1" | grep -q "^planned_issues="' _ "$outLockFail" + # --------------------------------------------------------------------------- # sort -V regression lock (issue #174 follow-up): two open milestones titled # "v1.9" and "v1.10", where LEXICAL and VERSION order genuinely DIVERGE