Skip to content

Harden three CI guards, found while porting this pipeline - #4

Open
Baasie wants to merge 4 commits into
mainfrom
harden-ci-guards
Open

Harden three CI guards, found while porting this pipeline#4
Baasie wants to merge 4 commits into
mainfrom
harden-ci-guards

Conversation

@Baasie

@Baasie Baasie commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

This pipeline was ported to weave-it.org over the last few days. Three things surfaced there that are latent here too. Each is one commit against one file, so they can be taken separately.

Workflow-only; no application code touched.

1. The content guard fails on any merge commit — a live bug

deploy.yml asks git log -1 --format='%an' "$SHA" and compares it to virtualddd-sync. That is "who changed the content" only when the sync's commit is the last one to land.

Merge a PR that carries a sync commit and the tip is the merge commit, authored by whoever pressed the button — so the deploy rejects files nobody touched. Verified against this repo's own history:

case current proposed
plain sync push (7b7370a) pass pass
merge commit carrying that sync commit fails — reads "Kenny Baas-Schwegler" pass
genuine hand edit (c6311f5) fails fails

It asks the range instead: who authored the commits between BEFORE and SHA that actually touched src/content. A merge commit touches nothing, so it never appears.

A squash still fails, and should — squashing rewrites the content's author into whoever opened the PR, leaving no evidence the sync produced it. The failure message now says so, since the next person to hit it will be holding a legitimate sync.

2. The sync has no guard for an unset NOTION_TOKEN

Every other secret here degrades to "skip and say so" — deploy.yml guards on [ -z "$HOST" ]. This one exits 1, hourly, on a schedule.

Nothing is wrong today because the secret is set. It matters when it's rotated, or a fork runs the workflow. On weave-it.org the hourly sync failed for seven hours with an unset token before anyone connected the red cross to the open task that caused it — a red run meaning "not configured" is indistinguishable from one meaning "broken".

3. node --test on an empty glob exits 0

Both globs match here, so this changes nothing today.

It's here because weave-it.org spent its entire migration with an empty tests/ directory: the step ran, went green on every push, and the brief went on saying that tier fails the deploy. Nobody was careless — the output said pass.

A green tick for "there was nothing to check" is worse than a red one, because it's indistinguishable from "everything passed".


Validated: all six workflows still parse as YAML, the three changed run: blocks pass bash -n, and the guard logic was replayed against real commits from this repository's history (above).

🤖 Generated with Claude Code

https://claude.ai/code/session_01QeS7uT29MpUrFiefwb4jzW

Baasie and others added 4 commits August 2, 2026 12:54
The guard reads `git log -1 --format='%an' "$SHA"` — the author of the tip
commit — and compares it to `virtualddd-sync`. That is the same thing as "who
changed the content" only when the sync's own commit happens to be the last one
to land.

Merge a pull request that carries synced content and it is not. The tip becomes
the merge commit, authored by whoever pressed the button, and the deploy fails
on content nobody touched. Verified against this repository's own history: with
a merge commit on top of 7b7370a, the current check reads "Kenny Baas-Schwegler"
and rejects files the sync wrote. A plain sync push still passes, and a genuine
hand edit (c6311f5) still fails — which is the case worth not breaking.

That is a false positive on the workflow whose whole job is to be trusted, and
the obvious workaround for it — push straight to main — is worse than the rule
being enforced.

Asks the range instead: who authored the commits between BEFORE and SHA that
actually touched `src/content`. A merge commit touches nothing, so it does not
appear. A squash still fails, and should: squashing rewrites the author of the
content into whoever opened the pull request, and there is then no evidence left
that the sync produced it. The failure message says so, because the next person
to hit it will be holding a legitimate sync.

Found while porting this pipeline to weave-it.org, where merging the first
branch that carried a sync commit would have hit it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every other secret in this repository degrades to "skip and say so" —
deploy.yml guards on `[ -z "$HOST" ]` and reports what it did. The sync does
not: an empty or rotated `NOTION_TOKEN` makes the script exit 1, on a schedule,
hourly, with nothing actually wrong.

Nothing is wrong here today, because the secret is set. It matters the day it is
rotated, or a fork runs the workflow, or somebody clears it while debugging —
and it matters because a red run meaning "not configured" is indistinguishable
from a red run meaning "broken", and a schedule full of both teaches everyone to
read neither.

Not hypothetical: on weave-it.org, which runs this same workflow, the hourly
sync failed for seven hours with an unset token before anyone connected the red
cross to the open task that caused it.

Guards the job rather than the step, because everything downstream — the diff,
the commit, the deploy it triggers — only means anything if something was
fetched. The "What changed?" step is guarded too, so a skipped run cannot report
"nothing changed in Notion", which is a different claim from "we never looked".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`node --test` on a glob that matches nothing prints "pass 0" and exits 0.

Both globs match here, so this changes nothing today. It is here because the
sibling repository this pipeline was ported to spent an entire migration with an
empty `tests/` directory: the step ran, went green on every push, and the brief
went on saying that tier fails the deploy. Nobody was careless — the output said
"pass".

A green tick for "there was nothing to check" is worse than a red one, because
it is indistinguishable from "everything passed".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three fixes from review, all inside the guards this branch adds.

The range form has a hole the tip form did not: when `github.event.before` is
newer than `github.sha`, which is what a force-push backwards looks like, the
diff still shows content moving while `BEFORE..SHA` is empty. An empty author
list then read as "nobody but the sync" and passed. A guard whose whole job is
attribution must not pass when it cannot attribute, so an empty list is now a
refusal. Replayed against this repository's history: a plain sync push still
passes, a range mixing sync commits with hand-written code still passes because
only the sync touched src/content, and the rollback case now fails where it
used to wave through.

`paste -sd', '` cycles its delimiter list rather than using it whole, so three
authors came out as "alice,bob carol". It is the line someone reads while they
are already confused about why their deploy failed.

And the escaped quotes in the sync.yml comment were characters, not quoting.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Baasie

Baasie commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Reviewed, and pushed three fixes to the branch (7023be9). All three are inside the guards this PR adds; nothing about the design changed.

The range form has a hole the tip form did not. When github.event.before is newer than github.sha — a force-push backwards — git diff BEFORE SHA still reports content moving while BEFORE..SHA is empty. The author list then came back empty, others was empty, and the guard passed. A guard whose whole job is attribution should not pass when it cannot attribute, so an empty list is now a refusal with its own message. Replayed against this repository's history:

case result
plain sync push (bb7c822) pass
range mixing sync commits with hand-written code (ff6ac6f~1..714dd6e) pass — only the sync touched src/content
rollback, range empty and diff not fail (was: silent pass)

paste -sd', ' cycles its delimiter list rather than using it whole, so three authors rendered as alice,bob carol. Now paste -sd, | sed 's/,/, /g'. Message-only, but it is the line someone reads while they are already confused.

The escaped quotes in the sync.yml comment were characters rather than quoting: \"nothing changed in Notion\".

Everything else stands as written. I checked what the token gate does downstream rather than assuming: every later step is gated on steps.diff.outputs.changed == 'true', which is empty when the diff step is skipped, and the deploy job needs the same output — so a skipped sync deploys nothing and claims nothing. compgen -G works under the bash -e shell Actions uses. All six workflows still parse and the changed run: blocks pass bash -n.

One thing worth a sentence in the file before this lands, because the next reader meets both comments in one sitting: changes 2 and 3 point in opposite directions on purpose. An empty test directory goes red because it is a promise this repo makes to itself; an unset secret goes green because it describes an environment that was never configured, such as a fork. That is defensible, but only if it is written down.

The squash caveat in the description does not apply to this PR itself — it touches no src/content, so squashing it is fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant