Skip to content

fix(system-update): match the npm install-script gate to measured risk - #60

Merged
verlyn13 merged 2 commits into
mainfrom
fix/npm-script-gate-risk-tiered
Aug 2, 2026
Merged

fix(system-update): match the npm install-script gate to measured risk#60
verlyn13 merged 2 commits into
mainfrom
fix/npm-script-gate-risk-tiered

Conversation

@verlyn13

@verlyn13 verlyn13 commented Aug 2, 2026

Copy link
Copy Markdown
Member

Purpose

The install-script gate required an exact version on every --allow-scripts grant. A walk of the live global tree shows what that was policing, and it is not much:

  • 5 packages / 7 versions out of 891 carry an install script at all — 0.79%
  • zero of the 12 top-level globals declares one in its own manifest
  • zero of those scripts fetches from a non-registry host
  • @openai/codex has none anywhere in its tree, so strict mode on that lane has zero enforcement targets and can only ever exit 0

Meanwhile the exact-version grain generated the whole maintenance cost: over 2026-07 the gate named 7 dependency identities but 15 version tuples, and workerd alone moved 8 times in 33 days. None of those edits was a review.

What changed

Grain → dependency identity. wrangler=esbuild,fsevents,workerd, owner-scoped as before. A new script-bearing identity still stops the install. SYSTEM_UPDATE_NPM_ALLOW_SCRIPTS_REQUIRE_PIN=true restores the old grain. Semver ranges, dist-tags and globs are now rejected by name — npm discards them silently, which fails open on a non-strict npm. Globs matter most: the sibling knob min-release-age-exclude is minimatch, allow-scripts is not.

Per-lane release-age cooldown. 2 days on stable globals, 0 on the AI channels. Across seven documented npm hijacks the dwell-time distribution is bimodal — five inside 24h, then 8–10 days and 72 days — so 1, 2 and 7 all catch the same five; 2 buys margin against detection-to-removal lag. The AI lane is 0 because a cooldown on a dist-tag does not delay a channel, it moves it: verified, codex@alpha resolves to stable 0.146.0 and copilot@prerelease to 1.0.77, exit 0, no warning. --min-release-age-exclude is never emitted and a test enforces it — npm ≥ 12 only, and on 11.16 the unknown flag becomes a positional spec and kills the step with E404.

Artifact verification where the allowlist never applied. ai_cli_platform_dep_ok existed but was wired only into --check; it now runs in update mode on a parent version change. The platform-binary drop is a plain tarball no script touches, and command health cannot see it — the loader falls back and --version answers from the JS wrapper.

The npm 12 timebomb this defuses

npm 12.0.2 has been latest since 2026-07-08 and inverts the default: npm 11 runs an uncovered install script with a notice, npm 12 blocks it and still exits 0. It also renamed the log prefix allow-scriptsinstall-scripts. The counters matched only the npm-11 wording and would have silently returned zero — a v12 host would have installed globals without their native binaries and reported a clean green row.

Both wordings now match, npm_scripts_blocked_at_exit_zero separates blocked-at-success from pending review, and both steps print npm-major:. The arrival lane is mise upgrade --yes against a floating node = "24" — no operator decision involved. Runway exists: node 26.5.1 still bundles npm 11.17.0. npm is deliberately neither pinned nor upgraded (the mise shim shadows ~/.npm-global/bin, corepack is gone in Node 26).

Two defects fixed in passing

  • The AI lane could never reach blocked in production. npm_script_review_is_sole_failure rejected any log containing ^ai-cli-channel-failed: — which is the gate's own marker. The existing assertion passed only because its fixture omitted the line production always emits. §16.1 documented behavior the code did not have.
  • grep | grep -qv misclassified large logs. Under pipefail the reader's short-circuit hands the writer SIGPIPE and the pipeline returns 141, read as "no second cause". Reproduced at ≥800 matched lines. Now a single awk pass.

Reporting contract

Every tolerated state is stated rather than rendered green — npm-script-identity-grant, npm-global-resolved-drift, ai-cli-cooldown-hazard, ai-cli-platform-drift. All are printed markers promoted by collect_notices, since step functions run in a subshell.

Validation

  • scripts/validate-repo.shPASS (shellcheck 0.11.0 pinned)
  • tests/system-update/run.sh153 passed, 0 failed (was 96 before this line of work)
  • Live --check run: clean on both lanes
  • Live update run: gemini moved 2026080120260802, so the version-change branch executed — platform check ran and passed, and the run disclosed 1 package(s) ran install scripts under an identity-scoped grant, so the script's version was not reviewed this run
  • A transient network failure during one run was correctly classified network — retryable, not a hard failure

Risk

Does not protect against runtime compromise — 43% of the same incident set, including the most recent (node-ipc, 2026-05-14), which explicitly bypassed lifecycle scripts. For a global CLI invoked daily, "runtime" means every invocation. Also misses long-dwell compromise (event-stream, ~72 days), and a 2-day cooldown delays this host's own npm audit fix by 2 days. All three are stated in §22.

The gate required an exact version on every allow-scripts grant. A walk of the
live global tree shows what that was policing: 5 packages / 7 versions out of
891 carry an install script at all (0.79%), none of the 12 top-level globals
declares one in its own manifest, none fetches from a non-registry host, and
@openai/codex has none anywhere in its tree — so strict mode on that lane has
zero enforcement targets and can only ever exit 0.

The exact-version grain generated the whole maintenance cost. Over 2026-07 the
gate named 7 dependency identities but 15 version tuples; workerd alone moved 8
times in 33 days. None of those edits was a review.

Grain is now dependency identity, owner-scoped as before; a new script-bearing
identity still stops the install. SYSTEM_UPDATE_NPM_ALLOW_SCRIPTS_REQUIRE_PIN
restores the old behaviour. Semver ranges, dist-tags and globs are rejected by
name — npm discards them silently, which fails open on a non-strict npm. Globs
matter most: min-release-age-exclude IS minimatch, allow-scripts is not.

Adds a per-lane release-age cooldown: 2 days on stable globals, 0 on the AI
channels. Across seven documented npm hijacks the dwell-time distribution is
bimodal (five inside 24h, then 8-10 days and 72 days), so 1, 2 and 7 days catch
the same five; 2 buys margin against detection-to-removal lag. The AI lane is 0
because a cooldown on a dist-tag does not delay a channel, it moves it —
verified, codex@alpha resolves to stable 0.146.0 and copilot@prerelease to
1.0.77, exit 0, no warning. --min-release-age-exclude is never emitted and a
test enforces that: it is npm>=12 only and on 11.16 the unknown flag becomes a
positional spec and kills the step with E404.

Wires ai_cli_platform_dep_ok into update mode on a parent version change. The
platform-binary drop is a plain tarball no script touches, so no allowlist could
catch it, and command health cannot see it — the loader falls back and
--version answers from the JS wrapper.

npm 12 readiness. npm 12.0.2 (latest since 2026-07-08) inverts the default:
npm 11 runs an uncovered script with a notice, npm 12 blocks it and still exits
0, and the log prefix changed allow-scripts -> install-scripts. The counters
matched only npm-11 wording and would have returned zero, so a v12 host would
have installed globals without native binaries and reported a clean green row.
Both wordings now match, npm_scripts_blocked_at_exit_zero separates
blocked-at-success from pending review, and both steps print npm-major.

Fixes two defects found while mapping:
- the AI lane could never reach `blocked`. The sole-failure predicate rejected
  any log with ^ai-cli-channel-failed:, which is the gate's OWN marker; the
  assertion passed only because its fixture omitted the line production always
  emits. Heal failure and retry exhaustion remain genuine second causes.
- `grep | grep -qv` returned 141 under pipefail once the reader short-circuited,
  reading as "no second cause". Reproduced at >=800 matched lines. Now one awk.

Every tolerated state is stated rather than rendered green:
npm-script-identity-grant, npm-global-resolved-drift, ai-cli-cooldown-hazard,
ai-cli-platform-drift — printed markers promoted by collect_notices, since step
functions run in a subshell.

Does not protect against runtime compromise (43% of the same incident set,
including node-ipc 2026-05-14, which bypassed lifecycle scripts).

153 assertions pass; scripts/validate-repo.sh green.
Adds a superseding section for the install-script policy stated across §11
(item 4), §12, §13, §15, §16.1, §17 and §20. Those sections stay as history;
the exact-version-pin invariant they assert is retired here with the measured
reasoning — 0.79% of the tree carries an install script, 15 version tuples for
7 identities over one month, and zero enforcement targets on the codex lane.

Records the npm 11 -> 12 default inversion and why npm is deliberately neither
pinned nor upgraded, the two defects fixed (AI lane could never reach `blocked`;
pipefail/SIGPIPE misclassification), the per-lane cooldown reasoning, and what
the design does not protect against.

Also corrects a stale cross-reference: the REQUIRE_PIN comment cited §21, which
is the cask receipt/artifact section.
@verlyn13
verlyn13 requested a review from a team as a code owner August 2, 2026 19:51
@verlyn13
verlyn13 merged commit bdec977 into main Aug 2, 2026
2 checks passed
@verlyn13
verlyn13 deleted the fix/npm-script-gate-risk-tiered branch August 2, 2026 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant