Skip to content

Allow code-owner approval to override breaking-change check#7469

Open
alfonso-noriega wants to merge 5 commits into
mainfrom
breaking-change-codeowner-override
Open

Allow code-owner approval to override breaking-change check#7469
alfonso-noriega wants to merge 5 commits into
mainfrom
breaking-change-codeowner-override

Conversation

@alfonso-noriega

@alfonso-noriega alfonso-noriega commented May 5, 2026

Copy link
Copy Markdown
Contributor

WHY are these changes introduced?

The breaking-change check fails hard (exit 1) whenever the script detects a removed Zod field, command, flag, or major changeset. There's currently no way to override it short of force-merging or admin-bypassing branch protection — and after PR #7468 in this stack, the previous (broken) "approval" path is gone too.

In practice, breaking changes are sometimes intentional: removing a deprecated field at a major-version boundary, restructuring a schema after a migration, etc. The team needs a way to acknowledge "yes, we know this is breaking, ship it" that's auditable and lightweight.

The simplest, most natural mechanism is the one already in place for everything else: a code-owner approval on the PR.

WHAT is this pull request doing?

Adds a code-owner-approval override to the breaking-change check. When a code owner of the changed files approves the PR, the check re-runs automatically (via a new pull_request_review trigger) and turns green.

.github/workflows/breaking-change-check.yml (new)

The breaking-change job moved into its own workflow file so we can trigger it on pull_request_review events without re-running the entire test suite (unit tests, e2e, type-diff, etc.) every time someone clicks Approve. Triggers:

  • pull_request — same as before
  • pull_request_review: types: [submitted, dismissed, edited] — re-runs when an approval is added or withdrawn so the check turns green/red automatically
  • merge_group — same as before

Concurrency is keyed per PR (or per merge-queue head) so the latest review event always wins.

.github/workflows/tests-pr.yml

  • Removes the now-relocated major-change-check job.

workspace/src/major-change-check.js

When breaking changes are detected, the script now:

  1. Fetches the PR's reviews via GET /repos/{owner}/{repo}/pulls/{N}/reviews.
  2. Computes the latest review per author (matches GitHub's "latest review wins" semantics — a later CHANGES_REQUESTED cancels an earlier APPROVED, exactly like dismiss_stale_reviews).
  3. For each approver, checks GET /repos/{owner}/{repo}/collaborators/{user}/permission. If the permission is admin, maintain, or write, treat the approval as a code-owner override.
  4. If override is granted: emit has_breaking_changes=false, log the approver, append an "Override: approved by @user" footer to the sticky comment, and exit cleanly.
  5. Otherwise: behave exactly as before — sticky comment + exit 1.

The script also reads CODEOWNERS (.github/CODEOWNERS) and logs which teams own the changed paths for transparency in the run output. The override decision uses the repo-permission check (not direct team-membership lookup) because the default GITHUB_TOKEN can't list arbitrary org-team memberships across Shopify, but it can read repo-level collaborator permissions. Anyone with write on this repo got that access via a code-owning team grant — same security posture as branch protection's "require code-owner review", evaluated earlier so the check can flip without manual CI re-runs.

The override does not auto-approve when the GitHub API fails — every degradation path returns approved: false. We'd rather over-flag than silently grant.

workspace/src/major-change-check.test.js

10 new tests:

  • parseCodeowners — comments and blank lines stripped, owners preserved
  • codeownersPatternToRegExp*, **, anchored, and unanchored patterns
  • ownersForFiles — last-matching-rule-wins (matches CODEOWNERS semantics, including theme overriding the catch-all)
  • findCodeownerApproval:
    • Approver with write access overrides (✓)
    • Latest review per author wins (a later CHANGES_REQUESTED cancels an earlier APPROVED, sticky-as-dismiss_stale_reviews)
    • No approving reviewer with write access → not approved
    • Missing PR number bails immediately
    • Reviews API failure bails (does NOT silently grant)

Stickiness behaviour

Per the requirement: matches dismiss_stale_reviews. Concretely:

  • If the same author submits a later CHANGES_REQUESTED or DISMISSED, the override is gone (we always look at latest review per author).
  • After a git push, the PR's existing approving reviews remain (since dismiss_stale_reviews: false on main's branch protection), so the override survives a force-push. If branch-protection ever flips dismiss_stale_reviews to true, GitHub will dismiss the reviews and our check will correctly go red again on the next event.

How to test your changes?

cd workspace
node --test src/major-change-check.test.js

All 21 tests pass locally (11 pre-existing + 10 new). End-to-end testing requires a PR that introduces a real or fake breaking change; smoke-test plan:

  1. Open a PR that removes a Zod field. The check should be red with the existing sticky-comment.
  2. A code owner clicks Approve.
  3. The workflow re-triggers (visible in Actions → Breaking change check). The check turns green and the sticky-comment gains the "Override: approved by @user" footer.
  4. The reviewer requests changes. The check turns red again.

Post-release steps

None — the workflow takes effect immediately on first run on main.

Checklist

  • I've considered possible cross-platform impacts (Mac, Linux, Windows) — pure Node + workflow YAML, no platform calls
  • I've considered possible documentation changes
  • I've considered analytics changes to measure impact — N/A (CI tooling)
  • The change is user-facing — N/A (internal CI tooling, no changeset needed)

alfonso-noriega commented May 5, 2026

Copy link
Copy Markdown
Contributor Author

@github-actions github-actions Bot added the no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users. label May 5, 2026
@alfonso-noriega alfonso-noriega marked this pull request as ready for review May 12, 2026 10:50
@alfonso-noriega alfonso-noriega requested a review from a team as a code owner May 12, 2026 10:50
isaacroldan
isaacroldan previously approved these changes May 14, 2026
@alfonso-noriega alfonso-noriega dismissed isaacroldan’s stale review May 14, 2026 10:14

Testing ci checks behavior

@alfonso-noriega alfonso-noriega force-pushed the breaking-change-codeowner-override branch from d04a8f9 to d31b575 Compare May 14, 2026 10:37
@github-actions

Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/conf-store.d.ts
@@ -128,11 +128,10 @@ interface RunWithRateLimitOptions {
 export declare function runWithRateLimit(options: RunWithRateLimitOptions, config?: LocalStorage<ConfSchema>): Promise<boolean>;
 /**
  * Get auto-upgrade preference.
- * Defaults to true if the preference has never been explicitly set.
  *
- * @returns Whether auto-upgrade is enabled.
+ * @returns Whether auto-upgrade is enabled, or undefined if never set.
  */
-export declare function getAutoUpgradeEnabled(config?: LocalStorage<ConfSchema>): boolean;
+export declare function getAutoUpgradeEnabled(config?: LocalStorage<ConfSchema>): boolean | undefined;
 /**
  * Set auto-upgrade preference.
  *
packages/cli-kit/dist/private/node/constants.d.ts
@@ -8,7 +8,6 @@ export declare const environmentVariables: {
     env: string;
     firstPartyDev: string;
     noAnalytics: string;
-    optOutInstrumentation: string;
     appAutomationToken: string;
     partnersToken: string;
     runAsUser: string;
packages/cli-kit/dist/public/node/metadata.d.ts
@@ -34,7 +34,7 @@ export type SensitiveSchema<T> = T extends RuntimeMetadataManager<infer _TPublic
  * @returns A container for the metadata.
  */
 export declare function createRuntimeMetadataContainer<TPublic extends AnyJson, TSensitive extends AnyJson = Record<string, never>>(defaultPublicMetadata?: Partial<TPublic>): RuntimeMetadataManager<TPublic, TSensitive>;
-type CmdFieldsFromMonorail = PickByPrefix<MonorailEventPublic, 'cmd_all_'> & PickByPrefix<MonorailEventPublic, 'cmd_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_create_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_theme_'> & PickByPrefix<MonorailEventPublic, 'store_'> & PickByPrefix<MonorailEventPublic, 'env_auto_upgrade_'>;
+type CmdFieldsFromMonorail = Pick<MonorailEventPublic, 'shop_id'> & PickByPrefix<MonorailEventPublic, 'cmd_all_'> & PickByPrefix<MonorailEventPublic, 'cmd_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_create_app_'> & PickByPrefix<MonorailEventPublic, 'cmd_theme_'> & PickByPrefix<MonorailEventPublic, 'store_'> & PickByPrefix<MonorailEventPublic, 'env_auto_upgrade_'>;
 declare const coreData: RuntimeMetadataManager<CmdFieldsFromMonorail, {
     commandStartOptions: {
         startTime: number;
packages/cli-kit/dist/public/node/monorail.d.ts
@@ -2,7 +2,7 @@ import { JsonMap } from '../../private/common/json.js';
 import { DeepRequired } from '../common/ts/deep-required.js';
 export { DeepRequired };
 type Optional<T> = T | null;
-export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.24";
+export declare const MONORAIL_COMMAND_TOPIC = "app_cli3_command/1.23";
 export interface Schemas {
     [MONORAIL_COMMAND_TOPIC]: {
         sensitive: {
@@ -32,7 +32,7 @@ export interface Schemas {
             node_version: string;
             is_employee: boolean;
             store_fqdn_hash?: Optional<string>;
-            store_fqdn_validated?: Optional<boolean>;
+            shop_id?: Optional<number>;
             user_id: string;
             cmd_all_alias_used?: Optional<string>;
             cmd_all_launcher?: Optional<string>;
packages/cli-kit/dist/public/node/node-package-manager.d.ts
@@ -13,8 +13,8 @@ export declare const bunLockfile = "bun.lockb";
 export declare const pnpmWorkspaceFile = "pnpm-workspace.yaml";
 /** An array containing the lockfiles from all the package managers */
 export declare const lockfiles: Lockfile[];
-export declare const lockfilesByManager: Record<PackageManager, Lockfile[]>;
-export type Lockfile = 'yarn.lock' | 'package-lock.json' | 'pnpm-lock.yaml' | 'bun.lockb' | 'bun.lock';
+export declare const lockfilesByManager: Record<PackageManager, Lockfile | undefined>;
+export type Lockfile = 'yarn.lock' | 'package-lock.json' | 'pnpm-lock.yaml' | 'bun.lockb';
 /**
  * A union type that represents the type of dependencies in the package.json
  * - dev: devDependencies
packages/cli-kit/dist/public/node/upgrade.d.ts
 import { getAutoUpgradeEnabled, setAutoUpgradeEnabled } from '../../private/node/conf-store.js';
 export { getAutoUpgradeEnabled, setAutoUpgradeEnabled };
 /**
  * Utility function for generating an install command for the user to run
  * to install an updated version of Shopify CLI.
  *
  * @returns A string with the command to run, or undefined if the package manager cannot be determined.
  */
 export declare function cliInstallCommand(): string | undefined;
 /**
- * Options for {@link runCLIUpgrade}.
- */
-export interface RunCLIUpgradeOptions {
-    /**
-     * `true` when the upgrade is being triggered by the automatic postrun hook.
-     * In that case we skip project-local upgrades — those should only happen when the
-     * user explicitly runs `shopify upgrade` so we don't surprise them by mutating
-     * their `package.json` / lockfile in the background.
-     */
-    autoupgrade?: boolean;
-}
-/**
  * Runs the CLI upgrade using the appropriate package manager.
  * Determines the install command and executes it.
  *
- * @param options - See {@link RunCLIUpgradeOptions}.
  * @throws AbortError if the package manager or command cannot be determined.
  */
-export declare function runCLIUpgrade(options?: RunCLIUpgradeOptions): Promise<void>;
+export declare function runCLIUpgrade(): Promise<void>;
 /**
  * Returns the version to auto-upgrade to, or undefined if auto-upgrade should be skipped.
- * Auto-upgrade is enabled by default and can be disabled via `setAutoUpgradeEnabled(false)`.
+ * Auto-upgrade is disabled by default and must be enabled via `shopify upgrade`.
  * Also skips for CI, pre-release versions, or when no newer version is available.
  *
  * @returns The version string to upgrade to, or undefined if no upgrade should happen.
  */
 export declare function versionToAutoUpgrade(): string | undefined;
 /**
  * Checks the freshly fetched notifications feed for a kill-switch notification that
  * disables auto-upgrade. A blocking notification is one with `surface: "autoupgrade"`,
  * `type: "error"`, and matching version/date ranges for the current CLI.
  *
  * Fails open: any error fetching or parsing the feed results in `false`, so a broken
  * notifications endpoint never prevents users from auto-upgrading. Intentionally silent
  * (no logs) — this is invoked on the auto-upgrade hot path.
  *
  * @returns `true` when an active blocking notification is found, `false` otherwise.
  */
 export declare function hasBlockingAutoUpgradeNotification(): Promise<boolean>;
 /**
  * Shows a daily upgrade-available warning for users who have not enabled auto-upgrade.
  * Skipped in CI and for pre-release versions. When auto-upgrade is enabled this is a no-op
  * because the postrun hook will handle the upgrade directly.
  */
 export declare function warnIfUpgradeAvailable(): Promise<void>;
 /**
  * Generates a message to remind the user to update the CLI.
  * For major version bumps, appends a link to the GitHub release notes so users
  * can review breaking changes before deciding to upgrade.
  *
  * @param version - The version to update to.
  * @param isMajor - Whether the version bump is a major version change.
  * @returns The message to remind the user to update the CLI.
  */
 export declare function getOutputUpdateCLIReminder(version: string, isMajor?: boolean): string;
+/**
+ * Prompts the user to enable or disable automatic upgrades, then persists their choice.
+ *
+ * @returns Whether the user chose to enable auto-upgrade.
+ */
+export declare function promptAutoUpgrade(): Promise<boolean>;
packages/cli-kit/dist/public/node/context/local.d.ts
@@ -53,7 +53,7 @@ export declare function isUnitTest(env?: NodeJS.ProcessEnv): boolean;
  * Returns true if reporting analytics is enabled.
  *
  * @param env - The environment variables from the environment of the current process.
- * @returns True unless SHOPIFY_CLI_NO_ANALYTICS or OPT_OUT_INSTRUMENTATION is truthy, or debug mode is enabled.
+ * @returns True unless SHOPIFY_CLI_NO_ANALYTICS is truthy or debug mode is enabled.
  */
 export declare function analyticsDisabled(env?: NodeJS.ProcessEnv): boolean;
 /**

@alfonso-noriega alfonso-noriega force-pushed the remove-fake-approval-gate branch from 04411a5 to 15299d0 Compare May 14, 2026 10:54
@alfonso-noriega alfonso-noriega force-pushed the breaking-change-codeowner-override branch from d31b575 to 0872730 Compare May 14, 2026 10:57
@alfonso-noriega alfonso-noriega force-pushed the remove-fake-approval-gate branch from 6e6bea7 to 8fdaa1f Compare May 14, 2026 10:58
@alfonso-noriega alfonso-noriega force-pushed the breaking-change-codeowner-override branch from 0872730 to 27d7554 Compare May 14, 2026 10:58
Base automatically changed from remove-fake-approval-gate to main May 14, 2026 13:14

@isaacroldan isaacroldan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Review assisted by pair-review

Comment thread .github/workflows/breaking-change-check.yml Outdated
Comment thread workspace/src/major-change-check.js
Comment thread workspace/src/major-change-check.js
Comment thread .github/workflows/breaking-change-check.yml Outdated
@alfonso-noriega alfonso-noriega force-pushed the breaking-change-codeowner-override branch from 10ffab5 to d2b3f35 Compare May 19, 2026 12:21
@alfonso-noriega alfonso-noriega force-pushed the breaking-change-codeowner-override branch from f3ea402 to 04d5fb5 Compare May 22, 2026 11:13

@isaacroldan isaacroldan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


Review assisted by pair-review

Comment thread .github/workflows/breaking-change-check.yml
Comment thread .github/workflows/breaking-change-check.yml Outdated
Comment thread workspace/src/major-change-check.js
Comment thread workspace/src/major-change-check.js Outdated
Comment thread workspace/src/major-change-check.js
Moving the job into its own workflow file accidentally dropped fetch-depth
from 0 to 1. Without full history, `git merge-base origin/<base> HEAD`
fails and the scoping logic falls back to scanning all changesets/schemas,
re-flagging every pre-existing major change in the repo.
…views

- Workflow: split the post-check sticky-comment step so the override
  footer ('Override: approved by @user') survives instead of being
  deleted alongside the 'no breaking changes' clear-out.
- major-change-check.js: filter the latest-review-per-author map to
  actionable states only (APPROVED / CHANGES_REQUESTED / DISMISSED).
  Without this, an approver who later leaves an inline review comment
  has their APPROVED entry overwritten by the resulting COMMENTED
  review and the override is silently lost.
- Adds regression test for the COMMENTED-after-APPROVED case.
Comment #3: resolveContext only returned {baselineRef, changedFiles}, so
runMain called findCodeownerApproval with undefined repo/prNumber and
the override always bailed at 'no PR number in event'. The feature was
a silent no-op in CI even though unit tests passed (they injected
repo/prNumber directly, bypassing the integration path).

- Reads GITHUB_REPOSITORY and GITHUB_EVENT_PATH inside resolveContext,
  with the same dependency-injection shape (`repository`, `eventPath`,
  `readEvent`) we use for runGit so tests can drive it deterministically.
- Returns {baselineRef, changedFiles, repo, prNumber} from every code
  path; merge_group / local invocations cleanly degrade to null.
- Adds an integration-shaped test that exercises the actual env-var
  path (pull_request and pull_request_review payloads), plus coverage
  for merge_group, missing env, malformed slugs, and unreadable event
  files. This is the exact regression the unit tests previously missed.
- Logs the resolved PR context (or its absence) so CI runs make it
  obvious whether the override check is eligible.

Comment #4: removed the `pnpm nx run-many --target=build` step from the
workflow. major-change-check.js operates on git-tracked source files
and `git diff` output only \u2014 it never imports compiled artefacts, and
runMain already explicitly skips install+build for the baseline clone
for the same reason. Verified locally: the script runs cleanly against
a fresh checkout with no dist/. Approval-triggered re-runs now take
~30s instead of ~5\u20138min.
5 follow-ups from Isaac's review:

1. Drop `edited` from the pull_request_review trigger types. The
   `edited` action fires when a review's body text is mutated, which
   doesn't affect approval state \u2014 so it re-runs the full clone+diff
   for no functional effect.

2. Bump DEFAULT_NODE_VERSION 24.1.0 \u2192 26.1.0 to match tests-pr.yml,
   tests-main.yml, and tests-manual.yml. The 24.1.0 value was an
   oversight from the workflow extraction.

3. Add a module-level comment above the code-owner override section
   clarifying that the CODEOWNERS parsing helpers (parseCodeowners,
   codeownersPatternToRegExp, ownersForFiles) are informational-only:
   they log which teams own the changed paths but do *not* gate the
   override. Gating is on repo write access, per the existing
   doc-comment on findCodeownerApproval. This addresses the 'misleading
   surface area' concern without ripping out tested primitives that we
   may want to promote into the gating path later (e.g. once a token
   that can resolve org-team membership is available).

4. Delete the unused `defaultFetchCompare` helper. Verified no callers
   anywhere in the codebase \u2014 it was scaffolded for a compare-API path
   that was replaced by the git-based diff in resolveContext.

5. Add an explicit `!repo` guard to findCodeownerApproval. This is a
   follow-on to the previous round's resolveContext change: that fix
   started returning `repo: null` when GITHUB_REPOSITORY is unset or
   malformed (local invocation), but the downstream guard was missed.
   Without it, defaultFetchReviews(null, prNumber) would throw
   TypeError reading 'owner' \u2014 accidentally fail-safe (non-zero exit)
   but with a baffling error message. Adds a regression test covering
   the case and asserting that no GitHub API URL is constructed when
   repo is null.
@alfonso-noriega alfonso-noriega force-pushed the breaking-change-codeowner-override branch from 372e473 to 0511a7e Compare May 25, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-changelog This PR doesn't include a changeset entry. Is an internal only change not relevant to end users.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants