From d3a58772e718243df229bf7a9c7d50cc8cd82dbd Mon Sep 17 00:00:00 2001 From: Zoey Rose Date: Wed, 12 Aug 2026 00:02:44 +0000 Subject: [PATCH 1/3] feat(automation): govern Classic dependency updater identity --- CONTRIBUTING.md | 11 ++ README.md | 113 ++++++++++++++ bin/validate | 122 +++++++++++++++ bin/verify-manual-settings | 165 ++++++++++++++++++++- config/manual-settings.json | 33 ++++- tests/validate-manual-credentials.sh | 95 +++++++++++- tests/verify-manual-settings.sh | 214 ++++++++++++++++++++++++++- 7 files changed, 742 insertions(+), 11 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b54e194..9e13b56 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,6 +27,17 @@ material. Changes require credential-schema negative tests, the read-only `bin/verify-manual-settings` plan, and the documented manual and scheduled post-rotation checks. +Actions GitHub Apps require their own value-free inventory entry with stable +App, installation, and repository IDs; exact permissions and event set; +selected-repository verification; consumer paths; secret and variable names; +accountable ownership; verification and rotation dates; and a runbook. Never +store a private key, installation token, token response, or credential value. +Changes require negative schema tests, read-only installation and metadata-name +verification, and organization-owner confirmation of the exact selected +repository set in GitHub's installation UI. Keep the App out of bypass lists, +preserve read-only Actions defaults, and prove the consumer's disposable pull +request receives ordinary checks without self-approval or self-merge. + Organization identity changes must preserve the exact public description and canonical website in `config/organization.json`, remain delta-aware in plan and apply modes, and include unchanged-state coverage. Keep the public profile in diff --git a/README.md b/README.md index d9b052e..7b11ff8 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,12 @@ read-only and are skipped on later runs. secret-name presence plus inventoried environment identity, deployment branches, reviewer rules, and exact secret and variable name sets without reading or proving any value. +- Repository-scoped GitHub Apps used by Actions have separate value-free + lifecycle contracts in `config/manual-settings.json`. The inventory binds + the stable App and installation IDs, exact permissions and event set, + intended repository and consumer, credential metadata names, accountable + owner, verification date, rotation deadline, and runbook. The App private + key and installation tokens never belong in this repository. - GitHub Actions defaults to read-only, cannot approve pull requests, and may use only Atrinik, GitHub, Codecov coverage, and explicitly allowed Docker actions. @@ -225,6 +231,113 @@ functionality proof: apply mode verifies the classic PAT's advertised scopes, Project update capability, and write access for every repository needing an issue-type change before the first mutation. +### Classic dependency update App + +`atrinik-classic-dependency-updater` is an organization-owned GitHub App used +only by the planned `atrinik/classic` consumer workflow +`.github/workflows/update-content.yml`. App ID `4564008` and installation ID +`153045686` are public stable identifiers. The installation selects only +`atrinik/classic`, repository ID `1327289971`, and has exactly these repository +permissions: + +- metadata: read; +- contents: write; +- pull requests: write. + +The App subscribes to no webhook events and has no Actions, checks, +deployments, environments, issues, packages, secrets, organization +administration, or ruleset-bypass permission. Organization Actions defaults +remain read-only and Actions cannot approve pull-request reviews. The App ID +is stored as the repository Actions variable `DEPENDENCY_UPDATE_APP_ID`; its +private key is stored as the repository Actions secret +`DEPENDENCY_UPDATE_APP_PRIVATE_KEY`. Never commit, print, cache, artifact, or +place either the private key or an installation token in a pull-request body, +step output, command line, fixture, or log. + +The public REST response available to the settings administration credential +proves the organization owner, stable App and installation IDs, selected-mode +installation, exact effective permissions, empty event set, and unsuspended +state. GitHub does not expose the installation's exact selected-repository list +to that credential. An organization owner must therefore also inspect + +and confirm that the only selected repository is `classic` whenever the record +is created, rotated, or reviewed. `repository_scope_verification` records this +manual boundary; `bin/verify-manual-settings` additionally verifies Classic's +stable repository identity and both credential metadata names without reading +or logging their values. + +GitHub App installation access tokens expire after one hour. The consumer must +mint a token only in the branch/PR mutation job, scope it to `classic`, avoid +exporting it beyond the necessary steps, and allow the pinned token action to +revoke it when the job finishes. The automation boundary is the single stable +branch `automation/content-update` and one open pull request from that branch. +Use one non-cancelling concurrency group, `classic-content-update`; a queued +run re-evaluates current state after the prior run completes. A run updates the +existing App-owned branch and PR only when their base, head owner, author, and +changed-path contract are intact. Zero matching PRs permits creation, one +permits refresh, and multiple matches or unexpected commits fail closed. + +GitHub has no branch-only App permission: `contents: write`, which is required +to update the automation branch, also authorizes Git-reference and release API +operations. GitHub likewise has no separate create-or-update-PR permission that +excludes review APIs: `pull requests: write` covers both. The App credential +therefore cannot by itself prove that tag, release, or review calls are +technically impossible. Because the App owns its generated pull request, it +cannot provide the distinct human approval required by the ordinary gate, and +it has no bypass. The consumer must never review, approve, merge, tag, publish, +dispatch a release, write the default branch, or change repository settings. +Its reviewed code, exact branch and changed-path checks, ordinary pull-request +gate, protected release-tag rules, workflow contract tests, and audit trail +jointly enforce that operational boundary. Do not describe the credential as +release- or review-incapable; treat any use outside the exact automation branch +and pull-request operations as an incident. + +Provision or rotate the key as an Atrinik organization owner: + +1. Prepare a reviewed change that advances `last_verified_on` and `rotate_by` + by no more than the recorded 90-day cadence. Reconfirm the App owner, + installation ID, Classic-only selection, empty event set, and exact three + permissions in the GitHub UI before changing credentials. +2. Generate a new App private key while the prior key remains valid. Store the + complete new PEM in the approved private credential manager and replace the + `DEPENDENCY_UPDATE_APP_PRIVATE_KEY` repository secret without echoing it. + Keep `DEPENDENCY_UPDATE_APP_ID` equal to the public numeric App ID. +3. Run `bin/verify-manual-settings`. This proves metadata and name presence, + not the key value. After the Classic consumer exists, dispatch only + `update-content.yml` and require one App-authored disposable pull request to + receive ordinary `Classic validation`, `CodeQL validation`, and + `Conventional PR title` checks. +4. Confirm the App-authored pull request cannot satisfy its own human approval + or merge gate, close the disposable pull request, remove its branch, and + only then delete the previous private key. A later scheduled run must also + succeed before rotation is considered complete. + +For suspected disclosure or misuse, disable the Classic updater workflow, +suspend installation `153045686`, delete the repository secret, and revoke the +affected private key immediately. Review Actions logs, App and organization +audit events, open pull requests, branches, tags, and releases; remove only +verified App-owned disposable state. Rotate to a new key and repeat the full +proof before re-enabling. To revoke permanently, uninstall the App from +Classic, delete its keys plus the repository secret and variable, and reconcile +`config/manual-settings.json` through a reviewed rollback. Do not leave a +credential inventory entry claiming a revoked installation is active. + +Read non-secret live metadata and run the complete verifier with: + +```sh +gh api orgs/atrinik/installations \ + --jq '[.installations[] | {id, app_id, app_slug, repository_selection, permissions, events, suspended_at}]' +gh api repos/atrinik/classic/actions/secrets \ + --jq '{total_count, names: [.secrets[].name]}' +gh api repos/atrinik/classic/actions/variables \ + --jq '{total_count, names: [.variables[].name]}' +bin/verify-manual-settings +``` + +If any ID, permission, event, suspension state, selected repository, or +credential name differs, stop. Disable the consumer and reconcile reviewed +desired state before minting another installation token. + ## Cross-repository planning Review and apply the planning layers in their dependency order: diff --git a/bin/validate b/bin/validate index 5b1dfc3..a61227b 100755 --- a/bin/validate +++ b/bin/validate @@ -226,6 +226,7 @@ jq -e '. == {state: "not-configured"}' \ jq -e ' (keys == [ "codecov_github_app_repositories", + "github_actions_apps", "github_actions_credentials", "github_actions_environments", "github_packages_actions_access", @@ -241,6 +242,127 @@ jq -e ' ]) ' "${manual_settings_config}" >/dev/null +jq -e \ + --arg today "${validation_today}" \ + --slurpfile repositories "${repositories_config}" ' + def day_epoch($value): + ($value + "T00:00:00Z" | fromdateiso8601); + def valid_day($value): + ($value | type == "string" and test("^[0-9]{4}-[0-9]{2}-[0-9]{2}$")) and + (try ((day_epoch($value) | strftime("%Y-%m-%d")) == $value) catch false); + .github_actions_apps as $apps | + ($apps | type == "array" and length > 0) and + ([$apps[].app_id] | length == (unique | length)) and + ([$apps[].installation_id] | length == (unique | length)) and + ( + [$apps[] | [.repository, .secret_scope, .secret_name]] | + length == (unique | length) + ) and + ( + [$apps[] | [.repository, .variable_scope, .variable_name]] | + length == (unique | length) + ) and + all( + $apps[]; + (keys == [ + "app_id", + "app_slug", + "consumers", + "events", + "installation_id", + "last_verified_on", + "owner", + "permissions", + "purpose", + "repository", + "repository_id", + "repository_scope_verification", + "repository_selection", + "rotate_by", + "rotation_cadence_days", + "rotation_owner", + "runbook", + "secret_name", + "secret_scope", + "variable_name", + "variable_scope" + ]) and + (.app_id | type == "number" and floor == . and . > 0) and + ( + .app_slug | + type == "string" and + test("^[a-z0-9]+(?:-[a-z0-9]+)*$") + ) and + (.installation_id | type == "number" and floor == . and . > 0) and + .repository_selection == "selected" and + .repository_scope_verification == "organization_owner_ui" and + .events == [] and + .permissions == { + contents: "write", + metadata: "read", + pull_requests: "write" + } and + ( + .repository | + type == "string" and + test("^atrinik/[A-Za-z0-9_.-]+$") + ) and + ( + .repository as $full_name | + ($full_name | split("/")[1]) as $repository | + ($repositories[0].pull_request_gate | index($repository)) != null and + ($repositories[0].archive | index($repository)) == null + ) and + (.repository_id | type == "number" and floor == . and . > 0) and + .secret_scope == "repository" and + .variable_scope == "repository" and + (.secret_name | type == "string" and test("^[A-Z][A-Z0-9_]*$")) and + (.variable_name | type == "string" and test("^[A-Z][A-Z0-9_]*$")) and + .secret_name != .variable_name and + ( + .consumers | + type == "array" and + length > 0 and + . == (sort | unique) and + all( + .[]; + type == "string" and + test("^\\.github/workflows/[A-Za-z0-9_.-]+\\.ya?ml$") + ) + ) and + (.purpose | type == "string" and length > 0) and + (.owner | type == "string" and length > 0) and + (.rotation_owner | type == "string" and length > 0) and + ( + .rotation_cadence_days | + type == "number" and floor == . and . >= 1 and . <= 365 + ) and + valid_day(.last_verified_on) and + valid_day(.rotate_by) and + (day_epoch(.last_verified_on) <= day_epoch($today)) and + (day_epoch(.rotate_by) >= day_epoch($today)) and + (day_epoch(.rotate_by) > day_epoch(.last_verified_on)) and + ( + (day_epoch(.rotate_by) - day_epoch(.last_verified_on)) / 86400 <= + .rotation_cadence_days + ) and + ( + .runbook | + type == "string" and + test("^[A-Za-z0-9._/-]+\\.md#[a-z0-9-]+$") + ) + ) and + all( + $apps[]; + . as $app | + all( + $apps[]; + .repository != $app.repository or + .repository_id == $app.repository_id + ) + ) +' "${manual_settings_config}" >/dev/null + jq -e ' .organization_pins as $pins | ($pins | keys == ["apply_path", "repositories", "verification"]) and diff --git a/bin/verify-manual-settings b/bin/verify-manual-settings index 734af6c..4f97339 100755 --- a/bin/verify-manual-settings +++ b/bin/verify-manual-settings @@ -51,6 +51,7 @@ github_api() { declare -A repository_data_files=() declare -A repository_secret_files=() +declare -A repository_variable_files=() load_repository_identity() { local repository=$1 @@ -132,6 +133,73 @@ load_repository_secrets() { fi } +load_repository_variables() { + local repository=$1 + local collected expected page page_size repository_key response status + + if [[ -v repository_variable_files["${repository}"] ]]; then + return + fi + if load_repository_identity "${repository}"; then + : + else + status=$? + return "${status}" + fi + + repository_key=${repository//\//-} + repository_variable_files["${repository}"]=${temporary}/${repository_key}-variables.jsonl + : >"${repository_variable_files["${repository}"]}" + expected= + collected=0 + page=1 + while true; do + if response=$(github_api \ + "list ${repository} Actions variables (page ${page})" \ + "repos/${repository}/actions/variables?per_page=100&page=${page}"); then + : + else + status=$? + return "${status}" + fi + if ! jq -e ' + (.total_count | type == "number" and floor == . and . >= 0) and + (.variables | type == "array") and + all( + .variables[]; + (.name | type == "string") and + (.created_at | type == "string") and + (.updated_at | type == "string") + ) + ' <<<"${response}" >/dev/null; then + echo "error: Actions variable metadata response is invalid for ${repository} (page ${page})" >&2 + return 1 + fi + if [[ -z ${expected} ]]; then + expected=$(jq -r '.total_count' <<<"${response}") + elif [[ $(jq -r '.total_count' <<<"${response}") != "${expected}" ]]; then + echo "error: Actions variable metadata changed while listing ${repository}" >&2 + return 1 + fi + jq -c '.variables[] | {name, created_at, updated_at}' <<<"${response}" \ + >>"${repository_variable_files["${repository}"]}" + page_size=$(jq -r '.variables | length' <<<"${response}") + ((collected += page_size)) + if ((collected >= expected)); then + break + fi + if ((page_size == 0)); then + echo "error: Actions variable metadata ended before total_count for ${repository}" >&2 + return 1 + fi + ((page += 1)) + done + if ((collected != expected)); then + echo "error: Actions variable metadata count mismatch for ${repository}: expected ${expected}, got ${collected}" >&2 + return 1 + fi +} + load_paginated_metadata() { local operation=$1 local endpoint=$2 @@ -400,6 +468,101 @@ while IFS= read -r environment_record; do echo "KEEP ${repository} environment ${environment} metadata" done < <(jq -c '.github_actions_environments[]' "${manual_settings}") +installations_file=${temporary}/${organization}-installations.jsonl +if load_paginated_metadata \ + "list ${organization} GitHub App installations" \ + "orgs/${organization}/installations" \ + installations "${installations_file}"; then + : +else + status=$? + exit "${status}" +fi + +while IFS= read -r app_record; do + app_id=$(jq -r '.app_id' <<<"${app_record}") + app_slug=$(jq -r '.app_slug' <<<"${app_record}") + installation_id=$(jq -r '.installation_id' <<<"${app_record}") + repository=$(jq -r '.repository' <<<"${app_record}") + repository_owner=${repository%%/*} + repository_id=$(jq -r '.repository_id' <<<"${app_record}") + secret_name=$(jq -r '.secret_name' <<<"${app_record}") + variable_name=$(jq -r '.variable_name' <<<"${app_record}") + + if [[ ${repository_owner} != "${organization}" ]]; then + echo "error: GitHub App repository owner does not match ${organization}: ${repository}" >&2 + exit 1 + fi + if verify_repository_identity "${repository}" "${repository_id}"; then + : + else + status=$? + exit "${status}" + fi + + expected_permissions=$(jq -c '.permissions' <<<"${app_record}") + installation=$(jq -sc \ + --argjson app_id "${app_id}" \ + --argjson installation_id "${installation_id}" ' + [.[] | select(.app_id == $app_id and .id == $installation_id)] | + if length == 1 then .[0] else empty end + ' "${installations_file}") + if [[ -z ${installation} ]]; then + echo "error: GitHub App installation is missing or ambiguous for ${app_slug}" >&2 + exit 1 + fi + if ! jq -e \ + --arg app_slug "${app_slug}" \ + --arg organization "${organization}" \ + --argjson expected_permissions "${expected_permissions}" ' + .app_slug == $app_slug and + .target_type == "Organization" and + .account.login == $organization and + .account.type == "Organization" and + .repository_selection == "selected" and + .permissions == $expected_permissions and + .events == [] and + .suspended_at == null + ' <<<"${installation}" >/dev/null; then + echo "error: GitHub App installation metadata or permission drift for ${app_slug}" >&2 + exit 1 + fi + + if load_repository_secrets "${repository}"; then + : + else + status=$? + exit "${status}" + fi + secret_updated_at=$(jq -sr --arg name "${secret_name}" ' + [.[] | select(.name == $name) | .updated_at] | + if length == 1 then .[0] else empty end + ' "${repository_secret_files["${repository}"]}") + if [[ -z ${secret_updated_at} ]]; then + echo "error: GitHub App Actions secret name is missing for ${repository}: ${secret_name}" >&2 + exit 1 + fi + + if load_repository_variables "${repository}"; then + : + else + status=$? + exit "${status}" + fi + variable_updated_at=$(jq -sr --arg name "${variable_name}" ' + [.[] | select(.name == $name) | .updated_at] | + if length == 1 then .[0] else empty end + ' "${repository_variable_files["${repository}"]}") + if [[ -z ${variable_updated_at} ]]; then + echo "error: GitHub App Actions variable name is missing for ${repository}: ${variable_name}" >&2 + exit 1 + fi + + echo "KEEP ${app_slug} installation metadata and exact permissions" + echo "KEEP ${repository} repository Actions secret ${secret_name} (updated ${secret_updated_at})" + echo "KEEP ${repository} repository Actions variable ${variable_name} (updated ${variable_updated_at})" +done < <(jq -c '.github_actions_apps[]' "${manual_settings}") + # GraphQL variables must remain literal for GitHub to substitute. # shellcheck disable=SC2016 pins_response=$(github_api \ @@ -442,4 +605,4 @@ if [[ ${live_pins} != "${desired_pins}" ]]; then fi echo "KEEP ${organization} organization pins match the exact governed order" -echo "Manual settings live credential, environment, and organization pin metadata is present." +echo "Manual settings live credential, GitHub App, environment, and organization pin metadata is present." diff --git a/config/manual-settings.json b/config/manual-settings.json index 876e579..0376796 100644 --- a/config/manual-settings.json +++ b/config/manual-settings.json @@ -70,6 +70,37 @@ "variable_names": [] } ], + "github_actions_apps": [ + { + "app_id": 4564008, + "app_slug": "atrinik-classic-dependency-updater", + "consumers": [ + ".github/workflows/update-content.yml" + ], + "events": [], + "installation_id": 153045686, + "last_verified_on": "2026-08-11", + "owner": "Atrinik organization owners", + "permissions": { + "contents": "write", + "metadata": "read", + "pull_requests": "write" + }, + "purpose": "Authenticate the branch and pull-request operations for one reviewed Classic content-lock update without access to other Atrinik repositories.", + "repository": "atrinik/classic", + "repository_id": 1327289971, + "repository_scope_verification": "organization_owner_ui", + "repository_selection": "selected", + "rotate_by": "2026-11-09", + "rotation_cadence_days": 90, + "rotation_owner": "Atrinik organization owners", + "runbook": "README.md#classic-dependency-update-app", + "secret_name": "DEPENDENCY_UPDATE_APP_PRIVATE_KEY", + "secret_scope": "repository", + "variable_name": "DEPENDENCY_UPDATE_APP_ID", + "variable_scope": "repository" + } + ], "github_actions_credentials": [ { "credential_type": "classic_pat", @@ -100,5 +131,5 @@ "github_packages_actions_access": [], "require_two_factor_authentication": true, "require_secure_two_factor_methods": true, - "reason": "The public GitHub API does not expose supported update parameters for every setting in this file, including organization pins, and external credential values must never be stored here. Confirm these settings, the exact ordered pin set, credential lifecycle metadata and live secret-name presence, Codecov GitHub App repository access, any listed GitHub Actions environments, and GitHub Packages Actions access in the corresponding organization, repository, and package settings UI." + "reason": "The public GitHub API does not expose supported update parameters for every setting in this file, including organization pins and the exact selected-repository set for an organization GitHub App installation to an administrative token, and external credential values must never be stored here. Confirm these settings, the exact ordered pin set, credential lifecycle metadata and live secret-name presence, GitHub App repository selection, Codecov GitHub App repository access, any listed GitHub Actions environments, and GitHub Packages Actions access in the corresponding organization, repository, and package settings UI." } diff --git a/tests/validate-manual-credentials.sh b/tests/validate-manual-credentials.sh index 5f6a80d..eedec04 100755 --- a/tests/validate-manual-credentials.sh +++ b/tests/validate-manual-credentials.sh @@ -15,7 +15,7 @@ cp -R "${root}/.github" "${temporary}/.github" assert_invalid() { local description=$1 - if ATRINIK_VALIDATION_TODAY=2026-08-10 \ + if ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${temporary}/bin/validate" >/dev/null 2>&1; then echo "error: validator accepted ${description}" >&2 exit 1 @@ -36,8 +36,37 @@ rewrite_manual_settings() { mv "${output}" "${temporary}/config/manual-settings.json" } -ATRINIK_VALIDATION_TODAY=2026-08-10 \ +ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${temporary}/bin/validate" >/dev/null +jq -e ' + .github_actions_apps == [{ + app_id: 4564008, + app_slug: "atrinik-classic-dependency-updater", + consumers: [".github/workflows/update-content.yml"], + events: [], + installation_id: 153045686, + last_verified_on: "2026-08-11", + owner: "Atrinik organization owners", + permissions: { + contents: "write", + metadata: "read", + pull_requests: "write" + }, + purpose: "Authenticate the branch and pull-request operations for one reviewed Classic content-lock update without access to other Atrinik repositories.", + repository: "atrinik/classic", + repository_id: 1327289971, + repository_scope_verification: "organization_owner_ui", + repository_selection: "selected", + rotate_by: "2026-11-09", + rotation_cadence_days: 90, + rotation_owner: "Atrinik organization owners", + runbook: "README.md#classic-dependency-update-app", + secret_name: "DEPENDENCY_UPDATE_APP_PRIVATE_KEY", + secret_scope: "repository", + variable_name: "DEPENDENCY_UPDATE_APP_ID", + variable_scope: "repository" + }] +' "${root}/config/manual-settings.json" >/dev/null jq -e ' .github_actions_credentials == [{ credential_type: "classic_pat", @@ -62,6 +91,66 @@ jq -e ' }] ' "${root}/config/manual-settings.json" >/dev/null +rewrite_manual_settings \ + '.github_actions_apps += [.github_actions_apps[0]]' +assert_invalid 'a duplicate GitHub App record' +reset_manual_settings + +rewrite_manual_settings \ + 'del(.github_actions_apps[0].installation_id)' +assert_invalid 'a GitHub App without an installation ID' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].permissions.actions = "write"' +assert_invalid 'a GitHub App with Actions permission' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].permissions.contents = "read"' +assert_invalid 'a GitHub App without contents write permission' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].events = ["push"]' +assert_invalid 'a GitHub App with webhook events' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].repository_selection = "all"' +assert_invalid 'a GitHub App installed on all repositories' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].repository_scope_verification = "unverified"' +assert_invalid 'a GitHub App without owner-verified repository scope' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].repository = "atrinik/unknown"' +assert_invalid 'a GitHub App for an ungoverned repository' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].secret_name = "private-key"' +assert_invalid 'a malformed GitHub App Actions secret name' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].variable_name = "app-id"' +assert_invalid 'a malformed GitHub App Actions variable name' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].rotate_by = "2026-08-10"' +assert_invalid 'a stale GitHub App rotate-by date' +reset_manual_settings + +rewrite_manual_settings \ + '.github_actions_apps[0].private_key = "secret"' +assert_invalid 'GitHub App private-key material' +reset_manual_settings + rewrite_manual_settings \ '.github_actions_credentials += [.github_actions_credentials[0]]' assert_invalid 'a duplicate repository credential' @@ -125,7 +214,7 @@ assert_invalid 'a non-leap-year February 29' reset_manual_settings rewrite_manual_settings \ - '.github_actions_credentials[0].last_verified_on = "2026-08-11"' + '.github_actions_credentials[0].last_verified_on = "2026-08-12"' assert_invalid 'a future verification date' reset_manual_settings diff --git a/tests/verify-manual-settings.sh b/tests/verify-manual-settings.sh index 3683fa8..3b9c462 100755 --- a/tests/verify-manual-settings.sh +++ b/tests/verify-manual-settings.sh @@ -126,6 +126,132 @@ repos/atrinik/classic) }' fi ;; +"orgs/atrinik/installations?per_page=100&page=1") + if [[ ${FAKE_GH_SCENARIO} == app-api-failure ]]; then + echo "gh: organization installation administration denied" >&2 + exit 45 + elif [[ ${FAKE_GH_SCENARIO} == malformed-app-installations ]]; then + jq -n '{total_count: 1, installation: []}' + elif [[ ${FAKE_GH_SCENARIO} == app-installations-page2 || \ + ${FAKE_GH_SCENARIO} == app-installations-page2-failure ]]; then + jq -n '{ + total_count: 101, + installations: [range(0; 100) | { + id: (1000 + .), + app_id: (2000 + .), + app_slug: ("unrelated-" + tostring), + target_type: "Organization", + account: {login: "atrinik", type: "Organization"}, + repository_selection: "selected", + permissions: {metadata: "read"}, + events: [], + suspended_at: null + }] + }' + elif [[ ${FAKE_GH_SCENARIO} == missing-app ]]; then + jq -n '{total_count: 0, installations: []}' + else + app_id=4564008 + installation_id=153045686 + app_slug=atrinik-classic-dependency-updater + repository_selection=selected + account_login=atrinik + events='[]' + permissions='{"contents":"write","metadata":"read","pull_requests":"write"}' + suspended_at=null + case ${FAKE_GH_SCENARIO} in + app-id-drift) app_id=1 ;; + app-installation-id-drift) installation_id=1 ;; + app-slug-drift) app_slug=other-app ;; + app-selection-drift) repository_selection=all ;; + app-account-drift) account_login=other ;; + app-events-drift) events='["push"]' ;; + app-permission-drift) permissions='{"actions":"write","contents":"write","metadata":"read","pull_requests":"write"}' ;; + app-suspended) suspended_at='"2026-08-11T00:00:00Z"' ;; + esac + jq -n \ + --argjson app_id "${app_id}" \ + --argjson installation_id "${installation_id}" \ + --arg app_slug "${app_slug}" \ + --arg repository_selection "${repository_selection}" \ + --arg account_login "${account_login}" \ + --argjson events "${events}" \ + --argjson permissions "${permissions}" \ + --argjson suspended_at "${suspended_at}" '{ + total_count: 1, + installations: [{ + id: $installation_id, + app_id: $app_id, + app_slug: $app_slug, + target_type: "Organization", + account: {login: $account_login, type: "Organization"}, + repository_selection: $repository_selection, + permissions: $permissions, + events: $events, + suspended_at: $suspended_at + }] + }' + fi + ;; +"orgs/atrinik/installations?per_page=100&page=2") + if [[ ${FAKE_GH_SCENARIO} == app-installations-page2-failure ]]; then + echo "gh: second installation page failed" >&2 + exit 47 + fi + [[ ${FAKE_GH_SCENARIO} == app-installations-page2 ]] + jq -n '{ + total_count: 101, + installations: [{ + id: 153045686, + app_id: 4564008, + app_slug: "atrinik-classic-dependency-updater", + target_type: "Organization", + account: {login: "atrinik", type: "Organization"}, + repository_selection: "selected", + permissions: { + contents: "write", + metadata: "read", + pull_requests: "write" + }, + events: [], + suspended_at: null + }] + }' + ;; +"repos/atrinik/classic/actions/secrets?per_page=100&page=1") + if [[ ${FAKE_GH_SCENARIO} == missing-app-secret ]]; then + jq -n '{total_count: 0, secrets: []}' + else + jq -n '{ + total_count: 1, + secrets: [{ + name: "DEPENDENCY_UPDATE_APP_PRIVATE_KEY", + created_at: "2026-08-11T23:47:15Z", + updated_at: "2026-08-11T23:47:15Z" + }] + }' + fi + ;; +"repos/atrinik/classic/actions/variables?per_page=100&page=1") + if [[ ${FAKE_GH_SCENARIO} == app-variable-api-failure ]]; then + echo "gh: Actions variable administration denied" >&2 + exit 46 + elif [[ ${FAKE_GH_SCENARIO} == malformed-app-variables ]]; then + jq -n '{total_count: 1, variable: []}' + elif [[ ${FAKE_GH_SCENARIO} == missing-app-variable ]]; then + jq -n '{total_count: 0, variables: []}' + else + jq -n '{ + total_count: 1, + variables: [{ + name: "DEPENDENCY_UPDATE_APP_ID", + value: "not-inspected-by-the-verifier", + created_at: "2026-08-11T23:47:00Z", + updated_at: "2026-08-11T23:47:00Z" + }] + }' + fi + ;; "repos/atrinik/github-settings/actions/secrets?per_page=100&page=1") if [[ ${FAKE_GH_SCENARIO} == missing ]]; then jq -n '{total_count: 0, secrets: []}' @@ -313,7 +439,7 @@ run_verify() { FAKE_GH_LOG="${temporary}/gh.log" \ FAKE_GH_SCENARIO="${scenario}" \ GITHUB_ACTIONS=true GH_TOKEN=test-token \ - ATRINIK_VALIDATION_TODAY=2026-08-10 \ + ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${root}/bin/verify-manual-settings" } @@ -322,8 +448,14 @@ output=$(run_verify present) grep -Fq 'KEEP atrinik/github-settings repository Actions secret ATRINIK_SETTINGS_TOKEN' \ <<<"${output}" grep -Fq 'KEEP atrinik/classic environment discord-release metadata' <<<"${output}" +grep -Fq 'KEEP atrinik-classic-dependency-updater installation metadata and exact permissions' \ + <<<"${output}" +grep -Fq 'KEEP atrinik/classic repository Actions secret DEPENDENCY_UPDATE_APP_PRIVATE_KEY' \ + <<<"${output}" +grep -Fq 'KEEP atrinik/classic repository Actions variable DEPENDENCY_UPDATE_APP_ID' \ + <<<"${output}" grep -Fq 'KEEP atrinik organization pins match the exact governed order' <<<"${output}" -grep -Fq 'Manual settings live credential, environment, and organization pin metadata is present.' \ +grep -Fq 'Manual settings live credential, GitHub App, environment, and organization pin metadata is present.' \ <<<"${output}" : >"${temporary}/gh.log" @@ -352,7 +484,7 @@ output=$(PATH="${temporary}/bin:${PATH}" \ FAKE_GH_LOG="${temporary}/gh.log" \ FAKE_GH_SCENARIO=environment-page2 \ GITHUB_ACTIONS=true GH_TOKEN=test-token \ - ATRINIK_VALIDATION_TODAY=2026-08-10 \ + ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${environment_page_root}/bin/verify-manual-settings") grep -Fq 'KEEP atrinik/classic environment discord-release metadata' <<<"${output}" grep -Fq 'deployment-branch-policies?per_page=100&page=2' "${temporary}/gh.log" @@ -376,7 +508,7 @@ output=$(PATH="${temporary}/bin:${PATH}" \ FAKE_GH_LOG="${temporary}/gh.log" \ FAKE_GH_SCENARIO=shared-repository \ GITHUB_ACTIONS=true GH_TOKEN=test-token \ - ATRINIK_VALIDATION_TODAY=2026-08-10 \ + ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${shared_root}/bin/verify-manual-settings") grep -Fq 'SECOND_SETTINGS_TOKEN' <<<"${output}" [[ $(grep -Fc 'repos/atrinik/github-settings' "${temporary}/gh.log") == 2 ]] @@ -398,6 +530,76 @@ for scenario in pin-drift malformed-pins; do exit 1 fi done + +app_failures=( + missing-app + app-id-drift + app-installation-id-drift + app-slug-drift + app-selection-drift + app-account-drift + app-events-drift + app-permission-drift + app-suspended + missing-app-secret + missing-app-variable + malformed-app-installations + malformed-app-variables + app-api-failure + app-variable-api-failure + app-installations-page2-failure +) +for scenario in "${app_failures[@]}"; do + : >"${temporary}/gh.log" + if run_verify "${scenario}" \ + >"${temporary}/${scenario}.out" 2>"${temporary}/${scenario}.err"; then + echo "error: manual-settings verifier accepted ${scenario}" >&2 + exit 1 + fi +done +grep -Fq 'GitHub App installation is missing or ambiguous' \ + "${temporary}/missing-app.err" +grep -Fq 'GitHub App installation metadata or permission drift' \ + "${temporary}/app-permission-drift.err" +grep -Fq 'GitHub App Actions secret name is missing' \ + "${temporary}/missing-app-secret.err" +grep -Fq 'GitHub App Actions variable name is missing' \ + "${temporary}/missing-app-variable.err" +grep -Fq 'returned invalid metadata' \ + "${temporary}/malformed-app-installations.err" +grep -Fq 'Actions variable metadata response is invalid' \ + "${temporary}/malformed-app-variables.err" +grep -Fq 'organization installation administration denied' \ + "${temporary}/app-api-failure.err" +grep -Fq 'Actions variable administration denied' \ + "${temporary}/app-variable-api-failure.err" +grep -Fq 'second installation page failed' \ + "${temporary}/app-installations-page2-failure.err" + +: >"${temporary}/gh.log" +output=$(run_verify app-installations-page2) +grep -Fq 'KEEP atrinik-classic-dependency-updater installation metadata and exact permissions' \ + <<<"${output}" +grep -Fq 'orgs/atrinik/installations?per_page=100&page=2' "${temporary}/gh.log" + +for failure in \ + app-api-failure:45 \ + app-variable-api-failure:46 \ + app-installations-page2-failure:47; do + scenario=${failure%%:*} + expected_status=${failure#*:} + : >"${temporary}/gh.log" + set +e + run_verify "${scenario}" \ + >"${temporary}/${scenario}-status.out" \ + 2>"${temporary}/${scenario}-status.err" + status=$? + set -e + if ((status != expected_status)); then + echo "error: ${scenario} exited ${status}, expected ${expected_status}" >&2 + exit 1 + fi +done grep -Fq 'organization pin order or identity drift' \ "${temporary}/pin-drift.err" grep -Fq 'invalid organization pin metadata' \ @@ -497,7 +699,7 @@ fi : >"${temporary}/gh.log" if PATH="${temporary}/bin:${PATH}" \ FAKE_GH_LOG="${temporary}/gh.log" FAKE_GH_SCENARIO=present \ - GITHUB_ACTIONS=true GH_TOKEN='' ATRINIK_VALIDATION_TODAY=2026-08-10 \ + GITHUB_ACTIONS=true GH_TOKEN='' ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${root}/bin/verify-manual-settings" \ >"${temporary}/empty.out" 2>"${temporary}/empty.err"; then echo "error: manual-settings verifier accepted an empty workflow credential" >&2 @@ -506,4 +708,4 @@ fi grep -Fq 'ATRINIK_SETTINGS_TOKEN is unavailable' "${temporary}/empty.err" [[ ! -s ${temporary}/gh.log ]] -echo "Manual settings live credential and environment verification tests passed." +echo "Manual settings live credential, GitHub App, and environment verification tests passed." From dfe99a2de0e8a78205db032ac38cc51e31d4d2b3 Mon Sep 17 00:00:00 2001 From: Zoey Rose Date: Wed, 12 Aug 2026 00:08:02 +0000 Subject: [PATCH 2/3] test(governance): align manual-state validation dates --- tests/check-project-health.sh | 6 +++--- tests/validate-planning-health.sh | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/check-project-health.sh b/tests/check-project-health.sh index 87f1eb9..d436983 100755 --- a/tests/check-project-health.sh +++ b/tests/check-project-health.sh @@ -311,7 +311,7 @@ run_health() { ATRINIK_SETTINGS_TOKEN=settings-token \ ATRINIK_PROJECT_HEALTH_NOW=2026-08-10T05:00:00Z \ ATRINIK_PROJECT_HEALTH_SYNC_COMMAND="${temporary}/bin/sync-plan" \ - ATRINIK_VALIDATION_TODAY=2026-08-10 \ + ATRINIK_VALIDATION_TODAY=2026-08-11 \ GITHUB_STEP_SUMMARY="${temporary}/step-summary" \ "${root}/bin/check-project-health" "$@" } @@ -396,7 +396,7 @@ if PATH="${temporary}/bin:${PATH}" \ GH_TOKEN=health-token ATRINIK_SETTINGS_TOKEN='' \ ATRINIK_PROJECT_HEALTH_NOW=2026-08-10T05:00:00Z \ ATRINIK_PROJECT_HEALTH_SYNC_COMMAND="${temporary}/bin/sync-plan" \ - ATRINIK_VALIDATION_TODAY=2026-08-10 \ + ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${root}/bin/check-project-health" --apply \ >"${temporary}/missing.out" 2>"${temporary}/missing.err"; then echo "error: health check accepted a missing settings credential" >&2 @@ -471,7 +471,7 @@ if PATH="${temporary}/bin:${PATH}" \ GH_TOKEN=health-token ATRINIK_SETTINGS_TOKEN='' \ ATRINIK_PROJECT_HEALTH_NOW=2026-08-10T05:00:00Z \ ATRINIK_PROJECT_HEALTH_SYNC_COMMAND="${temporary}/bin/sync-plan" \ - ATRINIK_VALIDATION_TODAY=2026-08-10 \ + ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${root}/bin/check-project-health" --apply \ >"${temporary}/reopen.out" 2>"${temporary}/reopen.err"; then echo "error: health check accepted a missing credential during recurrence" >&2 diff --git a/tests/validate-planning-health.sh b/tests/validate-planning-health.sh index 3226665..9303d4d 100755 --- a/tests/validate-planning-health.sh +++ b/tests/validate-planning-health.sh @@ -13,14 +13,14 @@ assert_invalid() { jq "${filter}" "${root}/config/planning-health.json" \ >"${temporary}/repository/config/planning-health.json" - if ATRINIK_VALIDATION_TODAY=2026-08-10 \ + if ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${temporary}/repository/bin/validate" >/dev/null 2>&1; then echo "error: validator accepted ${description}" >&2 exit 1 fi } -ATRINIK_VALIDATION_TODAY=2026-08-10 \ +ATRINIK_VALIDATION_TODAY=2026-08-11 \ "${temporary}/repository/bin/validate" >/dev/null assert_invalid '.freshness_threshold_minutes = 30' \ From 96f037333b5c17814b3c5a7853d93742ab5a5906 Mon Sep 17 00:00:00 2001 From: Zoey Rose Date: Wed, 12 Aug 2026 00:13:36 +0000 Subject: [PATCH 3/3] test(governance): keep lifecycle fixtures coherent --- tests/validate-manual-credentials.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/validate-manual-credentials.sh b/tests/validate-manual-credentials.sh index eedec04..88732d1 100755 --- a/tests/validate-manual-credentials.sh +++ b/tests/validate-manual-credentials.sh @@ -239,6 +239,8 @@ assert_invalid 'a top-level credential-value field' reset_manual_settings rewrite_manual_settings ' + .github_actions_apps[0].last_verified_on = "2028-02-29" | + .github_actions_apps[0].rotate_by = "2028-05-29" | .github_actions_credentials[0].last_verified_on = "2028-02-29" | .github_actions_credentials[0].rotate_by = "2028-05-29" '