Skip to content

add github-org-* tools, dm-fork-sync, dm-github-policy, CI mock + live-audit#3

Merged
assisted-by-ai merged 1 commit into
masterfrom
claude/configure-repo-access-hZI4X
Apr 30, 2026
Merged

add github-org-* tools, dm-fork-sync, dm-github-policy, CI mock + live-audit#3
assisted-by-ai merged 1 commit into
masterfrom
claude/configure-repo-access-hZI4X

Conversation

@assisted-by-ai
Copy link
Copy Markdown
Owner

@assisted-by-ai assisted-by-ai commented Apr 26, 2026

Summary

Single-commit PR adding GitHub-org tooling, project-policy wrappers, and a mock-API CI test suite to developer-meta-files. The code is ready to apply project security policy at the org level, with a deliberate testing detour through org-ai-assisted before flipping back to Kicksecure + Whonix.

Tools (usr/bin/github-org-*)

  • github-org-clone <source-org> [<dest-dir>] -- clone or fast-forward every public, non-archived, non-fork repo into <dest-dir>/<repo-name>/. Idempotent.
  • github-org-fork <source-owner> <target-owner> -- fork every selected source repo into the target. Both may be User or Organization. Optional flags configure target settings.
  • github-org-push <source-dir> <target-owner> -- push HEAD (and optionally tags) from each clone under source-dir to a same-named target repo. Refuses if missing.
  • github-org-set-fork-approval <org> [<policy>] -- set the org-level fork-PR contributor approval policy. Default all_external_contributors.

Common across all four: --dry-run, -v/--verbose, --include/--exclude REGEX, --ssh|--https, --jobs N. Auth via GITHUB_TOKEN env or chmod-600 ~/.config/github-token. No gh CLI dependency.

Shared library (usr/libexec/developer-meta-files/github-org-lib.bsh)

  • Token resolution (env / chmod-600 file; refuses symlinks).
  • Hardened ghorg_api: token via curl --config - from stdin (no argv leak), pinned to https, bounded by --connect-timeout / --max-time / --max-filesize. Token-handling functions wrap with local -; set +x so an enclosing set -x cannot leak the secret.
  • Rate-limit retry on HTTP 429, 5xx, and 403-rate-limit body. Honors Retry-After and X-RateLimit-Reset. Falls back to exponential backoff. Caps via GHORG_MAX_RETRIES and GHORG_MAX_BACKOFF_SECONDS.
  • Mock mode: GHORG_MOCK=1 GHORG_MOCK_DIR=path serves canned fixture files. Missing fixture = HTTP 599 (loud failure).
  • Account-type probe (User vs Organization) with caching; --include-private for User owners switches to /user/repos?affiliation=owner and refuses if listed user is not the auth user.
  • Strict allowlist ghorg_validate_name: ^[A-Za-z0-9._-]+$, length-capped, no leading dash, no embedded ... Allows .github / .gitignore. Reserved names (., .., .git) explicitly rejected.
  • Safe display via sanitize-string from helper-scripts.

Project-policy wrappers (usr/bin/)

  • dm-fork-sync -- thin wrapper around github-org-fork. Mirrors source orgs into the bot account.

  • dm-github-policy -- applies project security policy. Modes: default --apply, --dry-run, --audit (read-only). End-of-run summary with ok / warn / dry-run / skip counts; non-zero exit on any warn.

    Settings applied at org level:

    • Fork-PR contributor approval = all_external_contributors
    • Default workflow GITHUB_TOKEN permissions = read; cannot create or approve PRs
    • Allowed Actions = github-owned + Marketplace-verified-creators only
    • Member policy: default_repository_permission=read, members_can_create_repositories=false
    • Code Security Configurations API (modern; replaces the deprecated security-defaults PATCH /orgs/{org} fields and the deprecated /orgs/{org}/{product}/enable_all family): one configuration enables secret scanning + push protection + Dependabot alerts/updates + dependency graph; CodeQL default-setup and private vulnerability reporting explicitly disabled. Attached to all existing repos and set as default for new repos.
    • Default-branch ruleset: block force-push, block deletion, require signed commits. bypass_actors=[] -- nobody, including org owners, can override.
    • Tag ruleset: block deletion, block re-pointing, require signed tags. Same bypass_actors=[].

    Per-repo settings (no org-wide endpoint): has_wiki=false, has_issues=true.

    SKIP+notify with UI URL pointers for: 2FA enforcement, PAT policies, App/OAuth policies. Verified against the OpenAPI spec to be UI-only at this time.

    Audit-only sections (--audit): members lacking 2FA, existing rulesets, fine-grained PAT activity, installed GitHub Apps, org-level webhooks lacking a secret (Scorecard "Webhooks" Critical-risk gap), and .github/dependabot.yml file presence per repo (Scorecard "Dependency-Update-Tool" file-based detection).

    Help text documents the OSSF Scorecard mapping per check (Branch-Protection ceiling = Tier 1 deliberate; Code-Review, CodeQL, Maintained = out of scope).

Testing-only configuration (currently active on this branch)

dm-github-policy::ORGS, dm-fork-sync::SOURCE_ORGS, and dm-packaging-helper-script::batch_current_package_remote_list all currently target org-ai-assisted only. The production list (Kicksecure, Whonix, plus the assisted-by-ai user mirror) is preserved as a single-line comment above each active line for one-keystroke revert. pkg_git_remotes_add defines remotes for both assisted-by-ai and org-ai-assisted so either can be activated without further edits.

CI infrastructure (ci/ + .github/workflows/)

  • ci/fixtures/ -- 32 canned API responses for the test org, the auth user, /user/repos, every org-policy GET that audit and dry-run paths hit, plus webhooks list and per-repo dependabot.yml content lookups. All shapes verified against the current OpenAPI spec at github/rest-api-description@main.
  • ci/tests/test_*.sh -- 10 mock-API tests covering arg parsing, dry-run flow, --get / --audit, validator attack-pattern rejection, rate-limit-wait header parsing, and a shellcheck-clean enforcement test.
  • ci/test-github-org-tools.sh -- runs every test_*.sh. Refuses to run outside CI=true.
  • ci/install-genmkfile.sh + ci/install-helper-scripts.sh -- standalone CI install scripts using the genmkfile install flow (no ad-hoc symlinks).
  • ci/probe-live-unauth.sh -- best-effort live unauth smoke that skips when the rate-limit bucket is exhausted.
  • .github/workflows/test-github-org-tools.yml -- runs the mock-API suite on push/PR. paths: filter so it only runs when github-related files change. Container debian:trixie. permissions: contents: read only.
  • .github/workflows/audit-live.yml -- manual-trigger (workflow_dispatch only) live audit using a repo secret GHORG_AUDIT_TOKEN (read-only fine-grained PAT). Guarded by github.repository check so forks cannot use the secret; pre-flight step verifies the secret is present and exits 1 otherwise.

Coding standards used

  • Strict shell options on every authored script: errexit nounset pipefail errtrace inherit_errexit shift_verbose.
  • Long option names everywhere (--quiet, --fixed-strings, --ignore-case, --extended-regexp, --invert-match, --count, --line-number, --lines=N).
  • End-of-options separator -- on grep, head, sed, find, basename, sanitize-string calls that take patterns or potentially untrusted strings.
  • Every authored file carries ## AI-Assisted after the standard copyright + COPYING header. Neutral disclosure, no name, no fame-creep on minor contributions.
  • ASCII-only across code, commit message, and this PR description.

Live audit setup (post-merge)

  1. Mint a fine-grained PAT under the assisted-by-ai user account, resource-owner = org-ai-assisted, read-only scopes: Repository (Administration, Contents, Metadata, Webhooks) + Organization (Administration, Members, Webhooks). Expiration 7 days.
  2. Settings -> Secrets and variables -> Actions -> New repository secret GHORG_AUDIT_TOKEN.
  3. Actions tab -> "github-org tools (live audit)" -> Run workflow.
  4. To revoke: delete the secret or revoke the PAT.

Test plan

  • Mock-API suite: 10 / 10 tests pass locally.
  • Shellcheck-clean across every authored file.
  • OpenAPI spec verified for all endpoint paths and request/response shapes.
  • Live audit run via audit-live.yml after merge; review report.
  • Once audit looks clean: flip the org-ai-assisted test config back to Kicksecure + Whonix (uncomment the production line in three files: dm-github-policy, dm-fork-sync, dm-packaging-helper-script).
  • First production dm-github-policy --dry-run, then real --apply, then manual UI flips for the SKIP-flagged settings (2FA, PAT policies, App/OAuth policies).
  • Manually confirm the bot account / mirror remote works end-to-end with dm-packaging-helper-script push flow.

https://claude.ai/code

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f42c5c804

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread usr/bin/dm-fork-sync Outdated
local owner="$1" is_source="$2"
local page=1 result body status
while : ; do
result="$(gh_api GET "/orgs/${owner}/repos?per_page=100&page=${page}&type=all")"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use account-aware repo listing endpoint

The new sync script documents assisted-by-ai as a bot account, but list_repos hardcodes the org endpoint (/orgs/${owner}/repos). For a user account this returns 404, so the initial mirror repo lookup fails and the script exits before any fork/configuration work happens. Please branch on account type (or probe once) and use /users/{owner}/repos for user accounts.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 045c0c8. detect_account_type() probes /users/{owner} (which works for both User and Organization accounts) and reads .type, then list_repos routes to /users/{owner}/repos or /orgs/{owner}/repos accordingly. Result is cached per owner. create_fork similarly omits the organization body field when the mirror is a User.


Generated by Claude Code

Comment thread usr/bin/dm-packaging-helper-script Outdated
## remote names, rather than duplicating info.
set_batch_current_package_remote_list() {
batch_current_package_remote_list=( "github-${batch_current_project_name,,}" )
batch_current_package_remote_list=( "github-${batch_current_project_name,,}" "assisted-by-ai" )
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Don’t push assisted-by-ai for unsupported source orgs

This change adds assisted-by-ai to every package’s push/fetch remote list, including the special derivative-maker repo, but dm-fork-sync only provisions mirrors from Kicksecure and Whonix. When a derivative-maker mirror branch is missing, pkg_git_push_remotes now attempts diffs/pushes against nonexistent refs/remotes/assisted-by-ai/*, causing background git failures and aborting at wait_for_git_processes. Gate this remote by project/source org (or mirror derivative-maker as well).

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipping — the premise doesn't hold for this repo layout:

  1. derivative-maker exists as Kicksecure/derivative-maker and Whonix/derivative-maker, so dm-fork-sync does mirror it (the first source org wins on the name collision in assisted-by-ai, which is the intended behavior).
  2. The "missing remote-tracking ref" case is already handled by internal_git_diff at usr/bin/dm-packaging-helper-script:678-700: it catches git's exit 128, pauses for human inspection (read -r _), then deliberately falls through and returns 0 so the caller attempts git push, which fails visibly with the same git error rather than silently skipping. That's the documented design — see the comment block at line 681-687.

So the worst case after a fresh remote add is a visible push failure (and only until the first dm-fork-sync run + pkg_git_fetch_remotes populate the tracking refs), not the silent abort the comment describes. Gating per source org would add complexity for no practical gain.


Generated by Claude Code

assisted-by-ai pushed a commit that referenced this pull request Apr 26, 2026
Codex review P1 (PR #3): list_repos hardcoded /orgs/{owner}/repos,
which 404s for User accounts. assisted-by-ai is currently a User
account, so the mirror lookup failed before any fork ran.

Add detect_account_type() that probes /users/{owner} (returns .type
"User" or "Organization") and caches the result. list_repos now
selects /users/{owner}/repos or /orgs/{owner}/repos accordingly,
including the right query parameters per endpoint.

create_fork also adapted: the forks endpoint takes `organization` only
when targeting an org. For a User mirror, omit it - the fork goes to
the authenticated user, which must therefore be MIRROR_ORG.
@assisted-by-ai assisted-by-ai force-pushed the claude/configure-repo-access-hZI4X branch from accf1ab to 049d68e Compare April 26, 2026 17:18
@assisted-by-ai assisted-by-ai changed the title add dm-fork-sync and mirror push integration add github-org-* tools, dm-fork-sync, dm-github-policy, CI mock test suite Apr 27, 2026
@assisted-by-ai assisted-by-ai force-pushed the claude/configure-repo-access-hZI4X branch 2 times, most recently from bc4ddf0 to f1e84ce Compare April 29, 2026 17:01
Three coordinated additions:
1. Generic GitHub-org tooling under usr/bin (github-org-clone,
   github-org-fork, github-org-push, github-org-set-fork-approval)
   plus the shared library github-org-lib.bsh.
2. Project-policy wrappers dm-fork-sync and dm-github-policy.
3. CI mock-API test suite under ci/, plus two GitHub Actions
   workflows (mock-tests, live audit).

Tools (usr/bin/github-org-*):

- github-org-clone <source-org> [<dest-dir>]
  Clone or fast-forward every public, non-archived, non-fork repo
  from the source into <dest-dir>/<repo-name>/. Idempotent.
- github-org-fork <source-owner> <target-owner>
  Fork every selected source repo into the target. Source/target
  may each be a User or an Organization. Optional flags configure
  target settings (--disable-issues/wiki/projects, --actions
  {enable|disable|leave}, --workflow-perms {read|write|leave}).
- github-org-push <source-dir> <target-owner>
  Push HEAD (and optionally tags) from each clone under source-dir
  to a same-named target repo. Refuses to push if missing.
- github-org-set-fork-approval <org> [<policy>]
  Set the org-level fork-PR contributor approval policy. Default
  policy: all_external_contributors.

Common defaults: skip private/archived/fork repos on the source
side; 4 parallel jobs where applicable; --dry-run, -v/--verbose,
--include/--exclude REGEX, --ssh|--https. Auth via GITHUB_TOKEN env
or chmod-600 ~/.config/github-token. No gh CLI dependency.

Shared library (usr/libexec/developer-meta-files/github-org-lib.bsh):

- Token resolution from env or chmod-600 token file. Refuses
  symlinks (defeats stat-then-read perm-check).
- ghorg_api: token via curl --config from stdin so it never appears
  in argv (no ps aux leak). Pinned to https with --proto / --proto-
  redir; bounded by --connect-timeout / --max-time; capped at
  --max-filesize 10 MB.
- Rate-limit retry on HTTP 429, 5xx, and 403-rate-limit body.
  Honors Retry-After and X-RateLimit-Reset; falls back to
  exponential backoff. GHORG_MAX_RETRIES, GHORG_MAX_BACKOFF_SECONDS.
- Mock mode: GHORG_MOCK=1 GHORG_MOCK_DIR=path serves canned
  responses from local fixtures. Missing fixture returns HTTP 599
  so tests fail loudly.
- Token-handling functions wrap with `local -; set +x` so an
  enclosing set -x cannot leak the secret.
- Account-type probe (User vs Organization) with caching;
  paginated listing routed to the right endpoint. For User owners
  with --include-private, switches to /user/repos?affiliation=owner
  and refuses if the listed user is not the auth user.
- ghorg_repo_lookup: 0 = exists, 1 = 404, 2 = other failure.
- ghorg_authenticated_user (cached) + ghorg_repo_parent (for
  cross-source name-collision detection).
- ghorg_validate_name: every API-supplied identifier must match
  ^[A-Za-z0-9._-]+$, length-capped, no leading dash, no embedded
  "..", not equal to . / .. / .git. Allows .github / .gitignore.
- Safe display via sanitize-string for any error message that
  includes API-derived bytes.

Project-policy wrappers (usr/bin/):

- dm-fork-sync: thin wrapper around github-org-fork. Mirrors
  source orgs into the bot account.
- dm-github-policy: applies project-wide GitHub security policy.
  Modes: default --apply, --dry-run, --audit (read-only). End-of-
  run summary with ok / warn / dry-run / skip counts.

  Settings applied at the org level:
  - Fork-PR contributor approval = all_external_contributors
  - Default workflow GITHUB_TOKEN permissions = read; cannot
    create or approve pull requests
  - Allowed Actions = github-owned + verified creators
  - Member policy: default-perm=read, no member create
  - Code Security Configurations API: a single org-level
    configuration enables secret scanning, push protection,
    Dependabot alerts and security updates, dependency graph,
    explicitly disables CodeQL default setup and private
    vulnerability reporting; attached to all existing repos and
    set as the default for new repos. Replaces the deprecated
    PATCH /orgs/{org} security-defaults fields and the
    /orgs/{org}/{product}/enable_all family.
  - Default-branch ruleset: block force-push, block deletion,
    require signed commits, bypass_actors=[] (no admin override).
    Deliberately does not include the pull_request rule
    (maintainers push directly to master).
  - Tag ruleset: block deletion, block re-pointing, require
    signed tags, bypass_actors=[].

  Per-repo settings (no org-wide endpoint exists):
  - has_wiki=false, has_issues=true.

  SKIP+notify with UI URL pointers for: 2FA enforcement, PAT
  policies, App/OAuth policies. Verified against the GitHub
  OpenAPI spec to be UI-only.

  --audit reports current state of every applied setting plus
  members lacking 2FA, existing rulesets, fine-grained PAT
  activity, installed GitHub Apps, org-level webhooks lacking a
  secret (Scorecard "Webhooks" Critical-risk gap), and
  .github/dependabot.yml file presence per repo (Scorecard
  "Dependency-Update-Tool" file-based detection).

  Help text documents the OSSF Scorecard mapping per check
  (Branch-Protection ceiling = Tier 1 deliberate; Code-Review,
  CodeQL, Maintained = out of scope; etc.).

Test-only configuration: ORGS in dm-github-policy and SOURCE_ORGS
in dm-fork-sync currently target ('org-ai-assisted'). The
production list ('Kicksecure', 'Whonix') is preserved as a
single-line comment above the active line for easy revert.
dm-packaging-helper-script: same comment-out pattern; the
batch_current_package_remote_list mirror is org-ai-assisted (was
assisted-by-ai); the corresponding remote-add line is added in
pkg_git_remotes_add.

CI infrastructure (ci/ + .github/workflows/):

- ci/fixtures/: 32 canned responses for the test org, the auth
  user, /user/repos, every org-policy GET that audit and dry-run
  paths hit, plus webhooks list and per-repo dependabot.yml
  contents lookups.
- ci/tests/test_*.sh: 10 mock-API tests covering github-org-*
  arg parsing, dry-run flow, --get/--audit, the validator's
  attack-pattern rejection, the rate-limit-wait header parser,
  and a shellcheck-clean enforcement test that subjects every
  authored file to project-wide .shellcheckrc rules.
- ci/test-github-org-tools.sh: runs every test_*.sh; quiet on
  success, prints captured logs on failure. Refuses to run
  outside CI=true.
- ci/install-genmkfile.sh + ci/install-helper-scripts.sh:
  standalone install scripts for CI; refuse to run outside
  CI=true.
- ci/probe-live-unauth.sh: best-effort live unauth smoke against
  the real REST API; exits 0 with a skip message if the rate-
  limit bucket is exhausted.
- .github/workflows/test-github-org-tools.yml: runs the suite
  on push to master/claude/** and on every PR. Container
  debian:trixie. paths: filter so it only runs on github-org-*,
  dm-* tools, github-org-lib, ci/, or the workflow itself.
  permissions: contents: read only.
- .github/workflows/audit-live.yml: manual-trigger
  (workflow_dispatch only) live audit using a repo secret
  GHORG_AUDIT_TOKEN (read-only fine-grained PAT). Guarded by
  github.repository check so forks cannot use the secret;
  pre-flight step verifies the secret is present and exits 1
  otherwise.

Strict shell options on every authored script:
  set -o errexit / nounset / pipefail / errtrace
  shopt -s inherit_errexit / shift_verbose

Long option names everywhere (--quiet, --fixed-strings,
--ignore-case, --extended-regexp, --invert-match, --count,
--line-number, --lines=N). End-of-options separator (--) on
grep, head, sed, find, basename, sanitize-string calls that
take patterns or potentially untrusted strings.

Header on every authored file:
  ## Copyright (C) 2026 - 2026 ENCRYPTED SUPPORT LLC <...>
  ## See the file COPYING for copying conditions.

  ## AI-Assisted

The "AI-Assisted" marker discloses substantial AI involvement
without naming any specific tool or person, so the disclosure
does not invite per-file fame creep on minor contributions.

Defensive permissions on the workflow files: actions: write
intentionally not granted. actions/upload-artifact@v7 was tested
empirically against contents: read alone (run id 24967508066);
the artifact backend uses the workflow run's own identity, not
the GITHUB_TOKEN's permission scope.
@assisted-by-ai assisted-by-ai force-pushed the claude/configure-repo-access-hZI4X branch from 521e080 to d5a8747 Compare April 30, 2026 21:09
@assisted-by-ai assisted-by-ai changed the title add github-org-* tools, dm-fork-sync, dm-github-policy, CI mock test suite add github-org-* tools, dm-fork-sync, dm-github-policy, CI mock + live-audit Apr 30, 2026
@assisted-by-ai assisted-by-ai merged commit 0b42fa3 into master Apr 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants