Skip to content

feat(ops): add recurring workflow audit and runbook - #486

Merged
ss-o merged 6 commits into
mainfrom
codex/recurring-operations-audit
Jul 27, 2026
Merged

feat(ops): add recurring workflow audit and runbook#486
ss-o merged 6 commits into
mainfrom
codex/recurring-operations-audit

Conversation

@ss-o

@ss-o ss-o commented Jul 27, 2026

Copy link
Copy Markdown
Member

Summary

  • add a read-only Ruby CLI that inventories scheduled workflows across active organization repositories
  • include synthetic fixtures and tests for pagination, disabled workflows, malformed data, API failures, private redaction, and private output permissions
  • add the canonical recurring-operations runbook with the retain, event-driven, reusable, manual, and remove classifications
  • route the runbook through the public instruction manifest, README, and policy validator

Why

The organization currently has repeated scheduled maintenance with inconsistent ownership, permissions, timing, failure handling, and retirement criteria. Issue #485 needs a reproducible evidence source and one canonical operating procedure before repository changes or agent schedules are introduced.

Impact

Maintainers and supported agents can generate the same inventory, distinguish time-driven work from event-driven work, and prepare reviewable recommendations. This PR does not create scheduled tasks, modify repository settings, dispatch workflows, or mutate other repositories.

Instruction impact review

  1. Classification: The runbook is canonical operational guidance. The manifest entry is instruction routing. The audit CLI and validator test provide executable evidence.
  2. Consumers: Codex, Claude Code, Copilot, maintainers, and public policy validation receive the recurring-operations route. Gemini CLI is intentionally excluded under the maintainer's deprecation policy.
  3. Canonical owner: runbooks/recurring-operations.md owns recurring-operation decisions; the audit script owns reproducible inventory collection. Existing specialist runbooks retain their domains.
  4. Duplication and contradiction: The new runbook links existing organization review, triage, label, dependency, tracker, release, and learning-capture owners instead of duplicating their procedures. Review found no conflicting canonical owner.
  5. Manifest impact: .github/instruction-surfaces.json adds exactly one recurring-operations route and the README exposes both the runbook and audit command.
  6. Runtime delivery: Every supported runtime receives the required route through the public manifest. No optional hook, skill, or local memory owns a mandatory rule.
  7. Generated output and limits: Public policy validation and all 69 policy tests pass. The audit suite verifies public redaction and requires private JSON output to use an explicit mode-0600 file.

Safety boundaries

  • inventory collection uses GitHub GET requests only
  • public Markdown output omits private repository records
  • private JSON requires an explicit output path and mode 0600
  • unavailable API data becomes a structured error instead of stale inference
  • the workflow remains draft-only and does not authorize scheduled-task creation or repository mutation

Validation

  • ruby scripts/test-audit-scheduled-workflows.rb: 26 tests passed
  • python3 -m unittest scripts/test_validate_agent_policy.py -v: 69 tests passed
  • python3 scripts/validate-agent-policy.py: passed
  • ruby -c scripts/audit-scheduled-workflows.rb: syntax OK
  • ruby -c scripts/test-audit-scheduled-workflows.rb: syntax OK
  • python3 -m json.tool .github/instruction-surfaces.json: passed
  • git diff --check origin/main...HEAD: passed

Relates to #485 and #412.

Agent handoff

Status: Ready for review
Repository: z-shell/.github
Branch/PR: codex/recurring-operations-audit
Tracker/Issue: #485

Current state

  • The reviewed inventory CLI and recurring-operations runbook are published with four focused review follow-up commits.
  • No organization schedule, workflow, setting, or other repository was changed.
  • The wider program remains in progress, including the August 1 organization-review cycle and separately authorized repository changes.

Verification

  • Audit suite: 26 passed
  • Policy suite: 69 passed
  • Public policy validator: passed
  • Ruby syntax, JSON parsing, and diff checks: passed

Blockers

  • None for review of this draft PR.
  • The weekly scheduled-review decision remains time-gated until the third pilot cycle on or after 2026-08-01.

Next steps

  1. Review this draft PR and address actionable feedback.
  2. Merge only after maintainer approval.
  3. Continue issue ops: rationalize scheduled workflows and recurring organization maintenance #485 through the remaining classification, implementation, and scheduled-task gates.

Copilot AI review requested due to automatic review settings July 27, 2026 21:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a reproducible, read-only inventory tool and canonical runbook to standardize how the org audits and classifies recurring/scheduled automation, with policy-manifest routing and tests to keep the guidance discoverable and enforced.

Changes:

  • Added scripts/audit-scheduled-workflows.rb Ruby CLI to inventory scheduled workflows via read-only GitHub API calls, with JSON/Markdown output and private-redaction safety boundaries.
  • Added a canonical runbooks/recurring-operations.md runbook defining the classification model (retain/event-driven/reusable/manual/remove) and draft-only review rules.
  • Added fixtures/tests and wired the runbook into the public instruction manifest (plus README documentation) so supported runtimes can route to it.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
scripts/audit-scheduled-workflows.rb New read-only CLI that inventories scheduled workflows and renders JSON/Markdown outputs.
scripts/test-audit-scheduled-workflows.rb New Ruby test harness covering parsing, pagination, disabled workflows, redaction, and error handling.
scripts/test_validate_agent_policy.py Adds a manifest-routing assertion for the new recurring-operations runbook surface.
scripts/fixtures/scheduled-workflows/active-utc.yml Fixture for a UTC scheduled workflow with permissions/concurrency and reusable call.
scripts/fixtures/scheduled-workflows/active-iana-timezone.yml Fixture for explicit IANA timezone handling.
scripts/fixtures/scheduled-workflows/comment-only-workflow.yml Fixture for comment-only workflow content (no active triggers).
scripts/fixtures/scheduled-workflows/directory-api-error.yml Fixture simulating malformed directory API response object.
scripts/fixtures/scheduled-workflows/disabled-inactivity.yml Fixture for disabled_inactivity workflow state.
scripts/fixtures/scheduled-workflows/job-level-controls.yml Fixture for job-level permissions/concurrency detection.
scripts/fixtures/scheduled-workflows/missing-workflow-directory.yml Fixture for missing workflows directory (404).
scripts/fixtures/scheduled-workflows/multiple-cron.yml Fixture for multiple cron schedules.
scripts/fixtures/scheduled-workflows/private-no-schedule.yml Fixture for private repo with only workflow_dispatch (no schedule).
scripts/fixtures/scheduled-workflows/private-redaction.yml Fixture for private repo schedule used to validate redaction / private-output rules.
scripts/fixtures/scheduled-workflows/rate-limit-error.yml Fixture for API rate-limit error behavior.
scripts/fixtures/scheduled-workflows/reusable-workflow-caller.yml Fixture for detecting uses: reusable workflow calls.
scripts/fixtures/scheduled-workflows/top-level-controls.yml Fixture for top-level permissions/concurrency detection.
runbooks/recurring-operations.md New canonical runbook for classifying and reviewing recurring operations and scheduled workflows.
.github/README.md Documents how to run the scheduled workflow inventory and links to the runbook.
.github/instruction-surfaces.json Adds a required instruction surface routing recurring-operations guidance to supported consumers.

Comment thread .github/instruction-surfaces.json
Comment thread scripts/test_validate_agent_policy.py
Comment thread scripts/audit-scheduled-workflows.rb
Comment thread scripts/audit-scheduled-workflows.rb
Comment thread scripts/audit-scheduled-workflows.rb
Copilot AI review requested due to automatic review settings July 27, 2026 21:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Comment thread scripts/audit-scheduled-workflows.rb
Comment thread scripts/audit-scheduled-workflows.rb
Copilot AI review requested due to automatic review settings July 27, 2026 22:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Comment thread scripts/audit-scheduled-workflows.rb
Copilot AI review requested due to automatic review settings July 27, 2026 22:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Comment thread scripts/audit-scheduled-workflows.rb
Comment thread scripts/audit-scheduled-workflows.rb
Copilot AI review requested due to automatic review settings July 27, 2026 22:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Comment thread scripts/audit-scheduled-workflows.rb
@ss-o
ss-o marked this pull request as ready for review July 27, 2026 22:24
@ss-o
ss-o requested a review from a team as a code owner July 27, 2026 22:24
@ss-o
ss-o enabled auto-merge (squash) July 27, 2026 22:25
@ss-o
ss-o disabled auto-merge July 27, 2026 22:48
@ss-o
ss-o merged commit 3b9272d into main Jul 27, 2026
9 checks passed
@ss-o
ss-o deleted the codex/recurring-operations-audit branch July 27, 2026 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants