Skip to content

[scout] Add automation prompt contract check #85

Description

@s-hiraoku

Summary

Add a small default-branch contract check for the Codex automation source prompts so role boundaries, issue prefixes, creation limits, and source-path conventions stay aligned with the organization automation docs.

Target repository and affected paths

  • Repository: kaizen-agents-org/.github
  • Likely paths/components: automations/*.prompt.md, docs/automation-roles.md, docs/repo-improvement-scout.md, docs/org-monitor.md, and a focused script or workflow check under scripts/ / .github/workflows/.

Observed evidence from default branch

  • docs/automation-roles.md defines the authoritative automation layers, allowed issue prefixes, per-repository creation limits, and PR permissions for scout, monitor, weekly readiness review, and readiness issue creator.
  • automations/README.md says this repository stores GitHub-managed source prompts and that runtime copies under $CODEX_HOME/automations are not the source of truth.
  • automations/kaizen-agents-repo-improvement-scout.prompt.md and automations/kaizen-agents-org-monitor.prompt.md duplicate critical role-boundary text, active repository scope, issue creation limits, and safety rules in prose.
  • Existing contract checks cover dogfood/shared-skill sync behavior, for example scripts/check-daily-dogfood-sync-contract.sh, but a targeted search found no equivalent script or workflow check for automation prompt role-boundary alignment.

Recommended action

Add a focused contract check that validates the automation source prompts still contain the key organization-level invariants needed by the runtime automations. At minimum, cover:

  • each managed prompt references or reads docs/automation-roles.md where appropriate;
  • scout creates only [scout] issues and does not create monitor/readiness issues or PRs;
  • monitor creates only [monitor] issues and does not act as the general improvement scout;
  • weekly readiness review creates readiness report PRs but not issues;
  • readiness issue creator reads merged default-branch readiness reports and creates [readiness-review] issues only;
  • the per-repository creation limits remain consistent with docs/automation-roles.md.

Keep the check deterministic and small, similar in spirit to scripts/check-daily-dogfood-sync-contract.sh. If exact prose matching is too brittle, prefer checking stable phrases or adding explicit machine-readable markers near the prompt headers.

Documentation basis

  • docs/automation-roles.md, "Boundaries", "Issue Prefixes", and "Limits": defines the responsibilities and limits that the source prompts must preserve.
  • docs/repo-improvement-scout.md, "Issue Creation" and "Safety Boundaries": says the scout may create [scout] issues only under bounded conditions and must not edit files or open implementation PRs.
  • docs/documentation-sources.md, "Documentation Maintenance": says automation docs should be updated when automated issue creation rules or safety boundaries change.
  • automations/README.md, "Managed Prompts": establishes automations/*.prompt.md as the GitHub-managed source for runtime automation behavior.

Duplicate-check summary

Checked open .github issues and PRs for automation prompt contract, role boundary, prompt alignment, source prompt, and automation roles keywords. Existing open .github scout issues cover README automation guidance (#83), dogfood scheduler contract coverage (#82), and docs README flow alignment (#79/#80), but none cover a deterministic source-prompt contract check.

Related issues or PRs

PR linkage requirement

The implementation PR for this issue must target the repository default branch,
include Closes #85 for same-repository work or Closes kaizen-agents-org/.github#85
for cross-repository work in the PR body, and verify
gh pr view <pr> --json baseRefName,closingIssuesReferences,isDraft before
reporting the PR ready. Do not rely on a PR title, branch name, or issue comment
as proof that GitHub will close the issue on merge.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kaizenIssue processed by Kaizen Loopkaizen:needs-humanNeeds human input before retry

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions