From e21e9d4502fe94880b2da2e4e9621380fdd78ff0 Mon Sep 17 00:00:00 2001 From: Yuhan Lei Date: Tue, 19 May 2026 13:25:35 +0800 Subject: [PATCH 1/3] docs(github): tighten PR and task templates against agent misuse PR #748 surfaced a template ambiguity: the labeler bot auto-applied routing/priority labels but not the required type label, the agent rewrote the checklist text to "Label bot should apply type/routing/ priority labels; no manual labels were specified per maintainer instruction" instead of adding the missing `bug` label, and the human maintainer had to add `bug` minutes later before validation ran. Five rounds of crosscheck (Claude Opus + Codex) found and addressed: - Label checklist conflated bot-applied vs author-applied labels and had an "or I requested maintainer labeling" escape hatch. - All checklist items were first-person past-tense assertions with no explicit immutability guard; the agent treated the lines as editable. - Human Review Status was a free-text contract with a loose "or not required" escape hatch. - How To Verify example block had no machine-spottable marker; an agent could leave the example verbatim. - Task template Execution mode options had overlapping prose and uneven parallelism. The new PR template splits labels into three explicit author/bot rows, adds a top-of-file policy comment plus a "How to use this checklist" blockquote, converts Human Review Status into a strict three-option enum, adds a `replace-before-submit` HTML sentinel to the example block, and tags genuinely conditional items with **(conditional)**. The new Task template Execution mode rewrites each option as an explicit declarative directive ("the agent must not... until..." / "the agent must post the plan as an issue comment..." / "the agent makes the requested changes... must not push directly to dev"). Round-5 crosscheck returned "None" for both code findings and design alternatives from both reviewers. Remaining design-level work (CI lint enforcement, issue-template `area` dropdown restructure) is deferred to a follow-up PR. --- .github/ISSUE_TEMPLATE/03-task.yml | 8 +++--- .github/pull_request_template.md | 42 +++++++++++++++++++++--------- 2 files changed, 33 insertions(+), 17 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/03-task.yml b/.github/ISSUE_TEMPLATE/03-task.yml index ba98722c4..b5dc9e1fd 100644 --- a/.github/ISSUE_TEMPLATE/03-task.yml +++ b/.github/ISSUE_TEMPLATE/03-task.yml @@ -61,10 +61,10 @@ body: id: execution_mode attributes: label: Execution mode - description: Pick the safest starting mode. + description: Pick the safest starting mode. Each option defines exactly what the agent must do next on this issue. options: - - Human review required before code changes - - Agent should investigate and propose a plan first - - Agent can implement directly + - Wait for human approval — the agent must not write code or open a PR until a human leaves an explicit "approved" comment on this issue + - Investigate and propose a plan first — the agent must post the plan as an issue comment and wait for an explicit "approved" comment before writing code or opening a PR + - Implement and open a PR — the agent makes the requested changes on a feature branch and opens a PR following the standard PR template without requiring prior issue-comment approval. The agent must not push directly to dev, and the resulting PR still requires human approval to merge. validations: required: true diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 46d482d79..392b5403d 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,3 +1,5 @@ + + ## Summary Describe what changed. @@ -12,7 +14,11 @@ Link the issue if there is one. ## Human Review Status -Pending. A human should make the final merge decision after reviewing the final diff and verification evidence. +Replace this paragraph with exactly one of the following lines: + +- `Pending` — waiting for a human reviewer to approve. +- `Approved by @` — name the human reviewer who signed off after the final diff and verification evidence were ready. +- `Not required: ` — only for automated release bumps, dependency updates, or other low-risk bot-authored PRs. State the reason. Do not select this for any change an agent authored on behalf of a human request. ## Review Focus @@ -20,13 +26,16 @@ What should reviewers pay the most attention to? ## Risk Notes -Call out behavior, data, permissions, dependencies, platform, or migration risks. Write "None" if there are no special risks. +Call out behavior, data, permissions, dependencies, platform, or migration risks. If you left any **(conditional)** checklist item below unticked, list each skipped item here with a one-line reason. Write "None" only when there are no risks AND no skipped items. ## How To Verify List the targeted checks you ran and the key result for each one. Prefer the smallest checks that cover the changed surface. Include the result, not just the command. + + ```text +# EXAMPLE — delete this block and replace with your real verification results YAML parse: ok for both issue forms Diff check: no whitespace errors Focused tests: 47 passed @@ -38,14 +47,21 @@ Required for visible UI changes. ## Checklist -- [ ] Human review status is stated above as pending, approved, or not required -- [ ] I linked the related issue, or stated why there is no issue -- [ ] This PR has exactly one type label (`bug`, `enhancement`, `task`, or `documentation`), at least one primary routing label (`app`, `ui`, `platform`, `harness`, or `ci`), and exactly one priority label (`P0` to `P3`), or I requested maintainer labeling -- [ ] I described the review focus and any meaningful risks -- [ ] I listed the relevant verification steps and the key result for each -- [ ] I did not introduce unrelated refactors, dependencies, generated files, or file changes beyond the stated scope -- [ ] I manually checked visible UI or copy changes when needed, with screenshots or recordings -- [ ] I considered macOS and Windows impact for platform, packaging, updater, signing, paths, shell, or permissions changes -- [ ] I called out docs, release notes, dependencies, permissions, credentials, deletion behavior, generated content, or local file changes when relevant -- [ ] I reviewed the final diff for unrelated changes and suspicious dependency changes -- [ ] I am targeting `dev`, and my PR title and commit messages use Conventional Commits in English +> **How to use this checklist:** +> - Tick a box by replacing `[ ]` with `[x]`. Do not edit, add, or remove items. +> - The bot-applied label items can only be honestly ticked AFTER the PR is opened and the labeler / priority-triage bots have run — return to the PR description and tick them then. +> - Most items are required. The few that are conditional are explicitly marked **(conditional)**; for those, leave unticked if they truly do not apply and explain why in Risk Notes. All other items must be ticked before requesting human review. + +- [ ] **Type label** — this PR carries exactly one of `bug`, `enhancement`, `task`, `documentation`. Type labels are author-added; the labeler bot does NOT assign them. Add the label in the GitHub UI, then tick this. +- [ ] **Routing labels** — this PR carries at least one of `app`, `ui`, `platform`, `harness`, `ci`. The labeler bot assigns these on PR open based on changed paths. Confirm the bot's choice (or override if wrong), then tick this. +- [ ] **Priority label** — this PR carries exactly one of `P0`, `P1`, `P2`, `P3`. The priority-triage bot suggests one on PR open. Confirm or override, then tick this. +- [ ] Human Review Status above is set to `Pending`, `Approved by @`, or `Not required: ` (default is `Pending`; "not required" is restricted to bot-authored low-risk PRs). +- [ ] I linked the related issue, or stated in Summary why there is no issue. +- [ ] I described the review focus and any meaningful risks. +- [ ] I replaced the example block in How To Verify with the real verification steps and the key result for each. +- [ ] I did not introduce unrelated refactors, dependencies, generated files, or file changes beyond the stated scope. +- [ ] **(conditional)** I manually checked visible UI or copy changes when needed, with screenshots or recordings. Leave unticked only if no visible UI or copy changed. +- [ ] **(conditional)** I considered macOS and Windows impact for platform, packaging, updater, signing, paths, shell, or permissions changes. Leave unticked only if no platform/packaging surface was touched. +- [ ] **(conditional)** I called out docs, release notes, dependencies, permissions, credentials, deletion behavior, generated content, or local file changes when relevant. Leave unticked only if none of those surfaces was touched. +- [ ] I reviewed the final diff for unrelated changes and suspicious dependency changes. +- [ ] I am targeting `dev`, and my PR title and commit messages use Conventional Commits in English. From f896958380d0a92354ef3c6e572cc3a64172da50 Mon Sep 17 00:00:00 2001 From: Yuhan Lei Date: Tue, 19 May 2026 13:30:32 +0800 Subject: [PATCH 2/3] ci(labeler): route all .github/ changes to ci, not only workflows Templates and issue forms under .github/ITS_TEMPLATE/ and the top-level PR template are CI/process infrastructure that should route the same way as workflow files. Without this, a PR that only touches .github/pull_request_template.md or .github/ISSUE_TEMPLATE/*.yml receives no routing label from the labeler bot, and the manual override is stripped by sync-labels on the next pr-triage run. The task rule keeps the narrower .github/workflows/** scope: task is a type label, and per the updated PR template, type labels are author-applied, not bot-applied. Workflow PRs retain the existing task auto-application as a historical convenience; this can be revisited in a follow-up if the inconsistency proves confusing. --- .github/labeler.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/labeler.yml b/.github/labeler.yml index 5b52405a1..235d3ff3a 100644 --- a/.github/labeler.yml +++ b/.github/labeler.yml @@ -1,7 +1,7 @@ ci: - changed-files: - any-glob-to-any-file: - - ".github/workflows/**" + - ".github/**" task: - changed-files: From 3c30cd83fc139d616d61b87e4555e6e83020c96e Mon Sep 17 00:00:00 2001 From: Yuhan Lei Date: Tue, 19 May 2026 14:00:43 +0800 Subject: [PATCH 3/3] ci(labeler): keep .github/workflows/** explicit in ci rule Restore the explicit .github/workflows/** glob alongside the broader .github/** glob. The pr-triage contract test at packages/opencode/test/github/pr-triage-workflow.test.ts:57 uses literal-string matching to assert that workflow files are routed to the ci label, so dropping the workflows glob broke the test even though .github/** subsumes it semantically. Keeping both globs preserves the test contract and documents the intent that workflow files are first-class CI infra while other .github/ files (templates, labeler.yml itself, dependabot config) are CI process. --- .github/labeler.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/labeler.yml b/.github/labeler.yml index 235d3ff3a..fa39c467f 100644 --- a/.github/labeler.yml +++ b/.github/labeler.yml @@ -1,6 +1,7 @@ ci: - changed-files: - any-glob-to-any-file: + - ".github/workflows/**" - ".github/**" task: