Skip to content

Add triage-issues-local skill for OS issue triage#170

Merged
rachaelrenk merged 5 commits into
mainfrom
docs/issue-triage-automation
Jun 3, 2026
Merged

Add triage-issues-local skill for OS issue triage#170
rachaelrenk merged 5 commits into
mainfrom
docs/issue-triage-automation

Conversation

@rachaelrenk

@rachaelrenk rachaelrenk commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Add automated issue triage for the newly open-sourced docs repo. The docs repo is already onboarded to the Oz OSS repo management agent (thanks Safia). This PR adds a triage-issue-local companion skill with docs-specific rules the core triage-issue workflow uses to specialize its behavior for this repo.

Changes

New: .agents/skills/triage-issue-local/SKILL.md

Repo-specific triage rules for the management agent, following the same pattern as the warp repo's version. Includes:

  • Heuristics — Site bugs vs content issues, URL-to-source-file mapping, source code verification for disputed docs, support question handling
  • Label taxonomy — Priority labels (priority/high, priority/medium, priority/low), status labels (triage, ready-to-implement), and template labels (bug, improve or update documentation)
  • Follow-up question limit — 2 max per triage response
  • Information to check — What to look for in the issue before asking follow-ups
  • Content structure reference — Docs section map for page identification

Labels created on the repo (already live)

  • priority/high (red) — Factually incorrect info, broken page, security-related
  • priority/medium (yellow) — Outdated content, confusing instructions, incomplete coverage
  • priority/low (green) — Typos, minor formatting, nice-to-have improvements
  • triage (gray) — Applied automatically; removed once a human reviews
  • ready-to-implement (green) — Issue is triaged and ready for someone to work on
  • improve or update documentation (blue) — Fixes the missing label referenced by the CONTENT.yml issue template

Context

Co-Authored-By: Oz oz-agent@warp.dev

Add a new skill for the Oz management agent to triage community
issues on the docs repo. The skill classifies issues, identifies
affected pages, assesses priority, checks for duplicates, applies
labels (triage + priority), and posts a structured comment with
a welcome preamble and reviewer instructions.

Labels (priority/high, priority/medium, priority/low, triage, and
improve or update documentation) were created directly on the repo.

Co-Authored-By: Oz <oz-agent@warp.dev>
@cla-bot cla-bot Bot added the cla-signed label Jun 2, 2026
@vercel

vercel Bot commented Jun 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jun 3, 2026 5:12pm

Request Review

@oz-for-oss

oz-for-oss Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

@rachaelrenk

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@rachaelrenk rachaelrenk self-assigned this Jun 2, 2026

@oz-for-oss oz-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR adds a repo-local docs issue triage skill. The main concerns are that the skill is not located/named where the triage workflow resolves repo-local companion skills, and its instructions redefine the core workflow by asking the agent to mutate GitHub directly instead of returning triage_result.json for the workflow to apply.

Concerns

  • The added skill uses .agents/skills/triage-issues-local/SKILL.md / triage-issues-local, but the dispatch resolver looks for .agents/skills/triage-issue-local/SKILL.md, so this guidance will not be loaded.
  • The skill asks the agent to apply labels and post comments directly, which conflicts with the core triage workflow's JSON-artifact contract and can produce duplicate or unauthorized GitHub mutations.
  • The duplicate search instruction uses an unpaginated gh issue list call instead of the shared repository-wide dedupe workflow, so it can miss older duplicate issues.

Verdict

Found: 0 critical, 4 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Comment thread .agents/skills/triage-issues-local/SKILL.md Outdated
Comment thread .agents/skills/triage-issues-local/SKILL.md Outdated
Comment thread .agents/skills/triage-issues-local/SKILL.md Outdated
Comment thread .agents/skills/triage-issues-local/SKILL.md Outdated
…nation

- Rename triage-issues-local -> triage-issue-local (singular) to match
  the management agent's resolver convention
- Switch from direct label/comment application to triage_result.json
  artifact output, consistent with the review-pr workflow pattern
- Fix unpaginated gh issue list by adding --limit 100
- Add JSON validation step
- Update example to show JSON schema instead of raw markdown comment

Co-Authored-By: Oz <oz-agent@warp.dev>
@rachaelrenk

Copy link
Copy Markdown
Contributor Author

Review request — questions for @captainsafia

This PR adds a triage-issue-local skill for the Oz management agent to triage docs issues. A few open questions about how the skill integrates with the management system:

  1. triage_result.json contract — The skill currently outputs a triage_result.json file (consistent with the review.json pattern in review-pr.yml). Is this the expected artifact contract for the management agent's triage workflow, or does the agent apply labels/comments directly?

  2. Comment posting — The skill includes an issue_body field in the JSON for the workflow to post as a comment. Does the management workflow handle posting from the JSON artifact, or should the skill instruct the agent to post directly? We want to avoid duplicate comments.

  3. Skill name — We went with triage-issue-local (singular). Can you confirm this matches the resolver's expected path?

@@ -0,0 +1,117 @@
---
name: triage-issue-local

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually skip most of the content that tells the agent to read the issue and write out the triage_result.json. The OSS workflows will handle that. You can keep this one focused strictly on the rules.

The version in the warp repo is a good example of what the target structure would look like.

Rewrite triage-issue-local to match the pattern in the warp repo.
The core triage-issue skill (managed by the OSS system) handles
reading issues, writing triage_result.json, applying labels, posting
comments, and checking duplicates. This local skill only provides
repo-specific rules:

- Heuristics for docs issues (site bugs vs content, URL mapping,
  source code verification)
- Label taxonomy (priority labels, triage, ready-to-implement)
- Follow-up question limit (2 max)
- Information to check before asking follow-ups
- Content structure reference

Co-Authored-By: Oz <oz-agent@warp.dev>
@rachaelrenk

Copy link
Copy Markdown
Contributor Author

/oz-review

@oz-for-oss

oz-for-oss Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

@rachaelrenk

I'm re-reviewing this pull request in response to a review request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@oz-for-oss oz-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR adds repo-specific triage guidance for docs issues, including label taxonomy, follow-up patterns, and content structure references.

Concerns

  • The companion skill currently instructs agents to apply ready-to-implement, but the core triage-issue workflow reserves that label for maintainers and forbids local overrides from changing reserved label rules.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Comment thread .agents/skills/triage-issue-local/SKILL.md Outdated
Co-authored-by: oz-for-oss[bot] <277970191+oz-for-oss[bot]@users.noreply.github.com>
@rachaelrenk rachaelrenk merged commit 385e81d into main Jun 3, 2026
8 checks passed
@rachaelrenk rachaelrenk deleted the docs/issue-triage-automation branch June 3, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants