Skip to content

docs: explain org-member review flow for external contributor PRs #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sayt-0 merged 2 commits into from
Jun 25, 2026
+27 −0
Merged

docs: explain org-member review flow for external contributor PRs #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e53c701
docs: explain org-member review flow for external contributor PRs
Sayt-0 Jun 24, 2026
d4b7ac3
docs: address review feedback on external contributor review flow
Sayt-0 Jun 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ cd tests
- Update docs if needed
- Be responsive to feedback

## Automated PR Review

This repo uses the `docker-agent` AI reviewer on pull requests. How a review is triggered depends on who opened the PR:

- **Org members:** request a review from `docker-agent` in the PR sidebar (Reviewers → add `docker-agent`). The review starts automatically once requested.
- **External / fork contributors:** the same request step applies, but GitHub gates Actions on these PRs, so an org member must also approve the workflow run first:
1. **Approve the workflow run.** GitHub holds workflows on PRs from first-time and external contributors until a maintainer clicks **Approve and run workflows**.
2. **Request the review.** In the PR sidebar, under **Reviewers**, add `docker-agent`. The review starts and appears as a check run.

No special commands or workflow inputs are needed, and an external contributor cannot trigger a review of their own PR. The deprecated `/review` comment still works, but requesting `docker-agent` as a reviewer is the supported path. See the [PR Review documentation](review-pr/README.md#external-and-fork-contributor-prs) for the full flow.

## Security Issues

**Do not** open public issues for vulnerabilities. Contact maintainers privately first.
Expand Down
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,8 @@ jobs:

For comprehensive documentation on setting up AI-powered PR reviews, including features like automatic reviews, requesting a review from `docker-agent`, feedback learning, and customization options, see the **[PR Review documentation](review-pr/README.md)**.

For external or fork contributor PRs, an org member approves the workflow run and then requests a review from `docker-agent` via GitHub's native review request UI (no special commands or workflow inputs required). See [External and fork contributor PRs](review-pr/README.md#external-and-fork-contributor-prs).

### Manual Trigger with Inputs

```yaml
Expand Down
14 changes: 14 additions & 0 deletions review-pr/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,20 @@ pull_request:
```
Adds `synchronize` to also trigger on every push to the PR branch. Opt in if your team wants the reviewer to automatically re-examine every update, at the cost of more workflow runs.

### External and fork contributor PRs

> [!NOTE]
> The requester-authorized path below requires the `check-org-membership` update from PR #16 (merge that PR first). Until it ships, membership is checked against the PR author rather than the requesting org member, so requesting `docker-agent` on an external or fork PR is silently skipped.

Auto-review only runs on PRs authored by org members. A PR opened by an external or fork contributor is **not** reviewed automatically. To get one reviewed, an org member drives it through GitHub's native UI in two steps:

1. **Approve the workflow run.** For PRs from first-time and external contributors, GitHub holds all Actions runs until a maintainer approves them (governed by the repository's `Settings` → `Actions` → `General` fork-PR approval policy). Click **Approve and run workflows** on the PR; until then nothing runs, including the PR review trigger.
2. **Request a review from `docker-agent`.** In the PR sidebar, under **Reviewers**, add `docker-agent`. This fires a `review_requested` event and starts the review, shown as a check run (if `checks: write` is granted).

That is the entire flow. **No special commands or workflow inputs are needed**: not the deprecated `/review` comment, not `workflow_dispatch`, and no caller-side configuration. The review is authorized by the requesting org member rather than the PR author, which is what lets an external contributor's PR be reviewed on demand. The request is safe by construction: GitHub only lets users with triage or write access request a reviewer, and the reusable workflow verifies org membership before any review work runs. An external contributor cannot trigger a review of their own PR.

@derekmisler derekmisler Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: this documents behavior that doesn't exist yet on main without PR #16. in the current check-org-membership/index.ts, for PR_SOURCE=event the username is COMMENT_AUTHOR, which is "" on a pull_request event (github.event.comment is null). passing an empty string to the membership check returns 404 → is_member=false → review silently skipped. the REQUESTER env var and evaluateMembership/resolveReviewRequester logic are all added by PR #16.

your PR description says "merge #16 first", which is the right call. but that's engineering metadata in the PR body, not user-facing. if merge order slips, anyone following these docs hits a two-step flow that quietly does nothing.

recommend adding a visible caveat in the doc itself, e.g.:

> [!NOTE]
> This flow requires the `check-org-membership` update shipping with PR #16. Merge that PR before this one.

@Sayt-0 Sayt-0 Jun 25, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in d4b7ac3 as a > [!NOTE] at the top of the section: the requester-authorized path requires the check-org-membership update from PR #16; until it ships, membership is checked against the PR author rather than the requester, so the review is silently skipped. This makes the merge-order dependency visible in the doc itself rather than only in the PR description.


To re-run the review after new commits, re-request the review from `docker-agent` in the sidebar (the refresh icon next to their name).

### Customizing

```yaml
Expand Down