diff --git a/.github/workflows/pr-review-by-openhands.yml b/.github/workflows/pr-review-by-openhands.yml index 744de8a6..fdaa5d4d 100644 --- a/.github/workflows/pr-review-by-openhands.yml +++ b/.github/workflows/pr-review-by-openhands.yml @@ -45,6 +45,7 @@ jobs: llm-base-url: https://llm-proxy.app.all-hands.dev # [DEPRECATED] review-style is no longer used; standard and roasted are merged # review-style: roasted + use-sub-agents: 'true' # Matches the default; kept for clarity llm-api-key: ${{ secrets.LLM_API_KEY }} github-token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }} lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }} diff --git a/plugins/pr-review/README.md b/plugins/pr-review/README.md index c5f9ebc9..6eee790d 100644 --- a/plugins/pr-review/README.md +++ b/plugins/pr-review/README.md @@ -24,7 +24,7 @@ Then configure the required secrets (see [Installation](#installation) below). - **A/B Testing**: Support for testing multiple LLM models - **Review Context Awareness**: Considers previous reviews and unresolved threads - **Evidence Enforcement**: Optional check that PR descriptions include concrete end-to-end proof the code works, not just test output -- **Sub-Agent Delegation** *(Experimental)*: Split large PR reviews across multiple sub-agents, one per file, then consolidate findings (see [Known Limitations](#known-limitations-sub-agent-delegation)) +- **Sub-Agent Delegation**: Split large PR reviews across multiple sub-agents, one per file, then consolidate findings (see [Known Limitations](#known-limitations-sub-agent-delegation)) - **Observability**: Optional Laminar integration for tracing and evaluation ## Plugin Contents @@ -142,7 +142,7 @@ PR reviews are automatically triggered when: | `llm-base-url` | No | `''` | Custom LLM endpoint URL | | `review-style` | No | `roasted` | **[DEPRECATED]** Previously chose between `standard` and `roasted` review styles. Now ignored — the styles have been merged into a single unified skill. | | `require-evidence` | No | `'false'` | Require the reviewer to enforce an `Evidence` section in the PR description with end-to-end proof: screenshots/videos for frontend work, commands and runtime output for backend or scripts, and an agent conversation link when applicable. Test output alone does not qualify. | -| `use-sub-agents` | No | `'false'` | **(Experimental)** Enable sub-agent delegation for file-level reviews. The main agent acts as a coordinator that delegates per-file review work to `file_reviewer` sub-agents via the SDK TaskToolSet, then consolidates findings into a single PR review. Useful for large PRs with many changed files. | +| `use-sub-agents` | No | `'true'` | Enable sub-agent delegation for file-level reviews. The main agent acts as a coordinator that delegates per-file review work to `file_reviewer` sub-agents via the SDK TaskToolSet, then consolidates findings into a single PR review. Useful for large PRs with many changed files. To restore the previous single-agent behavior, set to `'false'`. | | `extensions-repo` | No | `OpenHands/extensions` | Extensions repository | | `extensions-version` | No | `main` | Git ref (tag, branch, or SHA) | | `llm-api-key` | Yes | - | LLM API key | @@ -165,14 +165,13 @@ Python dependency caching is **disabled by default**. `uv run --with ...` re-dow ## Known Limitations: Sub-Agent Delegation -The `use-sub-agents` feature is **experimental** and has the following known constraints: +The following are known constraints of the sub-agent delegation feature. These are acceptable tradeoffs for the improved review depth it provides, and none pose a security risk — in the worst case a review may be less thorough than expected, which the single-agent fallback (`use-sub-agents: 'false'`) addresses. - **LLM-driven JSON parsing**: The coordinator agent relies on the LLM to parse and merge JSON responses from sub-agents. There is no code-level validation of sub-agent output, so malformed responses may cause incomplete reviews. - **Potential information loss during consolidation**: When merging findings from multiple sub-agents, the coordinator may lose or deduplicate findings imperfectly, especially for cross-file issues. -- **No integration tests yet**: Current test coverage verifies prompt formatting only. End-to-end validation of the delegation flow requires manual workflow testing. - **Sub-agents have read-only tools**: File reviewer sub-agents have access to `terminal` and `file_editor` for inspecting full source files and surrounding context, but they cannot query the GitHub API or post reviews — only the coordinator handles GitHub interaction. -These limitations are acceptable for an opt-in experimental feature and will be addressed as the feature matures. +To opt out, set `use-sub-agents: 'false'` in your workflow. ## A/B Testing Multiple Models diff --git a/plugins/pr-review/action.yml b/plugins/pr-review/action.yml index 48211754..3f302d42 100644 --- a/plugins/pr-review/action.yml +++ b/plugins/pr-review/action.yml @@ -29,11 +29,12 @@ inputs: default: 'false' use-sub-agents: description: > - Enable sub-agent delegation for file-level reviews (experimental). + Enable sub-agent delegation for file-level reviews. When true, the agent gets the TaskToolSet and decides at runtime whether to delegate based on diff size and complexity. + Set to 'false' to restore the previous single-agent behavior. required: false - default: 'false' + default: 'true' extensions-repo: description: GitHub repository for extensions (owner/repo) required: false diff --git a/plugins/pr-review/workflows/pr-review-by-openhands.yml b/plugins/pr-review/workflows/pr-review-by-openhands.yml index 744de8a6..fdaa5d4d 100644 --- a/plugins/pr-review/workflows/pr-review-by-openhands.yml +++ b/plugins/pr-review/workflows/pr-review-by-openhands.yml @@ -45,6 +45,7 @@ jobs: llm-base-url: https://llm-proxy.app.all-hands.dev # [DEPRECATED] review-style is no longer used; standard and roasted are merged # review-style: roasted + use-sub-agents: 'true' # Matches the default; kept for clarity llm-api-key: ${{ secrets.LLM_API_KEY }} github-token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }} lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }}