Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 3 additions & 9 deletions .github/workflows/qa-changes-by-openhands.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,11 @@
---
# EXPERIMENTAL: Automated QA validation of PR changes using OpenHands.
# Automated QA validation of PR changes using OpenHands.
#
# Unlike pr-review (which reads diffs and posts code-review comments),
# this workflow actually runs the code — setting up the environment,
# executing tests, exercising changed behavior, and posting a structured
# QA report as a PR comment.
#
# This is an early experiment; expect rough edges. The plugin source is
# pinned to the extensions feature branch while we iterate.
name: QA Changes by OpenHands [experimental]
name: QA Changes by OpenHands

on:
pull_request:
Expand Down Expand Up @@ -39,16 +36,13 @@ jobs:
timeout-minutes: 30
steps:
- name: Run QA Changes
# EXPERIMENTAL: pointing at feature branch while iterating
uses: OpenHands/extensions/plugins/qa-changes@feat/qa-changes-plugin
uses: OpenHands/extensions/plugins/qa-changes@main
with:
llm-model: litellm_proxy/claude-sonnet-4-5-20250929
llm-base-url: https://llm-proxy.app.all-hands.dev
max-budget: '10.0'
timeout-minutes: '30'
max-iterations: '500'
# EXPERIMENTAL: use the feature branch of extensions
extensions-version: feat/qa-changes-plugin
llm-api-key: ${{ secrets.LLM_API_KEY }}
github-token: ${{ secrets.OPENHANDS_BOT_GITHUB_PAT_PUBLIC }}
lmnr-api-key: ${{ secrets.LMNR_SKILLS_API_KEY }}
5 changes: 1 addition & 4 deletions .github/workflows/qa-changes-evaluation.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: QA Changes Evaluation [experimental]
name: QA Changes Evaluation

# This workflow evaluates how well QA validation performed.
# It runs when a PR is closed to assess QA effectiveness.
Expand Down Expand Up @@ -48,14 +48,11 @@ jobs:
echo "No trace file found for PR #$PR_NUMBER - skipping evaluation"
fi

# EXPERIMENTAL: pinned to feature branch while qa-changes plugin is in development.
# Switch to @main (and remove ref:) once the plugin is merged.
- name: Checkout extensions repository
if: steps.check-trace.outputs.trace_exists == 'true'
uses: actions/checkout@v6
with:
repository: OpenHands/extensions
ref: feat/qa-changes-plugin
path: extensions

- name: Set up Python
Expand Down
Loading