Skip to content

Commit 27deb73

Browse files
Add PR-blocking CI for QNN model tests, operator/model testsuite, and pass unit tests (#18854)
Summary: OSS PRs that change QNN backend APIs can pass external CI but break internal CI (e.g., D99197179 had hundreds of internal failures). The existing `test-qnn-models-linux` job in pull.yml was an empty placeholder, the backend test suite (operators + models) only ran on nightly, and `test_passes.py` had no external CI coverage. Wire up three new PR-blocking CI jobs in pull.yml: 1. `test-qnn-models-linux` — runs model export tests (mv2, mv3, dl3) via test_model.sh, replacing the empty placeholder 2. `test-qnn-testsuite-linux` — runs the standardized backend test suite (operators + models with flow_qnn) via _test_backend.yml reusable workflow 3. `test-qnn-passes-linux` — runs QNN AOT pass unit tests (test_passes.py) Differential Revision: D100708421
1 parent 4b7facb commit 27deb73

File tree

1 file changed

+45
-2
lines changed

1 file changed

+45
-2
lines changed

.github/workflows/pull.yml

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -857,6 +857,8 @@ jobs:
857857
id-token: write
858858
contents: read
859859
strategy:
860+
matrix:
861+
model: [mv2, mv3, dl3]
860862
fail-fast: false
861863
with:
862864
runner: linux.2xlarge
@@ -868,9 +870,50 @@ jobs:
868870
# The generic Linux job chooses to use base env, not the one setup by the image
869871
CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]")
870872
conda activate "${CONDA_ENV}"
873+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh --build-tool cmake
874+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-qnn-deps.sh
875+
PYTHON_EXECUTABLE=python bash .ci/scripts/build-qnn-sdk.sh
876+
PYTHON_EXECUTABLE=python bash .ci/scripts/test_model.sh ${{ matrix.model }} "cmake" "qnn"
877+
878+
test-qnn-testsuite-linux:
879+
name: test-qnn-testsuite-linux
880+
permissions:
881+
id-token: write
882+
contents: read
883+
uses: ./.github/workflows/_test_backend.yml
884+
with:
885+
backend: qnn
886+
flows: '["qnn"]'
887+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
888+
timeout: 120
889+
run-linux: true
890+
runner-linux: linux.2xlarge
891+
892+
test-qnn-passes-linux:
893+
name: test-qnn-passes-linux
894+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
895+
permissions:
896+
id-token: write
897+
contents: read
898+
strategy:
899+
fail-fast: false
900+
with:
901+
runner: linux.2xlarge
902+
docker-image: ci-image:executorch-ubuntu-22.04-qnn-sdk
903+
submodules: 'recursive'
904+
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
905+
timeout: 30
906+
script: |
907+
# The generic Linux job chooses to use base env, not the one setup by the image
908+
CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]")
909+
conda activate "${CONDA_ENV}"
910+
911+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-linux.sh --build-tool cmake
912+
PYTHON_EXECUTABLE=python bash .ci/scripts/setup-qnn-deps.sh
913+
PYTHON_EXECUTABLE=python bash .ci/scripts/build-qnn-sdk.sh
871914
872-
# placeholder for running test_qnn_delegate.py, can use matrix such that we can trigger different jobs, refers to test-llama-runner-qnn-linux
873-
# reminder: make sure each job runs fast
915+
# Run QNN pass unit tests
916+
pytest -xvs backends/qualcomm/tests/test_passes.py
874917
875918
test-phi-3-mini-runner-linux:
876919
name: test-phi-3-mini-runner-linux

0 commit comments

Comments
 (0)