Skip to content

Commit a6afb0d

Browse files
authored
[ML] Run allowlist validation in PyTorch edge pipeline (#2989)
When the PyTorch edge build triggers a test run, also invoke the Python-based allowlist validation (validate_allowlist.py) which traces live HuggingFace models with the new PyTorch version and checks every op against ALLOWED_OPERATIONS / FORBIDDEN_OPERATIONS. This ensures that if a PyTorch upgrade introduces new TorchScript operations for any supported model architecture, the pipeline fails with a clear message before the change reaches a release. Made-with: Cursor
1 parent d3df09c commit a6afb0d

1 file changed

Lines changed: 24 additions & 1 deletion

File tree

.buildkite/scripts/steps/run_tests.sh

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,27 @@ else
105105
-P cmake/run-all-tests-parallel.cmake || TEST_OUTCOME=$?
106106
fi
107107

108+
# --- PyTorch allowlist validation ---
109+
# When triggered from the PyTorch edge pipeline, run the Python-based
110+
# allowlist validation which traces live HuggingFace models with the
111+
# new PyTorch version and verifies every op is in ALLOWED_OPERATIONS.
112+
VALIDATION_OUTCOME=0
113+
if [[ "${GITHUB_PR_COMMENT_VAR_ACTION:-}" == "run_pytorch_tests" ]]; then
114+
echo "--- Validating PyTorch allowlist against HuggingFace models"
115+
cmake \
116+
-DSOURCE_DIR="$(pwd)" \
117+
-DVALIDATE_CONFIG="$(pwd)/dev-tools/extract_model_ops/validation_models.json" \
118+
-DVALIDATE_PT_DIR="$(pwd)/dev-tools/extract_model_ops/es_it_models" \
119+
-DVALIDATE_VERBOSE=TRUE \
120+
-P cmake/run-validation.cmake || VALIDATION_OUTCOME=$?
121+
122+
if [[ $VALIDATION_OUTCOME -ne 0 ]]; then
123+
echo "^^^ +++"
124+
echo "Allowlist validation failed — the new PyTorch version may introduce ops not in ALLOWED_OPERATIONS."
125+
echo "See dev-tools/extract_model_ops/README.md for how to update the allowlist."
126+
fi
127+
fi
128+
108129
# Upload test results
109130
echo "--- Uploading test results"
110131
TEST_RESULTS_ARCHIVE=${OS}-${HARDWARE_ARCH}-unit_test_results.tgz
@@ -117,4 +138,6 @@ else
117138
echo "No test results archive created"
118139
fi
119140

120-
exit $TEST_OUTCOME
141+
if [[ $TEST_OUTCOME -ne 0 || $VALIDATION_OUTCOME -ne 0 ]]; then
142+
exit 1
143+
fi

0 commit comments

Comments
 (0)