Skip to content

Add Metal test job to CI test matrix#1862

Open
aditvenk wants to merge 1 commit intomainfrom
aditvenk/stack/24
Open

Add Metal test job to CI test matrix#1862
aditvenk wants to merge 1 commit intomainfrom
aditvenk/stack/24

Conversation

@aditvenk
Copy link
Copy Markdown
Contributor

@aditvenk aditvenk commented Mar 28, 2026

Stacked PRs:


Add Metal test job to CI test matrix

Add a macOS Metal entry to matrix.json (macos-m2-26, cpu runtime,
metal backend) and update test.yml to handle macOS:

  • Gate apt-get on runner.os == 'Linux'
  • Add cpu runtime branch for PyTorch install
  • Add MPS availability check for metal backend
  • Run full test suite for metal (most tests skip via requires_gpu)
  • Skip grid range tests that need loop_index_expr

@aditvenk aditvenk marked this pull request as draft March 28, 2026 06:46
@aditvenk aditvenk changed the base branch from aditvenk/stack/13 to main March 28, 2026 06:46
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch from 74ea2e7 to c1e193e Compare March 28, 2026 06:46
@aditvenk aditvenk changed the base branch from main to aditvenk/stack/13 March 28, 2026 06:47
@aditvenk aditvenk marked this pull request as ready for review March 28, 2026 06:47
@aditvenk aditvenk requested a review from oulgen March 28, 2026 06:50
TEST_PATH="."
EXTRA_FLAGS="--ignore=test/test_examples_dist.py"
PARALLEL=""
elif [[ "${{ matrix.alias }}" == "m2-metal" ]]; then
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You dont need this, above sets HELION_BACKEND=metal and that should cover this

Copy link
Copy Markdown
Contributor Author

@aditvenk aditvenk Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It collects all tests in "." and I was worried that there might be imports which cannot be resolved in mac. Checking if any imports cannot be resolved...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there are imports that cannot be resolved, then they should be guarded by the backend. i.e. pallas backend cannot import triton, so all triton imports are guarded

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You dont need this, above sets HELION_BACKEND=metal and that should cover this

@oulgen -- just to clarify:

  1. are you suggesting to not add a conditional here? If I don't, by default, we try to run all tests in the folder (incl test_metal.py), but a bunch of tests fail to import triton on mac and several more fail to find CUDA and fail. We will need to touch-up these files to exit gracefully on mac?

Or, 2) are you suggesting to use HELION_BACKEND in the condition's expression?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done (2) in the recent commit. PTAL

Copy link
Copy Markdown
Contributor

@oulgen oulgen Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each test file is guarded by @onlybackend decorator. So metal would only run metal tests. You dont need this if block at all. Try running locally HELION_BACKEND=metal pytest test and you'll see

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thanks for clarifying! Didn’t know that. Will try it and amend this PR

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified as suggested.

I had to skip a few tests that fail on Metal due to some missing feature not yet added

@aditvenk aditvenk marked this pull request as draft March 28, 2026 17:18
@aditvenk aditvenk changed the base branch from aditvenk/stack/13 to main March 28, 2026 17:18
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch 2 times, most recently from d4395e8 to 065e6d8 Compare April 9, 2026 04:50
@aditvenk aditvenk changed the base branch from main to aditvenk/stack/26 April 9, 2026 04:50
@aditvenk aditvenk marked this pull request as ready for review April 9, 2026 04:50
@aditvenk aditvenk marked this pull request as draft April 9, 2026 20:43
@aditvenk aditvenk changed the base branch from aditvenk/stack/26 to main April 9, 2026 20:43
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch from 065e6d8 to eb80c4b Compare April 9, 2026 20:43
@aditvenk aditvenk changed the base branch from main to aditvenk/stack/26 April 9, 2026 20:43
@aditvenk aditvenk marked this pull request as ready for review April 9, 2026 20:43
@aditvenk aditvenk marked this pull request as draft April 9, 2026 20:57
@aditvenk aditvenk changed the base branch from aditvenk/stack/26 to main April 9, 2026 20:57
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch from eb80c4b to 4b81409 Compare April 9, 2026 20:57
@aditvenk aditvenk changed the base branch from main to aditvenk/stack/26 April 9, 2026 20:58
@aditvenk aditvenk marked this pull request as ready for review April 9, 2026 20:58
@aditvenk aditvenk marked this pull request as draft April 9, 2026 21:50
@aditvenk aditvenk changed the base branch from aditvenk/stack/26 to main April 9, 2026 21:50
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch from 4b81409 to 087ce82 Compare April 9, 2026 21:50
@aditvenk aditvenk changed the base branch from main to aditvenk/stack/26 April 9, 2026 21:50
@aditvenk aditvenk marked this pull request as ready for review April 9, 2026 21:50
@aditvenk aditvenk marked this pull request as draft April 13, 2026 02:14
@aditvenk aditvenk changed the base branch from aditvenk/stack/26 to main April 13, 2026 02:14
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch from 087ce82 to c0b4d5b Compare April 13, 2026 02:14
@aditvenk aditvenk changed the base branch from main to aditvenk/stack/26 April 13, 2026 02:14
@aditvenk aditvenk marked this pull request as ready for review April 13, 2026 02:15
@aditvenk aditvenk marked this pull request as draft April 13, 2026 02:54
@aditvenk aditvenk changed the base branch from aditvenk/stack/26 to main April 13, 2026 02:54
@aditvenk aditvenk force-pushed the aditvenk/stack/24 branch from c0b4d5b to 1b74bae Compare April 13, 2026 02:54
Add a macOS Metal entry to matrix.json (macos-m2-26, cpu runtime,
metal backend) and update test.yml to handle macOS:
- Gate apt-get on runner.os == 'Linux'
- Add cpu runtime branch for PyTorch install
- Add MPS availability check for metal backend
- Run full test suite for metal (most tests skip via requires_gpu)
- Skip grid range tests that need loop_index_expr

stack-info: PR: #1862, branch: aditvenk/stack/24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants