You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All Metal shaders currently live inline in crates/inference/src/forward/metal_qwen35.rs as Rust string constants (~14K lines, growing). This worked for bootstrapping but is hitting limits:
No MSL syntax highlighting or linting — shaders are opaque strings to the Rust compiler
No reuse across models — Qwen3.5-specific but GEMV/softmax/RMSNorm are generic
Hard to diff — shader changes appear as string diffs, not structured code changes
Can't benchmark shaders in isolation — coupled to the full forward pass
Can't study/adapt MLX techniques easily — no clean place to put reference implementations
Proposal
Extract Metal shaders into a dedicated module with .metal source files:
Context
All Metal shaders currently live inline in
crates/inference/src/forward/metal_qwen35.rsas Rust string constants (~14K lines, growing). This worked for bootstrapping but is hitting limits:Proposal
Extract Metal shaders into a dedicated module with
.metalsource files:Benefits
.metalfiles get IDE syntax highlighting + Metal validation (xcrun metal -c)metal-shader-benchbinary that benchmarks individual kernelsxcrun metalcompilation catches shader errors before runtimeMigration plan
.metalfiles (no behavior change)include_str!withmetal-rspipeline compilationxcrun metal -cvalidation to CI (macOS runner only)metal_qwen35.rsto use the shader library via dispatch.rsNon-goals
Priority
P2 — prerequisite for sustainable Metal kernel development (issue #85) but not blocking current work.
References
crates/inference/src/forward/metal_qwen35.rs(~14K lines)