Add JACCL expert-parallel distributed inference#351

Open

machiabeli wants to merge 22 commits into

antirez:mainfrom

machiabeli:feat/jaccl-distributed

Commits on May 27, 2026

[jaccl] add C shim header for JACCL integration

machiabeli
and
claude
committed
[jaccl] add C++ shim implementation wrapping JACCL Group API

machiabeli
and
claude
committed
[jaccl] integrate JACCL static lib build into Makefile (opt-in JACCL=1)

machiabeli
and
claude
committed
[jaccl] add shim linkage test

machiabeli
and
claude
committed
[jaccl] add distributed state fields to ds4_engine

machiabeli
and
claude
committed
[jaccl] init/teardown JACCL in engine lifecycle

machiabeli
and
claude
committed
[jaccl] add --distributed CLI flag

machiabeli
and
claude
committed
[jaccl] distribute expert accumulation with all_sum (CPU path)

machiabeli
and
claude
committed
[jaccl] distribute batch expert accumulation (CPU prefill path)

machiabeli
and
claude
committed
[jaccl] skip non-owned expert gate/up projections

machiabeli
and
claude
committed
[jaccl] mask non-owned experts before fused Metal MoE kernel

machiabeli
and
claude
committed
[jaccl] insert all_sum on routed_out after fused Metal MoE kernel

machiabeli
and
claude
committed
[jaccl] distribute batch Metal MoE path

machiabeli
and
claude
committed
[jaccl] fix thread-safety: move all_sum out of prealloc into callers

machiabeli
and
claude
committed
[jaccl] add distributed launch script

machiabeli
and
claude
committed
[jaccl] add distributed correctness test

machiabeli
and
claude
committed
[jaccl] document distributed benchmark methodology

machiabeli
and
claude
committed
[jaccl] fix Metal distributed: flush router kernel before CPU mask read

machiabeli
and
claude
committed
[jaccl] Metal expert mask kernel — eliminates batch break overhead

machiabeli
and
claude
committed
[jaccl] Metal shader early-exit for non-owned experts

machiabeli
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add JACCL expert-parallel distributed inference#351

Add JACCL expert-parallel distributed inference#351
machiabeli wants to merge 22 commits into
antirez:mainfrom
machiabeli:feat/jaccl-distributed

Commits on May 27, 2026

[jaccl] add C shim header for JACCL integration

[jaccl] add C++ shim implementation wrapping JACCL Group API

[jaccl] integrate JACCL static lib build into Makefile (opt-in JACCL=1)

[jaccl] add shim linkage test

[jaccl] add distributed state fields to ds4_engine

[jaccl] init/teardown JACCL in engine lifecycle

[jaccl] add --distributed CLI flag

[jaccl] distribute expert accumulation with all_sum (CPU path)

[jaccl] distribute batch expert accumulation (CPU prefill path)

[jaccl] skip non-owned expert gate/up projections

[jaccl] mask non-owned experts before fused Metal MoE kernel

[jaccl] insert all_sum on routed_out after fused Metal MoE kernel

[jaccl] distribute batch Metal MoE path

[jaccl] fix thread-safety: move all_sum out of prealloc into callers

[jaccl] add distributed launch script

[jaccl] add distributed correctness test

[jaccl] document distributed benchmark methodology

[jaccl] fix Metal distributed: flush router kernel before CPU mask read

[jaccl] Metal expert mask kernel — eliminates batch break overhead

[jaccl] Metal shader early-exit for non-owned experts

Commits on Jun 7, 2026

[jaccl] fix distributed inference: GPU sync + defense-in-depth MoE safety

[jaccl] fp16 all_sum — halve RDMA payload for 12% generation speedup