Support for Muown

[fmac2000]

Is your feature request related to a problem? Please describe.
Yes. Muown is currently not supported, despite being one of the most promising Muon variants for training performance.

Describe the solution you'd like
Please add support for the Muown optimizer: https://github.com/kcc-lion/muown

Describe alternatives you've considered
I have tested other emerging optimizers, including existing Muon-style variants, but Muown has performed best in my independent experiments so far.

Additional context
Paper: https://arxiv.org/html/2605.10797v1
I have independently verified that Muown is currently the best-performing optimizer I’ve found, and I would really like to see it supported.

[skyw]

Contribution welcome. Otherwise, I can add it when I have time or ask agent to give it a try.

[skyw]

Asked AI to yolo one, seems decent https://github.com/NVIDIA-NeMo/Emerging-Optimizers/blob/skyw/muown/emerging_optimizers/orthogonalized_optimizers/muown.py.

Will need some time to clean up code and figure out how to test it. but basic function looks good enough.

[fmac2000]

Yep looks great thanks Sky

Only thing I'd suggest is adding an arg for scaling the Adam LR, I usually default it to 0.1, simply just multiplies against the LR on the magnitude Adam branch/ anywhere Adam is used - its better performing in my tests and follows the typical Muon practice more closely

Thanks!