Skip to content

Refactor monarch variable names to boost readbility#40

Open
Edenzzzz wants to merge 3 commits into
HazyResearch:mainfrom
Edenzzzz:main
Open

Refactor monarch variable names to boost readbility#40
Edenzzzz wants to merge 3 commits into
HazyResearch:mainfrom
Edenzzzz:main

Conversation

@Edenzzzz

@Edenzzzz Edenzzzz commented Dec 13, 2024

Copy link
Copy Markdown

Previously block dimensions such as q, p are very non-intuitive and make the code hard to understand and build on. In developing my Monarch PEFT project, I renamed them to fix the block-diagonal matmul taxonomy with terms like blk_in, blk_out to clearly indicate block dims.
This could significantly encourage follow-up work using this code base, as folks have complained about hard-to-use code :)

@Edenzzzz

Edenzzzz commented Dec 13, 2024

Copy link
Copy Markdown
Author

@DanFu09

DanFu09 commented Dec 14, 2024

Copy link
Copy Markdown
Collaborator

Cool! Has this been tested in this repo in any models?

@Edenzzzz

Copy link
Copy Markdown
Author

If you mean convergence tests, not yet--only passed precision tests. Can probably try that in a few days. Thanks!

@DanFu09

DanFu09 commented Dec 15, 2024 via email

Copy link
Copy Markdown
Collaborator

@Edenzzzz

Edenzzzz commented Jan 19, 2025

Copy link
Copy Markdown
Author

@DanFu09 Convergence curves match (monarch-mixer-finetune-glue-768dim-80m-parameters.yaml)
image

@Edenzzzz Edenzzzz changed the title Refractor monarch variable names to boost readbility Refactor monarch variable names to boost readbility Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants