You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is there any guidance on how to tune lr and lr_1d for MARS?
I noticed that in the README, under the hyperparameters for GPT-2 models, lr is set to be 2× lr_1d. Is this ratio optimal, and does it also apply to other models?
Is there any guidance on how to tune
lrandlr_1dfor MARS?I noticed that in the README, under the hyperparameters for GPT-2 models,
lris set to be 2×lr_1d. Is this ratio optimal, and does it also apply to other models?