Skip to content

Comments

chaning learning rate back to default of 1e-3#156

Open
schnamo wants to merge 2 commits intodevfrom
fix/default_learning_rate
Open

chaning learning rate back to default of 1e-3#156
schnamo wants to merge 2 commits intodevfrom
fix/default_learning_rate

Conversation

@schnamo
Copy link
Collaborator

@schnamo schnamo commented Feb 18, 2026

Changes
For model config files the default is now set to 1e-3.

Solves issue 150

@sfluegel05
Copy link
Collaborator

Thanks for fixing this. I took the chance to delete a redundant pretraining config and fix some other parameters. Also, there are 5 toxicity electra configs. You have changed one learning rate to 1e-3, the others are 1e-5. Is there a reason for (a) having 5 different toxicity electra configs (but none for other OPT finetuning tasks) and (b) for having different learning rates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants