Hello!
Thank you for your great work. Recently I have followed the code you provided in github and your hyperparameters to train the Magicoder. However, The results I reproduced are different from the model you provided in huggingface. I am sure that everything is the same as your paper.
Here's my result.
To clarify, I use my own evaluation code to evaluate the two models. But since the two models share the same evaluation code, I think it doesn't matter.
Best regards,
Shen
Hello!
Thank you for your great work. Recently I have followed the code you provided in github and your hyperparameters to train the Magicoder. However, The results I reproduced are different from the model you provided in huggingface. I am sure that everything is the same as your paper.
Here's my result.
To clarify, I use my own evaluation code to evaluate the two models. But since the two models share the same evaluation code, I think it doesn't matter.
Best regards,
Shen