Very low BLEU Score during copynet training

[justeuer]

Hi,

I'm using the default configuration of the copynet model. I updated Allennlp and Pytorch to the latest stable versions (2.7.0, 1.9.1) because I also got the nan error described in this issue. I didn't install the package list given there as this leads to version conflicts. It solved the nan problem, but now the BLEU scores during validation are very low (close to zero). Is that an artifact of the validation loop (the paper doesn't give a BLEU score)?

[tomerwolgithub]

Hi @justeuer. It's been a while, but zero BLEU score should not happen. Note that the exact string match score is a lower-bound on BLEU and it is not zero.