The pre-trained model provided in this repository different from the model used in the paper?

[doiken23]

Using the pre-trained model provided in this repository, I ran the following command to calculate the performance of the model on the test data.

python test.py --cfg configs/dynamic/dynamic.yaml --visualize --snapshot 9000 --gpu 0

python evaluate.py \
    --results_dir ./experiments/dynamic/test_output/captions \
    --anno ./data/total_change_captions_reformat.json \
    --type_file ./data/type_mapping.json

The calculated scores are as follows:

BLEU-4	CIDEr	METEOR	SPICE
0.5355	1.1496	0.3794	0.3128

This score is slightly higher than the score provided in the paper. Is the model provided in this repository different from the model used in the paper?

[Ph0eNiX0803]

Hi! @doiken23 Did you reproduce the score by training a new model using the code provided in this repository?

[Ph0eNiX0803]

I train a model that score can't reach the pre-trained model.

[doiken23]

@Ph0eNiX0803

Sorry, I didn't reproduce the score in the paper...
Maybe the seed is not fixed, so the score will vary.