Very High Loss

[vinevix]

Hi, I've read your paper and I was trying to train your model. Despite I didn't change hyperparameters or any other model component, I get a very high Loss: that's my result after 15 epochs

epoch 4 average x_t_loss, x_1_loss, prob_loss, val losses: 13.204826354980469, 12.088323593139648, 3156.693359375, 12.881041526794434, 11.268576622009277, 2969.4931640625

Do you know what might be the problem?

[xu-shitong]

I suggest you start with the provided Flickr8k dataset clip feature and train for 4-5 epochs, to verify the model is optimized; or use the model to perform inference and see if the model was trained successfully but just with wrong loss calculation.