transformer performance

[SUSTechBruce]

Hi, I have run your code, but the performance is not better than VASnet, it seems like that the transformer model doesn't work very well in this supervised video summary assignment, what do you think?

[tim-roderick]

Here's my take on transformers for this problem.

I wasn't able to improve on the results of VASnet, as you'll see in the paper linked on the readme.

I think transformers do have the power and applicability in this task to get better than state of the art results, but not in the form I provide. I can safely say that the global pandemic limited the scope of this project.

In my conclusion I provide a description of directions I would take this project if I had further time. There I suggest decreasing the vocabulary size so that the weighting placed among words in the vocabulary wasn't so sparse. This would probably be a good idea for significant improvement.

After that, moving to a transformer implementation as found in BERT would also probably be a good idea.

As the video data is far more intricate and complex than words in a vocabulary, this model may not be applicable unless used with huge amount of model parameters.

Then again, there's a good chance this model isn't a good fit for this problem at all. I'll leave that to your judgement 😉

[SUSTechBruce]

Great analysis, I have learned a lot, you did a good job, thinks, bro~

[tim-roderick]

No problem man, anytime 👍