Problems for new words

[abhranil08]

What should I do when I have new words in my test corpus?

[vermasrijan]

Hi @abhranil08 ,
Regarding your question, you could deal with a new word with either of the followings -

1. If new word is not present in vocabulary dict, then get its transliterate from any translate API (like Google)
2. Check out OpenNMT-py or OpenNMT (Lua language). They have a hack around to deal with a new word.
3. You could also train it using transfer learning

Note: This notebook is very old to be honest, and is not for production. Pretty sure there are better implementation now out there. I have listed some in the README.
Hope this helps :)

[alice-cool]

I got a error like the following:
return [lang.word2index[word] for word in sentence.split(' ')]
KeyError: "i'd"