-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Description
Traceback (most recent call last):
File "/home/quiana/work/python-toturial/Tokenization/BBPE/train.py", line 11, in
BBPETokenizer.train_tokenizer(data, vocab_size, vocab_outfile=vocab_outfile, merges_outfile=merges_outfile)
File "/home/quiana/work/python-toturial/Tokenization/BBPE/bbpe.py", line 170, in train_tokenizer
most_common_pair = pair_freq.most_common(1)[0][0]
~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels