The crate provides the port of the original BERT tokenizer from the Google BERT repository.
MIT license. See LICENSE file for full license.
yigit353/bert_tokenizer
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
| Name | Name | Last commit date | ||
|---|---|---|---|---|
The crate provides the port of the original BERT tokenizer from the Google BERT repository.
MIT license. See LICENSE file for full license.