Skip to content

使用的是哪个bert?tokenizer设置。未解析的引用 #22

@4C4247

Description

@4C4247

def init_tokenizer():
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
tokenizer.add_special_tokens({'bos_token': '[DEC]'})
tokenizer.add_special_tokens({'additional_special_tokens': ['[ENC]']})
tokenizer.enc_token_id = tokenizer.additional_special_tokens_ids[0]
return tokenizer
我设置了联网下载,但是连接失败,我换了国内镜像,需要手动指定bert是哪一个。如你所见,太多了,可不可以留言告诉我~
还有三个未解析的引用,是我少了什么包吗。
bert
unreferenced

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions