Skip to content

Is there a way to get vocabulary size of an initialized tokenizer? #47

Open
Open
Is there a way to get vocabulary size of an initialized tokenizer?#47
@zhuzihan728

Description

@zhuzihan728
zhuzihan728
opened on Apr 21, 2025

Is there a way to get the vocabulary size of an initialized tokenizer from var encoding = GptEncoding.GetEncoding(encodingName);?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions