Skip to content

Assess model perplexity on eval split #5

Description

@irhum

The original splits over UniRef50 can be found in the original repo. Using a subset of them, need to compute either:

  • the randomized masking perplexity
  • the pseudo-perplexity with exhaustive single-token masking

Will likely do the former for computational efficiency.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions