Feature Request: Add logprobs metadata to model database
Why we need logprobs data
Logprobs are a natural model fingerprint — different models produce distinct token probability distributions on the same prompt, and these distributions are inherently difficult to forge. If an API claims to serve Model A but returns logprobs inconsistent with Model A's baseline, users can immediately detect misrepresentation.
Current problems
- No central reference: There's no public dataset of per-model logprobs baselines. Users who suspect mislabeling have nothing to compare against.
- High collection cost: Building baselines requires many API calls with consistent prompts — impractical for individual users to do across dozens of models.
- models.dev gap: The database already tracks pricing, context limits, and capability flags, but has zero information about logprobs support.
What to add
A logprobs section per model recording:
- Whether logprobs are available
- Access method (built-in by default, or requires parameter)
Happy to help collect baselines for a first batch of mainstream models if accepted.
Feature Request: Add logprobs metadata to model database
Why we need logprobs data
Logprobs are a natural model fingerprint — different models produce distinct token probability distributions on the same prompt, and these distributions are inherently difficult to forge. If an API claims to serve Model A but returns logprobs inconsistent with Model A's baseline, users can immediately detect misrepresentation.
Current problems
What to add
A logprobs section per model recording:
Happy to help collect baselines for a first batch of mainstream models if accepted.