Skip to content

Dynamic Model Discovery for LLM Nodes #643

@asclearuc

Description

@asclearuc

LLM provider nodes currently hard-code their model lists inside services.json
files. When providers release new models or deprecate old ones, a developer must
manually edit the JSON — a fragile, error-prone process that causes model lists
to fall behind the real provider catalogue.

Create a sync tool that discovers currently-available models directly from provider APIs and propagates
that information into the services configuration. The tool should support all cloud LLM providers that are currently present: OpenAI, Anthropic, Gemini, Mistral, DeepSeek, xAI, Perplexity, and Qwen.

Key behaviours:

  • Dry-run by default. Changes are previewed before being written. --apply
    commits them to disk.
  • Non-destructive. Existing profile titles, model fields, and manual customisations are preserved.
  • Deprecation tracking. Models no longer listed by a provider are marked
    deprecated: true with a migration message, rather than silently removed.
    Existing pipelines continue to work.
  • Token limit accuracy. Limits are resolved automatically.
  • CI/CD ready. Designed to run on a weekly schedule, opening a pull request
    with the full add/update/deprecate summary for developer review before merge.
  • If a key for the corresponding LLM provider is present, the tool runs a smoke test for that provider.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions