Skip to content

Skill Forge rewriter/gate ignore agents.defaults.model, fall back to region-blocked provider default #155

Description

@arelchan

Summary

Skill Forge's auxiliary LLM calls (the query rewriter and the LLM gate)
do not use the configured agent model (agents.defaults.model). When no
Skill-Forge-specific model is set, they fall through to the provider's
hardcoded default_model, which for the openrouter provider is
openrouter/anthropic/claude-sonnet-4-5. If that default model is unavailable
(e.g. region-blocked), the rewrite/gate step hard-fails even though the agent is
configured with a working model -- so the main turn succeeds while
skill.rewrite / skill.gate error out.

Evidence (from tracing)

Config: agents.defaults.model = "minimax/minimax-m3", provider openrouter.

In one turn, the trace shows:

  • llm.call with invocation_source = session.turn (main reasoning) ->
    llm.model = minimax/minimax-m3, status OK.
  • llm.call with invocation_source = skill.rewrite ->
    llm.model = openrouter/anthropic/claude-sonnet-4-5, status ERROR:
    OpenrouterException - {"error":{"message":"This model is not available in your region.","code":403}}.

So the rewriter used a different model than the configured minimax/minimax-m3,
and that model is region-blocked. The rewriter then falls back to
need_retrieval = True with an empty rewritten query.

Root cause

  • raven/context_engine/factory.py: QueryRewriter(provider, max_tokens=...)
    is constructed with no model; LLMGateFilter(..., model=skill_forge.llm_gate_model or None, ...)
    is None unless llm_gate_model is set.
  • QueryRewriter.analyze / LLMGateFilter.filter call
    provider.chat_with_retry(...) without a model, so they use
    provider.default_model.
  • raven/providers/registry.py: the openrouter provider spec hardcodes
    default_model = "openrouter/anthropic/claude-sonnet-4-5".

The main agent loop passes model=agents.defaults.model explicitly on every
call, so it is unaffected; only the auxiliary Skill-Forge calls hit the
hardcoded default.

Impact

  • Query rewrite and skill gate fail whenever the provider's hardcoded default
    model is unavailable/region-blocked, regardless of the configured agent model.
  • The rewriter has no config override (unlike the gate's llm_gate_model),
    so this cannot be worked around by configuration alone.

Proposed fix

Make the Skill-Forge auxiliary calls inherit agents.defaults.model when no
Skill-Forge-specific model is configured:

  • Thread the configured agent model into QueryRewriter / LLMGateFilter in
    factory.py (add a model parameter to QueryRewriter).
  • Pass that model on the chat_with_retry calls, so rewrite/gate use the same
    working model as the main loop by default; a Skill-Forge override (e.g.
    llm_gate_model) still wins when set.

Workarounds (until fixed)

  • Gate: set skill_forge.llm_gate_model to a model available in your region
    (e.g. minimax/minimax-m3).
  • Rewriter: no config knob; either disable it (skill_forge.rewrite_enabled = false)
    or apply the code fix above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions