Model Router engine — route requests across existing model engines

[JoshuaDivineBlessing]

Summary

Adds a new MODEL_ROUTER engine type: a virtual model engine that owns no model itself and dispatches each request to one of the model engines already in the catalog. Routing is configured entirely in a router.json file in the engine's assets folder — the SMSS only carries the standard engine identity keys, no routing config.

{
  "mode": "llm",
  "sticky": true,
  "classifier_engine": "<engineId>",
  "default_route": "<engineId>",
  "fallbacks": ["<engineId>"],
  "embeddings_engine": "<engineId>",
  "routes": [
    { "name": "code", "engine_id": "<engineId>",
      "description": "Programming questions: debugging, writing and reviewing code",
      "keywords": ["java", "python", "debug"], "weight": 70 }
  ]
}

Routing modes

• keyword — first route with a whole-word keyword match on the latest user message wins (alphanumeric-lookaround matching, so score does not match "underscore"); otherwise the default route.
• llm — a classifier engine picks the route by reading each route's description (required on every route in this mode; validated at engine open). Falls back to keyword matching when classification fails, so keywords act as the optional deterministic safety net.
• weighted — deterministic weighted round-robin via an atomic cycle counter; weights 30/70 give an exact 30/70 split over every full cycle, not a probabilistic one.

Behavior

• Sticky conversations (default on): the engine that first serves a room is pinned (bounded in-memory LRU), so a conversation stays on one model and llm mode pays the classifier once per room, not per turn. Pins drop when the pinned engine fails.
• Failover: selected target → fallbacks in order → default route. Each attempt gets a fresh copy of the parameters; the last failure is rethrown (preserving SemossModelEngineException for the FE) when every candidate fails.
• Delegation path: askCall delegates straight to the target's askRoom with the caller's room, so message_json/tools pass through verbatim, tool-calling works through the router, and the target's inference log lands under the real room id. Route selection uses the latest user-authored message, not the serialized conversation blob.
• Access control: router access does NOT implicitly grant the backing engines — every ask/embeddings target is checked with SecurityEngineUtils.userCanViewEngine and denied candidates are skipped (classifier is exempt as internal plumbing).
• Inference logs: requests log twice by design (router id for attribution + target id for actual usage). Router rows are tagged route_ask/route_embeddings via a new inferenceLogMessageMethod hook in AbstractModelEngine, so ask-history queries and usage aggregations only count the target's rows.
• Context window: getContextWindow() returns the minimum across all serving targets (lazy, cached per config load) so agent auto-compaction works when a router is the agent's model. An explicit CONTEXT_WINDOW in SMSS/metadata still wins.
• Embeddings: served only when embeddings_engine is configured; fails fast with a clear message otherwise.
• Validation at open: unknown mode, llm without classifier or route descriptions, weighted without positive weights, duplicate route names, and any self-reference (route/default/classifier/embeddings/fallback pointing at the router) all fail engine open loudly.

Runtime config editing

• GetModelRouterConfig(engine=[...]) — returns the raw router.json (edit access required; the config exposes target engine ids).
• UpdateModelRouterConfig(engine=[...], map=[{...}]) — validates with the same rules as engine open (nothing is written when validation fails), persists the file, and applies it to the running instance immediately via reloadConfig() — no engine reload. Pushes the engine in clustered deployments.
• Supporting refactor: config parsing/validation is a static parseAndValidateConfig shared by open and update; config-derived state is swapped as immutable volatile snapshots; applying config resets sticky pins, the RR counter, and the derived context window.
• New IModelRouterEngine interface (readConfigJson/updateConfig/reloadConfig), advertised by the EngineProxyFactory guard proxy so the reactors work on guarded engines.

Files

File	Change
engine/api/ModelTypeEnum.java	register MODEL_ROUTER
engine/api/IModelRouterEngine.java	new interface for router config operations
engine/impl/model/ModelRouterEngine.java	the engine (new)
engine/impl/model/AbstractModelEngine.java	inferenceLogMessageMethod hook (identity by default)
engine/impl/pipeline/EngineProxyFactory.java	guard proxy advertises IModelRouterEngine for routers
reactor/model/GetModelRouterConfigReactor.java	new
reactor/model/UpdateModelRouterConfigReactor.java	new

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.