feat(inference): add llmisvc_model_provider_resolver

[jland-redhat]

Summary

• Add llmisvc_model_provider_resolver, porting only the LLMISvc / KServe BBR body-rewrite path from IPP’s model-provider-resolver.
• Prefer a configurable model header (default X-Model, aligned with model_to_header), fall back to body "model", and when the value is a publisher ID (publishers/.../models/<name>) rewrite the body "model" to <name> only.
• Leave the routing header untouched so KServe can still route on the publisher ID; stash the original ID in llmisvc_model_provider_resolver.publisher_id for metering.
• Includes unit tests, example config, integration coverage, and generated filter docs.

Does not port ExternalModel / ExternalProvider resolution, weighted provider selection, Host rewrite, api-format detection, or credential handling.

Sister PR (merge after this)

• praxis-extproc: fix ExtProc body-mutation path to set content-length to the mutated body length (avoids Envoy mismatch_between_content_length_and_the_length_of_the_mutated_body / 500).
• Link: fix(extproc): sync content-length on body mutate + rustls CryptoProvider opendatahub-io/praxis-extproc#23

Without the ExtProc follow-up, body rewrites that change length will fail in Envoy BUFFERED + header SEND mode even though this filter’s rewrite is correct.

Test plan

• Unit tests for rewrite / header preference / body fallback / non-publisher passthrough
• Example config + integration tests
• Validated on local cluster with publisher-ID model request; upstream returned a completion:

{
  "id": "chatcmpl-efb19481-6952-5be9-9572-ebfb8aa9070b",
  "model": "demo/sim-stream",
  "object": "chat.completion",
  "choices": [
    {
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "I am fine, how are you today? ..."
      }
    }
  ]
}

[praxis-bot-app]

Missing Signed-off-by: 5113bb3. All commits require sign-off (via git commit --signoff).

[praxis-bot-app]

AI tool authorship detected:

• d11ef9e: Co-authored-by: Cursor <cursoragent@cursor.com>

Sorry, this project does not accept commits authored by tools as valid.
Commits need to be authored by and signed-off by the human(s) responsible for the PR, with their name and contact.

[praxis-bot-app]

Unsigned commits: d03e60a. Please sign your commits.

[aslakknutsen]

Why this if model_to_header ?

[jland-redhat]

model_to_header puts the model in the header.

But the model serving needs the header to be a "canonical id" publishers//models/<MODEL_NAME>. So the way BBR works in 3.5 is that we have the user put this value in the body "models" value and we move it into the header and replace it with the real <MODEL_NAME>

This resolver is that second part.

External Models do something similar but replacement will happen based on the ExternalModel CRD.

[aslakknutsen]

I mean, this seems to "if it's not in the header as provided by model_to_header", it will fall back to getting the value from where model_to_header would/should have gotten the value.

[jland-redhat]

Hmm maybe we can sync on this so I can better understand but I would think that is ok right?

It is faster to get this from the header and we fallback to the model if it is not there seems reasonable. But I can just remove the fallback if we think that makes more sense.

[aslakknutsen]

Is there ever a chance that "model" won't be in the request body on a valid request?

[jland-redhat]

Not if it is following the OpenAI spec I don't believe, and it has to follow that spec if it is a vLLM model which should be the only thing that this filter would catch.

[aslakknutsen]

Ok. I'm hinting at we could pretty easy do a StringBuffer splice to inject the new model value to avoid the full DOM deserialize and serialize of the complete body. (handling missing model fields is a little bit more tricky, but a replace in a buffer is easy)

[praxis-bot]

praxis-bot review

Clean filter implementation with good test coverage, proper config validation, and correct use of replace_json_body. One edge case around body mutation when the "model" field is absent.

Findings: 1 medium

[praxis-bot]

[Medium] When resolve_model_name obtains the model from the header (not the body), the body might not contain a "model" key at all. In that case obj.insert(...) silently adds a "model" field the caller never sent, which could surprise backends that do not expect one (e.g. non-completions endpoints that happen to share a pipeline).

Guard the insert so it only rewrites an existing field:

if !obj.contains_key("model") {
    return Ok(FilterAction::Continue);
}

obj.insert("model".to_owned(), serde_json::Value::String(short_name.to_owned()));

Add a unit test: header carries a publisher ID, body is {"messages":[]} (no "model"), assert the body is unchanged after the filter runs.

[praxis-bot]

praxis-bot review (round 2)

Fix commit is clean: backtick formatting, CI metrics_route field, and test loop refactor all look correct. One nit below. The prior Medium (body mutation when "model" absent) remains open.

Findings: 1 nit

[praxis-bot]

[Nit] Assertion message says SplitN but the implementation (line 279) uses split_once. They are semantically similar, but the message should match the actual method for accuracy.

[praxis-bot]

praxis-bot review (round 3)

Clean implementation with good separation of concerns. Config validation (deny_unknown_fields, empty-header rejection, validate_max_body_bytes), the header-vs-body resolution chain, and the split_once-based publisher-ID parsing are all correct. Integration tests exercise the core rewrite, routing-header preservation, and non-publisher passthrough end-to-end. The register.rs test refactor to a loop is a nice cleanup.

The prior Medium from round 1 (body mutation when "model" absent -- obj.insert(...) adds a field the caller never sent) remains the only actionable concern and is not yet addressed.

Findings: 0 new (prior Medium still open)