feat: Support explicit model control mode in OpenAI frontend

[pskiran1]

What does the PR do?

This PR adds explicit model-control support to the OpenAI-compatible FastAPI frontend, enabling runtime model load/unload (EXPLICIT mode) and startup model selection via CLI flags, along with tests and documentation.

Changes:

• Add --model-control-mode and --load-model CLI options and pass them into tritonserver.Server startup configuration.
• Introduce /v1/models/{model_name}/load and /v1/models/{model_name}/unload endpoints plus API restriction grouping for model-management.
• Extend TritonLLMEngine with dynamic model load/unload support and add comprehensive model-management tests/docs.

Checklist

• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

• feat

Related PRs:

Where should the reviewer start?

Test plan:

• CI Pipeline ID: 46877415

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

[Copilot]

Pull request overview

This PR adds explicit model-control support to the OpenAI-compatible FastAPI frontend, enabling runtime model load/unload (EXPLICIT mode) and startup model selection via CLI flags, along with tests and documentation.

Changes:

• Add --model-control-mode and --load-model CLI options and pass them into tritonserver.Server startup configuration.
• Introduce /v1/models/{model_name}/load and /v1/models/{model_name}/unload endpoints plus API restriction grouping for model-management.
• Extend TritonLLMEngine with dynamic model load/unload support and add comprehensive model-management tests/docs.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
python/openai/openai_frontend/main.py	Adds CLI flags and wires model control mode + startup models into Triton server creation.
python/openai/openai_frontend/frontend/fastapi_frontend.py	Registers the new model-management router in the FastAPI app.
python/openai/openai_frontend/frontend/fastapi/routers/model_management.py	Implements load/unload REST endpoints that call the engine.
python/openai/openai_frontend/frontend/fastapi/middleware/api_restriction.py	Adds model-management to restricted API mapping.
python/openai/openai_frontend/engine/engine.py	Extends the engine protocol with async load/unload methods.
python/openai/openai_frontend/engine/triton_engine.py	Implements load/unload and refreshes model metadata handling.
python/openai/tests/utils.py	Adds an explicit-mode server helper and a port parameter to OpenAIServer.
python/openai/tests/test_model_management.py	Adds test coverage for none/explicit modes, concurrency, CLI flags, and restricted API behavior.
python/openai/README.md	Documents model management modes, CLI usage, endpoints, and restriction options.

Comments suppressed due to low confidence (1)

python/openai/tests/utils.py:111

• OpenAIServer.__init__ accepts a port parameter and uses it to build URLs, but it never passes --openai-port (or otherwise configures the subprocess) to actually start the server on that port. If a caller overrides port, health checks and requests will hit the wrong port. Either remove the parameter, or ensure the subprocess is launched with the matching --openai-port value when port is provided.

    def __init__(
        self,
        cli_args: List[str],
        *,
        port: int = 9000,
        env_dict: Optional[Dict[str, str]] = None,
    ) -> None:
        # TODO: Incorporate caller's cli_args passed to this instance instead
        self.host = "localhost"
        self.port = port

        env = os.environ.copy()
        if env_dict is not None:
            env.update(env_dict)

        this_dir = Path(__file__).resolve().parent
        script_path = this_dir / ".." / "openai_frontend" / "main.py"
        self.proc = subprocess.Popen(
            ["python3", script_path] + cli_args,
            env=env,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[whoisj]

Overall, this is an excellent PR. Thanks for your hard work. I've left a couple of comments/questions that are preventing me from immediately approving things.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[whoisj]

approved w/ 1 question

[whoisj]

an await under lock? how long are we expecting the lock to be held here?

[yinggeh]

Depending on how long core loads/unloads the model