Add /set command and context length configuration

[ericcurtin]

Add a /set command to the interactive mode to allow users to configure
parameters like num_ctx during runtime. Also implement environment
variable support for default context length (DMR_CONTEXT_LENGTH) and
ensure the scheduler respects these configurations when setting up
backend runners.

Signed-off-by: Eric Curtin eric.curtin@docker.com

[gemini-code-assist]

Summary of Changes

Hello @ericcurtin, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the flexibility of configuring model context length. It provides users with an interactive command to adjust num_ctx on the fly and allows for a system-wide default to be set through an environment variable. The core scheduling logic has been adapted to seamlessly integrate and apply these new configuration options.

Highlights

• Interactive Configuration Command: Introduced a new /set parameter num_ctx <value> command in the interactive CLI mode, allowing users to dynamically adjust the model's context length during runtime.
• Environment Variable Support: Added support for configuring a default context length via the DMR_CONTEXT_LENGTH environment variable, which is parsed and validated at application startup.
• Scheduler Integration: The backend scheduler has been updated to respect both the runtime /set command and the DMR_CONTEXT_LENGTH environment variable, prioritizing explicit request-specific context sizes over the environment default.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• The interactive help for /set now has two separate entries (/set system and /set with parameter) and a usage line that only mentions the parameter form; consider consolidating the help text so users see a single, clear description of both forms and their syntax.
• The list of supported parameters (num_ctx) is hard-coded multiple times in the /set handling (usage line and unknown-parameter message); consider centralizing the allowed parameters (e.g., a map or const slice) so future additions or renames stay consistent across help and validation.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The interactive help for `/set` now has two separate entries (`/set system` and `/set` with `parameter`) and a usage line that only mentions the `parameter` form; consider consolidating the help text so users see a single, clear description of both forms and their syntax.
- The list of supported parameters (`num_ctx`) is hard-coded multiple times in the `/set` handling (usage line and unknown-parameter message); consider centralizing the allowed parameters (e.g., a map or const slice) so future additions or renames stay consistent across help and validation.

## Individual Comments

### Comment 1
<location> `cmd/cli/commands/run.go:238-235` </location>
<code_context>
+		case strings.HasPrefix(line, "/set"):
</code_context>

<issue_to_address>
**issue (bug_risk):** Avoid matching `/set` handler for commands like `/settings` by requiring an exact command token

`strings.HasPrefix(line, "/set")` will also catch `/settings`, `/setup`, etc., and send them through the `/set` handler with an incorrect usage message. Instead, match the first token exactly, e.g.:

```go
case strings.HasPrefix(line, "/set ") || line == "/set":
    // or:
    // fields := strings.Fields(line)
    // if len(fields) > 0 && fields[0] == "/set" { ... }
```
so only the `/set` command is routed here.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces a /set command for runtime configuration and support for the DMR_CONTEXT_LENGTH environment variable. The changes are well-implemented, particularly the logic for handling the default context length in the scheduler. My main feedback is around the user experience of the new /set command, which could be made more intuitive.