Skip to content

[WIP] Update LM Studio client with REST API #3

Draft
rugvedS07 wants to merge 7 commits intomainfrom
rugved/update-with-rest
Draft

[WIP] Update LM Studio client with REST API #3
rugvedS07 wants to merge 7 commits intomainfrom
rugved/update-with-rest

Conversation

@rugvedS07
Copy link
Owner

External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated "Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text with a detailed and high quality description of your changes.

Include a link to a bug report or enhancement request.

@@ -10,6 +10,8 @@ pub struct LMStudioClient {
}

const LMSTUDIO_CONNECTION_ERROR: &str = "LM Studio is not responding. Install from https://lmstudio.ai/download and run 'lms server start'.";
// 8192 tokens provides headroom above the observed ~6000 token initial input.
const DEFAULT_CONTEXT_LENGTH: u32 = 8192;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly think we probably want even more than this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but not sure if we want to push a higher context length and error out. That is why I kept is conservative as much as possible

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants