Send system role, not developer, to Ollama-compatible endpoints - #71
Open
map-c wants to merge 1 commit into
Open
Send system role, not developer, to Ollama-compatible endpoints#71map-c wants to merge 1 commit into
map-c wants to merge 1 commit into
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
The Ollama provider slot is what users point at arbitrary OpenAI-compatible endpoints (vLLM, LM Studio, self-hosted gateways). pi defaults supportsDeveloperRole on for providers it doesn't recognize, and with reasoning: true that sends the system prompt as role "developer", which strictly-validating endpoints reject with a 400 on the first message. Pin supportsDeveloperRole: false on the ollama branch, mirroring workersAiCompat(). Regression tests assert the compat flag and the role on the wire. Fixes cloudflare#36
map-c
force-pushed
the
fix-ollama-developer-role
branch
from
August 7, 2026 09:51
d44e727 to
164bb4c
Compare
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #36.
Problem
A model added through the Ollama provider slot pointing at a self-hosted OpenAI-compatible endpoint fails on the first message with HTTP 400: the system prompt goes out with
role: "developer", which strictly-validating endpoints (vLLM, LM Studio, llama.cpp, self-hosted gateways) reject.pi's
openai-completionsAPI picks the system-prompt role frommodel.reasoning && compat.supportsDeveloperRole.detectCompat()doesn't recognize theollamaprovider or an arbitrary self-hosted hostname, so the flag defaults on — and the ollama branch hardcodesreasoning: true, supplying the other half of the condition.Fix
Pin
compat: { supportsDeveloperRole: false }on the ollama branch ingetModelDirect(), mirroring whatworkersAiCompat()already does for the sibling Workers AI branch.Tests
Two regression tests in
__tests__/ai-models.test.ts(which drives the real pi-ai stack with an injected fetch stub):supportsDeveloperRole: false;role: "system"— red before the fix (it captured"developer", exactly the issue's repro), green after.pnpm exec vitest run(workshop-backend, 281 tests),pnpm lint:check, andtsc --noEmitall pass.