Fix(ChattyLLM): Fix title generation to work with languages other than english #451
Conversation
…n english tested with Llama 3.1 8B and Mistral Small 24B Signed-off-by: Marcel Klehr <mklehr@gmx.net>
marcelklehr
force-pushed
the
fix/title-gen-multilang
branch
from
297ed81 to
90c1691
Compare
|
|
||
| try { | ||
| $taskId = $this->scheduleLLMChatTask($userInstructions, $systemPrompt, $history, $sessionId, false); | ||
| $taskId = $this->scheduleLLMChatTask($userInstructions, $userInstructions, $history, $sessionId, false); |
There was a problem hiding this comment.
Duplicate prompt for moar accuracy.
There was a problem hiding this comment.
hehe
but why remove system prompt. That's meant for universal guidelines for the LLMs and probably should be taken into account here as well.
does the default one have a negative effect on the response?
There was a problem hiding this comment.
I believe the two prompts are competing for the llm's attention and IMHO the general assistant guidelines are not really needed for title generation.
There was a problem hiding this comment.
I agree with Marcel here. The "chat" system prompt won't help much on generating a title.
There was a problem hiding this comment.
ah yeah I see that now. Maybe the language guidelines should only be in the system prompt, the user instructions could be about only outputing the title and the examples (which are noice).
but it's alright for the title generation I suppose if it's not user-influenced.
| 'role' => $message->getRole(), | ||
| 'content' => $message->getContent(), | ||
| ]); | ||
| ], JSON_THROW_ON_ERROR); |
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
tested with Llama 3.1 8B and Mistral Small 24B
The current prompt would fail with Mistral Small 24B (tested on c.nc.c), ie. it would output swedish titles if sweden was mentioned in the conversation.