Skip to content

Fix(ChattyLLM): Fix title generation to work with languages other than english #451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
marcelklehr merged 2 commits into from
Feb 3, 2026
+4 −10
Merged

Fix(ChattyLLM): Fix title generation to work with languages other than english #451

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
90c1691
Fix(ChattyLLM): Fix title generation to work with languages other tha…
marcelklehr Feb 3, 2026
86c4199
fix: Address review comments
marcelklehr Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/AppInfo/Application.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ class Application extends App implements IBootstrap {
public const ASSISTANT_DATA_FOLDER_NAME = 'Assistant';

public const CHAT_USER_INSTRUCTIONS = 'This is a conversation in a specific language between the user and you, Nextcloud Assistant. You are a kind, polite and helpful AI that helps the user to the best of its abilities. If you do not understand something, you will ask for clarification. Detect the language that the user is using. Make sure to use the same language in your response. Do not mention the language explicitly. Format your answers properly in markdown.';
public const CHAT_USER_INSTRUCTIONS_TITLE = 'Above is a chat session in a specific language between the user and you, Nextcloud Assistant. Generate a suitable title summarizing the conversation in the same language. Output only the title in plain text, nothing else.';
public const CHAT_USER_INSTRUCTIONS_TITLE = 'This is a conversation between the user and Nextcloud Assistant. Generate a suitable title for the conversation that summarizes it. Detect the language of the conversation. The title that you output should be in the same language as the conversation. Output only the title in plain text, nothing else. Do not mention the language explicitly. For example, if the conversation is about trees in sweden but is written in Spanish, the title could be "Àrboles en Suecia", if it was in English, the title could be "Trees in Sweden". Do not write the title in e.g. Swedish just because Sweden is mentioned in the conversation.';
public const MAX_TEXT_INPUT_LENGTH = 64_000;

private IAppConfig $appConfig;
Expand Down
12 changes: 3 additions & 9 deletions lib/Controller/ChattyLLMController.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -880,28 +880,22 @@ public function generateTitle(int $sessionId): JSONResponse {
) ?: Application::CHAT_USER_INSTRUCTIONS_TITLE;
$userInstructions = str_replace('{user}', $user->getDisplayName(), $userInstructions);

$systemPrompt = '';
$firstMessage = $this->messageMapper->getFirstNMessages($sessionId, 1);
if ($firstMessage->getRole() === 'system') {
$systemPrompt = $firstMessage->getContent();
}

$history = $this->getRawLastMessages($sessionId);
// history is a list of JSON strings
$history = array_map(static function (Message $message) {
return json_encode([
'role' => $message->getRole(),
'content' => $message->getContent(),
]);
], JSON_THROW_ON_ERROR);
Copy link
Member Author

@marcelklehr marcelklehr Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throw JSON errors

}, $history);

try {
$taskId = $this->scheduleLLMChatTask($userInstructions, $systemPrompt, $history, $sessionId, false);
$taskId = $this->scheduleLLMChatTask($userInstructions, $userInstructions, $history, $sessionId, false);
Copy link
Member Author

@marcelklehr marcelklehr Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate prompt for moar accuracy.

Copy link
Contributor

@kyteinsky kyteinsky Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hehe

but why remove system prompt. That's meant for universal guidelines for the LLMs and probably should be taken into account here as well.
does the default one have a negative effect on the response?

Copy link
Member Author

@marcelklehr marcelklehr Feb 3, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the two prompts are competing for the llm's attention and IMHO the general assistant guidelines are not really needed for title generation.

Copy link
Member

@julien-nc julien-nc Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Marcel here. The "chat" system prompt won't help much on generating a title.

Copy link
Contributor

@kyteinsky kyteinsky Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah I see that now. Maybe the language guidelines should only be in the system prompt, the user instructions could be about only outputing the title and the examples (which are noice).

but it's alright for the title generation I suppose if it's not user-influenced.

} catch (\Exception $e) {
return new JSONResponse(['error' => $e->getMessage()], Http::STATUS_BAD_REQUEST);
}
return new JSONResponse(['taskId' => $taskId]);
} catch (\OCP\DB\Exception $e) {
} catch (\OCP\DB\Exception|\JsonException $e) {
$this->logger->warning('Failed to generate a title for the chat session', ['exception' => $e]);
return new JSONResponse(['error' => $this->l10n->t('Failed to generate a title for the chat session')], Http::STATUS_INTERNAL_SERVER_ERROR);
}
Expand Down