Skip to content

Replace fixed-interval LLM log polling with exponential backoff #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
CyberSecDef merged 3 commits into from
Apr 9, 2026
Merged

Replace fixed-interval LLM log polling with exponential backoff #160

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
121ab17
Initial plan
Copilot Apr 8, 2026
9cd0b47
Implement exponential backoff for LLM log polling (setTimeout-based, …
Copilot Apr 8, 2026
f5ce62a
Update static/js/script.js
CyberSecDef Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 33 additions & 5 deletions static/js/script.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1517,7 +1517,18 @@ $(function () {
// LLM Log Display
// -------------------------------------------------------------------
var _seenLogSignatures = {};
var _logPollInterval = null;
var _logPollTimeout = null;
var _logPollSameCount = 0; // consecutive polls with no new entries
// Backoff schedule: 0-1 same polls → 15s, 2 → 30s, 3 → 60s, 4 → 2min, 5 → 5min, 6+ → 10min
var _LOG_POLL_DELAYS = [15000, 15000, 30000, 60000, 120000, 300000, 600000];

function _scheduleLogPoll() {
if (_logPollTimeout) {
clearTimeout(_logPollTimeout);
}
var delay = _LOG_POLL_DELAYS[Math.min(_logPollSameCount, _LOG_POLL_DELAYS.length - 1)];
_logPollTimeout = setTimeout(pollLLMLog, delay);
}
Comment on lines +1529 to +1531
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With exponential backoff, polling can reach a 10-minute delay. If a user starts a new LLM action after a long idle period, the UI may not show new /llm_log entries (and sticky status updates) for up to the full backoff delay because the backoff is only reset after a poll detects new entries. Consider resetting _logPollSameCount (and triggering an immediate/near-immediate poll) when client-side actions that generate LLM traffic start (e.g., outline generation, chapter generation/revision, illustration generation), so logs remain responsive while still backing off during true idle periods.

Copilot uses AI. Check for mistakes.

function entrySignature(entry) {
if (!entry) return "";
Expand Down Expand Up @@ -1609,23 +1620,30 @@ $(function () {
_hasInitializedLogSnapshot = true;
_activeLLMRequests = 0;
setStickyStatus(DEFAULT_STICKY_STATUS, { force: true });
_scheduleLogPoll();
return;
}

if (entries.length === 0) return;
if (entries.length === 0) {
_logPollSameCount++;
_scheduleLogPoll();
return;
}

// Clear placeholder on first visible log entry
if (Object.keys(_seenLogSignatures).length === 0) {
$("#llm-chat-messages").empty();
}

var foundNew = false;
for (var i = 0; i < entries.length; i++) {
var entry = entries[i];
var signature = entrySignature(entry);
if (!signature || _seenLogSignatures[signature]) {
continue;
}

foundNew = true;
_seenLogSignatures[signature] = true;

if (entry.type === "request") {
Expand All @@ -1646,16 +1664,24 @@ $(function () {
addLogMessage(formatted);
}
}

if (foundNew) {
_logPollSameCount = 0;
} else {
_logPollSameCount++;
}
_scheduleLogPoll();
},
error: function() {
// Silently fail - log polling is non-critical
_logPollSameCount++;
_scheduleLogPoll();
}
});
}

// Start polling for LLM log updates
_logPollInterval = setInterval(pollLLMLog, 15000); // Poll every 15 seconds
pollLLMLog(); // Initial poll
// Start polling for LLM log updates (with adaptive backoff)
pollLLMLog(); // Initial poll; subsequent polls are scheduled inside pollLLMLog

// Clear log button — clears display and server-side log file
$("#btn-clear-log").on("click", function() {
Expand All @@ -1667,6 +1693,8 @@ $(function () {
_seenLogSignatures = {};
_activeLLMRequests = 0;
_hasInitializedLogSnapshot = false;
_logPollSameCount = 0;
Comment thread
CyberSecDef marked this conversation as resolved.
_scheduleLogPoll();
setStickyStatus(DEFAULT_STICKY_STATUS, { force: true });

// Clear the server-side log file
Expand Down
Loading