Access log reader returns the oldest entries and loads the whole file into memory

[Adriel2503]

Summary

readMonitoringConfig() backs both endpoints of the Requests page on self-hosted
instances (settings.readStatsLogs and settings.readStats). It has two distinct
problems:

1. Without a date range it returns the oldest 500 entries of access.log, not the
   most recent ones.
2. With a date range it reads the entire file synchronously into a single string.

Both are on canary @ 53feb9dd5. Cloud is unaffected — IS_CLOUD short-circuits both
procedures before the read.

---

1. The default view shows the oldest requests, not the newest

packages/server/src/utils/traefik/application.ts — readMonitoringConfig, the
readAll === false branch:

const fileStream = createReadStream(configPath, { encoding: "utf8" });
const readline = createInterface({ input: fileStream, ... });

for await (const line of readline) {
    // ...
    if (log.ServiceName !== "dokploy-service-app@file") {
        content += `${line}\n`;
        validCount++;
        if (validCount >= 500) {
            break;                       // <-- 500 entries from the START of the file
        }
    }
}

access.log is append-only, so reading forward and stopping at 500 yields the 500
oldest requests. parseRawConfig then sorts them by time descending, which makes the
table look correct — it is showing the newest of a stale window.

On an instance with steady traffic the Requests page shows requests from hours ago and
does not advance until the log is truncated.

2. Filtering by date reads the entire file

Same function, the readAll === true branch:

return fs.readFileSync(configPath, "utf8");

This runs whenever the user picks a date range. Three consequences:

• readFileSync blocks the event loop. It is not just that this request is slow — the
  whole panel is unresponsive for every other user while the read is in flight.

• It fails outright on large logs. Past Node's maximum string length the call throws
  ERR_STRING_TOO_LONG and the page errors instead of degrading.

• The result is fully materialized before pagination. In
  packages/server/src/utils/access-log/utils.ts, parseRawConfig JSON.parses every
  line into an array, filters, sorts, and only then slices the requested page:

  const totalCount = parsedLogs.length;
  if (sort) { parsedLogs = _.orderBy(parsedLogs, [sort.id], [sort.desc ? "desc" : "asc"]); }
  else      { parsedLogs = _.orderBy(parsedLogs, ["time"], ["desc"]); }
  
  if (page) {
      const startIndex = page.pageIndex * page.pageSize;
      parsedLogs = parsedLogs.slice(startIndex, startIndex + page.pageSize);   // <-- last
  }

  Rendering 20 rows parses and sorts the entire log first.

Current mitigation and its cost

packages/server/src/utils/access-log/handler.ts truncates the log daily:

await execAsync(
    `tail -n 1000 ${accessLogPath} > ${accessLogPath}.tmp && mv ${accessLogPath}.tmp ${accessLogPath}`,
);

That keeps the memory problem from surfacing on most instances, but it means access log
history is discarded every day, and it does not help an instance that accumulates a large
log between two runs of the cron.

---

Proposed fix

Read the file backwards from the end in bounded chunks and stop as soon as enough
entries are collected:

• no date range → walk back until N valid entries are collected, then stop
• with a date range → walk back until an entry older than start is reached, then stop

This is safe because access.log is chronological and append-only, so walking backwards
visits entries newest-first and the first out-of-range entry means every remaining entry
is also out of range.

Result:

	before	after
Default view	oldest 500 entries	newest N entries
Memory	entire file as one string + parsed array	bounded by entries returned
Event loop	blocked by readFileSync	never blocked
Large logs	ERR_STRING_TOO_LONG	unaffected

The exported signatures of parseRawConfig and processLogs stay unchanged, so the
existing tests in apps/dokploy/__test__/requests/request.test.ts keep passing as-is.

I have this implemented and tested locally, and will open a PR referencing this issue.
Verified against a seeded access.log of 2000 entries (2 MB) on a local dev instance: the
Requests page now lists the newest entries first, and a 30-minute date range reads 29
entries in ~5 ms instead of parsing the whole file. Happy to adjust the approach if you
would prefer a different one.

Possible follow-up (separate PR)

Reading backwards fixes the read path, but filtering and sorting still happen in memory
and the daily truncation still discards history. A natural next step — as its own PR, not
this one — would be to let the existing apps/monitoring Go service ingest the access log
into the SQLite store it already maintains. Filtering, sorting and pagination would become
SQL queries, retention would be handled by the cleanup cron already implemented there, and
the tail -n 1000 job could be dropped. Happy to open a separate issue for that if it is
of interest.

Environment

• Affects self-hosted only
• canary @ 53feb9dd5

[linear-code]

DOK-616