Scale Logger for Production Use

[yaredtsy]

Description:
The current logger needs to scale to production workloads. We should improve the logging pipeline and investigate how production systems like Sentry handle ingestion, storage, filtering, and analytics.

This work should focus on making the logger reliable, fast, and useful at scale.

Scope:

• Review how tools like Sentry, Datadog, and OpenTelemetry design their logging pipelines
• Decide whether the logger should run as:
  • A deployed service
  • A standalone microservice
  • Or a pluggable external backend
• Design a production‑ready ingestion and storage flow

Requirements:

• Support high‑volume logging without blocking the app
• Allow logs to be filtered by:
  • Date / time range
  • Severity level
  • Function / node
  • Error type
• Improve error reporting:
  • Clear grouping of similar errors
  • Stack traces tied to the function that triggered them
  • Useful context without noise
• Add basic analytics:
  • Error frequency over time
  • Most failing functions
  • Recent regressions

Acceptance Criteria:

• Logger can handle production‑level traffic
• Logs can be queried efficiently by date and filters
• Error reports are readable and grouped
• Architecture supports future scaling and extensions

Notes:
This logger is a core part of the system’s philosophy—logs should live close to the function that produced them, while still being scalable and production‑ready.