Skip to content

Clarify benchmark metadata wording #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LimiNode merged 1 commit into from
Dec 5, 2025
Merged

Clarify benchmark metadata wording #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions README-RU.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -701,10 +701,12 @@ LogIt++ включает библиотеку *fmt* для форматиров

Харнесс меряет end-to-end латентность (*вызов лога → доставка в sink*) и суммарную пропускную. Он полезен для поиска регрессий и сравнения дизайна пайплайнов, но это **не** идеальное соревнование «кто быстрее». LogIt++ осознанно тратит больше работы в духе Python `icecream`: один `LOGIT_*` может парсить имена аргументов, собирать `args_array` из `VariableValue` и опционально форматировать структуру. Классические printf-логгеры вроде spdlog оптимизируются под быстрое форматирование строк и очереди, без этой «леденцовой» ветки. Для корректного сравнения держите оба лагеря в одном режиме:

В этой методике LogIt++ проходит путь «record → formatter → sink/queue» с IceCream-подобными метаданными (имена/значения аргументов), а spdlog в адаптере получает уже готовую строку и измеряет «string → queue → sink».

- *Только текст / passthrough* показывает стоимость dispatch/очереди/sink и ближе всего к поведению spdlog по умолчанию.
- *IceCream-стиль метаданных* (`LOGIT_*` с захватом аргументов) включает парсинг имён и упаковку значений; тут LogIt++ делает больше работы на вызов намеренно.

Асинхронные цифры включают enqueue + пробуждение воркера/планирование ОС + работу sink; для file sink добавляется разброс из-за буферов/flush.
Асинхронные цифры включают enqueue + пробуждение воркера/планирование ОС + работу sink; для file sink добавляется разброс из-за буферов/flush. Латентности в async сильно зависят от размера thread_pool/overflow policy и поведения sink; числа ниже отражают именно адаптер из этого репозитория, а не «spdlog в целом».

### Последний снимок (05.12.2025)

Expand All @@ -713,6 +715,7 @@ LogIt++ включает библиотеку *fmt* для форматиров
- Метрики: медианная задержка (`p50`) в наносекундах и достигнутая пропускная способность (сообщений/с).
- Железо: 3 vCPU (Intel Xeon E5-2673 v4 @ 2.30GHz) в виртуальной машине, одна NUMA-нода.
- Данные: обновлено по `bench/results/latency-2025-12-05-10k.csv` (05.12.2025, 03:18 UTC).
- Таблица отражает только этот сценарий; полный набор — в CSV.

| Режим | Приёмник | LogIt++ p50 | Пропускная (LogIt++) | spdlog p50 | Пропускная (spdlog) |
|-------|----------|-------------|----------------------|------------|---------------------|
Expand All @@ -721,7 +724,7 @@ LogIt++ включает библиотеку *fmt* для форматиров
| Async | Null | 20 916 нс | 1 846 272 сооб./с | 1 248 779 нс | 1 303 573 сооб./с |
| Async | File | 255 323 нс | 651 384 сооб./с | 5 001 140 нс | 1 153 976 сооб./с |

**Выводы:** LogIt++ держит sub-µs p50 в синхронных режимах, даже с путём метаданных в стиле IceCream; spdlog выигрывает по скорости на null/file за счёт более лёгкого форматирования. В async обе стороны включают enqueue + пробуждения + sink: LogIt++ остаётся в десятках–сотнях микросекунд, тогда как адаптер spdlog в этой конфигурации уходит в миллисекунды.
**Выводы:** В синхронных режимах LogIt++ показывает p50 ~120–130 нс при IceCream-подобном пути метаданных; адаптер spdlog работает с готовой строкой, поэтому на Null/File быстрее в этой методике. В async обе стороны меряют enqueue + пробуждения + sink и чувствительны к конфигурации thread_pool/overflow/sink: здесь LogIt++ остаётся в десятках–сотнях микросекунд, а spdlog-адаптер уходит в миллисекунды и требует отдельной настройки/профиля для других конфигураций. При необходимости можно включить passthrough/fmt_only и отключить лишние метаданные.

### Как устроен бенч-харнесс (LatencyRecorder)

Expand Down
10 changes: 7 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -775,15 +775,18 @@ The harness times end-to-end latency (*log call → delivery into the sink*) and
regressions and comparing pipeline designs, but it is **not** a perfect “fastest logger wins” contest. LogIt++ intentionally does
extra work inspired by Python’s `icecream`: a single `LOGIT_*` call can extract argument names, build `args_array` with
`VariableValue`, and optionally format those structured values. Classic printf-style loggers such as spdlog focus on fast string
formatting and queueing instead of this metadata path. If you want an apples-to-apples view, keep the comparison within the same
formatting and queueing instead of this metadata path. In this harness LogIt++ travels the “record → formatter → sink/queue” path
with IceCream-inspired metadata (argument names/values), while the spdlog adapter receives an already formatted string and measures “string → queue → sink.”
If you want an apples-to-apples view, keep the comparison within the same
mode:

- *Text-only/passthrough* shows dispatch/queue/sink cost and is the closest to spdlog’s default path.
- *Metadata-heavy* (`LOGIT_*` with argument capture) includes parsing and packing the structured arguments; LogIt++ will do more
work per call here by design.

Async numbers also include enqueue + worker wakeup/scheduling + sink time; file sinks add I/O variance from buffering and flush
policies.
policies. Async latencies depend heavily on thread pool size/overflow policy and sink behavior; the values below reflect the
adapter in this repository rather than spdlog at large.

### Latest snapshot (Dec 05, 2025)

Expand All @@ -793,6 +796,7 @@ policies.
- Metrics: median (`p50`) latency in nanoseconds and achieved throughput (messages/sec).
- Hardware: 3 vCPU VM (Intel Xeon E5-2673 v4 @ 2.30GHz), single NUMA node.
- Data: refreshed from `bench/results/latency-2025-12-05-10k.csv` (Dec 05, 2025 @ 03:18 UTC).
- The table captures that single scenario; see the CSV for the full matrix.

| Mode | Sink | LogIt++ p50 | LogIt++ throughput | spdlog p50 | spdlog throughput |
|------|------|-------------|--------------------|------------|-------------------|
Expand All @@ -801,7 +805,7 @@ policies.
| Async | Null | 20,916 ns | 1,846,272 msg/s | 1,248,779 ns | 1,303,573 msg/s |
| Async | File | 255,323 ns | 651,384 msg/s | 5,001,140 ns | 1,153,976 msg/s |

**Takeaways:** LogIt++ keeps sub-microsecond p50s in synchronous modes while carrying the IceCream-style metadata path; spdlog’s lean formatting stays faster on the null/file sinks. Asynchronously, both numbers include enqueue + worker wakeups + sink work; LogIt++ stays in the tens-to-hundreds of microseconds, while the spdlog adapter lands in low-to-mid milliseconds for this run.
**Takeaways:** In synchronous modes LogIt++ shows p50 ~120–130 ns while carrying the IceCream-inspired metadata path; the spdlog adapter receives preformatted strings, so it remains faster on the null/file sinks in this scenario. Asynchronously, both sides measure enqueue + worker wakeups + sink work and are sensitive to thread-pool/overflow/sink configuration; here LogIt++ stays in the tens-to-hundreds of microseconds, while the spdlog adapter lands in low-to-mid milliseconds and would need tuning/profiling for other setups. Passthrough/fmt-only modes remain available if you want to trim the metadata cost.

### Benchmark harness notes (LatencyRecorder)

Expand Down
7 changes: 5 additions & 2 deletions docs/mainpage.dox
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -686,10 +686,12 @@ Run `./build/bench/logit_bench` to capture the full matrix (sync/async × null/f

The harness tracks end-to-end latency (*log call → sink delivery*) and throughput. It is great for regression hunting and pipeline design comparisons, but it is **not** a perfect “which logger is fastest” race. LogIt++ intentionally mirrors Python `icecream`: a single `LOGIT_*` call may parse argument names, build `args_array` with `VariableValue`, and optionally format those values. Many printf-style loggers (e.g., spdlog) optimize for lightweight formatting and queueing instead of this metadata path. Compare implementations inside the same mode:

In this harness LogIt++ travels the “record → formatter → sink/queue” path with IceCream-inspired metadata (argument names/values), while the spdlog adapter receives an already formatted string and measures “string → queue → sink.”

- *Text-only/passthrough* emphasizes dispatch/queue/sink cost and is closest to the spdlog default path.
- *Metadata-heavy* (`LOGIT_*` with argument capture) measures parsing and packing of structured arguments; LogIt++ does more per-call work here by design.

Async results include enqueue + worker wakeup/scheduling + sink time. File sinks add variability from buffering and flush policy.
Async results include enqueue + worker wakeup/scheduling + sink time. File sinks add variability from buffering and flush policy. Async latencies depend heavily on thread pool sizing/overflow policy and sink behavior; the numbers below reflect the adapter in this repository rather than spdlog at large.

\subsection bench_latest Latest snapshot (Dec 05, 2025)

Expand All @@ -698,6 +700,7 @@ Async results include enqueue + worker wakeup/scheduling + sink time. File sinks
- Metrics: median (`p50`) latency in nanoseconds and achieved throughput (messages/sec).
- Hardware: 3 vCPU VM (Intel Xeon E5-2673 v4 @ 2.30GHz), single NUMA node.
- Data: refreshed from `bench/results/latency-2025-12-05-10k.csv` (Dec 05, 2025 @ 03:18 UTC).
- The table reflects that single scenario; see the CSV for the full matrix.

| Mode | Sink | LogIt++ p50 | LogIt++ throughput | spdlog p50 | spdlog throughput |
|------|------|-------------|--------------------|------------|-------------------|
Expand All @@ -706,7 +709,7 @@ Async results include enqueue + worker wakeup/scheduling + sink time. File sinks
| Async | Null | 20,916 ns | 1,846,272 msg/s | 1,248,779 ns | 1,303,573 msg/s |
| Async | File | 255,323 ns | 651,384 msg/s | 5,001,140 ns | 1,153,976 msg/s |

**Takeaways:** LogIt++ keeps sub-microsecond p50s in synchronous modes while carrying the IceCream-style metadata path; spdlog’s lean formatting stays faster on the null/file sinks. Asynchronously, both numbers include enqueue + worker wakeups + sink work; LogIt++ stays in the tens-to-hundreds of microseconds, while the spdlog adapter lands in low-to-mid milliseconds for this run.
**Takeaways:** In synchronous modes LogIt++ shows p50 ~120–130 ns while carrying the IceCream-inspired metadata path; the spdlog adapter receives preformatted strings, so it remains faster on the null/file sinks in this scenario. Asynchronously, both sides measure enqueue + worker wakeups + sink work and are sensitive to thread-pool/overflow/sink configuration; here LogIt++ stays in the tens-to-hundreds of microseconds, while the spdlog adapter lands in low-to-mid milliseconds and would need tuning/profiling for other setups. Passthrough/fmt-only modes remain available if you want to trim the metadata cost.

\subsection bench_harness Benchmark context (LatencyRecorder)

Expand Down
Loading