diff --git a/.agents/tasks/otel-backend-implementation.md b/.agents/tasks/otel-backend-implementation.md new file mode 100644 index 000000000..5a34d4d0c --- /dev/null +++ b/.agents/tasks/otel-backend-implementation.md @@ -0,0 +1,263 @@ +--- +slug: otel-backend-implementation +branch: otel +owner: claude +status: in-review +started: 2026-06-27 +related-memories: [] +--- + +## Goal + +Ship an OpenTelemetry logger backend for Spine Logging that maps `LogData` to OTel +log records (`Logger.emit(...)`) and, on top of the same call path, a log-based +*events* surface. Success = an app can route Spine logs to an OTel pipeline by +adding one backend artifact and pointing it at an `OpenTelemetry` instance; the +record carries severity, message, log-site, throwable, Spine scope/log-site +metadata as attributes, and active-span trace correlation; all proven with tests. + +The companion research spec is [`otel-backend-report.md`](otel-backend-report.md) +(referred to below as "the report"). **This plan corrects several factual errors +in the report** (see *Context → Corrections*); where the two disagree, this plan +wins. + +## Context + +The plan is grounded in three sources, all read on 2026-06-27: + +1. **The Spine SPI** (verified `confirmed` against source) — the report describes it + accurately: + - [`BackendFactory.kt:81`](logging/src/commonMain/kotlin/io/spine/logging/backend/BackendFactory.kt) — `abstract fun create(loggingClass: String): LoggerBackend` + - [`LoggerBackend.kt`](logging/src/commonMain/kotlin/io/spine/logging/backend/LoggerBackend.kt) — `loggerName: String?` (65), `isLoggable(Level)` (72), `log(LogData)` (80), `handleError(RuntimeException, LogData)` (118) + - [`LogData.kt`](logging/src/commonMain/kotlin/io/spine/logging/backend/LogData.kt) — `level`, `timestampNanos: Long`, `loggerName`, `logSite`, `metadata`, `wasForced()`, `literalArgument` + - [`Level.kt:56`](logging/src/commonMain/kotlin/io/spine/logging/Level.kt) — `data class Level(name, value)`; FATAL=2000, ERROR/SEVERE=1000, WARNING=900, INFO=800, CONFIG=700, DEBUG/FINE=500, FINER/TRACE=400, FINEST=300 + - [`MetadataProcessor.kt`](logging/src/commonMain/kotlin/io/spine/logging/backend/MetadataProcessor.kt) — `forScopeAndLogSite(scope, logged)` (83), `process(handler, ctx)` (133), `getSingleValue(key)` (151), `keySet()` (168) + - [`MetadataHandler.kt`](logging/src/commonMain/kotlin/io/spine/logging/backend/MetadataHandler.kt) — `handle(key, value, ctx)` (62) and the repeated-key hook **`handleRepeated(key, values: Iterator, ctx)`** (78); build via `MetadataHandler.builder()` + - [`SimpleMessageFormatter.kt:167`](logging/src/commonMain/kotlin/io/spine/logging/backend/SimpleMessageFormatter.kt) — `getLiteralLogMessage(logData): String` (message text without the `[CONTEXT …]` suffix) + - [`Platform.kt:182`](logging/src/commonMain/kotlin/io/spine/logging/backend/Platform.kt) — `getInjectedMetadata(): Metadata` + - [`LogContext.kt:585`](logging/src/commonMain/kotlin/io/spine/logging/LogContext.kt) — `Key.LOG_CAUSE: MetadataKey` + +2. **The reference backends to mirror**: + - [`log4j2-backend`](backends/log4j2-backend) — the mapping template ([`LogEvents.kt`](backends/log4j2-backend/src/main/kotlin/io/spine/logging/backend/log4j2/LogEvents.kt): `MetadataProcessor.forScopeAndLogSite` → format → `getSingleValue(LOG_CAUSE)` → `MetadataHandler.builder()`, with `Tags` special-cased and `ValueQueue` for repeated keys). + - [`probe-backend`](backends/probe-backend) — the `@AutoService` + KSP registration pattern ([`build.gradle.kts`](backends/probe-backend/build.gradle.kts): `plugins { ksp }`, `implementation(AutoService.annotations)`, `ksp(AutoServiceKsp.processor)`) and the memoizing test backend. + - [`logging/build.gradle.kts`](logging/build.gradle.kts) — the **`kmp-module`** shape: `kotlin { sourceSets { commonMain {…}; jvmMain { runtimeOnly(project(":jvm-default-platform")) }; jvmTest {…} } }`, JVM target only. + +3. **The proven OTel-Kotlin precedent — the `core-jvm` repo's `server-otel` module** + (sibling checkout at `/Users/sanders/Projects/Spine/core-jvm`). It already uses + the native Kotlin SDK successfully and gives us copy-ready artifacts: + - `core-jvm/buildSrc/src/main/kotlin/io/spine/dependency/lib/OpenTelemetryKotlin.kt` — a `Dependency()` object pinning **0.4.0**, group `io.opentelemetry.kotlin`, exposing `api`, `noop`, `core`, `implementation`, `compat`. **Copy it verbatim into this repo's `buildSrc/.../lib/`.** + - `core-jvm/server-otel/build.gradle.kts` — production depends on **`api` only**; tests add `core` + `implementation` (the native SDK). + - `core-jvm/.../given/TestOtel.kt` — the SDK is built with `createOpenTelemetry { tracerProvider { export { processor } } }` and a hand-rolled recording processor. **The logs test path mirrors this with a recording `LogRecordProcessor` — no in-memory-exporter artifact required.** + + The actual **0.4.0 Logs API** was read from the api-discovery cache + (`io.opentelemetry.kotlin:api:0.4.0`) and matches the report's skeletons: + `Logger.emit(body, eventName, timestamp: Long?, observedTimestamp, context, severityNumber, severityText, exception, attributes)`, + `Logger.enabled(context, severityNumber, eventName)`, + `LoggerProvider.getLogger(name, version, schemaUrl, attributes)`, + `OpenTelemetry.loggerProvider`, and the `SeverityNumber` enum (1–24; `DEBUG4=8`). + +### Corrections to the report (verified 2026-06-27) + +| # | Report says | Reality | Consequence | +|---|-------------|---------|-------------| +| C1 | `opentelemetry-kotlin` **0.5.0** | Latest is **0.4.0** (2026-05-20); 0.5.0 does not exist | Pin `0.4.0` (matches `core-jvm`). | +| C2 | Version catalog `gradle/libs.versions.toml` | Repo uses **buildSrc Kotlin objects** | Copy `core-jvm`'s `OpenTelemetryKotlin` object into `buildSrc/.../lib/`, not a TOML entry. | +| C3 | Service file in `jvmMain/resources`, hand-written | Repo standard is **`@AutoService` + KSP** (probe-backend) | Annotate the JVM factory `@AutoService(BackendFactory::class)`; KSP generates the service file. | +| C4 | Test with `exporters-in-memory` (Kotlin); compat bridge for the SDK | Kotlin in-memory exporter artifact unconfirmed; `core-jvm` uses the **native Kotlin SDK** + a hand-rolled recording processor | Use the **native SDK** (`core`+`implementation`); tests capture records with a recording `LogRecordProcessor`. No compat, no in-memory artifact. | +| C5 | A `std` backend exists | Only `log4j2`, `jul`, `probe` exist | Ignore; mirror `log4j2`. | +| C6 | `OtelBackendSettings.use(...)` reuses an existing pattern | No backend has programmatic injection today; JVM precedent is the `spine.logging.backend_factory` system-property override ([`DefaultPlatform.kt`](platforms/jvm-default-platform/src/main/kotlin/io/spine/logging/backend/system/DefaultPlatform.kt)) | `OtelBackendSettings` is a **new** convention; model its `@Volatile`/no-op-default shape on `DefaultPlatform`'s loader and document it. | + +Two API refinements (vs the report's prose): +- `AttributesMutator` has **no `Int`/`Float` setters** — only `setLongAttribute`/`setDoubleAttribute` (+ `*List*`, `setByteArrayAttribute`, `setAnyValueAttribute`). The attribute handler must widen `Int`/`Short`/`Byte` → `Long` and `Float` → `Double`. +- `Logger.emit(timestamp: Long?)` — the param exists, but **the unit (epoch nanos vs millis) is not documented in the API**; it is resolved by the SDK. Confirm against `:implementation` 0.4.0 in Phase 0 before wiring `data.timestampNanos`. + +### Decisions (resolved) + +| Topic | Decision | +|-------|----------| +| **JVM SDK strategy** | **Native Kotlin SDK** (`core`+`implementation`) from the start — proven in `core-jvm/server-otel`. No compat bridge. | +| **Module platform** | **`kmp-module`** with the JVM target only (mirrors core `logging`). Mapping in `commonMain` (only touches OTel `:api` + Spine commonMain SPI); JVM-only registration in `jvmMain`. Aims at KMP up front to cut future migration, even though only JVM is wired today. | +| **Registration** | **`@AutoService(BackendFactory::class)` + KSP** (`kspJvm`), as probe-backend does. | +| **Events / OTEP 4430** | Events emit through the Logs API (`emit(eventName = …)`); never `Span.AddEvent`. | +| **Body vs attributes** | Body = `getLiteralLogMessage(...)` (no `[CONTEXT]` suffix); metadata → attributes (no double-encoding). | +| **CONFIG severity** | `DEBUG4` (judgment call, documented in KDoc) — open to `INFO` if preferred. | + +## Architecture + +``` +backends/otel-backend/ ← kmp-module, JVM target only (for now) + src/commonMain/kotlin/io/spine/logging/backend/otel/ + OtelLoggerBackend.kt ← LoggerBackend; LogData → Logger.emit(...) [api only] + OtelBackendSettings.kt ← @Volatile holder; default NoopOpenTelemetry; use(otel) + SeverityMapping.kt ← Level.toSeverityNumber() + AttributeMapping.kt ← MetadataHandler + LogSite → code.* semconv + src/jvmMain/kotlin/io/spine/logging/backend/otel/ + OtelBackendFactory.kt ← @AutoService(BackendFactory::class); resolves OpenTelemetry from settings + src/jvmTest/kotlin/... ← native SDK + recording LogRecordProcessor + Kotest/JUnit5 specs + +backends/otel-backend-bootstrap/ ← OPTIONAL, Phase 2 (jvmMain SDK init, turnkey wiring) +``` + +- `commonMain` depends on `OpenTelemetryKotlin.api` + `.noop` + `project(":logging")`. +- `jvmMain` adds `AutoService.annotations` (+ `kspJvm(AutoServiceKsp.processor)`), and + the factory carries `@AutoService` directly — it is a no-arg `class`, so unlike + probe-backend's `object` it needs no adapter shim. +- `jvmTest` adds `OpenTelemetryKotlin.core` + `.implementation`, `logging-testlib`, and + `runtimeOnly(project(":jvm-default-platform"))` so `DefaultPlatform` discovers the + `@AutoService` factory end-to-end. +- **Build wrinkle to validate (Phase 0):** KSP on a `kmp-module` uses `kspJvm(...)`, + not the plain `ksp(...)` probe-backend uses on its `jvm-module`. If KSP-on-KMP + misbehaves with the `kmp-module` convention plugin, fall back to a hand-written + `src/jvmMain/resources/META-INF/services/io.spine.logging.backend.BackendFactory`. + +## Open decisions + +1. **CONFIG severity** — `DEBUG4` (default) vs `INFO`. Cosmetic; documented either way. +2. **Bootstrap module SDK config surface** (Phase 2) — env-var driven vs explicit DSL. + Defer until Phase 1 lands. + +## Plan + +### Phase 0 — spike: native SDK end-to-end on JVM (de-risk) ✅ +- [x] Copy `core-jvm`'s `OpenTelemetryKotlin.kt` into + [`buildSrc/.../lib/OpenTelemetryKotlin.kt`](buildSrc/src/main/kotlin/io/spine/dependency/lib/OpenTelemetryKotlin.kt) + (0.4.0; api/noop/core/implementation/compat). `AutoService`/`AutoServiceKsp` already exist. +- [x] Register the module: `"otel-backend"` added to `includeBackend(...)` in + [`settings.gradle.kts`](settings.gradle.kts). +- [x] [`build.gradle.kts`](backends/otel-backend/build.gradle.kts): `plugins { kmp-module; ksp }`; + `commonMain` → `api(OpenTelemetryKotlin.api)` + `noop` + `:logging`; `jvmMain` → + `AutoService.annotations` + `kspJvm(AutoServiceKsp.processor)`; `jvmTest` → `core` + + `implementation` + `logging-testlib` + `runtimeOnly(:jvm-default-platform)`. + **Two build wrinkles found & fixed:** (1) KSP-on-KMP works via `kspJvm` (no fallback + needed); (2) `kmp-module` does NOT put the JUnit Platform on `jvmTest` (only + `jvm-module` configures the `test` task), so added an explicit + `tasks.named("jvmTest") { useJUnitPlatform() }`. +- [x] **Logs SDK DSL + timestamp unit confirmed** (upstream `v0.4.0` source): + `createOpenTelemetry { loggerProvider { export { processor } } }`, + `LogRecordProcessor.onEmit(ReadWriteLogRecord, Context)`, `ReadableLogRecord` + getters, and `emit(timestamp)` = **epoch nanoseconds** (passes through unchanged). +- [x] [`RecordingLogRecordProcessor`](backends/otel-backend/src/jvmTest/kotlin/io/spine/logging/backend/otel/given/RecordingLogRecordProcessor.kt) + (mirrors `core-jvm`'s `RecordingSpanProcessor`) drives the end-to-end test — no + in-memory-exporter artifact needed. + +### Phase 1 — production backend ✅ +- [x] [`OtelBackendSettings.kt`](backends/otel-backend/src/commonMain/kotlin/io/spine/logging/backend/otel/OtelBackendSettings.kt): + `@Volatile` holder, `NoopOpenTelemetry` default, `use()`/`current()`. Documented as new convention. +- [x] [`SeverityMapping.kt`](backends/otel-backend/src/commonMain/kotlin/io/spine/logging/backend/otel/SeverityMapping.kt): + numeric-threshold `Level.toSeverityNumber()`; CONFIG → `DEBUG4`. +- [x] [`OtelLoggerBackend.kt`](backends/otel-backend/src/commonMain/kotlin/io/spine/logging/backend/otel/OtelLoggerBackend.kt): + `isLoggable` via `enabled`; `log` maps body (`getLiteralLogMessage`), severity, nanos + timestamp, cause→exception, attributes; implicit context for correlation; `handleError`. +- [x] [`AttributeMapping.kt`](backends/otel-backend/src/commonMain/kotlin/io/spine/logging/backend/otel/AttributeMapping.kt): + `MetadataHandler.builder()` with default single + `setDefaultRepeatedHandler` + (repeated→`*ListAttribute`), `.ignoring(LOG_CAUSE)`, `value is Tags` → `spine.tag.`; + LogSite → `code.*`; numeric widening (no `Int`/`Float` setters in `AttributesMutator`). +- [x] [`OtelBackendFactory.kt`](backends/otel-backend/src/jvmMain/kotlin/io/spine/logging/backend/otel/OtelBackendFactory.kt): + `@AutoService(BackendFactory::class)`; resolves the logger from `OtelBackendSettings`. +- [x] [`OtelLoggerBackendSpec`](backends/otel-backend/src/jvmTest/kotlin/io/spine/logging/backend/otel/OtelLoggerBackendSpec.kt) — + **10 tests, all passing**: body, no `[CONTEXT]`, severity number+text, `code.*`, + `spine.*` single + repeated→list, cause→`exception.*` (not `spine.cause`), timestamp + passthrough, and factory resolution via `OtelBackendSettings`. + +### Phase 2 — bootstrap module + correlation hardening +- [x] **Trace-correlation test** — [`OtelLoggerBackendSpec`](backends/otel-backend/src/jvmTest/kotlin/io/spine/logging/backend/otel/OtelLoggerBackendSpec.kt) + `correlate the record with the active span`: a span is made current via + `otel.context.implicit().storeSpan(span).attach()`, then `log()` (with implicit + `context = null`) stamps the span's trace/span ids onto the record. **Passes** — + closes the report §5.5 / §10.4 correlation risk. (Coroutine-dispatch propagation + is a consumer concern; left for a consumer-side test.) +- [x] [`backends/otel-backend-bootstrap`](backends/otel-backend-bootstrap) (`jvm-module`): + [`OtelLogging`](backends/otel-backend-bootstrap/src/main/kotlin/io/spine/logging/backend/otel/bootstrap/OtelLogging.kt) + `installOtlpHttp(endpoint)` / `fromEnvironment()` builds a native SDK via + `createOpenTelemetry { loggerProvider { export { batchLogRecordProcessor(otlpHttpLogRecordExporter(endpoint)) } } }`, + installs it through `OtelBackendSettings.use(...)`, and returns an `AutoCloseable` + that uninstalls + shuts the SDK down (`runBlocking { (otel as TelemetryCloseable).shutdown() }`). + Real 0.4.0 OTLP API confirmed from upstream source: `otlpHttpLogRecordExporter` + (`exporters-otlp`) + `batchLogRecordProcessor` (`exporters-core`), both extensions on + `LogExportConfigDsl` in `io.opentelemetry.kotlin.logging.export`. **Builds green; + smoke test passes.** +- [ ] Optional early-record ring buffer in `OtelBackendSettings`, replayed on `use(...)` + (report §4) — deferred; only if bootstrap-order logs prove to matter. + +### Phase 3 — log-based events API + domain-event integration +- [x] Low-level: [`EVENT_NAME`](backends/otel-backend/src/commonMain/kotlin/io/spine/logging/backend/otel/OtelEvents.kt) + `MetadataKey` (label `otelEventName` — Spine labels disallow dots) the + backend pulls out and passes as `emit(eventName = …)`; ignored as an attribute. + Verified by `emit a named event when the event-name metadata is set`. +- [x] Ergonomic [`WithLogging.logEvent(name, level, message)`](backends/otel-backend/src/commonMain/kotlin/io/spine/logging/backend/otel/OtelEvents.kt) — + sugar over `at(level).with(EVENT_NAME, name).log { … }`. Verified end-to-end by + `emit a named event through the 'logEvent' API end-to-end`. +- [x] Domain-event → observability-event bridge: **documented**, not coded — it needs Spine + proto/domain types from consumer repos (e.g. `core-jvm`, `io.spine:spine-server`), so it + belongs in the consumer, not this library. The recipe (proto type name → `event.name`, + selected fields + scope → attributes, active span → correlation) and the + domain-event-vs-observability-event boundary are in + [`README.md`](backends/otel-backend/README.md) (report §6.3). + +### Phase 4 — non-JVM targets (deferred; separate initiative) +- [ ] Only when Spine grows non-JVM backend registration (today only + `jvm-default-platform` + ServiceLoader exist; `actual fun loadPlatform()` is + JVM-only). The `commonMain` mapping already built here is the reusable part; + each target needs its own discovery mechanism + an OTLP exporter (iOS/native OTLP + is **not** available upstream yet — correction C4). + +## Risks & open questions (status after verification) + +1. **`opentelemetry-kotlin` maturity** — 0.4.0, `@ExperimentalApi`, logs in *Development*. + `@OptIn` everywhere; pin exactly; expect breaking changes. *(Open — accepted.)* +2. **`emit(timestamp)` unit** — param is `Long?`, unit undocumented in `:api`. Confirm + against `:implementation` in Phase 0. *(Open — verify.)* +3. **Logs SDK DSL** — `loggerProvider { export { … } }` inferred by analogy to the + tracer DSL in `core-jvm`; confirm against `:implementation` 0.4.0 in Phase 0. *(Open — verify.)* +4. **KSP on `kmp-module`** — `kspJvm` path is unexercised in this repo; validate in + Phase 0, fallback = hand-written service file. *(Open — verify.)* +5. **Coroutine context propagation** — correlation across suspension unproven; Phase 2 + test. *(Open.)* +6. **Test exporter** — resolved: hand-rolled recording `LogRecordProcessor` like + `core-jvm`, no in-memory artifact. *(Resolved.)* +7. **Non-JVM registration** — genuinely unpaved in Spine; keeps KMP export out of + near-term scope. *(Resolved → Phase 4.)* + +## Testing + +- JVM, native Kotlin SDK (`core`+`implementation`) configured via + `createOpenTelemetry { loggerProvider { export { recordingProcessor } } }`, mirroring + `core-jvm/server-otel`'s `TestOtel`/`RecordingSpanProcessor`. +- Assert: severity number, body without context suffix, namespaced + widened + repeated→list + attributes, exception recorded, `event.name` when set, trace/span id under an active + span, and the coroutine-correlation case. +- `runtimeOnly(:jvm-default-platform)` in `jvmTest` for the `@AutoService` discovery path. + +## Log + +- 2026-06-27 — Drafted from `otel-backend-report.md` after a 6-agent verification pass + (SPI signatures, log4j2 template, build conventions, platform selection, task-doc + format, upstream `opentelemetry-kotlin` status). Recorded corrections C1–C6. +- 2026-06-27 — Scope confirmed by maintainer: JVM-first, Phases 0–3, KMP (Phase 4) + deferred. +- 2026-06-27 — Two implementation decisions taken: **native Kotlin SDK** from the start + (proven in `core-jvm/server-otel`) and **`@AutoService` + KSP** registration. Module + retargeted to **`kmp-module`** (JVM only) with the mapping in `commonMain` to minimise + future KMP migration. Verified the real 0.4.0 Logs API from the api-discovery cache + and the `core-jvm` `OpenTelemetryKotlin` dep object / native-SDK test pattern. +- 2026-06-27 — **Phases 0 & 1 implemented and verified.** `./gradlew :otel-backend:build` + is green (compile + detekt + kover + license); `:otel-backend:jvmTest` runs **10/10 + passing** (had to JDK-17 the build and add `useJUnitPlatform()` to the kmp `jvmTest`). + KSP emits the `@AutoService` service file naming `OtelBackendFactory`. Timestamp unit + and logs SDK DSL confirmed against upstream `v0.4.0`. +- 2026-06-27 — **Phase 2 (correlation + bootstrap) and Phase 3 (events + docs) done.** + Added the correlation test, the `logEvent`/`EVENT_NAME` events surface, the README + (incl. the consumer-side domain-event recipe), and the `otel-backend-bootstrap` OTLP + module. otel-backend **13/13** tests; bootstrap **2/2**; both detekt-clean; Dokka clean. + The real 0.4.0 OTLP API (`otlpHttpLogRecordExporter` in `exporters-otlp`, + `batchLogRecordProcessor` in `exporters-core`) was confirmed from upstream source — the + getting-started docs' helpers postdate 0.4.0, so empirical compilation + upstream + reading were needed. +- 2026-06-27 — **Reviewed.** `kotlin-engineer` → APPROVE (no MUST violations; `@Volatile` + holder, `runBlocking` in `close()`, numeric widening, severity thresholds, null-safety, + explicit-API all verified correct). `spine-code-review` → APPROVE WITH CHANGES (nits; + version gate satisfied at `2.0.0-SNAPSHOT.418`). Applied: endpoint constant in the smoke + test, a `fromEnvironment()` test, `@AfterEach` restoring the no-op holder, safe cast in + `close()`, null-message guard in `handleError`, a widening-KDoc note, and README + line-length/Dokka-link fixes. Status → `in-review`. **Not committed** — awaiting the + maintainer to review and open the PR. diff --git a/.agents/tasks/otel-backend-report.md b/.agents/tasks/otel-backend-report.md new file mode 100644 index 000000000..7cc72ef13 --- /dev/null +++ b/.agents/tasks/otel-backend-report.md @@ -0,0 +1,642 @@ +# Implementing an OpenTelemetry Logger Backend for Spine Logging (KMP) + +**Status:** Research / implementation spec — ready to drive Claude Code sessions +**Target library:** `SpineEventEngine/logging` +**OTel client:** `open-telemetry/opentelemetry-kotlin` (KMP), `io.opentelemetry.kotlin:*` v0.5.0 ( +`@ExperimentalApi`) +**Scope:** Logger backend (Spine `LogData` → OTel log records) **plus** a log-based events surface +**SDK lifecycle:** support both *inject an existing instance* and *bootstrap from config* + +--- + +## 0. How to use this document in Claude Code + +This is a self-contained spec. A Claude Code session does not have the conversation that produced +it, so everything needed is inline: the exact Spine SPI signatures, the exact `opentelemetry-kotlin` +API surface, the mapping rules, code skeletons, and the open risks. Treat the code blocks as +*starting skeletons to verify against the pinned dependency versions*, not as copy-paste-final — +`opentelemetry-kotlin` is pre-1.0 and `@ExperimentalApi`, so signatures can shift between releases. + +Recommended working order is the phased plan in §11. Start with the Phase 0 JVM-via-compat spike +before committing to the native KMP path. + +--- + +## 1. Decision record + +| Decision | Choice | Rationale | +|-----------------|-------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| +| OTel client | `opentelemetry-kotlin` (KMP) | Multiplatform reach matching Spine's `commonMain` ethos; idiomatic Kotlin API. Trade-off: logs are in *Development* status and the API is `@ExperimentalApi`. | +| Platform target | KMP, JVM-first | JVM via the `compat` bridge over `opentelemetry-java` de-risks maturity; native/JS follow. | +| Scope | Backend + log-based events | `Logger.emit(eventName = …)` makes events the *same* call path as logs — minimal extra surface. | +| SDK lifecycle | Both (inject default, optional bootstrap) | Core backend stays `:api`-only; a separate optional module bootstraps a real SDK. | +| Span events | Log-based only (no `Span.AddEvent`) | Aligns with OTEP 4430 (Span Event API deprecation). | + +--- + +## 2. Background — the two sides + +### 2.1 Spine Logging backend SPI (ground truth from the repo) + +The SPI lives in `logging/src/commonMain/kotlin/io/spine/logging/backend/`. It is Flogger-derived +but slimmed down. The four types you implement against: + +```kotlin +// BackendFactory.kt — discovered + instantiated by the platform +public abstract class BackendFactory { + public abstract fun create(loggingClass: String): LoggerBackend +} + +// LoggerBackend.kt — the thing you build +public abstract class LoggerBackend { + public abstract val loggerName: String? + public abstract fun isLoggable(level: Level): Boolean + public abstract fun log(data: LogData) + public abstract fun handleError(error: RuntimeException, badData: LogData) +} + +// LogData.kt — the record handed to log() +public interface LogData { + public val level: Level + public val timestampNanos: Long + public val loggerName: String? + public val logSite: LogSite // className, methodName, fileName, lineNumber + public val metadata: Metadata + public fun wasForced(): Boolean + public val literalArgument: Any? +} +``` + +`Level` is a plain data class (not `java.util.logging.Level`), so it is multiplatform-safe: + +```kotlin +public data class Level(val name: String, val value: Int) +// OFF=MAX, FATAL=2000, ERROR/SEVERE=1000, WARNING=900, INFO=800, +// CONFIG=700, DEBUG/FINE=500, FINER/TRACE=400, FINEST=300, ALL=MIN +``` + +`Metadata` is an indexed collection of typed keys; `MetadataKey` carries a `label`, a `canRepeat` +flag, and `cast()`: + +```kotlin +public abstract class Metadata { + public abstract fun size(): Int + public abstract fun getKey(n: Int): MetadataKey + public abstract fun getValue(n: Int): Any + public abstract fun findValue(key: MetadataKey): T? +} +``` + +Two helper types you will use rather than hand-rolling iteration: + +- `MetadataProcessor` — merges *scope* metadata (`ScopedLoggingContext`) with *log-site* metadata + and iterates them: `forScopeAndLogSite(scope, logged)`, then `process(handler, context)`, plus + `getSingleValue(key)`, `keySet()`. +- `MetadataHandler` — visitor: + `abstract fun handle(key: MetadataKey, value: T, context: C)` (with separate handling + hooks for repeated keys). +- `SimpleMessageFormatter` — renders the message. Critically it exposes + `getLiteralLogMessage(logData): String` (message text **without** the `[CONTEXT …]` suffix) in + addition to the full `format(logData, metadata)`. The default formatter already ignores + `LogContext.Key.LOG_CAUSE` when appending context. + +The cause/throwable is carried as metadata under `LogContext.Key.LOG_CAUSE`, extracted via +`metadata.getSingleValue(LogContext.Key.LOG_CAUSE)`. + +### 2.2 Reference implementation — the Log4j2 backend + +The shipped `backends/log4j2-backend` is the template to mirror. Its shape, condensed: + +```kotlin +public class Log4j2BackendFactory : BackendFactory() { + override fun create(loggingClass: String): LoggerBackend { + val name = loggingClass.replace('$', '.') + val logger = LogManager.getLogger(name) as Logger + return Log4j2LoggerBackend(logger) + } +} + +internal class Log4j2LoggerBackend(private val logger: Logger) : LoggerBackend() { + override val loggerName get() = logger.name + override fun isLoggable(level: Level) = logger.isEnabled(level.toLog4j()) + override fun log(data: LogData) = logger.get().log(toLog4jLogEvent(logger.name, data)) + override fun handleError(error: RuntimeException, badData: LogData) = + logger.get().log(toLog4jLogEvent(logger.name, error, badData)) +} +``` + +Registered via +`backends/log4j2-backend/src/main/resources/META-INF/services/io.spine.logging.backend.BackendFactory` +containing one line: the factory FQN. Note this is **JVM `ServiceLoader`** — see §9 for why that +matters for KMP. + +Inside `toLog4jLogEvent` the pattern you will replicate is: + +1. `MetadataProcessor.forScopeAndLogSite(Platform.getInjectedMetadata(), logData.metadata)` +2. format the message, +3. pull the cause via `getSingleValue(LOG_CAUSE)`, +4. project log-site + metadata into the target representation. + +### 2.3 opentelemetry-kotlin Logs/Events API (ground truth from the repo) + +Repo: `open-telemetry/opentelemetry-kotlin` (donated by Embrace). Version **0.5.0**. The logs API is +annotated `@ExperimentalApi`. + +Module map (relevant subset): + +| Artifact | Use | +|----------------------------------------------------|------------------------------------------------------------------------| +| `io.opentelemetry.kotlin:api` | Instrumentation API — **the only dependency the backend itself needs** | +| `io.opentelemetry.kotlin:noop` | `NoopOpenTelemetry` default instance | +| `io.opentelemetry.kotlin:core` + `:implementation` | The native Kotlin SDK (only where you initialize) | +| `io.opentelemetry.kotlin:compat` | JVM/Android bridge over `opentelemetry-java` | +| `io.opentelemetry.kotlin:exporters-otlp` | OTLP exporters | +| `io.opentelemetry.kotlin:exporters-in-memory` | Test exporter | + +The API entry point and logs surface: + +```kotlin +public interface OpenTelemetry { + public val loggerProvider: LoggerProvider + public val tracerProvider: TracerProvider + public val context: ContextFactory + public val span: SpanFactory + public val baggage: BaggageFactory + // … meterProvider, propagator, etc. +} + +public interface LoggerProvider { + public fun getLogger( + name: String, version: String? = null, schemaUrl: String? = null, + attributes: (AttributesMutator.() -> Unit)? = null, + ): Logger +} + +public interface Logger { + public fun enabled( + context: Context? = null, + severityNumber: SeverityNumber? = null, + eventName: String? = null, + ): Boolean + + public fun emit( + body: Any? = null, + eventName: String? = null, // ← log-based EVENTS are just this param + timestamp: Long? = null, + observedTimestamp: Long? = null, + context: Context? = null, + severityNumber: SeverityNumber? = null, + severityText: String? = null, + exception: Throwable? = null, + attributes: (AttributesMutator.() -> Unit)? = null, + ) +} +``` + +`SeverityNumber` is the standard 1–24 OTel scale ( +`TRACE=1 … DEBUG=5 … INFO=9 … WARN=13 … ERROR=17 … FATAL=21`, each with `2/3/4` sub-levels). +`AttributesMutator` is a typed builder: + +```kotlin +public interface AttributesMutator { + public fun setBooleanAttribute(key: String, value: Boolean) + public fun setStringAttribute(key: String, value: String) + public fun setLongAttribute(key: String, value: Long) + public fun setDoubleAttribute(key: String, value: Double) + public fun setStringListAttribute( + key: String, + value: List + ) // + Boolean/Long/Double list + public fun setByteArrayAttribute(key: String, value: ByteArray) + public fun setAnyValueAttribute(key: String, value: AnyValue) +} +``` + +**Compat bridge (JVM/Android only, `compat` module):** + +```kotlin +// wrap an existing opentelemetry-java instance +val otelKotlin: OpenTelemetry = otelJava.toOtelKotlinApi() +// or create a Kotlin API backed by the Java SDK +val otelKotlin: OpenTelemetry = createCompatOpenTelemetry { /* configure */ } +``` + +This is the key maturity hedge: on the JVM you get the Kotlin API surface while the **Stable** +`opentelemetry-java` logs SDK does the actual work. + +--- + +## 3. Architecture + +``` +backends/otel-backend/ ← KMP module, commonMain depends ONLY on :api (+ :noop) + commonMain/ + OtelBackendFactory.kt ← BackendFactory; resolves OpenTelemetry from holder + OtelLoggerBackend.kt ← LoggerBackend; maps LogData → Logger.emit(...) + SeverityMapping.kt ← Level → SeverityNumber + AttributeMapping.kt ← Metadata/LogSite → AttributesMutator + OtelBackendSettings.kt ← holder for the OpenTelemetry instance (inject mode) + events/ ← log-based events surface (Phase 3) + jvmMain/ + resources/META-INF/services/io.spine.logging.backend.BackendFactory + +backends/otel-backend-bootstrap/ ← OPTIONAL, separate artifact (bootstrap mode) + depends on :core/:implementation/:exporters-otlp OR :compat + opentelemetry-java BOM + builds a real SDK from env/config and calls OtelBackendSettings.use(...) +``` + +**Why the split:** the OTel docs are explicit that `core`/`compat`/`implementation` should not be a +dependency of any module that isn't initializing the SDK. Keeping the backend `:api`-only means +consumers who already run an OTel SDK just point the backend at it; consumers who want turnkey +wiring add the bootstrap artifact. + +--- + +## 4. SDK lifecycle — supporting both modes + +The SPI constraint: `BackendFactory` is no-arg-constructed by the platform (via `ServiceLoader` on +JVM), so the factory **cannot take the `OpenTelemetry` instance through its constructor**. It must +resolve it from a settable holder, defaulting to no-op. + +```kotlin +// OtelBackendSettings.kt (commonMain, :api only) +public object OtelBackendSettings { + @Volatile + private var instance: OpenTelemetry = NoopOpenTelemetry + + /** Inject mode: app calls this once at startup. */ + public fun use(openTelemetry: OpenTelemetry) { + instance = openTelemetry + } + + internal fun current(): OpenTelemetry = instance +} +``` + +- **Inject mode (recommended default):** the app sets the instance at bootstrap. On JVM this can be + a native Kotlin SDK *or* `javaOtel.toOtelKotlinApi()`: + ```kotlin + OtelBackendSettings.use(GlobalOpenTelemetry.get().toOtelKotlinApi()) // JVM + compat + // or + OtelBackendSettings.use(createOpenTelemetry { loggerProvider { export { … } } }) // native + ``` +- **Bootstrap mode:** the optional `otel-backend-bootstrap` module reads config (e.g. + `OTEL_EXPORTER_OTLP_ENDPOINT`), builds an SDK, and calls `OtelBackendSettings.use(...)`. It owns + the SDK lifecycle and shutdown. + +**Early-record caveat (mirror the OTel appender behavior):** anything logged before `use(...)` runs +goes to `NoopOpenTelemetry` and is dropped. If bootstrap-order logs matter, add a small bounded ring +buffer in the holder that replays into the real instance on `use(...)`. Treat as a Phase 2 +enhancement. + +--- + +## 5. The mapping — `LogData` → `Logger.emit(...)` + +This is the core of the backend. The `OtelLoggerBackend`: + +```kotlin +internal class OtelLoggerBackend( + private val logger: Logger, // io.opentelemetry.kotlin.logging.Logger + override val loggerName: String?, +) : LoggerBackend() { + + override fun isLoggable(level: Level): Boolean = + logger.enabled(severityNumber = level.toSeverityNumber()) + + override fun log(data: LogData) { + val metadata = MetadataProcessor.forScopeAndLogSite( + Platform.getInjectedMetadata(), data.metadata + ) + val cause = metadata.getSingleValue(LogContext.Key.LOG_CAUSE) + logger.emit( + body = SimpleMessageFormatter.getLiteralLogMessage(data), // text only, no [CONTEXT] + timestamp = data.timestampNanos, // VERIFY unit (see §10) + severityNumber = data.level.toSeverityNumber(), + severityText = data.level.name, + exception = cause, + context = null, // null ⇒ active context (see §5.5) + attributes = { applyAttributes(data, metadata) }, + ) + } + + override fun handleError(error: RuntimeException, badData: LogData) { + logger.emit( + body = "Spine logging backend error: ${error.message}", + severityNumber = SeverityNumber.ERROR, + severityText = "ERROR", + exception = error, + attributes = { setStringAttribute("spine.logging.bad_data", badData.toString()) }, + ) + } +} +``` + +### 5.1 Severity — `Level` → `SeverityNumber` + +Use a numeric-threshold mapping rather than name-matching, so custom Spine levels degrade sensibly: + +```kotlin +fun Level.toSeverityNumber(): SeverityNumber = when { + value >= Level.FATAL.value -> SeverityNumber.FATAL // 2000 → 21 + value >= Level.ERROR.value -> SeverityNumber.ERROR // 1000 → 17 + value >= Level.WARNING.value -> SeverityNumber.WARN // 900 → 13 + value >= Level.INFO.value -> SeverityNumber.INFO // 800 → 9 + value >= Level.CONFIG.value -> SeverityNumber.DEBUG4 // 700 → 8 (judgment call) + value >= Level.DEBUG.value -> SeverityNumber.DEBUG // 500 → 5 + value >= Level.FINER.value -> SeverityNumber.TRACE2 // 400 → 2 + else -> SeverityNumber.TRACE // 300 → 1 +} +``` + +`CONFIG` has no clean OTel equivalent (it sits between INFO and DEBUG in JUL semantics); `DEBUG4` +keeps it just above plain DEBUG. Mapping it to `INFO` is also defensible — pick one and document it. + +### 5.2 Body / message + +Use `SimpleMessageFormatter.getLiteralLogMessage(data)` to get the rendered message **without** the +appended `[CONTEXT …]` block. Metadata goes to attributes (§5.4), so using the full `format(...)` +here would double-encode it (once as text, once as structured attributes). This is a deliberate +divergence from the Log4j2 backend, which appends context into the text because Log4j2's +structured-data story is weaker. + +### 5.3 Timestamp & exception + +- `data.timestampNanos` → `emit(timestamp = …)`. **Confirm the unit** `opentelemetry-kotlin` + expects (epoch nanos vs millis) against the pinned version; OTLP is epoch nanos, but verify the + Kotlin API contract. +- Cause via `metadata.getSingleValue(LogContext.Key.LOG_CAUSE)` → `emit(exception = …)`. The Kotlin + SDK records it per the exception semantic conventions. + +### 5.4 Metadata + LogSite → attributes + +```kotlin +private fun AttributesMutator.applyAttributes(data: LogData, metadata: MetadataProcessor) { + // log-site → OTel code.* semconv + val s = data.logSite + setStringAttribute("code.namespace", s.className) + setStringAttribute("code.function", s.methodName) + s.fileName?.let { setStringAttribute("code.filepath", it) } + if (s.lineNumber >= 0) setLongAttribute("code.lineno", s.lineNumber.toLong()) + + // scope + log-site metadata → attributes (LOG_CAUSE already consumed as exception) + metadata.process(AttributeHandler, this) +} + +private object AttributeHandler : MetadataHandler() { + override fun handle(key: MetadataKey, value: T, ctx: AttributesMutator) { + if (key == LogContext.Key.LOG_CAUSE) return // handled as exception + val name = "spine.${key.label}" // namespace to avoid semconv clashes + when (value) { + is Boolean -> ctx.setBooleanAttribute(name, value) + is Int -> ctx.setLongAttribute(name, value.toLong()) + is Long -> ctx.setLongAttribute(name, value) + is Float -> ctx.setDoubleAttribute(name, value.toDouble()) + is Double -> ctx.setDoubleAttribute(name, value) + is String -> ctx.setStringAttribute(name, value) + else -> ctx.setStringAttribute(name, value.toString()) + } + } + // Override the repeated-key hook to accumulate into setStringListAttribute / setLongListAttribute, etc. +} +``` + +Notes: + +- **Namespace metadata keys** (`spine.