feat(dev): ship runner module logs to dispatch (log pipeline)#36
Merged
Conversation
Slice 3 of the module log pipeline — the dev runner's write path. The
in-container `dev --all` runner already line-reads each module's stdout/stderr
(spawn_forwarder); now it also parses each line and ships it to dispatch's
ingest, which buffers it into the per-module ring the developer console reads.
- log_shipper: parse_line turns a line into {ts,level,msg,app_id} — extracting
the SDK's structured JSON (slog) fields, falling back to the stream
(stderr→warn, stdout→info) for non-JSON lines. A per-module background thread
batches entries (cap 200 / ~1s) and POSTs them to
{MS_DISPATCH_URL}/internal/modules/{id}/logs with X-MS-Service-Secret ==
MS_INTERNAL_SECRET (the host-minted per-session secret the ingest binds to).
Best-effort: a failed or unconfigured ship never blocks the module.
- app_id rides each entry so the app-side console can filter a shared dev
module's logs per app (dispatch ignores the field until the Go Entry.AppID
follow-up).
Verified: cargo check + cargo test (5 log_shipper tests) green; cargo fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Slice 3 of the module log pipeline — the dev runner's write path. The in-container
dev --allrunner already line-reads each module's stdout/stderr (spawn_forwarder); now it also ships each line to dispatch's ingest (#275), which buffers it into the per-module ring the developer console (#94) reads.log_shipper::parse_lineturns a line into{ts, level, msg, app_id}— extracting the SDK's structured JSON (slog)level/msg/time/app_id, falling back to the stream (stderr→warn,stdout→info) with the raw line for non-JSON output.{MS_DISPATCH_URL}/internal/modules/{id}/logswithX-MS-Service-Secret == MS_INTERNAL_SECRET— the host-minted per-session secret the ingest binds to (confirmed: the host injects it into the compose env, so the runner's value == the session's registeredInternalSecret).MS_DISPATCH_URL/secret unset) ship never blocks the module — logs just stay in the terminal as before.app_idrides each entry so the app-side console (thelogs:readwork) can filter a shared dev module's logs per app. Dispatch ignores the field until the GoEntry.AppIDfollow-up.Verification
cargo checkclean;cargo test log_shipper— 5 tests green (JSON parse incl.app_id, message-alias + error level, non-JSON stream fallback, level normalization, shipping-disabled-without-config);cargo fmtclean.🤖 Generated with Claude Code