Telemetry: service skeleton + slice 1 (TelemetryPolicyConfigured config-as-events) - #42
Merged
Merged
Conversation
…ig-as-events)
Stands up CritterCab.Telemetry as the repo's second service and realizes W006
slice 1 in one PR (skeleton + first-slice cadence exception; mirrors the Dispatch
skeleton + slice-5.1 precedent). Opening PR of the W006 transport chain — config-
as-events is the dependency-correct first slice (slice 2's gRPC ingest reads this
slice's TelemetryPolicy view), and it lands no transport.
- TelemetryPolicyConfigured singleton event stream (full-replacement, ADR-011)
- ConfigureTelemetryPolicy command + nested AbstractValidator (boundary validation)
- TelemetryPolicy self-aggregating live-stream view; throttlePolicyVersion as long
(the Marten stream version, matching the proto int64)
- IInitialData migration-time idempotent bootstrap seed (defaults 9/30/5,
operatorId=system-bootstrap) per ADR-011 Option A
- 3 Alba GWTs (bootstrap / reconfigure / reject) + DB-less smoke test
- apphost: Telemetry + crittercab_telemetry DB (5315/5316); Kafka deliberately
deferred to slice 3
First config-as-events instance in code (ADR-011's third instance; Dispatch and
Onboarding were design-only) and first FluentValidation use. Four jasperfx-source-
verifier gates ran before wiring. No ADR fired (W006 §11 candidates are later-arc).
Prompt + retro under docs/{prompts,retrospectives}/implementations/006-*. DEBT row
registered for the config-as-events bootstrap-seed skill (ADR-011's deferred follow-up).
…liation The critter-skill-auditor Phase 2 pass caught a design-vs-implementation gap: W006 §6.1 locks the seed as "ADR-011 Option A (migration-time seed)", but the Marten IInitialData realization runs at host startup — structurally Option B, the option ADR-011 rejected over a multi-instance seed race. IInitialData does serve Option A's intent under the JasperFx `resources setup` deploy step, but also self- seeds at host start; the race is benign here (idempotent guard + full-replacement) and irrelevant at single-instance MVP scale. Corrects artifacts to stop bare-claiming "Option A" and registers the reconciliation (plus the write-path optimistic-concurrency question — Append with no expected version, because [WriteAggregate] has no binding to a constant singleton id) as questions the future canonical-pattern skill must resolve. The ADR-011 amendment itself is a user-owned decision, not made here. - TelemetryPolicyBootstrap.cs / Program.cs: accurate seed comments + reconciliation note - W006 Document History: open-reconciliation note - DEBT.md: two design questions the reference impl must resolve before enshrining - retro: "design meets code" section capturing the finding
…est) CI caught it: the reject GWT got 200 instead of 400 — the boundary validator never ran. UseFluentValidationProblemDetailMiddleware resolves IValidator<T> from DI, but nothing registered the validators. Add opts.UseFluentValidation() (WolverineFx.FluentValidation) which discovers and registers all IValidator<> in the app assembly — the idiomatic pairing the Wolverine HTTP sample uses, and it auto-discovers future validators rather than needing a per-validator registration. Local build green; the reject test's 400 is verified on CI (local Docker wedged).
…wiring DEBT Records the confirmed CI outcome (run 29119259592: Telemetry 4/4 + Dispatch 11/11) and the CI-caught validator-registration bug as a methodology lesson (a Docker-wedged session's DB-less smoke test gives false confidence about paths it never touches). Registers the Wolverine.HTTP FluentValidation two-call wiring as a wolverine-http-handlers skill gap.
… LWW Resolves the Option-A/B-for-Marten gap the slice-1 Phase-2 audit surfaced (user- directed, in-PR). Amends ADR-011 with a 2026-07-10 section recording that IInitialData (via .InitializeWith<T>()) is the canonical Marten realization of Option A — it seeds at the deploy-time apply step and idempotently at host start as a self-healing safety net; the multi-instance race is mitigated by the idempotent guard + full-replacement and avoided by the deploy step / single-instance MVP. Also records last-writer-wins as the accepted config-singleton write semantic (full-replacement has no invariant to defend; optimistic concurrency not required; manual constant-id append since [WriteAggregate] has no id field to bind). Cascades the "resolved" status through DEBT (skill now unblocked), W006 history, the retro (design-meets-code + spec-delta + outstanding items, documenting the mid-flight user-directed expansion), both README indexes, and the two seed code comments. Prompt left as authored (historical record), not retroactively re-scoped.
erikshafer
added a commit
that referenced
this pull request
Jul 10, 2026
…nsport build Introduces three handoff notes tracking Telemetry chain progression from PR #39 (protos) through PR #42 (skeleton + slice 1). Post-PR-42 handoff orients next session on PR B: slices 4+2 paired (LastKnownPosition store + ReportLocations gRPC ingest), CritterCab's first gRPC surface in code and first Kafka testcontainer. Documents verified WolverineFx.Grpc client-streaming gap (must hand-wire against IMessageBus through 5.37.2; 6.17 unconfirmed), local Docker wedged state (lean on CI), and ADR-011 amendment locked (IInitialData bootstrap + last-writer-wins singletons). Supersedes three earlier untracked handoffs after PR #40 (narrative decision—does not apply, recorded) and PR #41 (package refresh to 6.17.0) both merged. Carries forward design-locked guardrails, three W006 §11 ADR candidates, and open skill-debt items. Includes narrative-decision and post-PR-41 intermediate handoffs for session lineage; all disposable once next session orients.
erikshafer
added a commit
that referenced
this pull request
Jul 24, 2026
…unk C) W006 §6.2. CritterCab's first gRPC surface that serves traffic, and the client-streaming shape that was blocked for three months until WolverineFx.Grpc 6.21.0 shipped auto-codegen for it. Nothing here is hand-wired. TelemetryGrpcService is an empty abstract [WolverineGrpcService] deriving the generated TelemetryServiceBase. Two codegen layers stack: protoc emits the virtual ReportLocations over a raw IAsyncStreamReader, then Wolverine emits the override that adapts it to IAsyncEnumerable and forwards to IMessageBus.StreamAsync. ReportLocationsHandler therefore never sees a gRPC type — the whole stream is simply the message. The per-ping pipeline lives inside the handler, which is forced rather than stylistic: Wolverine cannot weave Before/Validate frames for client-streaming because a before-frame needs a concrete request at method entry and a stream cannot supply one. So unlike slice 1, which validates at the HTTP boundary with FluentValidation, validation and the publish trigger are handler concerns here. Trigger, per §6.2: shouldPublish = heartbeatDue OR (cellChanged AND throttleFloorElapsed), evaluated against slice 4's document. Policy and baseline are read once at window open and the baseline kept current in memory — safe because within a window the driver's own pings are the sole writer of their own document. An absent baseline publishes immediately, which is what closes §6.4's Return case. On publish it goes publish-first, then store: that ordering fails benignly (a duplicate the consumer's dedup absorbs) where the reverse fails as a missed cell change. Invalid pings are silently dropped, never errored. Seams, both ready-to-swap rather than final: - IDriverPrincipalAccessor resolves driverId from the principal, never the payload (R5) — which is why the proto has no driver_id field. The dev stub reads x-driver-id; gRPC metadata travels as HTTP/2 headers so it arrives there. Documented swap site for a real Entra claim. - IDriverLocationPublisher carries the generated DriverLocationUpdated — the real published language, so PR C swaps only the implementation, never the contract. driver_location_updated.proto joins codegen for this (GrpcServices="None", message-only). No Kafka is wired. H3CellIndexer wraps one line because that line has a footgun on both axes: H3's LatLng is (lat, lon) in radians while NTS Coordinate is (lon, lat) in degrees. The degrees-native Coordinate path removes the unit conversion entirely. The pinning test computes the same point down both paths — opposite on order AND units, so they can only agree if production has both right — and separately proves a swapped Coordinate yields a different cell, since "is a valid cell" passes happily for the wrong place on Earth. apphost.cs did not compile, and had not since PR #42 added the Telemetry service block without its #:project directive; CI never builds the apphost so nothing caught it. Fixed here because deliverable 3 cannot land on a file that does not build. gRPC rides the existing HTTPS endpoint over HTTP/2, matching the convention the Dispatch block already documents, so it needs no port of its own. Tests: 31/31 green. Five drive a real gRPC client stream against the Alba host — gate 11 resolves cleanly, since Alba wraps WebApplicationFactory over TestServer, so GetTestServer().CreateHandler() reaches the same in-memory transport and CritterCab keeps its Alba-first default with no parallel host recipe. Covers Happy publish, Throttled, Heartbeat, Window close and unauthenticated, plus the §6.4 No-write GWT that slice 4 could not reach alone because deciding not to write is this slice's trigger.
erikshafer
added a commit
that referenced
this pull request
Jul 24, 2026
…portLocations ingest (#45) * docs: prompt 007 — telemetry slices 4+2 (LastKnownPosition + gRPC ingest) Session prompt for PR B of the W006 Telemetry chain: slice 4 (LastKnownPosition store + eviction) and slice 2 (gRPC ReportLocations client-streaming ingest). Includes the 2026-07-21 Verify-before-wiring source-verification pass and the 2026-07-24 consistency alignment: - fork 3 and gate 8a now name the NTS Coordinate-degrees H3 recipe; the H3Index.FromLatLng tuple path is marked as the radians trap it is - gate 8b closed: package id is pocketken.H3 4.5.0.1, ships a native lib/net10.0, pulls NetTopologySuite 2.6.0 transitively - deliverables 3 and 6 carry gate 5's BackgroundService + EvictStalePositions handler split (Wolverine has no first-class recurring primitive) and gate 6's HardDeleteWhere refinement - deliverable 1 pins the bump to 6.21.0 (the source-verified V6.21.0-12 baseline, not the newer 6.22.0) and makes it lockstep across all 11 WolverineFx entries The previous subject on this commit claimed to implement slices 4 and 2; it only ever contained this prompt document. * Telemetry: WolverineFx 6.21.0 lockstep + proto-first gRPC codegen (chunk A) First codegen-behind-a-proto in the repo. Chunk A of prompt 007 — the version bump and codegen wiring that slices 4 and 2 build on. Directory.Packages.props - all 11 WolverineFx* entries 6.19.0 -> 6.21.0 in lockstep. Only WolverineFx and WolverineFx.Grpc strictly need it (6.21.0 is the release that added client-streaming auto-codegen), but bumping the pair alone would leave the other nine as a version island. - 6.21.0 rather than the newer 6.22.0: 6.21.0 is what prompt 007's Verify-before-wiring citations were source-verified against (local checkout V6.21.0-12), so the gate line numbers keep matching the binaries we build on. 6.22.0 is a later tidy: packages bump. - new Grpc.AspNetCore / Grpc.Tools / Google.Protobuf entries, pinned to exactly what WolverineFx.Grpc 6.21.0 declares (2.76.0 / 2.76.0 / 3.31.1) so the codegen toolchain matches Wolverine's own tested pairing. CritterCab.Telemetry.csproj - adds WolverineFx.Grpc plus the three gRPC packages. - adds the <Protobuf> item compiling report_locations.proto in from the repo-root protos/ tree. The proto stays there because the contract is the artifact (ADR-009) and is shared across services; ProtoRoot points at that tree so the proto's package path resolves. Verified: dotnet build emits TelemetryService.TelemetryServiceBase with Task<LocationIngestAck> ReportLocations( IAsyncStreamReader<LocationPing>, ServerCallContext) and MethodType.ClientStreaming — the shape gate 1 predicted. Full solution builds with 0 warnings; 15/15 tests pass locally against the new dependency line (Marten 9.16.1, JasperFx 2.30.1), resolving gate 7. * Telemetry: slice 4 — LastKnownPosition store + heartbeat-absence eviction (chunk B) W006 §6.4 and §3.3. CritterCab's first non-event-sourced document write path and first recurring/scheduled work in the repo. LastKnownPosition — a plain Marten document, overwrite-in-place, defending no invariant: LWW on ServerReceivedAt is the whole concurrency story. Ships the minimal single-timestamp shape: ServerReceivedAt serves BOTH §6.2's lastPublishedAt trigger baseline AND §6.4's eviction key, because §6.4 locks upsert-on-publish-only, which makes them the same instant by construction. A separate LastPublishedAt would have held a duplicate value and forced eviction and the trigger to read different names for one moment. §3.3's DeviceTimestamp / Speed / Heading are omitted deliberately — slice 3 passes speed and heading to Kafka straight from the ping, so the document never reads them. Field is Lon, not §3.3's Lng: the proto is the contract (ADR-009). Eviction is split, per the prompt's gate 5 — Wolverine has no first-class recurring-message primitive, so the timer is a plain BackgroundService and only the work is Wolverine's: - LastKnownPositionEvictionService holds no logic and stays untested: it loops on Task.Delay, opens a scope, and calls InvokeAsync (not PublishAsync, so a slow sweep back-pressures its own timer rather than overlapping the next tick). It takes IServiceScopeFactory rather than IMessageBus because Wolverine registers IMessageBus scoped (HostBuilderExtensions.cs:232) and a BackgroundService is a singleton — injecting the bus directly fails DI scope validation at host build. - EvictStalePositionsHandler holds everything testable: reads HeartbeatIntervalSeconds from the TelemetryPolicy view, derives the 3-missed-heartbeats threshold (a documented constant, not a v1 policy param), and HardDeleteWhere's the stale rows. HardDeleteWhere rather than DeleteWhere so "the row is gone" survives a future soft-delete config — DeleteWhere would silently degrade to a flag and break the Return GWT. No staleness event is published (v1, R3/R8). Registered inside Program.cs's Marten guard: without a store there is nothing to sweep and the timer would only log failures. Tests (6 new, 21/21 green): Upsert, Evict, threshold-respects-fresh-documents, Return, and a proof the threshold is read from policy rather than hardcoded (the same 10-second-old document survives under the seeded 30s heartbeat and is swept under a reconfigured 1s one). The §6.4 "No-write" GWT is not here — deciding not to write is slice 2's trigger, which lands in chunk C. Also covers the production host wiring, which nothing else reaches: the fixture host strips the timer for determinism and the smoke test runs without a connection string, so the registration would otherwise be untested. Verified by mutation that the guard actually fails when the bus is injected directly. * Telemetry: slice 2 — gRPC ReportLocations client-streaming ingest (chunk C) W006 §6.2. CritterCab's first gRPC surface that serves traffic, and the client-streaming shape that was blocked for three months until WolverineFx.Grpc 6.21.0 shipped auto-codegen for it. Nothing here is hand-wired. TelemetryGrpcService is an empty abstract [WolverineGrpcService] deriving the generated TelemetryServiceBase. Two codegen layers stack: protoc emits the virtual ReportLocations over a raw IAsyncStreamReader, then Wolverine emits the override that adapts it to IAsyncEnumerable and forwards to IMessageBus.StreamAsync. ReportLocationsHandler therefore never sees a gRPC type — the whole stream is simply the message. The per-ping pipeline lives inside the handler, which is forced rather than stylistic: Wolverine cannot weave Before/Validate frames for client-streaming because a before-frame needs a concrete request at method entry and a stream cannot supply one. So unlike slice 1, which validates at the HTTP boundary with FluentValidation, validation and the publish trigger are handler concerns here. Trigger, per §6.2: shouldPublish = heartbeatDue OR (cellChanged AND throttleFloorElapsed), evaluated against slice 4's document. Policy and baseline are read once at window open and the baseline kept current in memory — safe because within a window the driver's own pings are the sole writer of their own document. An absent baseline publishes immediately, which is what closes §6.4's Return case. On publish it goes publish-first, then store: that ordering fails benignly (a duplicate the consumer's dedup absorbs) where the reverse fails as a missed cell change. Invalid pings are silently dropped, never errored. Seams, both ready-to-swap rather than final: - IDriverPrincipalAccessor resolves driverId from the principal, never the payload (R5) — which is why the proto has no driver_id field. The dev stub reads x-driver-id; gRPC metadata travels as HTTP/2 headers so it arrives there. Documented swap site for a real Entra claim. - IDriverLocationPublisher carries the generated DriverLocationUpdated — the real published language, so PR C swaps only the implementation, never the contract. driver_location_updated.proto joins codegen for this (GrpcServices="None", message-only). No Kafka is wired. H3CellIndexer wraps one line because that line has a footgun on both axes: H3's LatLng is (lat, lon) in radians while NTS Coordinate is (lon, lat) in degrees. The degrees-native Coordinate path removes the unit conversion entirely. The pinning test computes the same point down both paths — opposite on order AND units, so they can only agree if production has both right — and separately proves a swapped Coordinate yields a different cell, since "is a valid cell" passes happily for the wrong place on Earth. apphost.cs did not compile, and had not since PR #42 added the Telemetry service block without its #:project directive; CI never builds the apphost so nothing caught it. Fixed here because deliverable 3 cannot land on a file that does not build. gRPC rides the existing HTTPS endpoint over HTTP/2, matching the convention the Dispatch block already documents, so it needs no port of its own. Tests: 31/31 green. Five drive a real gRPC client stream against the Alba host — gate 11 resolves cleanly, since Alba wraps WebApplicationFactory over TestServer, so GetTestServer().CreateHandler() reaches the same in-memory transport and CritterCab keeps its Alba-first default with no parallel host recipe. Covers Happy publish, Throttled, Heartbeat, Window close and unauthenticated, plus the §6.4 No-write GWT that slice 4 could not reach alone because deciding not to write is this slice's trigger. * Telemetry: docs + ledger for slices 4 and 2 (chunk D) Closes the session's documentation deliverables and retires the client-streaming forward-constraint everywhere it was recorded. report_locations.proto — the stale comment directing a hand-wire against IMessageBus is corrected. This one mattered beyond tidiness: the proto's leading comments become doc-comments in the generated ReportLocationsGrpc.cs, so the obsolete instruction was being compiled into the build output. The replacement also records the middleware caveat at contract level, since it is a property of the RPC shape rather than of our implementation. wolverine-grpc-handlers — corrected under the session-runner-blocking exception: a session cannot follow a skill that tells it to hand-wire a shape the library now generates. Seven stale claims fixed (frontmatter, intro, why-not-code-first, the abstract-stub note, two pitfalls, see-also), and a new Client-streaming handlers section added mirroring the server-streaming one. It documents the empty stub, the two stacked codegen layers, the IAsyncEnumerable handler shape, and three things the session had to discover from source rather than docs: middleware does NOT weave and fails silently rather than loudly; identity must come from an IHttpContextAccessor seam because there is no [WolverineBefore] to read ServerCallContext from; and the handler slot is keyed on the request type, so two client-streaming RPCs sharing a request type collide. wolverine-grpc-bidirectional-handlers — banner plus mental-model corrections only, as scoped. The ~145-line hand-written-workaround body is left intact under an explicit obsolete banner rather than half-rewritten; its removal is a DEBT row. The mental-model table and the prose around it were corrected too, because leaving "Rejected at startup" in the most-read section would have kept the file contradicting shipped code regardless of the banner. DEBT.md — six rows: the bidirectional rewrite; test-class naming across all three Telemetry suites (a decision, not a cleanup, and explicitly not to be fixed slice-locally); no skill for plain-document write paths; no skill for recurring work; identity-acl's gRPC auth pattern not applying to streaming shapes; and the feature-folder/type-name collision now on its third occurrence. W006 Document History — slices 2 and 4 designed -> realized, the forward constraint closed, and the §11 windowed-client-streaming candidate recorded as firing into a skill rather than an ADR. It also records that MaxAccuracyMeters = 100 was invented at implementation time, since §6.2 names the threshold but fixes no value. Retro — records what the session learned rather than what it did: that the verification pass earned its cost by disconfirming its own prompt twice, that both load-bearing tests were built to fail a naive implementation, that the DI guard was mutation-verified, and that the apphost break survived two weeks because CI's existing completeness guard cannot see a file-based app. 31/31 still green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opening PR of the W006 Telemetry transport chain. Stands up
CritterCab.Telemetryas the repo's second service and realizes W006 slice 1 (TelemetryPolicyConfiguredconfig-as-events) in one PR, per the named skeleton-plus-first-slice cadence exception (mirrors the Dispatch skeleton + slice-5.1 precedent).Config-as-events is the dependency-correct first slice: slice 2's gRPC ingest reads this slice's
TelemetryPolicyview (throttlePolicyVersion,h3Resolution, intervals). So this opening PR of the "first real transport build" deliberately lands no transport — that's the correct dependency order, not an omission.What landed
Skeleton — new
Microsoft.NET.Sdk.Webservice mirroring Dispatch (owncrittercab_telemetryDB, ports 5315/5316), paired Alba test project, apphost + slnx wiring. Kafka deliberately not wired intoapphost.cs— transport lands with the slice that needs it (slice 3), the same deferral the Dispatch skeleton made.Slice 1 (
TelemetryPolicy/feature folder)TelemetryPolicyConfiguredsingleton event stream (full-replacement, ADR-011)ConfigureTelemetryPolicycommand + nestedAbstractValidator(boundary validation: intervals positive,heartbeat ≥ minPublish,h3Resolutionin 0–15)TelemetryPolicyself-aggregating live-stream view exposingthrottlePolicyVersionas along(the Marten stream version, matching the protoint64)IInitialDatamigration-time idempotent bootstrap seed (defaults9/30/5,operatorId=system-bootstrap) per ADR-011 Option ATwo firsts in code
DEBT.mdrow (not authored here).WolverineFx.Http.FluentValidation.Verification
jasperfx-source-verifiergates ran before wiring:IInitialDataseed vehicle,longstream-version property, self-aggregatingLiveStreamAggregationregistration + Marten-9partialscope, Wolverine.HTTP FluentValidation middleware. All confirmed at build.dotnet buildgreen; smoke test green locally. Local Docker's container-start path was wedged all session (the handoff-flagged condition), so the 3 slice-1 Testcontainers tests are gated on CI here rather than run locally.No ADR fires — W006 §11's three candidates are all later-arc (Kafka topic-naming → slice 3; stream-processing-4th-shape → slices 2–4; windowed client-streaming → slice 2). No narrative anchor (PR #40: the narrative layer does not apply to Telemetry).
Prompt + retro:
docs/{prompts,retrospectives}/implementations/006-telemetry-skeleton-and-slice-1-config.md.Follow-on PRs
LastKnownPosition+ eviction) + slice 2 (gRPC ingest, hand-wired againstIMessageBus)DriverLocationUpdated→ Kafka; wires Kafka into apphost)NearbyAvailableDriversStub)