feat(lab): CL-09 passive production evidence - #1489
Merged
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughCL-09 adds passive production evidence for Compatibility Lab. Production attempts receive exact opaque route-subject linkage, bounded usage history is queried, and summaries are exposed through the management API, CLI, and Compatibility Matrix without changing Lab verdicts or routing. ChangesCompatibility Lab progression
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant ProductionRequest
participant RouteSubjectResolver
participant UsageLog
participant PassiveQuery
participant CompatibilityMatrix
ProductionRequest->>RouteSubjectResolver: resolveProductionRouteSubject(final route, inbound wire)
RouteSubjectResolver-->>ProductionRequest: opaque subject ID
ProductionRequest->>UsageLog: persist labRouteSubjectId
CompatibilityMatrix->>PassiveQuery: queryPassiveProductionSignals(subject ID, limit)
PassiveQuery->>UsageLog: read bounded recent usage entries
UsageLog-->>PassiveQuery: usage history
PassiveQuery-->>CompatibilityMatrix: passive production summary
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
the
documentation
Contributor
|
✅ Deterministic PR hygiene checks passed. |
Wibias
marked this pull request as ready for review
August 11, 2026 21:31
Wibias
marked this pull request as draft
August 11, 2026 21:32
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md`:
- Line 260: Add exactly one trailing newline to the end of the Markdown file
after the “CL-05: not started” entry so it satisfies the MD047 check.
In
`@devlog/_plan/260807_compatibility_lab/009_cl09_passive_production_evidence.md`:
- Around line 98-116: Clarify the CL-09 V1 invariants so “0 production request
mutation” prohibits outbound request and user-payload changes, while explicitly
permitting the metadata-only addition of labRouteSubjectId to the existing
attempt record. Apply the same clarification wherever the passive evidence
capture rules are defined, including the section around the existing
attempt-record update.
- Line 1: Rename the document title from “Passive Production Evidence / Shadow
Correlation” to “Passive Production Evidence / Exact-Route Correlation,” leaving
the rest of the document unchanged.
- Around line 174-188: Constrain the diagnostic fields in PassiveRouteSignalV1
by replacing unrestricted terminalStatus, closeReason, and errorCode values with
finite allowlisted or bounded normalized tokens. Update the usage normalizer to
validate and map unrecognized values to unknown or omit them, and apply the same
validation before every Lab projection and API, CLI, or UI serialization.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c50b6160-58ab-4db2-a4db-f270a8399169
📒 Files selected for processing (3)
devlog/_plan/260807_compatibility_lab/000_master_plan.mddevlog/_plan/260807_compatibility_lab/001_pr_stack_status.mddevlog/_plan/260807_compatibility_lab/009_cl09_passive_production_evidence.md
Wibias
force-pushed
the
feat/cl-09-passive-production-evidence
branch
from
August 12, 2026 01:01
263cc12 to
597b684
Compare
Contributor
✅ READY
UI screenshot waived by the Hygiene✅ Deterministic PR hygiene checks passed. |
Wibias
changed the title
github-actions
Bot
added
enhancement
Wibias
marked this pull request as ready for review
August 12, 2026 01:53
Wibias
marked this pull request as draft
August 12, 2026 01:53
Contributor
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@devlog/_plan/260807_compatibility_lab/001_pr_stack_status.md`:
- Around line 30-31: Choose one authoritative CL-09 lifecycle state, using
001_pr_stack_status.md as the source of truth, and update its lines 30-31 and
170-171 to match the actual PR scope and phase table. Reconcile
devlog/_plan/260807_compatibility_lab/009_cl09_passive_production_evidence.md
lines 13 and 449-460 with that authorization, removing or updating claims that
runtime implementation proceeds or is absent. Update
docs/superpowers/plans/2026-08-12-cl09-passive-production-evidence.md lines
66-70 so its implementation claim is retained only if authorized; otherwise make
it contract-only.
In
`@devlog/_plan/260807_compatibility_lab/009_cl09_passive_production_evidence.md`:
- Around line 288-290: Redact the internal requestRef and decisionRef fields
before exposing passive production signals through the management API and CLI
JSON output. Add or reuse a public DTO/serializer around
queryPassiveProductionSignals, while retaining both references for internal
correlation, and add tests covering both output paths.
In `@docs/superpowers/plans/2026-08-12-cl09-passive-production-evidence.md`:
- Line 7: Update the “Current base” entry in the plan document to retain SHA
e8db4e0365b12a314d1c08ec2cf81599efe5b2d3 and explicitly identify it as a later
dev descendant of the CL-08 merge 3b8f9487676fe258d76295e49e7db75aca26a4cb.
In `@gui/src/pages/CompatibilityMatrix.tsx`:
- Around line 194-203: Add locale entries for the production section title,
status text, and each metric label shown in the production block, then update
the JSX in the CompatibilityMatrix production section to render all of them
through t(). Preserve the existing detail values and conditional rendering.
In `@src/cli/lab.ts`:
- Around line 241-247: Update the production-signals flow in the CLI case to
pass the resolved deps.configDir into queryPassiveProductionSignals(), then
propagate that directory through queryPassiveProductionSignals() to
readRecentUsageEntries() so usage.jsonl is read from the selected configuration
directory. Add a regression test covering a non-default config directory.
In `@src/lab/query/passive-production.ts`:
- Around line 132-134: Update queryPassiveProductionSignals and its
readRecentUsageEntries call to fetch one row beyond
PASSIVE_PRODUCTION_MAX_SCAN_ROWS, then ensure derivation caps processing while
detecting that extra row and reports scan truncation accurately. Replace the
exact-length signals.length === maxResults heuristic with an overflow-aware
result, preserving the separate max-results truncation behavior only when an
additional matching signal is found.
- Around line 106-115: Add a short comment immediately before the scanRows
selection in readRecentUsageEntries documenting that the selected append-only
log entries are ordered oldest-first. Explicitly note that tail selection,
reverse iteration, and the signals[0].observedAt calculation rely on this
ordering contract.
In `@src/server/responses/core.ts`:
- Around line 1788-1794: Memoize installation-salt reads in the flow used by
readExistingInstallationSalt/readSaltFile with a module-wide cache keyed by the
resolved salt path, avoiding repeated synchronous file reads. Update or
invalidate the corresponding cache entry whenever readInstallationSalt creates
or reads a salt, while preserving existing salt values and lookup behavior.
In `@tests/lab-passive-production-evidence.test.ts`:
- Around line 108-117: Add focused tests near the existing passive-production
tests to cover all remaining classifyOutcome branches: a final attempt with
closeReason "client_cancel" should produce "client_cancel", an error code from
ENVIRONMENTAL_ERROR_CODES should produce "environmental", and an error code from
ROUTE_ERROR_CODES should produce "route_error" while incrementing
recentRouteErrorSignals. Reuse the existing usageEntryWithAttempt setup and
subject identifier patterns.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c874aa9a-29ed-42be-afda-ea991dce817d
📒 Files selected for processing (13)
devlog/_plan/260807_compatibility_lab/001_pr_stack_status.mddevlog/_plan/260807_compatibility_lab/009_cl09_passive_production_evidence.mddocs/superpowers/plans/2026-08-12-cl09-passive-production-evidence.mdgui/src/pages/CompatibilityMatrix.tsxgui/src/pages/compatibility-matrix-api.tssrc/cli/lab.tssrc/lab/query/index.tssrc/lab/query/passive-production.tssrc/routing/compatibility/subject.tssrc/server/management/lab-routes.tssrc/server/responses/core.tssrc/usage/log.tstests/lab-passive-production-evidence.test.ts
Wibias
marked this pull request as ready for review
August 12, 2026 02:10
Wibias
added
the
gui-screenshot-waived
Wibias
marked this pull request as ready for review
August 12, 2026 02:12
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Collaborator
Author
|
Review triage on head Fixed and validated: CL-09 lifecycle-state consistency, non-default Skipped as not applicable: the Stale: the three findings against |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements CL-09 V1 passive production evidence on top of merged CL-08.
RouteSubjectV1identity on each actual production attemptnot_verificationBase
Implementation was rebased onto then-current
devate8db4e0365b12a314d1c08ec2cf81599efe5b2d3before runtime work began.Architecture
Exact attempt linkage
PersistedUsageAttempthas an optional installation-locallabRouteSubjectId. Production requests use the existing Lab route-subject construction authority for the actual inbound wire (responses,chat, oranthropic). Subject construction is best-effort: failure omits the link and never changes, retries, or delays the request.Fallback/combo recursion executes the same attempt-linkage seam for the actual child route, so different execution attempts can carry different exact subjects.
Passive query layer
src/lab/query/passive-production.tsscans at most 2,000 normalized recent usage rows and returns at most 200 signals. The reader fetches one extra row only to detect scan truncation accurately. It creates no second ledger, SQLite authority, artifact store, provider call, or Lab observation.Lab-facing signals expose only an explicit metadata allowlist. Raw
upstreamError, terminal-status strings, error-code strings, prompts, responses, tool payloads, credentials, account metadata, headers, URLs, and reasoning are not serialized. Exact known error codes are used internally only for conservative diagnostic outcome classification.The CLI propagates its selected
configDirthrough the passive query into the bounded usage reader, so alternate OpenCodex homes cannot be mixed with the default usage history.Read surfaces
GET /api/lab/production-signalsocx lab production-signals --subject <id> [--limit <n>] [--json]Not Lab verificationlabelingNo new product area or combined compatibility score is introduced.
Safety / isolation
CL-09 V1 guarantees:
Tests
Focused CL-09 regressions cover:
configDirusage-history isolationsuccess,client_cancel,environmental,route_error, and conservative generic HTTP-failure classificationReview status
The PR remains open. Valid CodeRabbit findings have been remediated or explicitly dispositioned against the frozen CL-09 contract. The exact final head must pass Cross-platform CI and React Doctor before independent final review. Do not merge based only on implementation completion.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation