feat(orb): content-addressed decision records on every verdict - #8842
Merged
Conversation
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
A contributor closed by ORB cannot see WHICH ruleset version or clause closed them -- and the authoritative config being private makes that worse, not better. Every terminal disposition now persists a canonical, content-addressed record and the public review surface renders it: 'the bot closed me' becomes 'clause X of config abc123 closed me' -- inspectable, arguable, stable under challenge. Shape follows SLSA's Verification Summary Attestation. Closes #8836. Part of epic #8828 (Phase 4 -- trust surface). Stacked on the #8830 branch (migration contiguity with 0178). - decision-record.ts: canonicalJson (recursively key-sorted -- key order is construction artifact, never meaning; refuses un-JSON-able values loudly), sha256Hex via Web Crypto, buildDecisionRecord (normalizes optional-shaped fields once, at the builder), persistDecisionRecord (latest-finalize-wins per target@sha, best-effort -- legibility never breaks finalization), renderDecisionRecordSection (bounded), loadDecisionRecordCollapsible (fail-safe null) - migration 0179: decision_records - processors: persist at the SAME finalize site as recordNativeGateDecision with the SAME reasonCode expression, so the record and the calibration row can never disagree about WHY; the unified comment appends the stored record as a collapsible on republish (first publish precedes first finalize by design) - model/prompt commitments are null for rule-only decisions; wiring the AI-review contribution is the tracked follow-up on #8836
JSONbored
force-pushed
the
feat/decision-record
branch
from
July 26, 2026 11:54
dd85f5a to
f237a05
Compare
❌ 2 Tests Failed:
View the top 2 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
JSONbored
added a commit
that referenced
this pull request
Jul 26, 2026
… in the decision record (#8847) When an AI-judgment finding (ai_consensus_defect / ai_review_split) shaped a decision, the decision record now carries the finding's calibrated confidence and a digest of the prompt template commitment (REVIEW_PROMPT_VERSION + REVIEW_SYSTEM_PROMPT) -- the fields that join every decision to the risk-control calibration set (#8835) and complete #8842's tracked follow-up. Advances #8834 (the persistence half its issue text names as the core deliverable; the flag-gated rotated-exemplar N-run variant remains -- the existing dual-model consensus already IS a two-sample agreement signal, so the extra runs are a cost knob, not the prerequisite). - DecisionRecord schema v2: + aiConfidence (null when no AI judgment contributed); normalized once at the builder like its siblings - processors: source the confidence from the gate's own AI-judgment blocker; modelId deliberately stays null at this site -- the finding does not carry which concrete models ran, and a guess would be worse than nothing (reviewDiagnostics holds per-run identities) - render: the model line gains the confidence when present
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every terminal verdict now persists a content-addressed decision record, and the public review surface renders it. Closes #8836 (epic #8828, Phase 4 — the highest impact/cost item of the trust-surface research). Stacked on #8840 (migration contiguity: 0179 follows its 0178).
Why
A contributor closed by ORB today cannot see which ruleset version or clause closed them — and the authoritative config being private (the operator's config dir overrides the public yml) makes that worse. The remedy from the epic's research: publish the digest even where contents stay private. "The bot closed me" becomes "clause
guardrail_holdof configabc123…decided this" — inspectable, arguable, and stable under challenge. The shape follows SLSA's Verification Summary Attestation, which exists for exactly this delegated-decision pattern.This record is also the input schema the golden-corpus replay (#8832) and the deterministic replay harness (#8838) consume — one schema, three consumers, so "what we published" and "what we can replay" can never drift apart.
How
canonicalJson— recursively key-sorted, whitespace-free, the ONE serialization every digest is computed over. Key order is a construction artifact, never meaning; un-JSON-able values throw loudly (a silent wrong digest is worse). Property-tested: key-order invariance at depth, array-order preservation, unicode stability.DecisionRecord— action (the acted disposition, never the raw conclusion — gate_decision can contradict the action taken — stale 'merge' verdicts recorded after CI-failure closes distort calibration #8825's lesson), the clause (reasonCode, same expression asrecordNativeGateDecisionat the same site, so record and calibration row can never disagree about why), resolved-config digest, gate pack, and model/prompt commitments (null for rule-only decisions; AI-review wiring is the tracked follow-up).(target, head sha), best-effort — legibility must never break finalization. The stored JSON is the canonical form: re-digesting the stored bytes reproduces the stored digest (tested — that is the replay check).Verification