From b6d8e596e71ae7e42b153a3487523eeca271a607 Mon Sep 17 00:00:00 2001
From: JSONbored <49853598+JSONbored@users.noreply.github.com>
Date: Mon, 27 Jul 2026 03:45:40 -0700
Subject: [PATCH] =?UTF-8?q?docs:=20publish=20the=20tenant=20verification?=
=?UTF-8?q?=20contract=20=E2=80=94=20claim,=20artifact,=20trust=20assumpti?=
=?UTF-8?q?on=20(#9186)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Adds "What you can verify" as a sibling to verify-this-review: for every claim
LoopOver makes, the artifact that proves it, the endpoint or CLI that checks it,
who can run it, and the trust assumption behind it.
States the boundaries as plainly as the guarantees: the ledger is
tamper-evident rather than tamper-proof until external anchoring ships, live
gate execution is not attested by standing design, ground-truth honesty is
closed by no artifact including attestation, digests commit to inputs and never
to non-deterministic model outputs, and the hosted-tenant column is weaker than
the self-host one.
---
.../content/docs/verify-this-review.mdx | 3 +
.../content/docs/what-you-can-verify.mdx | 130 ++++++++++++++++++
.../src/components/site/docs-nav.tsx | 1 +
3 files changed, 134 insertions(+)
create mode 100644 apps/loopover-ui/content/docs/what-you-can-verify.mdx
diff --git a/apps/loopover-ui/content/docs/verify-this-review.mdx b/apps/loopover-ui/content/docs/verify-this-review.mdx
index c9eaa32ff0..dcef38683d 100644
--- a/apps/loopover-ui/content/docs/verify-this-review.mdx
+++ b/apps/loopover-ui/content/docs/verify-this-review.mdx
@@ -14,6 +14,9 @@ its checksum, replay the same scorer over it, and compare what you get against w
Everything below runs read-only against a database export and pure functions from
`@loopover/engine`. Nothing posts anywhere, nothing needs an API key.
+For the wider contract — every claim, its artifact, and its trust assumption, including the ones you
+cannot check — see [what you can verify](/docs/what-you-can-verify).
+
## 1. Export the corpus snapshot
Every rule's fired/override history exports as a versioned, checksummed JSON snapshot
diff --git a/apps/loopover-ui/content/docs/what-you-can-verify.mdx b/apps/loopover-ui/content/docs/what-you-can-verify.mdx
new file mode 100644
index 0000000000..63785eb4ce
--- /dev/null
+++ b/apps/loopover-ui/content/docs/what-you-can-verify.mdx
@@ -0,0 +1,130 @@
+---
+title: What you can verify
+description: Every claim LoopOver makes, the artifact that proves it, the command that checks it, and the trust assumption behind it — including the things you cannot verify.
+eyebrow: Core concepts
+---
+
+## Why this page exists
+
+[Verify this review](/docs/verify-this-review) walks through *one* check end to end: re-run the
+published backtest corpus and compare the numbers. This page is the wider contract — for **every**
+claim LoopOver makes, what artifact proves it, who can check it, and what you still have to take on
+trust.
+
+The last column is the important one. A verification story that only lists what works is marketing;
+the assumptions and the gaps are what let you decide whether the guarantee is worth anything to you.
+
+## The matrix
+
+### 1. Gate-decision integrity
+
+**Claim:** the sequence of decisions was not silently reordered, deleted, or rewritten.
+
+Every persisted verdict appends to a hash-chained ledger — each row's hash covers the previous row's
+hash, so any edit to history breaks the chain at a point you can locate.
+
+```bash
+curl -s "https://api.loopover.ai/v1/public/decision-ledger/verify" | jq
+```
+
+Returns `{ ok, checked, nextAfterSeq, tipSeq, tipHash, totalCount }`, and a `break` object with a
+`409` status if the chain is inconsistent. No API key — **anyone** can run it.
+
+
+ **Trust assumption: tamper-evident, not tamper-proof.** This detects sequence gaps, predecessor
+ and row-hash mismatches, truncated tails, and records whose content no longer matches their
+ chained digest. It does **not** detect an operator deleting the chain wholesale and re-chaining
+ from genesis — catching that requires an external anchor the operator does not control, which is
+ tracked but **not built today**.
+
+
+### 2. Decision-record authenticity
+
+**Claim:** this specific verdict is the document its digest names.
+
+Every review comment prints its decision record's full 64-character digest, and the record itself is
+public:
+
+```bash
+curl -s "https://api.loopover.ai/v1/public/decision-records/OWNER/REPO/123" | jq
+```
+
+Returns `{ record, recordDigest }`. The digest is a SHA-256 over the record's canonical JSON — keys
+sorted, no whitespace — so you can recompute it yourself and confirm nothing was edited after the
+fact. **Anyone** can run it.
+
+Corrections are visible history, not silent replacement: a superseded verdict for the same commit is
+stored as a new revision, never an overwrite.
+
+### 3. Published accuracy numbers
+
+**Claim:** the precision figures on the [fairness report](/fairness) are real computations over real,
+replayable history — not hand-entered numbers.
+
+This is the walkthrough in [Verify this review](/docs/verify-this-review): export the checksummed
+corpus, verify its checksum, re-run the same public scoring functions from `@loopover/engine`, and
+compare. **Anyone** can do this for public repositories.
+
+
+ **Private repositories are the exception.** A hosted tenant's review history cannot be published
+ for anonymous third-party re-runs. Per-tenant checksummed export — so *you* can verify your own
+ numbers even when the public cannot — is tracked and not yet shipped.
+
+
+### 4. Attested execution
+
+**Claim:** the backtest replay ran unmodified, inside genuine AMD SEV-SNP hardware.
+
+When a run carries an attestation envelope, a skeptic verifies it entirely offline — no contact with
+LoopOver at all, trusting only AMD's published root certificate:
+
+```bash
+npx tsx scripts/verify-attested-run.ts --envelope envelope.json --expected expected.json
+```
+
+Exit `0` means verified; `1`–`8` each name a distinct failure class (sample attestation, bad
+envelope, malformed report, untrusted chain, bad signature, TCB mismatch, measurement mismatch,
+`report_data` mismatch); `9` is a usage or IO error.
+
+
+ **Not reachable in production yet.** The verification software is complete and shipped, but no
+ SEV-SNP hardware is deployed, so no production run carries an envelope today. This row describes a
+ path you can audit now and use once the hardware lands — not a guarantee currently in force.
+
+
+### 5. What the digests actually pin
+
+**Claim:** the configuration and prompt in force at decision time are committed to.
+
+Each decision record carries a `configDigest` (the resolved gate policy), a `settingsDigest` (raw
+effective settings), a `promptDigest` (the exact system prompt sent), and the model identifiers. A
+changed policy or prompt produces a different digest, so silent reconfiguration is detectable.
+
+
+ **Digests commit to inputs, never to outputs.** Identical model identifiers and an identical
+ `promptDigest` can still produce different completions — inference is not deterministic. A digest
+ match means *the same question was asked*, never *the same answer came back*.
+
+
+## What you cannot verify
+
+Stated plainly, because a boundary you discover later is worse than one published up front.
+
+| Not verifiable | Why | What would change it |
+| --- | --- | --- |
+| **Live gate execution** | The merge/close calls acting on your PR are not attested. Putting an attestation service in the live request path trades real availability for a proof that replay already provides more cheaply. | Standing design, not a pending gap. Revisited only if a tenant contractually requires attested live decisions. |
+| **Ground-truth honesty** | Accuracy numbers are scored against recorded human-override events on maintainer infrastructure. Attestation proves *computation*, not *data provenance* — a perfectly attested run over cherry-picked labels is still cherry-picked. | Provenance at capture time (receipts verifiable by the humans whose overrides they record). Different mechanism entirely. |
+| **Wholesale ledger replacement** | Row 1's limit — self-operated chains are tamper-evident against everyone except the operator. | External anchoring: signed checkpoints, a transparency log, or an on-chain commitment. |
+| **Model behavior** | Row 5's limit — no artifact makes a non-deterministic model reproducible. | Nothing planned; this is a property of the models, not of our record-keeping. |
+
+## Self-hosting versus hosted
+
+| | Self-host operator | Hosted tenant |
+| --- | --- | --- |
+| **What you verify** | Everything, locally — your own database, your own corpus export, your own ledger | Public artifacts: ledger integrity, record authenticity, published numbers (public repos), digest scope |
+| **Corpus access** | Direct, against your own Postgres | Public repos today; private-repo export is tracked and not yet shipped |
+| **Trust surface** | You *are* the operator — you trust your own infrastructure | You trust LoopOver's operator honesty exactly as far as the tamper-evident limit allows |
+
+Self-hosting genuinely closes more of the gap, and the hosted column is the weaker of the two. That
+asymmetry is real, and pretending otherwise would defeat the purpose of publishing this contract at
+all.
diff --git a/apps/loopover-ui/src/components/site/docs-nav.tsx b/apps/loopover-ui/src/components/site/docs-nav.tsx
index 29f2786532..5030175182 100644
--- a/apps/loopover-ui/src/components/site/docs-nav.tsx
+++ b/apps/loopover-ui/src/components/site/docs-nav.tsx
@@ -107,6 +107,7 @@ export const docsNav: DocsGroup[] = [
{ to: "/docs/upstream-drift", label: "Upstream drift" },
{ to: "/docs/backtest-calibration", label: "Backtest & calibration" },
{ to: "/docs/verify-this-review", label: "Verify this review" },
+ { to: "/docs/what-you-can-verify", label: "What you can verify" },
],
},
{