Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions apps/loopover-ui/content/docs/self-hosting-configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,68 @@ Lets LoopOver open a pull request that refreshes this repo's own `AGENTS.md`/`CL
]}
/>

## Decision-ledger anchoring (optional)

Your instance keeps a hash-chained ledger of every verdict it persists. That chain is **tamper-evident**
on its own — anyone can walk it and locate an edit. Anchoring makes it **tamper-proof against you**: a
scheduled job publishes a signed checkpoint of the chain's tip to places you do not control (a Sigstore
Rekor transparency log, and a git commit that GH Archive and Software Heritage mirror), so rewriting
history back past a published checkpoint would also require forging a signature or fabricating matching
evidence at an external mirror.

**This is a self-host capability, and only a self-host one.** Hosted review execution is retired, so
`api.loopover.ai` never decides anything and has no chain of its own to anchor — its ledger endpoints
answer for an empty chain and say so (`status: "empty_ledger"`). The chain that matters is the one *your*
instance wrote, so anchoring runs where the decisions were made.

Anchoring is **off until you provision a signing key**, and stays silent-but-visible until then: the public
anchors endpoint reports `status: "unconfigured"` rather than an empty list that could be mistaken for a
healthy idle instance.

### 1. Generate the keypair

```bash
npm run ledger:anchor-keygen
```

Prints both halves already in the encodings the runtime expects, with the key id derived from the public
half so the two cannot drift apart. It writes nothing to disk — the private key exists only in that output,
so run it on a machine you trust and paste straight into your secret store.

### 2. Provision both halves

| Variable | Secret? | What it is |
| --- | --- | --- |
| `LOOPOVER_LEDGER_ANCHOR_KEYS` | No — publish it | The JSON array of published public keys. Served verbatim by `/v1/public/decision-ledger/anchor-key`, which is how a third party verifies your anchors. |
| `LOOPOVER_LEDGER_ANCHOR_PRIVATE_KEY` | **Yes** | PKCS8 PEM. Treat it like any other credential: `wrangler secret put`, a compose secret, or a vault entry — never a committed file. |

Optional, for the git-commit backend (Rekor needs no credentials and uses its public shard by default):

| Variable | What it is |
| --- | --- |
| `LOOPOVER_LEDGER_ANCHOR_GIT_OWNER` / `_REPO` | The repo checkpoints are appended to. A dedicated, public, otherwise-empty repo is the intended shape. |
| `LOOPOVER_LEDGER_ANCHOR_GIT_BRANCH` / `_PATH` | Default `main` / `anchors.jsonl`. |
| `LOOPOVER_LEDGER_ANCHOR_GIT_INSTALLATION_ID` | The GitHub App installation used to commit. Unset means the git backend simply does not run; Rekor is unaffected. |

### 3. Confirm it is live

```bash
curl -s "$ORB/v1/public/decision-ledger/anchors?limit=1" | jq '{status, anchor: .anchors[0]}'
```

`status` distinguishes the states that used to be indistinguishable:

- `anchored` — checkpoints are publishing; the newest one is in `anchor`.
- `pending` — key provisioned and a ledger exists; the first checkpoint is due (hourly, or every 256 rows).
- `empty_ledger` — nothing has been decided yet. Expected on a fresh instance.
- `unconfigured` — no signing key. Anchoring is not running.

### Rotating the key

Keep the retired entry in `LOOPOVER_LEDGER_ANCHOR_KEYS` with its `notAfter` set to the rotation instant,
and append the new one. **A retired key is never removed, only closed** — anchors signed under it must stay
verifiable forever, and a verifier checking a two-year-old checkpoint needs the key it was signed with.

## Next steps

Configure the GitHub integration in [GitHub App and Orb](/docs/self-hosting-github-app), then add optional context through [AI providers](/docs/self-hosting-ai-providers), [REES](/docs/self-hosting-rees), or [RAG](/docs/self-hosting-rag). For the full gate-mode and per-repo settings reference — including the AI-review combine modes and a complete worked manifest — see [Tuning your reviews](/docs/tuning).
36 changes: 31 additions & 5 deletions apps/loopover-ui/content/docs/what-you-can-verify.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,14 @@ only tampering within it.
Every persisted verdict appends to a hash-chained ledger — each row's hash covers the previous row's
hash, so any edit to history breaks the chain at a point you can locate.

Run this against **the instance that reviewed the PR** — the host in the review comment's own links.
A ledger belongs to the runtime that made the decisions, so `api.loopover.ai` is the wrong host to
ask (see "Which host you ask matters" below):

```bash
curl -s "https://api.loopover.ai/v1/public/decision-ledger/verify" | jq
ORB=https://orb.example.org # the instance that reviewed the PR

curl -s "$ORB/v1/public/decision-ledger/verify" | jq
```

Returns `{ ok, checked, nextAfterSeq, tipSeq, tipHash, totalCount, prunedRecords }`, and a `break`
Expand Down Expand Up @@ -63,13 +69,33 @@ Two response fields encode deliberate, published semantics rather than tolerance
every row individually external-checkable in real time.
</Callout>

<Callout variant="warn">
**Which host you ask matters.** A decision ledger belongs to the runtime that *made* the decisions,
and hosted review execution is retired — only self-host runtimes execute reviews, so each one
writes and anchors **its own** chain in its own database. `api.loopover.ai` therefore has no ledger
of its own to anchor and never will: its `/v1/public/decision-ledger/*` endpoints answer for an
empty chain, and its `anchors` response says so explicitly via `status: "empty_ledger"` rather
than an ambiguous empty list.

So every command on this page runs against `$ORB`, the instance whose reviews you are checking —
never against `api.loopover.ai`.

This is a deliberate boundary, not a gap. Aggregating every instance's records into one central
chain would produce an anchored artifact that looks *more* authoritative while proving *less*: the
central chain would be a re-chained aggregate nobody's verifier cares about, and each instance's
real chain would still be unanchored.
</Callout>

**Verify an anchor end to end**

a. Fetch the most recent anchor and the currently-published signing key:
a. Fetch the most recent anchor and the currently-published signing key. `status` tells you which
state you are in before you read the list — `anchored`, `empty_ledger` (nothing decided yet),
`unconfigured` (no signing key provisioned), or `pending` (a ledger exists and is due its first
anchor):

```bash
curl -s "https://api.loopover.ai/v1/public/decision-ledger/anchors?limit=1" | jq '.anchors[0]' > anchor.json
curl -s "https://api.loopover.ai/v1/public/decision-ledger/anchor-key" | jq -c '.keys' > anchor-keys.json
curl -s "$ORB/v1/public/decision-ledger/anchors?limit=1" | jq '{status, anchor: .anchors[0]}' > anchor.json
curl -s "$ORB/v1/public/decision-ledger/anchor-key" | jq -c '.keys' > anchor-keys.json
```

b. Fetch the actual signed payload the anchor committed to. For the git-commit backend,
Expand Down Expand Up @@ -108,7 +134,7 @@ hash yourself:

```bash
SEQ=$(jq -r '.payload.seq' anchor-signed.json)
curl -s "https://api.loopover.ai/v1/public/decision-ledger/row/$SEQ" | jq > row.json
curl -s "$ORB/v1/public/decision-ledger/row/$SEQ" | jq > row.json
npx tsx -e '
import { readFileSync } from "node:fs";
import { ledgerRowHash } from "./src/review/decision-record.ts";
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"deploy": "wrangler deploy",
"deploy:api": "turbo run build --filter=@loopover/engine --filter=@loopover/contract && wrangler d1 migrations apply loopover --remote && wrangler deploy",
"selfhost:postgres:migrate": "tsx scripts/migrate-selfhost-sqlite-to-postgres.ts",
"ledger:anchor-keygen": "tsx scripts/gen-ledger-anchor-keypair.ts",
"selfhost:env-reference": "node --experimental-strip-types scripts/gen-selfhost-env-reference.ts",
"selfhost:env-reference:check": "node --experimental-strip-types scripts/gen-selfhost-env-reference.ts --check",
"miner:env-reference": "tsx packages/loopover-miner/scripts/generate-env-reference.ts",
Expand Down
82 changes: 82 additions & 0 deletions scripts/gen-ledger-anchor-keypair.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/usr/bin/env node
// Generate the decision-ledger anchor signing keypair (#9719, epic #9267).
//
// Anchoring shipped with no way to provision the key it needs. Both halves of the scheduler's second guard
// (`LOOPOVER_LEDGER_ANCHOR_KEYS`, `LOOPOVER_LEDGER_ANCHOR_PRIVATE_KEY`) had to be produced by hand, in the
// exact encodings `parseAnchorPublicKeys` and `signLedgerAnchorPayload` expect, with a keyId that is
// `computeAnchorKeyId` of the public half -- and nothing in the repo said how. An operator who guessed any
// of those wrong got `ledger_anchor_skipped_unconfigured` and an empty anchor list, which until #9755 was
// indistinguishable from a healthy idle instance.
//
// This prints both values ready to paste, deriving the keyId with the SAME function the runtime uses, so
// the published key and the anchors that reference it cannot disagree.
//
// npm run ledger:anchor-keygen
//
// PRINTS a private key to stdout. It is never written to disk, never committed, and the output is meant to
// go straight into your secret store (`wrangler secret put`, a compose env file, a vault entry). Run it on a
// machine you trust, and do not paste the private half into a shell history you keep.
import { computeAnchorKeyId } from "../src/review/ledger-anchor";

function toPem(base64: string, label: string): string {
return `-----BEGIN ${label}-----\n${(base64.match(/.{1,64}/g) ?? []).join("\n")}\n-----END ${label}-----`;
}

function bytesToBase64(bytes: Uint8Array): string {
let binary = "";
for (const byte of bytes) binary += String.fromCharCode(byte);
return btoa(binary);
}

export type GeneratedAnchorKeypair = {
keyId: string;
/** Exactly the string `LOOPOVER_LEDGER_ANCHOR_KEYS` takes -- a JSON array, parseable by
* `parseAnchorPublicKeys` without further massaging. */
publishedKeys: string;
/** Exactly the string `LOOPOVER_LEDGER_ANCHOR_PRIVATE_KEY` takes -- PKCS8 PEM, importable by
* `signLedgerAnchorPayload`. */
privateKeyPem: string;
};

/**
* Produce both halves in the encodings the runtime expects. Exported and returning values rather than
* printing them, so a test can assert the OUTPUT actually round-trips -- generating a key that the runtime
* then refuses is precisely the failure this script exists to prevent, and a print-only script could not be
* checked for it.
*/
export async function generateAnchorKeypair(now: string = new Date().toISOString()): Promise<GeneratedAnchorKeypair> {
// P-256 / SHA-256 -- the pair `signLedgerAnchorPayload` imports and Rekor's `PKIX_ECDSA_P256_SHA_256`
// verifier expects. Any other curve produces anchors Rekor rejects and third parties cannot check.
const pair = (await crypto.subtle.generateKey({ name: "ECDSA", namedCurve: "P-256" }, true, ["sign", "verify"])) as CryptoKeyPair;
const pkcs8 = bytesToBase64(new Uint8Array((await crypto.subtle.exportKey("pkcs8", pair.privateKey)) as ArrayBuffer));
const publicKeySpki = bytesToBase64(new Uint8Array((await crypto.subtle.exportKey("spki", pair.publicKey)) as ArrayBuffer));
// Derived with the SAME function the runtime uses, so the published key and the anchors referencing it
// cannot disagree about the id.
const keyId = await computeAnchorKeyId(publicKeySpki);
// `notBefore` is now: an anchor signed before its key's validity window would not verify against it.
return {
keyId,
publishedKeys: JSON.stringify([{ keyId, publicKeySpki, notBefore: now, notAfter: null }]),
privateKeyPem: toPem(pkcs8, "PRIVATE KEY"),
};
}

async function main(): Promise<void> {
const { keyId, publishedKeys, privateKeyPem } = await generateAnchorKeypair();

console.log("# ── LOOPOVER_LEDGER_ANCHOR_KEYS (public, safe to publish and to commit) ──");
console.log("# Serve this verbatim; it is what /v1/public/decision-ledger/anchor-key returns.");
console.log(`LOOPOVER_LEDGER_ANCHOR_KEYS='${publishedKeys}'`);
console.log("");
console.log("# ── LOOPOVER_LEDGER_ANCHOR_PRIVATE_KEY (SECRET — never commit) ──");
console.log("# wrangler secret put LOOPOVER_LEDGER_ANCHOR_PRIVATE_KEY (paste the block below)");
console.log(privateKeyPem);
console.log("");
console.log(`# keyId ${keyId} — derived from the public half, so it cannot drift from the key it names.`);
console.log("# ROTATION: keep the retired entry in LOOPOVER_LEDGER_ANCHOR_KEYS with notAfter set to the");
console.log("# rotation instant and append the new one. Anchors signed under the old key must stay");
console.log("# verifiable forever, so a retired key is never removed -- only closed.");
}

// Only when run directly, so importing this for a test does not print a private key into the test log.
if (process.argv[1]?.endsWith("gen-ledger-anchor-keypair.ts")) await main();
83 changes: 83 additions & 0 deletions test/unit/gen-ledger-anchor-keypair.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
import { describe, expect, it } from "vitest";
import { generateAnchorKeypair } from "../../scripts/gen-ledger-anchor-keypair";
import {
anchorKeyById,
buildLedgerAnchorPayload,
computeAnchorKeyId,
currentAnchorKey,
parseAnchorPublicKeys,
signLedgerAnchorPayload,
verifyLedgerAnchorSignature,
} from "../../src/review/ledger-anchor";

// #9719: anchoring shipped with no way to provision the key it needs, so both halves had to be produced by
// hand in the exact encodings the runtime expects. A generator that emits a key the runtime then REFUSES
// would reproduce the original failure (`ledger_anchor_skipped_unconfigured`, empty anchor list) while
// looking like a fix — so these tests run the generated values through the real parse/sign/verify path
// rather than checking their shape.

const AT = "2026-07-29T00:00:00.000Z";

describe("gen-ledger-anchor-keypair (#9719)", () => {
it("REGRESSION: the generated values feed the REAL runtime path end to end — parse, select, sign, verify", async () => {
const generated = await generateAnchorKeypair(AT);

// 1. The published half parses through the same function the anchor-key route serves from.
const keys = parseAnchorPublicKeys(generated.publishedKeys);
expect(keys).toHaveLength(1);
// 2. It is SELECTABLE as the current key -- an entry the scheduler cannot pick is the exact
// `no_current_signing_key_published` early return that left anchors empty.
const current = currentAnchorKey(keys);
expect(current?.keyId).toBe(generated.keyId);

// 3. The private half imports and signs.
const payload = buildLedgerAnchorPayload({ seq: 7, rowHash: "a".repeat(64), totalCount: 7 }, AT);
const signed = await signLedgerAnchorPayload(payload, generated.privateKeyPem, generated.keyId);

// 4. And a third party holding ONLY the published half verifies it -- the whole point of the artifact.
const resolved = anchorKeyById(keys, signed.keyId);
expect(resolved).not.toBeNull();
expect(await verifyLedgerAnchorSignature(signed, String(resolved?.publicKeySpki))).toBe(true);
});

it("the keyId is DERIVED from the public half, so the published key cannot drift from the id naming it", async () => {
const generated = await generateAnchorKeypair(AT);
const [published] = parseAnchorPublicKeys(generated.publishedKeys);
expect(await computeAnchorKeyId(String(published?.publicKeySpki))).toBe(generated.keyId);
});

it("emits an OPEN validity window at the supplied instant — a key valid from 'now', never expired", async () => {
const generated = await generateAnchorKeypair(AT);
const [published] = parseAnchorPublicKeys(generated.publishedKeys);
expect(published).toMatchObject({ notBefore: AT, notAfter: null });
});

it("INVARIANT: every run is a fresh key — a generator that repeated itself would be catastrophic", async () => {
const [a, b] = await Promise.all([generateAnchorKeypair(AT), generateAnchorKeypair(AT)]);
expect(a.keyId).not.toBe(b.keyId);
expect(a.privateKeyPem).not.toBe(b.privateKeyPem);
});

it("REGRESSION: a signature does NOT verify against a different run's key", async () => {
// Guards the pairing itself: publishing key A while signing with key B would produce anchors that
// silently fail every third-party check.
const mine = await generateAnchorKeypair(AT);
const other = await generateAnchorKeypair(AT);
const signed = await signLedgerAnchorPayload(
buildLedgerAnchorPayload({ seq: 1, rowHash: "b".repeat(64), totalCount: 1 }, AT),
mine.privateKeyPem,
mine.keyId,
);
const [otherPublished] = parseAnchorPublicKeys(other.publishedKeys);
expect(await verifyLedgerAnchorSignature(signed, String(otherPublished?.publicKeySpki))).toBe(false);
});

it("the PEM is real PKCS8 armor, so it pastes into a secret store unmodified", async () => {
const generated = await generateAnchorKeypair(AT);
expect(generated.privateKeyPem.startsWith("-----BEGIN PRIVATE KEY-----\n")).toBe(true);
expect(generated.privateKeyPem.trimEnd().endsWith("-----END PRIVATE KEY-----")).toBe(true);
// Wrapped at 64 columns like every other PEM, so a copy-paste through a YAML block stays valid.
const body = generated.privateKeyPem.split("\n").slice(1, -1);
for (const line of body) expect(line.length).toBeLessThanOrEqual(64);
});
});
Loading