feat: agent skills for usage, receipts, distillation, and backtesting - #11
Merged
Conversation
Four agentskills.io-format skills in skills/: sqlite-predict (core usage: operation selection, the aggregate convention, statuses vs errors, defaults), prediction-receipts (agent-owned provenance: a _predict_receipts table convention, hashing the result document, replay verification; possible because serving is deterministic and models are content-hashed), distill-lifecycle (verify holdout before serving, drift and re-distill cadence, license discipline), and interpret-backtest (MASE against the naive floor, coverage honesty, conformal judgment, when to narrow auto's pool). README gains an agent skills pointer. Skills carry judgment, not wrappers: the SQL surface is already the API. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 11 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (11)
📝 WalkthroughWalkthroughAdds agent skill documentation for sqlite-predict usage, prediction receipts, distillation lifecycle management, and backtest interpretation. Implements a receipt CLI with deterministic result hashing, SQL hash support, model pinning, replay verification, and integration tests. ChangesAgent skills and operational workflows
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Agent
participant receipt_py
participant SQLite
participant sqlite_predict
Agent->>receipt_py: Record prediction SQL
receipt_py->>sqlite_predict: Load extension and execute SQL
sqlite_predict->>SQLite: Read inputs and model metadata
SQLite-->>receipt_py: Return canonical result
receipt_py->>SQLite: Store result hash receipt
Agent->>receipt_py: Verify receipt
receipt_py->>SQLite: Replay recorded SQL
SQLite-->>receipt_py: Return replayed result
receipt_py-->>Agent: Return match report
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
All four validate with the agentskills reference validator (skills-ref). Add the optional license field and quote metadata values per the spec's string-map recommendation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
skills/prediction-receipts/scripts/receipt.py is the interoperability anchor: one stdlib-only implementation of the convention (record, verify with exit-code semantics, list), with canonicalization defined in one place (aggregate documents hashed verbatim; row results as compact JSON with shortest round-trip floats). The skill now points at it as the preferred path, keeping the hand-rolled steps as the spec. tests/test_skills.py wires both into the normal pytest run: every SKILL.md is held to the agentskills spec (name/dir match, name charset, description bounds and when-to-use, line budget, house dash rule), and the receipt script is proven end to end: record, replay-match, re-record determinism, tamper detection via exit code 2, and registry content_hash pinning for bundled models and distilled students. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/distill-lifecycle/SKILL.md`:
- Around line 70-76: Revise the licensing guidance in the distillation section
to state that the teacher’s license may impose obligations depending on the
exact weights, license, distribution method, and jurisdiction. Require recording
the exact teacher license with the student, and clarify that accept_license is
only an opt-in mechanism, not proof of compliance.
- Around line 49-55: Update the soft-label guidance in the distillation
lifecycle documentation to qualify calibration preservation and imbalance
benefits on the teacher continuing to emit meaningful probabilities; avoid
claiming soft labels rescue severely imbalanced datasets that collapse to a
single class, while retaining the existing refusal behavior.
- Around line 16-19: Update the introductory description in the distillation
lifecycle documentation to avoid presenting “a few kilobytes,” “microseconds,”
and “seconds” as guaranteed performance claims. Either add authoritative sources
for each figure or soften the wording to clearly identify them as illustrative,
while preserving the surrounding explanation of the teacher-to-student
lifecycle.
In `@skills/interpret-backtest/SKILL.md`:
- Around line 43-45: Replace the standalone JSON fragment in the conformal
interval example with a complete backtest(...) call, including the required
query and horizon arguments and the interval_method configuration. Keep the
example in the existing SQL block and ensure it clearly shows where the
configuration is passed.
- Around line 47-51: Revise the conformal intervals guidance to avoid
guaranteeing nominal coverage: state that they can improve empirical calibration
based on measured residuals, while retaining the caveat about needing sufficient
folds and the instruction to verify coverage with backtest on the target series.
- Around line 39-40: Update the default Gaussian band discussion in SKILL.md to
either cite the benchmark source and specify the model and dataset scope
supporting the 0.57 coverage figure, or remove the exact figure while preserving
the broader overconfidence statement.
In `@skills/prediction-receipts/scripts/receipt.py`:
- Around line 62-68: Update canonical_document and its callers so
canonicalization depends on the operation type: preserve raw string output only
for operations that require aggregate-document hashing, while predict and
backtest TVF results always serialize as {"columns": ..., "rows": ...},
including one-row, one-TEXT-column results. Pass the operation context through
the affected call sites rather than inferring behavior from result shape.
- Around line 52-59: Update connect so extension loading is disabled immediately
after db.load_extension(extension) completes, including before returning the
connection; preserve the existing failure handling and ensure cmd_verify’s
subsequent input_sql execution uses the connection with extension loading
disabled.
In `@tests/test_skills.py`:
- Around line 76-136: Extend the receipt tests around
test_receipt_record_verify_and_tamper and test_receipt_pins_a_registered_student
with adversarial cases for one-row textual TVF results, malformed options,
unknown options, and verification of a nonexistent receipt. Exercise extension
failure paths and assert stderr or structured errors contain the exact
PREDICT_ERR_* code, not descriptive prose, while preserving existing success and
tamper assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: a9484e45-1705-4622-aaa1-e7dba4553ab7
📒 Files selected for processing (7)
README.mdskills/distill-lifecycle/SKILL.mdskills/interpret-backtest/SKILL.mdskills/prediction-receipts/SKILL.mdskills/prediction-receipts/scripts/receipt.pyskills/sqlite-predict/SKILL.mdtests/test_skills.py
Python was the receipt script's only dependency, and the environments this extension targets (phone, embedded, wasm) may not have it. The lightest dependency is the extension itself: expose the vendored SHA-256 (the hash that already pins model weights) as a deterministic, innocuous predict_sha256(x) scalar (TEXT/BLOB, NULL passthrough), and rewrite the prediction-receipts skill so record and replay-verify run in pure SQL. The Python script stays as optional convenience for row-shaped results (canonical float serialization) and mismatch diagnostics. CI: hash vectors against hashlib, plus a pure-SQL record/verify/tamper round trip; functions reference documents the new scalar. Also gitignores autogluon's local model dumps, which must never be committed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Security (critical): receipt.py locks extension loading immediately
after loading predict0, so a tampered receipt's replayed input_sql can
no longer call load_extension() on an arbitrary library; a regression
test replays exactly that attack and asserts sqlite refuses it.
Interoperability (major): canonicalization now follows the recorded
operation instead of the result shape, so a one-text-cell backtest or
predict result hashes as {"columns","rows"} rather than masquerading
as an aggregate document; verify() reads the stored operation.
Claims discipline (the reviewer enforcing our own path instructions):
size/latency claims in distill-lifecycle now cite the benchmarks and
state measured ranges; the soft-label rescue is scoped and qualified;
the licensing section says obligations 'may' apply, tells users to
record the exact license with each student, and states that
accept_license is an acknowledgment, not compliance; the 0.57 coverage
figure carries its model/dataset scope and source; the conformal
fragment became a complete backtest() call; and conformal coverage is
'substantially improves, reached nominal in our benchmarks, no
finite-sample guarantee' instead of a promise.
Tests: adversarial cases for the one-cell TVF canonicalization, unknown
option keys asserting the exact PREDICT_ERR_OPTIONS code, missing
receipt ids, and the load_extension replay attack. Eight pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 8
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/prediction-receipts/SKILL.md`:
- Around line 90-94: Update the fenced command block in the prediction receipts
documentation to include the text language tag, leaving the commands and
surrounding documentation unchanged.
- Around line 71-81: The receipt SQL example must not be presented as full
receipt verification. Label it as manual verification for the specific hardcoded
forecast query, state that it ignores stored operation, input_sql, options,
model_id, and content_hash metadata, and direct users to the reference script
for full replay verification.
- Around line 51-67: Make the receipt-recording example executable when
_predict_models is absent by separating the registered-model and
unregistered-model SQL paths, or by creating the registry table before the
INSERT. In the unregistered path, set content_hash to NULL while keeping
input_sql identical to the SQL used to compute doc; preserve the existing lookup
for registered models.
- Around line 51-67: Update the receipt example around _predict_receipts so
input_sql follows one consistent replay contract: store the exact SQL used to
compute doc verbatim, or revise the doc query to represent the supplied-row
query, ensuring both expressions match exactly. Also make the content_hash
lookup optional by recording NULL when _predict_models is unavailable, while
preserving the existing model and hash behavior when it exists.
- Around line 47-48: Update the workflow description around predict_sha256() to
distinguish result-document hashing from model-weight pinning: describe
predict_sha256(doc) as hashing the prediction result, and identify the registry
content_hash as the separate provenance value for pinned model weights. Remove
the claim that they are the same hash.
In `@sqlite-predict.c`:
- Around line 423-453: Update predict_sha256_fn to accept only SQLITE_TEXT and
SQLITE_BLOB values; for all other non-NULL types, return the appropriate
existing PREDICT_ERR_* error through the SQLite context instead of converting
with sqlite3_value_text(). Preserve NULL-to-NULL behavior and the current
hashing behavior for TEXT and BLOB inputs.
In `@tests/test_skills.py`:
- Around line 217-220: Replace the prose stderr check in the missing-receipt
test around run_receipt("verify", ...) with an assertion for the appropriate
PREDICT_ERR_* code representing a missing receipt. Preserve the nonzero
return-code assertion and ensure the test validates the CLI’s stable structured
error identifier rather than message wording.
In `@website/src/content/docs/reference/functions.md`:
- Line 126: Update the predict_sha256(x) documentation row after the C
implementation contract is finalized: either document only TEXT/BLOB inputs if
numeric arguments are rejected, or explicitly include INTEGER/REAL inputs and
their defined representation if numeric support is retained. Ensure the
description matches the behavior implemented by the predict_sha256 function and
does not claim a narrower input scope.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 5e842f56-9b37-4ad3-be51-a24453b9eb00
📒 Files selected for processing (8)
.gitignoreskills/distill-lifecycle/SKILL.mdskills/interpret-backtest/SKILL.mdskills/prediction-receipts/SKILL.mdskills/prediction-receipts/scripts/receipt.pysqlite-predict.ctests/test_skills.pywebsite/src/content/docs/reference/functions.md
🚧 Files skipped from review as they are similar to previous changes (3)
- skills/distill-lifecycle/SKILL.md
- skills/interpret-backtest/SKILL.md
- skills/prediction-receipts/scripts/receipt.py
predict_sha256 rejects INTEGER/REAL with PREDICT_ERR_SCHEMA instead of hashing SQLite's 15-digit number-to-text coercion, which could collapse distinct doubles into one hash; the functions reference documents the contract. The receipt CLI gains stable RECEIPT_ERR_* failure codes so agents branch on outcomes the way they do on PREDICT_ERR_* (the test asserts the code, not prose). The receipts skill now distinguishes result hashing from weight pinning explicitly, ships an executable no-registry recording variant, labels the pure-SQL verify as a spot check of one query's hash and routes full replay to the reference script, and tags the command block for MD040. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
It lands on main with the scale results. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@coderabbitai review All round-two findings addressed in 18ceff0: predict_sha256 rejects numeric arguments with PREDICT_ERR_SCHEMA, the receipt CLI has stable RECEIPT_ERR_* codes asserted by the tests, the receipts skill distinguishes result hashing from weight pinning, ships the executable no-registry variant, scopes the pure-SQL verify as a spot check, and the command block is tagged. 3aaef19 removes an unrelated file that slipped into the branch. |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '=== PR commits and final changed files ==='
git log --oneline --decorate -12
git diff --name-status HEAD~3..HEAD
printf '%s\n' '=== Updates stated by the author ==='
git show --stat --oneline 18ceff0
git show --stat --oneline 3aaef19
git show --format=fuller --no-ext-diff 18ceff0 -- \
sqlite-predict.c \
skills/prediction-receipts/SKILL.md \
skills/prediction-receipts/scripts/receipt.py \
tests/test_skills.py
printf '%s\n' '=== Target file outlines ==='
ast-grep outline sqlite-predict.c --items all --match 'predict_sha256|sqlite3_predict_init' || true
ast-grep outline skills/prediction-receipts/scripts/receipt.py --items all
ast-grep outline tests/test_skills.py --items all --match 'receipt|pure_sql|adversarial|sha256' || trueLength of output: 13237 🏁 Script executed: #!/bin/bash
set -euo pipefail
base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main 2>/dev/null || true)"
printf '=== merge base: %s ===\n' "${base:-unavailable}"
if [ -n "${base:-}" ]; then
git diff --stat "$base"..HEAD
git diff --name-status "$base"..HEAD
fi
printf '%s\n' '=== receipt CLI ==='
nl -ba skills/prediction-receipts/scripts/receipt.py | sed -n '1,270p'
printf '%s\n' '=== hash UDF and registration ==='
nl -ba sqlite-predict.c | sed -n '400,470p'
nl -ba sqlite-predict.c | sed -n '630,700p'
printf '%s\n' '=== receipt skill ==='
nl -ba skills/prediction-receipts/SKILL.md | sed -n '1,180p'
printf '%s\n' '=== receipt tests ==='
nl -ba tests/test_skills.py | sed -n '45,255p'
printf '%s\n' '=== benchmark result references and tracked state ==='
git ls-files benchmarks/results/scale-arena.jsonl benchmarks/results/scale.md benchmarks/scale_arena.py
rg -n -C2 'scale-arena|scale\.md|scale_arena' README.md benchmarks .gitignore 2>/dev/null || trueLength of output: 2609 🏁 Script executed: #!/bin/bash
set -euo pipefail
numbered() {
awk '{printf "%6d\t%s\n", NR, $0}' "$1" | sed -n "${2},${3}p"
}
printf '%s\n' '=== receipt CLI ==='
numbered skills/prediction-receipts/scripts/receipt.py 1 270
printf '%s\n' '=== hash UDF and registration ==='
numbered sqlite-predict.c 400 475
numbered sqlite-predict.c 630 705
printf '%s\n' '=== receipt skill ==='
numbered skills/prediction-receipts/SKILL.md 1 185
printf '%s\n' '=== receipt tests ==='
numbered tests/test_skills.py 45 260
printf '%s\n' '=== all fail() call sites ==='
rg -n -C2 '\bfail\s*\(' skills/prediction-receipts/scripts/receipt.py
printf '%s\n' '=== retained benchmark results ==='
for f in benchmarks/results/scale-arena.jsonl benchmarks/results/scale.md; do
printf '\n--- %s ---\n' "$f"
awk '{printf "%6d\t%s\n", NR, $0}' "$f"
done
printf '%s\n' '=== references outside the retained results ==='
rg -n -C2 'scale-arena|scale_arena|scale\.md' \
-g '!benchmarks/results/scale-arena.jsonl' \
-g '!benchmarks/results/scale.md' . || trueLength of output: 35001
✅ Action performedReview finished.
|
The skill claimed the reference script re-derives everything from stored fields; it only replayed input_sql and compared hashes, so a tampered model_id or options column still verified. Now verify() re-derives the serving model from the replayed result and fails the verification (exit 2, id_matches false) when it contradicts the recorded model_id; row-form receipts recorded via --model-id report id_matches null since no model is derivable. The options column is declared informational in both the report (options_verified: false) and the skill text: its authoritative copy is the options text inside input_sql, which the replay executes verbatim, and the receipt row itself is self-attested. Adversarial tests cover the detected forgery (model_id) and pin the documented limit (options). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@coderabbitai review Round-three finding addressed in 1e49364: verify() now re-derives the serving model from the replayed result and fails verification (exit 2, id_matches false) on a forged model_id; row-form receipts report id_matches null since no model is derivable. The options column is declared informational in the report (options_verified: false) and the skill text, with the receipts row's self-attested nature stated plainly. Adversarial tests cover the detected forgery and pin the documented options limit. |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '=== target commit ==='
git show --stat --oneline 1e49364
git show --format= --no-ext-diff 1e49364 -- \
skills/prediction-receipts/scripts/receipt.py \
skills/prediction-receipts/SKILL.md \
tests/test_skills.py
printf '%s\n' '=== receipt CLI outline ==='
ast-grep outline skills/prediction-receipts/scripts/receipt.py --items all
printf '%s\n' '=== verifier and receipt-related implementation ==='
nl -ba skills/prediction-receipts/scripts/receipt.py | sed -n '1,290p'
printf '%s\n' '=== receipt skill ==='
nl -ba skills/prediction-receipts/SKILL.md | sed -n '1,220p'
printf '%s\n' '=== receipt tests ==='
nl -ba tests/test_skills.py | sed -n '1,330p'
printf '%s\n' '=== all receipt-field references ==='
rg -n -C3 '\b(model_id|options|id_matches|options_verified|result_sha256|content_hash)\b' \
skills/prediction-receipts/scripts/receipt.py \
skills/prediction-receipts/SKILL.md \
tests/test_skills.pyLength of output: 8024 🏁 Script executed: #!/bin/bash
set -euo pipefail
numbered() {
awk '{printf "%6d\t%s\n", NR, $0}' "$1" | sed -n "${2},${3}p"
}
printf '%s\n' '=== PR changed files from merge base ==='
base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main 2>/dev/null || true)"
printf 'merge base: %s\n' "${base:-unavailable}"
if [ -n "${base:-}" ]; then
git diff --name-status "$base"..HEAD
git diff --check "$base"..HEAD
fi
printf '%s\n' '=== complete receipt script ==='
numbered skills/prediction-receipts/scripts/receipt.py 1 300
printf '%s\n' '=== complete receipts skill ==='
numbered skills/prediction-receipts/SKILL.md 1 230
printf '%s\n' '=== full receipt-test context ==='
numbered tests/test_skills.py 1 340
printf '%s\n' '=== receipt schema and verification query references ==='
rg -n -C4 'CREATE TABLE|INSERT INTO _predict_receipts|SELECT .*_predict_receipts|UPDATE _predict_receipts|cmd_record|cmd_verify|model_fields|canonical_document' \
skills/prediction-receipts/scripts/receipt.py \
tests/test_skills.py
printf '%s\n' '=== current changes to reviewed files ==='
git diff --no-ext-diff "$base"..HEAD -- \
skills/prediction-receipts/scripts/receipt.py \
skills/prediction-receipts/SKILL.md \
tests/test_skills.pyLength of output: 70720
✏️ Learnings added
✅ Action performedReview finished.
|
Locking extension loading closed one door and left the write door open: a tampered receipt's input_sql could DELETE, DROP, ATTACH, or flip pragmas during verification. verify() and list now open the database read-only at the file level and install an authorizer that permits only reads and function calls; every legitimate replay (aggregates, predict, backtest) is pure and passes, every write shape is denied at prepare time. Adversarial tests replay DELETE, DROP, ATTACH, and PRAGMA receipts, assert refusal, and assert the verifier's data is untouched. The skill states the replay posture explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@coderabbitai review Round-four finding addressed: verify() and list now run on a read-only (mode=ro) connection with an allow-list authorizer (reads and function calls only), so a tampered input_sql cannot write, drop, attach, or change pragmas during replay. Adversarial tests replay DELETE/DROP/ATTACH/PRAGMA receipts and assert both refusal and untouched data. The skill documents the untrusted-replay posture. |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
base="$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main 2>/dev/null || true)"
printf '=== merge base ===\n%s\n' "${base:-unavailable}"
if [ -n "${base:-}" ]; then
printf '\n=== changed files ===\n'
git diff --name-status "$base"..HEAD
printf '\n=== latest commits ===\n'
git log --oneline "$base"..HEAD
printf '\n=== whitespace check ===\n'
git diff --check "$base"..HEAD
fi
printf '\n=== receipt script outline ===\n'
ast-grep outline skills/prediction-receipts/scripts/receipt.py --items all
printf '\n=== receipt script relevant sections ===\n'
nl -ba skills/prediction-receipts/scripts/receipt.py | sed -n '1,360p'
printf '\n=== security-related call sites ===\n'
rg -n -C4 'connect\(|set_authorizer|authorizer|mode=ro|uri=|execute\(|cmd_verify|cmd_list|SQLITE_' \
skills/prediction-receipts/scripts/receipt.py
printf '\n=== receipt documentation ===\n'
nl -ba skills/prediction-receipts/SKILL.md | sed -n '1,260p'
printf '\n=== receipt-related tests ===\n'
rg -n -C8 'DELETE|DROP|ATTACH|PRAGMA|authoriz|read-only|readonly|cmd_verify|cmd_list|adversarial|receipt' \
tests/test_skills.pyLength of output: 3530 🏁 Script executed: #!/bin/bash
set -euo pipefail
numbered() {
awk '{printf "%6d\t%s\n", NR, $0}' "$1"
}
printf '%s\n' '=== round-four security commit ==='
git show --format=fuller --stat --no-ext-diff cb0f5c6
git show --format= --no-ext-diff cb0f5c6 -- \
skills/prediction-receipts/scripts/receipt.py \
skills/prediction-receipts/SKILL.md \
tests/test_skills.py
printf '%s\n' '=== complete receipt script ==='
numbered skills/prediction-receipts/scripts/receipt.py
printf '%s\n' '=== receipt skill ==='
numbered skills/prediction-receipts/SKILL.md
printf '%s\n' '=== receipt tests ==='
numbered tests/test_skills.py
printf '%s\n' '=== Python SQLite authorizer constants used by this script ==='
rg -n -C2 '\bsqlite3\.SQLITE_[A-Z_]+\b' skills/prediction-receipts/scripts/receipt.py tests/test_skills.py
printf '%s\n' '=== replay and list invocation sites ==='
rg -n -C5 '\b(cmd_verify|cmd_list|connect)\s*\(|\bset_authorizer\s*\(|\bexecute\s*\(' \
skills/prediction-receipts/scripts/receipt.pyLength of output: 46758
✅ Action performedReview finished.
|
Rather than waiting for the next review round to find the next adjacent gap, this closes the full remaining surface found by an adversarial self-audit of the receipt tool: a VM-step budget aborts runaway replays (recursive CTE bombs) instead of hanging the verifier; BLOB cells canonicalize deterministically as their sha256 instead of crashing json serialization; --options must parse as JSON before anything executes; database-open and SQL failures exit with stable RECEIPT_ERR_* codes instead of tracebacks. Every path has a test: the bomb aborts under a small step budget, blob receipts round-trip deterministically, malformed options and missing databases fail clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
@coderabbitai full review Beyond the round-four fix (read-only replay with an allow-list authorizer, tested against DELETE/DROP/ATTACH/PRAGMA receipts), cb0f5c6 and this push close the rest of the surface from our own adversarial audit: VM-step budget against runaway replays, deterministic BLOB canonicalization, upfront options validation, and stable RECEIPT_ERR_* codes on every failure path instead of tracebacks. Eleven tests cover the attack and hardening set. Requesting a full review rather than incremental so the whole file is judged as it now stands. |
|
✅ Action performedFull review finished. Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 11 minutes. |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Four agentskills.io-format skills in
skills/, plus a README pointer. Skills carry judgment, not wrappers, since the SQL surface is already the whole API for an agent:auto, conformal, candidates)._predict_receiptstable convention: record the input SQL, options, serving model id and its registrycontent_hash, extension version, and a SHA-256 of the exact result document; replay by re-running and comparing hashes. This is the receipts design living at the layer it always belonged: serving determinism and content-addressed models make it verifiable with zero support from the C core.holdout_metricbefore serving, the measured shape of distillation loss (regression tails are worse than classification), drift and versioned re-distillation, and license discipline (orientation only; points at the docs' not-legal-advice stance).auto's candidate pool.Why now
Main is locked at the release state; this lands as follow-on activity. It also gives the new CodeRabbit setup its first real PR to review.
Test plan
_predict_modelscolumns).skills/exists in this branch).🤖 Generated with Claude Code
Summary by CodeRabbit
predict_sha256(x)for deterministic hashing of TEXT/BLOB inputs.predict_sha256.