Skip to content

test: add GuardrailDecision source-provenance vectors#168

Draft
safal207 wants to merge 6 commits into
mainfrom
agent/guardrail-provenance-vectors
Draft

test: add GuardrailDecision source-provenance vectors#168
safal207 wants to merge 6 commits into
mainfrom
agent/guardrail-provenance-vectors

Conversation

@safal207

Copy link
Copy Markdown
Owner

What changed

Adds a provider-neutral source-provenance conformance corpus for the GuardrailDecisionV1 discussion in crewAIInc/crewAI#4877:

  • ten positive and negative vectors;
  • a stdlib-only independent checker with no ProofPath, CrewAI, CKG, or vendor imports;
  • frozen and changed source representations;
  • explicit proof levels: COMMITTED, REPRODUCIBLE, FRESH, and INVALID;
  • fail-closed cases for tampered snapshots, missing snapshots, malformed hashes, and ambiguous byte representations;
  • explicit signals for stale sources, unproven temporal precedence, and declared-but-unverified external anchors.

Why

knowledge_source_ref + source_content_hash can detect that a live source changed, but a hash alone does not preserve the historical bytes needed for independent reproduction. The corpus separates commitment, historical reproducibility, current freshness, and temporal anchoring instead of treating them as one guarantee.

It also pins the hashed representation (response-body-raw or response-body-decompressed) so two clients do not silently hash different byte streams for the same URL.

Validation

PASS hash-only-is-committed-not-reproducible -> COMMITTED
PASS snapshot-closes-historical-reproduction -> REPRODUCIBLE
PASS matching-current-source-is-fresh -> FRESH
PASS changed-current-source-is-stale-but-history-remains-reproducible -> REPRODUCIBLE
PASS tampered-snapshot-fails-closed -> INVALID
PASS missing-snapshot-fails-closed -> INVALID
PASS ambiguous-representation-fails-closed -> INVALID
PASS issued-at-without-anchor-does-not-prove-precedence -> REPRODUCIBLE
PASS declared-anchor-is-not-treated-as-verified -> REPRODUCIBLE
PASS malformed-hash-fails-closed -> INVALID

Guardrail provenance conformance passed: 10

Run with:

python3 scripts/check_guardrail_provenance_vectors.py \
  conformance/guardrail-decision-v1/manifest.json

Scope

This PR does not define the full GuardrailDecision schema, policy semantics, signatures, or an OTS/RFC 3161 verifier. It deliberately reports an external anchor reference as unverified until a profile-specific verifier actually checks the proof.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 25ce1d03-3734-4d83-9813-6f5c5368c336

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/guardrail-provenance-vectors

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant