Skip to content

test: add LiveKit Agents drift proof - #48

Merged
MaazAhmed47 merged 3 commits into
mainfrom
codex/livekit-agents-drift-proof
Aug 15, 2026
Merged

test: add LiveKit Agents drift proof#48
MaazAhmed47 merged 3 commits into
mainfrom
codex/livekit-agents-drift-proof

Conversation

@MaazAhmed47

@MaazAhmed47 MaazAhmed47 commented Aug 15, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a fixed-scope, synthetic LiveKit Agents proof for this exact path:

real LiveKit MCPToolset attached to a real Agent
  -> directly invoked discovered FunctionTool wrapper
  -> local legacy-MCP compatibility adapter
  -> Interlock POST /mcp/call
  -> synthetic MCP 2026-07-28 server

The proof records an explicitly approved read-only/internal boundary, performs one allowed call, explicitly refreshes discovery after a same-identity read-plus-export/external mutation, observes Interlock quarantine only the changed tool, and proves the later wrapper call is held with zero upstream execution delta. It also retains review evidence and verifies all currently supported Security Receipt checks.

No Interlock core behavior is changed.

Exact provenance

  • Interlock base SHA: a78d5a0a4557a63f0db71e41a70d451c03bb13bb
  • Integration commit SHA: 3c24b45a614f1c2f4e201b7a4152d6aee30aeb6c
  • Base-is-ancestor verification: true
  • LiveKit Agents source commit: 06c71f9c718e24a151630447755a9fa86851b389
  • LiveKit Agents: 1.6.10
  • MCP SDK: 1.28.1
  • Python: 3.12.9

SHA-bound local verification

At 3c24b45a614f1c2f4e201b7a4152d6aee30aeb6c:

  • Complete proof: passed.
  • proof.json records the pinned base SHA, exact integration SHA, and successful ancestry verification.
  • Explicit approval endpoint and its server-bound audit event were verified before MCPToolset.setup().
  • adapter_enforced_explicit_approval is false.
  • Initial upstream execution delta: 1.
  • Held-call upstream execution delta: 0.
  • Receipt checks: record_found, chain, receipt_match, evidence_digest, and binding all true.
  • Focused LiveKit integration: 22 passed.
  • Gateway/evidence/receipt gate: 125 passed.
  • Full pytest: 1977 passed, 134 skipped, 6 xfailed, 4 warnings.
  • Focused Black check: 5 integration Python files unchanged.
  • Ruff: passed.
  • Targeted mypy: 3 source files, no issues.
  • compileall: passed.
  • git diff --check: passed.
  • Artifact and credential scans: passed.
  • Documented cleanup: passed; committed worktree clean.

Limitations

  • This is a local synthetic proof only.
  • It uses a compatibility adapter; it does not prove native LiveKit/Interlock MCP interoperability.
  • No AgentSession is started and no LLM selects or invokes the tool.
  • The proof harness invokes the discovered wrapper directly.
  • Explicit approval is runner-sequenced and verified, not adapter-enforced; /mcp/tools exposes no distinct explicit-approval marker.
  • Detection requires an explicit successful POST /mcp/discover observation refresh.
  • A failed refresh remains fail-stale at the last successfully observed state.
  • requirements-pinned.txt contains direct dependency pins and is not a resolved transitive lock.
  • This is not a LiveKit endorsement, partnership, customer validation, production canary, native compatibility claim, or production-readiness claim.

CI repair

Two focused follow-up commits repair the old-SHA Backend CI failures without changing Interlock runtime behavior:

  • c93092ed76b506b0cfead88505ab3cc8df6ed2a9 - ci: run LiveKit proof in backend checks
    • Backend checkout now uses fetch-depth: 0, allowing the proof's pinned-base ancestry check to run against the pull-request merge checkout.
    • Backend installs both requirements.txt and examples/livekit_agents/requirements-pinned.txt while retaining the whole-directory python -m pytest tests/ -q -ra gate.
    • Blocking Dependency audit installs and audits both requirement files. The proof pins remain direct pins, not a resolved transitive lock.
  • 3c24b45a614f1c2f4e201b7a4152d6aee30aeb6c - test: remove stochastic payment evidence assertion
    • Removes only the broad 4242 substring scan and the timestamp-stripping helper made unused by that removal.
    • Preserves the deterministic cus_secret, pm_secret, ch_secret, and acct_secret sanitization assertions and the hash-evidence assertion.

New-SHA verification

Clean full-history verification at 3c24b45a614f1c2f4e201b7a4152d6aee30aeb6c used the revised CI install:

  • pip check: no broken requirements.
  • Blocking pip-audit 2.10.1: no known vulnerabilities.
  • Focused LiveKit integration: 22 passed.
  • Complete SHA-bound proof and documented cleanup: passed; all five receipt checks true and held upstream delta zero.
  • Payment sanitization regression: 25/25 consecutive passes; complete payment proof file 3 passed.
  • Gateway/evidence/receipt gate: 125 passed.
  • Full local pytest: 1977 passed, 134 skipped, 6 xfailed, 4 warnings.
  • Complete Black check: 68 Python files unchanged.
  • Ruff: passed.
  • CI mypy: 61 source files, no issues.
  • Proof mypy: 3 source files, no issues.
  • compileall, git diff --check, workflow YAML validation, and secret/artifact scans: passed.
  • Clean verifier and primary worktree: clean after cleanup.

Exact-SHA GitHub checks

GitHub Actions run 31884920835 and all external statuses are bound to 3c24b45a614f1c2f4e201b7a4152d6aee30aeb6c:

  • Backend tests: passed: 1974 passed, 137 skipped, 6 xfailed, 4 warnings.
  • Dependency audit: passed, including the LiveKit proof dependency tree.
  • Secret scan, Dashboard build, Helm chart, Docker image build, Vercel, and Vercel Preview Comments: passed.

@vercel

vercel Bot commented Aug 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
interlock-web Ready Ready Preview Aug 15, 2026 12:33pm

@MaazAhmed47
MaazAhmed47 marked this pull request as ready for review August 15, 2026 15:02
@MaazAhmed47
MaazAhmed47 merged commit a5a05c5 into main Aug 15, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant