Skip to content

Harden WPT evidence validation#12

Merged
mertushka merged 2 commits into
mainfrom
ci/harden-wpt-evidence
Jun 13, 2026
Merged

Harden WPT evidence validation#12
mertushka merged 2 commits into
mainfrom
ci/harden-wpt-evidence

Conversation

@mertushka

Copy link
Copy Markdown
Owner

Purpose

Make the selected WPT contract and hosted conformance evidence independently verifiable instead of relying on summary counters or test counts alone.

Changes

  • recompute WPT pass, failure, retry, and identity data from result artifacts
  • reject duplicate identities, malformed results, empty reports, mixed workflow runs, and mismatched GitHub SHA metadata
  • bind selected and executed {file, name} identities to a SHA-256 digest in wpt-manifest.json
  • fail direct and sharded targeted runs when no subtests match while allowing legitimately empty individual shards
  • add runner-boundary tests using a small fake shard runner
  • refresh docs/verification-audit.md with the successful PR ci: shard full conformance tests #11 conformance evidence

Impact

  • Public API and native addon behavior: unchanged
  • WPT selection: unchanged at 620 subtests; the exact identity set is now pinned
  • Browser/WebRTC semantics: unchanged
  • CI: malformed or stale evidence that previously false-greened now fails explicitly

Validation

  • npm run check
  • npm test (51 tests)
  • npm run wpt:selection:check (620 identities, pinned digest)
  • npm run pack:check
  • targeted valid and invalid direct/sharded selector checks
  • synthetic 620-result evidence writer integration check

The full selected WPT suite was not run locally because this change does not alter WebRTC semantics. The Conformance workflow remains the hosted matrix validation path.

@mertushka mertushka marked this pull request as ready for review June 13, 2026 16:29
@mertushka mertushka merged commit 159338d into main Jun 13, 2026
19 checks passed
@mertushka mertushka deleted the ci/harden-wpt-evidence branch June 13, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant