Skip to content

Test coverage: complex verification functions are under-tested (cargo-crap audit) #127

Description

@avrabe

Context

A one-shot cargo-crap run (CRAP = Change Risk Anti-Patterns: cyclomatic complexity weighted by test coverage) over the wsc library surfaced a clear pattern: sigil's most complex verification functions are among its least unit-tested code — the exact risk quadrant for a signing/verification tool.

Run: cargo llvm-cov -p wsc --lib --lcovcargo crap --lcov lcov.info → 72/1251 functions over CRAP threshold 30.

Punch list — verification functions to add unit coverage for

Ordered by CRAP score; coverage % is from the --lib unit-test run (integration tests in tests/ were not included, so true coverage for some is higher — but "a complex verifier with no unit test" still stands):

CC unit cov function location
29 10.1% verify_attestation_signature composition/mod.rs:1942
24 0.0% PublicKey::verify_multi signature/multi.rs:211
21 0.0% AirGappedVerifier::verify_signature airgapped/verifier.rs:198
35 32.9% verify_attestation_recursive composition/mod.rs:2081
16 14.1% KeylessVerifier::verify signature/keyless/signer.rs:547
14 6.4% RekorKeyring::verify_inclusion_proof signature/keyless/rekor_verifier.rs:724
11 0.0% KeylessSigner::sign_module signature/keyless/signer.rs:261
8 0.0% SctVerifier::verify_embedded_scts sct.rs:289

Separate finding — complexity, not coverage

cli/main.rs::start has cyclomatic complexity 152 (and handle_bundle_command CC 63). These are CLI dispatch functions; the CRAP score is inflated because --lib coverage doesn't reach the CLI, but the raw complexity is a real decomposition smell worth a separate look.

Caveats / method notes

  • The CRAP score is only as accurate as the coverage report fed to it. This run used --lib-scoped coverage, so CLI/example functions show as catastrophic CRAP purely from missing coverage data — discount those.
  • A more rigorous follow-up is planned: re-evaluating coverage with MC/DC-style branch coverage (pulseengine/witness) against the WASM-component build, which is the coverage criterion aligned with sigil's DO-178C/DO-333 ambitions. cargo-crap is a heuristic prioritiser; MC/DC is certification-grade evidence.

Suggested approach

Work the punch list top-down — each entry is a focused unit-test addition. verify_attestation_signature, PublicKey::verify_multi, and AirGappedVerifier::verify_signature are the highest-value first targets.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions