Skip to content

ci: add mutation testing for validation and redaction helpers (#244)#344

Open
aaniya22 wants to merge 3 commits into
utksh1:mainfrom
aaniya22:ci/mutation-testing-244
Open

ci: add mutation testing for validation and redaction helpers (#244)#344
aaniya22 wants to merge 3 commits into
utksh1:mainfrom
aaniya22:ci/mutation-testing-244

Conversation

@aaniya22
Copy link
Copy Markdown
Contributor

What

Adds mutation testing for the two most security-critical helper modules:

  • backend/secuscan/validation.py
  • backend/secuscan/redaction.py

How

  • mutmut.toml — configures which files to mutate and which tests to run
  • .github/workflows/mutation.yml — new targeted CI job that:
    • triggers only when relevant source or test files change (no noise on unrelated PRs)
    • also runs on a weekly schedule (every Monday 03:00 UTC) to catch regressions
    • uploads an HTML report as a downloadable artifact (30-day retention)
    • enforces a hard 80% mutation score threshold — fails CI if too many mutants survive
  • backend/requirements-dev.txt — pins mutmut>=3.5.0

What was NOT changed

  • No UI changes
  • No doc changes
  • No lockfile churn
  • No formatting changes
  • No existing tests modified

Verification

Run locally:

mutmut run
mutmut results
mutmut html

Closes #244

aaniya22 added 3 commits May 27, 2026 03:53
Fixes utksh1#265

- config.py: removed 'secuscan-dev-key' hardcoded fallback in
  resolved_vault_key(). Now raises ValueError at startup if
  SECUSCAN_VAULT_KEY is not explicitly set in environment.

- vault.py: replaced broken XOR stream cipher with AES-256-GCM via
  the cryptography package. XOR with a 32-byte cycling keystream was
  trivially breakable via crib-dragging for secrets > 32 bytes.
  AES-256-GCM provides proper confidentiality and built-in integrity
  verification regardless of secret length.

- requirements.txt: added cryptography>=42.0.0 as explicit dependency.
@utksh1 utksh1 added area:ci CI, tooling, or automation work area:backend Backend API, database, or service work type:testing Testing work category bonus label type:devops DevOps or infrastructure work category bonus label level:advanced 55 pts difficulty label for advanced contributor PRs labels May 27, 2026
Copy link
Copy Markdown
Owner

@utksh1 utksh1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes. mutation-testing, backend-tests, and formatting-hygiene are failing. Please make CI green and avoid bundling unrelated vault/config dependency changes into the mutation testing workflow PR.

@utksh1
Copy link
Copy Markdown
Owner

utksh1 commented May 28, 2026

Thanks for following up. Clarifying the change request so it is actionable:

Why this is blocked:
Requesting changes. mutation-testing, backend-tests, and formatting-hygiene are failing. Please make CI green and avoid bundling unrelated vault/config dependency changes into the mutation testing workflow PR.

What to do next:

  • Fix the specific issues called out above.
  • Push the updated branch and make sure the relevant CI checks pass.
  • Reply here when ready for re-review.

@utksh1
Copy link
Copy Markdown
Owner

utksh1 commented May 29, 2026

Current status: still blocked. The mutation-testing CI branch is conflicting with current main. Please rebase, make sure the mutation job is advisory or clearly documented if required, and keep runtime reasonable for normal PR CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:backend Backend API, database, or service work area:ci CI, tooling, or automation work level:advanced 55 pts difficulty label for advanced contributor PRs type:devops DevOps or infrastructure work category bonus label type:testing Testing work category bonus label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Add mutation testing for validation and security-critical helpers

2 participants