Skip to content

Add authentication-review skill#62

Merged
richardmhope merged 3 commits into
mainfrom
add-authentication-review-skill
Jul 21, 2026
Merged

Add authentication-review skill#62
richardmhope merged 3 commits into
mainfrom
add-authentication-review-skill

Conversation

@richardmhope

Copy link
Copy Markdown
Collaborator

Summary

  • New authentication-review skill (0.1.0): a deep-slice review of authentication changes — password storage/policy and recovery, MFA bypass and OTP handling, sessions & cookies, JWT/API-token verification, OAuth 2.0, OIDC, SAML, and LDAP sign-in — classified against OWASP ASVS 5.0 (V6/V7/V9/V10, with V11/V3 for KDF and cookie-attribute findings) and grounded in RFC 9700, NIST SP 800-63B, and the OWASP Authentication, Password Storage, Session Management, MFA, Forgot Password, and SAML Security cheat sheets (all fetched during authoring).
  • security-review 0.3.2: reciprocal cross-reference (it keeps breadth; the new skill takes depth on the auth slice).
  • docs/finding-output.md: classifier rows for authentication-review, plus the previously missing ci-workflow-review/iac-review rows.
  • Eval fixtures: authentication-review (hand-rolled OAuth code flow with no state/PKCE) and authentication-review-saml variant (ACS verifies only the response-envelope signature, then reads NameID from the re-parsed raw document — XSW-shaped).
  • Plugin tree regenerated via scripts/build_plugin.py.

Test plan

  • uv run --extra dev ruff check . && uv run --extra dev ruff format --check . && uv run --extra dev mypy && uv run --extra dev pytest — 132 passed (structure lint, fixture validation, plugin freshness)
  • skilldeck install authentication-review --agent claude --scope project smoke-tested; rendered SKILL.md has correct frontmatter and stamp
  • Manual paid eval run (python evals/run_evals.py) against the two new fixtures — deferred, per evals/README

🤖 Generated with Claude Code

A deep-slice review skill for authentication changes: password storage
and policy, recovery/reset, MFA bypass and OTP handling, sessions and
cookies, JWT/API-token verification, OAuth 2.0, OIDC, SAML, and LDAP
sign-in. Classified against OWASP ASVS 5.0 (V6/V7/V9/V10, with V11/V3
for KDF and cookie findings), grounded in RFC 9700, NIST SP 800-63B,
and the OWASP Authentication/Password Storage/Session Management/MFA/
Forgot Password/SAML Security cheat sheets.

- security-review 0.3.2: reciprocal cross-reference to the new skill
- finding-output.md: classifier rows for authentication-review and the
  previously missing ci-workflow-review/iac-review entries
- eval fixtures: OAuth code flow missing state/PKCE, and a SAML ACS
  that checks only the response-envelope signature then reads NameID
  from the raw document
- plugin tree regenerated

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread evals/fixtures/authentication-review-saml/change/app/saml_acs.py Fixed
richardmhope and others added 2 commits July 21, 2026 22:09
The plant is the signature-wrapping defect (envelope-only verification
plus a raw-document NameID read); the unhardened lxml parse was
incidental and raised a CodeQL alert. Harden the parser — XSW is
unaffected by it, so the eval is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@richardmhope
richardmhope merged commit fa69c80 into main Jul 21, 2026
11 of 12 checks passed
@richardmhope
richardmhope deleted the add-authentication-review-skill branch July 21, 2026 12:12

@icebergai-review-bot icebergai-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IcebergAutoReview

Verdict: request_changes

The change is mechanically sound, but the new security guidance contains a blocking SAML correctness error that would generate false-positive findings for valid implementations.

Findings

  • src/skilldeck/skills/authentication-review/skill.md:196: The checklist says an unsigned Assertion inside a signed Response is attacker-writable and mandates rejecting unsigned assertions. A correctly verified signature over the enclosing Response also protects its contained Assertion; the essential requirement is to consume claims only from the verified signed subtree. The fixture’s own base implementation follows that model. Revise this guidance to accept either a signed Assertion or a signed Response whose signature reference covers the trusted Assertion, while retaining the signed-subtree/XSW checks. This is consistent with the cited OWASP SAML guidance.

Validation

  • git diff --check main...HEAD — passed.
  • python scripts/build_plugin.py --check — passed; generated plugin tree is current.
  • Targeted fixture preparation probe for authentication-review and authentication-review-saml — passed.
  • Targeted pytest invocation could not start because uv attempted to create .venv on the read-only checkout; system Python lacks pytest. Current-head GitHub CI is green as stated.

Residual risks / optional notes

  • The paid agent evaluation for both new fixtures remains deferred, so their real detection and false-positive behavior has not been demonstrated.

Automated review by Codex 76eaed20bdbd using IcebergAutoReview.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants