Add authentication-review skill#62
Merged
Merged
Conversation
A deep-slice review skill for authentication changes: password storage and policy, recovery/reset, MFA bypass and OTP handling, sessions and cookies, JWT/API-token verification, OAuth 2.0, OIDC, SAML, and LDAP sign-in. Classified against OWASP ASVS 5.0 (V6/V7/V9/V10, with V11/V3 for KDF and cookie findings), grounded in RFC 9700, NIST SP 800-63B, and the OWASP Authentication/Password Storage/Session Management/MFA/ Forgot Password/SAML Security cheat sheets. - security-review 0.3.2: reciprocal cross-reference to the new skill - finding-output.md: classifier rows for authentication-review and the previously missing ci-workflow-review/iac-review entries - eval fixtures: OAuth code flow missing state/PKCE, and a SAML ACS that checks only the response-envelope signature then reads NameID from the raw document - plugin tree regenerated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The plant is the signature-wrapping defect (envelope-only verification plus a raw-document NameID read); the unhardened lxml parse was incidental and raised a CodeQL alert. Harden the parser — XSW is unaffected by it, so the eval is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
IcebergAutoReview
Verdict: request_changes
The change is mechanically sound, but the new security guidance contains a blocking SAML correctness error that would generate false-positive findings for valid implementations.
Findings
src/skilldeck/skills/authentication-review/skill.md:196: The checklist says an unsigned Assertion inside a signed Response is attacker-writable and mandates rejecting unsigned assertions. A correctly verified signature over the enclosing Response also protects its contained Assertion; the essential requirement is to consume claims only from the verified signed subtree. The fixture’s own base implementation follows that model. Revise this guidance to accept either a signed Assertion or a signed Response whose signature reference covers the trusted Assertion, while retaining the signed-subtree/XSW checks. This is consistent with the cited OWASP SAML guidance.
Validation
git diff --check main...HEAD— passed.python scripts/build_plugin.py --check— passed; generated plugin tree is current.- Targeted fixture preparation probe for
authentication-reviewandauthentication-review-saml— passed. - Targeted pytest invocation could not start because
uvattempted to create.venvon the read-only checkout; system Python lacks pytest. Current-head GitHub CI is green as stated.
Residual risks / optional notes
- The paid agent evaluation for both new fixtures remains deferred, so their real detection and false-positive behavior has not been demonstrated.
Automated review by Codex 76eaed20bdbd using IcebergAutoReview.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
authentication-reviewskill (0.1.0): a deep-slice review of authentication changes — password storage/policy and recovery, MFA bypass and OTP handling, sessions & cookies, JWT/API-token verification, OAuth 2.0, OIDC, SAML, and LDAP sign-in — classified against OWASP ASVS 5.0 (V6/V7/V9/V10, with V11/V3 for KDF and cookie-attribute findings) and grounded in RFC 9700, NIST SP 800-63B, and the OWASP Authentication, Password Storage, Session Management, MFA, Forgot Password, and SAML Security cheat sheets (all fetched during authoring).security-review0.3.2: reciprocal cross-reference (it keeps breadth; the new skill takes depth on the auth slice).docs/finding-output.md: classifier rows forauthentication-review, plus the previously missingci-workflow-review/iac-reviewrows.authentication-review(hand-rolled OAuth code flow with nostate/PKCE) andauthentication-review-samlvariant (ACS verifies only the response-envelope signature, then readsNameIDfrom the re-parsed raw document — XSW-shaped).scripts/build_plugin.py.Test plan
uv run --extra dev ruff check . && uv run --extra dev ruff format --check . && uv run --extra dev mypy && uv run --extra dev pytest— 132 passed (structure lint, fixture validation, plugin freshness)skilldeck install authentication-review --agent claude --scope projectsmoke-tested; rendered SKILL.md has correct frontmatter and stamppython evals/run_evals.py) against the two new fixtures — deferred, per evals/README🤖 Generated with Claude Code