[REVIEW] iam-review: add token revocation, device-code, and MFA fatigue evidence gates

[wowsofine]

Skill Being Reviewed

Skill name: iam-review
Skill path: skills/identity/iam-review/

False Positive Analysis

Benign identity posture that should be credited as stronger than generic MFA/CAE:

provider: Entra ID
human_users:
  mfa_required: true
  phishing_resistant_for_admins: true
  number_matching_required: true
  additional_context_in_push: true
legacy_authentication:
  pop_imap_smtp_basic_auth: blocked
session_controls:
  sign_in_frequency_hours: 8
  persistent_browser_session: disabled_for_admins
  continuous_access_evaluation: enabled
risk_response:
  sign_in_risk_policy: enabled
  user_risk_policy: enabled
  token_revocation_on_password_reset: verified
attack_scenarios_tested:
  - device_code_phishing_blocked_by_conditional_access
  - refresh_token_revoked_after_user_disable
  - impossible_travel_requires_step_up
logs:
  sign_in_logs_include_authentication_requirement: true
  audit_logs_include_revoke_sessions_event: true

Why this is a false positive:
The current skill can flag broad MFA and zero-trust gaps, but it does not distinguish a tenant that only has generic MFA from one that has tested token revocation, device-code controls, sign-in risk policies, number matching, and CAE behavior. Without those evidence fields, a review can under-credit a strong implementation or over-credit a weak one based only on "MFA enabled" and "CAE mentioned".

Coverage Gaps

Missed variant 1: Device-code phishing succeeds despite MFA coverage.

authentication:
  mfa_required: true
  default_method: push_notification
conditional_access:
  device_code_flow: allowed
  require_compliant_device_for_device_code: false
  sign_in_risk_policy: report_only
user_experience:
  number_matching: disabled
  additional_context: disabled
observed_attack:
  attacker_initiates_device_code: true
  user_approves_push: true
  token_issued_to_attacker: true

Why it should be caught:
IAM-AUTH-01 through IAM-AUTH-06 cover MFA presence, phishing-resistant authenticators, bypasses, and recovery flows, but they do not require evidence for device-code flow restrictions, number matching, additional context, or sign-in risk enforcement. A tenant can satisfy generic MFA while remaining exposed to device-code phishing and MFA fatigue.

Missed variant 2: Refresh tokens remain valid after disablement, password reset, or risk change.

user: contractor@example.com
status: disabled
password_reset: completed
risk_state: high
continuous_access_evaluation: enabled_in_policy
validation:
  revoke_sessions_event: missing
  refresh_token_replay_after_disable: succeeds
  sign_in_frequency_policy: 90_days
  resource_app_honors_cae: unknown

Why it should be caught:
IAM-ZT-04 and IAM-ZT-05 mention excessive session lifetime and no CAE, but the skill does not ask for proof that refresh tokens are revoked on critical events or that relying applications honor revocation. CAE on paper is weaker than tested revocation evidence.

Missed variant 3: Risk-based access is report-only or partially scoped.

conditional_access:
  sign_in_risk_policy:
    state: report_only
    included_users: admins_only
  user_risk_policy:
    state: disabled
  workload_identity_risk: not_evaluated
logs:
  risky_users_reviewed: false
  risky_sign_ins_reviewed: false
review_output:
  risk_based_authentication: enabled

Why it should be caught:
IAM-ZT-06 asks whether risk-based or adaptive authentication exists, but it does not require enforcement mode, scope denominator, risk signal source, or audit evidence that risky users/sign-ins are reviewed and remediated.

Edge Cases

• CAE is enabled for the tenant but not supported by a legacy or third-party resource app.
• Sign-in frequency is strict for browser sessions but refresh-token lifetime remains long for mobile/desktop clients.
• Number matching is enabled for some push methods but admins can still use SMS, voice, or legacy OTP.
• Risk policies apply only to internal users while external guests, service principals, or break-glass accounts are excluded.
• Device-code flow is needed for CLI tooling, but no approved device-code client list or monitoring exists.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found: Add a token/session assurance gate to the authentication and zero-trust sections rather than treating MFA, Conditional Access, and CAE as single binary controls.

Recommended additions:

1. Add findings such as IAM-SESSION-01 through IAM-SESSION-08 for unrestricted device-code flow, missing number matching/additional context, report-only sign-in risk policy, missing refresh-token revocation evidence, unsupported CAE resource apps, excessive sign-in frequency, risky-user remediation gaps, and legacy-auth exclusions.
2. Add output fields for device_code_policy, mfa_push_protection, risk_policy_mode, refresh_token_revocation_test, cae_resource_coverage, sign_in_frequency, legacy_auth_status, and risky_sign_in_review_evidence.
3. Require one vulnerable fixture for device-code phishing with generic MFA and one benign fixture for enforced number matching plus tested refresh-token revocation.
4. Make severity depend on privilege level, token replay success, legacy auth exposure, and whether the affected resource app honors CAE.

Comparison to Other Tools

Tool	Catches this?	Notes
Semgrep	No	This is identity control-plane behavior, not source-code pattern matching.
CodeQL	No	CodeQL does not evaluate IdP session controls or token revocation behavior.
Entra ID Conditional Access / Sign-in logs	Partial	Can show policy results and risk events, but the skill should require review output that records enforcement mode, scope, and validation.
Cloud IAM analyzers	Partial	They help with policy structure, but refresh-token and device-code attack resistance needs IdP/session evidence.

Overall Assessment

Strengths:
The skill already has useful checks for MFA coverage, phishing-resistant authenticators, recovery-flow bypasses, session lifetime, CAE, and risk-based authentication.

Needs improvement:
The current checklist treats several session controls as high-level yes/no items. It does not require attack-path evidence for device-code phishing, MFA fatigue, refresh-token replay after risk change, or report-only risk policies.

Priority recommendations:

1. Add a token/session assurance gate covering device-code flow, push-MFA protections, risk policy enforcement mode, and refresh-token revocation tests.
2. Add platform evidence fields for Entra ID, AWS IAM Identity Center, and Google Workspace / Cloud Identity where equivalent logs and controls exist.
3. Add fixtures that separate generic MFA from tested phishing-resistant, revocation-aware session control.

Sources Checked

• Current skill reviewed locally: skills/identity/iam-review/SKILL.md
• Existing related issues checked in local cache: [REVIEW] iam-review: add recovery-flow and MFA exception evidence gates #894 recovery-flow and MFA exception gates, [REVIEW] iam-review: add external trust and session-constraint evidence gates #1426 external trust/session-constraint gates, [REVIEW] iam-review: add workload identity federation trust-claim gates #685 workload identity federation trust-claim gates, [REVIEW] iam-review: refresh NIST SP 800-63B-4 authenticator evidence #195 authenticator evidence, [REVIEW] iam-review: add effective privilege evidence matrix #67 effective privilege evidence matrix.
• This review is distinct because it focuses on token/session behavior after authentication: device-code flow, MFA fatigue controls, refresh-token revocation, sign-in risk enforcement mode, and CAE resource coverage.

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms
• Preferred payment method: I can provide payment details privately after maintainer acceptance.

[lcwLcw123]

/attempt #1522

I am taking a focused implementation pass on skills/identity/iam-review/ for token/session assurance: device-code phishing, MFA push fatigue protections, risk-policy enforcement mode, refresh-token revocation tests, CAE resource coverage, sign-in frequency, and risky sign-in review evidence.

Planned scope:

• add IAM-SESSION findings for unrestricted device-code flow, missing number matching/additional context, report-only risk policies, missing refresh-token revocation evidence, unsupported CAE resource apps, excessive sign-in frequency, risky-user remediation gaps, and legacy-auth exclusions;
• extend output fields for device-code policy, MFA push protection, risk policy mode, revocation test evidence, CAE app coverage, sign-in frequency, legacy auth status, and risky sign-in review evidence;
• add benign and vulnerable fixtures;
• validate with marker checks, formatting checks, frontmatter/fence checks, ASCII/prompt-injection scans, fixture parsing, and official reference checks.

Bounty target: Improver Moderate ($100) if accepted. Preferred payment method: GitHub Sponsors.