Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,11 +160,11 @@ There are no conditional edges: after `resolve_input` → `build_context`, all a
| `pattern_defaults.py` | Shared pattern metadata (category, explanation, remediation) |
| `static_yara.py` | YARA-based static analyzer |
| `osv_client.py` | OSV.dev API client for live vulnerability lookups (SC4); batch queries with caching and fallback |
| `static_patterns_*.py` | 13 pattern-based analyzers (prompt_injection, data_exfiltration, anti_refusal, etc.) |
| `static_patterns_*.py` | 14 pattern-based analyzers (prompt_injection, data_exfiltration, anti_refusal, etc.) |
| `behavioral_ast.py` | AST-based behavioral analyzer (AST1–AST8): detects exec, eval, subprocess, os.system, compile, dynamic import/getattr, and dangerous execution chains |
| `behavioral_taint_tracking.py` | Taint-tracking behavioral analyzer (TT1–TT5): source→sink data-flow analysis over Python AST |
| `mcp_least_privilege.py`, `mcp_tool_poisoning.py` | MCP analyzers (LP1–LP4 least-privilege; TP1–TP4 tool poisoning) |
| `mcp_rug_pull.py` | MCP rug-pull analyzer (stub; RP1–RP3 planned) |
| `mcp_rug_pull.py` | MCP rug-pull analyzer (RP1–RP3): detects manifest/tool-definition changes between scans |
| `semantic_security_discovery.py`, `semantic_developer_intent.py`, `semantic_quality_policy.py` | Semantic (LLM) analyzers; emit findings only when `use_llm` is enabled |

---
Expand Down Expand Up @@ -251,9 +251,9 @@ Use [static_runner.run_static_patterns](../src/skillspector/nodes/analyzers/stat

Use [pattern_defaults](../src/skillspector/nodes/analyzers/pattern_defaults.py) for category and remediation. Examples: [static_patterns_prompt_injection.py](../src/skillspector/nodes/analyzers/static_patterns_prompt_injection.py), [static_patterns_data_exfiltration.py](../src/skillspector/nodes/analyzers/static_patterns_data_exfiltration.py).

### Stub analyzers
### Placeholder analyzers

Return `{"findings": []}`. Most analyzer nodes are implemented; `mcp_rug_pull` remains a stub (returns no findings) pending rug-pull detection (RP1–RP3). Use this pattern for any new placeholder analyzer. The LLM-backed semantic analyzers also return `{"findings": []}` when `use_llm` is False.
Return `{"findings": []}`. All analyzer nodes are currently implemented; use this pattern for any new placeholder analyzer added before its detection logic lands. The LLM-backed semantic analyzers also return `{"findings": []}` when `use_llm` is False.

---

Expand Down
14 changes: 7 additions & 7 deletions docs/LLM_ANALYZER_BASE_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,15 +347,15 @@ evaluates *existing* static findings rather than discovering new ones:
- Overrides `parse_response` to return dicts (not `Finding` objects)
- Adds `apply_filter` to match LLM results back to originals by `(file, rule_id)`

### Semantic Analyzer Stubs
### Semantic Analyzers

These are ready to be implemented using `LLMAnalyzerBase`:
These are implemented on top of `LLMAnalyzerBase` and emit findings only when `use_llm` is enabled:

| Stub | SADD Reference | Purpose |
|------|---------------|---------|
| `semantic_security_discovery` | B.4.1 | Intent and attack-phrasing risks |
| `semantic_developer_intent` | B.4.2 | Description-behavior mismatch |
| `semantic_quality_policy` | B.4.3 | Quality/safety rubric violations |
| Analyzer | Purpose |
|----------|---------|
| `semantic_security_discovery` | Intent and attack-phrasing risks |
| `semantic_developer_intent` | Description-behavior mismatch |
| `semantic_quality_policy` | Quality/safety rubric violations |

---

Expand Down
2 changes: 1 addition & 1 deletion src/skillspector/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ def __str__(self) -> str:


class AnalyzerPlugin(Protocol):
"""Analyzer protocol from SADD A.1.1."""
"""Analyzer plugin protocol: name/stage/availability and an ``analyze`` entry point."""

name: str
stage: str
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

"""Behavioral taint-tracking analyzer (SADD B.2.2): sources -> sinks data-flow analysis.
"""Behavioral taint-tracking analyzer (TT1–TT5): sources -> sinks data-flow analysis.

Parses Python AST to identify data sources (env vars, file reads, network input)
and sinks (network output, exec, file writes), then tracks flows between them
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

"""Semantic developer-intent analyzer node (SADD B.4.2).
"""Semantic developer-intent analyzer node.

Detects context-dependent risk and semantic description–behavior mismatches
by comparing the skill's manifest (name, description, permissions) against
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

"""Semantic quality-policy analyzer node (SADD B.4.3).
"""Semantic quality-policy analyzer node.

Evaluates AI agent skill files against a quality and safety rubric using
LLM-based discovery. Flags vague triggers, missing user warnings, and
Expand Down
2 changes: 1 addition & 1 deletion tests/nodes/analyzers/test_behavioral_taint_tracking.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

"""Tests for behavioral_taint_tracking analyzer (SADD B.2.2): source→sink data-flow."""
"""Tests for behavioral_taint_tracking analyzer (TT1–TT5): source→sink data-flow."""

from __future__ import annotations

Expand Down
10 changes: 5 additions & 5 deletions tests/nodes/analyzers/test_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.

"""Tests for analyzer node registry alignment with SADD spec."""
"""Tests for analyzer node registry alignment with the workflow reference table."""

from __future__ import annotations

from skillspector.nodes.analyzers import ANALYZER_NODE_IDS, ANALYZER_NODES

# Expected analyzer node IDs per SADD spec workflow reference table.
# Expected analyzer node IDs per the workflow reference table.
# Order: static (14), behavioral (2), mcp (3), semantic (3).
EXPECTED_ANALYZER_NODE_IDS: list[str] = [
"static_patterns_prompt_injection",
Expand Down Expand Up @@ -49,10 +49,10 @@


class TestAnalyzerRegistry:
"""Registry matches SADD spec node set and order."""
"""Registry matches the expected node set and order."""

def test_analyzer_node_ids_match_sadd_spec(self):
"""ANALYZER_NODE_IDS equals expected list from SADD spec."""
def test_analyzer_node_ids_match_expected(self):
"""ANALYZER_NODE_IDS equals the expected list."""
assert ANALYZER_NODE_IDS == EXPECTED_ANALYZER_NODE_IDS

def test_analyzer_nodes_has_entry_for_every_id(self):
Expand Down