feat(inspect): converge on a repository-first golden path (#32)

[trionnemesis]

What changed

An engineer can open a local repository and get a ranked list of its AI-agent attack surface — agents, skills, MCP servers, hooks, tool grants, memory/RAG — with, per risk, whether anything here can turn it into a deterministic verdict. Before this, project/discovery.py produced a careful surface inventory that nothing consumed, and the entry point (agentsec run) could not be reached without a configured target.

agentsec init → agentsec scan → agentsec scan --verify -t <id> → dashboard

agentsec scan needs a checkout and nothing else. Converges #32.

Type

• New scenario (Attack–Detection Contract)
• Evidence collector
• Executor
• Evaluator / verdict logic
• MCP gateway surface
• Docs / ADR
• Fix

Purple coverage

Question	Answer
Which scenario(s) cover this change?	None directly — this plane routes to scenarios rather than being one. AGT-CONFIG-003 is retagged (see below), which changes which risks and static findings it correlates with.
Does it change how a verdict is produced?	No. The four axes, precedence and evaluator are untouched. repo_risk is composed beside purple and never merged; a risk is never a PurpleVerdict. tests/test_inspect.py asserts the verdict vocabulary never appears in the plane, and that risk counts never reach purple.
Does it widen what the MCP gateway can reach?	One read-only resource, agentsec://project/risks, taking no arguments — no path, no locator. It reads surfaces already resolved through project/resolver.py at the process boundary. No new tool; no write.

Why repository-first rather than the agentsec check façade #32 proposed

Writing that façade surfaced the problem underneath it. agentsec check --target order-agent-staging needs the target to exist: an allowlist entry, a staging agent, usually a Wazuh or OTel backend. That is a security or platform team's work, so the engineer #32 names as the primary user cannot reach the first command of the golden path without someone else's sprint.

It is also backwards on its own terms — configuring a target is a real cost, and nothing told anyone why it was worth paying for their repository. scan answers exactly that, then hands the provable subset to the harness. The CI gate is still the destination; it is now the second step. Full reasoning in ADR 0009.

The plane: a risk is a reason to test, not a result

Ten deterministic rules in inspect/rules.py — no model, for the reason ADR 0002 gives one level downstream: a risk plane whose output changed between two runs of the same commit could not be diffed or argued with.

Each risk resolves to one of three states, reusing the config-surface: tag convention from #25 (extracted into scenario/surface_tags.py so the risk and posture planes cannot answer the same question differently):

State	Meaning
verified	A covering scenario produced a verdict — the answer is in purple
verifiable	One exists, has not run. scan --verify drains this queue
not_verifiable	Nothing in the catalogue exercises this surface

not_verifiable is the default and the common case: neither a pass nor a failure, but AgentSec saying it found something it cannot settle. scan exits 0 even with critical risks outstanding — a gate that blocks on a static match teaches its team to bypass the gate.

Rules never carry file content. Evidence is counts, line numbers, Unicode codepoint names and the rule's own marker vocabulary. project/discovery.py earns publication without a second redaction pass by not reading values; a plane that quoted the offending line would spend that on the way out. Pinned by a test that plants known strings and asserts none reach the serialised output.

Two surfaces that did not exist

• tool_grants — one entry per settings.json permission rule, not a count. A repository can hold no skills, no agents and no hooks and still hand a model unattended shell access in four words of JSON; "is this configured" is not the question, which tool under which constraint is.
• memory — a declared manifest location defaulting to .claude/memory. Absent in most repositories, which inventories as empty rather than as safe.

Two fixes the plane exposed by being run

• AGT-CONFIG-003 retagged from .claude/hooks/guard_agentsec.py to .claude/hooks. Nothing in that scenario is specific to one hook — it seeds an untrusted filename and asserts on run_shell_hook — so the narrow tag made it correlate with this repository and with nothing in anyone else's. With it fixed, a critical hook-injection risk in an arbitrary repo becomes verifiable instead of unprovable.
• Hook rules strip comments before matching. The first run against this repository reported network egress from guard_agentsec.py, on the strength of a comment explaining what a proxied curl would do. A rule that reports the documentation of a risk as the risk teaches its reader to skip the plane. Fixed, and pinned as a regression test.

Gaps this exposes, stated rather than hidden

Recorded in docs/feature-matrix.md and promoted to open Core items on the roadmap:

1. No scenario covers the tool-grant / settings surface. ASI-TOOL-PERMISSION-BYPASS fires at critical and reports not_verifiable. The highest-value gap the plane found; it needs a fixture, which the guard hook correctly refuses to let an agent write.
2. No scenario covers the memory surface. AGT-XPIA-001 is the right shape but carries no config-surface: tag.
3. AGT-CONFIG-* still has no recorded fixtures, so --verify against demo-agent-fixture (environment local) refuses with exit 2 rather than selecting nothing and reporting success — verified by hand.

Docs

• docs/feature-matrix.md — every capability classified Core / Supporting / Experimental / Parked against the one path, which is the "收斂功能矩陣" deliverable.
• docs/adr/0009-repository-first-golden-path.md — the decision, four rejected alternatives, five accepted costs.
• docs/roadmap.md — re-sorted by layer rather than completion status, as product: 收斂 AgentSec 的工程師使用路徑與功能邊界 #32 asked.
• READMEs (en + zh-TW), docs/deployment.md, packaging/claude-desktop/README.md realigned; resource counts corrected to 10 total / 7 published.

Checks

• make check passes — ruff clean, mypy clean on 63 files, 379 tests (345 before + 34 new)
• agentsec validate --strict passes on all 8 scenarios
• agentsec run --target demo-agent-fixture --profile nightly still exits 1 with exactly AGT-TENANT-001 and AGT-MEMPOIS-001 blocking

If this touches the MCP surface

• The capability exists on HarnessService first (inspect_repository), so CLI and CI reach it too
• Input schema is closed — the resource takes no parameters at all, so no locator, credential or free-text command is expressible
• tests/test_mcp_contract.py still passes; test_publish_redaction.py updated to include the new URI in the published allowlist
• The resource has a publication policy (repo_risk_document); the gateway refuses to boot without one

Notes for the reviewer

Three decisions worth challenging:

1. PUBLISH_SCHEMA_VERSION 1.2.0 → 1.3.0, not 2.0.0. repo_risk is required on the composed dashboard, so a consumer validating strictly against the old schema sees a new key. I judged that minor because every shape a consumer was already reading is untouched — but the same argument was made for 1.2.0 and the required-key question will recur.
2. scan exits 0 with critical risks outstanding. Deliberate, and the most likely thing someone will want to change. If it ever exits 1, the plane stops being "a reason to test" and becomes a second gate with no evidence behind it.
3. Rule severities are fixed per rule, not configurable. No --fail-on threshold, no severity overrides. That is one fewer knob, and also one fewer way to make the plane say what someone wanted it to say.

Also worth a look: _SECRET_MARKERS / _SINK_MARKERS / _IMPERATIVE in inspect/rules.py. ASI-INSTR-EXFIL-DIRECTIVE requires all three within a 3-line window specifically so security documentation does not trip it, and there is a test asserting prose about exfiltration does not fire the exfiltration rule. That balance is tuned, not derived.

---

Generated by Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1bf51cbddb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Scope the verification rollup to the requested target

When a covering scenario already has a verdict for target A, agentsec scan --verify --target B uses this unfiltered rollup, marks the risk as verified, removes the scenario from verify_queue, and exits without testing target B. Because verdicts are target-specific, pass the requested target into repository inspection and filter this rollup before deciding which scenarios still need to run.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Bound and report truncated inspection reads

For an instruction or hook larger than 512 KiB, read_bytes() first loads the entire file and then silently discards everything after the limit. A malicious directive placed in the discarded tail therefore produces neither a risk nor a truncation problem while the report still says inspected, and a sufficiently large file can also exhaust memory before the slice is applied. Read a bounded amount from the file and record when additional bytes were omitted.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Handle memory symlinks whose targets leave the memory directory

If .claude/memory contains a symlink to another file inside the repository but outside the declared memory directory, files_under() accepts it and returns the resolved target path, so this relative_to(base) call raises ValueError. That exception is not converted to an inspection problem and causes the entire scan to fail; retain the symlink-relative path or explicitly reject targets outside the declared memory directory.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Include the surface identity in each risk ID

When the same rule fires for multiple surfaces stored in one file—for example, two credential-bearing MCP servers in .mcp.json or multiple broad tool grants in settings.json—every result receives the same rule_id:file ID. Dashboard consumers that key or diff rows by the documented stable ID can therefore overwrite distinct risks or treat them as one; include surface_id (or another per-surface discriminator) in the ID.

Useful? React with 👍 / 👎.