feat: Add adoption-step bridge and agent-ops guardrail signals to /assess

[bjcoombs]

Summary

Incorporates the two repo-observable takeaways from Boris Cherny's Steps of AI Adoption (July 2026) into /assess:

1. Adoption-step bridge: the report's scorecard fold gains a "What this score unlocks" section mapping the maturity band onto the adoption ladder (Not Ready → Step 1 Assisted, Solid → Step 2 Parallel, AI-Native → Step 3 Supervised autonomy). Framed explicitly as a bound on safe agent autonomy, never a claim about how the team actually operates.
2. Agent-operations guardrail signals: a new deterministic lib/agent_ops.py scan surfaces encoded operational guardrails - .claude/settings.json / .claude/settings.local.json permission allow/deny/ask counts, hook events, sandbox config, .claude/hooks/ scripts, and routine definitions under .claude/workflows/ / .claude/routines/ - as Layer 8 workflow-maturity evidence.

Cherny's per-step guardrails map directly onto existing layers (self-verification loops = L5/L6, automated review = L7, agent instructions = L0, routines/permissions = L8), which is what makes the bridge sound. The org-side content of the ladder (procurement, spend caps, agent-count telemetry) is deliberately out of scope - not repo-observable.

Changes Made

• skills/assess/scripts/lib/agent_ops.py (new): pure-stdlib scan; summary booleans credit git-tracked evidence only, mirroring the Layer 0 rule. Deliberately excludes .claude/agents/ and .claude/skills/ (Layer 0 evidence) to prevent double-counting.
• skills/assess/scripts/assess_core.py: wires agent_ops into run-context.json via _safe.
• agents/assess-layer-scorer.md: Layer 8 reads agent_ops; each true summary boolean strengthens the workflow-maturity dimension, absence never lowers a verdict, and agent-ops evidence is never sufficient alone.
• skills/assess-findings/SKILL.md: the bridge table in the scorecard fold, with the fraction-based band rule reused for non-software archetypes.
• README.md: single attribution to Cherny's post (primary source, his Threads announcement linking the artifact), following the existing Karpathy single-attribution convention.
• skills/assess/scripts/lib/README.md: agent_ops.py module entry.
• .claude-plugin/plugin.json: 1.55.31 → 1.56.0 (MINOR - new feature).

Testing

• skills/assess pytest: 1031 passed (7 new in tests/test_agent_ops.py: tracked/untracked credit, malformed JSON degrade, non-git dir, hooks dir, routine dirs, empty repo), 2 skipped.
• scripts/ pytest: 152 passed. Plugin contract pytest: 539 passed.
• Ruff + mypy gates: clean.
• End-to-end: self-run of assess_core.py produces the agent_ops block (this repo has no .claude/ dir, so it reads all-false - the honest negative); .assess/ self-run churn reverted, not committed.

Risk Assessment

Low-to-medium. The scan is additive read-side context behind _safe (a failure degrades to available: false, never blocks a run). The report/rubric changes alter LLM-rendered prose only; scoring semantics change in one place (Layer 8 may score higher on repos with committed agent-ops config - lift-only by design).

Deployment Notes

Standard plugin release flow; version bump included. No migration or config changes for existing users.

Summary by CodeRabbit

• New Features

  • Assessment reports now include evidence of AI-operation guardrails, including permissions, hooks, workflows, and routines.
  • Maturity scores explain the level of AI autonomy and safeguards supported by the repository.
  • Only committed repository evidence contributes to guardrail scoring.

• Documentation

  • Updated assessment guidance and score explanations with maturity-band mappings to AI adoption stages.
  • Added documentation for the new operational guardrail evidence.

• Improvements

  • Malformed or unavailable configuration is handled gracefully without interrupting assessments.
  • Updated the plugin version to 1.56.0.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7403259c-0303-4ad0-85da-006924324646

📥 Commits

Reviewing files that changed from the base of the PR and between d9fc537 and 681992f.

📒 Files selected for processing (8)

• .claude-plugin/plugin.json
• README.md
• agents/assess-layer-scorer.md
• skills/assess-findings/SKILL.md
• skills/assess/scripts/assess_core.py
• skills/assess/scripts/lib/README.md
• skills/assess/scripts/lib/agent_ops.py
• skills/assess/tests/test_agent_ops.py

---

📝 Walkthrough

Walkthrough

The assessment gains a tracked agent-operations scanner for Claude settings, hooks, workflows, and routines. Its evidence is added to run context and Layer 8 scoring, with corresponding tests, documentation, maturity guidance, and a plugin version bump.

Changes

Agent operations assessment

Layer / File(s)	Summary
Agent operations scanner and validation skills/assess/scripts/lib/agent_ops.py, skills/assess/tests/test_agent_ops.py, skills/assess/scripts/lib/README.md	Adds defensive scanning of settings and operation directories, credits only tracked evidence, and tests parsing, tracking, hooks, routines, and failure cases.
Assessment context and Layer 8 scoring skills/assess/scripts/assess_core.py, agents/assess-layer-scorer.md	Adds agent_ops to run context and uses its summary booleans as supporting Layer 8 workflow-maturity evidence.
Maturity bounds and release metadata skills/assess-findings/SKILL.md, README.md, .claude-plugin/plugin.json	Documents maturity-band autonomy bounds and adoption-ladder mapping, and bumps the plugin version to 1.56.0.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Assessment
  participant AgentOpsScanner
  participant RunContext
  participant Layer8Scorer

  Assessment->>AgentOpsScanner: scan repository guardrails
  AgentOpsScanner-->>RunContext: add agent_ops evidence
  Layer8Scorer->>RunContext: read agent_ops summary
  Layer8Scorer-->>Assessment: calculate workflow maturity

Loading

Possibly related PRs

• bjcoombs/ai-native-toolkit#108: Introduced the Layer 8 scoring specification extended by this change.

Suggested labels: feat

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 35.71% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the two main changes: the adoption-step bridge and agent-ops guardrail signals for /assess.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cherny-adoption-bridge

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Checkov (3.3.8)

.claude-plugin/plugin.json

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[claude]

Claude Code Review approved. Detailed summary follows as a PR comment.

[claude]

Claude Code Review

Commit: 681992f | CI: required checks passing (skills/assess pytest, scripts/ pytest, plugin contract pytest, Validate PR title all green; claude-review + CodeRabbit still running, both non-blocking)

Summary

Clean, well-scoped MINOR feature. Two additions land against the repo own standard:

• lib/agent_ops.py stays inside the deterministic-core boundary: pure stdlib + git_churn.tracked_files, no AI, no network. scan_agent_ops never raises and is wired behind _safe in assess_core.py, so a failure degrades to available: false rather than blocking a run. The tracked-only credit rule mirrors Layer 0 correctly (untracked settings reported with tracked: false, never credited), and .claude/agents/ / .claude/skills/ are deliberately excluded to avoid double-counting Layer 0 evidence.
• Matching test (tests/test_agent_ops.py, 7 cases) covers the load-bearing paths: tracked vs untracked credit, malformed-JSON degrade, non-git dir, hooks dir, routine dirs, empty repo. That is the contract the deterministic core needs.

Contract checks all clean: version bumped 1.55.31 to 1.56.0 (MINOR, correct tier for a new feature); no dead links, em dashes, placeholder tokens, or secrets introduced. skills/assess-findings/SKILL.md is vendored into the assess standalone ZIP (as references/assess-findings.md), but the added "What this score unlocks" content is plugin-agnostic (a table plus an external Threads URL, no ARGUMENTS / SKILL_DIR / namespaced command / CC-only tool), so no chat-skip markers are required.

The adoption-ladder bridge is consistently framed as a bound on safe autonomy rather than a claim about team behaviour, and the Cherny-to-layer mapping is advisory prose rendered by the LLM, not scoring logic - appropriate.

Findings

• Note (skills/assess/scripts/lib/agent_ops.py:91): hook_events = len(hooks) counts event keys regardless of whether their command arrays are populated, so a settings file that declares an event name with an empty array (an event declared but no hook wired) still sets hooks_present: true. Your own test_tracked_settings_credit_summary exercises exactly this (empty PreToolUse / PostToolUse arrays give hooks_present True). It is lift-only, advisory, and absence never lowers a verdict, so not blocking - but if you want the signal to mean a hook actually fires, count only events whose value is a non-empty list.

Bot Review Notes

No unresolved CodeRabbit threads at review time (its check is still in progress).

Verdict: APPROVE.