diff --git a/CHANGELOG.md b/CHANGELOG.md index 4b7a576..f13209a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,35 @@ All notable changes to this project will be documented in this file. Format follows [Keep a Changelog](https://keepachangelog.com/). This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.7.2] - 2026-07-18 + +### Fixed + +- **The honesty guard no longer fires on honest text.** `tests/test_honesty.py`'s overclaim pattern matched negated phrasing — `"This is not fully isolated"`, `"Do not treat this as a sandbox"`, and `"It is never sandboxed"` all tripped it. A guard meant to catch overclaiming would have failed CI on wording that states the disclaimer *more* strongly, inverting its purpose. Matching is now negation-aware: candidates are discounted when a negator precedes them **in the same sentence**, so a disclaimer cannot excuse a genuine claim in the next sentence. +- **Guard failures now name the offending text.** The pattern used capturing groups with `re.findall()`, so a failure reported tuples of fragments (`[('', '', '')]` for the `fully isolated` branch) instead of the matched phrase. Groups are now non-capturing and matching uses `finditer()` + `group(0)`. Also anchored `sandbox\b` so a match inside `sandboxed` reports `"sandboxed"` rather than the truncated `"is sandbox"`. +- Added guard-the-guard coverage in both directions — 8 honest phrasings that must not trip it, 5 affirmative claims that must, plus sentence-boundary and failure-message regressions (23 new cases). Both findings raised by Qodo on PR #2. + +## [0.7.1] - 2026-07-18 + +### Fixed + +- **`test_learn_text` no longer asserts a vestigial substring.** It checked `"shell-cli" in out`, which after the executable rename passes only incidentally via the GitHub issue URL in the `learn` body — the assertion would have survived the entire command map being deleted. It now asserts `"shell whoami"`, tying the test to the command map it is meant to cover. Found by a `colleague` review run (see agentculture/colleague#353). + +## [0.7.0] - 2026-07-18 + +### Added + +- **`shell explain safety`** — the guard-not-a-sandbox threat model is now readable in-band, so an agent can query the posture before deciding what to trust rather than inferring it from the package name. +- **`docs/threat-model.md`** — assets and actors, a protected-vs-not table, the known bypasses (`sh -c`, shell expansion, here-docs, interpreters, symlink/TOCTOU), and what real isolation would actually require. Documents the contract the pending colleague extraction must uphold; the posture is inherited with the code and must not be upgraded in transit. +- **`tests/test_honesty.py`** — a guard test making the safety disclaimer load-bearing rather than droppable prose. Asserts the posture is present in `learn` (text and the new `safety_posture` JSON field), the `explain` root, `explain safety`, `README.md`, `CLAUDE.md`, and `docs/threat-model.md`, and that no shipped surface makes a positive isolation claim. +- **`CLAUDE.md`** — expanded from the self-init seed into a full runtime prompt (issue #1): the four non-negotiable constraints, the CLI contracts already enforced by tests, the verified extraction map into `colleague` 1.51.0 (handlers, safety helpers, and `policy.py` symbols with line numbers), the decoupling work that is the real difficulty, the planned VM/in-container backend, and the parked open questions. + +### Changed + +- **The CLI now names its own executable correctly.** argparse's `prog` was `shell-cli` while `[project.scripts]` installs `shell`, so `--help`, every `explain` body, and the `learn` text printed commands that do not exist (`uv run shell-cli whoami` failed outright). `prog`, all usage examples, the `cli overview` subject, and the `doctor` status line now say `shell`; `shell-cli` is retained only where it is genuinely the repo/PyPI distribution token. Both `explain shell` and `explain shell-cli` resolve to the root entry. +- **CLI self-description now describes this agent, not the template it was cloned from.** `learn`, `explain`, and `overview` called shell-cli "a clonable template for AgentCulture mesh agents" and listed template-onboarding artifacts. They now state the actual mission (the file-and-shell tool surface), the safety posture, and an honest Status section recording that the six primitives, path confinement, and approval policy are not extracted yet. `overview` gains Mission and Status sections; `learn --json` gains `tool`/`distribution`/`safety_posture`/`status` fields. +- **`README.md`** — rewritten around the mission, with the guard-not-a-sandbox warning promoted above the fold per the build brief, an explicit Status section, the two surfaces in priority order (library first), the four constraints, and a corrected quickstart. + ## [0.6.0] - 2026-07-18 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 8cdcd4b..9a3d9cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,28 +1,300 @@ -# CLAUDE.md — seed / bootstrap placeholder +# CLAUDE.md -> **This is a self-initializing seed, not a finished runtime prompt.** -> Run `/init` (or describe the agent's domain to your AI assistant) to -> re-initialize this file into a full runtime prompt, using the description -> below and the scaffolded repo as context. +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. -## Agent +## What this repo is -This repository hosts the **shell-cli** agent. +**shell-cli owns the file-and-shell tool surface for AI coding agents.** An agent +harness needs a small, boring, *safe* set of primitives: read a file, write a +file, edit a file, list a directory, look at an image, and run a shell command. +Today that surface lives inside +[`colleague`](https://github.com/agentculture/colleague) welded to one host, so +every other harness reimplements it — and reimplements the safety model with it. -## Description +This repo lifts that surface out into a reusable, pure-stdlib package. colleague +becomes the first consumer, not the owner. The authoritative build brief is +[issue #1](https://github.com/agentculture/shell-cli/issues/1) — read it before +large-scale work; it is the source of truth for scope. -Agent-first CLI and agent owning the file-and-shell tool surface for AI coding agents: read, write, edit, list, view media, and gated shell execution, with path confinement and an operator approval policy. Pure-stdlib core, extracted from colleague so any host imports one safe execution layer. +Identity is settled and must not drift: repo token `shell-cli`, console command +**`shell`**, import package **`shell`**, PyPI distribution **`shell-cli`**. -## Re-init instruction +### Current state -This file is a seed. To expand it into your full runtime prompt: +The repo is **scaffold only**. The agent-first CLI skeleton (`whoami`, `learn`, +`explain`, `overview`, `doctor`, `cli`, `explain safety`) is real and green; +**none of the six primitives, the path confinement, or the approval policy has +been extracted yet.** The CLI says so itself — `learn`, `overview`, and the +`explain` root each carry a Status section. Keep those honest as the extraction +lands; they are the first thing an agent consumer reads. -1. Open this repo in Claude Code (or your preferred AI assistant). -2. Run `/init` — the assistant will read the repo, incorporate the description - above, and replace this seed with a complete `CLAUDE.md`. -3. Commit the result. +## Commands -Until you run `/init`, `shell-cli` satisfies the `steward doctor` -`prompt-file-present` and `backend-consistency` invariants (a `CLAUDE.md` -exists and `culture.yaml` declares `backend: claude`) but the prompt is not -yet tailored to this agent's domain. +```bash +uv sync # create/refresh .venv +uv run pytest -n auto # full suite (xdist) +uv run pytest tests/test_cli.py -v # one file +uv run pytest -k test_whoami_json -v # one test +uv run pytest --cov=shell --cov-report=term # with coverage (fail_under=60) + +uv run shell whoami # the console command is `shell`, NOT `shell-cli` +uv run shell doctor --json + +# The four lint gates CI runs, in CI order: +uv run black --check shell tests +uv run isort --check-only shell tests +uv run flake8 shell tests +uv run bandit -c pyproject.toml -r shell +markdownlint-cli2 "**/*.md" "#node_modules" "#.local" "#.claude/skills" +uv run teken cli doctor . --strict # the agent-first rubric gate +``` + +`teken` is a **dev dependency only** — it renders/validates the CLI shape; it +must never become a runtime dependency. + +## Non-negotiable constraints + +These four are contracts, not preferences. Violating any of them breaks a +downstream consumer or an explicit honesty commitment. + +### 1. The core is pure-stdlib — zero base dependencies + +colleague pins exactly one base dependency (`agentfront>=0.20.0`) and guards it +with `tests/test_zero_deps.py`, which allow-lists that single name. The operator +decided shell-cli clears the same bar and gets allow-listed as a **second** +sanctioned base dep. That means the core may import **nothing outside the Python +standard library, ever** — if you take a third-party dep, colleague cannot import +you at all. + +`pyproject.toml` already has `dependencies = []`. Keep it that way. Anything +optional (a richer renderer, an MCP surface, container backends) goes in a +`[project.optional-dependencies]` extra, following colleague's `[mcp]` / `[otel]` +/ `[tui]` pattern. + +**Write the guard test early** — mirror colleague's +`/home/spark/git/colleague/tests/test_zero_deps.py`, which does two things worth +copying: asserts the declared `[project].dependencies` list literally, and +snapshots `sys.modules` before/after importing the package to catch third-party +leaks that a manifest check would miss. Enforce it in CI rather than remembering +it. + +### 2. It is a guard, not a sandbox — never upgrade the claim + +A package *named* `shell-cli` whose headline is safe execution will be read as +offering a sandbox. It does not. colleague's own comments are candid about this +and that posture must survive the extraction verbatim: + +> Honest limitation: the guard is a substring check on the raw command string. A +> sufficiently obfuscated command (e.g. variable expansion, concatenation, +> here-docs) could bypass it. It is best-effort — an airtight sandbox is out of +> v0 scope. The guard covers the obvious / accidental case; document rather than +> overclaim. +> +> Honest limit: like the rest of this gate, it is bypassable by `sh -c`, +> pipelines, and shell expansion — **a guard, not a sandbox.** + +State the threat model explicitly and prominently in the README: this protects +against **accidental and careless** model behaviour, not against an adversarial +one. Do not let a docstring, a README line, or a commit message quietly imply +isolation the code does not provide. + +### 3. Write verbs are dry-run by default; `--apply` commits + +Agents call CLIs in loops. Any verb that mutates the filesystem previews by +default and only writes when `--apply` is passed. Safe-by-default is mandatory, +not a nicety. + +### 4. Every PR bumps the version + +Every push to `main` publishes to PyPI, and the `version-check` CI job fails a PR +whose `pyproject.toml` version matches `main` — including docs-only and CI-only +PRs. Use the `version-bump` skill (`/version-bump patch|minor|major`); it updates +`pyproject.toml` and prepends a Keep-a-Changelog entry. + +## Architecture + +### The two surfaces, in priority order + +You are building two things, and the order matters: + +1. **A library** — the primary consumer is colleague *importing* you. This is the + surface that carries the value. Design it first. +2. **A CLI** (`shell read`, `shell run`, `shell edit`) — so a human or an agent + can drive the same gated surface from a terminal and watch the policy decide. + +Do not let CLI ergonomics distort the library API. + +### CLI scaffold conventions (already established — follow them) + +The afi-cli/teken pattern, no `src/` wrapper, one module per subcommand under +`shell/cli/_commands/`. Each module exposes a `register(sub)` that adds its +parser and sets `func`; `shell/cli/__init__.py:_build_parser` wires them up. +Noun groups nest their own subparsers the same way (see `_commands/cli.py`). + +Three contracts are load-bearing and enforced by tests plus the `teken cli +doctor --strict` rubric gate: + +- **Errors never leak tracebacks.** Every failure raises `CliError` + (`shell/cli/_errors.py`); `_dispatch` catches it, routes it through + `emit_error`, and wraps any unexpected exception. Argparse errors are folded + into the same shape by `_CliArgumentParser.error`, with a `_json_hint` + class attribute pre-scanned from raw argv so parse-time failures still honour + `--json`. New subparsers must be built with `parser_class=_CliArgumentParser` + or they bypass this and exit 2 with raw argparse output. +- **Streams never mix.** Results to stdout, errors and diagnostics to stderr + (`shell/cli/_output.py`). Text-mode errors render `error: …` then `hint: …` — + the `hint:` prefix is what agent consumers parse. +- **Exit codes**: `0` success, `1` user error, `2` environment error, `3+` + reserved. + +Every verb takes `--json`. Every registered noun/verb path needs an entry in +`shell/explain/catalog.py`; `tests/test_cli.py::test_every_catalog_path_resolves` +walks the whole catalog, and the rubric gate requires any noun with action-verbs +to expose `overview`. + +`whoami`/`doctor` parse `culture.yaml` with a hand-rolled line scanner +(`_commands/whoami.py`) specifically to avoid a YAML dependency — re-read the +pure-stdlib constraint before "improving" that. + +### What to extract from colleague, and from where + +Surveyed against the live checkout at `/home/spark/git/colleague` (colleague +1.51.0). `colleague/tools.py` is 1360 lines declaring its tool schemas as a flat +list; six are yours, the rest are colleague's own. + +| Yours (extract) | colleague-specific (leave behind) | +|---|---| +| `read_file` | `culture` | +| `view_media` | `devague` | +| `write_file` | `subagent` / `subagents` | +| `edit_file` | `memory` | +| `list_dir` | `run_tests` / `check_test_integrity` | +| `run_command` | `finish`, `deepthink` | + +Handlers live in `ToolExecutor` (`colleague/tools.py:666`): `_read_file:813`, +`_view_media:827`, `_write_file:860`, `_edit_file:878`, `_list_dir:947`, +`_run_command:957`. + +**The safety machinery moves with them — this was decided explicitly and it is +the point of the split.** A package shipping the six primitives *without* the +safety model would be a thin wrapper worth very little. + +- `_safe_path` (`tools.py:730`) — resolves a relative path under the root and + refuses anything that escapes it. +- `_refuse_clone_write` (`tools.py:737`) — refuses writes into a read-only source + subtree. **Generalise this**: a configurable set of read-only paths, not a + hard-coded `.colleague/neighbours`. +- `_truncate` (`tools.py:724`) — caps each result so a huge file or command + output cannot blow the model's context window. +- `_number_lines` (`tools.py:599`) and the `_require` argument-validation helpers. +- The `run_command` approval policy — `colleague/policy.py` (400 lines): the + frozen `Verdict` dataclass (`:80`), `Policy.check_run_command` (`:205`), + `Policy.check_file` (`:249`), the checksum helpers `file_checksum` (`:101`) / + `verify_checksum` (`:121`), `_first_token` (`:309`), and policy-file loading + (`_parse_policy_file:326`, `load_policy:354`). colleague gates only + `run_command` today (`colleague/loop.py:895-903`); everything else passes + through. + +### The real work is decoupling, not copying + +Both files are stdlib-clean but **not** import-clean. Two couplings must break: + +- `tools.py` does `from colleague import culture, devague, media, memory, + testintegrity` and `from colleague.config import _DEFAULT_MAX_OUTPUT_CHARS, + MAX_SUBAGENT_FANOUT`. Most of that belongs to tools you are *not* taking — but + **`media` is imported for `view_media`, which is yours.** Decide deliberately: + either `media` travels along, or `view_media` stays behind. Say which and why. +- `policy.py` does `from colleague.configdir import resolve_file` and `from + colleague.layers import sanitize_model`. Both must become injected parameters + or travel along. + +`ToolExecutor` is one class mixing both halves, so extraction means splitting it. +The encouraging part: its constructor is already injection-shaped — `spawn`, +`batch_spawn`, `deepthink` callables plus an `allowlist` +(`tools.py:669-723`). A natural seam is shell-cli owning the base executor (six +primitives + confinement + policy) with colleague subclassing it and registering +its own tools on top. **Propose the seam before building it; do not assume that +shape is the only one.** + +Tests worth mining from `/home/spark/git/colleague/tests/`: +`test_loop_run_command_policy.py`, `test_policy_carveout.py`, +`test_policy_all_engines.py`, `test_write_apply_isolation.py`, +`test_zero_deps.py`. + +### Planned: VM / in-container execution + +Beyond issue #1, the intended direction is a **real** isolation backend — +running `run_command` inside a container or VM rather than the host shell. Two +things to hold onto when that work starts: + +- It does not retroactively make the *existing* guard a sandbox. Keep the two + claims separate: the substring/token guard is best-effort host-side; a + container backend is genuine isolation. Never let one sentence cover both. +- Constraint #1 still binds. Driving `docker`/`podman` by `subprocess` to a CLI + on `PATH` keeps the core pure-stdlib; pulling a Docker SDK does not, and would + have to live behind an extra. + +## Open questions — park them, do not guess + +Recorded in issue #1; still unanswered. Use `/think` (devague) if one needs real +framing rather than a quick call. + +- Does `media` / `view_media` travel here, or stay in colleague? +- Does the approval-policy *file format* stay colleague's, or does shell-cli + define its own and colleague adapt? (Policy files exist in the wild; a format + change is a migration.) +- Is the read-only-subtree guard generalised to arbitrary paths, or does it stay + a single configurable directory? +- Does the CLI expose the policy as a first-class verb (`shell policy check …`)? +- Does anything beyond colleague adopt this soon — and should that shape the API + now, or wait for a second real consumer? + +## Definition of done, first milestone + +1. Six primitives + confinement + truncation + approval policy live here, + pure-stdlib, with a `test_zero_deps`-style guard enforcing it. +2. Tests carried over and extended from the colleague suite named above. +3. A documented threat model that does **not** claim to be a sandbox. +4. A concrete migration proposal **filed as an issue on colleague**: how it drops + its copy, allow-lists `shell-cli` in its guard test, and keeps behaviour + identical. colleague's change is colleague's to make — **propose, don't push.** + +## Naming, and the prompt-file story + +- **`shell` is the executable; `shell-cli` is the repo, the agent nick, and the + PyPI distribution.** Both are correct in their own place, so neither is a + find-and-replace target. Usage examples, `prog`, and anything a reader might + paste into a terminal say `shell`. The nick reported by `whoami` (from + `culture.yaml`'s `suffix`), the distribution name, the Sonar project key, and + the GitHub URLs stay `shell-cli`. `explain` accepts both as the root path. +- **Prompt files.** `culture.yaml` declares `backend: colleague`, so `doctor`'s + backend-consistency check requires `AGENTS.colleague.md` — which exists and + passes. This `CLAUDE.md` is the Claude Code prompt and the contributor guide; + it does **not** change that mapping. Keep both files, and keep them from + contradicting each other. + +## Conventions and workflow + +- **Skills**: `.claude/skills/` is vendored cite-don't-import from guildmaster + (and directly from devague/colleague for a few tracked divergences). Provenance + and the re-sync procedure live in `docs/skill-sources.md` — read it before + editing anything under `.claude/skills/`; local edits are lost on re-sync + unless lifted upstream first. Vendored skills are excluded from markdownlint + and from Sonar analysis on purpose. +- **PR lane**: use the `cicd` skill (`devex pr` plus SonarCloud `status` / + `await`). CI blocks on the Sonar quality gate when `SONAR_TOKEN` is set. +- **Second opinion**: reach for `ask-colleague` reflexively — `review` before + opening a PR on a non-trivial diff, `explore` for a fresh read of an + unfamiliar area. Both are read-only and run in a throwaway worktree, so the + reflex is always safe; the side-effecting `write --apply` / `--pr` needs the + user's go-ahead. Given that colleague is *the* consumer of this package, its + read of an extraction seam is worth more here than usual. +- **Memory**: `/recall` before non-trivial work to build on prior decisions + rather than re-deriving them; `/remember` when a non-obvious decision, + constraint, fix-and-why, or hard-won gotcha surfaces. This repo's memory is + in-repo and public — records resolve to `/.eidetic/memory` + (committed, team- and mesh-shared). Pass `--visibility private` to route a + record to `$HOME` instead. +- **Cross-repo**: use the `communicate` skill to file issues on sibling repos + (auto-signs `- shell-cli (Claude)`); use `gh issue create` or `cicd` for issues + on this repo. diff --git a/README.md b/README.md index bfd0a1e..0846ead 100644 --- a/README.md +++ b/README.md @@ -1,26 +1,69 @@ # shell-cli -Agent-first CLI and agent owning the file-and-shell tool surface for AI coding agents: read, write, edit, list, view media, and gated shell execution, with path confinement and an operator approval policy. Pure-stdlib core, extracted from colleague so any host imports one safe execution layer. - -## What you get - -- **An agent-first CLI** cited from [teken](https://github.com/agentculture/teken) - (`afi-cli`) — the runtime package has no third-party dependencies. -- **A mesh identity** — `culture.yaml` (`suffix` + `backend`) and the matching - resident prompt file (`AGENTS.colleague.md`, since this template runs - `backend: colleague`). -- **The canonical guildmaster skill kit** (11 skills) under `.claude/skills/`, - vendored cite-don't-import. See [`docs/skill-sources.md`](docs/skill-sources.md). -- **A build + deploy baseline** — pytest, lint, the agent-first rubric gate, and - PyPI Trusted Publishing wired into GitHub Actions. +The file-and-shell tool surface for AI coding agents: read, write, edit, list, +view media, and gated shell execution — with path confinement and an operator +approval policy. Pure-stdlib core, extracted from +[`colleague`](https://github.com/agentculture/colleague) so any harness imports +one execution layer instead of reimplementing it, and reimplementing its safety +model with it. + +Repo token `shell-cli`, console command **`shell`**, import package **`shell`**, +PyPI distribution **`shell-cli`**. + +## ⚠️ A guard, not a sandbox + +**Read this before you rely on anything here.** + +The execution gate is **best-effort**. It inspects the command string it is +handed, so it is bypassable by `sh -c`, pipelines, shell expansion, here-docs, +and any interpreter that takes code as an argument. There is no namespace, +container, or seccomp isolation. + +It protects against **accidental and careless** model behaviour. It does **not** +protect against an adversarial one, and it is not a security boundary to put +untrusted input behind. + +Full detail — assets, actors, known bypasses, and what real isolation would +require — in [`docs/threat-model.md`](docs/threat-model.md). The same posture is +readable in-band via `shell explain safety`, and +[`tests/test_honesty.py`](tests/test_honesty.py) fails the build if any shipped +surface drops the disclaimer or claims isolation. + +## Status + +**Scaffold.** The agent-first CLI skeleton is real and green; the six +primitives, the path confinement, and the approval policy are still being +extracted from `colleague`. Only the introspection verbs below are implemented +today. The build brief and current scope live in +[issue #1](https://github.com/agentculture/shell-cli/issues/1). + +## What this will be + +Two surfaces, in priority order: + +1. **A library** — the primary consumer is another agent harness *importing* + this. That is the surface carrying the value, and it is designed first. +2. **A CLI** — `shell read`, `shell run`, `shell edit`, so a human or an agent + can drive the same gated surface from a terminal and watch the policy decide. + +Four constraints hold it together: + +- **Pure-stdlib core, zero base dependencies.** colleague allow-lists exactly + its sanctioned base deps; a third-party dependency here means colleague cannot + import this package at all. Optional features go behind extras. +- **A guard, not a sandbox** — see above. +- **Write verbs are dry-run by default**; `--apply` commits. Agents call CLIs in + loops, so safe-by-default is mandatory. +- **Every PR bumps the version** — every push to `main` publishes to PyPI. ## Quickstart ```bash uv sync uv run pytest -n auto # run the test suite -uv run shell-cli whoami # identity from culture.yaml -uv run shell-cli learn # self-teaching prompt (add --json) +uv run shell whoami # identity from culture.yaml +uv run shell learn # self-teaching prompt (add --json) +uv run shell explain safety # the safety posture, in-band uv run teken cli doctor . --strict # the agent-first rubric gate CI runs ``` @@ -30,7 +73,7 @@ uv run teken cli doctor . --strict # the agent-first rubric gate CI runs |------|--------------| | `whoami` | Report this agent's nick, version, backend, and model from `culture.yaml`. | | `learn` | Print a structured self-teaching prompt. | -| `explain ` | Markdown docs for any noun/verb path. | +| `explain ` | Markdown docs for any noun/verb path (`explain safety` for the threat model). | | `overview` | Read-only descriptive snapshot of the agent. | | `doctor` | Check the agent-identity invariants (prompt-file-present, backend-consistency). | | `cli overview` | Describe the CLI surface itself. | @@ -39,20 +82,17 @@ Every command supports `--json`. Results go to stdout, errors/diagnostics to stderr (never mixed). Exit codes: `0` success, `1` user error, `2` environment error, `3+` reserved. -## Make it your own +## Repo furniture -1. Rename the package `shell/` and the `shell-cli` - CLI/dist name throughout `pyproject.toml`, the package, `tests/`, - `sonar-project.properties`, and this `README.md`. The name is hard-coded in - ~100 places, so list every occurrence first — see the `git grep` discovery - command in [`CLAUDE.md`](CLAUDE.md), the authoritative rename procedure. -2. Edit `culture.yaml` with your `suffix` and `backend`. -3. Rewrite `CLAUDE.md` for your agent and run `/init`. -4. Re-vendor only the skills you need from guildmaster (see - [`docs/skill-sources.md`](docs/skill-sources.md)). - -See [`CLAUDE.md`](CLAUDE.md) for the full conventions (version-bump-every-PR, -the `cicd` PR lane, deploy setup). +- **Mesh identity** — `culture.yaml` (`suffix` + `backend`) and the matching + resident prompt file (`AGENTS.colleague.md`, since this agent runs + `backend: colleague`). `CLAUDE.md` is the Claude Code prompt and the + contributor guide. +- **The canonical guildmaster skill kit** under `.claude/skills/`, vendored + cite-don't-import. Provenance and the re-sync procedure: + [`docs/skill-sources.md`](docs/skill-sources.md). +- **Build + deploy baseline** — pytest, lint, the agent-first rubric gate, and + PyPI Trusted Publishing wired into GitHub Actions. ## License diff --git a/docs/threat-model.md b/docs/threat-model.md new file mode 100644 index 0000000..9bfbcf2 --- /dev/null +++ b/docs/threat-model.md @@ -0,0 +1,76 @@ +# Threat model + +**shell-cli ships a guard, not a sandbox.** + +This document exists because the package name invites the wrong assumption. A +library whose headline is *safe shell execution* reads as an isolation boundary. +It is not one, it has never been one, and no amount of hardening short of +namespaces/containers/seccomp would make it one. State that plainly rather than +letting a reader infer otherwise. + +## Status + +The six primitives, the path confinement, and the approval policy are **not yet +extracted** from [`colleague`](https://github.com/agentculture/colleague) (see +[issue #1](https://github.com/agentculture/shell-cli/issues/1)). This document +describes the contract the extraction must uphold — the posture is inherited +along with the code, and must not be quietly upgraded in transit. + +## Who this protects, and from what + +The asset is the operator's machine and repository. The actor is **a language +model driving a tool loop** — one that is capable, fallible, and occasionally +confidently wrong. + +| Protected against | Not protected against | +|---|---| +| A model that misreads a path and walks out of the repo root | A model (or a prompt injection steering it) that is *trying* to escape | +| A write into a tree the operator declared read-only | Anything at all once the command reaches a shell | +| A command the operator's policy has not approved | Obfuscation that hides the command's real target | +| A result large enough to blow the caller's context window | Resource exhaustion (CPU, disk, fork bombs) | + +The distinction that matters: this stops **accidental and careless** behaviour. +It does not stop **adversarial** behaviour, and it is not a security boundary +you should place untrusted input behind. + +## Known bypasses + +These are not bugs to be fixed. They are consequences of gating a string that a +shell will later re-interpret, and they are documented rather than papered over: + +- **`sh -c` and friends** — the gate inspects the command it is handed, not + what that command subsequently spawns. +- **Shell expansion** — variable expansion, command substitution, globbing, and + concatenation can assemble a blocked target out of pieces that individually + look fine. +- **Here-docs and pipelines** — the payload is not the token the gate examined. +- **Interpreters** — `python -c`, `perl -e`, and any other language runtime take + arbitrary code as an opaque argument. +- **Symlinks and TOCTOU** — path confinement resolves at check time; a path can + change between the check and the operation. + +Upstream in colleague, the token-aware `shlex` check is deliberately paired with +a conservative substring fallback for unparseable commands, and the guard is +written to never raise — a guard that crashes the drive is worse than one that +is merely incomplete. Preserve both properties. + +## What actual isolation would require + +Real isolation is separate, deliberate work — not a wording change and not an +incremental hardening of the existing guard: + +- OS-level containment (namespaces, cgroups, seccomp), or +- a container runtime, or +- a virtual machine. + +A container/VM execution backend is planned. When it lands, the two claims must +stay clearly separated in every surface that mentions them: *the host-side guard +remains best-effort* and *the isolated backend is the one that actually +contains*. One sentence must never cover both. + +## Enforcement + +`tests/test_honesty.py` asserts the disclaimer is present in `learn`, the +`explain` root, `explain safety`, the README, `CLAUDE.md`, and this file — and +that no shipped surface makes a positive isolation claim. If you find that test +inconvenient, the correct response is to change the wording, not the test. diff --git a/pyproject.toml b/pyproject.toml index 544d64f..c8fcf20 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "shell-cli" -version = "0.6.0" +version = "0.7.2" description = "Agent-first CLI and agent owning the file-and-shell tool surface for AI coding agents: read, write, edit, list, view media, and gated shell execution, with path confinement and an operator approval policy. Pure-stdlib core, extracted from colleague so any host imports one safe execution layer." readme = "README.md" license = "Apache-2.0" diff --git a/shell/cli/__init__.py b/shell/cli/__init__.py index 6018442..077ca2c 100644 --- a/shell/cli/__init__.py +++ b/shell/cli/__init__.py @@ -1,4 +1,4 @@ -"""Unified CLI entry point for shell-cli. +"""Unified CLI entry point for shell-cli (the ``shell`` executable). The agent-first global verbs (``whoami``, ``learn``, ``explain``, ``overview``, ``doctor``) are registered here under :mod:`shell.cli._commands`, @@ -70,8 +70,12 @@ def _build_parser() -> argparse.ArgumentParser: from shell.cli._commands import whoami as _whoami_cmd parser = _CliArgumentParser( - prog="shell-cli", - description="shell-cli — a clonable template for AgentCulture mesh agents.", + prog="shell", + description=( + "shell — the file-and-shell tool surface for AI coding agents: " + "read, write, edit, list, view media, and gated shell execution. " + "A guard, not a sandbox — see `shell explain safety`." + ), ) parser.add_argument( "--version", diff --git a/shell/cli/_commands/cli.py b/shell/cli/_commands/cli.py index c872fe2..e0bc804 100644 --- a/shell/cli/_commands/cli.py +++ b/shell/cli/_commands/cli.py @@ -1,4 +1,4 @@ -"""``shell-cli cli`` — noun grouping CLI-surface introspection. +"""``shell cli`` — noun grouping CLI-surface introspection. Exists to satisfy the agent-first rubric's ``overview_cli_noun_exists`` check: any noun with action-verbs must also expose ``overview``. There are no @@ -15,7 +15,7 @@ def cmd_cli_overview(args: argparse.Namespace) -> int: emit_overview( - "shell-cli cli", + "shell cli", cli_sections(), json_mode=bool(getattr(args, "json", False)), ) @@ -23,14 +23,14 @@ def cmd_cli_overview(args: argparse.Namespace) -> int: def _no_verb(args: argparse.Namespace) -> int: - # `shell-cli cli` with no sub-verb prints the noun's overview. + # `shell cli` with no sub-verb prints the noun's overview. return cmd_cli_overview(args) def register(sub: argparse._SubParsersAction) -> None: p = sub.add_parser( "cli", - help="CLI-surface introspection (see 'shell-cli cli overview').", + help="CLI-surface introspection (see 'shell cli overview').", ) p.add_argument("--json", action="store_true", help="Emit structured JSON.") p.set_defaults(func=_no_verb, json=False) @@ -38,6 +38,6 @@ def register(sub: argparse._SubParsersAction) -> None: # parser_class); propagate it so `cli overview` parse errors route through # the structured error contract instead of argparse's default stderr/exit 2. noun_sub = p.add_subparsers(dest="cli_command", parser_class=type(p)) - ov = noun_sub.add_parser("overview", help="Describe the shell-cli CLI surface.") + ov = noun_sub.add_parser("overview", help="Describe the shell CLI surface.") ov.add_argument("--json", action="store_true", help="Emit structured JSON.") ov.set_defaults(func=cmd_cli_overview) diff --git a/shell/cli/_commands/doctor.py b/shell/cli/_commands/doctor.py index 8427fd2..f5de22e 100644 --- a/shell/cli/_commands/doctor.py +++ b/shell/cli/_commands/doctor.py @@ -1,4 +1,4 @@ -"""``shell-cli doctor`` — check the agent-identity invariants. +"""``shell doctor`` — check the agent-identity invariants. Mirrors the two invariants ``steward doctor`` verifies for a mesh agent: @@ -105,7 +105,7 @@ def cmd_doctor(args: argparse.Namespace) -> int: emit_result(report, json_mode=True) else: status = "healthy" if report["healthy"] else "unhealthy" - lines = [f"shell-cli doctor: {status}", ""] + lines = [f"shell doctor: {status}", ""] for check in report["checks"]: mark = "ok" if check["passed"] else "FAIL" lines.append(f"[{mark}] {check['id']}: {check['message']}") diff --git a/shell/cli/_commands/explain.py b/shell/cli/_commands/explain.py index deab724..5103ed9 100644 --- a/shell/cli/_commands/explain.py +++ b/shell/cli/_commands/explain.py @@ -1,4 +1,4 @@ -"""``shell-cli explain ...`` — global markdown catalog lookup (stable-contract). +"""``shell explain ...`` — global markdown catalog lookup (stable-contract). ``explain`` is global (not nested under a noun). It takes zero or more path tokens and resolves them via the catalog in :mod:`shell.explain`. @@ -32,7 +32,7 @@ def register(sub: argparse._SubParsersAction) -> None: p.add_argument( "path", nargs="*", - help="Command path tokens; empty = root (same as 'shell-cli').", + help="Command path tokens; empty = root (same as 'shell').", ) p.add_argument("--json", action="store_true", help="Emit structured JSON.") p.set_defaults(func=cmd_explain) diff --git a/shell/cli/_commands/learn.py b/shell/cli/_commands/learn.py index 2122be3..b92cd98 100644 --- a/shell/cli/_commands/learn.py +++ b/shell/cli/_commands/learn.py @@ -1,4 +1,4 @@ -"""``shell-cli learn`` — the learnability affordance. +"""``shell learn`` — the learnability affordance. Prints a structured self-teaching prompt. Must satisfy the agent-first rubric: >=200 chars and mention purpose, command map, exit codes, --json, and explain. @@ -12,23 +12,37 @@ from shell.cli._output import emit_result _TEXT = """\ -shell-cli — a clonable template for AgentCulture mesh agents. +shell — the file-and-shell tool surface for AI coding agents. Purpose ------- -Scaffold for a new Culture mesh agent: an agent-first CLI (cited from the teken -`python-cli` reference), an identity (culture.yaml + CLAUDE.md), the canonical -guildmaster skill kit under .claude/skills/, and a deploy/CI baseline. Clone it, -rename the package, and edit culture.yaml to mint a new agent. +Read, write, edit, list, view media, and gated shell execution — with path +confinement and an operator approval policy. Packaged pure-stdlib so any agent +harness imports one safe execution layer instead of reimplementing it, and +reimplementing its safety model with it. Extracted from `colleague`, which is +the first consumer rather than the owner. + +Safety posture +-------------- +A guard, NOT a sandbox. The execution gate is best-effort and is bypassable by +`sh -c`, pipelines, and shell expansion. It protects against accidental and +careless behaviour, not an adversarial one. Read `shell explain safety` before +relying on it. + +Status +------ +Scaffold: the six primitives, the path confinement, and the approval policy are +not extracted yet. Only the introspection verbs below are implemented. +See https://github.com/agentculture/shell-cli/issues/1 Commands -------- - shell-cli whoami Identity from culture.yaml. - shell-cli learn This self-teaching prompt. - shell-cli explain ... Markdown docs for any noun/verb path. - shell-cli overview Descriptive snapshot of the agent. - shell-cli doctor Check the agent-identity invariants. - shell-cli cli overview Describe the CLI surface itself. + shell whoami Identity from culture.yaml. + shell learn This self-teaching prompt. + shell explain ... Markdown docs for any noun/verb path. + shell overview Descriptive snapshot of the agent. + shell doctor Check the agent-identity invariants. + shell cli overview Describe the CLI surface itself. Machine-readable output ----------------------- @@ -44,15 +58,30 @@ More detail ----------- - shell-cli explain shell-cli + shell explain shell + shell explain safety """ def _as_json_payload() -> dict[str, object]: return { - "tool": "shell-cli", + "tool": "shell", + "distribution": "shell-cli", "version": __version__, - "purpose": "Clonable scaffold for a new AgentCulture mesh agent.", + "purpose": ( + "The file-and-shell tool surface for AI coding agents: read, write, edit, " + "list, view media, and gated shell execution, with path confinement and an " + "operator approval policy. Pure-stdlib core." + ), + "safety_posture": ( + "A guard, not a sandbox. The execution gate is best-effort and bypassable " + "by `sh -c`, pipelines, and shell expansion; it protects against accidental " + "and careless behaviour, not an adversarial one." + ), + "status": ( + "scaffold — the six primitives, path confinement, and approval policy are " + "not extracted yet; only the introspection verbs are implemented" + ), "commands": [ {"path": ["whoami"], "summary": "Identity probe from culture.yaml."}, {"path": ["learn"], "summary": "Self-teaching prompt."}, @@ -67,7 +96,7 @@ def _as_json_payload() -> dict[str, object]: "2": "environment/setup error", }, "json_support": True, - "explain_pointer": "shell-cli explain ", + "explain_pointer": "shell explain ", } diff --git a/shell/cli/_commands/overview.py b/shell/cli/_commands/overview.py index c11eeec..830020f 100644 --- a/shell/cli/_commands/overview.py +++ b/shell/cli/_commands/overview.py @@ -1,7 +1,8 @@ -"""``shell-cli overview`` — read-only descriptive snapshot of the agent. +"""``shell overview`` — read-only descriptive snapshot of the agent. -Describes the agent to an agent reader: identity (from culture.yaml), the verb -surface, and the sibling-pattern artifacts this template carries. The shared +Describes the agent to an agent reader: identity (from culture.yaml), the +mission and current build status, the verb surface, and the artifacts this repo +carries. The shared section/render helpers here are reused by the ``cli`` noun's ``overview`` (see :mod:`shell.cli._commands.cli`). @@ -19,6 +20,7 @@ _ARTIFACTS = [ "culture.yaml + AGENTS.colleague.md — mesh identity (suffix + backend)", + "docs/threat-model.md — what the guard does and does not protect against", ".claude/skills/ — the canonical guildmaster skill kit (cite-don't-import)", "docs/skill-sources.md — skill provenance ledger", "pyproject.toml + .github/workflows/ — buildable, deployable package baseline", @@ -32,6 +34,20 @@ "doctor — check the agent-identity invariants", ] +_MISSION = [ + "owns the file-and-shell tool surface for AI coding agents", + "read / write / edit / list / view_media / run_command, gated", + "path confinement + operator approval policy travel with the primitives", + "pure-stdlib core (zero base dependencies) so any harness can import it", + "a guard, NOT a sandbox — see `shell explain safety`", +] + +_STATUS = [ + "scaffold — only the introspection verbs below are implemented", + "the six primitives, confinement, and policy are not extracted yet", + "tracking issue: https://github.com/agentculture/shell-cli/issues/1", +] + def agent_sections() -> list[dict[str, object]]: """Sections describing the agent (used by the global verb).""" @@ -46,8 +62,10 @@ def agent_sections() -> list[dict[str, object]]: f"model: {ident['model']}", ], }, + {"title": "Mission", "items": list(_MISSION)}, + {"title": "Status", "items": list(_STATUS)}, {"title": "Verbs", "items": list(_VERBS)}, - {"title": "Sibling-pattern artifacts", "items": list(_ARTIFACTS)}, + {"title": "Artifacts", "items": list(_ARTIFACTS)}, ] @@ -61,9 +79,11 @@ def cli_sections() -> list[dict[str, object]]: { "title": "Conventions", "items": [ + "the executable is `shell`; `shell-cli` is the repo + PyPI distribution", "every command supports --json", "results to stdout, errors/diagnostics to stderr (never mixed)", "exit codes: 0 success, 1 user error, 2 environment error, 3+ reserved", + "write verbs are dry-run by default; --apply commits", ], }, ] diff --git a/shell/cli/_commands/whoami.py b/shell/cli/_commands/whoami.py index 7f1431b..201e1e8 100644 --- a/shell/cli/_commands/whoami.py +++ b/shell/cli/_commands/whoami.py @@ -1,11 +1,12 @@ -"""``shell-cli whoami`` — the smallest identity probe. +"""``shell whoami`` — the smallest identity probe. Reports the agent's identity as declared in ``culture.yaml``: its nick (``suffix``), the backend it runs on, and the served model (if any) — plus the package version. Read-only; touches nothing but its own ``culture.yaml``. -When you clone this template, rename the package and update ``culture.yaml`` — -``whoami`` then reflects your new agent's identity with no code change. +The nick is the agent/repo token ``shell-cli``, which is deliberately NOT the +executable name (``shell``) — see the Conventions section of ``shell cli +overview``. """ from __future__ import annotations diff --git a/shell/cli/_errors.py b/shell/cli/_errors.py index fa12f8c..a96b6dc 100644 --- a/shell/cli/_errors.py +++ b/shell/cli/_errors.py @@ -13,7 +13,7 @@ from dataclasses import dataclass -# Exit-code policy. Documented in ``shell-cli learn`` output. +# Exit-code policy. Documented in ``shell learn`` output. # 0 = success # 1 = user-input error (bad flag, missing required arg, unknown path) # 2 = environment / setup error (tool not installed, file unreadable) diff --git a/shell/explain/__init__.py b/shell/explain/__init__.py index 7ec0eff..acc4680 100644 --- a/shell/explain/__init__.py +++ b/shell/explain/__init__.py @@ -16,7 +16,7 @@ def resolve(path: tuple[str, ...]) -> str: raise CliError( code=EXIT_USER_ERROR, message=f"no explain entry for: {display}", - remediation="list entries with: shell-cli explain shell-cli", + remediation="list entries with: shell explain shell", ) diff --git a/shell/explain/catalog.py b/shell/explain/catalog.py index 77a8c77..323616b 100644 --- a/shell/explain/catalog.py +++ b/shell/explain/catalog.py @@ -1,7 +1,9 @@ -"""Markdown catalog for ``shell-cli explain ``. +"""Markdown catalog for ``shell explain ``. -Each entry is verbatim markdown. Keys are command-path tuples. The empty tuple -and ``("shell-cli",)`` both resolve to the root entry. +Each entry is verbatim markdown. Keys are command-path tuples. The empty tuple, +``("shell",)`` and ``("shell-cli",)`` all resolve to the root entry — the +executable is ``shell``, the repo/distribution token is ``shell-cli``, and a +reader may reasonably reach for either. Keep bodies self-contained: an agent reading one entry should get enough context without chaining reads. @@ -10,22 +12,39 @@ from __future__ import annotations _ROOT = """\ -# shell-cli +# shell -A clonable template for AgentCulture mesh agents. It carries an agent-first CLI -(cited from the teken `python-cli` reference), a mesh identity (`culture.yaml` + -`CLAUDE.md`), the canonical guildmaster skill kit under `.claude/skills/`, and a -buildable/deployable package baseline. Clone it, rename the package, edit -`culture.yaml`, and you have a new agent. +The file-and-shell tool surface for AI coding agents: read, write, edit, list, +view media, and gated shell execution — with path confinement and an operator +approval policy. Packaged pure-stdlib so any harness imports one safe execution +layer instead of reimplementing it (and reimplementing its safety model). + +Two surfaces, in priority order: a **library** (the primary consumer is another +harness importing it) and a **CLI** (drive the same gated surface from a +terminal and watch the policy decide). + +## Status + +Scaffold. The six primitives, the path confinement, and the approval policy are +being extracted from `colleague` and are **not here yet** — only the +introspection verbs below are implemented. See +. + +## Safety posture + +A guard, not a sandbox. The execution gate is best-effort: it refuses the +obvious and accidental case, and it is bypassable by `sh -c`, pipelines, and +shell expansion. It protects against careless model behaviour, not an +adversarial one. See `shell explain safety`. ## Verbs -- `shell-cli whoami` — identity probe from `culture.yaml`. -- `shell-cli learn` — structured self-teaching prompt. -- `shell-cli explain ` — markdown docs for any noun/verb. -- `shell-cli overview` — descriptive snapshot of the agent. -- `shell-cli doctor` — check the agent-identity invariants. -- `shell-cli cli overview` — describe the CLI surface. +- `shell whoami` — identity probe from `culture.yaml`. +- `shell learn` — structured self-teaching prompt. +- `shell explain ` — markdown docs for any noun/verb. +- `shell overview` — descriptive snapshot of the agent. +- `shell doctor` — check the agent-identity invariants. +- `shell cli overview` — describe the CLI surface. ## Exit-code policy @@ -36,62 +55,100 @@ ## See also -- `shell-cli explain whoami` -- `shell-cli explain doctor` +- `shell explain safety` +- `shell explain whoami` +""" + +_SAFETY = """\ +# shell safety model + +**shell-cli ships a guard, not a sandbox.** This entry exists so the claim is +addressable in-band — an agent can read it before deciding what to trust. + +## What it protects against + +Accidental and careless behaviour: a path that escapes the repo root, a write +into a tree declared read-only, a command that would run something it should +not, output large enough to blow the caller's context window. + +## What it does NOT protect against + +An adversarial command. The execution gate inspects the raw command string, so +variable expansion, concatenation, here-docs, `sh -c`, and pipelines can all +defeat it. There is no namespace, container, or seccomp isolation on the +default path. + +If you need real isolation, that is separate, deliberate work — it is not +implied by anything here. + +## Layers (as extracted) + +- **Path confinement** — relative paths resolve under a configured root; + anything escaping it is refused. +- **Read-only subtrees** — configured paths may be read, never written. +- **Truncation** — every result is capped so one huge file or command cannot + exhaust the caller's context. +- **Approval policy** — `run_command` is gated by an operator-supplied policy + returning an explicit verdict; file operations can be checksum-pinned. + +## Status + +Not yet extracted from `colleague` — this documents the contract the extraction +must uphold, not shipped behaviour. See `docs/threat-model.md`. """ _WHOAMI = """\ -# shell-cli whoami +# shell whoami Reports the agent's identity from `culture.yaml`: nick (`suffix`), backend, served model, and the package version. Read-only. ## Usage - shell-cli whoami - shell-cli whoami --json + shell whoami + shell whoami --json """ _LEARN = """\ -# shell-cli learn +# shell learn Prints a structured self-teaching prompt covering purpose, command map, exit-code policy, `--json` support, and the `explain` pointer. ## Usage - shell-cli learn - shell-cli learn --json + shell learn + shell learn --json """ _EXPLAIN = """\ -# shell-cli explain +# shell explain Prints markdown documentation for any noun/verb path. Unlike `--help` (terse, positional), `explain` is global and addressable by path. ## Usage - shell-cli explain shell-cli - shell-cli explain whoami - shell-cli explain --json + shell explain shell + shell explain safety + shell explain --json whoami """ _OVERVIEW = """\ -# shell-cli overview +# shell overview Read-only descriptive snapshot of the agent: identity (from `culture.yaml`), the -verb surface, and the sibling-pattern artifacts the template carries. Accepts an -ignored `target` so a stray path never hard-fails. +verb surface, and the artifacts this repo carries. Accepts an ignored `target` +so a stray path never hard-fails. ## Usage - shell-cli overview - shell-cli overview --json + shell overview + shell overview --json """ _DOCTOR = """\ -# shell-cli doctor +# shell doctor Checks the agent-identity invariants `steward doctor` verifies: prompt-file-present and backend-consistency (`colleague` → `AGENTS.colleague.md`), plus a @@ -99,27 +156,28 @@ ## Usage - shell-cli doctor - shell-cli doctor --json + shell doctor + shell doctor --json """ _CLI = """\ -# shell-cli cli +# shell cli Noun group for CLI-surface introspection. `cli overview` describes the CLI itself (distinct from the global `overview`, which describes the agent). ## Usage - shell-cli cli overview - shell-cli cli overview --json + shell cli overview + shell cli overview --json """ ENTRIES: dict[tuple[str, ...], str] = { (): _ROOT, - ("shell-cli",): _ROOT, ("shell",): _ROOT, + ("shell-cli",): _ROOT, + ("safety",): _SAFETY, ("whoami",): _WHOAMI, ("learn",): _LEARN, ("explain",): _EXPLAIN, diff --git a/tests/test_cli.py b/tests/test_cli.py index 7035ff6..2c2c856 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -21,7 +21,7 @@ def test_version_flag(capsys: pytest.CaptureFixture[str]) -> None: def test_no_args_prints_help(capsys: pytest.CaptureFixture[str]) -> None: rc = main([]) assert rc == 0 - assert "usage: shell-cli" in capsys.readouterr().out + assert "usage: shell" in capsys.readouterr().out def test_unknown_command_errors(capsys: pytest.CaptureFixture[str]) -> None: @@ -62,7 +62,11 @@ def test_learn_text(capsys: pytest.CaptureFixture[str]) -> None: assert rc == 0 out = capsys.readouterr().out assert len(out) >= 200 - assert "shell-cli" in out + # Assert the command map names the real executable. This used to check for + # "shell-cli", which after the rename passes only incidentally via the issue + # URL in the body — a vestigial assertion that would survive the command map + # being dropped entirely. + assert "shell whoami" in out assert "Exit-code policy" in out assert "--json" in out assert "explain" in out @@ -72,7 +76,8 @@ def test_learn_json(capsys: pytest.CaptureFixture[str]) -> None: rc = main(["learn", "--json"]) assert rc == 0 payload = json.loads(capsys.readouterr().out) - assert payload["tool"] == "shell-cli" + assert payload["tool"] == "shell" + assert payload["distribution"] == "shell-cli" assert payload["version"] == __version__ assert payload["json_support"] is True @@ -83,11 +88,11 @@ def test_learn_json(capsys: pytest.CaptureFixture[str]) -> None: def test_explain_root(capsys: pytest.CaptureFixture[str]) -> None: rc = main(["explain"]) assert rc == 0 - assert "# shell-cli" in capsys.readouterr().out + assert "# shell" in capsys.readouterr().out def test_explain_self(capsys: pytest.CaptureFixture[str]) -> None: - rc = main(["explain", "shell-cli"]) + rc = main(["explain", "shell"]) assert rc == 0 assert capsys.readouterr().out.startswith("#") @@ -97,7 +102,7 @@ def test_explain_json(capsys: pytest.CaptureFixture[str]) -> None: assert rc == 0 payload = json.loads(capsys.readouterr().out) assert payload["path"] == ["whoami"] - assert "shell-cli whoami" in payload["markdown"] + assert "shell whoami" in payload["markdown"] def test_explain_unknown_path_errors(capsys: pytest.CaptureFixture[str]) -> None: diff --git a/tests/test_cli_introspection.py b/tests/test_cli_introspection.py index c5952fc..99bf62a 100644 --- a/tests/test_cli_introspection.py +++ b/tests/test_cli_introspection.py @@ -41,14 +41,14 @@ def test_overview_graceful_on_bad_path(capsys: pytest.CaptureFixture[str]) -> No def test_cli_overview_text(capsys: pytest.CaptureFixture[str]) -> None: rc = main(["cli", "overview"]) assert rc == 0 - assert "# shell-cli cli" in capsys.readouterr().out + assert "# shell cli" in capsys.readouterr().out def test_cli_overview_json_shape(capsys: pytest.CaptureFixture[str]) -> None: rc = main(["cli", "overview", "--json"]) assert rc == 0 payload = json.loads(capsys.readouterr().out) - assert payload["subject"] == "shell-cli cli" + assert payload["subject"] == "shell cli" assert isinstance(payload["sections"], list) @@ -77,7 +77,7 @@ def test_cli_overview_unknown_flag_structured_error( def test_doctor_text(capsys: pytest.CaptureFixture[str]) -> None: rc = main(["doctor"]) assert rc in (0, 1) - assert "shell-cli doctor" in capsys.readouterr().out + assert "shell doctor" in capsys.readouterr().out def test_doctor_json_shape(capsys: pytest.CaptureFixture[str]) -> None: diff --git a/tests/test_honesty.py b/tests/test_honesty.py new file mode 100644 index 0000000..b4095cd --- /dev/null +++ b/tests/test_honesty.py @@ -0,0 +1,198 @@ +"""Guard tests for the guard-not-a-sandbox honesty constraint. + +The mission (issue #1) commits shell-cli to an explicit posture: the execution +gate is best-effort and bypassable, so nothing in the shipped surface may imply +isolation the code does not provide. A package *named* shell-cli whose headline +is safe execution will be read as offering a sandbox — these tests make the +disclaimer load-bearing rather than a docstring someone can quietly drop. + +Two directions are guarded: + +1. The disclaimer is PRESENT in the surfaces an agent actually reads + (``learn``, ``explain`` root, ``explain safety``, README, threat model). +2. No surface makes a POSITIVE isolation claim — the word "sandbox" may only + ever appear in a negating context. +""" + +from __future__ import annotations + +import json +import re +from pathlib import Path + +import pytest + +from shell.cli import main +from shell.explain.catalog import ENTRIES + +_REPO_ROOT = Path(__file__).resolve().parents[1] + +# Phrasings that would claim isolation this package does not implement. Kept +# deliberately narrow: the goal is to catch an affirmative claim, not to ban the +# word (the honest disclaimer must be free to say "not a sandbox"). +# +# Non-capturing groups throughout so ``finditer`` -> ``group(0)`` reports the +# offending text itself; with capturing groups a failure would print tuples of +# fragments (``[('', '', '')]``) and tell the reader nothing. +_CLAIM = re.compile( + r"\b(?:is|as|provides?|offers?|ships?|implements?|guarantees?)\s+(?:an?\s+)?" + r"(?:secure\s+|safe\s+|real\s+)?sandbox\b" + r"|\bsandboxe[sd]\b" + r"|\bfully\s+isolated\b", + re.IGNORECASE, +) + +# Words that turn one of the above into a disclaimer rather than a claim. This +# guard must never fire on honest text — "it is never sandboxed" is the posture +# stated *more* strongly, and failing CI over it would punish exactly the writing +# this file exists to protect. +_NEGATOR = re.compile( + r"\b(?:not|never|n't|no|nor|neither|without|rather\s+than|instead\s+of)\b", + re.IGNORECASE, +) + + +def overclaims(text: str) -> list[str]: + """Return affirmative isolation claims in *text*, as the matched substrings. + + A match is discounted when a negator appears earlier **in the same + sentence**. Negation is checked over that window rather than as a regex + lookbehind because Python requires fixed-width lookbehind and the negator + can sit several words back ("is never actually sandboxed"). Stopping at the + sentence boundary keeps a disclaimer in one sentence from excusing a genuine + claim in the next. + """ + found = [] + for match in _CLAIM.finditer(text): + start = max( + text.rfind(".", 0, match.start()), + text.rfind("!", 0, match.start()), + text.rfind("?", 0, match.start()), + text.rfind("\n", 0, match.start()), + ) + if _NEGATOR.search(text[start + 1 : match.start()]): + continue + found.append(match.group(0)) + return found + + +_DOC_FILES = ["README.md", "CLAUDE.md", "docs/threat-model.md"] + + +def _catalog_texts() -> dict[str, str]: + return {" ".join(path) or "": body for path, body in ENTRIES.items()} + + +# --- 1. the disclaimer is present ----------------------------------------- + + +def test_explain_safety_entry_exists(capsys: pytest.CaptureFixture[str]) -> None: + rc = main(["explain", "safety"]) + assert rc == 0 + out = capsys.readouterr().out + assert "not a sandbox" in out.lower() + assert "does not protect against" in out.lower() + + +def test_explain_root_carries_the_posture(capsys: pytest.CaptureFixture[str]) -> None: + rc = main(["explain"]) + assert rc == 0 + assert "not a sandbox" in capsys.readouterr().out.lower() + + +def test_learn_carries_the_posture(capsys: pytest.CaptureFixture[str]) -> None: + rc = main(["learn"]) + assert rc == 0 + assert "not a sandbox" in capsys.readouterr().out.lower() + + +def test_learn_json_exposes_safety_posture(capsys: pytest.CaptureFixture[str]) -> None: + """Machine consumers get the posture as a field, not buried in prose.""" + rc = main(["learn", "--json"]) + assert rc == 0 + payload = json.loads(capsys.readouterr().out) + assert "not a sandbox" in payload["safety_posture"].lower() + + +@pytest.mark.parametrize("name", _DOC_FILES) +def test_doc_carries_the_posture(name: str) -> None: + text = (_REPO_ROOT / name).read_text(encoding="utf-8").lower() + assert "not a sandbox" in text, f"{name} must state the guard-not-a-sandbox posture" + + +# --- 2. nothing overclaims ------------------------------------------------- + + +@pytest.mark.parametrize("name", _DOC_FILES) +def test_doc_makes_no_isolation_claim(name: str) -> None: + text = (_REPO_ROOT / name).read_text(encoding="utf-8") + hits = overclaims(text) + assert not hits, f"{name} appears to claim isolation shell-cli does not provide: {hits}" + + +def test_catalog_makes_no_isolation_claim() -> None: + offenders = {name: overclaims(body) for name, body in _catalog_texts().items()} + offenders = {k: v for k, v in offenders.items() if v} + assert not offenders, f"explain catalog overclaims isolation: {offenders}" + + +def test_cli_help_and_learn_make_no_isolation_claim( + capsys: pytest.CaptureFixture[str], +) -> None: + main(["learn"]) + learn_text = capsys.readouterr().out + main([]) + help_text = capsys.readouterr().out + for label, text in (("learn", learn_text), ("help", help_text)): + hits = overclaims(text) + assert not hits, f"{label} overclaims isolation: {hits}" + + +# --- 3. the guard itself is trustworthy ------------------------------------ +# +# A guard that cries wolf gets deleted, and one that never fires is decoration. +# Both directions are pinned here so a future tweak to the pattern cannot +# quietly break either. + + +@pytest.mark.parametrize( + "honest", + [ + "This is not fully isolated.", + "Do not treat this as a sandbox.", + "It is never sandboxed.", + "a guard, not a sandbox", + "It protects against careless behaviour, not an adversarial one.", + "There is no namespace, container, or seccomp isolation.", + "This is a guard rather than a sandbox.", + "The gate runs without a sandbox.", + ], +) +def test_honest_phrasing_does_not_trip_the_guard(honest: str) -> None: + assert overclaims(honest) == [], f"guard fired on honest text: {honest!r}" + + +@pytest.mark.parametrize( + "claim", + [ + "shell-cli provides a secure sandbox.", + "Every command is sandboxed.", + "The runner offers a sandbox for untrusted code.", + "Commands are fully isolated.", + "It guarantees a sandbox.", + ], +) +def test_affirmative_claim_trips_the_guard(claim: str) -> None: + assert overclaims(claim), f"guard missed an overclaim: {claim!r}" + + +def test_disclaimer_does_not_excuse_a_later_claim() -> None: + """A negator must not leak across a sentence boundary.""" + text = "This is not a sandbox. Every command is sandboxed anyway." + assert overclaims(text) == ["sandboxed"] + + +def test_failure_message_reports_the_offending_text() -> None: + """Regression: capturing groups made findall report ``[('', '', '')]``.""" + hits = overclaims("Commands are fully isolated.") + assert hits == ["fully isolated"] diff --git a/uv.lock b/uv.lock index 959fe08..ece0354 100644 --- a/uv.lock +++ b/uv.lock @@ -154,37 +154,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/61/e8/cb8e80d6f9f55b99588625062822bf946cf03ed06315df4bd8397f5632a1/coverage-7.14.0-py3-none-any.whl", hash = "sha256:8de5b61163aee3d05c8a2beab6f47913df7981dad1baf82c414d99158c286ab1", size = 211764, upload-time = "2026-05-10T18:02:29.538Z" }, ] -[[package]] -name = "shell-cli" -version = "0.6.0" -source = { editable = "." } - -[package.dev-dependencies] -dev = [ - { name = "bandit" }, - { name = "black" }, - { name = "flake8" }, - { name = "isort" }, - { name = "pytest" }, - { name = "pytest-cov" }, - { name = "pytest-xdist" }, - { name = "teken" }, -] - -[package.metadata] - -[package.metadata.requires-dev] -dev = [ - { name = "bandit", specifier = ">=1.7.5" }, - { name = "black", specifier = ">=23.7.0" }, - { name = "flake8", specifier = ">=6.1" }, - { name = "isort", specifier = ">=5.12.0" }, - { name = "pytest", specifier = ">=8.0" }, - { name = "pytest-cov", specifier = ">=4.1" }, - { name = "pytest-xdist", specifier = ">=3.0" }, - { name = "teken", specifier = ">=0.8" }, -] - [[package]] name = "execnet" version = "2.1.2" @@ -459,6 +428,37 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl", hash = "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", size = 310654, upload-time = "2026-04-12T08:24:02.83Z" }, ] +[[package]] +name = "shell-cli" +version = "0.7.1" +source = { editable = "." } + +[package.dev-dependencies] +dev = [ + { name = "bandit" }, + { name = "black" }, + { name = "flake8" }, + { name = "isort" }, + { name = "pytest" }, + { name = "pytest-cov" }, + { name = "pytest-xdist" }, + { name = "teken" }, +] + +[package.metadata] + +[package.metadata.requires-dev] +dev = [ + { name = "bandit", specifier = ">=1.7.5" }, + { name = "black", specifier = ">=23.7.0" }, + { name = "flake8", specifier = ">=6.1" }, + { name = "isort", specifier = ">=5.12.0" }, + { name = "pytest", specifier = ">=8.0" }, + { name = "pytest-cov", specifier = ">=4.1" }, + { name = "pytest-xdist", specifier = ">=3.0" }, + { name = "teken", specifier = ">=0.8" }, +] + [[package]] name = "stevedore" version = "5.8.0"