From 576e2ed200251b04f06894c362786e7c6d9e0915 Mon Sep 17 00:00:00 2001 From: CoderDeltaLAN Date: Thu, 18 Jun 2026 22:13:35 +0100 Subject: [PATCH] docs: add output and exit-code contract --- CHANGELOG.md | 5 +- README.md | 2 +- docs/EXIT-CODES.md | 116 +++++++++++++++++++ docs/OUTPUTS.md | 281 ++++++++++++++++++++++++++++++++++++++++----- 4 files changed, 375 insertions(+), 29 deletions(-) create mode 100644 docs/EXIT-CODES.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 332c6c2..afd38de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,10 @@ This project has a published GitHub Release line, but no stable support or API g ## [Unreleased] -No unreleased changes. +### Added + +- Added v0.3 architecture and roadmap planning documentation. +- Added initial output and exit-code contract documentation for existing `check` and `init` behavior and planned v0.3 `doctor`, `budget`, and `explain` commands. ## [0.2.3] - 2026-06-18 diff --git a/README.md b/README.md index 7ae44e2..13c0f05 100644 --- a/README.md +++ b/README.md @@ -177,7 +177,7 @@ The `v0.2.0` GitHub Release introduced this governance rule set. The published ` For detailed rule purpose, evidence, limits, and false-positive notes, see `docs/RULES.md`. -For CLI output examples in console, JSON, and Markdown formats, see `docs/OUTPUTS.md`. +For CLI output examples in console, JSON, and Markdown formats, see `docs/OUTPUTS.md`. For the CLI exit-code contract, see `docs/EXIT-CODES.md`. --- diff --git a/docs/EXIT-CODES.md b/docs/EXIT-CODES.md new file mode 100644 index 0000000..8a72fd5 --- /dev/null +++ b/docs/EXIT-CODES.md @@ -0,0 +1,116 @@ +# Exit Codes + +This document defines the initial `agent-rules-kit` CLI exit-code contract. + +The contract is project-local and pre-v1.0. It is meant to guide tests, documentation, and future command implementation. It is not a stable public API guarantee before v1.0. + +## General rules + +- `0` means the command completed successfully according to its command-specific rules. +- `1` is reserved for supported no-result or review-needed states when documented by the command. +- `2` means invalid command-line usage, invalid repository input, unsupported command input, or another documented operational error. +- Findings do not currently make `check` fail. +- A successful exit code does not prove that a repository is safe, complete, compliant, production-ready, or free of secrets. + +## Current implemented exit codes + +### Global options + +| Invocation | Exit code | Meaning | +| --- | ---: | --- | +| `agent-rules-kit --version` | `0` | Version printed successfully. | +| `agent-rules-kit` | `0` | Help printed successfully because no subcommand was provided. | +| invalid option or invalid subcommand | `2` | Command-line usage error handled by argparse. | + +### `check` + +| Condition | Exit code | Stdout | Stderr | +| --- | ---: | --- | --- | +| Supported instruction files found | `0` | Console, JSON, or Markdown report | Empty unless lower-level runtime fails unexpectedly | +| No supported instruction files found | `1` | Console, JSON, or Markdown no-result report | Empty unless lower-level runtime fails unexpectedly | +| Invalid repository input or supported operational error | `2` | JSON/Markdown error payload when those formats were requested; empty console stdout | Console error message on stderr | +| Invalid `--format` value or command-line usage | `2` | Argparse-dependent | Argparse-dependent | + +Notes: + +- `check` returns `0` even when warning findings are present. +- `check` returns `1` only for the supported no-instruction-files state. +- `check` returns `2` for repository validation errors raised by discovery and for argparse usage errors. + +### `init --dry-run` + +| Condition | Exit code | Stdout | Stderr | +| --- | ---: | --- | --- | +| Plan completed | `0` | Dry-run plan | Empty unless lower-level runtime fails unexpectedly | +| Invalid repository input or symlink refusal | `2` | Empty | Error message | +| Missing mode, conflicting modes, or command-line usage error | `2` | Argparse-dependent or empty | Error message or argparse-dependent | + +Notes: + +- `init --dry-run` is read-only. +- It must not modify repository files. + +### `init --write` + +| Condition | Exit code | Stdout | Stderr | +| --- | ---: | --- | --- | +| Explicit write completed | `0` | Write summary | Empty unless lower-level runtime fails unexpectedly | +| Invalid repository input or symlink refusal | `2` | Empty | Error message | +| Missing mode, conflicting modes, or command-line usage error | `2` | Argparse-dependent or empty | Error message or argparse-dependent | + +Notes: + +- `init --write` is the only current write mode. +- It backs up an existing root `AGENTS.md` before replacement. +- It must remain explicit and separate from read-only diagnosis commands. + +## Planned v0.3 exit-code direction + +The following commands are not implemented yet. Their exit-code contracts are design targets for future implementation phases. + +### `doctor` + +Planned direction: + +| Condition | Exit code | +| --- | ---: | +| Repository diagnosis completed with supported instruction files | `0` | +| No supported instruction files found, if aligned with `check` | `1` | +| Invalid repository input or command-line usage error | `2` | + +The implementation phase must decide and test whether no supported instruction files should mirror `check` with `1`. + +### `budget` + +Planned direction: + +| Condition | Exit code | +| --- | ---: | +| Budget approximation completed for supported input | `0` | +| No supported instruction files found, if the command operates only on discovered instruction files | `1` | +| Invalid repository input or command-line usage error | `2` | + +The implementation must not promise tokenizer-specific exactness unless a later explicit tokenizer phase is approved. + +### `explain` + +Planned direction: + +| Condition | Exit code | +| --- | ---: | +| Known rule explained or known rules listed | `0` | +| Unknown rule ID, invalid input, or command-line usage error | `2` | + +Unknown rule IDs should fail predictably. They should not silently produce generic guidance. + +## Maintenance rules + +Update this document when: + +- a command changes exit-code behavior; +- a new command is added; +- a new supported no-result state is introduced; +- command-line usage handling changes; +- JSON/Markdown error payload behavior changes. + +Do not update this document after code changes by guesswork. Verify behavior with real commands, tests, or fixtures before changing the contract. diff --git a/docs/OUTPUTS.md b/docs/OUTPUTS.md index 1a1db06..c730448 100644 --- a/docs/OUTPUTS.md +++ b/docs/OUTPUTS.md @@ -1,32 +1,265 @@ -# Output Examples +# Output Contract and Examples -This document shows representative `agent-rules-kit` CLI outputs generated from fixtures in the current repository state. +This document describes the current `agent-rules-kit` CLI output contract and representative output examples. -`v0.2.0` release artifacts may differ from unreleased `main` if post-release fixes have not yet been cut into a new release. - -The examples document output shape, exit codes, and redaction expectations. They are not a claim that the analyzed repository is safe, complete, production-ready, compliant, or free of secrets. +The contract is intentionally narrow. It documents implemented behavior on current `main` and the planned v0.3 command direction without promising a stable public API before v1.0. ## Scope -Covered commands and formats: +Implemented command surface: + +- `agent-rules-kit --version`; +- `agent-rules-kit check`; +- `agent-rules-kit init --dry-run`; +- `agent-rules-kit init --write`. + +Planned v0.3 command surface: + +- `agent-rules-kit doctor`; +- `agent-rules-kit budget`; +- `agent-rules-kit explain`. + +The planned commands are not implemented yet. Their output contracts are design targets for future phases and must not be documented as available behavior until their implementation phases are merged. + +## Contract status + +This is an initial project-local CLI contract. + +It is: + +- stable enough to guide repository tests and golden output fixtures; +- allowed to evolve before v1.0 with changelog notes; +- not a stable public API guarantee; +- not a claim that clean output proves safety, completeness, compliance, or production readiness. + +A clean report only means the implemented checks completed according to their documented behavior. -- `agent-rules-kit check` console output; -- `agent-rules-kit check --format json`; -- `agent-rules-kit check --format markdown`. +## Output channels -These examples do not document `init --write` side effects. Write-mode behavior remains explicit, local, and backup-oriented. +Current behavior: + +- normal command output is written to stdout; +- `check` console operational errors are written to stderr; +- `init` validation and operational errors are written to stderr; +- Python `argparse` usage errors are handled before command dispatch and follow argparse behavior. + +Future behavior should preserve that distinction unless a dedicated phase changes it with tests. + +## Implemented command and format matrix + +| Command | Formats | Implemented | Notes | +| --- | --- | --- | --- | +| `--version` | console | yes | Prints package version and exits `0`. | +| `check` | console | yes | Human-readable repository diagnosis. | +| `check --format json` | JSON | yes | Machine-readable payload. | +| `check --format markdown` | Markdown | yes | Human-readable Markdown report. | +| `init --dry-run` | console | yes | Read-only plan; no files modified. | +| `init --write` | console | yes | Explicit write mode with backup behavior for existing root `AGENTS.md`. | +| `doctor` | to be defined | no | Planned v0.3 read-only repository summary. | +| `budget` | to be defined | no | Planned v0.3 read-only local size/context-pressure approximation. | +| `explain` | to be defined | no | Planned v0.3 local rule explanation command. | ## Exit codes -`check` uses these documented exit codes: +The authoritative exit-code documentation is `docs/EXIT-CODES.md`. + +Summary for current implemented commands: + +| Command | Exit code | Meaning | +| --- | ---: | --- | +| `--version` | `0` | Version printed successfully. | +| no command | `0` | Help printed successfully. | +| `check` | `0` | Supported instruction files were found and the command completed. Findings may still be present. | +| `check` | `1` | No supported instruction files were found. | +| `check` | `2` | Invalid repository input, operational error, or command-line usage error. | +| `init --dry-run` | `0` | Plan completed successfully without writing files. | +| `init --write` | `0` | Explicit write completed successfully. | +| `init` | `2` | Missing mode, conflicting modes, invalid repository input, symlink refusal, or command-line usage error. | + +## JSON contract for `check` + +Current `check --format json` emits one JSON object. + +Current top-level fields: + +| Field | Type | Meaning | +| --- | --- | --- | +| `command` | string | Current command name, currently `check`. | +| `status` | string | `ok`, `no_instruction_files`, or `error`. | +| `repository` | string | Repository path as provided/resolved by the command, redacted if needed. | +| `instruction_files` | array | Supported instruction files discovered by the command. | +| `summary` | object | Count summary. | +| `findings` | array | Governance and system findings. | +| `error` | object or null | Error payload for supported error states. | + +Current `instruction_files` item fields: + +| Field | Type | Meaning | +| --- | --- | --- | +| `path` | string | Repository-relative path, redacted if needed. | +| `kind` | string | Supported instruction file kind. | + +Current `summary` fields: + +| Field | Type | Meaning | +| --- | --- | --- | +| `supported_instruction_file_count` | integer | Number of supported instruction files discovered. | +| `finding_count` | integer | Number of findings emitted. | + +Current `findings` item fields: + +| Field | Type | Required | Meaning | +| --- | --- | --- | --- | +| `rule_id` | string | yes | Finding rule identifier. | +| `severity` | string | yes | Finding severity. | +| `message` | string | yes | Human-readable finding message, redacted if needed. | +| `path` | string | no | Repository-relative affected path, redacted if needed. | +| `line` | integer | no | 1-based line number. | +| `column` | integer | no | 1-based column number. | +| `evidence` | string | no | Finding evidence, redacted if needed. | + +Current `error` object fields: + +| Field | Type | Meaning | +| --- | --- | --- | +| `message` | string | Human-readable error message, redacted if needed. | + +Ordering notes: + +- JSON keys are currently sorted by key when emitted. +- Discovered instruction files use deterministic discovery order. +- Governance findings use the current rule evaluation order. +- Future changes to ordering must be tested before being treated as contract. + +## Console contract for `check` + +Current successful console output with supported files includes: + +- command header; +- supported instruction file count; +- one line per supported instruction file; +- optional `Findings:` section; +- one line per finding. + +Current no-supported-files console output includes: + +- command header; +- `No supported agent instruction files found.` + +Console output is intended for humans. It is not the machine-readable contract. Use JSON for automation. + +## Markdown contract for `check` + +Current Markdown output includes: + +- `# agent-rules-kit check`; +- repository line; +- status line; +- supported instruction file count; +- finding count; +- instruction-file table when files exist; +- findings table when findings exist; +- error text when a supported error state occurs. + +Markdown output is intended for reports and documentation, not strict machine parsing. -- `0`: supported instruction files were found; -- `1`: no supported instruction files were found; -- `2`: invalid input or operational error. +## Init output contract -A successful exit code does not mean that the repository is safe or production-ready. It only means that the command completed according to its documented behavior. +Current `init --dry-run` console output includes: -## Console output: supported instruction files found +- command header; +- `Mode: dry-run`; +- `No files will be modified.`; +- planned file actions. + +Current `init --write` console output includes: + +- command header; +- `Mode: write`; +- modified file actions; +- backup path when an existing root `AGENTS.md` was backed up. + +Current `init` does not support JSON or Markdown output. + +`init --dry-run` is read-only. `init --write` is explicit write mode and must remain separate from read-only diagnosis commands. + +## Planned v0.3 command contracts + +These commands are design targets. They are not available until their dedicated implementation phases are merged. + +### `doctor` + +Planned purpose: + +- read-only repository-level diagnosis summary; +- reuse discovery and governance findings; +- summarize supported instruction files, finding counts, and high-level review status. + +Planned output direction: + +- console summary first; +- JSON only if the implementation phase explicitly defines and tests it; +- no branch protection, CI, dependency, or security certification audit in v0.3. + +Planned exit-code direction: + +- `0`: diagnosis completed and supported instruction files were found; +- `1`: no supported instruction files were found, if this mirrors `check`; +- `2`: invalid input or command-line usage error. + +### `budget` + +Planned purpose: + +- read-only local size/context-pressure approximation; +- report deterministic local metrics such as bytes, characters, lines, approximate words, file count, and totals. + +Planned output direction: + +- no model-specific token-count promise; +- no remote tokenization; +- no LLM call; +- no pricing estimate; +- use the word approximation for non-token metrics. + +Planned exit-code direction: + +- `0`: budget calculation completed for supported input; +- `1`: no supported instruction files were found, if the command operates on discovered instruction files; +- `2`: invalid input or command-line usage error. + +### `explain` + +Planned purpose: + +- explain known rule IDs and their limits from local rule metadata or documentation-backed text; +- optionally list known rules. + +Planned output direction: + +- local explanations only; +- no external documentation fetch; +- no LLM-generated explanations; +- unsupported rule IDs must fail predictably. + +Planned exit-code direction: + +- `0`: explanation or rule list completed; +- `2`: unknown rule ID, invalid input, or command-line usage error. + +## Redaction expectations + +Output paths, messages, and evidence payloads are passed through supported secret-like value redaction where the CLI emits them. + +Users should still avoid placing real secrets, tokens, credentials, customer data, private URLs, or sensitive values in fixtures, examples, logs, screenshots, issues, pull requests, or documentation. + +Redaction is a risk-reduction control. It is not complete secret scanning. + +## Representative examples + +These examples are generated from fixtures in the repository and should be re-checked when CLI behavior changes. + +### Console output: supported instruction files found Command: @@ -58,7 +291,7 @@ Stderr: -## Console output: no supported instruction files +### Console output: no supported instruction files Command: @@ -75,7 +308,7 @@ Stderr: -## JSON output: governance findings +### JSON output: governance findings Command: @@ -132,7 +365,7 @@ Stderr: -## Markdown output: governance findings +### Markdown output: governance findings Command: @@ -165,14 +398,8 @@ Stderr: -## Redaction expectations - -Output paths, messages, and evidence payloads are passed through secret-like value redaction where the CLI emits them. - -Users should still avoid placing real secrets, tokens, credentials, customer data, or private URLs in fixtures, examples, logs, screenshots, issues, or pull requests. - ## Maintainer guidance -When output behavior changes, update this document together with the corresponding tests and fixtures. +When output behavior changes, update this document together with corresponding tests and fixtures. -Do not update examples by hand if the CLI behavior changed. Regenerate or re-check examples against the current code so the documentation stays honest. +Do not update examples by hand if CLI behavior changed. Regenerate or re-check examples against the current code so the documentation stays honest.