diff --git a/.claude/commands/cdd-pre-pr.md b/.claude/commands/cdd-pre-pr.md index 7f13187..5eedcee 100644 --- a/.claude/commands/cdd-pre-pr.md +++ b/.claude/commands/cdd-pre-pr.md @@ -53,7 +53,17 @@ Check and **update** documentation based on the changes: Read each relevant doc and compare against the actual code changes. Fix discrepancies directly when they are reconciliation (the doc is out of date relative to what landed). Ask before applying structural changes (adding new doc files, restructuring an existing doc). -## 5. CI improvement check (conditional) +## 5. Test coverage reconciliation + +For each behavioural change in the diff (a new function, a new branch, changed output, a fixed bug), check whether it is covered by a test. This is the recurring guardrail behind the "tested behaviour" row of `doc/knowledge_base/engineering-practices.md`. + +- **If the project has a test command** ("tested behaviour" marked *enforced*): confirm a test exercises the new behaviour. If a behavioural change landed with no accompanying test, flag it — the default expectation is that new behaviour ships with a test. +- **If a change is deliberately untested** (a throwaway script, generated code, a spike): that is allowed, but it must be *intentional and recorded*, not silent. State the reason in the PR summary. +- **If the project has no test command yet** ("tested behaviour" still *expected*): do not invent a framework. Note that the change shipped untested because there is no test harness, and confirm that standing one up is tracked as a roadmap task. If this change is exactly the kind of behaviour that motivates a first test, say so and let the user decide whether to pull that task forward. + +This step asks a question and records the answer; it does not mandate a specific framework, a coverage threshold, or that every change be tested. "Not tested, and here is why" is a valid, recorded outcome. Surface it — do not block on it. + +## 6. CI improvement check (conditional) If, and only if, the change introduces a category of work that the existing CI does not cover, propose specific improvements to the user. Examples that should trigger a proposal: @@ -64,7 +74,7 @@ If, and only if, the change introduces a category of work that the existing CI d Do **not** propose generic CI improvements every run. The default is silence. If you do propose, the user has two options: apply now in this PR, or defer as a new roadmap task. Apply only on approval. -## 6. Upstream drift check +## 7. Upstream drift check ```bash git fetch origin main @@ -86,9 +96,9 @@ The script renders the template through `bootstrap-cdd-project.sh --stage` with If the script exits 0, report "no drift" and continue. If it reports divergence, present each diff to the user; for each, the user decides whether to reconcile the repo copy, reconcile the template copy, or record a justified exception (a whitelist entry or a `cdd-only` fence). Apply fixes only on user approval. Do not auto-edit either tree from this step. -When presenting the step 7 checklist, append a `- [ ] Command-set drift clean` line to it. +When presenting the step 8 checklist, append a `- [ ] Command-set drift clean` line to it. -## 7. Summary +## 8. Summary Present a checklist summary: @@ -105,6 +115,7 @@ Present a checklist summary: - [ ] CLAUDE.md up to date - [ ] README up to date - [ ] Roadmap up to date +- [ ] New behaviour tested (or untested-with-reason recorded) - [ ] CI gaps surfaced: none / proposed (list them) - [ ] No upstream drift (or: /cdd-merge-main recommended) - [ ] Reconciliation edits committed @@ -112,9 +123,9 @@ Present a checklist summary: Mark each item as pass ✓ or needs attention ✗ with details. -## 8. Commit reconciliation edits +## 9. Commit reconciliation edits -Commit the documentation reconciliation edits this session made in steps 3–6 (architecture/feature docs, CLAUDE.md, README, the coding standard, and the roadmap). This is a local commit only — **no push**. Pushing happens, if at all, in step 9. +Commit the documentation reconciliation edits this session made in steps 3–7 (architecture/feature docs, CLAUDE.md, README, the coding standard, and the roadmap). This is a local commit only — **no push**. Pushing happens, if at all, in step 10. First check the entry snapshot from step 1: @@ -122,13 +133,13 @@ First check the entry snapshot from step 1: - **Otherwise**, commit only the files this session edited. Add them by path — do not `git add -A`: ```bash -git add +git add git commit -m '' ``` Follow the repo's commit conventions from CLAUDE.md. Print a one-line summary of the commit (subject + files included). If nothing was reconciled (no edits this session), say so and skip the commit. -## 9. Open PR (optional) +## 10. Open PR (optional) After the checklist, offer to open the PR. This is human-gated — never open a PR without explicit confirmation. @@ -140,7 +151,7 @@ gh auth status && git remote get-url origin # origin should be a github.com UR If either is missing, say so in one line and skip this step (the checklist above still stands). -If §6 found upstream drift, restate the recommendation to run `/cdd-merge-main` before opening the PR, and let the user decide whether to proceed anyway. +If §7 found upstream drift, restate the recommendation to run `/cdd-merge-main` before opening the PR, and let the user decide whether to proceed anyway. Ask: **"Open a PR now?"** Do not pre-show a title or body, and do not print manual `gh` instructions — just ask whether to proceed. diff --git a/CLAUDE.md b/CLAUDE.md index 7751d0b..f626216 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,6 +8,7 @@ CDD is a human-in-the-loop workflow for evolving software projects together with | ---------------------------------------------------- | ------------------------------------------------- | | Process document (philosophy, lifecycle, edit rules) | `doc/knowledge_base/claude-driven-development.md` | | Implementation roadmap | `doc/knowledge_base/roadmap.md` | +| Engineering practices (enforced vs expected) | `doc/knowledge_base/engineering-practices.md` | | Documentation map | `doc/index.md` | | Architecture of this repo | `doc/architecture/index.md` | | Architecture decision records | `doc/architecture/adr/` (Nygard style) | @@ -64,7 +65,7 @@ When `/cdd-pre-pr` runs in this repo, the "build / format / lint / test" gates c | Directory | Purpose | | ---------------------------------- | --------------------------------------------------------- | -| `doc/knowledge_base/` | Process doc, roadmap, decision records | +| `doc/knowledge_base/` | Process doc, roadmap, engineering practices, decision records | | `doc/architecture/` | How this repo is structured | | `doc/features/` | What this repo provides (process + template) | | `template/` | Copy-paste material for new projects | diff --git a/demo/seed/CLAUDE.md b/demo/seed/CLAUDE.md index b8082e3..675fd99 100644 --- a/demo/seed/CLAUDE.md +++ b/demo/seed/CLAUDE.md @@ -9,6 +9,7 @@ Markdown Renderer is a small local web app: paste Markdown into a box, see it re | System architecture & design | `doc/architecture/` (start with `index.md`) | | Feature documentation | `doc/features/` (start with `index.md`) | | Implementation roadmap | `doc/knowledge_base/roadmap.md` | +| Engineering practices | `doc/knowledge_base/engineering-practices.md` | | Design decisions | `doc/knowledge_base/` (decision records) | **Read `doc/architecture/` before planning any feature or structural change.** diff --git a/demo/seed/doc/knowledge_base/engineering-practices.md b/demo/seed/doc/knowledge_base/engineering-practices.md new file mode 100644 index 0000000..a98c737 --- /dev/null +++ b/demo/seed/doc/knowledge_base/engineering-practices.md @@ -0,0 +1,40 @@ +# Engineering Practices + +The engineering floor Markdown Renderer commits to. CDD distinguishes two kinds of practice: + +- **Enforced** — a CDD gate guarantees it on every change. If an enforced practice is failing, `/cdd-pre-pr` reports it and the change is not ready to merge. +- **Expected** — committed to but not yet mechanized here; tracked as a roadmap task until it becomes enforced. "Expected" is a promise with a due date, not an opt-out. + +When an expected practice gains its mechanism, move it to **Enforced** in the same PR that lands the mechanism. + +## Documentation — Enforced + +Architecture, feature, and roadmap docs are reconciled against the diff by `/cdd-pre-pr` (documentation reconciliation). A change isn't done until the docs match it. + +## Tested behaviour — Enforced + +New behaviour ships with a test, or an explicit, recorded reason it does not. `/cdd-pre-pr` (test-coverage reconciliation) checks this on every change, and `/cdd-pre-pr` build & QA runs the suites. + +- Test command: `pytest` +- Integration test command: `pytest tests/integration` + +## Continuous integration — Expected + +The checks below run locally through `/cdd-pre-pr`, but nothing runs them automatically on every PR yet. Adding a CI workflow (build + tests + lint on every PR) is tracked on the roadmap. + +- CI entry point: *none yet — expected* + +## Lint & format — Enforced + +`/cdd-pre-pr` build & QA runs both: + +- Lint command: `python -m pyflakes app` +- Format check command: `black --check app tests` + +## Dependency & toolchain hygiene — Expected + +`requirements.txt` lists dependencies; pinning them to exact versions and documenting the Python toolchain version is *expected*. + +## How this list grows + +New practices are added here as the project matures. `/cdd-pre-pr`'s CI-improvement check and the roadmap's infrastructure tasks feed it — closing the CI task above flips "Continuous integration" from **Expected** to **Enforced**. diff --git a/doc/architecture/adr/0001-name-and-guard-founding-objectives.md b/doc/architecture/adr/0001-name-and-guard-founding-objectives.md new file mode 100644 index 0000000..3b2b348 --- /dev/null +++ b/doc/architecture/adr/0001-name-and-guard-founding-objectives.md @@ -0,0 +1,35 @@ +# 0001: Name and guard CDD's under-guarded founding objectives + +**Status:** Accepted + +## Context + +CDD has three founding objectives as stated by the project owner: + +1. **Automate as much as possible, but keep human approval at the important gates.** +2. **Instil engineering best practices into any adopting project without invading how it works** — so juniors and non-engineers inherit architecture excellence, structured documentation, thorough unit/integration tests, and strong CI. +3. **Self-improvement** — whenever CDD or a CDD project finds a workflow improvement (a better `CLAUDE.md`, a command/CI/test/doc improvement), it incorporates it. + +Two of these were under-guarded. Objective 2's testing/CI/lint half was only *suggested* — no shipped CI workflow, no test scaffolding, no lint config, and no canonical best-practices artifact in either layer — while its documentation half was already enforced. Objective 3 had no standing channel in steady state: upstreaming existed only via `/cdd-retrofit` upgrade mode and CDD-on-CDD dogfooding, with the friction log deliberately retired. + +**Structural root cause:** §1 named exactly four commitments. Objective 2's testing/CI half was unnamed (only its documentation half was elevated), and objective 3 was entirely unnamed. The philosophy can only guard what it names. + +## Decision + +Elevate the two under-guarded objectives to named commitments, and give each a concrete guardrail: + +- **§1 reframed, 4 → 5 commitments.** "The project documents itself" broadened to "the project holds itself to engineering standards as it grows" — documentation as the leading *enforced* exemplar; tests/CI/lint instilled by **mechanism and floor, not prescription**. Added a fifth commitment: "the workflow improves itself." +- **An engineering-practices contract (§2.12)** — a canonical, enforced-vs-expected artifact, shipped in the template as `doc/knowledge_base/engineering-practices.md` and instantiated in both the CDD repo and the demo seed. +- **A test-coverage reconciliation step in `/cdd-pre-pr`** (both command copies) — the recurring objective-2 guardrail, parallel to doc reconciliation: it surfaces and records coverage drift but adds no new checkpoint. + +Objective 3 is named this session but its mechanism is deferred (see below); the decision is to name the commitment now and design the channel separately rather than reintroduce a standing friction log. + +## Consequences + +- Objective 2 now has a named commitment, a canonical enumerated artifact, and a recurring guardrail — closing the largest gap. Tests/CI/lint are held to a floor by mechanism rather than prescribed per project type, preserving the "without invading how it works" constraint. +- The engineering-practices contract is a third thing to keep consistent across the two layers (process doc + template) and the demo seed; drift there is now a defect like any other cross-layer drift. +- The `/cdd-pre-pr` reconciliation step adds work to every pre-PR run, but deliberately *records* rather than *blocks* — it does not become a seventh checkpoint. +- **Deferred (roadmap Phase 11):** + - *Objective-3 standing channel* — a recurring mechanism routing a discovered improvement into the roadmap/conventions (not a reintroduced log). Recorded as a §6 known gap. + - *Objective-1 mechanizations* — codifying when `/cdd-merge-main` should be recommended or auto-triggered, and any mechanical gate-honored check. + - *Objective-2 reinforcement at bootstrap* — a required bootstrap-phase task and/or checklist, once the recurring `/cdd-pre-pr` mechanism is proven. diff --git a/doc/architecture/index.md b/doc/architecture/index.md index a269e63..90994df 100644 --- a/doc/architecture/index.md +++ b/doc/architecture/index.md @@ -8,3 +8,4 @@ How this repo is structured. This index is a pointer list — the content lives - [Bootstrap & retrofit](bootstrap-and-retrofit.md) — the single substitution pipeline: bootstrap script, stage mode, overlay mode, `/cdd-bootstrap`, `/cdd-retrofit`, the baseline marker - [The demo layer](demo.md) — the third artifact: filled-in seed + create/teardown automation - `adr/` — architecture decision records (`adr/0000-template.md` for the format) + - [`0001-name-and-guard-founding-objectives.md`](adr/0001-name-and-guard-founding-objectives.md) — naming and guarding CDD's two under-guarded founding objectives (engineering practices, self-improvement) diff --git a/doc/architecture/overview.md b/doc/architecture/overview.md index 742f962..2729697 100644 --- a/doc/architecture/overview.md +++ b/doc/architecture/overview.md @@ -27,7 +27,7 @@ Changes flow process-first, template-second. A PR that touches the process doc b │ ├── index.md # documentation map │ ├── architecture/ # how this repo is structured │ ├── features/ # what this repo provides -│ └── knowledge_base/ # process doc, roadmap, decisions +│ └── knowledge_base/ # process doc, roadmap, engineering practices, decisions ├── scripts/ # template smoke assertions + command-set drift check (with whitelists) ├── template/ # copy-paste material for new projects └── tools/ diff --git a/doc/features/demo.md b/doc/features/demo.md index ac30594..6a09885 100644 --- a/doc/features/demo.md +++ b/doc/features/demo.md @@ -7,7 +7,7 @@ A `demo/` subsystem that instantiates a concrete project ("Markdown Renderer") f Contents: -- `demo/seed/`: filled-in `CLAUDE.md`, a 6-phase roadmap, and architecture/features docs for the Markdown Renderer (CDD scaffolding only — the app itself is built by running CDD cycles on a created instance). +- `demo/seed/`: filled-in `CLAUDE.md`, a 6-phase roadmap, the engineering-practices contract, and architecture/features docs for the Markdown Renderer (CDD scaffolding only — the app itself is built by running CDD cycles on a created instance). - `demo/setup.sh`: create an instance — wraps `bootstrap-cdd-project.sh --overlay demo/seed`, then always creates and pushes a GitHub repo. Auto-numbers disposable demo instances (`mdr_demo_NN`) checking both local dirs and existing repos; `mdr` is the kept dogfood instance. - `demo/teardown.sh`: reclaim an instance — remove the local directory and delete its GitHub repo (needs the `gh` `delete_repo` scope). diff --git a/doc/features/template.md b/doc/features/template.md index a0b3d69..fd8f266 100644 --- a/doc/features/template.md +++ b/doc/features/template.md @@ -7,7 +7,8 @@ A copy-paste directory (`template/`) plus a non-interactive bootstrap script (`t - `.claude/settings.json`: auto-allows worktree sessions to read their handoff file (`~/.claude-handoffs//**`), substituted at bootstrap. - `doc/index.md` plus `doc/{architecture,features,knowledge_base}/`: the documentation map and doc directory skeletons; the architecture and features skeletons follow the index-plus-per-topic-docs convention. - `doc/knowledge_base/project-overview.md`: the project-charter skeleton (what it is, goals, what it does and does not do, constraints, architecture intentions) — a living document, kept current. Filled by `/cdd-bootstrap` from discovery, or by hand otherwise. -- `doc/knowledge_base/roadmap.md`: roadmap skeleton with a pre-filled Phase 1 of CDD bootstrap tasks (codebase survey, initial architecture and feature docs, CLAUDE.md and overview stubs, roadmap fill) plus a suggested-infrastructure task list (CI, linting, tests, …) to distribute across the project's real phases. The pre-filled phase serves files-only starts (`/cdd-retrofit` install + the manual script); `/cdd-bootstrap` writes those docs through discovery and ships a real roadmap without it. +- `doc/knowledge_base/roadmap.md`: roadmap skeleton with a pre-filled Phase 1 of CDD bootstrap tasks (codebase survey, initial architecture and feature docs, CLAUDE.md and overview stubs, engineering-practices fill, roadmap fill) plus a suggested-infrastructure task list (CI, linting, tests, …) to distribute across the project's real phases. The pre-filled phase serves files-only starts (`/cdd-retrofit` install + the manual script); `/cdd-bootstrap` writes those docs through discovery and ships a real roadmap without it. +- `doc/knowledge_base/engineering-practices.md`: the engineering-practices contract skeleton — each practice marked *enforced* (a CDD gate guarantees it) or *expected* (committed, tracked on the roadmap until mechanized), with placeholders for the project's own test/CI/lint commands. Filled in during the bootstrap phase. - `tools/PROJECT-worktree.sh`: worktree helper, renamed and substituted to `-worktree.sh` by the bootstrap script. - `BOOTSTRAP.md`: meta-documentation for the bootstrap recipe. Not copied into the bootstrapped tree. diff --git a/doc/index.md b/doc/index.md index 20ebc4b..5e80348 100644 --- a/doc/index.md +++ b/doc/index.md @@ -6,6 +6,7 @@ Documentation map for the CDD repository. Each directory keeps an `index.md` poi - [`architecture/`](architecture/index.md) — how this repo is structured - [`features/`](features/index.md) — what this repo provides -- `knowledge_base/` — process doc, roadmap +- `knowledge_base/` — process doc, roadmap, engineering practices - [`claude-driven-development.md`](knowledge_base/claude-driven-development.md) — the process document: philosophy, artifacts, lifecycle, edit rules (the shipped product) - [`roadmap.md`](knowledge_base/roadmap.md) — implementation roadmap for this repo + - [`engineering-practices.md`](knowledge_base/engineering-practices.md) — this repo's engineering floor: which practices are enforced vs expected diff --git a/doc/knowledge_base/claude-driven-development.md b/doc/knowledge_base/claude-driven-development.md index af3ba89..588c726 100644 --- a/doc/knowledge_base/claude-driven-development.md +++ b/doc/knowledge_base/claude-driven-development.md @@ -6,16 +6,18 @@ This document describes the philosophy, the artifacts, the lifecycle, and the ru ## 1. Philosophy -Four commitments shape every decision in this workflow. +Five commitments shape every decision in this workflow. **The human is in the loop at every gate.** The agent never picks the next task, never approves a plan, never merges its own PR, never restructures the roadmap unilaterally. It proposes; the human disposes. The agent's value is throughput inside a clearly-scoped task and consistency in keeping docs current, not autonomous decision-making. **Automate everything except decisions.** The positive dual of the first commitment: CDD drives toward maximal SDLC automation — implementation, verification, documentation reconciliation, merge mechanics, even the consistency checks that keep the workflow itself honest — while reserving human attention for decisions. The six checkpoints (Section 4) are where automation deliberately stops. Everywhere else, a recurring manual step is a gap: convert it into a mechanism. -**The project documents itself as it grows.** Architecture and feature documentation are first-class deliverables, not afterthoughts. They serve dual duty: human reference and agent context. The same `pre-pr` step that runs CI also reconciles the docs against the code. A change isn't done until the docs match it. +**The project holds itself to engineering standards as it grows.** Sound architecture, structured documentation, tested behaviour, and a working CI gate are first-class deliverables, not afterthoughts; they serve dual duty as human reference and agent context. Documentation is the part CDD enforces directly today: the same `pre-pr` step that runs CI reconciles the docs against the code, and a change isn't done until the docs match it. The rest — that new behaviour ships with a test, that CI builds and checks the project, that dependencies and style stay honest — CDD instils by *mechanism and floor, not prescription*: it ships a written definition of what "engineering-ready" means (the engineering-practices contract, Section 2.12), asks at the pre-PR gate whether new behaviour is tested, and tracks the practices it does not yet enforce on the roadmap — while leaving the concrete tools, frameworks, and commands to the project. It raises the floor without dictating the house. This is how CDD instils engineering excellence into an adopting project without invading how it works. **Context is the scarcest resource.** Each Claude Code session has a finite, expensive context window. The workflow is structured to keep each session's context focused on one job: choosing the next task, implementing one task, reviewing one PR, resolving one merge. Sessions hand off via files (handoffs, the roadmap, the docs) rather than by trying to share context. +**The workflow improves itself.** CDD treats its own substrate — `CLAUDE.md`, the commands, the CI and test scaffolding, the docs, the conventions — as a product under continuous revision. When a session discovers a better way to work (a constraint that should have been in `CLAUDE.md`, a check the pre-PR gate should run, a convention worth adopting), the improvement does not evaporate at session end: it is routed into the project's own roadmap or conventions as a tracked change, and an improvement general enough to help any project is surfaced as a candidate to upstream into CDD itself (Section 6, `/cdd-retrofit` upgrade mode). A recurring friction that no artifact captures is a gap, the same way a recurring manual step is. + A non-goal: full autonomy. CDD is not an attempt to take the human out of the loop. It is a way to amplify a single developer (initially) by structuring how the agent participates. ## 2. Artifacts @@ -75,6 +77,7 @@ Project metadata and history. The roadmap lives here. So do: - The project overview (`project-overview.md`): the project's charter — what it is, why it exists, what it does and explicitly does not do, its constraints, and its architecture intentions. See below. - Decision records: why we chose this RTOS, this framework, this hardware target, this protocol. Append-only. - Coding standards: language-specific style and convention rules. +- The engineering-practices contract (`engineering-practices.md`): the project's engineering floor, with each practice marked *enforced* or *expected*. See Section 2.12. - Investigation notes: deep dives done in the course of the project that don't fit into architecture or feature docs. The knowledge base is mostly append-only. Decisions are not rewritten when they are superseded; new decisions are added that supersede them, with a reference back. This preserves the reasoning trail. @@ -170,6 +173,25 @@ Several sessions auto-commit at their gate so that a session never leaves a dirt Which sessions auto-commit: the implementation session (§3.3) and `/cdd-pre-pr` (§3.5) commit their own changes locally; `/cdd-process-pr` (§3.7) commits and pushes; `/cdd-merge-main` (§3.4) produces a merge commit and enforces a clean tree before merging. All four follow the rules above. +### 2.12 The engineering-practices contract (`doc/knowledge_base/engineering-practices.md`) + +The project's engineering floor, written down. It is the artifact that makes the second commitment of Section 1 — instilling engineering best practices — legible instead of implicit: a reader can see, at a glance, which practices CDD currently *guarantees* for this project and which it still *owes*. Each practice is marked one of two ways: + +- **Enforced** — a CDD gate guarantees it on every change. If an enforced practice is failing, `/cdd-pre-pr` reports it and the change is not ready. +- **Expected** — the project is committed to the practice but has not yet mechanized it here. Each expected practice is tracked as a roadmap task until it becomes enforced. "Expected" is a promise with a due date, not an opt-out. + +The canonical set of practices the contract enumerates: + +| Practice | Typical status | Enforcing gate (once enforced) | +| --------------------------------------------------------------------------------- | ------------------------------------------------------- | ------------------------------------------------------ | +| Structured documentation (architecture, feature, roadmap docs track the code) | **enforced** | `/cdd-pre-pr` doc reconciliation (§3.5) | +| Tested behaviour (new behaviour ships with a test, or a recorded reason it doesn't) | **enforced** once a test command exists; **expected** until then | `/cdd-pre-pr` test-coverage reconciliation (§3.5) | +| Continuous integration (build + checks run on every change) | **expected** until a CI entry point exists, then **enforced** | `/cdd-pre-pr` build & QA (§3.5) + the project's own CI | +| Lint & format | **expected** | `/cdd-pre-pr` build & QA, once a lint/format command exists | +| Dependency & toolchain hygiene (pinned/locked deps, documented toolchain) | **expected** | project-defined | + +A practice moves from **expected** to **enforced** in the same change that lands its mechanism (a test command, a CI job): the mechanism and the status flip ship together. The contract is deliberately generic and language-agnostic — it names *what* the floor is and carries placeholders for the project's own commands, never a shipped CI or lint config (opinionated per-project-type defaults are deferred design, §6). New practices are added as the project matures; the roadmap's suggested-infrastructure tasks and `/cdd-pre-pr`'s CI-improvement check (§3.5) feed it. Drop a row that genuinely does not apply (e.g. integration tests in a pure library), but record *why* in a clause rather than deleting it silently. + ## 3. Lifecycle A task flows through CDD in up to five sessions, two of them optional side-loops (`/cdd-merge-main` before the PR, `/cdd-process-pr` after review). Each session type has a name, one command, and one job: @@ -230,9 +252,9 @@ The five rows above are the per-task lifecycle. Three further session types sit │ │ │ Run build, format, lint, tests, │ │ integration tests. Code review. │ - │ Reconcile architecture and │ - │ feature docs. Propose roadmap │ - │ edits. Conditionally propose CI │ + │ Reconcile docs and test │ + │ coverage. Propose roadmap edits. │ + │ Conditionally propose CI │ │ improvements. │ └──────────────────────────────────┘ │ @@ -305,7 +327,7 @@ This is also where the agent can pull in improvements from main that are useful ### 3.5 Pre-PR session: `/cdd-pre-pr` -A fresh session on the feature branch, started after the implementation session has closed. This isolation is deliberate: a fresh context avoids the implementation session grading its own homework, and any "propose to the human" step in `/cdd-pre-pr` is a proposal to the human running this session, not to another session. Runs the CI gates, reads the diff, code-reviews changed files, and reconciles documentation. +A fresh session on the feature branch, started after the implementation session has closed. This isolation is deliberate: a fresh context avoids the implementation session grading its own homework, and any "propose to the human" step in `/cdd-pre-pr` is a proposal to the human running this session, not to another session. Runs the CI gates, reads the diff, code-reviews changed files, reconciles documentation, and checks that new behaviour is tested. Doc reconciliation has three parts: @@ -313,9 +335,11 @@ Doc reconciliation has three parts: - **Feature docs**: same, for feature docs. - **Roadmap**: tick newly-completed checkboxes directly; identify roadmap edits (add/modify/remove tasks) implied by what landed and present them to the human in this session for approval before applying. +Alongside doc reconciliation, `/cdd-pre-pr` reconciles **test coverage** — the recurring guardrail behind the engineering-practices contract's "tested behaviour" row (§2.12). For each behavioural change in the diff it checks whether a test exercises the new behaviour. If the project has a test command, a behavioural change that landed without a test is flagged; a deliberately-untested change is allowed but must be recorded, not silent. If the project has no test harness yet (the contract still marks tested behaviour *expected*), the step does not invent a framework — it notes that the change shipped untested and confirms that standing up tests is tracked on the roadmap. Like doc reconciliation, this surfaces and records; it does not add a checkpoint (§4). + `/cdd-pre-pr` also performs a conditional CI-improvement check: if the change introduces a category of work that the existing CI doesn't cover (new file type, new test category, a tool that should be linted but isn't), propose improvements to the human. On approval, apply them as part of the same PR; alternatively, the human may defer them as a new roadmap task. The default is silence. The agent should not propose CI improvements every run; only when the change genuinely surfaces a gap. -Output is a pass/fail summary across all the gates. After the summary, `/cdd-pre-pr` auto-commits the reconciliation edits it just made — the doc, CLAUDE.md, README, and roadmap changes from this session — locally and with no push, per the commit conventions (§2.11); if it entered an already-dirty tree it stops and surfaces that instead of committing. Pushing stays out of this commit: it happens only in the opt-in PR-open step below. That step is an optional, human-gated step to open the PR — a general capability available to every task, not just issue-sourced ones. It asks a single yes/no question — no pre-shown title/body, no manual `gh` instructions. On approval it derives the title and body and runs `gh pr create`; if the branch name carries the `gh_issue_NN` token, the PR body gets a `Closes #NN` line so the issue auto-closes on merge. If §6 detected upstream drift, the step restates the `/cdd-merge-main` recommendation before offering. If the human declines, the step simply stops; the checklist stands on its own. The gate preserves the checkpoint model: `/cdd-pre-pr` never opens a PR without explicit confirmation. +Output is a pass/fail summary across all the gates. After the summary, `/cdd-pre-pr` auto-commits the reconciliation edits it just made — the doc, CLAUDE.md, README, and roadmap changes from this session — locally and with no push, per the commit conventions (§2.11); if it entered an already-dirty tree it stops and surfaces that instead of committing. Pushing stays out of this commit: it happens only in the opt-in PR-open step below. That step is an optional, human-gated step to open the PR — a general capability available to every task, not just issue-sourced ones. It asks a single yes/no question — no pre-shown title/body, no manual `gh` instructions. On approval it derives the title and body and runs `gh pr create`; if the branch name carries the `gh_issue_NN` token, the PR body gets a `Closes #NN` line so the issue auto-closes on merge. If the upstream-drift check detected drift, the step restates the `/cdd-merge-main` recommendation before offering. If the human declines, the step simply stops; the checklist stands on its own. The gate preserves the checkpoint model: `/cdd-pre-pr` never opens a PR without explicit confirmation. ### 3.6 PR review and merge @@ -391,6 +415,8 @@ This raises a recurring question — *is the task a deliverable or a project?* **Template opinionation per project type.** The current template encodes the workflow, but the project-specific bits (build commands, language constraints, module layout) are placeholders. Different project archetypes (firmware, web app, library, data pipeline) probably want different opinionated defaults for those placeholders. Worth deriving from real usage rather than guessing up front. +**A standing self-improvement channel.** The fifth commitment (§1, "the workflow improves itself") is guarded today only at two moments: `/cdd-retrofit` upgrade mode surfaces a project's general customizations as upstream candidates, and the CDD repo dogfoods itself (§7.1). A project merely *running* CDD day to day has no recurring, lightweight channel to flag "this `CLAUDE.md` constraint, this CI check, this convention looks general — capture it." The retired friction log (a standing separate file) is deliberately not the answer; the mechanism should route a discovered improvement into machinery that already exists — a roadmap item, a conventions/`CLAUDE.md` edit, or an upstream candidate — rather than into a new log. The likely shape is a conditional `/cdd-pre-pr` prompt parallel to the CI-improvement check, or a `/cdd-next-step` intake; deferred until the design is worked out. + **Adapting an existing project.** Addressed by `/cdd-retrofit`, a command that lives in the CDD repo only (see Section 2.7) and is run from a CDD-repo session with the target project's path as argument. It auto-detects which of two modes applies: - *Install mode* — the target has no CDD scaffolding. A files-only install of the template: slash commands, doc skeletons, the worktree helper, `.claude/settings.json`, with placeholder substitution done by `bootstrap-cdd-project.sh` in a render-only staging mode (the command never reimplements substitution). Files missing from the target are copied; collisions with existing files (a project's own `CLAUDE.md`, for instance) are merged interactively, one file at a time, with human approval — never overwritten silently. No codebase survey happens at install time: the template roadmap ships with a pre-filled bootstrap phase (survey the codebase, draft the initial architecture docs, write the feature docs, fill in the roadmap), so the project's first `/cdd-next-step` picks those up as the next unchecked tasks. This pre-filled phase is for files-only starts — install mode here and the manual bootstrap-script path — where no docs have been written yet; `/cdd-bootstrap` writes the docs through guided discovery and so ships a real roadmap without it. diff --git a/doc/knowledge_base/engineering-practices.md b/doc/knowledge_base/engineering-practices.md new file mode 100644 index 0000000..09dc70d --- /dev/null +++ b/doc/knowledge_base/engineering-practices.md @@ -0,0 +1,36 @@ +# CDD Engineering Practices + +The engineering floor this repo — the CDD meta-project — commits to. CDD distinguishes two kinds of practice: + +- **Enforced** — a CDD gate guarantees it on every change. If an enforced practice is failing, `/cdd-pre-pr` or CI reports it and the change is not ready to merge. +- **Expected** — committed to but not yet mechanized here; tracked as a roadmap task until it becomes enforced. + +This repo is documentation and shell scripts: there is no compiled build, so "build" and "tests" take the shape of shell-syntax checks, the bootstrap smoke, and the command-drift check rather than a compiler and a unit-test runner. See `CLAUDE.md` → "Build & test" for the exact commands. + +## Documentation — Enforced + +The process doc, template, architecture/feature docs, and roadmap are reconciled against the diff by `/cdd-pre-pr` (documentation reconciliation), and the two-layer consistency rule — process-doc-first, then template — is part of every change. A change isn't done until the docs match it. + +## Tested behaviour — Enforced + +There is no unit-test suite; behaviour is exercised by integration-style smoke and consistency checks, run in CI by `.github/workflows/template-smoke.yml` and by hand per `CLAUDE.md`: + +- `bash -n` over all shell scripts (syntax). +- `./scripts/command-drift-check.sh` — repo `.claude/commands/` vs the rendered template, plus the handoff-schema and worktree-helper assertions. +- End-to-end bootstrap smoke: `tools/bootstrap-cdd-project.sh` into a tmpdir + `scripts/template-smoke-assert.sh` (clean, link-valid tree). +- Demo seed-overlay smoke: `demo/setup.sh … --local-only`. + +New behaviour in a script or the bootstrap path ships with the relevant smoke or assertion extended to cover it. + +## Continuous integration — Enforced + +`.github/workflows/template-smoke.yml` runs shellcheck, the command-drift check, the end-to-end bootstrap smoke, and the demo seed-overlay step on every PR. + +## Lint & format — Enforced (lint); Expected (format) + +- Lint: `shellcheck` over all repo shell scripts, in CI. +- Format: no automated formatter for Markdown or shell is enforced yet. *Expected.* + +## Dependency & toolchain hygiene — Expected + +The toolchain is bash + `gh` + standard POSIX tools, assumed present rather than pinned. Documenting or pinning the required tool versions is *expected*. diff --git a/doc/knowledge_base/roadmap.md b/doc/knowledge_base/roadmap.md index 5fa4c40..ed1bc23 100644 --- a/doc/knowledge_base/roadmap.md +++ b/doc/knowledge_base/roadmap.md @@ -133,3 +133,17 @@ Defects and gaps surfaced by retrofitting CDD onto real existing projects. Each - [ ] **Retrofit doc-reconciliation playbook for common pre-existing layouts.** PyGroundControl needed manual reconciliation that recurs across projects: a split architecture-doc layout (`doc/backend/`, `doc/frontend/`, a top-level `system-architecture.md`) folding into `doc/architecture/`; a `future-work.md`/TODO/backlog doc folding into `roadmap.md`; an oversized CLAUDE.md duplicating command/troubleshooting content that should be slimmed to pointers (per-session context cost). Capture these as explicit guidance/checklist in `/cdd-retrofit` (and the process doc's existing-project section) so they aren't rediscovered each time. **Milestone:** CDD retrofits cleanly onto a project regardless of default-branch name, repo directory name/location, or pre-existing doc layout, without per-project manual fixes to the scaffolding. + +## Phase 11: Founding-objective guardrails + +Elevate the two under-guarded founding objectives — instilling engineering best practices, and workflow self-improvement — from implicit to named-and-tracked. Decision and reasoning in `doc/architecture/adr/0001-name-and-guard-founding-objectives.md`. + +- [x] Audit the three founding objectives against the workflow and record the gap inventory. — ADR `0001-name-and-guard-founding-objectives.md` +- [x] Name the under-guarded objectives in §1: broaden "documents itself" into "holds itself to engineering standards", add "the workflow improves itself" (4 → 5 commitments). +- [x] Ship the engineering-practices contract (enforced vs expected): process doc §2.12 + template `doc/knowledge_base/engineering-practices.md`, instantiated in the CDD repo and the demo seed. +- [x] Add the `/cdd-pre-pr` test-coverage reconciliation step (both command copies) as the recurring objective-2 guardrail. +- [ ] Objective-3 standing channel: a recurring mechanism that routes a discovered improvement into the roadmap/conventions (not a reintroduced standing log). — §6 known gap; design deferred. +- [ ] Reinforce objective 2 at bootstrap: a required bootstrap-phase task and/or checklist, once the `/cdd-pre-pr` mechanism is proven. +- [ ] Objective-1 mechanizations: codify when `/cdd-merge-main` is recommended/auto-triggered; consider a mechanical gate-honored check. + +**Milestone:** all three founding objectives are named commitments in §1, each with at least one recurring guardrail or a tracked plan to add one. diff --git a/scripts/template-smoke-whitelist.txt b/scripts/template-smoke-whitelist.txt index 23f73de..582dbb2 100644 --- a/scripts/template-smoke-whitelist.txt +++ b/scripts/template-smoke-whitelist.txt @@ -13,7 +13,7 @@ - + diff --git a/template/.claude/commands/cdd-pre-pr.md b/template/.claude/commands/cdd-pre-pr.md index 877e776..c841c2b 100644 --- a/template/.claude/commands/cdd-pre-pr.md +++ b/template/.claude/commands/cdd-pre-pr.md @@ -53,7 +53,17 @@ Check and **update** documentation based on the changes: Read each relevant doc and compare against the actual code changes. Fix discrepancies directly when they are reconciliation (the doc is out of date relative to what landed). Ask before applying structural changes (adding new doc files, restructuring an existing doc). -## 5. CI improvement check (conditional) +## 5. Test coverage reconciliation + +For each behavioural change in the diff (a new function, a new branch, changed output, a fixed bug), check whether it is covered by a test. This is the recurring guardrail behind the "tested behaviour" row of `doc/knowledge_base/engineering-practices.md`. + +- **If the project has a test command** ("tested behaviour" marked *enforced*): confirm a test exercises the new behaviour. If a behavioural change landed with no accompanying test, flag it — the default expectation is that new behaviour ships with a test. +- **If a change is deliberately untested** (a throwaway script, generated code, a spike): that is allowed, but it must be *intentional and recorded*, not silent. State the reason in the PR summary. +- **If the project has no test command yet** ("tested behaviour" still *expected*): do not invent a framework. Note that the change shipped untested because there is no test harness, and confirm that standing one up is tracked as a roadmap task. If this change is exactly the kind of behaviour that motivates a first test, say so and let the user decide whether to pull that task forward. + +This step asks a question and records the answer; it does not mandate a specific framework, a coverage threshold, or that every change be tested. "Not tested, and here is why" is a valid, recorded outcome. Surface it — do not block on it. + +## 6. CI improvement check (conditional) If, and only if, the change introduces a category of work that the existing CI does not cover, propose specific improvements to the user. Examples that should trigger a proposal: @@ -64,7 +74,7 @@ If, and only if, the change introduces a category of work that the existing CI d Do **not** propose generic CI improvements every run. The default is silence. If you do propose, the user has two options: apply now in this PR, or defer as a new roadmap task. Apply only on approval. -## 6. Upstream drift check +## 7. Upstream drift check ```bash git fetch origin main @@ -73,7 +83,7 @@ git log --oneline HEAD..origin/main If `origin/main` has advanced beyond the branch point, mention it and recommend running `/cdd-merge-main` before opening the PR. Do not merge from this session. -## 7. Summary +## 8. Summary Present a checklist summary: @@ -90,6 +100,7 @@ Present a checklist summary: - [ ] CLAUDE.md up to date - [ ] README up to date - [ ] Roadmap up to date +- [ ] New behaviour tested (or untested-with-reason recorded) - [ ] CI gaps surfaced: none / proposed (list them) - [ ] No upstream drift (or: /cdd-merge-main recommended) - [ ] Reconciliation edits committed @@ -97,9 +108,9 @@ Present a checklist summary: Mark each item as pass ✓ or needs attention ✗ with details. -## 8. Commit reconciliation edits +## 9. Commit reconciliation edits -Commit the documentation reconciliation edits this session made in steps 3–6 (architecture/feature docs, CLAUDE.md, README, the coding standard, and the roadmap). This is a local commit only — **no push**. Pushing happens, if at all, in step 9. +Commit the documentation reconciliation edits this session made in steps 3–7 (architecture/feature docs, CLAUDE.md, README, the coding standard, and the roadmap). This is a local commit only — **no push**. Pushing happens, if at all, in step 10. First check the entry snapshot from step 1: @@ -107,13 +118,13 @@ First check the entry snapshot from step 1: - **Otherwise**, commit only the files this session edited. Add them by path — do not `git add -A`: ```bash -git add +git add git commit -m '' ``` Follow the repo's commit conventions from CLAUDE.md. Print a one-line summary of the commit (subject + files included). If nothing was reconciled (no edits this session), say so and skip the commit. -## 9. Open PR (optional) +## 10. Open PR (optional) After the checklist, offer to open the PR. This is human-gated — never open a PR without explicit confirmation. @@ -125,7 +136,7 @@ gh auth status && git remote get-url origin # origin should be a github.com UR If either is missing, say so in one line and skip this step (the checklist above still stands). -If §6 found upstream drift, restate the recommendation to run `/cdd-merge-main` before opening the PR, and let the user decide whether to proceed anyway. +If §7 found upstream drift, restate the recommendation to run `/cdd-merge-main` before opening the PR, and let the user decide whether to proceed anyway. Ask: **"Open a PR now?"** Do not pre-show a title or body, and do not print manual `gh` instructions — just ask whether to proceed. diff --git a/template/CLAUDE.md b/template/CLAUDE.md index 950b4a9..314c3d3 100644 --- a/template/CLAUDE.md +++ b/template/CLAUDE.md @@ -12,6 +12,7 @@ | Architecture decision records | `doc/architecture/adr/` (Nygard style) | | Feature documentation | `doc/features/index.md` | | Implementation roadmap | `doc/knowledge_base/roadmap.md` | +| Engineering practices | `doc/knowledge_base/engineering-practices.md` | | Design decisions | `doc/knowledge_base/` (decision records) | Each doc directory keeps an `index.md` pointer list: read the index, then load only the documents you need. **`index.md` files are pointer lists only — content belongs in named subdocuments, not in the index itself.** diff --git a/template/doc/index.md b/template/doc/index.md index 409dba0..303429b 100644 --- a/template/doc/index.md +++ b/template/doc/index.md @@ -7,7 +7,7 @@ Project documentation map. Each directory keeps an `index.md` pointer list: read - [`knowledge_base/project-overview.md`](knowledge_base/project-overview.md) — the project charter: what it is, why it exists, what it does and does not do. **Read this first.** - [`architecture/`](architecture/index.md) — system design documents: what the system is, structurally - [`features/`](features/index.md) — feature documentation: what the system does, per capability -- [`knowledge_base/`](knowledge_base/README.md) — the roadmap, decision records, coding guidelines, investigation notes +- [`knowledge_base/`](knowledge_base/README.md) — the roadmap, the engineering-practices contract, decision records, coding guidelines, investigation notes ## What goes where diff --git a/template/doc/knowledge_base/README.md b/template/doc/knowledge_base/README.md index bdc1cbf..7f0a94d 100644 --- a/template/doc/knowledge_base/README.md +++ b/template/doc/knowledge_base/README.md @@ -6,6 +6,7 @@ Project metadata and history. Mostly append-only. - `project-overview.md`: the project charter — what it is, why it exists, what it does and explicitly does not do, its constraints and architecture intentions. Kept current (see Conventions). - `roadmap.md`: implementation roadmap (the central workflow artifact). +- `engineering-practices.md`: the project's engineering floor — each practice marked *enforced* (a CDD gate guarantees it) or *expected* (committed, tracked on the roadmap until mechanized). - `.md`: language-specific style and convention rules. - `.md`: design and tooling decisions (why we chose X over Y). - `.md`: deep dives done in the course of the project that don't fit into architecture or feature docs. diff --git a/template/doc/knowledge_base/engineering-practices.md b/template/doc/knowledge_base/engineering-practices.md new file mode 100644 index 0000000..a33117d --- /dev/null +++ b/template/doc/knowledge_base/engineering-practices.md @@ -0,0 +1,38 @@ +# Engineering Practices + +The engineering floor this project commits to. CDD distinguishes two kinds of practice: + +- **Enforced** — a CDD gate guarantees it on every change. If an enforced practice is failing, `/cdd-pre-pr` reports it and the change is not ready to merge. +- **Expected** — the project is committed to growing this practice, but it is not yet mechanized here. Each expected practice is tracked as a roadmap task until it becomes enforced. "Expected" is a promise with a due date, not an opt-out. + +When an expected practice gains its mechanism (a test command, a CI job, a linter), move it to **Enforced** in the same PR that lands the mechanism. Drop a row that genuinely does not apply to this project (e.g. integration tests in a pure library), but record *why* in a clause rather than deleting it silently. + +## Documentation — Enforced + +Architecture, feature, and roadmap docs are reconciled against the diff by `/cdd-pre-pr` (documentation reconciliation). A change isn't done until the docs match it. + +## Tested behaviour — + +New behaviour ships with a test, or an explicit, recorded reason it does not. `/cdd-pre-pr` (test-coverage reconciliation) checks this on every change. + +- Test command: `` +- Integration test command: `` + +## Continuous integration — + +Build and checks run on every PR. + +- CI entry point: `` + +## Lint & format — + +- Lint command: `` +- Format check command: `` + +## Dependency & toolchain hygiene — Expected + +Dependencies are pinned or locked; toolchain versions are documented. + +## How this list grows + +New practices are added here as the project matures. `/cdd-pre-pr`'s CI-improvement check and the roadmap's "Suggested infrastructure tasks" feed it: when one of those surfaces a gap and the project closes it, add the corresponding row here or flip it from **Expected** to **Enforced**. diff --git a/template/doc/knowledge_base/roadmap.md b/template/doc/knowledge_base/roadmap.md index 010f923..f968d89 100644 --- a/template/doc/knowledge_base/roadmap.md +++ b/template/doc/knowledge_base/roadmap.md @@ -11,6 +11,7 @@ Get the CDD substrate to reflect reality: survey what exists, write the initial - [ ] Survey the codebase and draft the initial architecture docs under `doc/architecture/`: an `overview.md` with the high-level shape, plus per-topic docs as warranted. For a greenfield project, write architecture guidelines and intentions instead. Where architecture notes already exist (a README, design docs), fold them in rather than starting from scratch. - [ ] Write the initial feature docs under `doc/features/`: one doc per existing user-visible capability. Likely empty for a greenfield project. Where features are already documented elsewhere, adopt and reconcile that content here. - [ ] Fill in the project charter at `doc/knowledge_base/project-overview.md` (what it is, goals, non-goals, constraints, architecture intentions) and the `CLAUDE.md` stubs (project description, critical constraints, build/test commands, module layout). +- [ ] Fill in the engineering-practices contract (`doc/knowledge_base/engineering-practices.md`): mark each practice *enforced* or *expected* for this project, and fill the command placeholders for the ones that already exist. - [ ] Fill in this roadmap: replace the placeholder phases below with the project's real plan, slotting in items from "Suggested infrastructure tasks" where they fit. **Milestone: the docs describe the project as it actually is, and the roadmap below is a real plan.** @@ -39,7 +40,7 @@ Get the CDD substrate to reflect reality: survey what exists, write the initial ## Suggested infrastructure tasks -Slot these into the phases above where they fit — usually spread across the early phases, not bundled into one. Drop the ones that don't apply; delete this section once it has been folded in. +Slot these into the phases above where they fit — usually spread across the early phases, not bundled into one. Drop the ones that don't apply; delete this section once it has been folded in. Each one the project commits to but hasn't mechanized yet is an *expected* practice in `doc/knowledge_base/engineering-practices.md`; closing it flips that row to *enforced*. - Set up CI: build + tests on every PR. - Add linting and a format check (and a pre-commit hook if wanted).