feat(docs): design doc + implementation plan convention with CI check (closes #67 phase-1)

.github/workflows/design-doc-check.yml

@@ -0,0 +1,56 @@
+# Design Doc Check — issue #67 Phase 1
+#
+# This workflow runs the design doc checker on every PR. It detects
+# "new feature" PRs (by file pattern) and fails if no design doc is
+# included in docs/design/. The check is bypassable via the
+# 'skip-design-doc' PR label.
+#
+# The check is informational on first failure — the PR author can either
+# add a design doc or add the bypass label, then re-push.
+
+name: Design Doc Check
+
+on:
+  pull_request:
+    branches: [main]
+    # Only run when files that could trigger the check are changed. This
+    # avoids wasting CI minutes on doc-only or test-only PRs.
+    paths:
+      - 'scripts/commands/**'
+      - 'scripts/formatters/**'
+      - 'scripts/parsers/**'
+      - 'scripts/*_engine.py'
+      - 'docs/design/**'
+      - '.github/workflows/design-doc-check.yml'
+      - 'scripts/check_design_doc.py'
+
+permissions:
+  contents: read
+  pull-requests: read
+
+jobs:
+  check:
+    name: Design Doc Required
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          # We need the full PR diff, not just the merge commit. fetch-depth 0
+          # gets the full history so the check script can inspect the PR's
+          # changed files via the GitHub API instead.
+          fetch-depth: 1
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Run design doc check
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_REPOSITORY: ${{ github.repository }}
+          PR_NUMBER: ${{ github.event.pull_request.number }}
+        run: |
+          python3 scripts/check_design_doc.py

CONTRIBUTING.md

@@ -183,11 +183,55 @@ When adding a new CLI command, create a new file in the `commands/` directory:
 ## Pull Request Process
 
 1. **Update documentation** — README.md, SKILL.md, SKILL-QUICK.md, changelog.md
-2. **Add tests** for new features
-3. **Ensure all tests pass** — `python3 -m pytest tests/ -v`
-4. **Follow the PR template** — describe changes, motivation, testing
-5. **One PR per feature** — keep PRs focused and reviewable
-6. **Update skill.json version** if adding new commands
+2. **Add a design doc** for new features (see "Design Doc Requirement" below)
+3. **Add tests** for new features
+4. **Ensure all tests pass** — `python3 -m pytest tests/ -v`
+5. **Follow the PR template** — describe changes, motivation, testing
+6. **One PR per feature** — keep PRs focused and reviewable
+7. **Update skill.json version** if adding new commands
+
+### Design Doc Requirement (issue #67 Phase 1)
+
+PRs that add a **new feature** must include a design doc in `docs/design/`.
+The CI check (`.github/workflows/design-doc-check.yml`) automatically
+detects new-feature PRs by file pattern and fails if no design doc is
+included.
+
+**What counts as a "new feature"?**
+
+| Pattern | Example | Requires design doc? |
+|---------|---------|---------------------|
+| New file in `scripts/commands/` | `commands/yourfeature.py` | Yes |
+| New `scripts/*_engine.py` | `yourfeature_engine.py` | Yes |
+| New file in `scripts/formatters/` | `formatters/yourformat.py` | Yes |
+| New parser (non-fallback) in `scripts/parsers/` | `parsers/yourlang_parser.py` | Yes |
+| Fallback parser | `parsers/fallback_yourlang.py` | No (regex shadow of existing parser) |
+| Bug fix (modified file) | any existing file | No |
+| Test addition | `tests/test_*.py` | No |
+| Documentation change | `*.md` | No |
+
+**How to write a design doc:**
+
+1. Copy `docs/design/template.md` to `docs/design/NNNN-feature-name.md`
+   - `NNNN` is the next available number (zero-padded to 4 digits)
+   - `feature-name` is a short kebab-case slug
+2. Fill in the sections: Problem, Goal, Changes, Trade-offs, Open Questions,
+   Migration / Rollout
+3. The **Trade-offs** section is the most important — document alternatives
+   considered and why they were rejected. This prevents future contributors
+   from re-litigating decisions without context.
+4. See `docs/design/0001-taint-engine.md` through `0004-graph-model.md` for
+   retroactive examples documenting existing features.
+
+**Bypassing the check:**
+
+If a feature is genuinely trivial (e.g., a one-line command alias) and a
+design doc would be pure overhead, add the `skip-design-doc` label to the
+PR. Use this sparingly — the check exists to ensure design decisions are
+recorded for future contributors.
+
+See `docs/README.md` for full details on the design doc and implementation
+plan convention.
 
 ### PR Title Format
 

docs/README.md

@@ -0,0 +1,108 @@
+# CodeLens Documentation
+
+This directory contains CodeLens design documents, implementation plans, and
+process guides.
+
+## Structure
+
+```
+docs/
+├── README.md                  ← you are here
+├── design/                    ← design docs (why a feature exists)
+│   ├── template.md            ← copy this to start a new design doc
+│   ├── 0001-taint-engine.md
+│   ├── 0002-mcp-server.md
+│   ├── 0003-plugin-system.md
+│   └── 0004-graph-model.md
+└── plans/                     ← implementation plans (how/when to build)
+    └── template.md            ← copy this to start a new plan
+```
+
+## When Do I Need a Design Doc?
+
+A design doc is **required** for PRs that add a new feature. The CI check
+(`scripts/check_design_doc.py`, runs via `.github/workflows/design-doc-check.yml`)
+automatically detects new-feature PRs by file pattern and fails if no design
+doc is included.
+
+### What counts as a "new feature"?
+
+Any of these triggers the requirement:
+
+| Pattern | Example | Why |
+|---------|---------|-----|
+| New file in `scripts/commands/` | `commands/yourfeature.py` | New CLI command |
+| New `scripts/*_engine.py` file | `yourfeature_engine.py` | New analysis engine |
+| New file in `scripts/formatters/` | `formatters/yourformat.py` | New output format |
+| New parser in `scripts/parsers/` (non-fallback) | `parsers/yourlang_parser.py` | New language support |
+
+### What does NOT require a design doc?
+
+- Bug fixes (modifications to existing files)
+- Test additions
+- Documentation changes
+- Dependency updates
+- Refactors that don't change behavior
+- Fallback parsers (`parsers/fallback_*.py`) — these are regex versions of
+  existing tree-sitter parsers, not new features
+
+### Bypassing the check
+
+If a feature is genuinely trivial (e.g., a one-line command alias) and a
+design doc would be pure overhead, add the `skip-design-doc` label to the PR.
+Use this sparingly — the check exists to ensure design decisions are recorded
+for future contributors.
+
+## How to Write a Design Doc
+
+1. Copy `docs/design/template.md` to `docs/design/NNNN-feature-name.md`
+   - `NNNN` is the next available number (zero-padded to 4 digits)
+   - `feature-name` is a short kebab-case slug
+2. Fill in the sections:
+   - **Problem** — what pain exists today? Be concrete.
+   - **Goal** — what does "done" look like? User-visible outcome.
+   - **Changes** — concrete file changes, grouped by area
+   - **Trade-offs** — alternatives considered and why they were rejected
+     (the most important section — prevents re-litigating decisions)
+   - **Open Questions** — what's NOT yet decided, with owners
+   - **Migration / Rollout** — how users migrate, or "additive — no migration"
+3. Reference the design doc in your PR description
+
+See `docs/design/0001-taint-engine.md` through `0004-graph-model.md` for
+retroactive examples documenting existing features.
+
+## How to Write an Implementation Plan
+
+A plan is **recommended** (but not enforced) for multi-phase features. Copy
+`docs/plans/template.md` to `docs/plans/NNNN-feature-name.md` and break the
+work into independently-reviewable phases.
+
+Plans are living documents — update the checklist as you progress. When the
+feature is complete, the plan can be archived or merged into the design doc's
+"Changes" section.
+
+## Numbering Convention
+
+Design docs and plans use ADR-style numbering: `NNNN-feature-name.md` where
+`NNNN` is a zero-padded sequential number. This ensures sort stability and
+makes it easy to reference a doc by number (e.g., "see design doc 0003").
+
+The numbering is per-directory — `docs/design/0001-foo.md` and
+`docs/plans/0001-bar.md` are independent sequences.
+
+## Lifecycle
+
+```
+1. PROPOSE  → create design doc in docs/design/ as part of your feature PR
+2. ACCEPT   → BOS reviews the design doc as part of PR review
+3. IMPLEMENT → the design doc reflects the as-built design; update if the
+              implementation diverged from the proposal
+4. SUPERSEDE → if a future PR replaces this feature, mark the doc as
+              "Superseded by NNNN" and create a new doc for the replacement
+5. DEPRECATE → if the feature is removed, mark the doc as "Deprecated"
+              but keep it for historical reference
+```
+
+Design docs are **never deleted** — they form the historical record of why
+the codebase is structured the way it is. Even deprecated docs remain so
+future contributors can understand past decisions.

docs/design/0001-taint-engine.md

@@ -0,0 +1,141 @@
+# Design Doc 0001: Taint Analysis Engine
+
+> **Status:** Accepted
+> **Date:** 2026-06-15 (retroactive — backfilled 2026-07-02)
+> **Author:** Wolfvin
+> **Related issues:** #49 (Phase 1 consolidation)
+> **Related PRs:** #140 (consolidation), original implementation pre-#49
+
+---
+
+## Problem
+
+CodeLens needed taint analysis to detect source-to-sink data flow vulnerabilities
+(SQL injection, XSS, SSRF, path traversal, command injection). The first
+attempt — `semantic_engine.py` — used regex pattern matching on source code
+strings. This produced unacceptable false positives:
+
+- String literals containing `request` or `query` were flagged as taint
+  sources even when they appeared in comments or unrelated variable names.
+- No path sensitivity: if a sanitizer ran on one branch of an `if/else`,
+  the regex engine still flagged the other branch.
+- No scope awareness: a variable named `user_input` in function A was
+  treated as tainted in function B even though they had no data dependency.
+- No inter-procedural flow: `def f(x): return x` followed by
+  `f(request.args)` was not recognized as passing taint through `f`.
+
+Agents using CodeLens reported that taint findings were "noisy and
+untrustworthy" — they had to manually re-verify every finding, which defeated
+the purpose of automated analysis.
+
+## Goal
+
+Produce taint analysis with:
+- <5% false-positive rate on standard vulnerable-app fixtures
+- Full taint path rendering (source → intermediate → sink) so agents can
+  verify the finding without re-reading source code
+- Path sensitivity (different taint states on if/else branches)
+- Inter-procedural flow within a single file
+- Confidence scores so agents can prioritize high-confidence findings first
+
+## Changes
+
+### Architecture
+
+Six-phase pipeline per file:
+
+1. **Parse** — tree-sitter produces an AST (language-aware, no regex)
+2. **CFG construction** — basic blocks with branches and joins
+3. **Source identification** — built-in patterns + YAML rule definitions
+4. **Forward propagation** — taint flows through assignments, calls, returns
+5. **Sink check** — does taint arrive at a known sink?
+6. **Finding generation** — render full path + confidence score
+
+### New Files
+
+- `scripts/ast_taint_engine.py` — the engine itself (~3700 lines)
+- `scripts/crossfile_taint_engine.py` — cross-file wrapper (Phase 1 of #49
+  made this a thin compat layer over `ast_taint_engine.analyze_workspace(cross_file=True)`)
+- `scripts/commands/taint.py` — CLI command
+- `scripts/rules/python_security.yaml` — built-in taint rules
+- `scripts/rules/javascript_security.yaml` — built-in taint rules
+
+### Modified Files
+
+- `scripts/codelens.py` — auto-registers `taint` command via `commands/__init__.py`
+- `scripts/mcp_server.py` — `codelens_taint` MCP tool
+
+### Confidence Scoring
+
+| Score | Meaning |
+|-------|---------|
+| 0.95+ | Direct source→sink, no sanitizer, same scope |
+| 0.80+ | Source→sink through function call, no sanitizer |
+| 0.60+ | Source→sink with partial sanitizer |
+| 0.40+ | Indirect taint, may be sanitized |
+
+## Trade-offs
+
+### Alternative A: Regex-based (`semantic_engine.py`)
+
+- **Pros:** Fast, no tree-sitter dependency, simple to add new patterns
+- **Cons:** No path sensitivity, no scope awareness, high false-positive rate
+- **Why rejected:** False positives made the feature unusable for agents.
+  `semantic_engine.py` is now deprecated (PR #140) and prints a warning on
+  every use.
+
+### Alternative B: LSP-based taint analysis
+
+- **Pros:** Uses language servers' type inference — would catch flows the
+  AST-only approach misses (e.g., dynamic dispatch)
+- **Cons:** Requires a running LSP server per language, 10-30s startup time,
+  not all languages have LSP servers, results vary by LSP implementation
+- **Why rejected:** Too heavy a dependency for the core engine. LSP
+  verification is available as an optional `--deep` enhancement via
+  `hybrid_engine.py`, not as the primary path.
+
+### Alternative C: Dataflow engine reuse
+
+- **Pros:** `dataflow_engine.py` already does some flow tracking
+- **Cons:** It tracks variable assignments, not taint semantics (source/sink).
+  Reusing it would require grafting on taint-specific logic, producing a
+  Frankenstein engine.
+- **Why rejected:** Separation of concerns — dataflow answers "where does
+  this value come from?", taint answers "is this a security vulnerability?".
+
+### Chosen approach: Tree-sitter AST + CFG
+
+- **Why:** Language-aware (no regex false positives), path-sensitive (CFG
+  tracks branches), inter-procedural (follows calls within a file), and
+  tree-sitter is already a CodeLens dependency for parsing. The cross-file
+  extension (#49 Phase 1) adds inter-procedural flow across files without
+  changing the per-file algorithm.
+
+## Open Questions
+
+- [x] Q1: How to handle library method approximation? — **Resolved** by
+  #49 Phase 4 (not yet implemented as of 2026-07-01; cross-file flow is
+  Phase 1, library approximation is Phase 4).
+- [x] Q2: Should taint findings be persisted to SQLite? — **Resolved**:
+  yes, via `persistent_registry.store_scan_result()`.
+- [ ] Q3: How to handle taint through async/await boundaries? — **Open**.
+  Current engine treats `await f()` as a synchronous call, which may miss
+  taint flow through event-loop-mediated callbacks.
+
+## Migration / Rollout
+
+The AST taint engine is additive — it does not replace `semantic_engine.py`
+(which remains as a deprecated alias for backward compatibility). Users who
+had `semantic_engine` in their CI scripts see a deprecation warning but
+their scripts continue to work.
+
+No database migration — taint findings are stored in the existing
+`scan_results` table via the standard `store_scan_result()` path.
+
+## References
+
+- Issue: #49 (taint analysis depth — multi-phase consolidation)
+- PR: #140 (Phase 1 — cross-file consolidation)
+- Prior art: Semgrep's taint mode, CodeQL's dataflow analysis
+- Related design docs: [0003-plugin-system](0003-plugin-system.md) (taint
+  rules can be shipped as a `rule_pack` plugin)