diff --git a/.agent/issues/adr-kit-mcp-issue.md b/.agent/issues/adr-kit-mcp-issue.md deleted file mode 100644 index ad4e9e8..0000000 --- a/.agent/issues/adr-kit-mcp-issue.md +++ /dev/null @@ -1,693 +0,0 @@ -# ADR Kit MCP Server - Issue Report from Real-World Usage - -**Date**: 2026-01-29 -**ADR Kit Installed Version**: 0.2.5 (verified via `/site-packages/adr_kit-0.2.5.dist-info/METADATA`) -**ADR Kit Health Check Reports**: "Update available: v0.2.2 → v0.2.5" (INCORRECT - already on 0.2.5) -**Context**: Using ADR Kit MCP server with Claude Code to document architectural decisions for an existing project - -**Version Discrepancy Note**: The `adr-kit mcp-health` command incorrectly reports "Update available: v0.2.2 → v0.2.5" even though v0.2.5 is already installed. This is a separate bug in the health check version detection logic. - ---- - -## Executive Summary - -During a brownfield ADR adoption session, we discovered that while the ADR Kit MCP server **successfully creates and parses ADRs**, there is a **critical gap in the workflow**: the `adr_create` MCP tool does not provide guidance on the policy structure needed for constraint extraction to work. This results in ADRs that are created but cannot be used effectively by `adr_planning_context` for preflight checks. - -**What Works**: ✅ MCP connection, ADR creation, ADR parsing, ADR counting -**What Doesn't Work**: ❌ Constraint extraction, policy guidance, format documentation - ---- - -## Detailed Findings - -### 1. MCP Server Connection: ✅ **WORKS** - -The MCP server connects successfully and is accessible to Claude Code agents: - -```bash -$ claude mcp list -Checking MCP server health... - -playwright: npx @playwright/mcp@latest - ✓ Connected -adr-kit: /Users/.../uv/tools/adr-kit/bin/adr-kit mcp-server - ✓ Connected -``` - -**Verdict**: Connection and health checks work perfectly. - ---- - -### 2. ADR Creation via `adr_create`: ✅ **WORKS** (partially) - -The `adr_create` MCP tool successfully creates ADR files: - -**Input**: -```json -{ - "title": "Use Monorepo Structure", - "context": "This project needs...", - "decision": "Use a monorepo...", - "consequences": "Positive: ..., Negative: ...", - "alternatives": "Polyrepo: ...", - "tags": ["architecture", "monorepo"], - "deciders": ["architect"], - "adr_dir": "docs/adr" -} -``` - -**Output**: -```json -{ - "status": "success", - "message": "ADR ADR-0001 created successfully", - "data": { - "adr_id": "ADR-0001", - "file_path": "docs/adr/ADR-0001-use-monorepo-structure.md", - "status": "proposed", - "conflicts": [], - "related_adrs": [], - "validation_warnings": [] - } -} -``` - -**Result**: 9 ADRs successfully created in `docs/adr/` directory. - -**Verdict**: File creation works, but see issues below about content structure. - ---- - -### 3. ADR Analysis via `adr_analyze_project`: ✅ **WORKS** - -The `adr_analyze_project` tool successfully scans and counts ADRs: - -**Input**: -```json -{ - "adr_dir": "docs/adr" -} -``` - -**Output** (truncated, 405,971 characters total): -```json -{ - "status": "success", - "message": "Project analysis completed - found 11 technologies", - "data": { - "existing_adr_count": 9, - "existing_adr_directory": "/Users/.../docs/adr", - "detected_technologies": [ - "React", "Vue", "Angular", "Express.js", "FastAPI", - "Django", "Flask", "TypeScript", "Python", "JavaScript", "Docker" - ], - ... - } -} -``` - -**Verdict**: ADR discovery and counting works correctly. - ---- - -### 4. Constraint Extraction via `adr_planning_context`: ❌ **FAILS** - -This is the **critical failure point**. When querying for architectural context, the tool finds **0 relevant ADRs** and extracts **0 constraints**: - -**Input**: -```json -{ - "task_description": "Test how ADR Kit parses ADR content and extracts constraints", - "domain_hints": ["backend"], - "adr_dir": "docs/adr" -} -``` - -**Output**: -```json -{ - "status": "success", - "message": "Planning context provided with 0 relevant ADRs", - "data": { - "relevant_adrs": [], - "constraints": [], - "guidance": ["Before implementing, ensure your approach aligns with existing architectural decisions."], - "use_technologies": [], - "avoid_technologies": [], - "patterns": [], - "checklist": [ - "✓ Check that new components follow established patterns", - "✓ Verify that dependencies align with architectural decisions", - "✓ Ensure security considerations are addressed per ADRs" - ], - "related_decisions": [] - }, - "metadata": { - "task": "Test how ADR Kit parses ADR content and extracts constraints", - "context_type": "implementation", - "relevant_count": 0 - } -} -``` - -**Expected**: Should extract constraints like "Use FastAPI", "Don't use Flask", "Backend must be in apps/api/", etc. - -**Actual**: Extracted 0 constraints from 9 existing ADRs. - -**Verdict**: Constraint extraction completely fails despite ADRs existing. - ---- - -## Root Cause Analysis - -### Investigation into ADR Kit Source Code - -We examined the ADR Kit source code at: -``` -/Users/.../uv/tools/adr-kit/lib/python3.13/site-packages/adr_kit/ -``` - -#### Finding 1: Policy Extraction Logic Exists - -**File**: `adr_kit/core/policy_extractor.py` - -The policy extractor implements a **three-tier extraction strategy**: - -1. **Structured policy from front-matter** (primary) -2. **Pattern matching from content** (backup) -3. **Future AI-assisted extraction** (placeholder) - -**Pattern matching looks for**: -```python -# Import ban patterns -r"(?i)(?:don't\s+use|avoid|ban|deprecated?)\s+([a-zA-Z0-9\-_@/]+)" -r"(?i)no\s+longer\s+use\s+([a-zA-Z0-9\-_@/]+)" -r"(?i)([a-zA-Z0-9\-_@/]+)\s+is\s+deprecated" - -# Preference patterns -r"(?i)use\s+([a-zA-Z0-9\-_@/]+)\s+instead\s+of\s+([a-zA-Z0-9\-_@/]+)" -r"(?i)replace\s+([a-zA-Z0-9\-_@/]+)\s+with\s+([a-zA-Z0-9\-_@/]+)" -r"(?i)prefer\s+([a-zA-Z0-9\-_@/]+)\s+over\s+([a-zA-Z0-9\-_@/]+)" - -# Boundary patterns -r"(?i)([a-zA-Z0-9\-_]+)\s+should\s+not\s+(?:access|call|use)\s+([a-zA-Z0-9\-_]+)" -r"(?i)no\s+direct\s+access\s+from\s+([a-zA-Z0-9\-_]+)\s+to\s+([a-zA-Z0-9\-_]+)" -``` - -**Problem**: Our ADRs don't use this language. They say things like: -- "Rejected: Flask - Simpler but synchronous" -- "Alternatives Considered: Django REST Framework" - -These don't match the patterns, so **0 constraints are extracted**. - -#### Finding 2: Structured Policy Format Exists but is Undocumented - -**File**: `adr_kit/core/model.py` - -The code defines structured policy models: - -```python -class ImportPolicy(BaseModel): - """Policy for import restrictions and preferences.""" - disallow: list[str] | None - prefer: list[str] | None - -class BoundaryPolicy(BaseModel): - """Policy for architectural boundaries.""" - layers: list[BoundaryLayer] | None - rules: list[BoundaryRule] | None - -class PythonPolicy(BaseModel): - """Python-specific policy rules.""" - disallow_imports: list[str] | None - -class PolicyModel(BaseModel): - """Structured policy model for ADR enforcement.""" - imports: ImportPolicy | None - boundaries: BoundaryPolicy | None - python: PythonPolicy | None - rationales: list[str] | None -``` - -**File**: `adr_kit/mcp/models.py` (line 91-93) - -The `CreateADRRequest` accepts a `policy` parameter: - -```python -policy: dict[str, Any] = Field( - default_factory=dict, - description="Structured policy block for enforcement" # ❌ TOO VAGUE -) -``` - -**Problem**: The description "Structured policy block for enforcement" provides **no guidance** on: -- What keys should be in the dict? -- What's the expected structure? -- How does it map to `ImportPolicy`, `BoundaryPolicy`, etc.? -- Are there examples anywhere? - -#### Finding 3: MCP Tool Description is Insufficient - -**File**: `adr_kit/mcp/server.py` - -The `adr_create` tool description: - -```python -def adr_create(request: CreateADRRequest) -> dict[str, Any]: - """ - Create a new architectural decision record. - - WHEN TO USE: Document significant technical decisions. - RETURNS: Created ADR details in 'proposed' status. - """ -``` - -**Problem**: The docstring doesn't mention: -- The `policy` parameter -- How to structure policies for constraint extraction -- Pattern-matching language requirements -- Examples of valid input - -**Result**: AI agents (like Claude Code) call this tool with no knowledge of policy requirements, creating ADRs that **cannot be used for constraint extraction**. - ---- - -## What the Agent Expected vs What Happened - -### What We Expected (Ideal Workflow) - -1. **Agent calls `adr_create`** with decision details -2. **MCP tool provides guidance** via description or examples -3. **ADR is created** with proper policy structure -4. **Validation feedback** if policy is malformed -5. **Later, `adr_planning_context` works** because policies are properly structured - -### What Actually Happened - -1. ✅ Agent calls `adr_create` with decision details -2. ❌ **No guidance provided** - agent doesn't know about policy requirements -3. ✅ ADR is created (file successfully written) -4. ❌ **No validation** - ADR has no structured policy, but no warning -5. ❌ **Later, `adr_planning_context` returns empty** - constraints can't be extracted - ---- - -## The Disconnect - -**ADR Kit has two undocumented modes**: - -### Mode 1: Structured Policy (Preferred, Undocumented) - -```yaml ---- -id: "ADR-0002" -title: "Use FastAPI as Web Framework" -status: proposed -policy: # ← This is what's needed but NEVER documented - imports: - disallow: ["flask", "django", "litestar"] - prefer: ["fastapi"] - python: - disallow_imports: ["flask", "django"] ---- -``` - -**Problem**: Agents don't know to include this because: -- Not in MCP tool description -- Not in request model field description -- No examples provided -- No validation errors if missing - -### Mode 2: Pattern Matching (Backup, Undocumented) - -ADRs could use pattern-friendly language: - -```markdown -## Decision - -Use FastAPI as the web framework. **Don't use Flask** or Django as they don't meet our async requirements. **Prefer FastAPI over Flask** for this use case. - -## Consequences - -**Avoid** synchronous frameworks like Flask. Backend **should not use** Django REST Framework. -``` - -**Problem**: Agents don't know to use this language because: -- Patterns are not documented -- Standard MADR format doesn't use this language -- No guidance in MCP tool -- Feels unnatural ("Don't use Flask" vs "Rejected: Flask") - ---- - -## Impact on Agents - -When AI agents use ADR Kit, they: - -1. **Successfully create ADRs** ✅ (files are written) -2. **Think everything worked** ✅ (no errors returned) -3. **Later, preflight checks fail** ❌ (no constraints available) -4. **Cannot understand why** ❌ (no feedback loop) - -The agent has **no way to know** that the ADRs it created are "hollow" - they exist as files but lack the structure needed for constraint extraction. - ---- - -## Example: What We Created vs What Was Needed - -### What the Agent Created (Doesn't Work) - -```yaml ---- -id: "ADR-0002" -title: "Use FastAPI as Web Framework" -status: proposed -date: 2026-01-29 -deciders: ['architect'] -tags: ['backend', 'framework', 'python', 'fastapi'] ---- - -## Context - -This project's backend needs a Python web framework... - -## Decision - -Use FastAPI as the backend web framework. Leverage FastAPI's built-in Pydantic integration for request/response validation. - -## Consequences - -### Positive -- Native async/await support -- Automatic OpenAPI/Swagger documentation -- Excellent documentation and large community - -### Negative -- Smaller ecosystem compared to Django/Flask -- Team needs understanding of async Python patterns - -## Alternatives - -### Django + Django REST Framework -- Rejected: More opinionated and heavier for API-only use case - -### Flask -- Rejected: Less integrated type safety and validation -- Async support is a bolt-on rather than native - -### Litestar -- Rejected: Smaller community despite similar features -``` - -**Constraint extraction result**: `constraints: []` ❌ - -### What Was Needed (Option 1: Structured Policy) - -```yaml ---- -id: "ADR-0002" -title: "Use FastAPI as Web Framework" -status: proposed -date: 2026-01-29 -deciders: ['architect'] -tags: ['backend', 'framework', 'python', 'fastapi'] -policy: # ← CRITICAL: This was missing - imports: - disallow: ["flask", "django", "litestar", "django-rest-framework"] - prefer: ["fastapi"] - python: - disallow_imports: ["flask", "django"] - rationales: - - "FastAPI provides native async support required for I/O operations" - - "Automatic OpenAPI documentation reduces maintenance burden" ---- - -## Context -... -``` - -**Constraint extraction result**: `constraints: ["Don't use flask", "Don't use django", "Prefer fastapi"]` ✅ - -### What Was Needed (Option 2: Pattern-Friendly Language) - -```markdown -## Decision - -Use FastAPI as the backend web framework. **Don't use Flask** or **Django** as alternatives. **Prefer FastAPI over Flask** for async support and automatic documentation. - -## Consequences - -### Positive -- Native async/await support -- Automatic OpenAPI/Swagger documentation - -### Negative -- Smaller ecosystem compared to Django/Flask - -## Alternatives - -### Django + Django REST Framework -- **Avoid Django** for this use case - too heavy and opinionated - -### Flask -- **Don't use Flask** - lacks native async and integrated validation -``` - -**Constraint extraction result**: `constraints: ["Don't use Flask", "Don't use Django", "Prefer FastAPI over Flask", "Avoid Django"]` ✅ - ---- - -## Recommended Fixes - -### Fix 1: Update MCP Tool Description (High Priority) - -**File**: `adr_kit/mcp/server.py` - -**Current**: -```python -def adr_create(request: CreateADRRequest) -> dict[str, Any]: - """ - Create a new architectural decision record. - - WHEN TO USE: Document significant technical decisions. - RETURNS: Created ADR details in 'proposed' status. - """ -``` - -**Proposed**: -```python -def adr_create(request: CreateADRRequest) -> dict[str, Any]: - """ - Create a new architectural decision record with optional policy enforcement. - - WHEN TO USE: Document significant technical decisions. - RETURNS: Created ADR details in 'proposed' status. - - POLICY STRUCTURE (for constraint extraction): - The 'policy' parameter should contain structured enforcement rules: - - { - "imports": { - "disallow": ["library-to-ban", "another-banned-lib"], - "prefer": ["preferred-library"] - }, - "python": { - "disallow_imports": ["banned.module"] - }, - "boundaries": { - "rules": [ - {"forbid": "frontend -> database"} - ] - }, - "rationales": [ - "Reason for policy constraint" - ] - } - - ALTERNATIVE (pattern matching): - If policy is not provided, use pattern-friendly language in content: - - "Don't use X" / "Avoid X" / "X is deprecated" - - "Use Y instead of X" / "Prefer Y over X" - - "Layer A should not access Layer B" - - NOTE: Structured policy is preferred for reliable constraint extraction. - """ -``` - -### Fix 2: Update Request Model Field Description (High Priority) - -**File**: `adr_kit/mcp/models.py` (line 91-93) - -**Current**: -```python -policy: dict[str, Any] = Field( - default_factory=dict, - description="Structured policy block for enforcement" -) -``` - -**Proposed**: -```python -policy: dict[str, Any] = Field( - default_factory=dict, - description="""Structured policy block for enforcement. Schema: { - 'imports': {'disallow': [str], 'prefer': [str]}, - 'python': {'disallow_imports': [str]}, - 'boundaries': {'rules': [{'forbid': str}]}, - 'rationales': [str] - }. If omitted, pattern matching from content will be used as fallback.""" -) -``` - -### Fix 3: Add Policy Validation (Medium Priority) - -**File**: `adr_kit/workflows/creation.py` (or wherever ADR creation workflow lives) - -Add validation step that checks if: -1. Policy structure is provided and valid, OR -2. Content contains pattern-matching language - -If neither exists, return a **warning** (not error): - -```json -{ - "validation_warnings": [ - "No structured policy provided and no pattern-matching language detected in content. Constraint extraction may not work. Consider adding a 'policy' block or using phrases like 'Don't use X' in your decision text." - ] -} -``` - -### Fix 4: Add Template/Example to Response (Low Priority) - -When an ADR is created without a policy, include a `suggested_policy` in the response showing what could be added: - -```json -{ - "status": "success", - "data": { - "adr_id": "ADR-0002", - "file_path": "docs/adr/ADR-0002-...", - "suggested_policy": { - "imports": { - "disallow": [""], - "prefer": [""] - } - } - }, - "next_steps": [ - "Review ADR content", - "Consider adding structured policy for constraint extraction", - "Use 'Don't use X' or 'Prefer Y' language if policy is not provided" - ] -} -``` - -### Fix 5: Documentation Update (High Priority) - -Add a section to ADR Kit README/docs: - -**Title**: "Writing ADRs for Constraint Extraction" - -**Content**: -- Explain the two-tier approach (structured policy vs pattern matching) -- Provide examples of both approaches -- Show what gets extracted from each -- Document the policy schema with all available fields -- Show how constraints appear in `adr_planning_context` output - ---- - -## Testing Checklist - -To verify fixes work: - -1. ✅ **Create ADR with structured policy** - - Include `policy.imports.disallow` and `policy.imports.prefer` - - Call `adr_planning_context` - - Verify constraints are extracted - -2. ✅ **Create ADR with pattern-matching language** - - Use phrases like "Don't use Flask" - - Call `adr_planning_context` - - Verify constraints are extracted - -3. ✅ **Create ADR without policy or patterns** - - Omit `policy` parameter - - Use generic language like "Rejected: Flask" - - Verify warning is returned about constraint extraction - -4. ✅ **Invalid policy structure** - - Provide malformed policy dict - - Verify clear error message with expected schema - ---- - -## Summary - -**What Works**: -- ✅ MCP server connection and health -- ✅ ADR file creation -- ✅ ADR discovery and counting -- ✅ The constraint extraction **logic** (patterns and structured policy parsing) - -**What's Broken**: -- ❌ **No guidance** for agents on how to structure policies -- ❌ **No documentation** of policy schema in MCP tool -- ❌ **No validation** or feedback when policy is missing -- ❌ **No warnings** when ADRs can't be used for constraints -- ❌ **Silent failure** mode - everything appears to work but doesn't - -**Impact**: -- Agents successfully create ADRs that **appear valid** -- Constraint extraction **silently returns empty results** -- No feedback loop to indicate the problem -- Preflight checks cannot work without constraints -- The core value proposition of ADR Kit (automated enforcement) is broken for AI agents - -**Recommendation**: -Prioritize **Fix 1** (MCP tool description) and **Fix 2** (field description) as these are documentation-only changes that would immediately enable agents to create properly structured ADRs. - ---- - -## Additional Issues Found - -### Issue #2: Health Check Version Detection Bug - -**Command**: `adr-kit mcp-health` - -**Reports**: -``` -🔄 Update available: v0.2.2 → v0.2.5 -``` - -**Actual Installed Version** (verified): -```bash -$ cat ~/.local/share/uv/tools/adr-kit/lib/python3.13/site-packages/adr_kit-0.2.5.dist-info/METADATA -Metadata-Version: 2.4 -Name: adr-kit -Version: 0.2.5 -``` - -**Problem**: The health check incorrectly reports that an update is available when already on the latest version. This creates confusion about which version is actually running. - -**Impact**: Minor - causes confusion but doesn't affect functionality. - -**Suggested Fix**: Fix version detection logic in `adr-kit mcp-health` command to correctly identify installed version. - ---- - -## Additional Context - -- **Installation**: ADR Kit 0.2.5 via `uv tool install adr-kit` (despite health check claiming 0.2.2) -- **Usage Pattern**: Claude Code agent with @architect and @adr-clerk specialized agents -- **Project Type**: Brownfield ADR adoption (documenting existing decisions) -- **ADRs Created**: 9 ADRs covering monorepo structure, FastAPI, React, TypeScript, SQLite, etc. -- **All ADRs**: Successfully created as files but unusable for constraint extraction - ---- - -## For the ADR Kit Team - -This issue comes from real-world usage by an AI agent (Claude Code) attempting to use ADR Kit as designed. The agent followed the MCP tool interface exactly as documented, yet the resulting ADRs cannot fulfill their intended purpose. - -The fix is straightforward: **better documentation and validation at the MCP layer**. The underlying extraction logic works fine - agents just need guidance on how to structure input so the extraction has something to extract. - -Happy to provide more details or test fixes if needed! diff --git a/.agent/requirements/adr-kit-vision-architectural.md b/.agent/requirements/adr-kit-vision-architectural.md deleted file mode 100644 index f4f3c1d..0000000 --- a/.agent/requirements/adr-kit-vision-architectural.md +++ /dev/null @@ -1,131 +0,0 @@ -# ADR‑Kit — Vision & Architectural Blueprint -_Last updated: 2025-09-05 10:41:12Z_ - -This document captures the **vision**, **why it matters**, and a **high‑level architecture** for ADR‑Kit. It is intentionally **implementation‑agnostic**: it describes behaviors, contracts, and workflows rather than folder layouts or specific technologies. Use it to align design discussions, evaluate current implementations, and plan evolutions. - ---- - -## 1) Purpose & Outcomes -**Goal:** Keep the human architect in control while agents implement code. -**Outcomes we want:** -- Decisions are **explicit** (ADR), **approved by a human**, and **discoverable** by agents. -- Agents **follow** accepted decisions, **pause** to propose new ones when needed, and **never drift** silently. -- Enforcement is **deterministic** (policy + checks) and visible in **automation** (CI). -- Guidance to agents is **contextual and minimal** (tool‑local promptlets), not via giant system prompts. - -**Primary users:** human architects/maintainers, IDE‑embedded agents, CI pipelines. - ---- - -## 2) Capabilities (what the system must do) -- **ADR Management:** Create, view, update, relate, and change the lifecycle state of ADRs (draft → proposed → accepted → deprecated/superseded). -- **Constraints Contract:** Produce a compact, machine‑readable summary of all **accepted** rules (“the contract”) for agents to obey. -- **Policy Gate for New Choices:** Deterministic gating for major technical choices (e.g., runtime deps, frameworks). Default: **require ADR** before proceeding. -- **Planning Context:** Provide agents a small planning packet: hard constraints + most relevant decisions for the current task. -- **Guardrails:** Generate/maintain configuration fragments (lint, dependency rules, CI checks) that reflect accepted decisions; validate the repo for violations. -- **Guided Interactions:** Return short, imperative guidance (“promptlets”) with tool responses to steer agents at the moment of action. -- **Lifecycle Hooks:** When a decision’s status changes, automatically update the contract and guardrails accordingly. -- **Observability & Audit:** Record key events (proposals, acceptances, violations, guardrail changes) to enable review and learning. - ---- - -## 3) Architectural Tenets -- **Human‑first + Machine‑first:** ADRs remain human‑legible; their enforcement rules are machine‑readable. -- **Deterministic & Idempotent:** Same inputs → same outputs; merges and validations are rule‑based, not heuristic. -- **Local, Contextual Steering:** Guidance is attached to tool responses that need it; keep prompts small and specific. -- **Minimal Token Footprint:** Share IDs, summaries, and contracts, not full documents, unless specifically requested. -- **Composable & Extensible:** New languages/policies integrate via adapters; storage/retrieval strategies are pluggable. -- **Fail Loud & Early:** Clear error codes and “next steps” for agents; CI fails fast on violations. - ---- - -## 4) High‑Level Architecture (conceptual components) -- **ADR Store:** Human‑legible decision records with lifecycle fields and links (supersedes/superseded‑by/relates‑to). -- **Constraints Builder:** Merges enforcement rules from all **accepted** decisions into a single, small **contract** object. Detects and blocks conflicts. -- **Policy Engine:** Deterministic gate for new or changed technical choices. Supports default “require ADR”, allow/deny lists, categories, and simple keyword/alias maps. -- **Planning Context Service:** Produces a **planning packet**: the contract + a shortlist of relevant ADRs (via simple filtering and lexical ranking). -- **Guardrail Manager:** Calculates and applies **configuration fragments** (e.g., lint/import/dep rules, CI checks). Writes only inside clearly marked, tool‑owned blocks. -- **Validator:** Scans manifests, imports, and configs to detect violations; emits precise findings and suggested remedies. -- **MCP Interface:** Exposes these capabilities as tools with a **uniform response envelope** that can include short, imperative guidance for the agent. - -> Each component can be implemented with technologies that fit your stack; the design does not prescribe specific storage formats or folder structures. - ---- - -## 5) Key Contracts & Data (format‑agnostic) -- **ADR Record:** id, title, status, tags/scope, effective date, links (supersedes/superseded‑by/relates‑to), rationale sections, short “why” summary, and **optional** machine rules. -- **Constraints Contract:** a compact object derived from all accepted ADRs that agents can obey without reading ADR prose (e.g., allow/deny lists, banned imports, required checks). Includes provenance (which decisions contributed) and a change hash for caching. -- **Policy Definition:** default posture (e.g., require ADR for runtime deps), category hints for “major choices”, allow/deny lists, alias mapping for names. -- **Planning Packet:** hard constraints + shortlist of relevant ADRs for a task + small guidance for how to incorporate them. -- **Response Envelope:** ok/code/data + optional guidance for the agent (priority, next step), provenance, and optional token hints. -- **Error Taxonomy:** stable codes (e.g., NEW_DEP_REQUIRES_ADR, ADR_CONFLICT, VALIDATION_FAILED) with actionable next steps. - ---- - -## 6) Core Workflows (behavioral view) -### 6.1 Cold Start -- Establish default policy (e.g., “require ADR” for new runtime dependencies or frameworks). -- Create a small set of baseline decisions if desired (language/runtime, dependency governance). - -### 6.2 Planning a Feature -- Agent requests the **planning packet**. -- Packet returns the contract + a few relevant decisions. -- Agent cites decision IDs in its plan and proceeds in line with the contract. - -### 6.3 Introducing a New Major Choice -- Before adding a runtime dependency or framework‑level item, the policy engine evaluates the action. -- Outcomes: **Allowed**, **Requires ADR**, or **Blocked/Conflicts**. -- On **Requires ADR**, the agent drafts a proposal and asks the human to approve; once accepted, the system updates the contract and guardrails. - -### 6.4 Superseding a Decision -- Agent or human proposes a new decision that supersedes an existing one. -- On acceptance, the system updates links, refreshes the contract, and adjusts guardrails accordingly. - -### 6.5 Validation & Automation -- Validator checks for violations in manifests/imports/configs. -- CI integrates the validator to gate changes; the contract’s hash helps detect drift. - ---- - -## 7) Guidance & Promptlets -- Tool responses can include a **small, imperative instruction** to the agent (“explain conflict; ask for approval to draft ADR; do not proceed until resolved”). -- Promptlets are reusable snippets referenced by ID; the system ensures they do not contradict accepted decisions. On contradiction, return a specific error and stop. - ---- - -## 8) Extensibility -- **Language/Framework Adapters:** Pluggable modules for different ecosystems to interpret and enforce decisions (e.g., dependency/import checks, lint rules, CI glue). -- **Retrieval Strategy:** Start with simple filtering + lexical ranking for relevant ADRs; add semantics only when the decision set grows large. -- **Storage:** ADRs and contracts can live wherever is convenient (files, DB, etc.); the behavior is the same. -- **Interfaces:** MCP or other interfaces can expose the same capabilities; response envelopes remain consistent. - ---- - -## 9) Success Criteria & Non‑Goals -**Success:** -- Agents cite decisions in plans, pause for approval on major new choices, and follow the contract. -- Minimal drift: violations are caught early; automation blocks unsafe changes. -- Human effort is focused on **approving decisions**, not policing implementation details. - -**Non‑Goals (for v1):** background daemons by default, heavy semantic retrieval, auto‑refactors beyond owned config fragments, deep static analysis across all languages. - ---- - -## 10) Open Design Questions (to drive discussion) -- Decision **granularity**: which choices always need ADRs vs. can be policy‑allowed? -- **Taxonomy** for tags/scope to make retrieval predictable. -- **Authority model**: who approves/supersedes decisions? -- **CI strictness**: hard‑fail vs. warn for specific categories. -- **Observability**: which events are logged, and where? -- **Adapter roadmap**: which ecosystems to prioritize next? - ---- - -## 11) Minimal MVP (implementation‑agnostic) -- ADR management with lifecycle, short summaries, and links. -- Deterministic constraints contract built from **accepted** decisions. -- Policy gate that defaults to **require ADR** for major choices. -- Planning packet with small, relevant context for agents. -- Guardrail generation/validation with clearly marked, tool‑owned fragments. -- CI integration to block violations. -- Uniform response envelopes with optional promptlets and clear error codes. diff --git a/.agent/verification/verify_policy_issue.py b/.agent/verification/verify_policy_issue.py deleted file mode 100644 index a35c083..0000000 --- a/.agent/verification/verify_policy_issue.py +++ /dev/null @@ -1,227 +0,0 @@ -"""Verification script for policy extraction issue. - -This script reproduces the issue reported in .agent/issues/adr-kit-mcp-issue.md: -- ADRs created without policy structure -- Constraint extraction returns empty results -- No warnings or feedback provided -""" - -import tempfile -from pathlib import Path - -from adr_kit.core.model import ADR, ADRFrontMatter, ADRStatus, PolicyModel -from adr_kit.core.policy_extractor import PolicyExtractor -from adr_kit.workflows.creation import CreationInput, CreationWorkflow - - -def test_adr_without_policy(): - """Test Case 1: ADR without policy (mimics agent behavior from issue report).""" - print("\n" + "=" * 70) - print("TEST 1: ADR without policy (BAD - like the issue report)") - print("=" * 70) - - with tempfile.TemporaryDirectory() as tmpdir: - workflow = CreationWorkflow(adr_dir=tmpdir) - - # Create ADR exactly as the agent did in the issue report - input_data = CreationInput( - title="Use FastAPI as Web Framework", - context="This project's backend needs a Python web framework with async support.", - decision="Use FastAPI as the backend web framework. Leverage FastAPI's built-in Pydantic integration.", - consequences="""### Positive -- Native async/await support -- Automatic OpenAPI/Swagger documentation -- Excellent documentation and large community - -### Negative -- Smaller ecosystem compared to Django/Flask -- Team needs understanding of async Python patterns""", - alternatives="""### Django + Django REST Framework -- Rejected: More opinionated and heavier for API-only use case - -### Flask -- Rejected: Less integrated type safety and validation -- Async support is a bolt-on rather than native - -### Litestar -- Rejected: Smaller community despite similar features""", - tags=["backend", "framework", "python", "fastapi"], - deciders=["architect"], - # NOTE: NO POLICY PROVIDED - this is the problem! - ) - - result = workflow.execute(input_data=input_data) - - if result.success: - print(f"✅ ADR created: {result.data['creation_result'].adr_id}") - print( - f"📄 File: {result.data['creation_result'].file_path}" - ) - - # Check validation warnings - warnings = result.data["creation_result"].validation_warnings - print(f"\n⚠️ Validation Warnings: {len(warnings)}") - for warning in warnings: - print(f" - {warning}") - - # Try to extract policy - adr_file = Path(result.data["creation_result"].file_path) - from adr_kit.core.parse import parse_adr_file - - adr = parse_adr_file(adr_file) - extractor = PolicyExtractor() - policy = extractor.extract_policy(adr) - - print(f"\n🔍 Policy Extraction Results:") - print(f" - Disallowed imports: {policy.get_disallowed_imports()}") - print(f" - Preferred imports: {policy.get_preferred_imports()}") - print(f" - Has extractable policy: {extractor.has_extractable_policy(adr)}") - - if not extractor.has_extractable_policy(adr): - print( - "\n❌ PROBLEM CONFIRMED: No constraints extracted! " - "adr_planning_context would return constraints: []" - ) - if not warnings: - print( - "❌ WORSE: No warnings provided! Silent failure." - ) - else: - print(f"❌ Creation failed: {result.errors}") - - -def test_adr_with_structured_policy(): - """Test Case 2: ADR with structured policy (GOOD - should work).""" - print("\n" + "=" * 70) - print("TEST 2: ADR with structured policy (GOOD - should extract constraints)") - print("=" * 70) - - with tempfile.TemporaryDirectory() as tmpdir: - workflow = CreationWorkflow(adr_dir=tmpdir) - - input_data = CreationInput( - title="Use FastAPI as Web Framework", - context="This project's backend needs a Python web framework with async support.", - decision="Use FastAPI as the backend web framework.", - consequences="Native async support and automatic OpenAPI documentation.", - alternatives="Django/Flask rejected due to lack of native async support.", - tags=["backend", "framework"], - deciders=["architect"], - # STRUCTURED POLICY PROVIDED - this should work! - policy={ - "imports": { - "disallow": ["flask", "django", "litestar"], - "prefer": ["fastapi"], - }, - "python": {"disallow_imports": ["flask", "django"]}, - "rationales": [ - "FastAPI provides native async support required for I/O operations", - "Automatic OpenAPI documentation reduces maintenance burden", - ], - }, - ) - - result = workflow.execute(input_data=input_data) - - if result.success: - print(f"✅ ADR created: {result.data['creation_result'].adr_id}") - - # Try to extract policy - adr_file = Path(result.data["creation_result"].file_path) - from adr_kit.core.parse import parse_adr_file - - adr = parse_adr_file(adr_file) - extractor = PolicyExtractor() - policy = extractor.extract_policy(adr) - - print(f"\n🔍 Policy Extraction Results:") - print(f" - Disallowed imports: {policy.get_disallowed_imports()}") - print(f" - Preferred imports: {policy.get_preferred_imports()}") - print(f" - Has extractable policy: {extractor.has_extractable_policy(adr)}") - - if extractor.has_extractable_policy(adr): - print("\n✅ SUCCESS: Constraints extracted! adr_planning_context would work.") - else: - print( - "\n❌ UNEXPECTED: Policy provided but extraction failed!" - ) - else: - print(f"❌ Creation failed: {result.errors}") - - -def test_adr_with_pattern_language(): - """Test Case 3: ADR with pattern-matching language (SHOULD work).""" - print("\n" + "=" * 70) - print("TEST 3: ADR with pattern-matching language (should extract via patterns)") - print("=" * 70) - - with tempfile.TemporaryDirectory() as tmpdir: - workflow = CreationWorkflow(adr_dir=tmpdir) - - input_data = CreationInput( - title="Use FastAPI as Web Framework", - context="This project's backend needs a Python web framework with async support.", - decision="""Use FastAPI as the backend web framework. **Don't use Flask** -or Django as they lack native async support. **Prefer FastAPI over Flask** for this use case.""", - consequences="""**Avoid** synchronous frameworks like Flask. -Backend **should not use** Django REST Framework.""", - alternatives="Rejected Flask and Django.", - tags=["backend"], - deciders=["architect"], - # NO STRUCTURED POLICY - but using pattern-friendly language - ) - - result = workflow.execute(input_data=input_data) - - if result.success: - print(f"✅ ADR created: {result.data['creation_result'].adr_id}") - - # Try to extract policy - adr_file = Path(result.data["creation_result"].file_path) - from adr_kit.core.parse import parse_adr_file - - adr = parse_adr_file(adr_file) - extractor = PolicyExtractor() - policy = extractor.extract_policy(adr) - - print(f"\n🔍 Policy Extraction Results:") - print(f" - Disallowed imports: {policy.get_disallowed_imports()}") - print(f" - Preferred imports: {policy.get_preferred_imports()}") - print(f" - Has extractable policy: {extractor.has_extractable_policy(adr)}") - - if extractor.has_extractable_policy(adr): - print("\n✅ SUCCESS: Patterns detected! Constraint extraction works.") - else: - print( - "\n⚠️ PARTIAL: Pattern-matching failed. May need better patterns." - ) - else: - print(f"❌ Creation failed: {result.errors}") - - -if __name__ == "__main__": - print("\n" + "=" * 70) - print("VERIFICATION: ADR Kit Policy Extraction Issue") - print("Reproducing the issue from: .agent/issues/adr-kit-mcp-issue.md") - print("=" * 70) - - test_adr_without_policy() - test_adr_with_structured_policy() - test_adr_with_pattern_language() - - print("\n" + "=" * 70) - print("VERIFICATION COMPLETE") - print("=" * 70) - print( - """ -Expected Results: -- Test 1: Should show NO constraints extracted + NO warnings (❌ PROBLEM) -- Test 2: Should show constraints extracted (✅ WORKS) -- Test 3: Should show constraints extracted via patterns (✅ WORKS if patterns match) - -Next Steps: -1. Confirm Test 1 demonstrates the problem -2. Implement validation warnings -3. Update MCP documentation -""" - ) diff --git a/adr_kit/core/policy_extractor.py b/adr_kit/core/policy_extractor.py index 11a8096..4ca1d3e 100644 --- a/adr_kit/core/policy_extractor.py +++ b/adr_kit/core/policy_extractor.py @@ -1,239 +1,41 @@ -"""Policy extraction engine with hybrid approach. +"""Policy extraction engine. -This module implements the three-tier policy extraction strategy: -1. Structured policy from front-matter (primary) -2. Pattern matching from content (backup) -3. Future AI-assisted extraction (placeholder) - -Design decisions: -- Structured policy takes priority over pattern extraction -- Pattern matching handles common ADR language patterns -- Merges multiple sources into unified policy model -- Provides clear rationales for extracted policies +Reads structured policy from ADR front-matter only. +Policies are constructed by the agent via the policy guidance promptlet +during ADR creation — never extracted from natural language. """ -import re - from .model import ( ADR, - ImportPolicy, PolicyModel, - PythonPolicy, ) -class PolicyPatternExtractor: - """Extract policies from ADR content using pattern matching.""" - - def __init__(self) -> None: - # Common patterns for identifying banned imports/libraries - self.import_ban_patterns = [ - # "Don't use X", "Avoid X", "Ban X" - r"(?i)(?:don't\s+use|avoid|ban|deprecated?)\s+([a-zA-Z0-9\-_@/]+)", - # "No longer use X" - r"(?i)no\s+longer\s+use\s+([a-zA-Z0-9\-_@/]+)", - # "X is deprecated" - r"(?i)([a-zA-Z0-9\-_@/]+)\s+is\s+deprecated", - ] - - # Patterns for preferred alternatives - self.preference_patterns = [ - # "Use Y instead of X" - r"(?i)use\s+([a-zA-Z0-9\-_@/]+)\s+instead\s+of\s+([a-zA-Z0-9\-_@/]+)", - # "Replace X with Y" - r"(?i)replace\s+([a-zA-Z0-9\-_@/]+)\s+with\s+([a-zA-Z0-9\-_@/]+)", - # "Prefer Y over X" - r"(?i)prefer\s+([a-zA-Z0-9\-_@/]+)\s+over\s+([a-zA-Z0-9\-_@/]+)", - ] +class PolicyExtractor: + """Extract structured policy from ADR front-matter.""" - # Boundary/architecture patterns - self.boundary_patterns = [ - # "X should not access Y" - r"(?i)([a-zA-Z0-9\-_]+)\s+should\s+not\s+(?:access|call|use)\s+([a-zA-Z0-9\-_]+)", - # "No direct access from X to Y" - r"(?i)no\s+direct\s+access\s+from\s+([a-zA-Z0-9\-_]+)\s+to\s+([a-zA-Z0-9\-_]+)", - # "X must go through Y" - r"(?i)([a-zA-Z0-9\-_]+)\s+must\s+go\s+through\s+([a-zA-Z0-9\-_]+)", - ] + def extract_policy(self, adr: ADR) -> PolicyModel: + """Extract policy from structured front-matter. - # Common library name mappings for normalization - self.library_mappings = { - "react-query": "@tanstack/react-query", - "react query": "@tanstack/react-query", - "tanstack query": "@tanstack/react-query", - "axios": "axios", - "fetch": "fetch", - "lodash": "lodash", - "moment": "moment", - "momentjs": "moment", - "moment.js": "moment", - "date-fns": "date-fns", - "dayjs": "dayjs", - "jquery": "jquery", - "underscore": "underscore", - } + Returns the structured policy if present, otherwise an empty PolicyModel. + """ + structured_policy = adr.front_matter.policy - def extract_from_content(self, content: str) -> PolicyModel: - """Extract policy from ADR content using pattern matching.""" - imports = self._extract_import_policies(content) - python_policies = self._extract_python_policies(content) - rationales = self._extract_rationales(content) + if structured_policy: + return structured_policy return PolicyModel( - imports=imports, + imports=None, boundaries=None, - python=python_policies, + python=None, patterns=None, architecture=None, config_enforcement=None, - rationales=rationales, + rationales=None, ) - def _extract_import_policies(self, content: str) -> ImportPolicy | None: - """Extract import-related policies from content.""" - disallow = set() - prefer = set() - - # Extract banned imports - for pattern in self.import_ban_patterns: - matches = re.findall(pattern, content) - for match in matches: - normalized = self._normalize_library_name(match) - if normalized: - disallow.add(normalized) - - # Extract preferences - for pattern in self.preference_patterns: - matches = re.findall(pattern, content) - for match in matches: - if len(match) == 2: # (preferred, deprecated) - preferred, deprecated = match - preferred_norm = self._normalize_library_name(preferred) - deprecated_norm = self._normalize_library_name(deprecated) - - if preferred_norm: - prefer.add(preferred_norm) - if deprecated_norm: - disallow.add(deprecated_norm) - - if disallow or prefer: - return ImportPolicy( - disallow=list(disallow) if disallow else None, - prefer=list(prefer) if prefer else None, - ) - - return None - - def _extract_python_policies(self, content: str) -> PythonPolicy | None: - """Extract Python-specific policies from content.""" - python_libs = {"requests", "urllib", "urllib2", "httplib", "http.client"} - disallow = set() - - # Look for Python-specific import restrictions - for pattern in self.import_ban_patterns: - matches = re.findall(pattern, content) - for match in matches: - if match.lower() in python_libs: - disallow.add(match.lower()) - - if disallow: - return PythonPolicy(disallow_imports=list(disallow)) - - return None - - def _extract_rationales(self, content: str) -> list[str] | None: - """Extract rationales for the policies from content.""" - rationales = set() - - # Common rationale keywords - rationale_patterns = [ - r"(?i)for\s+(performance|security|maintainability|consistency|bundle\s+size)", - r"(?i)to\s+(?:improve|enhance|ensure)\s+(performance|security|maintainability|consistency)", - r"(?i)(?:better|improved)\s+(performance|security|maintainability|developer\s+experience)", - ] - - for pattern in rationale_patterns: - matches = re.findall(pattern, content) - for match in matches: - rationale = match.replace("_", " ").title() - rationales.add(rationale) - - return list(rationales) if rationales else None - - def _normalize_library_name(self, name: str) -> str | None: - """Normalize library names using common mappings.""" - name_lower = name.lower().strip() - return self.library_mappings.get(name_lower, name if len(name) > 1 else None) - - -class PolicyExtractor: - """Main policy extraction engine with hybrid approach.""" - - def __init__(self) -> None: - self.pattern_extractor = PolicyPatternExtractor() - - def extract_policy(self, adr: ADR) -> PolicyModel: - """Extract unified policy using hybrid approach. - - Priority: - 1. Structured policy from front-matter (preferred) - 2. Pattern extraction from content (backup) - 3. Merge both with structured taking priority - """ - # Tier 1: Structured policy (highest priority) - structured_policy = adr.front_matter.policy - - # Tier 2: Pattern extraction (backup) - pattern_policy = self.pattern_extractor.extract_from_content(adr.content) - - # Merge policies with structured taking priority - merged_policy = self._merge_policies(structured_policy, pattern_policy) - - return merged_policy - - def _merge_policies( - self, structured: PolicyModel | None, pattern: PolicyModel | None - ) -> PolicyModel: - """Merge structured and pattern-extracted policies.""" - if structured and pattern: - # Merge both, with structured taking priority - return PolicyModel( - imports=structured.imports or pattern.imports, - boundaries=structured.boundaries or pattern.boundaries, - python=structured.python or pattern.python, - patterns=structured.patterns or pattern.patterns, - architecture=structured.architecture or pattern.architecture, - config_enforcement=structured.config_enforcement - or pattern.config_enforcement, - rationales=self._merge_lists(structured.rationales, pattern.rationales), - ) - elif structured: - return structured - elif pattern: - return pattern - else: - # Return empty policy - explicit None values for mypy - return PolicyModel( - imports=None, - boundaries=None, - python=None, - patterns=None, - architecture=None, - config_enforcement=None, - rationales=None, - ) - - def _merge_lists( - self, list1: list[str] | None, list2: list[str] | None - ) -> list[str] | None: - """Merge two lists, removing duplicates.""" - if list1 and list2: - combined = set(list1) | set(list2) - return list(combined) - return list1 or list2 - def has_extractable_policy(self, adr: ADR) -> bool: - """Check if ADR has any extractable policy information. + """Check if ADR has a structured policy in front-matter. Note: Rationales alone don't count as extractable policy since they don't provide actionable constraints for adr_planning_context. @@ -285,8 +87,8 @@ def validate_policy_completeness(self, adr: ADR) -> list[str]: if adr.front_matter.status == ADRStatus.ACCEPTED: if not self.has_extractable_policy(adr): errors.append( - f"ADR {adr.front_matter.id} is accepted but has no extractable policy. " - "Add structured policy in front-matter or include decision rationales in content." + f"ADR {adr.front_matter.id} is accepted but has no structured policy. " + "Add a structured policy block in front-matter." ) return errors diff --git a/tests/fixtures/examples/good-adr-with-pattern-language.md b/tests/fixtures/examples/good-adr-with-pattern-language.md deleted file mode 100644 index 0d0e7dd..0000000 --- a/tests/fixtures/examples/good-adr-with-pattern-language.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -id: "ADR-0002" -title: "Use React Query for Server State Management" -status: proposed -date: 2026-02-06 -deciders: ["frontend-team", "architect"] -tags: ["frontend", "react", "state-management", "data-fetching"] ---- - -## Context - -Our React application needs a robust solution for managing server state (API data, caching, synchronization). Currently using ad-hoc fetch calls with useState, leading to: -- Duplicated loading/error states across components -- No caching strategy - redundant API calls -- Stale data issues when data changes on server -- Complex synchronization logic scattered throughout codebase - -Team has experience with Redux but finds it too verbose for simple data fetching. - -## Decision - -Use **React Query (TanStack Query)** for all server state management in the application. - -**Don't use Redux** for server state - Redux should only handle UI state (modals, form state, etc). **Avoid** manual fetch + useState patterns. **Prefer React Query over custom data fetching hooks** for consistency. - -All components fetching API data MUST use React Query's useQuery/useMutation hooks. Existing fetch calls **should be migrated** to React Query during regular development (no dedicated migration sprint needed). - -## Consequences - -### Positive - -- **Automatic Caching**: Eliminates redundant API calls, improves performance -- **Simplified Components**: No more manual loading/error state management -- **Background Refetching**: Data stays fresh automatically -- **Optimistic Updates**: Better UX for mutations -- **DevTools**: Excellent debugging experience with React Query DevTools -- **Reduced Bundle Size**: Removes need for Redux Toolkit for data fetching - -### Negative - -- **Learning Curve**: Team needs to understand stale-while-revalidate patterns -- **Different Mental Model**: Declarative data fetching vs imperative fetch calls -- **Cache Invalidation Complexity**: Need to plan cache keys and invalidation strategies -- **Testing Changes**: Need to mock React Query in tests differently than fetch - -### Risks - -- Improper cache key design could lead to stale data bugs -- Team might overuse optimistic updates causing race conditions - -### Mitigation - -- Create React Query wrapper with sensible defaults -- Document cache key naming conventions -- Provide training on stale-time and cache-time configuration -- Add lint rules to prevent direct fetch usage - -## Alternatives - -### Redux + RTK Query - -**Rejected**: Too heavyweight when we don't need global UI state management. - -- Pros: Integrated with Redux, good caching, TypeScript support -- Cons: Requires full Redux setup, more boilerplate, **avoid** for simple use cases -- Why not: We don't need Redux's complexity for our current application - -### SWR (Vercel) - -**Rejected**: Less feature-complete than React Query. - -- Pros: Lightweight, simple API, good caching -- Cons: Fewer features, less active development, smaller community -- Why not: React Query has better mutation support and more features - -### Apollo Client - -**Rejected**: GraphQL-specific, we use REST APIs. - -- Pros: Excellent for GraphQL, mature, good caching -- Cons: Overkill for REST APIs, large bundle size -- Why not: We use REST APIs, **don't use** Apollo for REST - -### Custom Hooks - -**Rejected**: Reinventing the wheel, hard to maintain. - -- Pros: Full control, no dependencies -- Cons: Need to implement caching, refetching, error handling, etc. -- Why not: React Query is battle-tested, **avoid custom fetch abstractions** - -## Implementation Notes - -- Install @tanstack/react-query v5 -- Create `src/lib/react-query.ts` with default configuration -- Add React Query DevTools in development mode -- Update component library docs with data fetching patterns -- Migrate high-traffic components first (dashboard, list views) - -## References - -- [React Query Documentation](https://tanstack.com/query/latest) -- [Practical React Query](https://tkdodo.eu/blog/practical-react-query) -- Internal: Frontend Architecture Guidelines (Notion) diff --git a/tests/unit/test_policy_validation.py b/tests/unit/test_policy_validation.py index fe50884..4cb0c0c 100644 --- a/tests/unit/test_policy_validation.py +++ b/tests/unit/test_policy_validation.py @@ -72,8 +72,8 @@ def test_creation_with_structured_policy_no_warning(self): ] assert len(policy_warnings) == 0 - def test_creation_with_pattern_language_no_warning(self): - """ADR with pattern-matching language should not trigger warning.""" + def test_creation_with_pattern_language_has_warning(self): + """ADR with pattern-matching language but no structured policy should trigger warning.""" with tempfile.TemporaryDirectory() as tmpdir: workflow = CreationWorkflow(adr_dir=tmpdir) @@ -90,13 +90,11 @@ def test_creation_with_pattern_language_no_warning(self): assert result.success creation_result = result.data["creation_result"] - # Should NOT have policy warnings (patterns detected) - policy_warnings = [ - w - for w in creation_result.validation_warnings - if "policy" in w.lower() or "constraint extraction" in w.lower() - ] - assert len(policy_warnings) == 0 + # Should have policy warnings — natural language patterns are not extracted, + # only structured front-matter policy is recognized + assert len(creation_result.validation_warnings) > 0 + warning_text = " ".join(creation_result.validation_warnings) + assert "policy" in warning_text.lower() def test_policy_extractor_with_structured_policy(self): """PolicyExtractor should extract from structured policy.""" @@ -121,38 +119,6 @@ def test_policy_extractor_with_structured_policy(self): assert policy.get_disallowed_imports() == ["flask"] assert policy.get_preferred_imports() == ["fastapi"] - def test_policy_extractor_with_pattern_language(self): - """PolicyExtractor should extract from pattern-matching language.""" - from datetime import date - - front_matter = ADRFrontMatter( - id="ADR-0001", - title="Test", - status=ADRStatus.PROPOSED, - date=date.today(), - ) - - content = """ -## Decision - -Use FastAPI. **Don't use Flask** or Django. -**Prefer FastAPI over Flask** for this use case. - -## Consequences - -**Avoid** synchronous frameworks. -""" - - adr = ADR(front_matter=front_matter, content=content) - - extractor = PolicyExtractor() - assert extractor.has_extractable_policy(adr) is True - - policy = extractor.extract_policy(adr) - # Should extract Flask from "Don't use Flask" - disallowed = policy.get_disallowed_imports() - assert any("flask" in item.lower() for item in disallowed) - def test_policy_extractor_without_policy(self): """PolicyExtractor should return False for ADR without policy.""" from datetime import date