draft

Author: tom-sapletta-com

CHANGELOG.md

@@ -1,3 +1,48 @@
+## [0.3.1] - 2026-03-01
+
+### Summary
+
+feat(flow): AST-based type inference + side-effect detection for enhanced CONTRACTS and DATA_TYPES
+
+### Added
+
+- **TypeInferenceEngine** (`analysis/type_inference.py`)
+  - Parses `->` return annotations from source AST
+  - Extracts argument type hints (`arg: Type`)
+  - Fallback: infers types from function name patterns (`parse_*` → str→dict)
+  - Batch mode: `extract_all_types()` for all project functions
+
+- **SideEffectDetector** (`analysis/side_effects.py`)
+  - AST scan: detects `open()`, `write()`, `self.x = ...`, `global`, `del`
+  - Classification: IO / Cache / Mutation / Pure
+  - `SideEffectInfo` with detailed breakdown and summary
+  - Heuristic fallback when source files unavailable
+
+- **26 new tests** (`tests/test_sprint2_flow.py`)
+  - TypeInferenceEngine: annotation extraction, defaults, signatures, batch
+  - SideEffectDetector: IO, pure, mutation, summary, batch, heuristic
+  - FlowExporter integration: contracts IN/OUT, data types, edge cases
+
+### Changed
+
+- **Enhanced CONTRACTS section** in `flow.toon`
+  - Per-stage: IN types, OUT type, SIDE-EFFECT summary
+  - INVARIANT inference (normalize → `len(output) <= len(input)`)
+  - SMELL markers for CC ≥ 15
+
+- **Enhanced DATA_TYPES section** in `flow.toon`
+  - Source counts: `[N annotated, M inferred / T functions]`
+  - Hub-type split recommendations with named sub-interfaces
+  - e.g. `AnalysisResult → split into: StructureResult, MetricsResult, FlowResult`
+
+- **FlowExporter** now uses `TypeInferenceEngine` and `SideEffectDetector`
+  - Typed signatures from AST (not just arg names)
+  - Purity scoring from AST body scan (not just name heuristics)
+
+- **Version bump** to 0.3.1
+
+---
+
 ## [0.3.0] - 2026-03-01
 
 ### Summary

README.md

@@ -1,60 +1,32 @@
 # code2flow
 
-**Python Code Flow Analysis Tool** — Static analysis for control flow graphs (CFG), data flow graphs (DFG), and call graph extraction with 4 purpose-built output formats.
+**Python Code Flow Analysis Tool** - Static analysis for control flow graphs (CFG), data flow graphs (DFG), and call graph extraction with optimized TOON format.
 
 ![img.png](img.png)
 
-## 🚀 New in v0.3.0: Format Taxonomy
+## 🚀 New: TOON Format v2
 
-**4 files, 4 purposes** — each format answers a different question:
+**TOON v2** is the default output format - scannable, severity-sorted, prompt-ready:
 
-| Format | File | Purpose | Answers |
-|--------|------|---------|---------|
-| **Map** | `project.map` | Structure | "What exists and how it's connected?" |
-| **Toon** | `analysis.toon` | Health diagnostics | "What's broken and how to fix it?" |
-| **Flow** | `flow.toon` | Data flow | "How do data flow through the system?" |
-| **Context** | `context.md` | LLM narrative | "Understand the system to rebuild it" |
+- **🎯 Health-first design** - issues sorted by severity (🔴/🟡)
+- **📊 Coupling matrix** - fan-in/fan-out analysis
+- **🔍 Duplicate detection** - find identical classes
+- **📈 Layered architecture** - package-level metrics
+- **⚡ Inline markers** - `!!` (CC≥15), `!` (CC≥10), `×DUP`
+- **🚫 Smart filtering** - excludes venv, site-packages
+- **📋 Actionable REFACTOR** - concrete steps, not just problems
 
 ```bash
-# Default: health diagnostics only
+# Default: TOON format only
 code2flow /path/to/project
 
-# Generate all 4 core formats
-code2flow /path/to/project -f toon,map,flow,context
-
-# Generate everything (all 8 formats)
+# Generate all formats
 code2flow /path/to/project -f all
 
-# Just the structural map
-code2flow /path/to/project -f map
-```
-
-### When to Use Which Format
-
-```
-"What's in the project?"              → project.map
-"What's broken?"                      → analysis.toon (HEALTH)
-"How to fix it?"                      → analysis.toon (REFACTOR)
-"How do data flow through the system?"→ flow.toon (PIPELINES)
-"Where to split a type?"              → flow.toon (DATA_TYPES)
-"Is the pipeline pure?"               → flow.toon (CONTRACTS)
-"How to rebuild in another language?" → context.md
-"What depends on this module?"        → analysis.toon (COUPLING)
-"What type does a function return?"   → project.map (signatures)
+# TOON + YAML (for comparison)
+code2flow /path/to/project -f toon,yaml
 ```
 
-## 🎯 TOON v2 — Health Diagnostics
-
-**TOON v2** (`analysis.toon`) is the default output — scannable, severity-sorted, prompt-ready:
-
-- **🎯 Health-first design** — issues sorted by severity (🔴/🟡)
-- **📊 Coupling matrix** — fan-in/fan-out analysis
-- **🔍 Duplicate detection** — find identical classes
-- **📈 Layered architecture** — package-level metrics
-- **⚡ Inline markers** — `!!` (CC≥15), `!` (CC≥10), `×DUP`
-- **🚫 Smart filtering** — excludes venv, site-packages
-- **📋 Actionable REFACTOR** — concrete steps, not just problems
-
 ## Performance Optimization
 
 For large projects (>1000 functions), use **Fast Mode**:
@@ -171,26 +143,17 @@ code2flow /path/to/project -o my_analysis
 
 ## Output Files
 
-### Core Formats (4 purpose-built files)
-
-| File | Format | Purpose | Size |
-|------|--------|---------|------|
-| `analysis.toon` | **Toon** | Health diagnostics (HEALTH, REFACTOR, COUPLING) | ~25KB |
-| `project.map` | **Map** | Structural map (modules, imports, signatures) | ~23KB |
-| `flow.toon` | **Flow** | Data-flow analysis (PIPELINES, CONTRACTS, DATA_TYPES) | ~10KB |
-| `context.md` | **Context** | LLM narrative (architecture, patterns, API) | ~30KB |
-
-### Additional Formats
-
 | File | Description | Size |
 |------|-------------|------|
+| `analysis.toon` | **🎯 Optimized TOON format** (default) | ~200KB |
 | `analysis.yaml` | Complete structured analysis data | ~2.5MB |
 | `analysis.json` | JSON format for programmatic use | ~2.6MB |
 | `flow.mmd` | Full Mermaid flowchart (all nodes) | ~9KB |
-| `compact_flow.mmd` | Compact flowchart — deduplicated nodes | ~9KB |
+| `compact_flow.mmd` | Compact flowchart - deduplicated nodes | ~9KB |
 | `calls.mmd` | Function call graph | ~9KB |
 | `cfg.png` | Control flow visualization | ~7MB |
 | `call_graph.png` | Call graph visualization | ~3.7MB |
+| `llm_prompt.md` | LLM-ready analysis summary | ~35KB |
 
 ## 🎯 TOON v2 Format Structure
 
@@ -210,10 +173,10 @@ REFACTOR[4]:
 
 COUPLING:
   ┌─────────────┬──────────────────────────────────────┐
-  │ Package     │ fan-in  fan-out  status             │
+  │ Package     │ fan-in  fan-out  status              │
   ├─────────────┼──────────────────────────────────────┤
   │ core        │ 12      45        !! split needed    │
-  │ exporters   │ 5       28        hub               │
+  │ exporters   │ 5       28        hub                │
   └─────────────┴──────────────────────────────────────┘
 
 LAYERS:
@@ -360,12 +323,9 @@ The analyzer is designed to be extensible. Key areas for enhancement:
 
 | Command | Output | Use Case |
 |---------|--------|----------|
-| `code2flow ./project` | `analysis.toon` | Quick health diagnostics (default) |
-| `code2flow ./project -f all` | All 8 formats | Complete analysis |
-| `code2flow ./project -f toon,map,flow,context` | 4 core formats | Full taxonomy |
-| `code2flow ./project -f map` | `project.map` | Structural map |
-| `code2flow ./project -f flow` | `flow.toon` | Data-flow analysis |
-| `code2flow ./project -f context` | `context.md` | LLM narrative |
+| `code2flow ./project` | `analysis.toon` | Quick analysis (default) |
+| `code2flow ./project -f all` | All formats | Complete analysis |
+| `code2flow ./project -f toon,yaml` | TOON + YAML | Comparison |
 | `code2flow ./project -m hybrid -v` | TOON + verbose | Detailed analysis |
 | `python validate_toon.py analysis.toon` | Validation | Quality check |
 
@@ -475,19 +435,19 @@ code2flow ./project -f toon,yaml
 # Both formats available for comparison
 ```
 
-## 📋 TOON v2 Format Specification
+## 📋 TOON Format Specification
 
 ### File Structure
 ```
 analysis.toon
-├── Header lines      # Project summary + key metrics
-├── HEALTH            # Highest-severity issues (🔴/🟡)
-├── REFACTOR          # Actionable refactoring steps
-├── COUPLING          # Package-level fan-in/fan-out summary
-├── LAYERS            # Package hierarchy + inline markers
-├── FUNCTIONS         # CC-filtered function list (focus on CC≥10)
-├── HOTSPOTS          # Top fan-out functions
-└── CLASSES           # Class-level complexity summary
+├── meta              # Metadata (project, mode, timestamp)
+├── stats             # Analysis statistics
+├── functions         # Function analysis with complexity
+├── classes           # Class information from function grouping
+├── modules           # Module-level statistics
+├── patterns          # Detected design patterns
+├── call_graph        # Top 50 most important functions
+└── insights           # Recommendations and summaries
 ```
 
 ### Complexity Scoring

ROADMAP.md

@@ -2,7 +2,7 @@
 
 This document outlines planned features, improvements, and milestones for the code2flow project.
 
-## Current Status (v0.3.0)
+## Current Status (v0.3.1)
 
 ✅ **Completed:**
 - Core analysis engine with caching and parallel processing
@@ -15,9 +15,14 @@ This document outlines planned features, improvements, and milestones for the co
 - **Format Taxonomy (v0.3.0)** — 4 purpose-built output formats:
   - `project.map` — structural map (modules, imports, signatures, types)
   - `analysis.toon` — health diagnostics (HEALTH, REFACTOR, COUPLING, LAYERS)
-  - `flow.toon` — **NEW** data-flow analysis (PIPELINES, TRANSFORMS, CONTRACTS, DATA_TYPES)
+  - `flow.toon` — data-flow analysis (PIPELINES, TRANSFORMS, CONTRACTS, DATA_TYPES)
   - `context.md` — LLM narrative (architecture, patterns, API surface)
   - CLI: `--format map,toon,flow,context,all`
+- **AST-based type inference + side-effect detection (v0.3.1)**:
+  - `TypeInferenceEngine` — parses return annotations, argument types, name-based fallback
+  - `SideEffectDetector` — AST scan for IO, cache, mutation, pure classification
+  - Enhanced CONTRACTS: IN/OUT types, SIDE-EFFECT, INVARIANT, SMELL markers
+  - Enhanced DATA_TYPES: source counts, hub-type split recommendations
 
 ---
 
@@ -305,7 +310,7 @@ pattern:
 |---------|-------------|-------|--------|
 | v0.2.5 | Mar 2026 | TOON v2 format implementation | ✅ Done |
 | v0.3.0 | Mar 2026 | Format taxonomy (map, toon, flow, context) | ✅ Done |
-| v0.3.1 | Q2 2026 | CONTRACTS + DATA_TYPES enhancement | 📋 Planned |
+| v0.3.1 | Mar 2026 | CONTRACTS + DATA_TYPES enhancement (AST type inference, side-effect detection) | ✅ Done |
 | v0.4.0 | Q3 2026 | IDE integration, real-time analysis | 📋 Planned |
 | v0.5.0 | Q4 2026 | JS/TS support | 📋 Planned |
 | v0.6.0 | Q1 2027 | Enterprise features | 📋 Planned |

TODO.md

@@ -9,24 +9,33 @@
   - [x] Update CLI: `--format map,toon,flow,context,all`
   - [x] 4 files, 4 purposes: map (structure), toon (health), flow (data-flow), context (LLM)
 
-## 🎯 Sprint 2 — CONTRACTS + DATA_TYPES (v0.3.1)
-
-### High Priority
-
-- [ ] **Type inference from AST**
-  - Parse `->` return annotations
-  - Parse arguments with type hints
-  - Fallback: infer from names (`parse_*` → str input, `to_dict` → dict output)
-
-- [ ] **CONTRACTS section enhancement**
-  - Per-pipeline: input→output for each stage
-  - Side-effect detection: `self.`, `.write`, `.save`, `cache`
-  - Purity scoring: pure / IO / cache / mutation
-
-- [ ] **DATA_TYPES section enhancement**
-  - Count consumed/produced per type
-  - Auto-detect hub-types (consumed ≥ 10)
-  - Recommend split for hub-types
+## ✅ Completed — Sprint 2 (v0.3.1)
+
+- [x] **Type inference from AST** (`analysis/type_inference.py`)
+  - [x] Parse `->` return annotations
+  - [x] Parse arguments with type hints
+  - [x] Fallback: infer from names (`parse_*` → str input, `to_dict` → dict output)
+  - [x] Batch mode: `extract_all_types()` for all project functions
+
+- [x] **CONTRACTS section enhancement**
+  - [x] Per-pipeline: IN types, OUT type for each stage
+  - [x] Side-effect detection via AST: `self.`, `.write`, `open()`, `global`, `cache`
+  - [x] Purity scoring: pure / IO / cache / mutation
+  - [x] INVARIANT inference (normalize → `len(output) <= len(input)`)
+  - [x] SMELL markers for CC ≥ 15
+
+- [x] **DATA_TYPES section enhancement**
+  - [x] Count consumed/produced per type (AST-based)
+  - [x] Auto-detect hub-types (consumed ≥ 10)
+  - [x] Hub-type split recommendations with named sub-interfaces
+  - [x] Source counts: `[N annotated, M inferred / T functions]`
+
+- [x] **SideEffectDetector** (`analysis/side_effects.py`)
+  - [x] AST scan: `open()`, `write()`, `self.x = ...`, `global`, `del`
+  - [x] Classification: IO / Cache / Mutation / Pure
+  - [x] Heuristic fallback when source unavailable
+
+- [x] **26 new tests** (`tests/test_sprint2_flow.py`)
 
 ## 🎯 Sprint 3 — PIPELINES auto-detection + SIDE_EFFECTS (v0.3.2)