diff --git a/docs/plans/moderate-context-optimization-plan.md b/docs/plans/moderate-context-optimization-plan.md new file mode 100644 index 00000000..822c8235 --- /dev/null +++ b/docs/plans/moderate-context-optimization-plan.md @@ -0,0 +1,416 @@ +# Implementation Plan: Moderate Context Optimization + +> Generated by `/PACT:plan-mode` on 2026-01-15 +> Status: PENDING APPROVAL + + + +## Summary + +Break up the 1,085-line `pact-protocols.md` into targeted files so agents and commands import only the context they need. This moderate approach extracts **2 new files** while keeping `pact-protocols.md` as the orchestrator reference—achieving ~85% context reduction for agents with minimal maintenance overhead. + +--- + +## Problem Statement + +**Current state:** Every PACT agent and command that references `pact-protocols.md` loads all 1,085 lines regardless of what they actually need: + +| Consumer | What They Need | What They Load | Waste | +|----------|----------------|----------------|-------| +| 7 Agents | S1 Autonomy (~75 lines) | 1,085 lines | ~93% | +| plan-mode.md | Variety (~60 lines) | 1,085 lines | ~94% | +| orchestrate.md | Multiple sections | 1,085 lines | Varies | + +**Goal:** Enable targeted imports so agents load ~100 lines instead of 1,085. + +--- + +## Specialist Perspectives + +### 📋 Preparation Phase +**Effort**: Low + +#### Research Completed +- [x] File sizes measured (pact-protocols.md = 1,085 lines) +- [x] Reference patterns analyzed (**24 @-references across 11 files** in `.claude/`; 49 total including `pact-plugin/` mirrors) +- [x] Agent-specific sections identified (S1 Autonomy, Self-Coordination, Algedonic pointer) +- [x] Existing patterns confirmed (algedonic.md already extracted—proves pattern works) + +#### Dependencies to Map + +| File | Reference Count | Notes | +|------|-----------------|-------| +| 7 agent files | 14 total (2 each) | Cross-Agent Coordination + Nested PACT | +| `pact-preparer.md` | 3 refs (exception) | Has additional S4 Environment Model ref → **stays pointing to main file** | +| `orchestrate.md` | 5 refs | Only Variety Management ref changes; S4/S2/S3* refs stay | +| `plan-mode.md` | 1 ref | Variety Management | +| `algedonic.md` | 2 refs | S5 Decision Framing refs → stay pointing to main file | +| `vsm-glossary.md` | 1 ref | General reference → stays pointing to main file | +| `pact-plugin/README.md` | 1 ref | General reference → stays pointing to main file | +| `pact-plugin/` | mirrors | Must mirror all changes to protocols/ and agents/ | + +**Total edits required**: ~32 files (14 agent edits + 2 command edits + 2 protocol files + pact-plugin mirrors) + +--- + +### 🏗️ Architecture Phase +**Effort**: Low + +#### Proposed File Structure + +``` +~/.claude/protocols/ +├── pact-protocols.md # Refactored: Orchestrator reference (~950 lines) +├── pact-agent-protocols.md # NEW: For all agents (~100 lines) +├── pact-variety.md # NEW: For orchestrate/plan-mode (~80 lines) +└── algedonic.md # Unchanged (already extracted) +``` + +#### Content Distribution + +| File | Content | Lines | Consumers | +|------|---------|-------|-----------| +| **pact-agent-protocols.md** (NEW) | S1 Autonomy & Recursion (full), Self-Coordination (from S2), Phase Handoffs, Algedonic pointer (summary + link to `algedonic.md`, NOT duplicated content) | ~100 | All 7 agents | +| **pact-variety.md** (NEW) | Variety Management (full), PACT Workflow Family table | ~80 | orchestrate.md, plan-mode.md | +| **pact-protocols.md** (refactored) | S5 Policy, S4 Checkpoints, S3/S4 Tension, S2 Coordination (minus Self-Coordination + pointer to agent file), S3* Audit, Workflow protocols, Decision logs, Documentation locations | ~950 | Orchestrator only | + +**Implementation note**: When extracting Self-Coordination from S2, add a one-line pointer in the main file's S2 section: "For agent-specific self-coordination protocols, see `pact-agent-protocols.md`." + +#### Design Approach + +**Consumer-aligned partitioning**: Group content by who needs it rather than by VSM system. This is the simplest mental model: +- "Agent file" = What agents need to know +- "Variety file" = What commands need for workflow selection +- "Main file" = Orchestrator's complete reference + +#### Key Decisions + +| Decision | Options | Recommendation | Rationale | +|----------|---------|----------------|-----------| +| How many files? | 2 new / 4 new / 8 new | **2 new** | Truly moderate; highest ROI with lowest overhead | +| Naming convention | VSM-aligned / Consumer-aligned | **Consumer-aligned** | More intuitive (`pact-agent-protocols` vs `s1-autonomy`) | +| What stays in main file? | Everything else / Slim index only | **Everything else** | Orchestrator still benefits from having full reference | + +#### Interface Contracts + +**Each new file includes back-reference header:** +```markdown +# {Title} + +> **Purpose**: {One line} +> **Referenced by**: {Consumer list} +> **See also**: @~/.claude/protocols/pact-protocols.md for full orchestration protocols +``` + +**Main file includes forward reference table** (add to `pact-protocols.md` header): +```markdown +## Extracted Protocols (For Specific Consumers) + +Some protocols are maintained in separate files for context efficiency: + +| Protocol | File | Primary Consumer | +|----------|------|------------------| +| Agent Autonomy & Coordination | `pact-agent-protocols.md` | All specialist agents | +| Variety Management | `pact-variety.md` | orchestrate.md, plan-mode.md | +| Emergency Signals | `algedonic.md` | All agents + orchestrator | +``` + +This bidirectional referencing ensures discoverability from either direction. + +--- + +### 💻 Code Phase +**Effort**: Low-Medium + +#### Files to Create + +| File | Purpose | +|------|---------| +| `.claude/protocols/pact-agent-protocols.md` | Agent-focused protocols: autonomy, coordination, handoffs | +| `.claude/protocols/pact-variety.md` | Variety scoring and workflow selection | + +#### Files to Modify + +| File | Changes | +|------|---------| +| `.claude/protocols/pact-protocols.md` | Add index header pointing to extracted files; minor cleanup | +| `.claude/agents/pact-architect.md` | Update refs: `pact-protocols.md` → `pact-agent-protocols.md` | +| `.claude/agents/pact-backend-coder.md` | Update refs: `pact-protocols.md` → `pact-agent-protocols.md` | +| `.claude/agents/pact-frontend-coder.md` | Update refs: `pact-protocols.md` → `pact-agent-protocols.md` | +| `.claude/agents/pact-database-engineer.md` | Update refs: `pact-protocols.md` → `pact-agent-protocols.md` | +| `.claude/agents/pact-test-engineer.md` | Update refs: `pact-protocols.md` → `pact-agent-protocols.md` | +| `.claude/agents/pact-preparer.md` | Update 2 of 3 refs → `pact-agent-protocols.md`; S4 Environment Model ref stays pointing to main file | +| `.claude/agents/pact-n8n.md` | Update refs: `pact-protocols.md` → `pact-agent-protocols.md` | +| `.claude/commands/PACT/orchestrate.md` | Update variety refs → `pact-variety.md` | +| `.claude/commands/PACT/plan-mode.md` | Update variety refs → `pact-variety.md` | +| `pact-plugin/protocols/` | Mirror new protocol files | +| `pact-plugin/agents/` | Mirror agent reference updates | + +#### Implementation Sequence + +1. Create `pact-agent-protocols.md` with extracted content +2. Create `pact-variety.md` with extracted content +3. Add index header to `pact-protocols.md` pointing to new files +4. Update all 7 agent files (parallel-safe: different files) +5. Update orchestrate.md and plan-mode.md +6. Sync to pact-plugin/ directory + +--- + +### 🧪 Test Phase +**Effort**: Low + +#### Test Scenarios + +| Scenario | Type | Priority | +|----------|------|----------| +| All @-references resolve to existing files | Validation | P0 | +| All section-level references find target sections | Validation | P0 | +| No content lost during extraction | Validation | P0 | +| Agent can read `pact-agent-protocols.md` | Smoke | P0 | +| orchestrate.md can read `pact-variety.md` | Smoke | P0 | +| pact-preparer S4 Environment Model ref still works | Smoke | P0 | +| orchestrate.md S4/S2/S3* refs still work (unchanged) | Smoke | P0 | +| No duplicate content across files | Validation | P1 | +| pact-plugin/ mirrors match .claude/ | Validation | P1 | +| Rollback restores working state | Validation | P1 | + +#### Coverage Targets +- Reference resolution: 100% +- Content preservation: 100% (verified by diff) + +#### Validation Approach + +**Hybrid**: Automated grep for reference inventory + manual spot-check of key consumers + +```bash +#!/bin/bash +set -euo pipefail +shopt -s globstar # Required for **/*.md pattern + +# Pre-change baseline +grep -r "@~/.claude/protocols" .claude/ pact-plugin/ > baseline-refs.txt + +# Post-change validation with proper exit codes +broken=0 + +# Check file-level references (existence + non-empty) +for ref in $(grep -roh "@~/.claude/protocols/[^)\" ]*" .claude/ pact-plugin/); do + filepath="${ref#@}" + filepath="${filepath/#\~/$HOME}" + if [[ ! -f "$filepath" ]]; then + echo "BROKEN FILE: $ref" + broken=1 + elif [[ ! -s "$filepath" ]]; then + echo "EMPTY FILE: $ref" + broken=1 + fi +done + +# Check section-level references (verify section headers exist in target files) +# NOTE: This regex matches common patterns like "for S1 Autonomy rules" but may miss +# other phrasings. Manual spot-check recommended for edge cases. +for file in .claude/**/*.md pact-plugin/**/*.md; do + [[ -f "$file" ]] || continue + while IFS= read -r line; do + section=$(echo "$line" | grep -oP 'for (the )?\K[A-Z][^.]*(?= (rules|protocol|section|assessment))' || true) + target=$(echo "$line" | grep -oP '@~/.claude/protocols/[^)\" ]*' | head -1 || true) + if [[ -n "$section" && -n "$target" ]]; then + target_path="${target#@}" + target_path="${target_path/#\~/$HOME}" + if [[ -f "$target_path" ]] && ! grep -q "## .*$section" "$target_path" 2>/dev/null; then + echo "MISSING SECTION: '$section' not found in $target_path (from $file)" + broken=1 + fi + fi + done < <(grep "@~/.claude/protocols" "$file" 2>/dev/null || true) +done + +exit $broken +``` + +**Key validation improvements**: +- Includes both `.claude/` and `pact-plugin/` directories +- Uses `globstar` for recursive file matching +- Returns proper exit code for CI integration +- Validates section-level references (not just file existence) +- Checks for empty files (not just existence) + +**Content preservation verification** (run before/after extraction): +```bash +# Before extraction: capture section inventory +grep -E "^## " .claude/protocols/pact-protocols.md > sections-before.txt + +# After extraction: verify all sections exist across files +cat .claude/protocols/pact-protocols.md \ + .claude/protocols/pact-agent-protocols.md \ + .claude/protocols/pact-variety.md | grep -E "^## " > sections-after.txt + +# Compare (should show only expected moves, no losses) +diff sections-before.txt sections-after.txt +``` + +**Rollback procedure** (if validation fails at any phase): +```bash +# Revert all protocol and agent changes +git checkout HEAD -- .claude/protocols/ .claude/agents/ pact-plugin/protocols/ pact-plugin/agents/ + +# Or revert entire working tree to last commit +git checkout HEAD -- . +``` + +--- + +## Synthesized Implementation Roadmap + +### Phase Sequence + +``` +Phase 1: Extract Agent Protocols (Low Risk) +├── Create pact-agent-protocols.md +└── Content: S1 Autonomy + Self-Coordination + Phase Handoffs + Algedonic quick-ref + │ + ▼ CHECKPOINT: New file exists and is well-formed + │ +Phase 2: Extract Variety Management (Low Risk) +├── Create pact-variety.md +└── Content: Variety Management + Workflow Family table + │ + ▼ CHECKPOINT: New file exists and is well-formed + │ +Phase 3: Update Index (Low Risk) +├── Add header to pact-protocols.md +└── List extracted files with purposes + │ + ▼ CHECKPOINT: Main file still coherent + │ +Phase 4: Update Agent References (Medium Risk) +├── Update all 7 agent files +└── Change @pact-protocols.md → @pact-agent-protocols.md (targeted refs only) + │ + ▼ CHECKPOINT: All agent refs resolve + │ +Phase 5: Update Command References (Low Risk) +├── Update orchestrate.md variety refs +└── Update plan-mode.md variety ref + │ + ▼ CHECKPOINT: All command refs resolve + │ +Phase 6: Plugin Sync (Low Risk) +├── Copy new protocols to pact-plugin/protocols/ +└── Copy updated agents to pact-plugin/agents/ + │ + ▼ FINAL VALIDATION +``` + +### Commit Sequence (Proposed) + +> **Note**: This sequence represents the intended final git history order, **not** the execution order. Independent commits may be implemented in parallel. + +1. `feat(protocols): extract agent-focused protocols to pact-agent-protocols.md` — S1 Autonomy, Self-Coordination, Phase Handoffs +2. `feat(protocols): extract variety management to pact-variety.md` — Variety scoring, workflow family +3. `refactor(protocols): add index header to pact-protocols.md` — Point to extracted files +4. `refactor(agents): update 7 agent files with targeted protocol refs` — All agents use new file +5. `refactor(commands): update orchestrate and plan-mode with targeted refs` — Commands use variety file +6. `chore(plugin): sync protocol and agent changes to pact-plugin/` — Mirror for distribution + +--- + +## Cross-Cutting Concerns + +| Concern | Status | Notes | +|---------|--------|-------| +| Security | N/A | No security implications | +| Performance | Ready | This is a context reduction optimization | +| Accessibility | N/A | Documentation only | +| Observability | N/A | No runtime implications | + +--- + +## Open Questions + +### Require User Decision + +None — straightforward moderate optimization with clear scope. + +### Require Further Research + +- [ ] **Token measurement**: How to verify actual token savings post-implementation? (Can defer to TEST phase) + +--- + +## Risk Assessment + +| Risk | Likelihood | Impact | Mitigation | +|------|------------|--------|------------| +| Broken @-references after update | Low | High | Pre-change baseline + automated validation script | +| Content lost during extraction | Low | Medium | Section-by-section diff before/after | +| Increased maintenance overhead | Low | Low | Only 2 new files; clear purposes | +| Agent needs content not in extracted file | Low | Medium | Include all agent-relevant sections; main file remains available | + +--- + +## Expected Outcomes + +| Metric | Before | After | Improvement | +|--------|--------|-------|-------------| +| Agent context load | 1,085 lines | ~100 lines | **~90% reduction** | +| plan-mode context load | 1,085 lines | ~80 lines | **~93% reduction** | +| orchestrate.md (variety refs) | 1,085 lines | ~80 lines | **~93% reduction** | +| Files to maintain | 1 | 3 | +2 files (moderate overhead) | +| Total protocol lines | 1,085 | 1,085 | No loss (redistributed) | + +--- + +## Scope Assessment + +- **Overall Complexity**: Low-Medium +- **New Files**: 2 protocol files + 2 plugin mirrors = 4 files +- **Modified Files**: 7 agents (14 ref edits) + 2 commands + 1 pact-protocols.md + 7 plugin agent mirrors + 1 plugin pact-protocols = ~18 files +- **Total File Touches**: ~22 files (some with multiple edits) +- **Total Edits**: ~32 (14 agent refs + 2 command refs + headers/content extraction + plugin mirrors) +- **Specialists Required**: pact-preparer (minimal), orchestrator for file edits +- **External Dependencies**: None +- **Estimated Time**: 2-3 hours implementation + 1 hour validation + +--- + +## Comparison to Previous Plan + +This plan supersedes the more comprehensive `context-architecture-protocols-plan.md` (PR #71): + +| Aspect | Previous Plan | This Plan | +|--------|---------------|-----------| +| New files | 8 | 2 | +| Scope | Full VSM-aligned partitioning | Consumer-aligned extraction | +| Complexity | High | Low-Medium | +| Maintenance overhead | High | Low | +| Agent savings | ~90% | ~90% | +| Time estimate | 6-7 hours | 2-3 hours | + +**Trade-off**: Similar agent context savings with ~60% less implementation effort and ongoing maintenance. + +--- + +## Next Steps + +To implement this plan after approval: +``` +/PACT:orchestrate implement moderate context optimization per approved plan +``` + +The orchestrator should reference this plan during execution. + +--- + +## Revision History + +| Date | Change | +|------|--------| +| 2026-01-15 | Initial plan created via `/PACT:plan-mode` | +| 2026-01-15 | Round 1 peer review: fixed reference count, added missing dependencies, documented special cases, enhanced validation script | +| 2026-01-15 | Round 2 peer review: corrected counts (24 refs, orchestrate=5, algedonic=2), added pact-plugin/README.md, added empty file check, content preservation verification, rollback procedure, clarified Algedonic pointer vs duplication, added S2 extraction note | diff --git a/docs/review/pr-72-moderate-context-optimization-review.md b/docs/review/pr-72-moderate-context-optimization-review.md new file mode 100644 index 00000000..cfceb6bf --- /dev/null +++ b/docs/review/pr-72-moderate-context-optimization-review.md @@ -0,0 +1,198 @@ +# PR #72 Peer Review Summary: Moderate Context Optimization Plan + +> **Initial Review**: 2026-01-15 +> **Second Review**: 2026-01-15 (post-feedback incorporation) +> **Final Review**: 2026-01-15 (verification complete) +> **Reviewers**: pact-architect, pact-test-engineer, pact-preparer +> **Overall Verdict**: **APPROVE** ✅ + +--- + +## Review History + +| Round | Date | Focus | Outcome | +|-------|------|-------|---------| +| 1 | 2026-01-15 | Initial plan review | APPROVE with suggestions (all addressed) | +| 2 | 2026-01-15 | Post-fix verification | APPROVE with minor suggestions | +| 3 | 2026-01-15 | Final verification | **APPROVE** ✅ (all items verified) | + +--- + +## Final Review: All Recommendations Verified + +All three reviewers have verified that Round 2 recommendations were fully addressed. + +| Reviewer | Verdict | Verification | +|----------|---------|--------------| +| **pact-architect** | **APPROVE** ✅ | S2 pointer note addressed, Algedonic clarification addressed | +| **pact-test-engineer** | **APPROVE** ✅ | Empty file check, regex note, diff command, rollback procedure all addressed | +| **pact-preparer** | **APPROVE** ✅ | All counts corrected (24 refs, orchestrate=5, algedonic=2), README.md added | + +--- + +## Round 2 Review: Consensus + +All three reviewers approved the updated plan. The previous high-priority items were addressed. + +| Reviewer | Verdict | Assessment | +|----------|---------|------------| +| **pact-architect** | **APPROVE** | Design is sound, bidirectional navigation implemented | +| **pact-test-engineer** | APPROVE with suggestions | Validation improved, minor regex concern | +| **pact-preparer** | APPROVE with suggestions | Minor count discrepancies found | + +--- + +## What Was Fixed (Round 1 → Round 2) + +All high-priority items from Round 1 have been addressed: + +| Item | Status | +|------|--------| +| Reference count accuracy | ✅ Updated to "21 refs across 11 files" | +| Missing dependencies | ✅ Added algedonic.md, vsm-glossary.md | +| pact-preparer special case | ✅ Documented (3 refs, S4 stays in main) | +| orchestrate.md update scope | ✅ Documented (1 of 6 refs changes) | +| Validation script gaps | ✅ Added globstar, exit codes, section-level validation | +| Forward reference table | ✅ Template added | +| Back-references | ✅ Template added | +| Total edit count | ✅ Documented (~32 edits across ~22 files) | + +--- + +## Second Review: Detailed Findings + +### pact-architect + +**Verdict: APPROVE** + +**Strengths**: +1. Consumer-aligned partitioning follows Interface Segregation Principle correctly +2. Minimal surface area (2 new files) balances optimization with maintainability +3. Bidirectional navigation contract (forward + back references) ensures discoverability +4. Implementation sequence is low-risk (create before reference) +5. Edge cases documented (pact-preparer, orchestrate.md partial updates) + +**Minor Recommendations**: +1. When extracting Self-Coordination from S2, add a one-line pointer in main file +2. Clarify that "Algedonic quick-ref" is a summary/pointer, not duplicated content + +**No blocking concerns.** + +--- + +### pact-test-engineer + +**Verdict: APPROVE WITH SUGGESTIONS** + +**Strengths**: +1. Validation script significantly improved with globstar, exit codes, section-level checks +2. Test scenarios correctly prioritized (P0 = reference resolution, content loss) +3. Hybrid validation approach (automated + manual) is appropriate +4. Phase checkpoints enable early failure detection + +**Remaining Concerns**: + +| Concern | Severity | Recommendation | +|---------|----------|----------------| +| Section-level regex is fragile | Medium | Pattern assumes "rules/protocol/section/assessment" suffix; may miss other phrasings | +| Empty file check missing | Low | Add `-s` check: `if [[ ! -s "$filepath" ]]` | +| Content preservation verification vague | Low | Add explicit diff command for section inventory | +| Rollback procedure minimal | Low | Document explicit `git checkout HEAD -- .claude/ pact-plugin/` | + +**Suggested validation script enhancement**: +```bash +# Add empty file check +if [[ ! -s "$filepath" ]]; then + echo "EMPTY FILE: $ref" + broken=1 +fi +``` + +--- + +### pact-preparer + +**Verdict: APPROVE WITH SUGGESTIONS** + +**Verification Results**: +- Actual reference count in `.claude/`: **24 refs** (plan says 21) +- Discrepancy sources: + - `orchestrate.md`: 5 refs (plan says 6) — plan is conservative + - `algedonic.md`: 2 refs (plan says 1) — missed one reference + +**Line Count Verification**: +| Content | Actual Lines | Plan Estimate | Assessment | +|---------|--------------|---------------|------------| +| S1 Autonomy & Recursion | ~75 | ~75 | ✅ Accurate | +| Phase Handoffs | ~11 | included in ~100 | ✅ Reasonable | +| Variety Management | ~61 | ~80 | ✅ Reasonable with headers | + +**Minor Discrepancies**: + +| Item | Plan States | Actual | Impact | +|------|-------------|--------|--------| +| Total refs in .claude/ | 21 | 24 | Low — doesn't affect approach | +| orchestrate.md refs | 6 | 5 | Low — plan is conservative | +| algedonic.md refs | 1 | 2 | Low — file stays in main anyway | + +**Missing from dependency table**: `pact-plugin/README.md` (1 ref) — this is a general reference that stays pointing to main file. + +--- + +## Remaining Suggestions (Non-Blocking) + +### For Implementation Phase + +| Suggestion | Source | Priority | +|------------|--------|----------| +| Fix reference count: 21 → 24 | Preparer | Low | +| Fix orchestrate.md count: 6 → 5 | Preparer | Low | +| Fix algedonic.md count: 1 → 2 | Preparer | Low | +| Add empty file check to validation | Test Engineer | Low | +| Make section regex more permissive | Test Engineer | Low | +| Add pact-plugin/README.md to dependency table | Preparer | Low | + +### For Execution + +1. **Self-Coordination extraction**: Add pointer in main S2 section +2. **Algedonic quick-ref**: Ensure it's a summary/pointer, not duplication +3. **Rollback command**: Use `git checkout HEAD -- .claude/ pact-plugin/` + +--- + +## Conflicting Opinions + +**None identified.** All reviewers agree on: +- Core strategy is correct +- Plan is ready for implementation +- Remaining items are minor and can be addressed during implementation + +--- + +## Conclusion + +All three rounds of peer review complete. All feedback has been addressed and verified. + +**Final Verdict**: **APPROVE FOR IMPLEMENTATION** ✅ + +The plan is ready for execution: +- Consumer-aligned partitioning (2 new files) achieves ~90% agent context reduction +- All reference counts verified accurate +- Validation approach is comprehensive with rollback procedures +- Bidirectional navigation ensures discoverability +- Implementation sequence is low-risk with checkpoints + +**Next Step**: +``` +/PACT:orchestrate implement moderate context optimization per approved plan +``` + +--- + +## Revision History + +| Date | Change | +|------|--------| +| 2026-01-15 | Initial review (Round 1) | +| 2026-01-15 | Updated with Round 2 findings | +| 2026-01-15 | Final verification (Round 3) — all items verified, status: APPROVE ✅ |