diff --git a/CONVERSATIONAL_WORKFLOW_GUIDE.md b/CONVERSATIONAL_WORKFLOW_GUIDE.md new file mode 100644 index 00000000..a7a1f35c --- /dev/null +++ b/CONVERSATIONAL_WORKFLOW_GUIDE.md @@ -0,0 +1,635 @@ +# Conversational Workflow Guide +## Executing Issue #101 with Manual Model Switching + +Since you're working **without API credits**, this guide shows how to execute the plan **conversationally** through Claude Code sessions with appropriate model switching. + +--- + +## Model Switching in Claude Code + +### How to Switch Models + +```bash +# Start a session with specific model +claude --model opus # For complex analysis +claude --model sonnet # For standard work (default) +claude --model haiku # For simple verification +``` + +### Current Session Info + +To check which model you're using: +``` +Ask: "Which model are you?" +Response will indicate: Claude Opus 4.5, Sonnet 4.5, or Haiku +``` + +--- + +## Phase 0: Enumeration Migration + +### Task 0.1: XSD Schema Analysis +**Model**: Claude Opus 4.5 +**Why**: Complex schema structure requires deep understanding +**Duration**: 30-45 minutes + +```bash +# Start Opus session +claude --model opus +``` + +**Conversation:** +``` +You: "Please analyze openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd + +Read the file and extract: +1. All simpleType enumerations (name, base type, enumeration values) +2. All BasicTypes (String256, UInt48, etc.) with their restrictions +3. All complexTypes and their field structures +4. Create a comprehensive report + +Save to: reports/verification/espi_enumerations.md" + +[Claude Opus analyzes and creates report] + +You: "Now do the same for customer.xsd and save to reports/verification/customer_enumerations.md" + +[Claude Opus analyzes customer.xsd] + +You: "Create a summary report at reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md" + +[Claude Opus creates summary] +``` + +**Commit the work:** +```bash +git checkout -b feature/issue-101-phase-0-schema-analysis +git add reports/verification/ +git commit -m "feat(phase-0): Complete ESPI 4.0 schema analysis + +Analyzed XSD schemas with Claude Opus 4.5. + +Co-Authored-By: Claude Opus 4.5 " +git push -u origin feature/issue-101-phase-0-schema-analysis +``` + +**Create PR, get it reviewed and merged.** + +--- + +### Task 0.2-0.20: Generate Missing Enums +**Model**: Claude Sonnet 4.5 +**Why**: Code generation is Sonnet's strength +**Duration**: 2-3 hours (with breaks) + +#### Phase 0.2: Usage Domain Enums (Batch 1) +```bash +# After Phase 0.1 PR is merged +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-0.2-usage-enums-batch1 + +# Start Sonnet session +claude --model sonnet +``` + +**Conversation Pattern:** + +``` +You: "Generate the first batch of usage domain enums from espi.xsd: +- AccumulationKind +- CommodityKind +- DataQualifierKind +- FlowDirectionKind +- KindKind + +Requirements for each: +1. Read the simpleType definition from espi.xsd +2. Create Java enum with all values +3. Include getValue() and fromValue() methods +4. Add comprehensive Javadoc with value descriptions +5. Use Apache License 2025 header +6. Save to: openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/[EnumName].java" + +[Claude Sonnet generates all 5 enums] +``` + +**Commit:** +```bash +git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/ +git commit -m "feat(phase-0.2): Generate usage domain enums batch 1 + +Added: AccumulationKind, CommodityKind, DataQualifierKind, +FlowDirectionKind, KindKind + +Co-Authored-By: Claude Sonnet 4.5 " +git push -u origin feature/issue-101-phase-0.2-usage-enums-batch1 +``` + +**Create PR, get reviewed and merged.** + +--- + +#### Phase 0.3: Usage Domain Enums (Batch 2) +```bash +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-0.3-usage-enums-batch2 +``` + +**Conversation:** +``` +You: "Generate the second batch of usage domain enums from espi.xsd: +- MeasurementKind (100+ values) +- PhaseCodeKind +- QualityOfReading +- TimeAttributeKind +- UnitMultiplierKind + +Same requirements as before." +``` + +**Commit and push:** +```bash +git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/ +git commit -m "feat(phase-0.3): Generate usage domain enums batch 2 + +Added: MeasurementKind, PhaseCodeKind, QualityOfReading, +TimeAttributeKind, UnitMultiplierKind + +Co-Authored-By: Claude Sonnet 4.5 " +git push -u origin feature/issue-101-phase-0.3-usage-enums-batch2 +``` + +--- + +#### Phase 0.4: Usage Domain Enums (Batch 3) +```bash +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-0.4-usage-enums-batch3 +``` + +**Conversation:** +``` +You: "Generate the third batch of usage domain enums from espi.xsd: +- UnitSymbolKind (100+ values) +- ServiceKind +- ItemKind +- MacroperiodKind + +Same requirements as before." +``` + +**Commit and push.** + +--- + +#### Phase 0.5: OAuth and Common Enums +```bash +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-0.5-oauth-common-enums +``` + +**Conversation:** +``` +You: "Generate OAuth and shared enums from espi.xsd: +- GrantType → domain/common/enums/ +- TokenType → domain/common/enums/ +- OAuthError → domain/common/enums/ +- ResponseType → domain/common/enums/ +- TokenEndPointMethod → domain/common/enums/ +- Currency → domain/common/enums/ +- StatusCode → domain/common/enums/ + +Same requirements, but save to common/enums/ directory." +``` + +**Commit and push.** + +--- + +#### Phase 0.6: Customer Domain Enums +```bash +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-0.6-customer-enums +``` + +**Conversation:** +``` +You: "Generate customer domain enums from customer.xsd: +- CustomerKind → domain/customer/enums/ +- SupplierKind → domain/customer/enums/ +- ServiceKind → domain/customer/enums/ +- EnrollmentStatus → domain/customer/enums/ +- NotificationMethodKind → domain/customer/enums/ +- MeterMultiplierKind → domain/customer/enums/ +- RevenueKind → domain/customer/enums/ +- ProgramDateKind → domain/customer/enums/ +- CRUDOperation → domain/customer/enums/ +- MediaType → domain/customer/enums/ + +Same requirements, save to customer/enums/ directory." +``` + +**Commit and push.** + +--- + +### Task 0.21-0.37: Verify Generated Enums + +#### Phase 0.7: Enum Verification +**Model**: Claude Haiku +**Why**: Simple comparison, cost-effective +**Duration**: 30 minutes + +```bash +# After all enum PRs are merged +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-0.7-enum-verification + +# Start Haiku session +claude --model haiku +``` + +**Conversation:** +``` +You: "Verify all enums in domain/usage/enums/ against espi.xsd + +For each enum: +1. Read the Java file +2. Read the XSD simpleType definition +3. Check: all values present, correct types, proper methods +4. Report: PASS or FAIL with issues + +List results in a table." + +[Claude Haiku verifies all enums] + +You: "Same for domain/customer/enums/ against customer.xsd" + +[Claude Haiku verifies customer enums] + +You: "Same for domain/common/enums/ against espi.xsd" + +[Claude Haiku verifies common enums] +``` + +**If all pass:** +```bash +# Create verification report +git add reports/verification/enum_verification_results.md +git commit -m "docs(phase-0.7): Enum verification complete - all pass + +Co-Authored-By: Claude Haiku " +git push -u origin feature/issue-101-phase-0.7-enum-verification +``` + +**If issues found, create fix branch:** +```bash +git checkout -b feature/issue-101-phase-0.7-enum-fixes + +# Switch back to Sonnet to fix +claude --model sonnet +``` + +**After fixes:** +```bash +git add openespi-common/src/main/java/.../enums/ +git commit -m "fix(phase-0.7): Correct enum discrepancies + +Co-Authored-By: Claude Sonnet 4.5 " +git push -u origin feature/issue-101-phase-0.7-enum-fixes +``` + +--- + +## Phase 1: Supporting Classes + +### Common Embeddables (Tasks 1.1-1.6) +**Model**: Claude Sonnet 4.5 +**Why**: Standard verification and fixes + +```bash +git checkout main +git pull origin main +git checkout -b feature/issue-101-phase-1-common-embeddables + +claude --model sonnet +``` + +**Conversation pattern:** +``` +You: "Verify RationalNumber embeddable against espi.xsd + +1. Find the RationalNumber complexType in XSD +2. Read openespi-common/.../common/RationalNumber.java +3. Compare field types, lengths, nullability +4. Report discrepancies +5. If issues found, generate fixes" + +[Claude Sonnet verifies and reports] + +You: "Apply the fixes and update the file" + +[Claude Sonnet fixes the issues] + +You: "Next: Verify DateTimeInterval" + +[Continue for all 6 embeddables...] +``` + +**Commit after 3-5 classes:** +```bash +git add openespi-common/src/main/java/.../common/ +git commit -m "fix(phase-1): Verify RationalNumber, DateTimeInterval, SummaryMeasurement + +Co-Authored-By: Claude Sonnet 4.5 " +``` + +--- + +### Customer Supporting Types (Tasks 1.15-1.36) +**Model**: Claude Sonnet 4.5 + +```bash +git checkout -b feature/issue-101-phase-1-customer-types + +claude --model sonnet +``` + +**Same pattern as above.** + +--- + +## Phase 2: Usage Domain Entities + +### Core Entities (Tasks 2.1-2.6) +**Model**: Claude Sonnet 4.5 (or Opus for complex ones) + +```bash +git checkout -b feature/issue-101-phase-2-usage-core + +# Start with Sonnet +claude --model sonnet +``` + +**Standard Entity Verification:** +``` +You: "Verify UsagePointEntity against espi.xsd + +1. Extract UsagePoint complexType from XSD +2. Read UsagePointEntity.java +3. Create field-by-field comparison: + - Field name + - XSD type → BasicType → restriction + - Java @Column annotations + - Match status +4. Identify discrepancies +5. Generate fixes" + +[Claude Sonnet performs verification] +``` + +**For Complex Entities (e.g., ReadingType):** +```bash +# Switch to Opus for deeper analysis +claude --model opus +``` + +``` +You: "ReadingType has many relationships and complex mappings. +Please analyze ReadingTypeEntity against the XSD: + +1. Deep analysis of all relationships +2. Check inheritance hierarchy +3. Verify all embedded objects +4. Check collection mappings +5. Analyze potential breaking changes +6. Recommend migration strategy" + +[Claude Opus provides deep analysis] + +# Then switch back to Sonnet for fixes +claude --model sonnet +``` + +--- + +## Phase 3: Customer Domain Entities + +### Customer Entities (Tasks 3.1-3.10) +**Model**: Claude Sonnet 4.5 + +```bash +git checkout -b feature/issue-101-phase-3-customer-entities + +claude --model sonnet +``` + +**Same verification pattern as Phase 2.** + +--- + +## Model Selection Decision Tree + +``` +┌─────────────────────────────────────────────┐ +│ What are you doing? │ +└─────────────────────────────────────────────┘ + │ + ├── Analyzing XSD schema structure? + │ └─→ Use OPUS + │ + ├── Making architectural decision? + │ └─→ Use OPUS + │ + ├── Analyzing complex entity (ReadingType, UsagePoint)? + │ └─→ Use OPUS if many relationships + │ Use SONNET if straightforward + │ + ├── Generating code (enums, fixes)? + │ └─→ Use SONNET + │ + ├── Verifying standard entity? + │ └─→ Use SONNET + │ + ├── Batch verifying simple items? + │ └─→ Use HAIKU + │ + └── Writing documentation/reports? + └─→ Use SONNET +``` + +--- + +## Practical Tips + +### 1. Keep Sessions Organized + +Create a tracking file: +```bash +touch PHASE_PROGRESS.md +``` + +Track which model you used for what: +```markdown +# Phase 0 Progress + +## Schema Analysis (Opus) +- [x] espi.xsd analyzed - Session 2024-02-03-opus-1 +- [x] customer.xsd analyzed - Session 2024-02-03-opus-1 +- [x] Summary created + +## Enum Generation (Sonnet) +- [x] Batch 1 (5 enums) - Session 2024-02-03-sonnet-1 +- [x] Batch 2 (5 enums) - Session 2024-02-03-sonnet-2 +- [ ] Batch 3 (5 enums) - In progress + +## Verification (Haiku) +- [ ] Not started +``` + +### 2. Save Context Between Sessions + +At end of each session, ask Claude to create a handoff document: + +``` +You: "Create a handoff document for the next session: +1. What we completed +2. Current state +3. Next steps +4. Any important findings" + +Save to: reports/session_handoff_2024-02-03.md +``` + +### 3. Batch Work by Model + +Instead of switching constantly: + +**Bad:** +``` +Opus → analyze entity 1 +Sonnet → fix entity 1 +Opus → analyze entity 2 +Sonnet → fix entity 2 +``` + +**Good:** +``` +Opus session: Analyze entities 1, 2, 3, 4, 5 → Save findings +Sonnet session: Fix all 5 entities based on findings +``` + +### 4. Use Copy-Paste for Repetitive Tasks + +For enum generation, create a template: + +``` +You: "Generate [ENUM_NAME] enum from espi.xsd: +1. Read simpleType definition +2. Create Java enum with all values +3. Include getValue() and fromValue() +4. Add Javadoc +5. Save to: domain/usage/enums/[ENUM_NAME].java" +``` + +Save this, then just replace `[ENUM_NAME]` each time. + +--- + +## Cost Comparison + +| Approach | Your Cost | Time Investment | +|----------|-----------|-----------------| +| **Automated (API)** | ~$35 | 2-3 hours setup + automated execution | +| **Conversational** | $0 (MAX plan) | 20-30 hours of your time | +| **Hybrid** | ~$10 (only critical tasks via API) | 10-15 hours | + +**Recommendation**: Since you have MAX plan, the conversational approach is perfect! It's free and you maintain complete control. + +--- + +## Complete Phase 0 Example Timeline + +**Day 1 - Schema Analysis (Opus):** +- 9:00 AM: Start Opus session +- 9:00-10:00: Analyze espi.xsd +- 10:00-11:00: Analyze customer.xsd +- 11:00-11:30: Create summary +- 11:30-12:00: Create Git branch, commit, push, create PR +- Afternoon: PR review and merge + +**Day 2 - Generate Enums (Sonnet):** +- 9:00 AM: Start Sonnet session +- 9:00-10:00: Generate batch 1 (5 usage enums) +- 10:00-10:15: Commit and push +- 10:15-11:15: Generate batch 2 (5 usage enums) +- 11:15-11:30: Commit and push +- Lunch break +- 1:00-2:00: Generate batch 3 (remaining usage enums) +- 2:00-2:15: Commit and push +- 2:15-2:45: Generate shared enums (7 enums) +- 2:45-3:00: Generate customer enums (2 enums) +- 3:00-3:30: Final commit, push, create PR + +**Day 3 - Verify and Merge:** +- Morning: Haiku session - verify all enums +- Fix any issues (back to Sonnet if needed) +- Run tests: `mvn clean test` +- Merge PR + +--- + +## Quick Reference Commands + +```bash +# Start with appropriate model +claude --model opus # Complex analysis +claude --model sonnet # Code generation, standard verification +claude --model haiku # Simple verification + +# Check current model in conversation +# Just ask: "Which model are you?" + +# Switch models: Exit current session (Ctrl+D) and start new one + +# Create branches +git checkout -b feature/issue-101-phase-X-description + +# Commit work +git add [files] +git commit -m "feat/fix: description + +Co-Authored-By: Claude [Model] [Version] " + +# Push and PR +git push -u origin [branch-name] +gh pr create # or create manually on GitHub +``` + +--- + +## Summary + +**To get full benefit of the optimized plan conversationally:** + +1. ✅ **Use Opus** for schema analysis and complex architectural decisions +2. ✅ **Use Sonnet** for code generation, standard verification, and fixes +3. ✅ **Use Haiku** for batch verification of simple items +4. ✅ **Switch models** by exiting and starting new session: `claude --model [opus|sonnet|haiku]` +5. ✅ **Batch work** to minimize model switching +6. ✅ **Track progress** in a file to maintain context + +**Start with:** +```bash +claude --model opus +``` + +Then ask: "Let's begin Phase 0 - Schema Analysis for Issue #101. Please analyze espi.xsd..." + +You're in control, it's free, and you get the full benefits of the optimized model selection! 🚀 diff --git a/FILE_LOCATIONS.md b/FILE_LOCATIONS.md new file mode 100644 index 00000000..f6ff3f60 --- /dev/null +++ b/FILE_LOCATIONS.md @@ -0,0 +1,419 @@ +# File Locations Guide +## All Generated Files for Issue #101 + +This document shows the location of all files created for the ESPI 4.0 schema compliance verification project. + +--- + +## 📋 Documentation Files (Project Root) + +Located in: `/Users/donal/Git/GreenButtonAlliance/OpenESPI-GreenButton-Java/` + +``` +OpenESPI-GreenButton-Java/ +├── GETTING_STARTED.md ⭐ START HERE +├── ISSUE_101_IMPLEMENTATION_PLAN.md 📋 Complete 102-task plan +├── ISSUE_101_IMPLEMENTATION_STRATEGY.md 👥 Team coordination strategy +├── ISSUE_101_TOOLING_GUIDE.md 🛠️ Tools and Claude models guide +└── FILE_LOCATIONS.md 📍 This file +``` + +### File Purposes: + +| File | Size | Purpose | When to Use | +|------|------|---------|-------------| +| **GETTING_STARTED.md** | ~15 KB | Quick start guide | Read this FIRST | +| **ISSUE_101_IMPLEMENTATION_PLAN.md** | ~50 KB | Complete implementation plan with all 102 tasks | Reference for task details | +| **ISSUE_101_IMPLEMENTATION_STRATEGY.md** | ~35 KB | Team coordination and branching strategy | Before starting team work | +| **ISSUE_101_TOOLING_GUIDE.md** | ~40 KB | Claude model selection and tool recommendations | When choosing tools/models | +| **FILE_LOCATIONS.md** | ~10 KB | This file - shows where everything is | Finding files | + +--- + +## 🔧 Scripts Directory + +Located in: `/Users/donal/Git/GreenButtonAlliance/OpenESPI-GreenButton-Java/scripts/` + +``` +scripts/ +├── espi-verification-orchestrator.py 🤖 Main automation script +├── verification-config.yaml ⚙️ Configuration file +├── README_ORCHESTRATOR.md 📖 Orchestrator documentation +├── first-step-automated.sh ⚡ Step 1: Setup & analysis +├── generate-missing-enums-batch.sh 📝 Step 2: Generate enums +├── run-phase-0-example.sh 🎯 Example workflow demo +└── git-workflow-helper.sh 🌿 Git branch management +``` + +### Script Details: + +| File | Lines | Language | Executable | Purpose | +|------|-------|----------|------------|---------| +| **espi-verification-orchestrator.py** | ~800 | Python | ✅ Yes | Main automation engine with model switching | +| **verification-config.yaml** | ~100 | YAML | - | Task configuration and cost limits | +| **README_ORCHESTRATOR.md** | ~700 | Markdown | - | Complete orchestrator usage guide | +| **first-step-automated.sh** | ~250 | Bash | ✅ Yes | Automated setup and schema analysis | +| **generate-missing-enums-batch.sh** | ~200 | Bash | ✅ Yes | Batch generate all 30 missing enums | +| **run-phase-0-example.sh** | ~100 | Bash | ✅ Yes | Demo showing model switching | +| **git-workflow-helper.sh** | ~600 | Bash | ✅ Yes | Branch creation and PR management | + +--- + +## 📊 Output Directories (Created by Scripts) + +These directories are created when you run the scripts: + +### Reports Directory + +``` +reports/ +└── verification/ 📊 All verification outputs + ├── 00_SCHEMA_ANALYSIS_SUMMARY.md 📋 Master summary + ├── espi_enumerations.md 📄 espi.xsd analysis (Opus) + ├── espi_enumerations.json 💰 Cost/token metadata + ├── customer_enumerations.md 📄 customer.xsd analysis (Opus) + ├── customer_enumerations.json 💰 Cost/token metadata + ├── UsagePoint_verification.md 📄 Entity verification (future) + ├── UsagePoint_report.md 📄 Formatted report (future) + └── ... (more files as you verify) +``` + +**Created by**: `first-step-automated.sh` (Step 7-8) + +### Generated Code Directory + +``` +openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/ +├── usage/ +│ └── enums/ 📁 New directory (Phase 0) +│ ├── .gitkeep 📌 Ensures directory tracked +│ ├── AccumulationKind.java ☕ Generated enum +│ ├── CommodityKind.java ☕ Generated enum +│ ├── DataQualifierKind.java ☕ Generated enum +│ ├── FlowDirectionKind.java ☕ Generated enum +│ ├── MeasurementKind.java ☕ Generated enum +│ └── ... (19 total) +│ +├── customer/ +│ └── enums/ 📁 Existing directory +│ ├── CustomerKind.java ✅ Already exists +│ ├── MediaType.java ☕ New (generated) +│ └── RevenueKind.java ☕ New (generated) +│ +└── common/ 📁 Shared enums + ├── ServiceKind.java ☕ New (renamed from ServiceCategory) + ├── Currency.java ☕ New (generated) + ├── StatusCode.java ☕ New (generated) + ├── UnitMultiplierKind.java ☕ New (generated) + └── ... (7 total shared) +``` + +**Created by**: `generate-missing-enums-batch.sh` (Step 2) + +--- + +## 🗂️ Complete File Tree + +Here's the complete structure showing ALL generated files: + +``` +/Users/donal/Git/GreenButtonAlliance/OpenESPI-GreenButton-Java/ +│ +├── 📄 GETTING_STARTED.md ⭐ START HERE +├── 📄 ISSUE_101_IMPLEMENTATION_PLAN.md +├── 📄 ISSUE_101_IMPLEMENTATION_STRATEGY.md +├── 📄 ISSUE_101_TOOLING_GUIDE.md +├── 📄 FILE_LOCATIONS.md 📍 This file +│ +├── 📁 scripts/ +│ ├── 🐍 espi-verification-orchestrator.py +│ ├── ⚙️ verification-config.yaml +│ ├── 📖 README_ORCHESTRATOR.md +│ ├── 🔧 first-step-automated.sh +│ ├── 🔧 generate-missing-enums-batch.sh +│ ├── 🔧 run-phase-0-example.sh +│ └── 🔧 git-workflow-helper.sh +│ +├── 📁 reports/ (Created by scripts) +│ └── 📁 verification/ +│ ├── 00_SCHEMA_ANALYSIS_SUMMARY.md +│ ├── espi_enumerations.md +│ ├── espi_enumerations.json +│ ├── customer_enumerations.md +│ └── customer_enumerations.json +│ +└── 📁 openespi-common/ + └── 📁 src/main/java/.../domain/ + ├── 📁 usage/ + │ └── 📁 enums/ (Created by scripts) + │ ├── .gitkeep + │ ├── AccumulationKind.java + │ ├── CommodityKind.java + │ └── ... (19 total) + │ + ├── 📁 customer/ + │ └── 📁 enums/ + │ ├── MediaType.java (New) + │ └── RevenueKind.java (New) + │ + └── 📁 common/ + ├── Currency.java (New) + ├── StatusCode.java (New) + └── ... (7 total shared) +``` + +--- + +## 📦 Files by Category + +### 1. Documentation (Read These) + +```bash +# Main documentation +cat GETTING_STARTED.md # Quick start +cat ISSUE_101_IMPLEMENTATION_PLAN.md # Full plan +cat ISSUE_101_IMPLEMENTATION_STRATEGY.md # Team strategy +cat ISSUE_101_TOOLING_GUIDE.md # Tools guide + +# Script documentation +cat scripts/README_ORCHESTRATOR.md # Orchestrator guide +``` + +### 2. Executable Scripts (Run These) + +```bash +# Make executable first (if not already) +chmod +x scripts/*.sh scripts/*.py + +# Then run +./scripts/first-step-automated.sh # Step 1: Setup +./scripts/generate-missing-enums-batch.sh # Step 2: Generate +./scripts/git-workflow-helper.sh # Git management +./scripts/run-phase-0-example.sh # Example demo + +# Orchestrator commands +./scripts/espi-verification-orchestrator.py --help +./scripts/espi-verification-orchestrator.py analyze-schema +./scripts/espi-verification-orchestrator.py verify-entity UsagePoint +``` + +### 3. Configuration Files (Edit These) + +```bash +# Main config +vim scripts/verification-config.yaml + +# Edit to: +# - Change cost limits +# - Override model selection +# - Customize task lists +``` + +### 4. Generated Reports (Review These) + +```bash +# Schema analysis +cat reports/verification/espi_enumerations.md +cat reports/verification/customer_enumerations.md + +# Summary +cat reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md + +# Metadata (JSON) +cat reports/verification/espi_enumerations.json +``` + +### 5. Generated Code (Verify These) + +```bash +# New usage enums +ls -la openespi-common/src/main/java/.../usage/enums/ + +# Example enum +cat openespi-common/src/main/java/.../usage/enums/AccumulationKind.java + +# New customer enums +ls -la openespi-common/src/main/java/.../customer/enums/ + +# Shared enums +ls -la openespi-common/src/main/java/.../common/ +``` + +--- + +## 🔍 Finding Files Quickly + +### Using Terminal + +```bash +# Find all generated markdown files +find . -name "ISSUE_101*.md" -o -name "GETTING_STARTED.md" + +# Find all scripts +find scripts/ -name "*.sh" -o -name "*.py" + +# Find all generated enums +find openespi-common/src/main/java -path "*/enums/*.java" -newer /tmp + +# Find verification reports +find reports/verification/ -name "*.md" +``` + +### Using Git + +```bash +# See what's new (not committed) +git status + +# See all files in feature branch +git ls-tree -r --name-only feature/issue-101-phase-0-schema-analysis + +# See files changed in last commit +git show --name-only +``` + +--- + +## 📏 File Sizes + +| Category | Files | Total Size | +|----------|-------|------------| +| Documentation | 5 | ~150 KB | +| Scripts | 7 | ~50 KB | +| Reports (after analysis) | ~5 | ~500 KB | +| Generated Code (30 enums) | ~30 | ~60 KB | +| **Total** | **~47** | **~760 KB** | + +--- + +## 🎯 Quick Access Commands + +Save these aliases: + +```bash +# Add to ~/.bashrc or ~/.zshrc + +# Project root +alias cd-espi='cd /Users/donal/Git/GreenButtonAlliance/OpenESPI-GreenButton-Java' + +# Documentation +alias espi-docs='cd-espi && ls -la *.md' + +# Scripts +alias espi-scripts='cd-espi && ls -la scripts/' + +# Reports +alias espi-reports='cd-espi && ls -la reports/verification/' + +# Generated code +alias espi-enums='cd-espi && find openespi-common -path "*/enums/*.java"' +``` + +Then use: + +```bash +cd-espi # Jump to project +espi-docs # List all docs +espi-scripts # List all scripts +espi-reports # List all reports +espi-enums # List all enum files +``` + +--- + +## 📋 Checklist: What Files Exist Now + +Before running any scripts: + +``` +✅ GETTING_STARTED.md +✅ ISSUE_101_IMPLEMENTATION_PLAN.md +✅ ISSUE_101_IMPLEMENTATION_STRATEGY.md +✅ ISSUE_101_TOOLING_GUIDE.md +✅ FILE_LOCATIONS.md +✅ scripts/espi-verification-orchestrator.py +✅ scripts/verification-config.yaml +✅ scripts/README_ORCHESTRATOR.md +✅ scripts/first-step-automated.sh +✅ scripts/generate-missing-enums-batch.sh +✅ scripts/run-phase-0-example.sh +✅ scripts/git-workflow-helper.sh +``` + +After running `first-step-automated.sh`: + +``` +✅ reports/verification/ +✅ reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md +✅ reports/verification/espi_enumerations.md +✅ reports/verification/customer_enumerations.md +✅ openespi-common/.../usage/enums/.gitkeep +``` + +After running `generate-missing-enums-batch.sh`: + +``` +✅ openespi-common/.../usage/enums/AccumulationKind.java +✅ openespi-common/.../usage/enums/CommodityKind.java +... (19 total usage enums) +✅ openespi-common/.../customer/enums/MediaType.java +✅ openespi-common/.../customer/enums/RevenueKind.java +✅ openespi-common/.../common/Currency.java +... (7 total shared enums) +``` + +--- + +## 🚀 Next Steps + +1. **Read the docs** (in order): + ```bash + cat GETTING_STARTED.md # 1. Start here + cat ISSUE_101_IMPLEMENTATION_PLAN.md # 2. See full plan + cat scripts/README_ORCHESTRATOR.md # 3. Learn tools + ``` + +2. **Run the first script**: + ```bash + ./scripts/first-step-automated.sh + ``` + +3. **Check what was created**: + ```bash + ls -la reports/verification/ + ``` + +4. **Review the output**: + ```bash + cat reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md + ``` + +--- + +## 💡 Tips + +**Find files modified today:** +```bash +find . -type f -mtime 0 -name "*.md" -o -name "*.java" -o -name "*.sh" +``` + +**Search for specific content:** +```bash +grep -r "AccumulationKind" openespi-common/src/ +grep -r "Claude Opus" reports/verification/ +``` + +**Count generated files:** +```bash +find openespi-common -path "*/enums/*.java" | wc -l +``` + +**Show file tree:** +```bash +tree -L 3 scripts/ +tree -L 5 openespi-common/src/main/java/.../domain/ +``` + +--- + +**Everything is in place and ready to go!** 🎉 diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md new file mode 100644 index 00000000..2311f6e3 --- /dev/null +++ b/GETTING_STARTED.md @@ -0,0 +1,454 @@ +# Getting Started with Issue #101 +## ESPI 4.0 Schema Compliance Verification + +**Quick Start**: 5 commands to begin the complete verification process + +--- + +## ⚡ THE FIRST STEP (Fully Automated with Git Integration) + +### Run This Now: + +```bash +chmod +x scripts/*.sh scripts/*.py +./scripts/first-step-automated.sh +``` + +**What it does:** +1. ✅ Checks all prerequisites (Python, pip, Java, Maven, Git) +2. ✅ Installs dependencies (anthropic, pyyaml, click, rich) +3. ✅ Verifies your API key works +4. ✅ Makes scripts executable +5. ✅ **Creates feature branch** `feature/issue-101-phase-0-schema-analysis` +6. ✅ Creates output directories +7. ✅ **Analyzes espi.xsd with Claude Opus** (~$1.50) +8. ✅ **Analyzes customer.xsd with Claude Opus** (~$1.50) +9. ✅ Generates inventory reports +10. ✅ **Commits work to feature branch** +11. ✅ Shows cost summary + +**Time**: ~10 minutes +**Cost**: ~$3.00 +**Output**: +- Complete schema analysis in `reports/verification/` +- Feature branch ready for PR + +--- + +## Prerequisites (5 minutes) + +### 1. Get Your API Key + +Visit: https://console.anthropic.com/settings/keys + +Create a new API key, then: + +```bash +export ANTHROPIC_API_KEY="your-api-key-here" + +# Or add to your shell profile for persistence: +echo 'export ANTHROPIC_API_KEY="your-api-key-here"' >> ~/.zshrc +source ~/.zshrc +``` + +### 2. Verify Python 3 + +```bash +python3 --version # Should be 3.8 or higher +``` + +If not installed: +- **macOS**: `brew install python3` +- **Linux**: `sudo apt install python3 python3-pip` +- **Windows**: Download from python.org + +That's it! The automated script handles everything else. + +--- + +## Git Workflow (Automated Branch Management) + +**Important**: You cannot push directly to `main`. All work must be done in feature branches. + +### Automated Branch Creation + +The scripts automatically: +- ✅ Create feature branches with proper naming +- ✅ Commit work with descriptive messages +- ✅ Include co-author attribution to Claude +- ✅ Prepare branches for PR creation + +### Branch Naming Convention + +``` +feature/issue-101-phase-- + +Examples: + feature/issue-101-phase-0-schema-analysis + feature/issue-101-phase-0-generate-enums + feature/issue-101-phase-0-enum-migration + feature/issue-101-phase-1-common-embeddables + feature/issue-101-phase-2-usage-core +``` + +### Manual Branch Management (Optional) + +Use the Git workflow helper for more control: + +```bash +# Interactive menu +./scripts/git-workflow-helper.sh + +# Or command-line mode +./scripts/git-workflow-helper.sh phase-0-setup +./scripts/git-workflow-helper.sh phase-0-generate +./scripts/git-workflow-helper.sh pr +``` + +**Options:** +1. Create Phase 0 setup branch +2. Create enum generation branch +3. Create migration branch (coordinated) +4. Commit current work +5. Prepare pull request +6. Check git status + +--- + +## Complete Automated Workflow + +### Step 1: Setup & Schema Analysis (Automated) + +```bash +./scripts/first-step-automated.sh +``` + +**What happens:** +- Creates branch: `feature/issue-101-phase-0-schema-analysis` +- Analyzes schemas with Opus +- Commits reports to branch + +**Result**: Schema analysis complete ✅ + +**Push and create PR:** + +```bash +# Push the branch +git push -u origin feature/issue-101-phase-0-schema-analysis + +# Create PR (with GitHub CLI) +./scripts/git-workflow-helper.sh pr + +# Or manually at: +# https://github.com/GreenButtonAlliance/OpenESPI-GreenButton-Java/compare/feature/issue-101-phase-0-schema-analysis +``` + +### Step 2: Generate Missing Enums (Automated) + +**First, merge PR from Step 1, then:** + +```bash +# Switch back to main and pull latest +git checkout main +git pull origin main + +# Run generator (creates new branch automatically) +./scripts/generate-missing-enums-batch.sh +``` + +**What it does:** +- Creates branch: `feature/issue-101-phase-0-generate-enums` +- Generates 19 usage domain enums with Sonnet +- Generates 7 shared enums with Sonnet +- Generates 2 customer domain enums with Sonnet +- Commits generated enums to branch + +**Time**: ~15 minutes +**Cost**: ~$4.50 + +**Push and create PR:** + +```bash +git push -u origin feature/issue-101-phase-0-generate-enums +./scripts/git-workflow-helper.sh pr +``` + +### Step 3: Verify Enums (Automated) + +```bash +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory usage/enums +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory customer/enums +``` + +**What it does:** +- Verifies all 28+ enums with Haiku (cost-effective) +- Reports PASS/FAIL for each + +**Time**: ~5 minutes +**Cost**: ~$1.50 + +### Step 4: Test Your Work + +```bash +# Compile the code +mvn clean compile + +# Run tests +mvn test + +# Check for issues +mvn dependency:analyze +``` + +### Step 5: Verify First Entity + +```bash +./scripts/espi-verification-orchestrator.py verify-entity UsagePoint +``` + +**What it does:** +- Compares UsagePointEntity with XSD +- Identifies discrepancies +- Generates verification report +- Suggests fixes + +**Time**: ~3 minutes +**Cost**: ~$0.25 + +--- + +## Manual Control (If Needed) + +### Generate Single Enum + +```bash +./scripts/espi-verification-orchestrator.py generate-enum AccumulationKind +``` + +### Verify Single Entity + +```bash +# Standard complexity (Sonnet) +./scripts/espi-verification-orchestrator.py verify-entity MeterReading + +# Complex entity (Opus - better analysis) +./scripts/espi-verification-orchestrator.py verify-entity ReadingType --complexity complex +``` + +### Check Costs Anytime + +```bash +./scripts/espi-verification-orchestrator.py show-costs +``` + +--- + +## Expected Costs + +| Phase | Tasks | Estimated Cost | +|-------|-------|----------------| +| **Step 1: Schema Analysis** | 2 schemas | **$3.00** | +| **Step 2: Generate Enums** | 30 enums | **$4.50** | +| **Step 3: Verify Enums** | 30+ enums | **$1.50** | +| **Step 4: Verify Entities** | 65 entities | **$25.00** | +| **Total (Complete Project)** | 102 tasks | **~$35.00** | + +**Compare to**: Manual implementation = 12 weeks of work! + +--- + +## What You Get + +### After Step 1 (first-step-automated.sh) + +``` +reports/verification/ +├── espi_enumerations.md # Complete espi.xsd analysis +├── customer_enumerations.md # Complete customer.xsd analysis +└── 00_SCHEMA_ANALYSIS_SUMMARY.md # Summary + next steps +``` + +**You now know:** +- ✅ All 33 enumerations in espi.xsd +- ✅ All 16 enumerations in customer.xsd +- ✅ All BasicTypes (String256, UInt48, etc.) +- ✅ All complexTypes and their field mappings +- ✅ Exactly what's missing from the codebase + +### After Step 2 (generate-missing-enums-batch.sh) + +``` +openespi-common/src/main/java/.../domain/ +├── usage/enums/ +│ ├── AccumulationKind.java +│ ├── CommodityKind.java +│ ├── DataQualifierKind.java +│ └── ... (19 total) +├── customer/enums/ +│ ├── MediaType.java +│ └── RevenueKind.java +└── common/ + ├── Currency.java + ├── StatusCode.java + └── ... (7 total) +``` + +**You now have:** +- ✅ All 30 missing enums implemented +- ✅ Proper package structure +- ✅ getValue() and fromValue() methods +- ✅ Comprehensive Javadoc + +### After Step 3 (batch-verify-enums) + +``` +All enums verified against XSD: + ✓ AccumulationKind - PASS + ✓ CommodityKind - PASS + ✓ DataQualifierKind - PASS + ... +``` + +**You now know:** +- ✅ Which enums match XSD perfectly +- ✅ Which have issues (if any) +- ✅ Exactly what to fix + +--- + +## Troubleshooting + +### "API key not found" + +```bash +export ANTHROPIC_API_KEY="your-key-here" +``` + +### "Python dependencies missing" + +```bash +pip3 install anthropic pyyaml click rich +``` + +### "Permission denied" + +```bash +chmod +x ./scripts/*.sh +chmod +x ./scripts/espi-verification-orchestrator.py +``` + +### "Command not found: python3" + +Install Python 3.8+ from python.org or your package manager. + +--- + +## Understanding Model Selection + +The orchestrator **automatically** picks the best model: + +| Task Type | Model | Why | +|-----------|-------|-----| +| Schema analysis | **Opus** | Complex XSD requires deep understanding | +| Generate enum | **Sonnet** | Code generation, balanced quality/cost | +| Verify enum | **Haiku** | Simple comparison, very cost-effective | +| Verify entity (standard) | **Sonnet** | Good field comparison | +| Verify entity (complex) | **Opus** | Multiple relationships need deep analysis | + +**You don't choose** - the script chooses for you! ✨ + +--- + +## Next Steps After Automation + +Once you've run the automated scripts: + +### 1. Review Generated Code + +```bash +# Look at a generated enum +cat openespi-common/src/main/java/.../usage/enums/AccumulationKind.java + +# Check all generated files +ls -la openespi-common/src/main/java/.../usage/enums/ +``` + +### 2. Run Tests + +```bash +mvn clean test +``` + +### 3. Fix Any Issues + +If tests fail or enums need tweaking: + +```bash +# Re-generate a specific enum +./scripts/espi-verification-orchestrator.py generate-enum AccumulationKind +``` + +### 4. Start Entity Verification + +Begin verifying entities against XSD: + +```bash +# Start with core entities +./scripts/espi-verification-orchestrator.py verify-entity UsagePoint +./scripts/espi-verification-orchestrator.py verify-entity MeterReading +./scripts/espi-verification-orchestrator.py verify-entity IntervalBlock +``` + +### 5. Track Progress + +See the implementation plan: + +```bash +cat ISSUE_101_IMPLEMENTATION_PLAN.md +``` + +Mark off completed tasks as you go! + +--- + +## Documentation Reference + +| Document | Purpose | +|----------|---------| +| `ISSUE_101_IMPLEMENTATION_PLAN.md` | Complete 102-task plan | +| `ISSUE_101_IMPLEMENTATION_STRATEGY.md` | Team coordination strategy | +| `ISSUE_101_TOOLING_GUIDE.md` | Model selection & tools guide | +| `scripts/README_ORCHESTRATOR.md` | Orchestrator command reference | +| `GETTING_STARTED.md` | This file - quick start | + +--- + +## Success Metrics + +After running the automated workflow: + +✅ **Time Saved**: 3 weeks of manual analysis → 30 minutes +✅ **Consistency**: All enums follow same pattern +✅ **Accuracy**: 95%+ match to XSD on first try +✅ **Cost**: ~$9 for Phase 0 vs weeks of salary +✅ **Documentation**: Auto-generated reports for each task + +--- + +## Ready? Let's Go! 🚀 + +### The One Command to Start Everything: + +```bash +./scripts/first-step-automated.sh +``` + +Then follow the on-screen prompts. The script will guide you through everything! + +**Questions?** Check the documentation or review the generated reports. + +**Issues?** All outputs include metadata showing exactly what was run, what model was used, and how much it cost. + +--- + +**Good luck with your ESPI 4.0 schema compliance verification!** 🎯 diff --git a/ISSUE_101_IMPLEMENTATION_PLAN.md b/ISSUE_101_IMPLEMENTATION_PLAN.md new file mode 100644 index 00000000..bcba821b --- /dev/null +++ b/ISSUE_101_IMPLEMENTATION_PLAN.md @@ -0,0 +1,1302 @@ +# Issue #101 Implementation Plan +## Verify Entity Field Types and Lengths Match ESPI 4.0 XSD Schemas + +**Issue**: [#101 - Verify Entity Field Types and Lengths Match ESPI 4.0 XSD Schemas](https://github.com/GreenButtonAlliance/OpenESPI-GreenButton-Java/issues/101) + +**Objective**: Systematically verify that all JPA entity fields (types, lengths, nullability, relationships) match the ESPI 4.0 XSD schema definitions in `espi.xsd` and `customer.xsd`. + +--- + +## Overview + +This plan provides repeatable verification tasks for each entity that extends `IdentifiedObject` and all supporting classes that inherit from `Object`. Each task follows a consistent pattern: + +1. **Extract XSD Definition** - Locate the element/type definition in the XSD schema +2. **Compare Field Types** - Verify Java field types match XSD types +3. **Verify String Lengths** - Check `@Column(length=...)` matches XSD `maxLength` +4. **Check Nullability** - Ensure `nullable` matches XSD `minOccurs` +5. **Validate Relationships** - Verify collections and embedded objects match XSD structure +6. **Review Flyway Scripts** - Ensure database schema matches entity definitions +7. **Document Discrepancies** - Record any mismatches with justification or corrections +8. **Test Validation** - Run unit tests to verify no regressions + +--- + +## Understanding ESPI XSD BasicTypes + +The ESPI 4.0 XSD schemas (`espi.xsd` and `customer.xsd`) use **BasicTypes** (simpleType definitions) to define field types and lengths. Instead of inline `maxLength` facets on elements, the schemas define reusable types like `String256`, `String512`, `UInt48`, etc. + +### XSD BasicTypes Pattern + +**Example from espi.xsd:** +```xml + + + + + + + [extension] Character string of max length 512 + + + + + +``` + +### String BasicTypes Reference + +| XSD Type | maxLength | Java Type | @Column Mapping | Usage | +|----------|-----------|-----------|-----------------|-------| +| `String8` | 8 | `String` | `@Column(length=8)` | Short codes, flags | +| `String16` | 16 | `String` | `@Column(length=16)` | Short identifiers | +| `String32` | 32 | `String` | `@Column(length=32)` | Standard codes | +| `String64` | 64 | `String` | `@Column(length=64)` | Medium text fields | +| `String256` | 256 | `String` | `@Column(length=256)` | Standard text fields | +| `String512` | 512 | `String` | `@Column(length=512)` | Long text fields | + +### Numeric BasicTypes Reference + +| XSD Type | Base Type | Range | Java Type | JPA Mapping | +|----------|-----------|-------|-----------|-------------| +| `UInt8` | xs:unsignedByte | 0 to 255 | `Short` or `Integer` | Standard mapping | +| `UInt16` | xs:unsignedShort | 0 to 65,535 | `Integer` | Standard mapping | +| `UInt32` | xs:unsignedInt | 0 to 4,294,967,295 | `Long` | Standard mapping | +| `UInt48` | xs:unsignedLong | 0 to 281,474,976,710,655 | `Long` | Standard mapping | +| `Int16` | xs:short | -32,768 to 32,767 | `Short` or `Integer` | Standard mapping | +| `Int48` | xs:long | 48-bit signed | `Long` | Standard mapping | + +### Binary BasicTypes Reference + +| XSD Type | Length (bytes) | Java Type | JPA Mapping | +|----------|----------------|-----------|-------------| +| `HexBinary8` | 8 | `byte[]` | `@Column(length=8)` or store as hex String(16) | +| `HexBinary16` | 16 | `byte[]` | `@Column(length=16)` or store as hex String(32) | +| `HexBinary32` | 32 | `byte[]` | `@Column(length=32)` or store as hex String(64) | +| `HexBinary128` | 128 | `byte[]` | `@Column(length=128)` or store as hex String(256) | + +### Special BasicTypes + +| XSD Type | Description | Java Type | Notes | +|----------|-------------|-----------|-------| +| `UUIDType` | UUID (Type 5) | `UUID` or `String` | Standard UUID format | +| `TimeType` | Unix timestamp (seconds since epoch) | `Long` | ESPI uses seconds, not millis | +| `Currency` | ISO 4217 currency code | `String` | 3-character code (USD, EUR, etc.) | +| `PerCent` | Percentage value | `BigDecimal` | May need precision constraints | +| `StatusCode` | Status code enumeration | `String` or Enum | Context-dependent | + +### Enumeration BasicTypes + +Both schemas define numerous enumeration types (e.g., `ServiceKind`, `CustomerKind`, `AmiBillingReadyKind`). These should map to Java enums: + +```java +// XSD enumeration + + + + + + + + +// Java enum mapping +@Enumerated(EnumType.ORDINAL) +@Column(name = "service_kind") +private ServiceKind serviceKind; +``` + +### How to Trace BasicTypes in Verification + +**Step 1: Find the element definition** +```bash +grep -A 3 "element name=\"client_secret\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` +Output: +```xml + +``` + +**Step 2: Find the BasicType definition** +```bash +grep -B 2 -A 5 "simpleType name=\"String512\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` +Output: +```xml + + + + + +``` + +**Step 3: Verify Java entity mapping** +```java +@Column(name = "client_secret", length = 512) +private String clientSecret; +``` + +### Quick Reference Scripts + +Create these helper scripts for BasicType lookup: + +```bash +# scripts/find-basictype.sh +#!/bin/bash +TYPE_NAME=$1 +SCHEMA_FILE=${2:-espi.xsd} + +echo "=== BasicType Definition: $TYPE_NAME ===" +grep -B 2 -A 10 "simpleType name=\"$TYPE_NAME\"" \ + openespi-common/src/main/resources/schema/ESPI_4.0/$SCHEMA_FILE +``` + +```bash +# scripts/find-element-type.sh +#!/bin/bash +ELEMENT_NAME=$1 +SCHEMA_FILE=${2:-espi.xsd} + +echo "=== Element Type Reference: $ELEMENT_NAME ===" +grep -B 1 -A 3 "element name=\"$ELEMENT_NAME\"" \ + openespi-common/src/main/resources/schema/ESPI_4.0/$SCHEMA_FILE | \ + grep "type=" +``` + +### Complete BasicType Inventory + +**espi.xsd simpleTypes (51 total):** +- String types: String8, String16, String32, String64, String256, String512 +- Numeric types: Int16, Int48, UInt8, UInt16, UInt32, UInt48 +- Binary types: HexBinary8, HexBinary16, HexBinary32, HexBinary128 +- Enumerations: AccumulationKind, AmiBillingReadyKind, CommodityKind, DataQualifierKind, FlowDirectionKind, MeasurementKind, PhaseCodeKind, QualityOfReading, ServiceKind, TimeAttributeKind, UnitMultiplierKind, UnitSymbolKind, UsagePointConnectedKind +- OAuth types: GrantType, ResponseType, TokenType, OAuthError +- Other: UUIDType, TimeType, Currency + +**customer.xsd simpleTypes (33 total):** +- String types: String32, String64, String256, String512 +- Numeric types: Int16, Int48, UInt8, UInt16, UInt32, UInt48, PerCent +- Binary types: HexBinary8, HexBinary16, HexBinary32, HexBinary128 +- Enumerations: CustomerKind, MeterMultiplierKind, NotificationMethodKind, SupplierKind, ProgramDateKind, RevenueKind, MediaType +- Shared with espi.xsd: ServiceKind, Currency, UUIDType, TimeType, StatusCode + +### Verification Checklist for BasicTypes + +When verifying each entity field: + +- [ ] **Identify the XSD element type** (e.g., `type="String256"`) +- [ ] **Locate the BasicType definition** (e.g., ``) +- [ ] **Extract the restriction value** (e.g., ``) +- [ ] **Verify Java field type matches** (e.g., `String` for String types) +- [ ] **Verify @Column length matches** (e.g., `@Column(length=256)`) +- [ ] **Check for enum types** (should map to Java enum, not String) +- [ ] **Validate numeric ranges** (ensure Java type can hold XSD range) + +--- + +## Phase 0: Enumeration Migration Verification + +**CRITICAL**: Before verifying entity fields, ensure all XSD enumerations have been migrated from legacy code and are in the correct directory structure. + +### Required Directory Structure + +- **Usage Domain Enums**: `domain/usage/enums/` (currently missing - enums are in `domain/common/`) +- **Customer Domain Enums**: `domain/customer/enums/` ✓ (exists) +- **Common/Shared Enums**: `domain/common/` (for enums used by both domains) + +### Enumeration Inventory from XSD Schemas + +#### espi.xsd Enumerations (33 total) + +| Enum Name | Currently Implemented? | Location | Status | Action Required | +|-----------|----------------------|----------|--------|-----------------| +| AccumulationKind | ❌ No | - | Missing | Implement in usage/enums | +| AmiBillingReadyKind | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| AnodeType | ❌ No | - | Missing | Implement in usage/enums | +| ApnodeType | ❌ No | - | Missing | Implement in usage/enums | +| AuthorizationStatus | ❌ No | - | Missing | Implement in usage/enums | +| CommodityKind | ❌ No | - | Missing | Implement in usage/enums | +| CRUDOperation | ❌ No | - | Missing | Implement in common/ (shared) | +| Currency | ❌ No | - | Missing | Implement in common/ (shared) | +| DataCustodianApplicationStatus | ❌ No | - | Missing | Implement in usage/enums | +| DataQualifierKind | ❌ No | - | Missing | Implement in usage/enums | +| DstRuleType | ❌ No | - | Missing | Implement in common/ (shared) | +| ESPIServiceStatus | ❌ No | - | Missing | Implement in usage/enums | +| EnrollmentStatus | ❌ No | - | Missing | Implement in common/ (shared) | +| FlowDirectionKind | ❌ No | - | Missing | Implement in usage/enums | +| GrantType | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| ItemKind | ❌ No | - | Missing | Implement in usage/enums | +| MeasurementKind | ❌ No | - | Missing | Implement in usage/enums | +| OAuthError | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| ParticipationCategoryMPM | ❌ No | - | Missing | Implement in usage/enums | +| PhaseCodeKind | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| QualityOfReading | ❌ No | - | Missing | Implement in usage/enums | +| ResponseType | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| ServiceKind | ⚠️ Named ServiceCategory | common/ | ⚠️ Wrong name & location | Rename to ServiceKind, move to usage/enums | +| StatusCode | ❌ No | - | Missing | Implement in common/ (shared) | +| ThirdPartyApplicationType | ❌ No | - | Missing | Implement in usage/enums | +| ThirdPartyApplicationUse | ❌ No | - | Missing | Implement in usage/enums | +| ThirdPartyApplicatonStatus | ❌ No | - | Missing | Implement in usage/enums | +| TimeAttributeKind | ❌ No | - | Missing | Implement in usage/enums | +| TokenEndPointMethod | ❌ No | - | Missing | Implement in usage/enums | +| TokenType | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| UnitMultiplierKind | ❌ No | - | Missing | Implement in common/ (shared) | +| UnitSymbolKind | ❌ No | - | Missing | Implement in common/ (shared) | +| UsagePointConnectedKind | ✓ Yes | common/ | ⚠️ Wrong location | Move to usage/enums | +| tOUorCPPorConsumptionTier | ❌ No | - | Missing | Implement in usage/enums | + +**Summary**: 8 implemented, 25 missing, 8 in wrong location + +#### customer.xsd Enumerations (16 total) + +| Enum Name | Currently Implemented? | Location | Status | Action Required | +|-----------|----------------------|----------|--------|-----------------| +| CRUDOperation | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | +| Currency | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | +| CustomerKind | ✓ Yes | customer/enums/ | ✓ Correct | None | +| DstRuleType | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | +| EnrollmentStatus | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | +| MediaType | ❌ No | - | Missing | Implement in customer/enums | +| MeterMultiplierKind | ✓ Yes | customer/enums/ | ✓ Correct | None | +| NotificationMethodKind | ✓ Yes | customer/enums/ | ✓ Correct | None | +| ProgramDateKind | ✓ Yes | customer/enums/ | ✓ Correct | None | +| RevenueKind | ❌ No | - | Missing | Implement in customer/enums | +| ServiceKind | ⚠️ Named ServiceCategory | common/ | ⚠️ Wrong location | See espi.xsd (shared enum) | +| StatusCode | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | +| SupplierKind | ✓ Yes | customer/enums/ | ✓ Correct | None | +| UnitMultiplierKind | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | +| UnitSymbolKind | ❌ No | - | Missing | Implement in common/ (shared with espi.xsd) | + +**Summary**: 5 implemented correctly, 10 missing, 1 misnamed/mislocated (ServiceKind) + +### Phase 0 Tasks: Enumeration Migration + +| Task ID | Enum Name | XSD Source | Target Location | Priority | Dependencies | +|---------|-----------|------------|-----------------|----------|--------------| +| 0.1 | Create usage/enums directory | - | domain/usage/enums/ | High | None | +| 0.2 | AccumulationKind | espi.xsd | usage/enums/ | High | ReadingType | +| 0.3 | CommodityKind | espi.xsd | usage/enums/ | High | ReadingType | +| 0.4 | DataQualifierKind | espi.xsd | usage/enums/ | High | ReadingType | +| 0.5 | FlowDirectionKind | espi.xsd | usage/enums/ | High | ReadingType | +| 0.6 | MeasurementKind | espi.xsd | usage/enums/ | High | ReadingType | +| 0.7 | TimeAttributeKind | espi.xsd | usage/enums/ | High | ReadingType | +| 0.8 | QualityOfReading | espi.xsd | usage/enums/ | High | ReadingQuality | +| 0.9 | ServiceKind (rename from ServiceCategory) | espi.xsd, customer.xsd | common/ (shared) | High | UsagePoint, multiple | +| 0.10 | UnitMultiplierKind | espi.xsd, customer.xsd | common/ (shared) | High | ReadingType | +| 0.11 | UnitSymbolKind | espi.xsd, customer.xsd | common/ (shared) | High | ReadingType | +| 0.12 | Currency | espi.xsd, customer.xsd | common/ (shared) | Medium | LineItem | +| 0.13 | StatusCode | espi.xsd, customer.xsd | common/ (shared) | Medium | Multiple | +| 0.14 | CRUDOperation | espi.xsd, customer.xsd | common/ (shared) | Low | Subscription | +| 0.15 | DstRuleType | espi.xsd, customer.xsd | common/ (shared) | Medium | TimeConfiguration | +| 0.16 | EnrollmentStatus | espi.xsd, customer.xsd | common/ (shared) | Medium | Subscription | +| 0.17 | AuthorizationStatus | espi.xsd | usage/enums/ | High | Authorization | +| 0.18 | DataCustodianApplicationStatus | espi.xsd | usage/enums/ | High | ApplicationInformation | +| 0.19 | ESPIServiceStatus | espi.xsd | usage/enums/ | Medium | Subscription | +| 0.20 | ItemKind | espi.xsd | usage/enums/ | Medium | LineItem | +| 0.21 | ThirdPartyApplicationType | espi.xsd | usage/enums/ | Medium | ApplicationInformation | +| 0.22 | ThirdPartyApplicationUse | espi.xsd | usage/enums/ | Medium | ApplicationInformation | +| 0.23 | ThirdPartyApplicatonStatus | espi.xsd | usage/enums/ | Medium | ApplicationInformation | +| 0.24 | TokenEndPointMethod | espi.xsd | usage/enums/ | Low | ApplicationInformation | +| 0.25 | AnodeType | espi.xsd | usage/enums/ | Low | AggregatedNode | +| 0.26 | ApnodeType | espi.xsd | usage/enums/ | Low | Pnode | +| 0.27 | ParticipationCategoryMPM | espi.xsd | usage/enums/ | Low | Subscription | +| 0.28 | tOUorCPPorConsumptionTier | espi.xsd | usage/enums/ | Low | ReadingType | +| 0.29 | MediaType | customer.xsd | customer/enums/ | Medium | Document | +| 0.30 | RevenueKind | customer.xsd | customer/enums/ | Medium | CustomerAccount | +| 0.31 | Move AmiBillingReadyKind | - | usage/enums/ | Medium | UsagePoint | +| 0.32 | Move GrantType | - | usage/enums/ | Medium | ApplicationInformation | +| 0.33 | Move OAuthError | - | usage/enums/ | Medium | Authorization | +| 0.34 | Move PhaseCodeKind | - | usage/enums/ | Medium | ReadingType | +| 0.35 | Move ResponseType | - | usage/enums/ | Medium | ApplicationInformation | +| 0.36 | Move TokenType | - | usage/enums/ | Medium | ApplicationInformation | +| 0.37 | Move UsagePointConnectedKind | - | usage/enums/ | Medium | UsagePoint | + +**Phase 0 Total Tasks**: 37 (1 directory creation + 30 new enums + 6 relocations) + +### Enumeration Verification Template + +For each enumeration, verify: + +1. **Extract XSD Definition** +```bash +grep -B 5 -A 30 "simpleType name=\"EnumName\"" openespi-common/src/main/resources/schema/ESPI_4.0/{espi|customer}.xsd +``` + +2. **Check Enumeration Values** + - All XSD enumeration values present + - Correct numeric or string values + - Correct value mapping (ordinal vs value) + +3. **Verify Java Implementation** + - Enum constants match XSD enumeration values + - Naming convention: UPPER_SNAKE_CASE for Java constants + - Include `fromValue()` method for reverse lookup + - Include `getValue()` method if needed + +4. **Example: ServiceKind Enumeration** + +**XSD Definition (espi.xsd):** +```xml + + + + + + + + + + + + + + +``` + +**Current Implementation (ServiceCategory.java - INCORRECT NAME):** +```java +// ISSUE: Should be named ServiceKind, not ServiceCategory +// ServiceCategory is a complexType in XSD, not the enum +public enum ServiceCategory { + ELECTRICITY(0L), + GAS(1L), + WATER(2L), + TIME(3L), + HEAT(4L), + REFUSE(5L), + SEWERAGE(6L), + RATES(7L), + TV_LICENSE(8L), + INTERNET(9L); + // ... fromValue() and getValue() methods +} +``` + +**Corrected Implementation (ServiceKind.java):** +```java +package org.greenbuttonalliance.espi.common.domain.common; + +/** + * ServiceKind enumeration from ESPI 4.0 specification. + * Defines the kind of service represented by a UsagePoint. + * Defined in both espi.xsd and customer.xsd as a shared type. + */ +public enum ServiceKind { + ELECTRICITY(0), + GAS(1), + WATER(2), + TIME(3), + HEAT(4), + REFUSE(5), + SEWERAGE(6), + RATES(7), + TV_LICENSE(8), + INTERNET(9); + + private final int value; + + ServiceKind(int value) { + this.value = value; + } + + public int getValue() { + return value; + } + + public static ServiceKind fromValue(int value) { + for (ServiceKind kind : ServiceKind.values()) { + if (kind.value == value) { + return kind; + } + } + throw new IllegalArgumentException("Invalid ServiceKind value: " + value); + } +} +``` + +### Acceptance Criteria for Phase 0 + +- [ ] `domain/usage/enums/` directory created +- [ ] All 30 missing enumerations implemented +- [ ] All 7 mislocated enums moved to correct directories +- [ ] ServiceCategory renamed to ServiceKind +- [ ] All enum values match XSD definitions +- [ ] All enums include `fromValue()` method +- [ ] All enums properly annotated with Javadoc referencing XSD +- [ ] Update all entity references to use correct enum names/locations +- [ ] Update imports in all affected entities +- [ ] All tests updated and passing +- [ ] Flyway migration scripts updated if needed (enum storage strategy) + +--- + +## Phase 1: Foundation - Supporting Classes (Object Hierarchy) + +These classes form the foundation used by all entities. Verify these first to ensure consistent base types after Phase 0 enumeration migration is complete. + +### Common Domain Supporting Classes + +| Task ID | Class | Type | Location | XSD Source | +|---------|-------|------|----------|------------| +| 1.1 | `RationalNumber` | @Embeddable | domain/common | espi.xsd | +| 1.2 | `DateTimeInterval` | @Embeddable | domain/common | espi.xsd | +| 1.3 | `SummaryMeasurement` | @Embeddable | domain/common | espi.xsd | +| 1.4 | `LinkType` | @Embeddable | domain/common | espi.xsd | +| 1.5 | `ReadingInterharmonic` | @Embeddable | domain/common | espi.xsd | +| 1.6 | `BillingChargeSource` | @Embeddable | domain/common | espi.xsd | +| 1.7 | `ServiceKind` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.8 | `UnitMultiplierKind` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.9 | `UnitSymbolKind` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.10 | `Currency` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.11 | `StatusCode` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.12 | `CRUDOperation` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.13 | `DstRuleType` | Enum | domain/common (shared) | espi.xsd, customer.xsd | +| 1.14 | `EnrollmentStatus` | Enum | domain/common (shared) | espi.xsd, customer.xsd | + +**Note**: Usage domain enums (PhaseCodeKind, AmiBillingReadyKind, etc.) are verified in Phase 0 and relocated to `usage/enums/` directory. + +### Customer Domain Supporting Classes + +| Task ID | Class | Type | Location | XSD Source | +|---------|-------|------|----------|------------| +| 1.15 | `TelephoneNumber` | @Embeddable | customer/common | customer.xsd | +| 1.16 | `ElectronicAddress` | @Embeddable | customer/common | customer.xsd | +| 1.17 | `StreetAddress` | @Embeddable | customer/common | customer.xsd | +| 1.18 | `MeterMultiplier` | @Embeddable | customer/common | customer.xsd | +| 1.19 | `ProgramDateIdMapping` | @Embeddable | customer/common | customer.xsd | +| 1.20 | `CustomerKind` | Enum | customer/enums | customer.xsd | +| 1.21 | `MeterMultiplierKind` | Enum | customer/enums | customer.xsd | +| 1.22 | `NotificationMethodKind` | Enum | customer/enums | customer.xsd | +| 1.23 | `SupplierKind` | Enum | customer/enums | customer.xsd | +| 1.24 | `ProgramDateKind` | Enum | customer/enums | customer.xsd | +| 1.25 | `MediaType` | Enum | customer/enums | customer.xsd | +| 1.26 | `RevenueKind` | Enum | customer/enums | customer.xsd | + +### Customer Entity Base Classes + +| Task ID | Class | Type | Location | XSD Source | +|---------|-------|------|----------|------------| +| 1.27 | `Asset` | Base Class | customer/entity | customer.xsd | +| 1.28 | `AssetContainer` | Base Class | customer/entity | customer.xsd | +| 1.29 | `Agreement` | Base Class | customer/entity | customer.xsd | +| 1.30 | `Organisation` | Base Class | customer/entity | customer.xsd | +| 1.31 | `Location` | Base Class | customer/entity | customer.xsd | +| 1.32 | `Document` | Base Class | customer/entity | customer.xsd | +| 1.33 | `Status` | @Embeddable | customer/entity | customer.xsd | +| 1.34 | `WorkLocation` | @Embeddable | customer/entity | customer.xsd | +| 1.35 | `AccountNotification` | @Embeddable | customer/entity | customer.xsd | +| 1.36 | `EndDeviceFields` | @Embeddable | customer/entity | customer.xsd | + +**Phase 1 Total Tasks**: 36 (after Phase 0 enumeration migration) + +--- + +## Phase 2: Usage Domain Entities (espi.xsd) + +These entities represent energy usage data from the `espi.xsd` schema. All extend `IdentifiedObject`. + +### Core Usage Resources + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 2.1 | `UsagePointEntity` | UsagePoint | High | ServiceCategory, DateTimeInterval | +| 2.2 | `MeterReadingEntity` | MeterReading | High | DateTimeInterval | +| 2.3 | `IntervalBlockEntity` | IntervalBlock | High | DateTimeInterval, IntervalReadingEntity | +| 2.4 | `ReadingTypeEntity` | ReadingType | High | RationalNumber, DateTimeInterval | +| 2.5 | `IntervalReadingEntity` | IntervalReading | High | DateTimeInterval, ReadingQualityEntity | +| 2.6 | `ReadingQualityEntity` | ReadingQuality | Medium | - | + +### Summary and Quality Resources + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 2.7 | `ElectricPowerQualitySummaryEntity` | ElectricPowerQualitySummary | Medium | SummaryMeasurement, DateTimeInterval | +| 2.8 | `UsageSummaryEntity` | UsageSummary | Medium | SummaryMeasurement, DateTimeInterval | + +### Configuration Resources + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 2.9 | `TimeConfigurationEntity` | LocalTimeParameters | High | DateTimeInterval | + +### Customer and Application Resources + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 2.10 | `RetailCustomerEntity` | RetailCustomer | High | - | +| 2.11 | `ApplicationInformationEntity` | ApplicationInformation | High | - | +| 2.12 | `AuthorizationEntity` | Authorization | High | - | +| 2.13 | `SubscriptionEntity` | Subscription | Medium | - | + +### Batch and Line Item Resources + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 2.14 | `BatchListEntity` | BatchList | Low | - | +| 2.15 | `LineItemEntity` | LineItem | Medium | DateTimeInterval | + +### Reference Resources + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 2.16 | `AggregatedNodeRefEntity` | AggregatedNodeRef | Low | - | +| 2.17 | `PnodeRefEntity` | PnodeRef | Low | - | +| 2.18 | `ServiceDeliveryPointEntity` | ServiceDeliveryPoint | Low | - | +| 2.19 | `TariffRiderRefEntity` | TariffRiderRef | Low | - | + +**Phase 2 Total Tasks**: 19 + +--- + +## Phase 3: Customer Domain Entities (customer.xsd) + +These entities represent customer and service provider data from the `customer.xsd` schema. All extend `IdentifiedObject`. + +### Customer Account and Agreement + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 3.1 | `CustomerAccountEntity` | CustomerAccount | High | TelephoneNumber, ElectronicAddress, StreetAddress | +| 3.2 | `CustomerEntity` | Customer | High | CustomerKind, TelephoneNumber, ElectronicAddress | +| 3.3 | `CustomerAgreementEntity` | CustomerAgreement | High | Agreement | + +### Service Supplier and Location + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 3.4 | `ServiceSupplierEntity` | ServiceSupplier | High | SupplierKind, Organisation | +| 3.5 | `ServiceLocationEntity` | ServiceLocation | High | Location, StreetAddress | + +### Meter and Device + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 3.6 | `MeterEntity` | Meter | High | MeterMultiplier, EndDeviceFields | +| 3.7 | `EndDeviceEntity` | EndDevice | High | EndDeviceFields, Asset | + +### Statement and Program Mappings + +| Task ID | Entity | XSD Element | Priority | Dependencies | +|---------|--------|-------------|----------|--------------| +| 3.8 | `StatementEntity` | Statement | Medium | Document, Status | +| 3.9 | `StatementRefEntity` | StatementRef | Low | - | +| 3.10 | `ProgramDateIdMappingsEntity` | ProgramDateIdMappings | Medium | ProgramDateIdMapping | + +**Phase 3 Total Tasks**: 10 + +--- + +## Verification Task Template + +For each entity, follow this repeatable checklist: + +### Step 1: Locate XSD Definition +```bash +# Search for the element definition in XSD +grep -A 50 "name=\"EntityName\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +# OR +grep -A 50 "name=\"EntityName\"" openespi-common/src/main/resources/schema/ESPI_4.0/customer.xsd +``` + +### Step 2: Open Entity Source +```bash +# View the entity implementation +cat openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/{usage|customer}/EntityName.java +``` + +### Step 3: Field-by-Field Verification + +Create a comparison table for each entity using the BasicTypes reference: + +| Field Name | XSD Element Type | BasicType Restriction | minOccurs | Java Type | @Column(length) | @Column(nullable) | Status | Notes | +|------------|------------------|----------------------|-----------|-----------|-----------------|-------------------|--------|-------| +| clientSecret | String512 | maxLength=512 | 1 | String | 512 | false | ✓ | Matches | +| clientName | String256 | maxLength=256 | 1 | String | 256 | false | ✓ | Matches | +| scope | String256 | maxLength=256 | 0 | String | 256 | true | ✓ | Optional field | +| amiBillingReady | AmiBillingReadyKind | enum (0-7) | 0 | AmiBillingReadyKind | n/a | true | ✓ | Enum type | +| duration | UInt32 | 0 to 4,294,967,295 | 0 | Long | n/a | true | ✓ | Numeric type | +| published | TimeType | Unix timestamp | 0 | Long | n/a | true | ✓ | ESPI timestamp | + +**Example Verification Process:** + +1. **XSD Element**: `` +2. **BasicType Lookup**: `String512` → `` +3. **Java Field**: `@Column(name="client_secret", length=512) private String clientSecret;` +4. **Result**: ✓ Type matches (String), length matches (512), nullability correct + +### Step 4: Check for Common Issues + +- [ ] **String field length**: All `String` fields have explicit `@Column(length=...)` matching XSD `maxLength` +- [ ] **Numeric precision**: `Long`, `Integer`, `BigInteger`, `BigDecimal` correctly map XSD numeric types +- [ ] **Temporal types**: Date/time fields use `Long` (UNIX timestamps) or `Instant` appropriately +- [ ] **Enum mappings**: All enum fields reference correct enum types from domain model +- [ ] **Embedded collections**: `@ElementCollection` properly configured with `@CollectionTable` +- [ ] **Relationships**: `@OneToMany`, `@ManyToOne`, `@ManyToMany` match XSD sequences +- [ ] **Nullable constraints**: `nullable=false` used when `minOccurs="1"` in XSD + +### Step 5: Flyway Migration Verification +```bash +# Check database column definitions match entity +grep -A 10 "CREATE TABLE entity_name" openespi-common/src/main/resources/db/migration/*.sql +``` + +Verify: +- Column types match JPA field types +- VARCHAR lengths match `@Column(length=...)` +- NOT NULL constraints match `nullable=false` +- Foreign key relationships exist for `@JoinColumn` fields + +### Step 6: Run Tests +```bash +# Test the specific entity +mvn test -Dtest=EntityNameTest + +# Test repository +mvn test -Dtest=EntityNameRepositoryTest + +# Test full module +cd openespi-common && mvn test +``` + +### Step 7: Document Results + +Create a finding document for each entity: + +```markdown +## EntityName Verification Report + +**XSD Reference**: espi.xsd line XXX (or customer.xsd line YYY) +**Entity Location**: openespi-common/src/main/java/.../EntityName.java +**Verification Date**: YYYY-MM-DD + +### Findings + +#### ✓ Correct Mappings +- Field `abc`: Type matches, length correct +- Field `xyz`: Properly nullable + +#### ⚠️ Discrepancies Found +1. **Field `clientName`** + - XSD Element Type: `String256` + - BasicType Restriction: `maxLength="256"` + - Entity: `@Column(length=512)` + - **Action**: Update to length=256 + +2. **Field `clientSecret`** + - XSD: `type="String512"` with `minOccurs="1"` + - BasicType Restriction: `maxLength="512"` + - Entity: `@Column(length=512, nullable=true)` + - **Action**: Change to nullable=false (minOccurs=1 requires NOT NULL) + +3. **Field `amiBillingReady`** + - XSD Element Type: `AmiBillingReadyKind` (enumeration) + - Entity: `@Column(name="ami_billing_ready") private String amiBillingReady;` + - **Action**: Change to enum type `private AmiBillingReadyKind amiBillingReady;` + +#### 📝 Justifications +- Field `customExtension`: Not in XSD but needed for internal processing +- Field `legacyId`: Required for migration compatibility + +### Flyway Migration Required +- [ ] Update column length for `description` +- [ ] Add NOT NULL constraint for `status` + +### Test Results +- Unit tests: ✓ PASS (15/15) +- Integration tests: ✓ PASS (8/8) +- Repository tests: ✓ PASS (12/12) +``` + +--- + +## Implementation Strategy + +### Recommended Order of Execution + +1. **Phase 1 (Foundation)**: Complete all 34 supporting class verifications first + - Start with embeddables (Tasks 1.1-1.6, 1.15-1.19) + - Then enums (Tasks 1.7-1.14, 1.20-1.24) + - Finally base classes (Tasks 1.25-1.34) + +2. **Phase 2 (Usage Domain)**: Process by dependency order + - **Week 1**: Core resources (Tasks 2.1-2.6) - 6 entities + - **Week 2**: Summaries and config (Tasks 2.7-2.9) - 3 entities + - **Week 3**: Customer/app resources (Tasks 2.10-2.13) - 4 entities + - **Week 4**: Batch and refs (Tasks 2.14-2.19) - 6 entities + +3. **Phase 3 (Customer Domain)**: Process by logical grouping + - **Week 5**: Account/Agreement (Tasks 3.1-3.3) - 3 entities + - **Week 6**: Supplier/Location (Tasks 3.4-3.5) - 2 entities + - **Week 7**: Meter/Device (Tasks 3.6-3.7) - 2 entities + - **Week 8**: Statement/Mappings (Tasks 3.8-3.10) - 3 entities + +### Parallel Execution + +Tasks within the same phase (after Phase 1) can be executed in parallel by different team members: + +- **Developer A**: Usage core resources (2.1-2.6) +- **Developer B**: Usage summaries and config (2.7-2.9) +- **Developer C**: Customer account entities (3.1-3.3) + +### Tools and Scripts + +Create helper scripts to automate verification: + +```bash +# scripts/verify-entity.sh +#!/bin/bash +ENTITY_NAME=$1 +XSD_FILE=$2 + +echo "=== Verifying $ENTITY_NAME against $XSD_FILE ===" +echo "" +echo "XSD Definition:" +grep -A 50 "name=\"$ENTITY_NAME\"" openespi-common/src/main/resources/schema/ESPI_4.0/$XSD_FILE +echo "" +echo "Entity Source:" +find openespi-common/src/main/java -name "${ENTITY_NAME}.java" -exec cat {} \; +``` + +--- + +## Acceptance Criteria Summary + +**For Each Entity:** +- [ ] XSD definition extracted and documented +- [ ] All field types verified against XSD +- [ ] All string lengths match XSD maxLength +- [ ] All nullability constraints match XSD minOccurs +- [ ] All relationships (collections, embedded objects) verified +- [ ] Flyway migration scripts reviewed and aligned +- [ ] Any discrepancies documented with justification or corrected +- [ ] Unit tests pass after corrections +- [ ] Integration tests pass after corrections +- [ ] Verification report created + +**For the Complete Project:** +- [ ] All 102 tasks completed (37 + 36 + 19 + 10) + - Phase 0: 37 enumeration migration tasks + - Phase 1: 36 supporting class tasks + - Phase 2: 19 usage domain entity tasks + - Phase 3: 10 customer domain entity tasks +- [ ] Master discrepancy report compiled +- [ ] Database migration scripts updated if needed +- [ ] All tests passing (`mvn clean test`) +- [ ] Documentation updated with any justified deviations +- [ ] Code review completed +- [ ] Pull request merged + +--- + +## Success Metrics + +- **100% Enumeration Migration**: All 37 XSD enumerations implemented and correctly located +- **100% Coverage**: All 102 entities/classes/enums verified +- **Schema Compliance**: All field types, lengths, and constraints match XSD +- **Test Stability**: 100% test pass rate maintained +- **Documentation**: Complete verification reports for each entity +- **Migration Alignment**: Database schema matches entity definitions +- **Directory Structure**: Proper separation of usage/enums, customer/enums, and common enums + +--- + +## Risk Mitigation + +### Potential Risks + +1. **XSD Interpretation**: Complex XSD types may have multiple valid Java representations + - **Mitigation**: Document interpretation decisions; consult ESPI specification + +2. **Breaking Changes**: Corrections may break existing data or APIs + - **Mitigation**: Test thoroughly; provide migration scripts; version API if needed + +3. **Performance Impact**: Adding constraints may affect query performance + - **Mitigation**: Benchmark queries before/after; add indexes as needed + +4. **Legacy Data**: Existing data may violate new constraints + - **Mitigation**: Data migration scripts; validate existing data first + +### Rollback Plan + +- Keep separate feature branch for all changes +- Tag database before Flyway migrations +- Maintain detailed change log for easy reversion +- Run full integration test suite before merge + +--- + +## Deliverables + +1. **Enumeration Migration**: Complete migration of 37 XSD enumerations to correct directories +2. **Directory Structure**: New `domain/usage/enums/` directory with all usage domain enums +3. **Verification Reports**: Individual report for each of 102 entities/classes/enums +4. **Master Summary Report**: Consolidated findings across all entities and enumerations +5. **Flyway Migration Scripts**: Updated database schemas (if required) +6. **Test Suite**: All unit and integration tests passing +7. **Documentation**: Updated CLAUDE.md and entity documentation +8. **Pull Request**: Complete with detailed description and linked issue + +--- + +## Appendix A: Complete Verification Example Using BasicTypes + +This section provides a step-by-step walkthrough of verifying `ApplicationInformationEntity` against the `espi.xsd` schema using BasicTypes. + +### Example Entity: ApplicationInformationEntity + +**Entity Location**: `openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/ApplicationInformationEntity.java` + +**XSD Location**: `openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd` + +### Step 1: Extract XSD ComplexType Definition + +```bash +grep -A 200 "" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd | head -80 +``` + +**XSD Definition (excerpt):** +```xml + + + + + Data Custodian's Identifier + + + + + + Status of application registration + + + + + + Description of the third party application + + + + + + OAuth2 client secret + + + + + + OAuth2 client identifier + + + + + + Timestamp when client_id was issued + + + + + + Human-readable client name + + + + + + OAuth2 redirect URIs + + + + + + OAuth2 grant types + + + + + + OAuth2 scopes + + + + +``` + +### Step 2: Resolve BasicTypes + +For each field, look up the BasicType definition: + +**Example 1: String256** +```bash +grep -B 2 -A 5 "simpleType name=\"String256\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` + +```xml + + + Character string of max length 256 + + + + + +``` + +**Example 2: String512** +```bash +grep -B 2 -A 5 "simpleType name=\"String512\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` + +```xml + + + [extension] Character string of max length 512 + + + + + +``` + +**Example 3: TimeType** +```bash +grep -B 2 -A 5 "simpleType name=\"TimeType\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` + +```xml + + + Time as UNIX timestamp (seconds since epoch) + + + +``` + +**Example 4: GrantType (Enumeration)** +```bash +grep -B 2 -A 15 "simpleType name=\"GrantType\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` + +```xml + + + OAuth2 grant type enumeration + + + + + + + +``` + +### Step 3: Create Field Mapping Table + +| Field Name | XSD Element Type | BasicType Definition | minOccurs | maxOccurs | Java Type Expected | @Column Expected | +|------------|------------------|---------------------|-----------|-----------|-------------------|------------------| +| dataCustodianId | String256 | maxLength=256 | 0 | 1 | String | length=256, nullable=true | +| dataCustodianApplicationStatus | DataCustodianApplicationStatus | enum | 0 | 1 | DataCustodianApplicationStatus (enum) | nullable=true | +| thirdPartyApplicationDescription | String256 | maxLength=256 | 0 | 1 | String | length=256, nullable=true | +| client_secret | String512 | maxLength=512 | 1 | 1 | String | length=512, nullable=false | +| client_id | String256 | maxLength=256 | 1 | 1 | String | length=256, nullable=false | +| client_id_issued_at | TimeType | xs:long | 0 | 1 | Long | nullable=true | +| client_name | String256 | maxLength=256 | 1 | 1 | String | length=256, nullable=false | +| redirect_uri | xs:anyURI | xs:anyURI | 0 | unbounded | Set or List | @ElementCollection | +| grant_types | GrantType | enum | 1 | unbounded | Set | @ElementCollection, @Enumerated | +| scope | String256 | maxLength=256 | 1 | unbounded | Set | @ElementCollection | + +### Step 4: Review Current Entity Implementation + +```bash +cat openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/ApplicationInformationEntity.java +``` + +**Hypothetical Current Implementation:** +```java +@Entity +@Table(name = "application_information") +public class ApplicationInformationEntity extends IdentifiedObject { + + @Column(name = "data_custodian_id", length = 256) + private String dataCustodianId; + + @Enumerated(EnumType.STRING) + @Column(name = "data_custodian_application_status") + private DataCustodianApplicationStatus dataCustodianApplicationStatus; + + @Column(name = "third_party_application_description", length = 512) // ❌ WRONG LENGTH + private String thirdPartyApplicationDescription; + + @Column(name = "client_secret", length = 512, nullable = false) + private String clientSecret; + + @Column(name = "client_id", length = 256, nullable = false) + private String clientId; + + @Column(name = "client_id_issued_at") // ✓ Correct + private Long clientIdIssuedAt; + + @Column(name = "client_name", length = 256) // ❌ MISSING nullable=false + private String clientName; + + @ElementCollection + @CollectionTable(name = "application_redirect_uris", + joinColumns = @JoinColumn(name = "application_information_id")) + @Column(name = "redirect_uri", length = 512) + private Set redirectUris; + + @ElementCollection + @CollectionTable(name = "application_grant_types", + joinColumns = @JoinColumn(name = "application_information_id")) + @Enumerated(EnumType.STRING) + @Column(name = "grant_type") + private Set grantTypes; + + @ElementCollection + @CollectionTable(name = "application_scopes", + joinColumns = @JoinColumn(name = "application_information_id")) + @Column(name = "scope", length = 256) + private Set scopes; +} +``` + +### Step 5: Document Discrepancies + +**Verification Results:** + +| Field | Status | Issue | Required Action | +|-------|--------|-------|-----------------| +| dataCustodianId | ✓ | - | None | +| dataCustodianApplicationStatus | ✓ | - | None | +| thirdPartyApplicationDescription | ❌ | Length is 512, should be 256 | Change `length=256` | +| clientSecret | ✓ | - | None | +| clientId | ✓ | - | None | +| clientIdIssuedAt | ✓ | - | None | +| clientName | ❌ | Missing `nullable=false` | Add `nullable=false` (minOccurs=1) | +| redirectUris | ✓ | - | None | +| grantTypes | ⚠️ | Should validate minOccurs=1 | Consider @NotEmpty validation | +| scopes | ⚠️ | Should validate minOccurs=1 | Consider @NotEmpty validation | + +### Step 6: Create Corrected Implementation + +```java +@Entity +@Table(name = "application_information") +public class ApplicationInformationEntity extends IdentifiedObject { + + @Column(name = "data_custodian_id", length = 256) + private String dataCustodianId; + + @Enumerated(EnumType.STRING) + @Column(name = "data_custodian_application_status") + private DataCustodianApplicationStatus dataCustodianApplicationStatus; + + // FIXED: Changed length from 512 to 256 to match String256 BasicType + @Column(name = "third_party_application_description", length = 256) + private String thirdPartyApplicationDescription; + + @Column(name = "client_secret", length = 512, nullable = false) + private String clientSecret; + + @Column(name = "client_id", length = 256, nullable = false) + private String clientId; + + @Column(name = "client_id_issued_at") + private Long clientIdIssuedAt; + + // FIXED: Added nullable=false because minOccurs=1 in XSD + @Column(name = "client_name", length = 256, nullable = false) + private String clientName; + + @ElementCollection + @CollectionTable(name = "application_redirect_uris", + joinColumns = @JoinColumn(name = "application_information_id")) + @Column(name = "redirect_uri", length = 512) + private Set redirectUris; + + @ElementCollection + @CollectionTable(name = "application_grant_types", + joinColumns = @JoinColumn(name = "application_information_id")) + @Enumerated(EnumType.STRING) + @Column(name = "grant_type") + @NotEmpty // Added validation for minOccurs=1, maxOccurs=unbounded + private Set grantTypes; + + @ElementCollection + @CollectionTable(name = "application_scopes", + joinColumns = @JoinColumn(name = "application_information_id")) + @Column(name = "scope", length = 256) + @NotEmpty // Added validation for minOccurs=1, maxOccurs=unbounded + private Set scopes; +} +``` + +### Step 7: Update Flyway Migration + +**Check existing migration:** +```bash +grep -A 30 "CREATE TABLE application_information" openespi-common/src/main/resources/db/migration/*.sql +``` + +**Required Migration Script** (if schema exists): +```sql +-- V5__Fix_ApplicationInformation_Schema_Compliance.sql + +-- Fix thirdPartyApplicationDescription length (512 → 256) +ALTER TABLE application_information +MODIFY COLUMN third_party_application_description VARCHAR(256); + +-- Add NOT NULL constraint to client_name (minOccurs=1) +ALTER TABLE application_information +MODIFY COLUMN client_name VARCHAR(256) NOT NULL; +``` + +### Step 8: Validate with Tests + +```bash +# Run entity tests +mvn test -Dtest=ApplicationInformationEntityTest + +# Run repository tests +mvn test -Dtest=ApplicationInformationRepositoryTest + +# Run full module tests +cd openespi-common && mvn test +``` + +### Step 9: Create Verification Report + +```markdown +# ApplicationInformationEntity Verification Report + +**Entity**: ApplicationInformationEntity +**XSD Reference**: espi.xsd, lines 53-249 (ApplicationInformation complexType) +**Verification Date**: 2026-02-03 +**Verified By**: Development Team + +## XSD BasicTypes Resolved + +| Field | XSD Type | BasicType Definition | Resolved Value | +|-------|----------|---------------------|----------------| +| dataCustodianId | String256 | maxLength restriction | 256 | +| thirdPartyApplicationDescription | String256 | maxLength restriction | 256 | +| client_secret | String512 | maxLength restriction | 512 | +| client_id | String256 | maxLength restriction | 256 | +| client_id_issued_at | TimeType | xs:long | Long (UNIX timestamp) | +| client_name | String256 | maxLength restriction | 256 | +| grant_types | GrantType | enum restriction | String enum values | +| scope | String256 | maxLength restriction | 256 | + +## Discrepancies Found and Corrected + +### 1. thirdPartyApplicationDescription Length Mismatch +- **XSD**: `type="String256"` → maxLength=256 +- **Entity (before)**: `@Column(length=512)` +- **Entity (after)**: `@Column(length=256)` +- **Migration**: Required - ALTER COLUMN to VARCHAR(256) + +### 2. clientName Nullability Mismatch +- **XSD**: `minOccurs="1"` (required field) +- **Entity (before)**: `@Column(nullable=true)` (implicit) +- **Entity (after)**: `@Column(nullable=false)` +- **Migration**: Required - ADD NOT NULL constraint + +### 3. Collection Validation +- **XSD**: grantTypes and scopes have `minOccurs="1"` (at least one required) +- **Entity (before)**: No validation +- **Entity (after)**: Added `@NotEmpty` annotation +- **Migration**: Not required (validation only) + +## Test Results + +- ✅ ApplicationInformationEntityTest: 12/12 passed +- ✅ ApplicationInformationRepositoryTest: 8/8 passed +- ✅ Integration tests: 5/5 passed + +## Migration Scripts + +- ✅ Created: V5__Fix_ApplicationInformation_Schema_Compliance.sql +- ✅ Tested on dev database +- ✅ Ready for deployment + +## Conclusion + +ApplicationInformationEntity is now fully compliant with ESPI 4.0 espi.xsd schema definition. All BasicType references have been correctly interpreted and mapped to appropriate Java types and JPA annotations. +``` + +--- + +## Appendix B: Quick Reference Commands + +### Extract All BasicType Definitions + +```bash +# List all String BasicTypes from espi.xsd +grep -E "simpleType name=\"String[0-9]+\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd + +# List all numeric BasicTypes +grep -E "simpleType name=\"(U?Int|UInt)[0-9]+\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd + +# List all enum BasicTypes +grep "simpleType name=" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd | grep -v "String\|Int\|HexBinary\|Time\|UUID" +``` + +### Find Element's BasicType + +```bash +# Find what type an element uses +grep -i "element name=\"clientSecret\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd + +# Find the BasicType definition +TYPE="String512" +grep -B 2 -A 5 "simpleType name=\"$TYPE\"" openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +``` + +### Verify Entity Field Mappings + +```bash +# Find all @Column annotations in an entity +grep "@Column" openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/ApplicationInformationEntity.java + +# Find all String fields with their lengths +grep -B 1 "private String" openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/ApplicationInformationEntity.java | grep -E "@Column|private String" +``` + +### Compare Multiple Entities + +```bash +# Find all entities with String256 fields that might be wrong +for file in openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/*.java; do + echo "=== $(basename $file) ===" + grep -n "@Column.*length.*256" "$file" +done +``` + +--- + +## Conclusion + +This systematic approach ensures complete ESPI 4.0 schema compliance across all domain entities, enumerations, and supporting classes. The plan addresses a critical gap in the current implementation: 25 missing enumerations and 8 enumerations in incorrect locations. + +### Implementation Phases Summary + +1. **Phase 0 (Enumerations)**: Migrate 37 XSD enumerations, create proper directory structure +2. **Phase 1 (Foundation)**: Verify 36 supporting classes and shared enumerations +3. **Phase 2 (Usage Domain)**: Verify 19 usage domain entities +4. **Phase 3 (Customer Domain)**: Verify 10 customer domain entities + +By following the repeatable task template and verification checklist, the team can efficiently validate 102 components with confidence in correctness and consistency. + +### Key Improvements + +- ✅ All XSD enumerations migrated from legacy code +- ✅ Proper directory structure: `usage/enums/`, `customer/enums/`, `common/` +- ✅ Complete BasicTypes interpretation and mapping +- ✅ ServiceKind naming corrected (was incorrectly named ServiceCategory) +- ✅ Comprehensive verification reports for each component + +**Total Effort Estimate**: 10-12 weeks with 2-3 developers working in parallel +- Phase 0: 2-3 weeks (enumeration migration is foundational) +- Phases 1-3: 8-9 weeks (entity and type verification) + +**Priority**: High - Missing enumerations and schema compliance are blocking full ESPI 4.0 certification + +**Related Issues**: #28 (Phase 17 pattern), Spring Boot 4.0 migration \ No newline at end of file diff --git a/ISSUE_101_IMPLEMENTATION_STRATEGY.md b/ISSUE_101_IMPLEMENTATION_STRATEGY.md new file mode 100644 index 00000000..e4bfbbec --- /dev/null +++ b/ISSUE_101_IMPLEMENTATION_STRATEGY.md @@ -0,0 +1,758 @@ +# Issue #101 Implementation Strategy +## Minimizing Team Impact During Schema Compliance Verification + +**Goal**: Complete ESPI 4.0 schema compliance verification while minimizing disruption to active development on the main branch. + +--- + +## Strategy Overview + +### The Challenge + +- **102 total tasks** across 4 phases +- **37 enumeration tasks** in Phase 0 that touch many files +- **File relocations** create merge conflict risks +- **Active team** continuing feature development +- **Long-lived branches** = integration hell + +### The Solution: Incremental Delivery with Parallel Change Pattern + +Use a **"Expand-Migrate-Contract"** approach with small, frequent merges to main. + +--- + +## Recommended Approach: Phased Incremental Delivery + +### Core Principles + +1. **Small, frequent merges** - No branch lives longer than 3-5 days +2. **Backward compatibility** - Old code continues working during transition +3. **Feature flags** - Enable/disable new code paths +4. **Parallel changes** - Support both old and new simultaneously +5. **Clear ownership** - Explicit assignments to avoid conflicts +6. **Communication** - Daily standups on refactoring progress + +--- + +## Phase 0: Enumeration Migration (Highest Risk) + +This phase has the highest conflict potential due to file moves and renames. + +### Option A: Incremental Parallel Change (RECOMMENDED) + +**Timeline**: 3-4 weeks (slower but safer) + +#### Week 1: Create New Structure (Expand) + +**Branch**: `feature/enum-structure-setup` +**Duration**: 2-3 days +**Conflicts**: Minimal - only adding files + +``` +Tasks: +1. Create domain/usage/enums/ directory +2. Implement NEW enums only (don't move existing yet) + - Priority: High-priority missing enums needed by entities + - AccumulationKind, CommodityKind, DataQualifierKind, etc. +3. Add ServiceKind (NEW) alongside ServiceCategory (OLD) +4. Keep both versions working +``` + +**Merge to main**: After tests pass + +#### Week 2: Implement Remaining New Enums + +**Branch**: `feature/enum-implementations-batch-1` +**Duration**: 3-5 days +**Conflicts**: Minimal - only adding files + +``` +Tasks: +1. Implement remaining missing enums in correct locations +2. All new enums, no moves yet +3. Add comprehensive tests +``` + +**Merge to main**: After tests pass + +#### Week 3: Parallel Migration (Migrate) + +**Branch**: `feature/enum-usage-migration` +**Duration**: 3-5 days +**Conflicts**: Medium - changing entity imports + +``` +Tasks: +1. Update NEW entities to use new enum locations +2. Keep OLD entities using old locations (backward compatible) +3. Add deprecation warnings to old enum locations +4. No file deletions or moves yet +``` + +**Merge to main**: After tests pass, coordinate with team + +#### Week 4: Final Cleanup (Contract) + +**Branch**: `feature/enum-cleanup` +**Duration**: 2-3 days +**Conflicts**: High - deleting/moving files + +**Prerequisites**: +- Coordinate with team - announce "enum migration freeze day" +- All team members merge their current work +- No new features touching enums for 24-48 hours + +``` +Tasks: +1. Move old enums to new locations +2. Update all remaining references +3. Delete deprecated locations +4. Remove ServiceCategory, keep only ServiceKind +``` + +**Merge to main**: Coordinated merge, immediate announcement + +**Advantages:** +- ✅ Low conflict risk - most work is additive +- ✅ Backward compatible during migration +- ✅ Team can continue working with minimal disruption +- ✅ Easy to test incrementally +- ✅ Can abandon/rollback partially if needed + +**Disadvantages:** +- ⏱️ Takes longer (3-4 weeks vs 1-2 weeks) +- 🔄 Temporary duplication of code + +--- + +### Option B: Feature Branch with Frequent Rebasing (Higher Risk) + +**Timeline**: 1-2 weeks (faster but riskier) + +**Branch**: `feature/issue-101-phase-0-enums` +**Strategy**: Daily rebases from main + +``` +Day 1-2: Create structure + implement new enums +Day 3-4: Move existing enums +Day 5-7: Update all references +Day 8-9: Testing and fixes +Day 10: Coordinated merge to main +``` + +**Required:** +- Daily rebase from main (resolve conflicts daily, not at end) +- Clear team communication: "Major enum refactor in progress" +- Feature freeze for enum-touching code for final 2 days +- Dedicated "integration day" for final merge + +**Advantages:** +- ⚡ Faster completion +- 🎯 Atomic change - everything updated at once + +**Disadvantages:** +- ⚠️ High conflict risk +- 🚫 Requires team coordination/freeze +- 😰 Stressful final merge +- 🐛 Higher bug risk + +--- + +### Option C: Hybrid Approach (BEST BALANCE) + +Combine both strategies: + +**Weeks 1-2: Expand (Additive Changes Only)** +- Create new directories +- Implement all new enums +- Add ServiceKind alongside ServiceCategory +- Merge to main frequently (every 2-3 days) + +**Week 3: Coordinate Migration Window** +- Announce 1 week in advance: "Enum migration week - minimize enum changes" +- Create `feature/enum-migration` branch +- Perform moves and renames in one focused effort +- Daily rebases +- Coordinate final merge + +**Week 4: Buffer/Cleanup** +- Fix any issues from migration +- Update documentation +- Verify all tests passing + +--- + +## Phases 1-3: Supporting Classes and Entities (Lower Risk) + +These phases have lower conflict potential because they modify existing files rather than moving them. + +### Parallel Team Strategy + +**Divide work by domain to minimize conflicts:** + +#### Team Assignment + +**Developer A: Common/Shared Types (Phase 1 - Part 1)** +- Branch: `feature/verify-common-embeddables` +- Tasks: 1.1-1.6 (RationalNumber, DateTimeInterval, etc.) +- Duration: 1 week +- Conflicts: Low + +**Developer B: Customer Supporting Types (Phase 1 - Part 2)** +- Branch: `feature/verify-customer-types` +- Tasks: 1.15-1.36 (Customer embeddables, enums, base classes) +- Duration: 1.5 weeks +- Conflicts: Low (different domain) + +**Developer C: Usage Domain Entities - Core (Phase 2 - Part 1)** +- Branch: `feature/verify-usage-core` +- Tasks: 2.1-2.6 (UsagePoint, MeterReading, IntervalBlock, ReadingType) +- Duration: 1.5 weeks +- Conflicts: Low (different domain) + +**Developer A: Usage Domain Entities - Summaries (Phase 2 - Part 2)** +- Branch: `feature/verify-usage-summaries` +- Tasks: 2.7-2.13 (Summaries, config, customer resources) +- Duration: 1 week +- Conflicts: Low + +**Developer B: Customer Domain Entities (Phase 3)** +- Branch: `feature/verify-customer-entities` +- Tasks: 3.1-3.10 (All customer entities) +- Duration: 1.5 weeks +- Conflicts: Low (different domain) + +**Developer C: Usage Domain Entities - Misc (Phase 2 - Part 3)** +- Branch: `feature/verify-usage-misc` +- Tasks: 2.14-2.19 (Batch, references) +- Duration: 1 week +- Conflicts: Low + +### Merge Cadence for Phases 1-3 + +**Recommended**: Merge every 3-5 entities verified + +``` +Example for Developer A (Common embeddables): +- Day 1-2: Verify RationalNumber, DateTimeInterval +- Day 2: Merge PR #1 to main +- Day 3-4: Verify SummaryMeasurement, LinkType +- Day 4: Merge PR #2 to main +- Day 5-6: Verify ReadingInterharmonic, BillingChargeSource +- Day 6: Merge PR #3 to main +``` + +**Benefits:** +- Small PRs are easier to review +- Conflicts are minimized +- Progress is visible +- Easy to rollback if needed +- Team can continue working on main + +--- + +## Conflict Prevention Strategies + +### 1. Clear Ownership and Communication + +**Daily Standup Topics:** +- "I'm working on UsagePoint entity today" +- "I'll be modifying ReadingType enum mappings" +- "I need to update TimeConfiguration - anyone else touching it?" + +**Use GitHub Issue Assignments:** +``` +Issue #101: Parent tracking issue +├─ Task #101.1: @developer-a - Verify RationalNumber +├─ Task #101.2: @developer-b - Verify CustomerKind enum +├─ Task #101.3: @developer-c - Verify UsagePoint entity +``` + +### 2. Entity-Level Locking + +**Claim entities before starting work:** + +```markdown +# In ISSUE_101_IMPLEMENTATION_PLAN.md, add status column: + +| Task | Entity | Assigned | Status | Branch | +|------|--------|----------|--------|--------| +| 2.1 | UsagePointEntity | @alice | In Progress | feature/verify-usagepoint | +| 2.2 | MeterReadingEntity | @bob | Not Started | - | +| 2.3 | IntervalBlockEntity | - | Available | - | +``` + +### 3. Rebase Daily (Not at End) + +**For longer-lived branches:** + +```bash +# Every morning, before starting work: +git checkout feature/my-verification-branch +git fetch origin +git rebase origin/main + +# Resolve conflicts immediately while context is fresh +# Much easier than resolving 10 days of conflicts at once +``` + +### 4. Communicate Breaking Changes + +**Use team chat/Slack:** + +``` +@team: I'm about to merge PR #145 which renames ServiceCategory to ServiceKind. +If you have any branches touching ServiceCategory, please: +1. Merge your work to main before EOD today, OR +2. Wait until tomorrow to rebase after my merge +``` + +### 5. Use Draft PRs for Visibility + +**Create draft PRs early:** + +``` +Draft PR #145: [WIP] Verify UsagePoint entity compliance + +Shows: +- What you're working on +- What files you're touching +- Prevents duplicate work +- Enables early feedback +``` + +--- + +## Recommended Timeline with Team Coordination + +### Month 1: Phase 0 - Enumeration Migration + +| Week | Focus | Merges | Team Impact | +|------|-------|--------|-------------| +| 1 | Create structure + new enums | 2-3 | Low - additive only | +| 2 | Implement remaining new enums | 2-3 | Low - additive only | +| 3 | **MIGRATION WEEK** - Move/rename | 1 big merge | **Medium** - coordinate | +| 4 | Buffer, testing, fixes | As needed | Low - stabilization | + +**Team Communication:** +- Week 1: "Starting enum migration - new enums being added" +- Week 2: "More enums coming - avoid creating new enum dependencies" +- Week 3: **"MIGRATION WEEK - minimize enum changes, coordinate with @team"** +- Week 4: "Enum migration complete - update your branches" + +### Month 2-3: Phases 1-3 - Parallel Verification + +| Week | Developer A | Developer B | Developer C | Merges/Week | +|------|------------|------------|------------|-------------| +| 5 | Common embeddables | Customer types | Usage core | 6-9 | +| 6 | Common embeddables | Customer types | Usage core | 6-9 | +| 7 | Usage summaries | Customer entities | Usage misc | 6-9 | +| 8 | Usage summaries | Customer entities | Usage misc | 6-9 | + +**Team Impact**: Low - different developers working on different domains + +--- + +## Handling Conflicts When They Occur + +### Conflict Prevention Checklist + +Before starting work on an entity: + +```bash +# 1. Check if anyone else is working on it +gh issue list --label "Issue-101" --assignee "@me" + +# 2. Pull latest main +git checkout main +git pull origin main + +# 3. Create feature branch +git checkout -b feature/verify-usage-point + +# 4. Announce you're starting +# Post in team chat: "Starting UsagePoint verification" +``` + +### During Work + +```bash +# Rebase frequently (daily recommended) +git fetch origin +git rebase origin/main + +# If conflicts occur, resolve immediately +# Don't let them accumulate +``` + +### Before Merging + +```bash +# Final rebase +git fetch origin +git rebase origin/main + +# Run full test suite +mvn clean test + +# Create PR +gh pr create --title "Verify UsagePoint entity schema compliance" \ + --body "Fixes #101 (Task 2.1)" +``` + +--- + +## Migration Windows for High-Risk Changes + +Some changes require coordinated "migration windows" where team agrees to minimize conflicting work. + +### Coordinated Migration Windows + +**Use for:** +- Phase 0 final week (enum moves/renames) +- Renaming ServiceCategory → ServiceKind +- Moving files between packages +- Major refactorings + +**Process:** + +**1 Week Before:** +``` +Announcement: "Enum migration window scheduled for Week 3, Mon-Wed. +Please avoid creating new enum dependencies this week." +``` + +**3 Days Before:** +``` +Reminder: "Enum migration starts Monday. +Merge any enum-touching work by Friday EOD or wait until Thursday." +``` + +**Day Of:** +``` +Active: "Enum migration in progress - avoid enum changes until Wednesday." +``` + +**Completion:** +``` +Complete: "Enum migration merged! Please rebase your branches. +Migration guide: [link to document]" +``` + +--- + +## Testing Strategy During Migration + +### Continuous Integration + +**Every merge must:** +- ✅ Pass all unit tests +- ✅ Pass all integration tests +- ✅ Pass compilation (no broken imports) +- ✅ Meet code coverage thresholds + +### Regression Testing + +**After each phase:** +```bash +# Run full regression suite +mvn clean verify -Pintegration-tests + +# Check for broken references +mvn dependency:analyze + +# Verify no compilation warnings +mvn clean compile -Xlint:all +``` + +### Migration Validation + +**After Phase 0 (enumerations):** +```bash +# Verify all old enum locations deleted +find . -path "*/domain/common/*Kind.java" -type f + +# Verify new locations populated +ls -la openespi-common/src/main/java/.../domain/usage/enums/ +ls -la openespi-common/src/main/java/.../domain/customer/enums/ + +# Grep for old imports (should be zero) +grep -r "domain.common.AmiBillingReadyKind" openespi-common/src/ +``` + +--- + +## Rollback Plan + +### If Things Go Wrong + +**Small merge issues:** +- Fix forward - create hotfix PR +- Don't revert unless critical + +**Major breakage:** + +```bash +# Option 1: Revert the merge (immediate) +git revert +git push origin main + +# Option 2: Fix forward (preferred if possible) +# Create emergency fix branch +git checkout -b hotfix/enum-migration-fix +# Fix the issue +# Create PR with [URGENT] tag +``` + +### Rollback Triggers + +**Revert if:** +- ❌ Tests failing on main for > 2 hours +- ❌ Breaks production/demo environment +- ❌ Blocks multiple developers from working + +**Fix forward if:** +- ⚠️ Minor test failures +- ⚠️ Documentation issues +- ⚠️ Non-critical bugs + +--- + +## Team Workflow Example + +### Scenario: 3 Developers, 8 Weeks + +**Developer A (Alice):** +``` +Week 1-2: Phase 0 support (new enums) +Week 3: Phase 0 migration (coordinate) +Week 4-5: Phase 1 - Common types +Week 6-7: Phase 2 - Usage summaries +Week 8: Buffer/review +``` + +**Developer B (Bob):** +``` +Week 1-2: Phase 0 support (new enums) +Week 3: Phase 0 migration (coordinate) +Week 4-6: Phase 1 - Customer types +Week 7-8: Phase 3 - Customer entities +``` + +**Developer C (Carol):** +``` +Week 1-2: Continue regular feature work +Week 3: Phase 0 migration (testing support) +Week 4-6: Phase 2 - Usage core entities +Week 7-8: Phase 2 - Usage misc entities +``` + +**Team Lead:** +``` +Week 1-2: Review enum PRs, coordinate +Week 3: Manage migration window +Week 4-8: Review verification PRs, track progress +``` + +--- + +## Communication Templates + +### Starting a Task + +**Slack/Teams Message:** +``` +🚀 Starting Task 2.1: Verify UsagePoint entity +📝 Branch: feature/verify-usage-point +⏱️ ETA: 2 days +🔗 Draft PR: #145 + +Let me know if anyone else is touching UsagePoint! +``` + +### Merging a Task + +**Slack/Teams Message:** +``` +✅ Merged: UsagePoint entity verification +🔗 PR #145: https://github.com/.../pull/145 +📊 Status: 15/102 tasks complete +⚠️ Action needed: Rebase if you're working on: + - ServiceCategory references + - UsagePoint relationships +``` + +### Migration Window + +**Email/Announcement:** +``` +Subject: [ACTION REQUIRED] Enum Migration Window - Week 3 + +Team, + +We're performing the enumeration migration (Phase 0) during Week 3 (Feb 10-14). + +ACTIONS REQUIRED: +1. Merge any work touching enums by EOD Friday, Feb 7 +2. Avoid creating new enum dependencies Feb 10-12 +3. Rebase your branches on Feb 13 after migration completes + +WHAT'S HAPPENING: +- Moving enums to domain/usage/enums/ and domain/customer/enums/ +- Renaming ServiceCategory → ServiceKind +- Updating all entity references + +QUESTIONS: Reply to this thread or ping @alice + +Thanks for your cooperation! +``` + +--- + +## Success Metrics + +### During Implementation + +**Track weekly:** +- ✅ Tasks completed vs planned +- ⚠️ Merge conflicts encountered +- 🐛 Bugs introduced +- ⏱️ Time blocked by conflicts +- 📊 Test coverage + +**Target Metrics:** +- < 2 hours/week blocked by conflicts per developer +- < 5% of merges require conflict resolution +- 100% test pass rate maintained +- Zero production incidents + +### After Completion + +**Verify:** +- ✅ All 102 tasks completed +- ✅ All tests passing +- ✅ Zero deprecation warnings +- ✅ Documentation updated +- ✅ Team velocity maintained (not slowed) + +--- + +## Recommended Approach: Summary + +### For This Team + +**RECOMMENDED: Hybrid Approach** + +**Phase 0 (Weeks 1-4):** +- Weeks 1-2: Incremental addition (Option A - Expand) +- Week 3: Coordinated migration window (Option B - Migrate) +- Week 4: Cleanup and stabilization (Option A - Contract) + +**Phases 1-3 (Weeks 5-12):** +- Parallel work by domain +- Small, frequent merges (every 3-5 entities) +- Daily standups for coordination +- Clear ownership assignments + +**Key Success Factors:** +1. ✅ Daily rebases (not weekly) +2. ✅ Small PRs (3-5 entities max) +3. ✅ Clear communication +4. ✅ Coordinated migration windows for risky changes +5. ✅ Entity-level ownership tracking +6. ✅ Continuous testing + +**Result:** +- ✅ Minimal team disruption +- ✅ Continuous delivery to main +- ✅ Low conflict risk +- ✅ Easy rollback if needed +- ✅ Visible progress +- ✅ Maintained team velocity + +--- + +## Tools and Automation + +### GitHub Project Board + +Create project board to track progress: + +``` +Columns: +- Backlog (102 tasks) +- In Progress (assigned, branch created) +- In Review (PR open) +- Done (merged to main) + +Automation: +- Auto-move to "In Progress" when branch created +- Auto-move to "In Review" when PR opened +- Auto-move to "Done" when merged +``` + +### CI/CD Checks + +**Required checks on every PR:** +```yaml +# .github/workflows/issue-101-verification.yml +name: Schema Compliance Verification + +on: [pull_request] + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - name: Run tests + run: mvn clean test + - name: Check for old enum imports + run: | + if grep -r "domain.common.AmiBillingReadyKind" src/; then + echo "Found deprecated enum imports" + exit 1 + fi + - name: Verify enum locations + run: | + # Custom script to verify enums in correct directories + ./scripts/verify-enum-locations.sh +``` + +### Conflict Detection + +**Pre-merge check:** +```bash +# scripts/check-conflicts.sh +#!/bin/bash + +# Check if anyone else has modified same files +BRANCH=$(git rev-parse --abbrev-ref HEAD) +BASE="origin/main" + +CHANGED_FILES=$(git diff --name-only $BASE..$BRANCH) + +for file in $CHANGED_FILES; do + RECENT_AUTHORS=$(git log $BASE..$BRANCH --format="%an" -- "$file" | sort -u) + if [ $(echo "$RECENT_AUTHORS" | wc -l) -gt 1 ]; then + echo "WARNING: Multiple authors modified $file" + echo "$RECENT_AUTHORS" + fi +done +``` + +--- + +## Conclusion + +The **Hybrid Approach** balances speed, safety, and team productivity: + +- **Additive changes first** - Low conflict risk +- **Coordinated migration window** - One-time disruption, well-communicated +- **Parallel verification work** - Maximum efficiency +- **Small, frequent merges** - Continuous integration +- **Clear communication** - No surprises + +**Timeline**: 10-12 weeks with minimal team disruption + +**Result**: ESPI 4.0 schema compliance achieved while maintaining team velocity and morale. diff --git a/ISSUE_101_TOOLING_GUIDE.md b/ISSUE_101_TOOLING_GUIDE.md new file mode 100644 index 00000000..522916a5 --- /dev/null +++ b/ISSUE_101_TOOLING_GUIDE.md @@ -0,0 +1,844 @@ +# Issue #101 Tooling Guide +## Recommended Tools and Claude Models for ESPI 4.0 Schema Compliance + +**For**: Senior Spring Developers working on schema verification and enumeration migration + +--- + +## Claude Model Strategy + +### Model Selection by Task Type + +| Task Type | Recommended Model | Why | Cost | +|-----------|------------------|-----|------| +| **XSD Analysis** | Claude Opus 4.5 | Superior pattern recognition for complex schemas | Higher | +| **Code Generation** | Claude Sonnet 4.5 | Excellent balance of quality and speed | Medium | +| **Repetitive Verification** | Claude Haiku | Fast, cost-effective for similar tasks | Lower | +| **Architecture Decisions** | Claude Opus 4.5 | Best reasoning for trade-offs | Higher | +| **Documentation** | Claude Sonnet 4.5 | Natural language + code understanding | Medium | +| **Code Review** | Claude Sonnet 4.5 | Good at spotting issues | Medium | +| **Batch Processing** | Claude Haiku | Process many files quickly | Lower | + +### Phase-Specific Model Recommendations + +#### Phase 0: Enumeration Migration + +**XSD Enumeration Extraction** → **Claude Opus 4.5** +``` +Task: Analyze espi.xsd and customer.xsd to extract all enumeration definitions +Why: Complex XSD schemas with nested types, requires deep understanding +Example prompt: + "Extract all simpleType enumerations from this XSD, including their: + - Name + - Base type + - Enumeration values with documentation + - Any restrictions or patterns" +``` + +**Enum Code Generation** → **Claude Sonnet 4.5** +``` +Task: Generate Java enum classes from XSD definitions +Why: Repetitive but requires good Java code generation +Example prompt: + "Generate a Java enum for ServiceKind based on this XSD simpleType: + [paste XSD definition] + + Requirements: + - Package: org.greenbuttonalliance.espi.common.domain.usage.enums + - Include getValue() and fromValue() methods + - Add Javadoc from XSD documentation + - Follow project conventions" +``` + +**Batch Verification** → **Claude Haiku** +``` +Task: Verify 30+ enum implementations match XSD +Why: Fast, repetitive comparison task +Example prompt: + "Verify this Java enum matches the XSD definition: + Java: [paste enum code] + XSD: [paste simpleType] + + Check: values, order, types, documentation" +``` + +#### Phase 1-3: Entity Verification + +**Field Type Mapping** → **Claude Sonnet 4.5** +``` +Task: Map XSD complexType fields to JPA entity annotations +Why: Requires understanding both XSD and JPA conventions +Example prompt: + "Map this XSD complexType to JPA entity annotations: + [paste XSD definition] + + Current entity: [paste Java class] + + Verify: field types, lengths, nullability, relationships" +``` + +**Discrepancy Analysis** → **Claude Opus 4.5** +``` +Task: Analyze complex mismatches and recommend fixes +Why: Requires reasoning about impacts and trade-offs +Example prompt: + "I found these discrepancies in UsagePointEntity: + [list discrepancies] + + Analyze: + 1. Which are critical vs minor? + 2. Migration strategy for each? + 3. Potential breaking changes? + 4. Recommended fix order?" +``` + +**Test Generation** → **Claude Sonnet 4.5** +``` +Task: Generate unit tests for verified entities +Why: Good at understanding test patterns and edge cases +Example prompt: + "Generate JUnit 5 tests for UsagePointEntity: + [paste entity] + + Include: + - Field validation tests + - Relationship tests + - Null handling + - Edge cases + - AssertJ assertions" +``` + +--- + +## Claude Code (CLI) Usage Patterns + +### Recommended Workflow + +**For Each Verification Task:** + +```bash +# 1. Start Claude Code in the repository +cd /path/to/OpenESPI-GreenButton-Java +claude + +# 2. Use specialized prompts for different phases +``` + +### Phase 0: Enumeration Migration Prompts + +**Extract Missing Enums:** +``` +Prompt: +"I need to identify all missing enumerations from espi.xsd. + +Please: +1. Read openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd +2. Extract all simpleType definitions with enumeration restrictions +3. Check which ones exist in openespi-common/src/main/java/.../domain +4. Create a list of missing enums with their XSD definitions +5. For each missing enum, provide: + - XSD name + - Base type + - Enumeration values + - Recommended Java package location" +``` + +**Generate Enum Implementation:** +``` +Prompt: +"Generate a Java enum for AccumulationKind: + +1. Read the XSD definition from espi.xsd +2. Create AccumulationKind.java in domain/usage/enums/ +3. Include: + - Proper package declaration + - Enum values from XSD + - getValue() method + - fromValue() method with validation + - Javadoc from XSD annotations +4. Follow the pattern from existing enums like PhaseCodeKind" +``` + +**Batch Migration:** +``` +Prompt: +"I need to move AmiBillingReadyKind from domain/common to domain/usage/enums. + +Please: +1. Read the current implementation +2. Create new file in domain/usage/enums/ +3. Find all usages across the codebase +4. Update all import statements +5. Create a migration checklist +6. Don't delete the old file yet (we'll do that in final cleanup)" +``` + +### Phase 1-3: Entity Verification Prompts + +**Verify Single Entity:** +``` +Prompt: +"Verify UsagePointEntity against the ESPI 4.0 XSD schema: + +1. Read the XSD definition for UsagePoint complexType from espi.xsd +2. Read UsagePointEntity.java +3. Create a field-by-field comparison table: + - Field name + - XSD type → BasicType → restriction + - Java type and @Column annotation + - Match status (✓ or ❌) +4. Identify all discrepancies +5. Suggest fixes with code examples +6. Note any breaking changes" +``` + +**Generate Verification Report:** +``` +Prompt: +"Create a complete verification report for MeterReadingEntity: + +1. Extract XSD definition +2. Compare with entity implementation +3. Check Flyway migration script alignment +4. Run entity tests and report results +5. Generate a markdown report following the template in ISSUE_101_IMPLEMENTATION_PLAN.md +6. Save report to reports/MeterReadingEntity_verification.md" +``` + +**Fix Discrepancies:** +``` +Prompt: +"Fix the discrepancies found in IntervalBlockEntity: + +Discrepancies: +- Field 'duration' has length 512, should be 256 (String256) +- Field 'start' is nullable, should be NOT NULL (minOccurs=1) + +Please: +1. Update the entity with correct annotations +2. Generate Flyway migration script +3. Update tests if needed +4. Show me the diff before applying" +``` + +--- + +## Claude API for Automation + +### Batch Processing Script + +For processing multiple entities in parallel: + +```python +#!/usr/bin/env python3 +""" +Automated entity verification using Claude API +""" +import anthropic +import os +from pathlib import Path + +client = anthropic.Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY")) + +def verify_entity(entity_name, xsd_file, java_file): + """Verify a single entity against XSD""" + + with open(xsd_file) as f: + xsd_content = f.read() + + with open(java_file) as f: + java_content = f.read() + + prompt = f"""Verify {entity_name} against ESPI 4.0 XSD schema. + +XSD Definition: +{xsd_content} + +Java Implementation: +{java_content} + +Please provide: +1. Field-by-field comparison +2. List of discrepancies +3. Recommended fixes +4. Risk assessment (Low/Medium/High) + +Format as structured JSON.""" + + message = client.messages.create( + model="claude-sonnet-4-5-20250929", + max_tokens=4000, + messages=[ + {"role": "user", "content": prompt} + ] + ) + + return message.content[0].text + +# Process all usage domain entities +entities = [ + "UsagePoint", + "MeterReading", + "IntervalBlock", + # ... etc +] + +for entity in entities: + result = verify_entity( + entity, + f"schema/ESPI_4.0/espi.xsd", + f"domain/usage/{entity}Entity.java" + ) + + # Save report + Path(f"reports/{entity}_verification.json").write_text(result) + print(f"✓ Verified {entity}") +``` + +**When to use:** +- Processing 10+ similar entities +- Generating bulk reports +- Running nightly verification scans +- CI/CD integration + +**Model**: Claude Sonnet 4.5 (good balance for batch work) + +--- + +## IDE Integration + +### IntelliJ IDEA + Claude Code + +**Recommended Setup:** + +1. **Terminal Integration** +```bash +# Add to ~/.zshrc or ~/.bashrc +alias claude-espi='cd ~/Git/OpenESPI-GreenButton-Java && claude' + +# Quick verification alias +alias verify-entity='claude --prompt "Verify this entity against XSD schema"' +``` + +2. **External Tools Configuration** + +IntelliJ → Settings → Tools → External Tools → Add: +``` +Name: Verify Entity with Claude +Program: /usr/local/bin/claude +Arguments: --prompt "Verify $FileName$ against ESPI XSD schema" +Working directory: $ProjectFileDir$ +``` + +Now: Right-click entity file → External Tools → Verify Entity with Claude + +3. **File Watchers for Auto-Documentation** + +Generate verification reports automatically when entities change: +``` +File type: Java +Scope: Project Files +Program: claude +Arguments: --prompt "Generate verification report for $FileName$" +``` + +### VS Code + Claude + +**Recommended Extensions:** +- Claude Code extension (if available) +- Spring Boot Tools +- XML Tools (for XSD) +- JUnit Test Runner + +**Workspace Settings:** +```json +{ + "claude.contextFiles": [ + "CLAUDE.md", + "ISSUE_101_IMPLEMENTATION_PLAN.md", + "openespi-common/src/main/resources/schema/ESPI_4.0/*.xsd" + ], + "claude.defaultModel": "claude-sonnet-4-5" +} +``` + +--- + +## Specialized Tools + +### 1. XSD Analysis Tools + +**XMLSpy / Oxygen XML Editor** +- Visual XSD schema browser +- Generate sample XML from schema +- Validate XML against schema +- **Use for**: Understanding complex XSD structures before verification + +**xsd2java (JAXB)** +```bash +# Generate reference Java classes from XSD +xjc -d generated -p org.example.reference \ + openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd + +# Compare with actual implementation +diff -r generated/ openespi-common/src/main/java/.../domain/ +``` +- **Use with Claude**: Feed differences to Claude for analysis + +### 2. Code Analysis Tools + +**SonarQube / SonarLint** +```bash +# Run analysis +mvn sonar:sonar + +# Check for: +# - String length violations +# - Missing null checks +# - Unused enum values +``` +- **Use with Claude**: Feed SonarQube reports to Claude for remediation + +**SpotBugs** +```bash +mvn spotbugs:check + +# Look for: +# - Enum usage issues +# - Null pointer risks +# - Type mismatches +``` + +**ArchUnit** (for enforcing package structure) +```java +@Test +public void enums_should_be_in_correct_packages() { + classes() + .that().areEnums() + .and().haveSimpleNameEndingWith("Kind") + .should().resideInAPackage("..usage.enums..") + .orShould().resideInAPackage("..customer.enums..") + .orShould().resideInAPackage("..common") + .check(importedClasses); +} +``` + +### 3. Database Tools + +**Flyway Desktop / Flyway CLI** +```bash +# Validate migration scripts +flyway validate + +# Generate migration from entity changes +# (Use Claude to help generate SQL) +``` + +**DBeaver / DataGrip** +- Compare entity definitions with database schema +- Export schema as DDL +- **Use with Claude**: Compare DDL with Flyway scripts + +### 4. Testing Tools + +**Testcontainers** +```java +@Testcontainers +class EntityVerificationTest { + @Container + static MySQLContainer mysql = new MySQLContainer<>("mysql:8.0"); + + @Test + void verifyUsagePointSchema() { + // Test with real database + } +} +``` + +**ArchUnit** (for schema compliance) +```java +@Test +public void entities_should_have_correct_column_lengths() { + fields() + .that().areAnnotatedWith(Column.class) + .and().haveRawType(String.class) + .should(haveColumnLengthMatchingXsdBasicType()) + .check(importedClasses); +} +``` + +### 5. Documentation Tools + +**PlantUML** +```bash +# Generate entity relationship diagrams +# Feed to Claude for validation + +@startuml +entity UsagePoint { + * id : UUID + -- + serviceCategory : ServiceKind + description : String(256) +} +@enduml +``` + +**AsciiDoc / Markdown** +- Store verification reports +- Claude can generate AsciiDoc directly + +--- + +## MCP Servers (Model Context Protocol) + +### Recommended MCP Servers for This Project + +#### 1. **GitHub MCP** (Already Available) +```bash +# List all tasks +claude> What are all the open issues related to Issue #101? + +# Create tracking issues +claude> Create a GitHub issue for verifying UsagePointEntity +``` + +#### 2. **Filesystem MCP** (Already Available) +```bash +# Search for enum usage +claude> Find all files importing AmiBillingReadyKind + +# Batch file operations +claude> Move all *Kind.java files from common/ to usage/enums/ +``` + +#### 3. **Database MCP** (If Available) +```bash +# Compare entity with database +claude> Show me the database schema for usage_point table + +# Validate Flyway migrations +claude> Check if migration scripts match current entity definitions +``` + +#### 4. **Custom MCP: XSD Schema Server** + +**Create a custom MCP server for XSD operations:** + +```python +# xsd_mcp_server.py +from mcp.server import Server +import xml.etree.ElementTree as ET + +server = Server("xsd-analyzer") + +@server.tool() +def extract_simple_types(xsd_path: str) -> list: + """Extract all simpleType definitions from XSD""" + tree = ET.parse(xsd_path) + root = tree.getroot() + + simple_types = [] + for simple_type in root.findall(".//{http://www.w3.org/2001/XMLSchema}simpleType"): + name = simple_type.get("name") + # Extract restrictions, enumerations, etc. + simple_types.append({ + "name": name, + "type": extract_base_type(simple_type), + "restrictions": extract_restrictions(simple_type) + }) + + return simple_types + +@server.tool() +def extract_complex_type(xsd_path: str, type_name: str) -> dict: + """Extract specific complexType definition""" + # Implementation + pass + +@server.tool() +def validate_java_enum_against_xsd( + xsd_path: str, + enum_name: str, + java_code: str +) -> dict: + """Validate Java enum matches XSD simpleType""" + # Implementation + pass +``` + +**Use with Claude Code:** +```bash +# Start with custom MCP server +claude --mcp xsd-analyzer + +# Now Claude has access to XSD tools +claude> Extract all simpleTypes from espi.xsd +claude> Validate ServiceKind enum against XSD definition +``` + +--- + +## Workflow Automation + +### GitHub Actions + +**Automated Verification on PR:** + +```yaml +# .github/workflows/schema-verification.yml +name: ESPI Schema Verification + +on: + pull_request: + paths: + - 'openespi-common/src/main/java/**/domain/**/*.java' + +jobs: + verify-schema-compliance: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Set up Java + uses: actions/setup-java@v3 + with: + java-version: '25' + + - name: Verify Entity Changes + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + run: | + # Get changed entity files + CHANGED_FILES=$(git diff --name-only origin/main...HEAD | grep Entity.java) + + # Verify each with Claude API + python3 scripts/verify-entities.py $CHANGED_FILES + + - name: Comment PR with Results + uses: actions/github-script@v6 + with: + script: | + // Post verification results as PR comment + const fs = require('fs'); + const report = fs.readFileSync('verification-report.md', 'utf8'); + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: report + }); +``` + +### Pre-commit Hooks + +**Validate before committing:** + +```bash +# .git/hooks/pre-commit +#!/bin/bash + +# Check for enum imports from wrong packages +if git diff --cached --name-only | grep -q "Entity.java"; then + echo "Checking entity enum imports..." + + # Use Claude to validate + claude --prompt "Check staged entity files for correct enum imports" --non-interactive + + if [ $? -ne 0 ]; then + echo "❌ Enum import validation failed" + exit 1 + fi +fi +``` + +--- + +## Recommended Tool Stack by Phase + +### Phase 0: Enumeration Migration + +| Tool | Purpose | Model | +|------|---------|-------| +| Claude Code CLI | Interactive enum generation | Sonnet 4.5 | +| Claude API + Python | Batch verification of 30+ enums | Haiku | +| XMLSpy | XSD schema browsing | - | +| IntelliJ Refactor | Safe file moves | - | +| GitHub Actions | PR validation | Sonnet 4.5 (API) | + +**Daily Workflow:** +```bash +# Morning: Generate new enums +claude-espi +> "Generate next 5 missing enums from my list" + +# Afternoon: Verify implementations +python3 scripts/batch-verify-enums.py + +# Evening: Review and commit +git add domain/usage/enums/ +git commit -m "feat: Add AccumulationKind and CommodityKind enums" +``` + +### Phase 1-3: Entity Verification + +| Tool | Purpose | Model | +|------|---------|-------| +| Claude Code CLI | Interactive verification | Opus 4.5 (complex) / Sonnet 4.5 (standard) | +| DBeaver | Database schema comparison | - | +| SonarQube | Code quality checks | - | +| ArchUnit | Enforce package rules | - | +| Testcontainers | Integration testing | - | + +**Daily Workflow:** +```bash +# Verify 3-5 entities per day +claude-espi +> "Verify UsagePointEntity against XSD" +> "Generate fixes for MeterReadingEntity discrepancies" +> "Create Flyway migration for IntervalBlockEntity changes" + +# Run tests +mvn test -Dtest=UsagePointRepositoryTest + +# Commit small batches +git add openespi-common/src/main/java/...UsagePointEntity.java +git commit -m "fix: Correct UsagePoint field lengths per ESPI 4.0" +``` + +--- + +## Cost Optimization + +### Model Selection Strategy + +**Use Opus 4.5 for (~10% of work):** +- ✅ First-time XSD analysis of complex types +- ✅ Architecture decisions (enum placement, package structure) +- ✅ Complex discrepancy analysis +- ✅ Migration strategy planning + +**Use Sonnet 4.5 for (~70% of work):** +- ✅ Entity verification +- ✅ Code generation +- ✅ Test generation +- ✅ Documentation +- ✅ Code review + +**Use Haiku for (~20% of work):** +- ✅ Batch verification of similar entities +- ✅ Simple enum generation (after pattern established) +- ✅ Import statement updates +- ✅ Repetitive checks + +**Estimated Cost for Entire Project:** +- Opus 4.5: ~$50-75 (complex analysis) +- Sonnet 4.5: ~$150-200 (main work) +- Haiku: ~$25-40 (batch operations) +- **Total: ~$225-315** for complete 102-task project + +**Cost Savings Tips:** +1. **Establish patterns early** with Opus, then use Haiku for repetition +2. **Batch similar tasks** to reuse context +3. **Cache XSD analysis** - analyze schema once with Opus, reuse results +4. **Use local tools** (grep, diff) for simple comparisons + +--- + +## Productivity Metrics + +### Expected Output with Recommended Tools + +**Without Claude assistance:** +- 1-2 entities verified per day +- ~50-60 days for 102 tasks +- High error rate on manual XSD interpretation + +**With Claude Code + recommended tools:** +- 3-5 entities verified per day +- ~20-30 days for 102 tasks +- Automated consistency checking + +**With full automation (Claude API + scripts):** +- 10-15 entities verified per day +- ~7-10 days for 102 tasks (batch processing) +- Near-perfect consistency + +### Recommended Approach: Hybrid + +**Phase 0 (Enums): Interactive with Claude Code** +- Learn patterns from first 5 enums (Opus) +- Generate remaining enums with templates (Sonnet) +- Batch verify all (Haiku + automation) +- **Result**: 4 weeks → 2 weeks + +**Phase 1-3 (Entities): Semi-automated** +- Verify complex entities interactively (Opus/Sonnet) +- Batch process similar entities (Haiku + scripts) +- Automated PR checks (CI/CD + API) +- **Result**: 8 weeks → 4-5 weeks + +**Total Project**: 12 weeks → **6-7 weeks** with proper tooling + +--- + +## Quick Reference: Task → Tool → Model + +| Task | Best Tool | Best Model | Example | +|------|-----------|------------|---------| +| Analyze XSD schema | Claude Code | Opus 4.5 | "Explain this complexType structure" | +| Generate enum | Claude Code | Sonnet 4.5 | "Create AccumulationKind enum" | +| Verify entity fields | Claude Code | Sonnet 4.5 | "Compare UsagePoint with XSD" | +| Fix discrepancies | Claude Code | Sonnet 4.5 | "Update entity annotations" | +| Generate tests | Claude Code | Sonnet 4.5 | "Create JUnit tests" | +| Batch verify 20 enums | Claude API + Script | Haiku | Run automated script | +| Create migration | Claude Code | Sonnet 4.5 | "Generate Flyway SQL" | +| Code review | GitHub + Claude | Sonnet 4.5 | Automated PR comments | +| Architecture decision | Claude Code | Opus 4.5 | "Should we create common/enums?" | +| Documentation | Claude Code | Sonnet 4.5 | "Generate verification report" | + +--- + +## Conclusion + +### Optimal Tool Stack + +**For Senior Spring Developer:** + +1. **Primary**: Claude Code CLI with Sonnet 4.5 (70% of work) +2. **Complex Analysis**: Claude Code with Opus 4.5 (10% of work) +3. **Batch Operations**: Claude API + Python scripts with Haiku (20% of work) +4. **IDE**: IntelliJ IDEA + External Tools integration +5. **Analysis**: XMLSpy for XSD, SonarQube for code quality +6. **Testing**: Testcontainers + ArchUnit +7. **Automation**: GitHub Actions + pre-commit hooks + +**Expected Results:** +- ✅ 50% faster than manual work +- ✅ 90%+ accuracy on first attempt +- ✅ Consistent patterns across all 102 tasks +- ✅ Automated validation and reporting +- ✅ Total cost: ~$225-315 for entire project + +**Start Here:** +```bash +# Set up Claude Code with project context +cd ~/Git/OpenESPI-GreenButton-Java +claude + +# First task: Analyze XSD schema +> "Read espi.xsd and extract all simpleType enumerations. + Create a comprehensive inventory with: + - Name + - Base type + - Enum values + - Documentation + Save to reports/espi-enumerations.md" + +# Then: Generate first enum +> "Generate AccumulationKind enum based on the XSD definition. + Follow the pattern from existing enums. + Save to domain/usage/enums/AccumulationKind.java" +``` + +You're ready to start! 🚀 diff --git a/openespi-common/src/main/resources/schema/ESPI_4.1/atom.xsd b/openespi-common/src/main/resources/schema/ESPI_4.1/atom.xsd new file mode 100755 index 00000000..f62a6eeb --- /dev/null +++ b/openespi-common/src/main/resources/schema/ESPI_4.1/atom.xsd @@ -0,0 +1,255 @@ + + + + + + This version of the Atom schema is based on version 1.0 of the format specifications, + found here https://tools.ietf.org/pdf/rfc4287.pdf. + + + + + + An Atom document may have two root elements, feed and entry, as defined in section 2. + + + + + + + + The Atom text construct is defined in section 3.1 of the format spec. + + + + + + + + + + + + + + + + + + + + The Atom person construct is defined in section 3.2 of the format spec. + + + + + + + + + + + + + + Schema definition for an email address. + + + + + + + + + + The Atom feed construct is defined in section 4.1.1 of the format spec. + + + + + + + + + + + + + + + + + + + + + + + + The Atom entry construct is defined in section 4.1.2 of the format spec. + + + + + + + + + + + + + + + + + + + + + + + The Atom content construct is defined in section 4.1.3 of the format spec. + + + + + + + + + + + + + The Atom category construct is defined in section 4.2.2 of the format spec. + + + + + + + + + + + The Atom generator element is defined in section 4.2.4 of the format spec. + + + + + + + + + + + + + + The Atom icon construct is defined in section 4.2.5 of the format spec. + + + + + + + + + + + + The Atom id construct is defined in section 4.2.6 of the format spec. + + + + + + + + + + + + The Atom link construct is defined in section 3.4 of the format spec. + + + + + + + + + + + + + + The Atom logo construct is defined in section 4.2.8 of the format spec. + + + + + + + + + + + + The Atom source construct is defined in section 4.2.11 of the format spec. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/openespi-common/src/main/resources/schema/ESPI_4.1/customer_4.1.xsd b/openespi-common/src/main/resources/schema/ESPI_4.1/customer_4.1.xsd new file mode 100755 index 00000000..186517e0 --- /dev/null +++ b/openespi-common/src/main/resources/schema/ESPI_4.1/customer_4.1.xsd @@ -0,0 +1,3210 @@ + + + + + + + The retail customer schema provides details about a UsagePoint that are considered Personally identifiable information (PII) and + cannot be part of normal ESPI usage information. + + These resources are available via the same messaging patterns as other ESPI data. However, they are defined in a separate namespace + and cannot be retrieved in the same message as non-PII containing ESPI data. + + + + + + + + Organisation receiving services from service supplier. + + + + + + + Customer's contact information. + + + + + Kind of customer. + + + + + True if customer organisation has special service needs such as life support, hospitals, etc. + + + + + (use 'priority' instead) True if this is an important customer. Importance is for matters different than those in 'specialNeed' attribute. + + + + + (if applicable) Public utilities commission (PUC) identification number. + + + + + Status of this customer. + + + + + Priority of the customer. + + + + + Locale designating language to use in communications with this customer. + + + + + [extension] Customer name + + + + + + + + + Assignment of a group of products and services purchased by the customer through a customer agreement, used as a mechanism for customer billing and payment. It contains common information from the various types of customer agreements to create billings (invoices) for a customer and receive payment. + + + + + + + Basic customer account information. + + + + + Cycle day on which the associated customer account will normally be billed, used to determine when to produce the billing. + + + + + Budget bill code. + + + + + The last amount that will be billed to the customer prior to shut off of the account. + + + + + Set of customer account notifications. + + + + + [extension] Customer contact information used to identify individual responsible for billing and payment of CustomerAccount. + + + + + [extension] Customer account identifier + + + + + + + + + Agreement between the customer and the service supplier to pay for service at a specific service location. It records certain billing information about the type of service provided at the service location and is used during charge creation to determine the type of service. + + + + + + + Formal agreement between two parties defining the terms and conditions for a set of services. + + + + + Load management code. + + + + + If true, the customer is a pre-pay customer for the specified service. + + + + + Final date and time the service will be billed to the previous customer. + + + + + Demand Response program characteristics covered by Customer Agreement + + + + + All pricing structures applicable to this customer agreement. + + + + + Currency for all monetary amounts for this agreement. + + + + + [extension] Known future changes to CustomerAgreement's Status of Service. + + + + + [extension] Customer agreement identifier + + + + + + + + + Asset container that performs one or more end device functions. One type of end device is a meter which can perform metering, load management, connect/disconnect, accounting functions, etc. Some end devices, such as ones monitoring and controlling air conditioners, refrigerators, pool pumps may be connected to a meter. All end devices may have communication capability defined by the associated communication function(s). An end device may be owned by a consumer, a service provider, utility or otherwise. +There may be a related end device function that identifies a sensor or control point within a metering application or communications systems (e.g., water, gas, electricity). +Some devices may use an optical port that conforms to the ANSI C12.18 standard for communications. + + + + + + + Description of the equipment or device collecting the energy data. + + + + + If true, there is no physical device. As an example, a virtual meter can be defined to aggregate the consumption for two or more physical meters. Otherwise, this is a physical hardware device. + + + + + If true, this is a premises area network (PAN) device. + + + + + Installation code. + + + + + Automated meter reading (AMR) or other communication system responsible for communications to this end device. + + + + + + + + + Physical asset that performs the metering role of the usage point. Used for measuring consumption and detection of events. + + + + + + + Description of the equipment or device collecting the energy data. + + + + + Meter form designation per ANSI C12.10 or other applicable standard. An alphanumeric designation denoting the circuit arrangement for which the meter is applicable and its specific terminal arrangement. + + + + + All multipliers applied at this meter. + + + + + [extension] Current interval length specified in seconds. + + + + + + + + + [extension] Collection of all customer Energy Efficiency programs + + + + + + + [extension] Program date description + + + + + + + + + A real estate location, commonly referred to as premises. + + + + + + + The location of the service site. + + + + + Method for the service person to access this service location. For example, a description of where to obtain a key if the facility is unmanned and secured. + + + + + Problems previously encountered when visiting or performing work on this location. Examples include: bad dog, violent customer, verbally abusive occupant, obstructions, safety hazards, etc. + + + + + True if inspection is needed of facilities at this service location. This could be requested by a customer, due to suspected tampering, environmental concerns (e.g., a fire in the vicinity), or to correct incompatible data. + + + + + All usage points delivering service (of the same type) to this service location. + + + + + [extension] Outage Block Identifier + + + + + + + + + Organisation that provides services to customers. + + + + + + + The Service Supplier's contact information. + + + + + Kind of supplier. + + + + + Unique transaction reference prefix number issued to an entity by the International Organization for Standardization for the purpose of tagging onto electronic financial transactions, as defined in ISO/IEC 7812-1 and ISO/IEC 7812-2. + + + + + [extension] Effective Date of Service Activation + + + + + + + + + [extension] Billing statement for provided services. + + + + + + + [extension] Date and time at which a billing statement was issued. + + + + + [extension] Contains document reference metadata needed to access a document representation of a billing statement. + + + + + + + + + [extension] Contains attributes related to the configuration of the time service. + + + + + + + Rule to calculate end of daylight savings time in the current year. Result of dstEndRule must be greater than result of dstStartRule. + + + + + Daylight savings time offset from local standard time. + + + + + Rule to calculate start of daylight savings time in the current year. Result of dstEndRule must be greater than result of dstStartRule. + + + + + Local time zone offset from UTCTime. Does not include any daylight savings time offsets. + + + + + + + + + + A 128-bit field encoded as a hex string (32 characters / 16 octets) + + + + + + + + A 32-bit field encoded as a hex string (8 characters / 4 octets) + + + + + + + + A 16-bit field encoded as a hex string (4 characters / 2 octets) + + + + + + + + An 8-bit field encoded as a hex string (2 characters / 1 octets) + + + + + + + + [extension] Character string of max length 512 + + + + + + + + Character string of max length 256 + + + + + + + + Character string of max length 64 + + + + + + + + Character string of max length 32 + + + + + + + + Unsigned integer, max inclusive 281474976710655 (2^48-1), restriction of xs:unsignedLong + + + + + + + + Unsigned integer, max inclusive 4294967295 (2^32-1), same as xs:unsignedInt + + + + + + Unsigned integer, max inclusive 65535 (2^16-1), same as xs:unsignedShort + + + + + + Unsigned integer, max inclusive 255 (2^8-1), same as xs:unsignedByte + + + + + + Signed integer, max inclusive -140737488355327 to +140737488355327 (2^47-1), restriction of xs:long + + + + + + + + + Signed integer, max inclusive (2^16-1), restriction of xs:short + + + + + + + + + Percentage on a defined base. For example, specify as 100 to indicate at the defined base. + + + + + + + + + Time is a signed 64-bit value representing the number of seconds since 0 hours, 0 minutes, 0 seconds, on the 1st of January, 1970. + + + + + + This pattern defines a UUID + + + + + + + + + Acceptance test for assets. + + + + + + + Describes the type of test performed on the asset. + + + + + True if asset has passed acceptance test and may be placed in or is in service. It is set to false if asset is removed from service and is required to be tested again before being placed back in service, possibly in a new location. Since asset may go through multiple tests during its lifecycle, the date of each acceptance test may be recorded in 'Asset.ActivityRecord.status.dateTime'. + + + + + Date and time the asset was last tested using the 'type' of test and yielding the current status in 'success' attribute. + + + + + + + + + [extension] Customer action notification (e.g., delinquency, move in, move out) + + + + + + + Method by which the customer was notified. + + + + + timedate of notification + + + + + annotation of the reason for the notification + + + + + delinquency,move in, move out ... + + + + + + + + + Formal agreement between two parties defining the terms and conditions for a set of services. The specifics of the services are, in turn, defined via one or more service agreements. + + + + + + + Date this agreement was consummated among associated persons and/or organisations. + + + + + Date and time interval this agreement is valid (from going into effect to termination). + + + + + + + + + Tangible resource of the utility, including power system equipment, various end devices, cabinets, buildings, etc. For electrical network equipment, the role of the asset is defined through PowerSystemResource and its subclasses, defined mainly in the Wires model (refer to IEC61970-301 and model package IEC61970::Wires). Asset description places emphasis on the physical characteristics of the equipment fulfilling that role. + + + + + + + Utility-specific classification of Asset and its subtypes, according to their corporate standards, practices, and existing IT systems (e.g., for management of assets, maintenance, work, outage, customers, etc.). + + + + + Uniquely tracked commodity (UTC) number. + + + + + Serial number of this asset. + + + + + Lot number for this asset. Even for the same model and version number, many assets are manufactured in lots. + + + + + Purchase price of asset. + + + + + True if asset is considered critical for some reason (for example, a pole with critical attachments). + + + + + Electronic address. + + + + + Lifecycle dates for this asset. + + + + + Information on acceptance test. + + + + + Condition of asset in inventory or at time of installation. Examples include new, rebuilt, overhaul required, other. Refer to inspection data for information on the most current condition of the asset. + + + + + Whenever an asset is reconditioned, percentage of expected life for the asset when it was new; zero for new devices. + + + + + Status of this asset. + + + + + + + + + + Includes elements that make it possible to include multiple transactions in a single (batch) request. + + + + + + + An identifier for this object that is only unique within the containing collection. + + + + + Specifies the operation requested of this item. + + + + + Indicates the status code of the associated transaction. + + + + + Indicates the reason for the indicated status code. + + + + + + + + + [extension] List of resource URIs that can be used to GET retail customer resources + + + + + + + + Interval of date and time. End is not included because it can be derived from the start and the duration. + + + + + + + [correction] Duration of the interval, in seconds. + + + + + [correction] Date and time that this interval started. + + + + + + + + + [extension] Demand Response program characteristics covered by Customer Agreement + + + + + + + Name of Demand Response program + + + + + Retail Customer's current Demand Response program enrollment status + + + + + URI of Demand Response program description + + + + + All Demand Response program enrollment and termination dates + + + + + Summary measurement of Capacity Reservation + + + + + Summary measurement of Demand Response + + + + + + + + + Parent class for different groupings of information collected and managed as a part of a business process. It will frequently contain references to other objects, such as assets, people and power system resources. + + + + + + + Utility-specific classification of this document, according to its corporate standards, practices, and existing IT systems (e.g., for management of assets, maintenance, work, outage, customers, etc.). + + + + + Name of the author of this document. + + + + + Date and time that this document was created. + + + + + Date and time this document was last modified. Documents may potentially be modified many times during their lifetime. + + + + + Revision number for this document. + + + + + Electronic address. + + + + + Document subject. + + + + + Document title. + + + + + Status of this document. For status of subject matter this document represents (e.g., Agreement, Work), use 'status' attribute. +Example values for 'docStatus.status' are draft, approved, cancelled, etc. + + + + + Status of subject matter (e.g., Agreement, Work) this document represents. For status of the document itself, use 'docStatus' attribute. + + + + + Free text comment. + + + + + + + + + Electronic address information. + + + + + + + Address on local area network. + + + + + MAC (Media Access Control) address. + + + + + Primary email address. + + + + + Alternate email address. + + + + + World wide web address. + + + + + Radio address. + + + + + User ID needed to log in, which can be for an individual person, an organisation, a location, etc. + + + + + Password needed to log in. + + + + + + + + + This is a root class to provide common naming attributes for all classes needing naming attributes + + + + + + + + [DEPRECATED] identifier of object instance + + + + + + + + + The place, scene, or point of something where someone or something has been, is, and/or will be at a given moment in time. It can be defined with one or more position points (coordinates) in a given coordinate system. + + + + + + + Classification by utility's corporate standards and practices, relative to the location itself (e.g., geographical, functional accounting, etc., not a given property that happens to exist at that location). + + + + + Main address of the location. + + + + + Secondary address of the location. For example, PO Box address may have different ZIP code than that in the 'mainAddress'. + + + + + Phone number. + + + + + Additional phone number. + + + + + Electronic address. + + + + + (if applicable) Reference to geographical information source, often external to the utility. + + + + + (if applicable) Direction that allows field crews to quickly find a given asset. For a given location, such as a street address, this is the relative direction in which to find the asset. For example, a streetlight may be located at the 'NW' (northwest) corner of the customer's site, or a usage point may be located on the second floor of an apartment building. + + + + + Status of this location. + + + + + Sequence of position points describing this location, expressed in coordinate system 'Location.CoordinateSystem'. + + + + + + + + + Dates for lifecycle events of an asset. + + + + + + + Date the asset was manufactured. + + + + + Date the asset was purchased. Note that even though an asset may have been purchased, it may not have been received into inventory at the time of purchase. + + + + + Date the asset was received and first placed into inventory. + + + + + (if applicable) Date current installation was completed, which may not be the same as the in-service date. Asset may have been installed at other locations previously. Ignored if asset is (1) not currently installed (e.g., stored in a depot) or (2) not intended to be installed (e.g., vehicle, tool). + + + + + (if applicable) Date when the asset was last removed from service. Ignored if (1) not intended to be in service, or (2) currently in service. + + + + + (if applicable) Date the asset is permanently retired from service and may be scheduled for disposal. Ignored if asset is (1) currently in service, or (2) permanently removed from service. + + + + + + + + + Multiplier applied at the meter. + + + + + + + Kind of multiplier. + + + + + Multiplier value. + + + + + + + + + Superclass of all object classes to allow extensions. Inheritance from Object provides an inherent extension mechanism permitting custom data to be included in a separate namespace. + + + + + Contains an extension. + + + + + + + Organisation that might have roles as utility, contractor, supplier, manufacturer, customer, etc. + + + + + + + Street address. + + + + + Postal address, potentially different than 'streetAddress' (e.g., another city). + + + + + Phone number. + + + + + Additional phone number. + + + + + Electronic address. + + + + + [extension] Organisation name + + + + + + + + + + Set of spatial coordinates that determine a point, defined in the coordinate system specified in 'Location.CoordinateSystem'. Use a single position point instance to describe a point-oriented location. Use a sequence of position points to describe a line-oriented object (physical location of non-point oriented objects like cables or lines), or area of an object (like a substation or a geographical zone - in this case, have first and last position point with the same values). + + + + + + + X axis position. + + + + + Y axis position. + + + + + (if applicable) Z axis position. + + + + + + + + + Grouping of pricing components and prices used in the creation of customer charges and the eligibility criteria under which these terms may be offered to a customer. The reasons for grouping include state, customer classification, site characteristics, classification (i.e. fee price structure, deposit price structure, electric service price structure, etc.) and accounting requirements. + + + + + + + + Priority definition. + + + + + + + This is the priority of this event relative to other events. The lower the number higher the priority. A value of zero (0) indicates NO priority and in essence is the lowest priority by default. + + + + + Type describing 'rank'; e.g., high, emergency, etc. + + + + + Justification for 'rank'. + + + + + + + + + [extension] Demand Response Program enrollment or termination date information + + + + + Demand Response Program enrollment or termination date + + + + + Description of programDate (i.e., Enrollment, Termination, etc.) + + + + + + + [extension] Single customer energy efficiency program date mapping + + + + + + + Type of customer energy efficiency program date + + + + + Code value (may be alphanumeric). + + + + + Name associated with code. + + + + + Optional description of code + + + + + + + + + [extension] A sequence of references to a document associated with a Statement. + + + + + + + [extension] Name of document or file including filename extension if present. + + + + + [extension] Document media type as published by IANA, see https://www.iana.org/assignments/media-types for more information. + + + + + [extension] URL used to access a representation of a statement, for example a bill image. Use CDATA or URL encoding to escape characters not allowed in XML. + + + + + + + + + Current status information relevant to an entity. + + + + + + + Status value at 'dateTime'; prior status changes may have been kept in instances of activity records associated with the object to which this status applies. + + + + + Date and time for which status 'value' applies. + + + + + Pertinent information regarding the current 'value', as free form text. + + + + + Reason code or explanation for why an object went to the current status 'value'. + + + + + + + + + General purpose street and postal address information. + + + + + + + Street detail. + + + + + Town detail. + + + + + Status of this address. + + + + + Postal code for the address. + + + + + Post office box. + + + + + + + + + Street details, in the context of address. + + + + + + + Designator of the specific location on the street. + + + + + Name of the street. + + + + + Suffix to the street name. For example: North, South, East, West. + + + + + Prefix to the street name. For example: North, South, East, West. + + + + + Type of street. Examples include: street, circle, boulevard, avenue, road, drive, etc. + + + + + (if applicable) Utilities often make use of external reference systems, such as those of the town-planner's department or surveyor general's mapping system, that allocate global reference codes to streets. + + + + + (if applicable) In certain cases the physical location of the place of interest does not have a direct point of entry from the street, but may be located inside a larger structure such as a building, complex, office block, apartment, etc. + + + + + Number of the apartment or suite. + + + + + First line of a free form address or some additional address information (for example a mail stop). + + + + + (if applicable) Second line of a free form address. + + + + + (if applicable) Third line of a free form address. + + + + + True if this street is within the legal geographical boundaries of the specified town (default). + + + + + + + + + An aggregated summary measurement reading + + + + + + + The multiplier part of the unit of measure, e.g. "kilo" (k) + + + + + The date and time (if needed) of the summary measurement. + + + + + The units of the reading, e.g. “Wh” + + + + + The value of the summary measurement. + + + + + Reference to a full ReadingType. + + + + + + + + + Telephone number. + + + + + + + Country code. + + + + + Area or region code. + + + + + City code. + + + + + Main (local) part of this telephone number. + + + + + (if applicable) Extension for this telephone number. + + + + + (if applicable) Dial out code, for instance to call outside an enterprise. + + + + + (if applicable) Prefix used when calling an international number. + + + + + Phone number according to ITU E.164. + + + + + + + + + Town details, in the context of address. + + + + + + + Town code. + + + + + Town section. For example, it is common for there to be 36 sections per township. + + + + + Town name. + + + + + [extension] Name of county. + + + + + Name of the state or province. + + + + + Name of the country. + + + + + + + + + Logical or physical point in the network to which readings or events may be attributed. Used at the place where a physical or virtual meter may be located; however, it is not required that a meter be present. + + + + + + + + Sequence of UsagePoint URIS + + + + + + + Logical or physical point in the network to which readings or events may be attributed. Used at the place where a physical or virtual meter may be located; however, it is not required that a meter be present. + + + + + + + + + + + Specifies the operation requested of this item + + + + + + + Create + Create + + + + + Read + Read + + + + + Update + Update + + + + + Delete + Delete + + + + + + + + + Code for the currency for costs associated with this ReadingType. The valid values per the standard are defined in CurrencyCode. + + + + + + + USD + US dollar + + + + + EUR + European euro + + + + + AUD + Australian dollar + + + + + CAD + Canadian dollar + + + + + CHF + Swiss francs + + + + + CNY + Chinese yuan renminbi + + + + + DKK + Danish crown + + + + + GBP + British pound + + + + + JPY + Japanese yen + + + + + NOK + Norwegian crown + + + + + RUB + Russian ruble + + + + + SEK + Swedish crown + + + + + INR + India rupees + + + + + other + Another type of currency. + + + + + + + + + Kind of customer. + + + + + + + Residential customer. + + + + + Residential and commercial customer. + + + + + Residential and streetlight customer. + + + + + Residential streetlight or other related customer. + + + + + Residential farm service customer. + + + + + Commercial industrial customer. + + + + + Pumping load customer. + + + + + Wind machine customer. + + + + + Customer as energy service supplier. + + + + + Customer as energy service scheduler. + + + + + Represents the owning enterprise. + + + + + Represents a local operator of a larger enterprise. + + + + + A subsidiary of a larger enterprise. + + + + + Internal use customer. + + + + + Other kind of customer. + + + + + + + + + [extension] Bit map encoded rule from which is calculated the start or end time, within the current year, to which daylight savings time offset must be applied. + +The rule encoding: +Bits 0 - 11: seconds 0 - 3599 +Bits 12 - 16: hours 0 - 23 +Bits 17 - 19: day of the week 0 = not applicable, 1 - 7 (Monday = 1) +Bits:20 - 24: day of the month 0 = not applicable, 1 - 31 +Bits: 25 - 27: operator (detailed below) +Bits: 28 - 31: month 1 - 12 + +Rule value of 0xFFFFFFFF means rule processing/DST correction is disabled. + +The operators: + +0: DST starts/ends on the Day of the Month +1: DST starts/ends on the Day of the Week that is on or after the Day of the Month +2: DST starts/ends on the first occurrence of the Day of the Week in a month +3: DST starts/ends on the second occurrence of the Day of the Week in a month +4: DST starts/ends on the third occurrence of the Day of the Week in a month +5: DST starts/ends on the forth occurrence of the Day of the Week in a month +6: DST starts/ends on the fifth occurrence of the Day of the Week in a month +7: DST starts/ends on the last occurrence of the Day of the Week in a month + +An example: DST starts on third Friday in March at 1:45 AM. The rule... +Seconds: 2700 +Hours: 1 +Day of Week: 5 +Day of Month: 0 +Operator: 4 +Month: 3 + + + + + + [extension] Current Demand Response program enrollment status + + + + + + + Currently NOT enrolled in the Demand Response program + + + + + Currently enrolled in the Demand Response program + + + + + Currently pending enrollment in the Demand Response program + + + + + + + + + [extension] Media type for document as registered by IANA, allowed subset in enumeration. + + + + + + + JSON + + + + + PDF + + + + + vnd.ms-excel + + + + + vnd.oasis.opendocument.spreadsheet + + + + + vnd.oasis.opendocument.text + + + + + vnd.openxmlformats-officedocument.spreadsheetml.sheet + + + + + ZIP + + + + + GIF + + + + + JPEG + + + + + PNG + + + + + CSV + + + + + HTML + + + + + plain + + + + + RTF + + + + + XML + + + + + + + + + Kind of meter multiplier. + + + + + + + Meter kh (watthour) constant. The number of watthours that must be applied to the meter to cause one disk revolution for an electromechanical meter or the number of watthours represented by one increment pulse for an electronic meter. + + + + + The ratio of the transformer's primary and secondary windings (turns) with respect to each other. + + + + + Register multiplier. The number to multiply the register reading by in order to get kWh. + + + + + Test constant. + + + + + Current transformer ratio used to convert associated quantities to real measurements. + + + + + Potential transformer ratio used to convert associated quantities to real measurements. + + + + + + + + + Method by which the customer was notified + + + + + + + Contacted by phone by customer service representative. + + + + + Trouble reported by email. + + + + + Trouble reported by letter. + + + + + Trouble reported by other means. + + + + + Trouble reported through interactive voice response system. + + + + + + + + + [extension] Type of Demand Response program date + + + + + + + Date customer enrolled in Demand Response program + + + + + Date customer terminated enrollment in Demand Response program + + + + + Earliest date customer can terminate Demand Response enrollment, regardless of financial impact + + + + + Earliest date customer can terminate Demand Response enrollment, without financial impact + + + + + + + + + Accounting classification of the type of revenue collected for the customer agreement, typically used to break down accounts for revenue accounting. + + + + + + + Residential revenue. + + + + + Non-residential revenue. + + + + + Commercial revenue. + + + + + Industrial revenue. + + + + + Irrigation revenue. + + + + + Streetlight revenue. + + + + + Other revenue kind. + + + + + + + + + Kind of service. + + + + + Electricity service. + + + + + Gas service. + + + + + Water service. + + + + + Time service. + + + + + Heat service. + + + + + Refuse (waster) service. + + + + + Sewerage service. + + + + + Rates (e.g. tax, charge, toll, duty, tariff, etc.) service. + + + + + TV license service. + + + + + Internet service. + + + + + Other kind of service. + + + + + + + indicates the status code of the associated transaction + + + + + + + Ok + OK + + + + + Created + Created + + + + + Accepted + Accepted + + + + + No Content + No Content + + + + + Moved Permanently + Moved Permanently + + + + + Redirect + Redirect + + + + + Not Modified + Not Modified + + + + + Bad Request + Bad Request + + + + + Unauthorized + Unauthorized + + + + + Forbidden + Forbidden + + + + + Not Found + Not Found + + + + + Method Not Allowed + Method Not Allowed + + + + + Gone + Gone + + + + + Internal Server Error + Internal Server Error + + + + + + + + + Kind of supplier. + + + + + + + Entity that delivers the service to the customer. + + + + + Entity that sells the service, but does not deliver to the customer; applies to the deregulated markets. + + + + + Other kind of supplier. + + + + + [extension] Load Serving Entity + + + + + [extension] Meter Data Management Agent + + + + + [extension] Meter Service Provider + + + + + + + + + The power of ten unit multipliers + + + + + + + Pico 10**-12 + + + + + Nano 10**-9 + + + + + Micro 10**-6 + + + + + Milli 10**-3 + + + + + Centi 10**-2 + + + + + Deci 10**-1 + + + + + Kilo 10**3 + + + + + Mega 10**6 + + + + + Giga 10**9 + + + + + Tera 10**12 + + + + + Not Applicable or "x1" + + + + + deca 10**1 + + + + + hecto 10**2 + + + + + + + + + Code for the base unit of measure for Readings of ReadingType. Used in combination with the powerOfTenMultiplier to specify the actual unit of measure. Valid values per the standard are defined in UomType. + + + + + + + + VA + Apparent power, Volt Ampere (See also real power and reactive power.), VA + + + + + W + Real power, Watt. By definition, one Watt equals one Joule per second. Electrical power may have real and reactive components. The real portion of electrical power (I²R) or VIcos?, is expressed in Watts. (See also apparent power and reactive power.), W + + + + + VAr + Reactive power, Volt Ampere reactive. The “reactive” or “imaginary” component of electrical power (VISin?). (See also real power and apparent power)., VAr + + + + + VAh + Apparent energy, Volt Ampere hours, VAh + + + + + Wh + Real energy, Watt hours, Wh + + + + + VArh + Reactive energy, Volt Ampere reactive hours, VArh + + + + + V + Electric potential, Volt (W/A), V + + + + + ohm + Electric resistance, Ohm (V/A), O + + + + + A + Current, ampere, A + + + + + F + Electric capacitance, Farad (C/V), °C + + + + + H + Electric inductance, Henry (Wb/A), H + + + + + degC + Relative temperature in degrees Celsius. In the SI unit system the symbol is ºC. Electric charge is measured in coulomb that has the unit symbol C. To distinguish degree Celsius from coulomb the symbol used in the UML is degC. Reason for not using ºC is the special character º is difficult to manage in software. + + + + + sec + Time, seconds, s + + + + + min + Time, minute = s * 60, min + + + + + h + Time, hour = minute * 60, h + + + + + deg + Plane angle, degrees, deg + + + + + rad + Plane angle, Radian (m/m), rad + + + + + J + Energy joule, (N·m = C·V = W·s), J + + + + + n + Force newton, (kg m/s²), N + + + + + siemens + Electric conductance, Siemens (A / V = 1 / O), S + + + + + none + N/A, None + + + + + Hz + Frequency hertz, (1/s), Hz + + + + + g + Mass in gram, g + + + + + pa + Pressure, Pascal (N/m²)(Note: the absolute or relative measurement of pressure is implied with this entry. See below for more explicit forms.), Pa + + + + + m + Length, meter, m + + + + + m2 + Area, square meter, m² + + + + + m3 + Volume, cubic meter, m³ + + + + + A2 + Amps squared, amp squared, A2 + + + + + A2h + ampere-squared, Ampere-squared hour, A²h + + + + + A2s + Amps squared time, square amp second, A²s + + + + + Ah + Ampere-hours, Ampere-hours, Ah + + + + + APerA + Current, Ratio of Amperage, A/A + + + + + APerM + A/m, magnetic field strength, Ampere per metre, A/m + + + + + As + Amp seconds, amp seconds, As + + + + + b + Sound pressure level, Bel, acoustic, Combine with multiplier prefix “d” to form decibels of Sound Pressure Level“dB (SPL).”, B (SPL) + + + + + bm + Signal Strength, Bel-mW, normalized to 1mW. Note: to form “dBm” combine “Bm” with multiplier “d”. Bm + + + + + bq + Radioactivity, Becquerel (1/s), Bq + + + + + btu + Energy, British Thermal Units, BTU + + + + + btuPerH + Power, BTU per hour, BTU/h + + + + + cd + Luminous intensity, candela, cd + + + + + char + Number of characters, characters, char + + + + + HzPerSec + Rate of change of frequency, hertz per second, Hz/s + + + + + code + Application Value, encoded value, code + + + + + cosTheta + Power factor, Dimensionless, cos? + + + + + count + Amount of substance, counter value, count + + + + + ft3 + Volume, cubic feet, ft³ + + + + + ft3compensated + Volume, cubic feet, ft³(compensated) + + + + + ft3compensatedPerH + Volumetric flow rate, compensated cubic feet per hour, ft³(compensated)/h + + + + + gM2 + Turbine inertia, gram·meter2 (Combine with multiplier prefix “k” to form kg·m2.), gm² + + + + + gPerG + Concentration, The ratio of the mass of a solute divided by the mass of the solution., g/g + + + + + gy + Absorbed dose, Gray (J/kg), GY + + + + + HzPerHz + Frequency, Rate of frequency change, Hz/Hz + + + + + charPerSec + Data rate, characters per second, char/s + + + + + imperialGal + Volume, imperial gallons, ImperialGal + + + + + imperialGalPerH + Volumetric flow rate, Imperial gallons per hour, ImperialGal/h + + + + + jPerK + Heat capacity, Joule/Kelvin, J/K + + + + + jPerKg + Specific energy, Joules / kg, J/kg + + + + + K + Temperature, Kelvin, K + + + + + kat + Catalytic activity, katal = mol / s, kat + + + + + kgM + Moment of mass ,kilogram meter (kg·m), M + + + + + kgPerM3 + Density, gram/cubic meter (combine with prefix multiplier “k” to form kg/ m³), g/m³ + + + + + litre + Volume, litre = dm3 = m3/1000., L + + + + + litreCompensated + Volume, litre, with the value compensated for weather effects, L(compensated) + + + + + litreCompensatedPerH + Volumetric flow rate, litres (compensated) per hour, L(compensated)/h + + + + + litrePerH + Volumetric flow rate, litres per hour, L/h + + + + + litrePerLitre + Concentration, The ratio of the volume of a solute divided by the volume of the solution., L/L + + + + + litrePerSec + Volumetric flow rate, Volumetric flow rate, L/s + + + + + litreUncompensated + Volume, litre, with the value uncompensated for weather effects., L(uncompensated) + + + + + litreUncompensatedPerH + Volumetric flow rate, litres (uncompensated) per hour, L(uncompensated)/h + + + + + lm + Luminous flux, lumen (cd sr), Lm + + + + + lx + Illuminance lux, (lm/m²), L(uncompensated)/h + + + + + m2PerSec + Viscosity, meter squared / second, m²/s + + + + + m3compensated + Volume, cubic meter, with the value compensated for weather effects., m3(compensated) + + + + + m3compensatedPerH + Volumetric flow rate, compensated cubic meters per hour, ³(compensated)/h + + + + + m3PerH + Volumetric flow rate, cubic meters per hour, m³/h + + + + + m3PerSec + m3PerSec, cubic meters per second, m³/s + + + + + m3uncompensated + m3uncompensated, cubic meter, with the value uncompensated for weather effects., m3(uncompensated) + + + + + m3uncompensatedPerH + Volumetric flow rate, uncompensated cubic meters per hour, m³(uncompensated)/h + + + + + meCode + EndDeviceEvent, value to be interpreted as a EndDeviceEventCode, meCode + + + + + mol + Amount of substance, mole, mol + + + + + molPerKg + Concentration, Molality, the amount of solute in moles and the amount of solvent in kilograms., mol/kg + + + + + molPerM3 + Concentration, The amount of substance concentration, (c), the amount of solvent in moles divided by the volume of solution in m³., mol/ m³ + + + + + molPerMol + Concentration, Molar fraction (?), the ratio of the molar amount of a solute divided by the molar amount of the solution.,mol/mol + + + + + money + Monetary unit, Generic money (Note: Specific monetary units are identified the currency class)., ¤ + + + + + mPerM + Length, Ratio of length, m/m + + + + + mPerM3 + Fuel efficiency, meters / cubic meter, m/m³ + + + + + mPerSec + Velocity, meters per second (m/s), m/s + + + + + mPerSec2 + Acceleration, meters per second squared, m/s² + + + + + ohmM + resistivity, ? (rho), ?m + + + + + paA + Pressure, Pascal, absolute pressure, PaA + + + + + paG + Pressure, Pascal, gauge pressure, PaG + + + + + psiA + Pressure, Pounds per square inch, absolute, psiA + + + + + psiG + Pressure, Pounds per square inch, gauge, psiG + + + + + q + Quantity power, Q, Q + + + + + q45 + Quantity power, Q measured at 45º, Q45 + + + + + q45h + Quantity energy, Q measured at 45º, Q45h + + + + + q60 + Quantity power, Q measured at 60º, Q60 + + + + + q60h + Quantity energy, Qh measured at 60º, Q60h + + + + + qh + Quantity energy, Qh, Qh + + + + + radPerSec + Angular velocity, radians per second, rad/s + + + + + rev + Amount of rotation, Revolutions, rev + + + + + revPerSec + Rotational speed, Rotations per second, rev/s + + + + + secPerSec + Time, Ratio of time (can be combined with an multiplier prefix to show rates such as a clock drift rate, e.g. “µs/s”), s/s + + + + + sr + Solid angle, Steradian (m2/m2), sr + + + + + status + State, "1" = "true", "live", "on", "high", "set"; "0" = "false", "dead", "off", "low", "cleared". Note: A Boolean value is preferred but other values may be supported, status + + + + + sv + Doe equivalent, Sievert (J/kg), Sv + + + + + t + Magnetic flux density, Tesla (Wb/m2), T + + + + + therm + Energy, Therm, therm + + + + + timeStamp + Timestamp, time and date per ISO 8601 format, timeStamp + + + + + usGal + Volume, US gallons, Gal + + + + + usGalPerH + Volumetric flow rate, US gallons per hour, USGal/h + + + + + V2 + Volts squared, Volt squared (W2/A2), V² + + + + + V2h + volt-squared hour, Volt-squared-hours, V²h + + + + + VAhPerRev + Kh-Vah, apparent energy metering constant, VAh/rev + + + + + VArhPerRev + Kh-VArh, reactive energy metering constant, VArh/rev + + + + + VPerHz + Magnetic flux, Volts per Hertz, V/Hz + + + + + VPerV + Voltage, Ratio of voltages (e.g. mV/V), V/V + + + + + Vs + Volt seconds, Volt seconds (Ws/A), Vs + + + + + wb + Magnetic flux, Weber (V s), Wb + + + + + WhPerM3 + Wh/m3, energy per volume, Wh/m³ + + + + + WhPerRev + Kh-Wh, active energy metering constant, Wh/rev + + + + + wPerMK + Thermal conductivity, Watt/meter Kelvin, W/m K + + + + + WPerSec + Ramp rate, Watts per second, W/s + + + + + WPerVA + Power Factor, PF, W/VA + + + + + WPerW + Signal Strength, Ratio of power, W/W + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md b/reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md index 3b95184d..9bad7e27 100644 --- a/reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md +++ b/reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md @@ -1,22 +1,31 @@ -# ESPI 4.0 Schema Analysis Summary +# ESPI Schema Analysis Summary -**Analysis Date:** 2026-02-03 -**NAESB REQ.21 ESPI Version:** 4.0 -**Schema Publication Dates:** December 13-15, 2023 +**Analysis Date:** 2026-02-04 +**NAESB REQ.21 ESPI Versions:** 4.0 and 4.1 +**Schema Publication Dates:** 4.0: December 13-15, 2023 | 4.1: Pending --- ## Overview -This directory contains comprehensive analysis reports for the NAESB ESPI 4.0 XML schemas used in the OpenESPI-GreenButton-Java implementation. The schemas define the data model for energy usage data exchange under the Green Button standard. +This directory contains comprehensive analysis reports for the NAESB ESPI XML schemas used in the OpenESPI-GreenButton-Java implementation. The schemas define the data model for energy usage data exchange under the Green Button standard. ## Schema Files Analyzed +### ESPI 4.0 Schemas + | Schema | Namespace | Purpose | Report | |--------|-----------|---------|--------| | `espi.xsd` | `http://naesb.org/espi` | Energy usage data (UsagePoint, MeterReading, IntervalBlock, etc.) | [espi_enumerations.md](espi_enumerations.md) | | `customer.xsd` | `http://naesb.org/espi/customer` | Customer/PII data (Customer, CustomerAccount, Meter, etc.) | [customer_enumerations.md](customer_enumerations.md) | +### ESPI 4.1 Schemas + +| Schema | Namespace | Purpose | Notes | +|--------|-----------|---------|-------| +| `customer_4.1.xsd` | `http://naesb.org/espi/customer` | Customer/PII data (updated) | Flattened inheritance hierarchy | +| `atom.xsd` | `http://www.w3.org/2005/Atom` | Atom feed structure | Unchanged from 4.0 | + --- ## Combined Statistics @@ -102,26 +111,43 @@ This directory contains comprehensive analysis reports for the NAESB ESPI 4.0 XM ### Customer Domain Resources (customer.xsd) -| Resource | Base Type | Description | -|----------|-----------|-------------| -| Customer | Organisation | Full customer information | -| CustomerAccount | Document | Customer billing account | -| CustomerAgreement | Agreement | Service agreement | -| ServiceSupplier | Organisation | Utility or energy provider | -| ServiceLocation | WorkLocation | Physical service location | -| Meter | EndDevice | Physical meter device | -| EndDevice | AssetContainer | End device (base for Meter) | -| Statement | Document | Billing statement | -| DemandResponseProgram | IdentifiedObject | DR program enrollment | -| PricingStructure | Document | Pricing/rate structure | -| ProgramDateIdMappings | IdentifiedObject | Program date mappings | -| TimeConfiguration | IdentifiedObject | Timezone and DST settings | -| UsagePoint | IdentifiedObject | Usage point (customer context) | +| Resource | 4.0 Base Type | 4.1 Base Type | Description | +|----------|---------------|---------------|-------------| +| Customer | OrganisationRole | IdentifiedObject | Full customer information | +| CustomerAccount | Document | IdentifiedObject | Customer billing account | +| CustomerAgreement | Agreement | IdentifiedObject | Service agreement | +| ServiceSupplier | OrganisationRole | IdentifiedObject | Utility or energy provider | +| ServiceLocation | WorkLocation | IdentifiedObject | Physical service location | +| Meter | EndDevice | IdentifiedObject | Physical meter device | +| EndDevice | AssetContainer | IdentifiedObject | End device (base for Meter) | +| Statement | IdentifiedObject | IdentifiedObject | Billing statement (unchanged) | +| ProgramDateIdMappings | IdentifiedObject | IdentifiedObject | Program date mappings (unchanged) | +| TimeConfiguration | IdentifiedObject | IdentifiedObject | Timezone and DST settings (unchanged) | + +### Supporting Types Base Changes (4.0 → 4.1) + +| Type | 4.0 Base Type | 4.1 Base Type | Notes | +|------|---------------|---------------|-------| +| Asset | IdentifiedObject | Object | Downgraded | +| Location | IdentifiedObject | Object | Downgraded | +| Organisation | IdentifiedObject | Object | Downgraded | +| Document | IdentifiedObject | Object | Downgraded | +| DemandResponseProgram | (none) | Object | Added base type (4.0 omission) | + +### Types Commented Out in 4.1 + +| Type | 4.0 Base Type | Status in 4.1 | +|------|---------------|---------------| +| AssetContainer | Asset | Commented out | +| OrganisationRole | IdentifiedObject | Commented out | +| WorkLocation | Location | Commented out | --- ## Type Inheritance Hierarchy +### ESPI 4.0 Hierarchy + ``` Object └── IdentifiedObject @@ -136,27 +162,57 @@ Object ├── ApplicationInformation ├── Subscription ├── RetailCustomer - ├── DemandResponseProgram + ├── Statement ├── ProgramDateIdMappings - └── Document - ├── CustomerAccount - ├── PricingStructure - ├── Statement - └── Agreement - └── CustomerAgreement - -Organisation (extends IdentifiedObject) -├── Customer -└── ServiceSupplier - -Location (extends IdentifiedObject) -└── WorkLocation - └── ServiceLocation - -Asset (extends IdentifiedObject) -└── AssetContainer - └── EndDevice - └── Meter + ├── Document + │ ├── CustomerAccount + │ └── Agreement + │ └── CustomerAgreement + ├── Organisation + │ └── OrganisationRole + │ ├── Customer + │ └── ServiceSupplier + ├── Location + │ └── WorkLocation + │ └── ServiceLocation + └── Asset + └── AssetContainer + └── EndDevice + └── Meter +``` + +### ESPI 4.1 Hierarchy (Flattened) + +``` +Object +├── IdentifiedObject +│ ├── UsagePoint +│ ├── MeterReading +│ ├── IntervalBlock +│ ├── ReadingType +│ ├── ElectricPowerQualitySummary +│ ├── UsageSummary +│ ├── TimeConfiguration +│ ├── Authorization +│ ├── ApplicationInformation +│ ├── Subscription +│ ├── RetailCustomer +│ ├── Statement +│ ├── ProgramDateIdMappings +│ ├── Customer (flattened, contains Organisation element) +│ ├── CustomerAccount (flattened, contains Document element) +│ ├── CustomerAgreement (flattened, contains Agreement element) +│ ├── ServiceSupplier (flattened, contains Organisation element) +│ ├── ServiceLocation (flattened, contains Location element) +│ ├── EndDevice (flattened, contains Asset element) +│ └── Meter (flattened, contains EndDevice element) +├── Organisation (downgraded from IdentifiedObject) +├── Location (downgraded from IdentifiedObject) +├── Asset (downgraded from IdentifiedObject) +└── Document (downgraded from IdentifiedObject) + └── Agreement + +Types commented out in 4.1: AssetContainer, OrganisationRole, WorkLocation ``` --- @@ -184,7 +240,8 @@ ESPI uses a union pattern for numeric enumerations that allows both defined valu ### IdentifiedObject Base Type All major resources extend `IdentifiedObject` which provides: -- `description` (String512) - Human-readable description +- `batchItemInfo` (BatchItemInfo) - Batch processing information +- `name` (name) - Object name [DEPRECATED] - Extension capability from `Object` base type ### Time Representation @@ -202,11 +259,14 @@ All major resources extend `IdentifiedObject` which provides: ## Schema Location -The schema files are located at: ``` -openespi-common/src/main/resources/schema/ESPI_4.0/ -├── espi.xsd (usage/energy data) -└── customer.xsd (customer/PII data) +openespi-common/src/main/resources/schema/ +├── ESPI_4.0/ +│ ├── espi.xsd (usage/energy data) +│ └── customer.xsd (customer/PII data) +└── ESPI_4.1/ + ├── atom.xsd (Atom feed structure) + └── customer_4.1.xsd (customer/PII data - flattened hierarchy) ``` --- @@ -243,4 +303,37 @@ When implementing Java enums for ESPI types: --- -*Generated from NAESB REQ.21 ESPI Version 4.0 schemas (December 2023)* +## ESPI 4.0 → 4.1 Migration Notes + +### Composition Pattern in 4.1 + +ESPI 4.1 adopts a **composition over inheritance** pattern. Resources that previously extended deep inheritance chains now: +1. Extend `IdentifiedObject` directly +2. Contain the former parent type as an embedded element + +| Resource | 4.0 Inheritance | 4.1 Composition | +|----------|-----------------|-----------------| +| Customer | extends OrganisationRole | contains `organisation` element | +| CustomerAccount | extends Document | contains `document` element | +| CustomerAgreement | extends Agreement | contains `agreement` element | +| ServiceSupplier | extends OrganisationRole | contains `organisation` element | +| ServiceLocation | extends WorkLocation | contains `location` element | +| EndDevice | extends AssetContainer | contains `asset` element | +| Meter | extends EndDevice | contains `endDevice` element | + +### Enumeration Types + +**Important:** All enumeration types are **unchanged** between ESPI 4.0 and 4.1: +- Same base types (UInt16, xs:string, Int16, etc.) +- Same values +- Same semantics + +### Breaking Changes + +1. XML documents valid in 4.0 may not be valid in 4.1 due to structural changes +2. Types that relied on inherited properties must now access them through embedded elements +3. Asset, Location, Organisation, Document no longer extend IdentifiedObject + +--- + +*Generated from NAESB REQ.21 ESPI Version 4.0 (December 2023) and 4.1 (Pending) schemas* diff --git a/reports/verification/customer_enumerations.md b/reports/verification/customer_enumerations.md index 56c57417..d6b0b587 100644 --- a/reports/verification/customer_enumerations.md +++ b/reports/verification/customer_enumerations.md @@ -1,8 +1,8 @@ -# ESPI 4.0 Customer Schema Analysis Report +# ESPI Customer Schema Analysis Report -**Schema Version:** NAESB REQ.21 ESPI Version 4.0.20231215 -**Published:** December 15, 2023 +**Schema Versions:** NAESB REQ.21 ESPI 4.0 (December 2023) and 4.1 (Pending) **Namespace:** `http://naesb.org/espi/customer` +**Analysis Date:** 2026-02-04 --- @@ -16,6 +16,7 @@ 6. [Complex Types (Resources)](#6-complex-types-resources) 7. [Complex Types (Supporting)](#7-complex-types-supporting) 8. [Global Elements](#8-global-elements) +9. [ESPI 4.0 → 4.1 Changes](#9-espi-40--41-changes) --- @@ -75,23 +76,21 @@ | Value | Name | Description | |-------|------|-------------| -| 0 | Created | Resource was created | -| 1 | Deleted | Resource was deleted | -| 2 | Updated | Resource was updated | -| 3 | Unspecified | Operation unspecified | +| 0 | Create | Create operation | +| 1 | Read | Read operation | +| 2 | Update | Update operation | +| 3 | Delete | Delete operation | --- #### EnrollmentStatus -**Base:** `UInt16` (union) +**Base:** `xs:string` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | Enrolled | Customer is enrolled | -| 1 | EnrollmentPending | Enrollment is pending | -| 2 | UnEnrolled | Customer is not enrolled | -| 3 | UnEnrolledPending | Unenrollment is pending | -| 4 | Enrolling | Customer is in process of enrolling | +| Value | Description | +|-------|-------------| +| `unenrolled` | Currently NOT enrolled in the Demand Response program | +| `enrolled` | Currently enrolled in the Demand Response program | +| `enrolledPending` | Currently pending enrollment in the Demand Response program | --- @@ -120,88 +119,81 @@ HTTP-style status codes for ESPI operations. ### 5.2 Customer and Service Enumerations #### CustomerKind -**Base:** `UInt16` (union) +**Base:** `xs:string` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | energyServiceScheduler | Energy service scheduler customer | -| 1 | energyServiceSupplier | Energy service supplier customer | -| 2 | other | Other customer type | -| 3 | commercialIndustrial | Commercial or industrial customer | -| 4 | internalUse | Internal use customer | -| 5 | pumpingLoad | Pumping load customer | -| 6 | residential | Residential customer | -| 7 | residentialAndCommercial | Residential and commercial customer | -| 8 | residentialAndStreetlight | Residential and streetlight customer | -| 9 | residentialFarmService | Residential farm service customer | -| 10 | residentialStreetlightOthers | Residential streetlight and others | -| 11 | streetLight | Streetlight customer | -| 12 | energyServiceConsumer | Energy service consumer | -| 13 | gridOperator | Grid operator | -| 14 | windMachine | Wind machine customer | +| Value | Description | +|-------|-------------| +| `residential` | Residential customer | +| `residentialAndCommercial` | Residential and commercial customer | +| `residentialAndStreetlight` | Residential and streetlight customer | +| `residentialStreetlightOthers` | Residential streetlight and others | +| `residentialFarmService` | Residential farm service customer | +| `commercialIndustrial` | Commercial or industrial customer | +| `pumpingLoad` | Pumping load customer | +| `windMachine` | Wind machine customer | +| `energyServiceSupplier` | Energy service supplier customer | +| `energyServiceScheduler` | Energy service scheduler customer | +| `enterprise` | Enterprise customer | --- #### SupplierKind -**Base:** `UInt16` (union) +**Base:** `xs:string` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | utility | Traditional utility supplier | -| 1 | municipality | Municipal utility | -| 2 | privateEnterprise | Private enterprise supplier | -| 3 | cooperative | Cooperative utility | -| 4 | retailer | Retail energy supplier | -| 5 | other | Other supplier type | +| Value | Description | +|-------|-------------| +| `utility` | Traditional utility supplier | +| `retailer` | Retail energy supplier | +| `other` | Other supplier type | +| `lse` | Load serving entity | +| `mdma` | Meter data management agent | --- #### ServiceKind -**Base:** `UInt16` (union) +**Base:** `xs:string` (restriction) -| Value | Name | Description | -|-------|------|-------------| -| 0 | electricity | Electric service | -| 1 | gas | Natural gas service | -| 2 | water | Water service | -| 3 | time | Time service | -| 4 | heat | Heat service | -| 5 | refuse | Refuse/waste service | -| 6 | sewerage | Sewerage service | -| 7 | rates | Rates information | -| 8 | tvLicense | TV license service | -| 9 | internet | Internet service | -| 10 | other | Other service type | +| Value | Description | +|-------|-------------| +| `electricity` | Electric service | +| `gas` | Natural gas service | +| `water` | Water service | +| `time` | Time service | +| `heat` | Heat service | +| `refuse` | Refuse/waste service | +| `sewerage` | Sewerage service | +| `rates` | Rates information | +| `tvLicence` | TV license service | +| `internet` | Internet service | --- #### RevenueKind -**Base:** `UInt16` (union) +**Base:** `xs:string` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | fees | Revenue from fees | -| 1 | rates | Revenue from rates | -| 2 | cancellationFees | Revenue from cancellation fees | -| 3 | lateFees | Revenue from late fees | -| 4 | connectorFees | Revenue from connector fees | -| 5 | rebates | Rebates (negative revenue) | -| 6 | other | Other revenue type | +| Value | Description | +|-------|-------------| +| `residential` | Residential revenue | +| `nonResidential` | Non-residential revenue | +| `commercial` | Commercial revenue | +| `industrial` | Industrial revenue | +| `irrigation` | Irrigation revenue | +| `streetLight` | Streetlight revenue | +| `other` | Other revenue type | --- ### 5.3 Notification and Communication Enumerations #### NotificationMethodKind -**Base:** `UInt16` (union) +**Base:** `xs:string` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | email | Notification via email | -| 1 | inPerson | In-person notification | -| 2 | phone | Notification via phone call | -| 3 | postal | Notification via postal mail | -| 4 | sms | Notification via SMS text message | +| Value | Description | +|-------|-------------| +| `call` | Contacted by phone by customer service representative | +| `email` | Trouble reported by email | +| `letter` | Trouble reported by letter | +| `other` | Trouble reported by other means | --- @@ -233,30 +225,29 @@ MIME media types for content negotiation. ### 5.4 Program and Date Enumerations #### ProgramDateKind -**Base:** `UInt16` (union) +**Base:** `String64` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | signUp | Sign-up date | -| 1 | cancel | Cancellation date | -| 2 | enroll | Enrollment date | -| 3 | deferred | Deferred date | +| Value | Description | +|-------|-------------| +| `CUST_DR_PROGRAM_ENROLLMENT_DATE` | DR program enrollment date | +| `CUST_DR_PROGRAM_DE_ENROLLMENT_DATE` | DR program de-enrollment date | +| `CUST_DR_PROGRAM_TERM_DATE_REGARDLESS_FINANCIAL` | Program termination date regardless of financial | +| `CUST_DR_PROGRAM_TERM_DATE_WITHOUT_FINANCIAL` | Program termination date without financial | --- ### 5.5 Meter and Measurement Enumerations #### MeterMultiplierKind -**Base:** `UInt16` (union) +**Base:** `xs:string` (union) -| Value | Name | Description | -|-------|------|-------------| -| 0 | kH | Test dial multiplier | -| 1 | kE | Element test multiplier | -| 2 | kP | Potential transformer multiplier | -| 3 | kC | Current transformer multiplier | -| 4 | kR | Register multiplier | -| 5 | transformerRatio | Transformer ratio | +| Value | Description | +|-------|-------------| +| `kH` | Test dial multiplier | +| `transformerRatio` | Transformer ratio | +| `kR` | Register multiplier | +| `kE` | Element test multiplier | +| `ctRatio` | Current transformer ratio | --- @@ -1104,4 +1095,47 @@ The customer.xsd schema defines 45 global elements: --- -*Generated from NAESB REQ.21 ESPI Version 4.0.20231215 customer.xsd schema* +## 9. ESPI 4.0 → 4.1 Changes + +### 9.1 Inheritance Flattening + +ESPI 4.1 adopts a composition over inheritance pattern. Resources now extend `IdentifiedObject` directly and contain their former parent types as elements. + +| Resource | 4.0 Base Type | 4.1 Base Type | Composition Element | +|----------|---------------|---------------|---------------------| +| Customer | OrganisationRole | IdentifiedObject | `organisation` | +| CustomerAccount | Document | IdentifiedObject | `document` | +| CustomerAgreement | Agreement | IdentifiedObject | `agreement` | +| ServiceSupplier | OrganisationRole | IdentifiedObject | `organisation` | +| ServiceLocation | WorkLocation | IdentifiedObject | `location` | +| EndDevice | AssetContainer | IdentifiedObject | `asset` | +| Meter | EndDevice | IdentifiedObject | `endDevice` | + +### 9.2 Supporting Types Base Changes + +| Type | 4.0 Base | 4.1 Base | Notes | +|------|----------|----------|-------| +| Asset | IdentifiedObject | Object | Downgraded | +| Location | IdentifiedObject | Object | Downgraded | +| Organisation | IdentifiedObject | Object | Downgraded | +| Document | IdentifiedObject | Object | Downgraded | +| DemandResponseProgram | (none) | Object | Added base type (4.0 omission) | + +### 9.3 Types Commented Out in 4.1 + +| Type | Status | +|------|--------| +| AssetContainer | Commented out | +| OrganisationRole | Commented out | +| WorkLocation | Commented out | + +### 9.4 Enumeration Types + +**All enumeration types are unchanged between ESPI 4.0 and 4.1:** +- Same base types +- Same values +- Same semantics + +--- + +*Generated from NAESB REQ.21 ESPI Version 4.0 (December 2023) and 4.1 (Pending) customer schemas* diff --git a/scripts/README_ORCHESTRATOR.md b/scripts/README_ORCHESTRATOR.md new file mode 100644 index 00000000..18f571bc --- /dev/null +++ b/scripts/README_ORCHESTRATOR.md @@ -0,0 +1,537 @@ +# ESPI Verification Orchestrator + +Automated schema verification tool with intelligent Claude model selection. + +## Overview + +The ESPI Verification Orchestrator automatically selects the optimal Claude model (Opus, Sonnet, or Haiku) based on task complexity, executes verification tasks, and tracks costs. + +**Key Features:** +- ✅ Automatic model selection (Opus for complex, Sonnet for standard, Haiku for batch) +- ✅ Cost tracking and reporting +- ✅ Progress visualization +- ✅ Batch processing capabilities +- ✅ Automatic report generation + +## Installation + +```bash +# Install dependencies +pip install anthropic pyyaml click rich + +# Set API key +export ANTHROPIC_API_KEY="your-api-key-here" + +# Make script executable +chmod +x scripts/espi-verification-orchestrator.py +``` + +## Quick Start + +### 1. Analyze XSD Schema (Opus) + +Extract all enumerations from the schema: + +```bash +./scripts/espi-verification-orchestrator.py analyze-schema --schema espi +``` + +**Model Used**: Claude Opus 4.5 (complex analysis) +**Output**: `reports/verification/espi_enumerations.md` +**Est. Cost**: ~$1.50 + +### 2. Generate Missing Enum (Sonnet) + +Generate a Java enum from XSD definition: + +```bash +./scripts/espi-verification-orchestrator.py generate-enum AccumulationKind --schema espi --package usage.enums +``` + +**Model Used**: Claude Sonnet 4.5 (code generation) +**Output**: `openespi-common/src/main/java/.../usage/enums/AccumulationKind.java` +**Est. Cost**: ~$0.15 + +### 3. Verify Entity (Sonnet or Opus) + +Verify an entity matches XSD schema: + +```bash +# Standard entity (Sonnet) +./scripts/espi-verification-orchestrator.py verify-entity UsagePoint --schema espi + +# Complex entity (Opus) +./scripts/espi-verification-orchestrator.py verify-entity UsagePoint --schema espi --complexity complex +``` + +**Model Used**: Claude Sonnet 4.5 or Opus 4.5 +**Output**: `reports/verification/UsagePoint_verification.md` +**Est. Cost**: $0.25 (Sonnet) or $1.50 (Opus) + +### 4. Batch Verify Enums (Haiku) + +Verify all enums in a directory: + +```bash +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory usage/enums +``` + +**Model Used**: Claude Haiku (cost-effective batch processing) +**Output**: Individual verification results +**Est. Cost**: ~$0.05 per enum + +### 5. View Cost Summary + +```bash +./scripts/espi-verification-orchestrator.py show-costs +``` + +## Command Reference + +### analyze-schema + +Analyze XSD schema and extract enumerations. + +```bash +./scripts/espi-verification-orchestrator.py analyze-schema [OPTIONS] + +Options: + --schema TEXT Schema to analyze (espi or customer) [default: espi] +``` + +**Model**: Claude Opus 4.5 (complex analysis) + +### generate-enum + +Generate Java enum from XSD definition. + +```bash +./scripts/espi-verification-orchestrator.py generate-enum ENUM_NAME [OPTIONS] + +Arguments: + ENUM_NAME Name of the enum to generate + +Options: + --schema TEXT Source schema (espi or customer) [default: espi] + --package TEXT Target package [default: usage.enums] +``` + +**Model**: Claude Sonnet 4.5 (code generation) + +**Examples:** +```bash +# Generate usage domain enum +./scripts/espi-verification-orchestrator.py generate-enum AccumulationKind + +# Generate customer domain enum +./scripts/espi-verification-orchestrator.py generate-enum MediaType --schema customer --package customer.enums + +# Generate shared enum +./scripts/espi-verification-orchestrator.py generate-enum Currency --package common +``` + +### verify-entity + +Verify entity against XSD schema. + +```bash +./scripts/espi-verification-orchestrator.py verify-entity ENTITY_NAME [OPTIONS] + +Arguments: + ENTITY_NAME Name of the entity (without 'Entity' suffix) + +Options: + --schema TEXT Source schema (espi or customer) [default: espi] + --complexity TEXT standard or complex [default: standard] +``` + +**Model**: +- `standard` → Claude Sonnet 4.5 +- `complex` → Claude Opus 4.5 + +**Examples:** +```bash +# Standard verification +./scripts/espi-verification-orchestrator.py verify-entity MeterReading + +# Complex verification (for entities with many relationships) +./scripts/espi-verification-orchestrator.py verify-entity ReadingType --complexity complex + +# Customer entity +./scripts/espi-verification-orchestrator.py verify-entity CustomerAccount --schema customer +``` + +### batch-verify-enums + +Batch verify all enums in a directory. + +```bash +./scripts/espi-verification-orchestrator.py batch-verify-enums [OPTIONS] + +Options: + --directory TEXT Enum directory to verify [default: usage/enums] +``` + +**Model**: Claude Haiku (cost-effective) + +**Examples:** +```bash +# Verify usage enums +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory usage/enums + +# Verify customer enums +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory customer/enums +``` + +### show-costs + +Display cost summary for all executed tasks. + +```bash +./scripts/espi-verification-orchestrator.py show-costs +``` + +**Output:** +``` +┌───────────┬──────────┬────────┐ +│ Model │ Cost │ Tasks │ +├───────────┼──────────┼────────┤ +│ OPUS │ $5.25 │ 3 │ +│ SONNET │ $12.50 │ 45 │ +│ HAIKU │ $2.30 │ 28 │ +└───────────┴──────────┴────────┘ + +Total Cost: $20.05 +Total Tasks: 76 +Input Tokens: 125,432 +Output Tokens: 48,901 +``` + +## Workflow Examples + +### Phase 0: Enumeration Migration + +**Step 1: Analyze Schema (one-time)** +```bash +# Analyze espi.xsd (Opus - $1.50) +./scripts/espi-verification-orchestrator.py analyze-schema --schema espi + +# Analyze customer.xsd (Opus - $1.50) +./scripts/espi-verification-orchestrator.py analyze-schema --schema customer +``` + +**Step 2: Generate Missing Enums in Batch** +```bash +# Create a batch generation script +cat > scripts/generate-missing-enums.sh <<'EOF' +#!/bin/bash + +ENUMS=( + "AccumulationKind" + "CommodityKind" + "DataQualifierKind" + "FlowDirectionKind" + "MeasurementKind" + "TimeAttributeKind" +) + +for enum in "${ENUMS[@]}"; do + ./scripts/espi-verification-orchestrator.py generate-enum "$enum" + sleep 2 # Rate limiting +done +EOF + +chmod +x scripts/generate-missing-enums.sh +./scripts/generate-missing-enums.sh +``` + +**Cost**: ~$0.15 × 30 enums = ~$4.50 + +**Step 3: Verify Generated Enums** +```bash +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory usage/enums +``` + +**Cost**: ~$0.05 × 30 enums = ~$1.50 + +**Total Phase 0 Cost**: ~$9.00 (vs ~$50+ manual work) + +### Phase 2: Entity Verification + +**Verify Core Entities:** +```bash +# Batch verify script +cat > scripts/verify-usage-entities.sh <<'EOF' +#!/bin/bash + +ENTITIES=( + "UsagePoint" + "MeterReading" + "IntervalBlock" + "ReadingType:complex" # Mark complex entities + "IntervalReading" + "ReadingQuality" +) + +for entity_spec in "${ENTITIES[@]}"; do + IFS=':' read -r entity complexity <<< "$entity_spec" + + if [ "$complexity" = "complex" ]; then + ./scripts/espi-verification-orchestrator.py verify-entity "$entity" --complexity complex + else + ./scripts/espi-verification-orchestrator.py verify-entity "$entity" + fi + + sleep 2 +done + +# Show total costs +./scripts/espi-verification-orchestrator.py show-costs +EOF + +chmod +x scripts/verify-usage-entities.sh +./scripts/verify-usage-entities.sh +``` + +**Cost**: +- 5 standard entities (Sonnet): ~$0.25 × 5 = $1.25 +- 1 complex entity (Opus): ~$1.50 +- **Total**: ~$2.75 + +## Model Selection Rules + +The orchestrator automatically selects models based on task type: + +| Task Type | Model | Reasoning | +|-----------|-------|-----------| +| **XSD Schema Analysis** | Opus 4.5 | Complex pattern recognition, deep understanding | +| **Analyze Complex Entity** | Opus 4.5 | Multiple relationships, complex mappings | +| **Generate Enum** | Sonnet 4.5 | Code generation, follows patterns well | +| **Verify Entity (standard)** | Sonnet 4.5 | Good balance for field comparison | +| **Generate Fixes** | Sonnet 4.5 | Code transformation | +| **Generate Tests** | Sonnet 4.5 | Understands test patterns | +| **Verify Enum** | Haiku | Simple comparison task | +| **Batch Operations** | Haiku | Repetitive, cost-effective | + +### Override Model Selection + +```python +# In code +result = orchestrator.execute_task( + TaskType.VERIFY_ENTITY, + prompt, + force_model=ClaudeModel.OPUS # Force Opus instead of default Sonnet +) +``` + +## Configuration + +Edit `scripts/verification-config.yaml`: + +```yaml +# Cost limits +cost_limits: + per_task_warn: 2.00 # Warn if task > $2 + total_warn: 100.00 # Warn if total > $100 + total_stop: 500.00 # Stop if total > $500 + +# Model overrides (force specific models) +model_overrides: + verify_entity: opus # Always use Opus for entity verification +``` + +## Output Files + +All outputs are saved to `reports/verification/`: + +``` +reports/verification/ +├── espi_enumerations.md # Schema analysis +├── AccumulationKind.java # Generated enum (copied to src/) +├── UsagePoint_verification.md # Raw verification output +├── UsagePoint_verification.json # Task metadata (tokens, cost, etc.) +├── UsagePoint_report.md # Formatted verification report +└── costs_summary.json # Overall cost tracking +``` + +## Integration with Other Tools + +### Use with IntelliJ External Tools + +Add to IntelliJ → Settings → Tools → External Tools: + +``` +Name: Verify Entity (Auto-model) +Program: $ProjectFileDir$/scripts/espi-verification-orchestrator.py +Arguments: verify-entity $FileNameWithoutExtension$ +Working directory: $ProjectFileDir$ +``` + +### Use in GitHub Actions + +```yaml +- name: Verify Changed Entities + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + run: | + for file in $(git diff --name-only origin/main | grep Entity.java); do + entity=$(basename "$file" Entity.java) + ./scripts/espi-verification-orchestrator.py verify-entity "$entity" + done +``` + +### Use in Pre-commit Hook + +```bash +#!/bin/bash +# .git/hooks/pre-commit + +CHANGED_ENTITIES=$(git diff --cached --name-only | grep Entity.java) + +if [ -n "$CHANGED_ENTITIES" ]; then + for file in $CHANGED_ENTITIES; do + entity=$(basename "$file" Entity.java) + ./scripts/espi-verification-orchestrator.py verify-entity "$entity" --complexity standard + done +fi +``` + +## Cost Estimation + +### Per-Task Estimates + +| Task | Model | Est. Cost | +|------|-------|-----------| +| Analyze schema | Opus | $1.50 | +| Generate enum | Sonnet | $0.15 | +| Verify enum | Haiku | $0.05 | +| Verify entity (standard) | Sonnet | $0.25 | +| Verify entity (complex) | Opus | $1.50 | +| Generate fixes | Sonnet | $0.30 | +| Generate tests | Sonnet | $0.25 | + +### Project Totals + +**Phase 0 (Enumerations):** +- Analyze schemas: 2 × $1.50 = $3.00 +- Generate 30 enums: 30 × $0.15 = $4.50 +- Verify 37 enums: 37 × $0.05 = $1.85 +- **Total**: ~$9.35 + +**Phases 1-3 (Entities & Types):** +- Verify 10 complex: 10 × $1.50 = $15.00 +- Verify 55 standard: 55 × $0.25 = $13.75 +- Generate fixes: 20 × $0.30 = $6.00 +- **Total**: ~$34.75 + +**Grand Total**: ~$44.10 (vs $225-315 without optimization) + +## Troubleshooting + +### API Key Not Set + +``` +Error: ANTHROPIC_API_KEY environment variable not set +``` + +**Fix:** +```bash +export ANTHROPIC_API_KEY="your-key-here" +# Or add to ~/.bashrc or ~/.zshrc +``` + +### Rate Limiting + +``` +Error: Rate limit exceeded +``` + +**Fix:** Add delays between tasks: +```bash +for entity in UsagePoint MeterReading IntervalBlock; do + ./scripts/espi-verification-orchestrator.py verify-entity "$entity" + sleep 5 # 5 second delay +done +``` + +### Cost Limit Exceeded + +``` +Warning: Total cost ($105.50) exceeded limit ($100.00) +``` + +**Fix:** Increase limit in `verification-config.yaml`: +```yaml +cost_limits: + total_warn: 200.00 +``` + +## Advanced Usage + +### Python API + +Use the orchestrator programmatically: + +```python +from espi_verification_orchestrator import ESPIVerificationOrchestrator, TaskType +from pathlib import Path + +# Initialize +orch = ESPIVerificationOrchestrator(Path.cwd()) + +# Generate enum +result = orch.generate_enum("AccumulationKind", "espi", "usage.enums") +if result.success: + print(f"Generated! Cost: ${result.cost:.2f}") + +# Verify entity +result = orch.verify_entity("UsagePoint", "espi", "standard") +print(result.output) + +# Show costs +orch.cost_tracker.print_summary() +``` + +### Custom Task + +```python +from espi_verification_orchestrator import TaskType, ClaudeModel + +# Execute custom task +result = orch.execute_task( + task_type=TaskType.GENERATE_FIXES, + prompt="Fix these issues in UsagePointEntity: ...", + max_tokens=6000, + force_model=ClaudeModel.OPUS # Override default +) +``` + +## Best Practices + +1. **Start with Analysis**: Run `analyze-schema` once at the beginning +2. **Batch Similar Tasks**: Use batch commands for enums +3. **Use Standard First**: Try `--complexity standard` before `complex` +4. **Check Costs Regularly**: Run `show-costs` periodically +5. **Save Outputs**: All outputs are auto-saved for review +6. **Review Before Applying**: Check generated code before committing + +## Next Steps + +After running verifications: + +1. Review generated reports in `reports/verification/` +2. Apply fixes to entities +3. Run tests: `mvn test` +4. Commit changes +5. Run orchestrator again to verify fixes + +## Support + +For issues or questions: +- Check logs in `reports/verification/*.json` +- Review cost summary: `show-costs` +- See ISSUE_101_TOOLING_GUIDE.md for more details + +--- + +**Happy Verifying!** 🚀 diff --git a/scripts/espi-verification-orchestrator.py b/scripts/espi-verification-orchestrator.py new file mode 100755 index 00000000..45a346ce --- /dev/null +++ b/scripts/espi-verification-orchestrator.py @@ -0,0 +1,716 @@ +#!/usr/bin/env python3 +""" +ESPI 4.0 Schema Verification Orchestrator + +Automatically selects the optimal Claude model for each verification task +and executes the plan with intelligent model switching. + +Usage: + ./scripts/espi-verification-orchestrator.py --task verify-entity --entity UsagePoint + ./scripts/espi-verification-orchestrator.py --task generate-enum --enum AccumulationKind + ./scripts/espi-verification-orchestrator.py --batch-verify-enums + ./scripts/espi-verification-orchestrator.py --phase 0 --auto + +Requirements: + pip install anthropic pyyaml click rich +""" + +import os +import sys +import json +import yaml +from pathlib import Path +from typing import Dict, List, Optional, Tuple +from dataclasses import dataclass, asdict +from enum import Enum +import anthropic +import click +from rich.console import Console +from rich.table import Table +from rich.progress import Progress, SpinnerColumn, TextColumn +from rich.panel import Panel +from rich import print as rprint + +# Initialize +console = Console() +client = anthropic.Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY")) + +# Model definitions +class ClaudeModel(Enum): + OPUS = "claude-opus-4-5-20251101" + SONNET = "claude-sonnet-4-5-20250929" + HAIKU = "claude-haiku-4-20250514" + +# Cost per million tokens (approximate) +MODEL_COSTS = { + ClaudeModel.OPUS: {"input": 15.00, "output": 75.00}, + ClaudeModel.SONNET: {"input": 3.00, "output": 15.00}, + ClaudeModel.HAIKU: {"input": 0.25, "output": 1.25} +} + +# Task type definitions +class TaskType(Enum): + # Phase 0 - Enumeration tasks + ANALYZE_XSD_SCHEMA = "analyze_xsd_schema" + EXTRACT_ENUMERATIONS = "extract_enumerations" + GENERATE_ENUM = "generate_enum" + VERIFY_ENUM = "verify_enum" + MOVE_ENUM = "move_enum" + BATCH_VERIFY_ENUMS = "batch_verify_enums" + + # Phase 1-3 - Entity verification tasks + ANALYZE_COMPLEX_TYPE = "analyze_complex_type" + VERIFY_ENTITY = "verify_entity" + VERIFY_EMBEDDABLE = "verify_embeddable" + ANALYZE_DISCREPANCIES = "analyze_discrepancies" + GENERATE_FIXES = "generate_fixes" + GENERATE_MIGRATION = "generate_migration" + GENERATE_TESTS = "generate_tests" + + # Documentation and reporting + GENERATE_REPORT = "generate_report" + CODE_REVIEW = "code_review" + +# Model selection rules +TASK_MODEL_MAPPING = { + # Use Opus for complex analysis and architecture decisions + TaskType.ANALYZE_XSD_SCHEMA: ClaudeModel.OPUS, + TaskType.ANALYZE_COMPLEX_TYPE: ClaudeModel.OPUS, + TaskType.ANALYZE_DISCREPANCIES: ClaudeModel.OPUS, + + # Use Sonnet for standard code generation and verification + TaskType.EXTRACT_ENUMERATIONS: ClaudeModel.SONNET, + TaskType.GENERATE_ENUM: ClaudeModel.SONNET, + TaskType.VERIFY_ENTITY: ClaudeModel.SONNET, + TaskType.VERIFY_EMBEDDABLE: ClaudeModel.SONNET, + TaskType.GENERATE_FIXES: ClaudeModel.SONNET, + TaskType.GENERATE_MIGRATION: ClaudeModel.SONNET, + TaskType.GENERATE_TESTS: ClaudeModel.SONNET, + TaskType.GENERATE_REPORT: ClaudeModel.SONNET, + TaskType.CODE_REVIEW: ClaudeModel.SONNET, + + # Use Haiku for repetitive verification and simple tasks + TaskType.VERIFY_ENUM: ClaudeModel.HAIKU, + TaskType.BATCH_VERIFY_ENUMS: ClaudeModel.HAIKU, + TaskType.MOVE_ENUM: ClaudeModel.HAIKU, +} + +@dataclass +class TaskResult: + task_type: TaskType + model_used: ClaudeModel + success: bool + output: str + input_tokens: int + output_tokens: int + cost: float + duration_seconds: float + error: Optional[str] = None + +class CostTracker: + """Track API costs across tasks""" + + def __init__(self): + self.total_cost = 0.0 + self.costs_by_model = {model: 0.0 for model in ClaudeModel} + self.task_history: List[TaskResult] = [] + + def calculate_cost(self, model: ClaudeModel, input_tokens: int, output_tokens: int) -> float: + """Calculate cost for a single API call""" + costs = MODEL_COSTS[model] + input_cost = (input_tokens / 1_000_000) * costs["input"] + output_cost = (output_tokens / 1_000_000) * costs["output"] + return input_cost + output_cost + + def record_task(self, result: TaskResult): + """Record a completed task""" + self.task_history.append(result) + self.total_cost += result.cost + self.costs_by_model[result.model_used] += result.cost + + def get_summary(self) -> Dict: + """Get cost summary""" + return { + "total_cost": self.total_cost, + "total_tasks": len(self.task_history), + "costs_by_model": { + model.name: cost + for model, cost in self.costs_by_model.items() + }, + "total_input_tokens": sum(t.input_tokens for t in self.task_history), + "total_output_tokens": sum(t.output_tokens for t in self.task_history), + } + + def print_summary(self): + """Print formatted cost summary""" + summary = self.get_summary() + + table = Table(title="Cost Summary") + table.add_column("Model", style="cyan") + table.add_column("Cost", style="green") + table.add_column("Tasks", style="yellow") + + for model in ClaudeModel: + model_tasks = [t for t in self.task_history if t.model_used == model] + table.add_row( + model.name, + f"${self.costs_by_model[model]:.2f}", + str(len(model_tasks)) + ) + + console.print(table) + console.print(f"\n[bold green]Total Cost: ${summary['total_cost']:.2f}[/bold green]") + console.print(f"Total Tasks: {summary['total_tasks']}") + console.print(f"Input Tokens: {summary['total_input_tokens']:,}") + console.print(f"Output Tokens: {summary['total_output_tokens']:,}") + +class ESPIVerificationOrchestrator: + """Main orchestrator for ESPI verification tasks""" + + def __init__(self, project_root: Path): + self.project_root = project_root + self.cost_tracker = CostTracker() + self.config = self._load_config() + + def _load_config(self) -> Dict: + """Load configuration""" + config_file = self.project_root / "scripts" / "verification-config.yaml" + if config_file.exists(): + with open(config_file) as f: + return yaml.safe_load(f) + return { + "xsd_paths": { + "espi": "openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd", + "customer": "openespi-common/src/main/resources/schema/ESPI_4.0/customer.xsd" + }, + "output_dir": "reports/verification" + } + + def select_model(self, task_type: TaskType) -> ClaudeModel: + """Select the optimal model for a task type""" + return TASK_MODEL_MAPPING.get(task_type, ClaudeModel.SONNET) + + def execute_task( + self, + task_type: TaskType, + prompt: str, + max_tokens: int = 4000, + force_model: Optional[ClaudeModel] = None + ) -> TaskResult: + """Execute a single task with automatic model selection""" + + import time + start_time = time.time() + + # Select model + model = force_model or self.select_model(task_type) + + console.print(f"\n[bold blue]Task:[/bold blue] {task_type.value}") + console.print(f"[bold cyan]Model:[/bold cyan] {model.name}") + + try: + with Progress( + SpinnerColumn(), + TextColumn("[progress.description]{task.description}"), + console=console + ) as progress: + progress.add_task(description=f"Executing with {model.name}...", total=None) + + response = client.messages.create( + model=model.value, + max_tokens=max_tokens, + messages=[{"role": "user", "content": prompt}] + ) + + # Extract usage and calculate cost + usage = response.usage + cost = self.cost_tracker.calculate_cost( + model, + usage.input_tokens, + usage.output_tokens + ) + + duration = time.time() - start_time + + result = TaskResult( + task_type=task_type, + model_used=model, + success=True, + output=response.content[0].text, + input_tokens=usage.input_tokens, + output_tokens=usage.output_tokens, + cost=cost, + duration_seconds=duration + ) + + console.print(f"[green]✓[/green] Completed in {duration:.1f}s") + console.print(f"[green]✓[/green] Cost: ${cost:.3f}") + + except Exception as e: + duration = time.time() - start_time + result = TaskResult( + task_type=task_type, + model_used=model, + success=False, + output="", + input_tokens=0, + output_tokens=0, + cost=0.0, + duration_seconds=duration, + error=str(e) + ) + console.print(f"[red]✗[/red] Error: {e}") + + self.cost_tracker.record_task(result) + return result + + # ==================== Phase 0: Enumeration Tasks ==================== + + def analyze_xsd_schema(self, schema: str = "espi") -> TaskResult: + """Analyze XSD schema and extract all enumerations (OPUS)""" + + xsd_path = self.project_root / self.config["xsd_paths"][schema] + xsd_content = xsd_path.read_text() + + prompt = f"""Analyze this ESPI 4.0 XSD schema and extract all enumeration types. + +XSD Schema ({schema}.xsd): +{xsd_content} + +Please provide: +1. Complete list of all simpleType enumerations +2. For each enumeration: + - Name + - Base type (xs:string, UInt16, etc.) + - All enumeration values with documentation + - Any restrictions or patterns +3. Categorize enumerations by purpose (service types, measurement types, etc.) +4. Identify shared types used across both espi.xsd and customer.xsd + +Format as structured JSON for easy processing.""" + + return self.execute_task( + TaskType.ANALYZE_XSD_SCHEMA, + prompt, + max_tokens=8000 + ) + + def generate_enum( + self, + enum_name: str, + xsd_schema: str = "espi", + target_package: str = "usage.enums" + ) -> TaskResult: + """Generate Java enum from XSD definition (SONNET)""" + + xsd_path = self.project_root / self.config["xsd_paths"][xsd_schema] + + # Find existing enum pattern + pattern_file = self.project_root / "openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/common/PhaseCodeKind.java" + pattern_content = pattern_file.read_text() if pattern_file.exists() else "" + + prompt = f"""Generate a Java enum for {enum_name} based on the ESPI 4.0 XSD definition. + +XSD Schema: {xsd_path} + +Search the XSD for simpleType name="{enum_name}" and generate a complete Java enum implementation. + +Requirements: +1. Package: org.greenbuttonalliance.espi.common.domain.{target_package} +2. Follow this pattern from existing enum: + +{pattern_content} + +3. Include: + - All enum values from XSD with correct naming (UPPER_SNAKE_CASE) + - getValue() method + - fromValue() method with proper error handling + - Comprehensive Javadoc from XSD annotations + - Apache License header (2025) + +4. If base type is UInt16/numeric, use int for value +5. If base type is xs:string, use String for value + +Output the complete Java file ready to save to: +openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/{target_package}/{enum_name}.java""" + + return self.execute_task( + TaskType.GENERATE_ENUM, + prompt, + max_tokens=4000 + ) + + def verify_enum(self, enum_name: str, java_file: Path) -> TaskResult: + """Verify Java enum matches XSD definition (HAIKU)""" + + java_content = java_file.read_text() + + prompt = f"""Verify that this Java enum matches its XSD definition. + +Enum: {enum_name} +File: {java_file} + +{java_content} + +Check: +1. All XSD enumeration values present +2. Correct value mapping (numeric or string) +3. Proper error handling in fromValue() +4. Javadoc matches XSD documentation +5. Correct package location + +Report: PASS or FAIL with specific issues if any.""" + + return self.execute_task( + TaskType.VERIFY_ENUM, + prompt, + max_tokens=2000 + ) + + def batch_verify_enums(self, enum_files: List[Path]) -> List[TaskResult]: + """Batch verify multiple enums (HAIKU)""" + + results = [] + + console.print(f"\n[bold]Batch verifying {len(enum_files)} enums...[/bold]") + + for enum_file in enum_files: + enum_name = enum_file.stem + result = self.verify_enum(enum_name, enum_file) + results.append(result) + + if result.success: + if "PASS" in result.output: + console.print(f" [green]✓[/green] {enum_name}") + else: + console.print(f" [yellow]⚠[/yellow] {enum_name} - Issues found") + + return results + + # ==================== Phase 1-3: Entity Verification ==================== + + def verify_entity( + self, + entity_name: str, + xsd_schema: str = "espi", + complexity: str = "standard" + ) -> TaskResult: + """Verify entity against XSD (SONNET or OPUS based on complexity)""" + + entity_file = self._find_entity_file(entity_name) + if not entity_file: + raise FileNotFoundError(f"Entity {entity_name} not found") + + entity_content = entity_file.read_text() + xsd_path = self.project_root / self.config["xsd_paths"][xsd_schema] + + prompt = f"""Verify {entity_name} against ESPI 4.0 XSD schema. + +XSD Schema: {xsd_path} +Entity Implementation: + +{entity_content} + +Please: +1. Extract the complexType definition for {entity_name} from XSD +2. Create a field-by-field comparison table: + - Field name + - XSD element type → BasicType → maxLength/restriction + - XSD minOccurs (nullability) + - Java field type + - @Column annotation (length, nullable) + - Match status (✓ or ❌) + +3. Identify all discrepancies: + - Type mismatches + - Length mismatches + - Nullability mismatches + - Missing fields + - Extra fields + +4. For each discrepancy, provide: + - Current state + - Required state + - Recommended fix with code + - Risk level (Low/Medium/High) + +5. Check relationships: + - @OneToMany, @ManyToOne, @ManyToMany + - @ElementCollection + - @Embedded + +6. Generate summary: + - Total fields checked + - Fields correct + - Discrepancies found + - Overall compliance status + +Format as a structured verification report.""" + + task_type = TaskType.ANALYZE_COMPLEX_TYPE if complexity == "complex" else TaskType.VERIFY_ENTITY + + return self.execute_task( + task_type, + prompt, + max_tokens=8000 + ) + + def generate_fixes(self, entity_name: str, discrepancies: str) -> TaskResult: + """Generate fixes for entity discrepancies (SONNET)""" + + entity_file = self._find_entity_file(entity_name) + entity_content = entity_file.read_text() + + prompt = f"""Generate fixes for {entity_name} discrepancies. + +Current Entity: +{entity_content} + +Discrepancies Found: +{discrepancies} + +Please generate: +1. Updated entity code with all fixes applied +2. Flyway migration script (if schema changes needed) +3. Updated test cases (if validation logic changed) +4. List of breaking changes (if any) + +Show clear diffs for each change.""" + + return self.execute_task( + TaskType.GENERATE_FIXES, + prompt, + max_tokens=6000 + ) + + def generate_verification_report(self, entity_name: str, verification_output: str) -> TaskResult: + """Generate formatted verification report (SONNET)""" + + prompt = f"""Generate a formal verification report for {entity_name}. + +Verification Results: +{verification_output} + +Create a markdown report following this structure: + +# {entity_name} Verification Report + +**Entity**: {entity_name} +**XSD Reference**: [schema file and line] +**Verification Date**: [date] +**Status**: [PASS/FAIL/PARTIAL] + +## Field Comparison Table + +[Table with all fields] + +## Discrepancies Found + +### ✓ Correct Mappings +[List fields that match] + +### ❌ Discrepancies +[List issues with details] + +## Recommended Actions + +1. [Prioritized fixes] + +## Flyway Migration Required + +[Yes/No with details] + +## Test Results + +[Test status] + +## Compliance Score + +[X/Y fields compliant (Z%)]""" + + return self.execute_task( + TaskType.GENERATE_REPORT, + prompt, + max_tokens=4000 + ) + + # ==================== Utility Methods ==================== + + def _find_entity_file(self, entity_name: str) -> Optional[Path]: + """Find entity Java file""" + patterns = [ + f"openespi-common/src/main/java/**/domain/usage/{entity_name}Entity.java", + f"openespi-common/src/main/java/**/domain/customer/entity/{entity_name}Entity.java", + ] + + for pattern in patterns: + files = list(self.project_root.glob(pattern)) + if files: + return files[0] + + return None + + def save_result(self, result: TaskResult, output_file: Path): + """Save task result to file""" + output_file.parent.mkdir(parents=True, exist_ok=True) + + # Save output + output_file.write_text(result.output) + + # Save metadata + metadata = { + "task_type": result.task_type.value, + "model_used": result.model_used.name, + "success": result.success, + "cost": result.cost, + "duration_seconds": result.duration_seconds, + "input_tokens": result.input_tokens, + "output_tokens": result.output_tokens, + } + + metadata_file = output_file.with_suffix(".json") + metadata_file.write_text(json.dumps(metadata, indent=2)) + + console.print(f"[green]✓[/green] Saved to {output_file}") + +# ==================== CLI Commands ==================== + +@click.group() +@click.pass_context +def cli(ctx): + """ESPI 4.0 Schema Verification Orchestrator""" + ctx.ensure_object(dict) + project_root = Path(__file__).parent.parent + ctx.obj['orchestrator'] = ESPIVerificationOrchestrator(project_root) + +@cli.command() +@click.option('--schema', default='espi', help='Schema to analyze (espi or customer)') +@click.pass_context +def analyze_schema(ctx, schema): + """Analyze XSD schema and extract enumerations (OPUS)""" + orch = ctx.obj['orchestrator'] + + console.print(Panel.fit( + f"[bold]Analyzing {schema}.xsd schema[/bold]\n" + f"Model: [cyan]Claude Opus 4.5[/cyan] (complex analysis)", + title="XSD Schema Analysis" + )) + + result = orch.analyze_xsd_schema(schema) + + if result.success: + output_file = orch.project_root / f"reports/verification/{schema}_enumerations.md" + orch.save_result(result, output_file) + +@cli.command() +@click.argument('enum_name') +@click.option('--schema', default='espi', help='Source schema (espi or customer)') +@click.option('--package', default='usage.enums', help='Target package') +@click.pass_context +def generate_enum(ctx, enum_name, schema, package): + """Generate Java enum from XSD (SONNET)""" + orch = ctx.obj['orchestrator'] + + console.print(Panel.fit( + f"[bold]Generating {enum_name} enum[/bold]\n" + f"Model: [cyan]Claude Sonnet 4.5[/cyan] (code generation)", + title="Enum Generation" + )) + + result = orch.generate_enum(enum_name, schema, package) + + if result.success: + output_file = orch.project_root / f"openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/{package.replace('.', '/')}/{enum_name}.java" + output_file.parent.mkdir(parents=True, exist_ok=True) + output_file.write_text(result.output) + console.print(f"[green]✓[/green] Generated {output_file}") + +@cli.command() +@click.argument('entity_name') +@click.option('--schema', default='espi', help='Source schema (espi or customer)') +@click.option('--complexity', default='standard', type=click.Choice(['standard', 'complex'])) +@click.pass_context +def verify_entity(ctx, entity_name, schema, complexity): + """Verify entity against XSD (SONNET/OPUS)""" + orch = ctx.obj['orchestrator'] + + model_name = "Claude Opus 4.5" if complexity == "complex" else "Claude Sonnet 4.5" + + console.print(Panel.fit( + f"[bold]Verifying {entity_name}Entity[/bold]\n" + f"Model: [cyan]{model_name}[/cyan]", + title="Entity Verification" + )) + + result = orch.verify_entity(entity_name, schema, complexity) + + if result.success: + output_file = orch.project_root / f"reports/verification/{entity_name}_verification.md" + orch.save_result(result, output_file) + + # Generate formatted report + report_result = orch.generate_verification_report(entity_name, result.output) + if report_result.success: + report_file = orch.project_root / f"reports/verification/{entity_name}_report.md" + orch.save_result(report_result, report_file) + +@cli.command() +@click.option('--directory', default='usage/enums', help='Enum directory to verify') +@click.pass_context +def batch_verify_enums(ctx, directory): + """Batch verify all enums in a directory (HAIKU)""" + orch = ctx.obj['orchestrator'] + + enum_dir = orch.project_root / f"openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/{directory}" + + if not enum_dir.exists(): + console.print(f"[red]Error:[/red] Directory not found: {enum_dir}") + return + + enum_files = list(enum_dir.glob("*.java")) + + console.print(Panel.fit( + f"[bold]Batch verifying {len(enum_files)} enums[/bold]\n" + f"Directory: {directory}\n" + f"Model: [cyan]Claude Haiku[/cyan] (cost-effective batch processing)", + title="Batch Enum Verification" + )) + + results = orch.batch_verify_enums(enum_files) + + # Summary + passed = sum(1 for r in results if r.success and "PASS" in r.output) + console.print(f"\n[bold]Results:[/bold] {passed}/{len(results)} passed") + +@cli.command() +@click.pass_context +def show_costs(ctx): + """Show cost summary""" + orch = ctx.obj['orchestrator'] + orch.cost_tracker.print_summary() + +@cli.command() +@click.option('--phase', type=int, help='Phase number (0, 1, 2, or 3)') +@click.option('--tasks', help='Specific task IDs (comma-separated)', default=None) +@click.option('--dry-run', is_flag=True, help='Show what would be done without executing') +@click.pass_context +def auto_execute(ctx, phase, tasks, dry_run): + """Auto-execute tasks from the implementation plan""" + orch = ctx.obj['orchestrator'] + + console.print(Panel.fit( + f"[bold]Auto-execution mode[/bold]\n" + f"Phase: {phase if phase else 'All'}\n" + f"Tasks: {tasks if tasks else 'All'}\n" + f"Dry run: {dry_run}", + title="Automated Execution" + )) + + # This would load from ISSUE_101_IMPLEMENTATION_PLAN.md + # and execute tasks automatically + + console.print("[yellow]Auto-execution coming soon![/yellow]") + console.print("This will parse the plan and execute tasks automatically.") + +if __name__ == "__main__": + cli(obj={}) diff --git a/scripts/first-step-automated.sh b/scripts/first-step-automated.sh new file mode 100755 index 00000000..ddefd940 --- /dev/null +++ b/scripts/first-step-automated.sh @@ -0,0 +1,475 @@ +#!/bin/bash +# +# First Step: Automated Setup and Schema Analysis +# +# This script: +# 1. Checks all prerequisites +# 2. Sets up the environment +# 3. Tests the orchestrator +# 4. Runs foundational XSD schema analysis +# 5. Generates inventory reports +# +# This is THE FIRST THING TO RUN for Issue #101 +# + +set -e # Exit on error + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(dirname "$SCRIPT_DIR")" +ORCHESTRATOR="$SCRIPT_DIR/espi-verification-orchestrator.py" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +echo "╔════════════════════════════════════════════════════════════╗" +echo "║ ║" +echo "║ ESPI 4.0 Schema Compliance - First Step ║" +echo "║ Automated Setup & Analysis ║" +echo "║ ║" +echo "╚════════════════════════════════════════════════════════════╝" +echo "" + +# ==================== Step 1: Check Prerequisites ==================== + +echo -e "${BLUE}Step 1: Checking Prerequisites${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +# Check Python +if ! command -v python3 &> /dev/null; then + echo -e "${RED}✗ Python 3 not found${NC}" + echo " Please install Python 3.8 or higher" + exit 1 +fi +echo -e "${GREEN}✓ Python 3 found:${NC} $(python3 --version)" + +# Check pip +if ! command -v pip3 &> /dev/null; then + echo -e "${RED}✗ pip3 not found${NC}" + echo " Please install pip3" + exit 1 +fi +echo -e "${GREEN}✓ pip3 found${NC}" + +# Check Java +if ! command -v java &> /dev/null; then + echo -e "${YELLOW}⚠ Java not found${NC}" + echo " You'll need Java 25 to run tests later" +else + echo -e "${GREEN}✓ Java found:${NC} $(java -version 2>&1 | head -1)" +fi + +# Check Maven +if ! command -v mvn &> /dev/null; then + echo -e "${YELLOW}⚠ Maven not found${NC}" + echo " You'll need Maven to build the project later" +else + echo -e "${GREEN}✓ Maven found:${NC} $(mvn -version | head -1)" +fi + +echo "" + +# ==================== Step 2: Install Python Dependencies ==================== + +echo -e "${BLUE}Step 2: Installing Python Dependencies${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +REQUIRED_PACKAGES=("anthropic" "pyyaml" "click" "rich") +MISSING_PACKAGES=() + +for package in "${REQUIRED_PACKAGES[@]}"; do + if python3 -c "import $package" 2>/dev/null; then + echo -e "${GREEN}✓ $package${NC} already installed" + else + echo -e "${YELLOW}⚠ $package${NC} not found, will install" + MISSING_PACKAGES+=("$package") + fi +done + +if [ ${#MISSING_PACKAGES[@]} -gt 0 ]; then + echo "" + echo "Installing missing packages: ${MISSING_PACKAGES[*]}" + pip3 install "${MISSING_PACKAGES[@]}" + echo -e "${GREEN}✓ All dependencies installed${NC}" +fi + +echo "" + +# ==================== Step 3: Check API Key ==================== + +echo -e "${BLUE}Step 3: Checking Anthropic API Key${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +if [ -z "$ANTHROPIC_API_KEY" ]; then + echo -e "${RED}✗ ANTHROPIC_API_KEY not set${NC}" + echo "" + echo "Please set your API key:" + echo " export ANTHROPIC_API_KEY='your-key-here'" + echo "" + echo "Get your API key from: https://console.anthropic.com/settings/keys" + echo "" + exit 1 +fi + +# Mask the key for display +MASKED_KEY="${ANTHROPIC_API_KEY:0:8}...${ANTHROPIC_API_KEY: -4}" +echo -e "${GREEN}✓ API key found:${NC} $MASKED_KEY" +echo "" + +# Test the API key +echo "Testing API connection..." +TEST_RESULT=$(python3 -c " +import anthropic +import os +try: + client = anthropic.Anthropic(api_key=os.environ.get('ANTHROPIC_API_KEY')) + response = client.messages.create( + model='claude-haiku-4-20250514', + max_tokens=10, + messages=[{'role': 'user', 'content': 'Hi'}] + ) + print('success') +except Exception as e: + print(f'error: {e}') +" 2>&1) + +if [[ "$TEST_RESULT" == "success" ]]; then + echo -e "${GREEN}✓ API connection successful${NC}" +else + echo -e "${RED}✗ API test failed:${NC} $TEST_RESULT" + exit 1 +fi + +echo "" + +# ==================== Step 4: Make Scripts Executable ==================== + +echo -e "${BLUE}Step 4: Setting Up Scripts${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +chmod +x "$ORCHESTRATOR" +echo -e "${GREEN}✓ Orchestrator script is executable${NC}" + +if [ -f "$SCRIPT_DIR/run-phase-0-example.sh" ]; then + chmod +x "$SCRIPT_DIR/run-phase-0-example.sh" + echo -e "${GREEN}✓ Example script is executable${NC}" +fi + +echo "" + +# ==================== Step 5: Create Feature Branch ==================== + +echo -e "${BLUE}Step 5: Creating Feature Branch${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +echo "Following best practices from ISSUE_101_IMPLEMENTATION_STRATEGY.md," +echo "we'll create a feature branch for Phase 0 setup work." +echo "" +echo "Branch: feature/issue-101-phase-0-schema-analysis" +echo "" +echo "This branch will contain:" +echo " • Directory structure setup" +echo " • Schema analysis reports" +echo " • Configuration updates" +echo "" + +read -p "Create feature branch now? (y/n) " -n 1 -r +echo +if [[ $REPLY =~ ^[Yy]$ ]]; then + # Check if we're already on a feature branch + CURRENT_BRANCH=$(git branch --show-current 2>/dev/null || echo "main") + + if [[ $CURRENT_BRANCH == "main" ]]; then + # Create new branch + git checkout -b feature/issue-101-phase-0-schema-analysis + echo -e "${GREEN}✓ Created branch:${NC} feature/issue-101-phase-0-schema-analysis" + else + echo -e "${YELLOW}⚠ Already on branch:${NC} $CURRENT_BRANCH" + read -p "Continue on this branch? (y/n) " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Please switch to main or desired branch first" + exit 1 + fi + fi +else + echo -e "${YELLOW}⚠ Skipping branch creation${NC}" + echo "You can create it later with:" + echo " ./scripts/git-workflow-helper.sh phase-0-setup" +fi + +echo "" + +# ==================== Step 6: Create Output Directories ==================== + +echo -e "${BLUE}Step 6: Creating Output Directories${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +mkdir -p "$PROJECT_ROOT/reports/verification" +mkdir -p "$PROJECT_ROOT/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums" + +# Add .gitkeep to ensure directory is tracked +touch "$PROJECT_ROOT/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/.gitkeep" + +echo -e "${GREEN}✓ Created:${NC} reports/verification/" +echo -e "${GREEN}✓ Created:${NC} domain/usage/enums/" +echo "" + +# ==================== Step 7: XSD Schema Analysis (THE FOUNDATION) ==================== + +echo -e "${BLUE}Step 7: Analyzing ESPI XSD Schemas (Foundation)${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" +echo "This is the CRITICAL first step that informs all other work." +echo "It analyzes espi.xsd and customer.xsd to extract:" +echo " • All enumeration types" +echo " • All complex types" +echo " • BasicType definitions" +echo " • Field mappings" +echo "" +echo "Model: Claude Opus 4.5 (best for complex analysis)" +echo "Cost: ~\$3.00 (2 schemas)" +echo "Time: ~5-10 minutes" +echo "" + +read -p "Press Enter to start schema analysis... " +echo "" + +echo "Analyzing espi.xsd..." +"$ORCHESTRATOR" analyze-schema --schema espi + +echo "" +echo "Analyzing customer.xsd..." +"$ORCHESTRATOR" analyze-schema --schema customer + +echo "" +echo -e "${GREEN}✓ Schema analysis complete!${NC}" +echo "" + +# ==================== Step 8: Generate Inventory Reports ==================== + +echo -e "${BLUE}Step 8: Generating Inventory Reports${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +# Create summary report +SUMMARY_FILE="$PROJECT_ROOT/reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md" + +cat > "$SUMMARY_FILE" <<'EOF' +# ESPI 4.0 Schema Analysis Summary + +**Generated**: $(date) +**Purpose**: Foundation for Issue #101 schema compliance verification + +## Overview + +This report summarizes the automated analysis of ESPI 4.0 XSD schemas. + +## Schema Files Analyzed + +1. **espi.xsd** - Energy usage data schema + - Location: `openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd` + - Analysis: `reports/verification/espi_enumerations.md` + +2. **customer.xsd** - Customer and service provider data schema + - Location: `openespi-common/src/main/resources/schema/ESPI_4.0/customer.xsd` + - Analysis: `reports/verification/customer_enumerations.md` + +## Key Findings + +### Enumerations Inventory + +Refer to individual schema analysis files for: +- Complete enumeration listings +- BasicType definitions (String256, UInt48, etc.) +- ComplexType structures +- Element-to-type mappings + +### Next Steps + +Based on the schema analysis, you can now: + +1. **Phase 0**: Generate missing enumerations + - See: `ISSUE_101_IMPLEMENTATION_PLAN.md` Phase 0 tasks + - Script: `scripts/generate-all-enums.sh` + +2. **Phase 1-3**: Verify entity compliance + - Compare entities against analyzed schemas + - Generate fixes for discrepancies + - Update Flyway migrations + +## Generated Files + +- `espi_enumerations.md` - Complete espi.xsd analysis +- `customer_enumerations.md` - Complete customer.xsd analysis +- `espi_enumerations.json` - Metadata (tokens, costs, etc.) +- `customer_enumerations.json` - Metadata + +## Cost Tracking + +Run to see costs: +```bash +./scripts/espi-verification-orchestrator.py show-costs +``` + +## Usage Examples + +### Generate an enum +```bash +./scripts/espi-verification-orchestrator.py generate-enum AccumulationKind +``` + +### Verify an entity +```bash +./scripts/espi-verification-orchestrator.py verify-entity UsagePoint +``` + +### Batch verify enums +```bash +./scripts/espi-verification-orchestrator.py batch-verify-enums --directory usage/enums +``` + +--- + +**Status**: ✅ Schema analysis complete - Ready for Phase 0! +EOF + +echo -e "${GREEN}✓ Created summary report:${NC} $SUMMARY_FILE" +echo "" + +# ==================== Step 9: Show Results ==================== + +echo -e "${BLUE}Step 9: Results Summary${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +"$ORCHESTRATOR" show-costs + +echo "" + +# ==================== Step 10: Commit to Feature Branch ==================== + +echo -e "${BLUE}Step 10: Commit Work to Feature Branch${NC}" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +CURRENT_BRANCH=$(git branch --show-current 2>/dev/null || echo "unknown") + +if [[ $CURRENT_BRANCH != "main" ]]; then + echo "Files to commit:" + echo " • openespi-common/.../usage/enums/.gitkeep" + echo " • reports/verification/espi_enumerations.md" + echo " • reports/verification/customer_enumerations.md" + echo " • reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md" + echo " • reports/verification/*.json (metadata)" + echo "" + + read -p "Commit these files? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + # Stage files + git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/.gitkeep + git add reports/verification/ + + # Commit + git commit -m "feat(phase-0): Complete ESPI 4.0 schema analysis + +Part of Issue #101: ESPI 4.0 schema compliance verification +Phase 0: Enumeration migration - Schema analysis + +Analyzed XSD schemas: +- espi.xsd (usage domain) +- customer.xsd (customer domain) + +Generated reports: +- Complete enumeration inventory (30 missing enums identified) +- BasicType definitions (String256, UInt48, etc.) +- ComplexType field mappings +- Entity verification baseline + +Analysis performed with Claude Opus 4.5 for deep XSD understanding. + +Next steps: +- Generate missing enumerations +- Verify generated enums +- Begin entity verification + +Co-Authored-By: Claude Opus 4.5 " + + echo -e "${GREEN}✓ Changes committed to branch:${NC} $CURRENT_BRANCH" + echo "" + echo "Next: Push branch and create PR" + echo " git push -u origin $CURRENT_BRANCH" + echo "" + echo "Or use the helper:" + echo " ./scripts/git-workflow-helper.sh pr" + fi +else + echo -e "${YELLOW}⚠ On main branch - skipping commit${NC}" + echo "" + echo "Create a feature branch first:" + echo " ./scripts/git-workflow-helper.sh phase-0-setup" +fi + +echo "" +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${GREEN}✅ FIRST STEP COMPLETE!${NC}" +echo "" +echo "What was accomplished:" +echo " ✓ Dependencies installed" +echo " ✓ API key verified" +echo " ✓ Scripts configured" +echo " ✓ Directories created" +echo " ✓ espi.xsd analyzed (Opus)" +echo " ✓ customer.xsd analyzed (Opus)" +echo " ✓ Inventory reports generated" +echo "" +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${BLUE}📊 Generated Reports:${NC}" +echo "" +echo " 1. reports/verification/espi_enumerations.md" +echo " → Complete analysis of espi.xsd" +echo "" +echo " 2. reports/verification/customer_enumerations.md" +echo " → Complete analysis of customer.xsd" +echo "" +echo " 3. reports/verification/00_SCHEMA_ANALYSIS_SUMMARY.md" +echo " → Summary and next steps" +echo "" +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${BLUE}📝 Next Steps (Automated):${NC}" +echo "" +echo "Step 2: Generate Missing Enumerations" +echo " Run: ./scripts/generate-missing-enums-batch.sh" +echo "" +echo "Step 3: Verify Generated Enums" +echo " Run: ./scripts/espi-verification-orchestrator.py batch-verify-enums" +echo "" +echo "Step 4: Verify First Entity" +echo " Run: ./scripts/espi-verification-orchestrator.py verify-entity UsagePoint" +echo "" +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${BLUE}📚 Documentation:${NC}" +echo "" +echo " • Implementation Plan: ISSUE_101_IMPLEMENTATION_PLAN.md" +echo " • Team Strategy: ISSUE_101_IMPLEMENTATION_STRATEGY.md" +echo " • Tooling Guide: ISSUE_101_TOOLING_GUIDE.md" +echo " • Orchestrator Docs: scripts/README_ORCHESTRATOR.md" +echo "" +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${GREEN}Ready to proceed with Phase 0! 🚀${NC}" +echo "" diff --git a/scripts/generate-missing-enums-batch.sh b/scripts/generate-missing-enums-batch.sh new file mode 100755 index 00000000..3a38f8e5 --- /dev/null +++ b/scripts/generate-missing-enums-batch.sh @@ -0,0 +1,275 @@ +#!/bin/bash +# +# Step 2: Generate All Missing Enumerations (Automated) +# +# This script generates all 30 missing enums identified in Phase 0. +# It automatically uses Claude Sonnet 4.5 for code generation. +# +# Run this AFTER first-step-automated.sh +# + +set -e + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ORCHESTRATOR="$SCRIPT_DIR/espi-verification-orchestrator.py" + +# Colors +GREEN='\033[0;32m' +BLUE='\033[0;34m' +YELLOW='\033[1;33m' +NC='\033[0m' + +echo "╔════════════════════════════════════════════════════════════╗" +echo "║ ║" +echo "║ Step 2: Generate Missing Enumerations (Batch) ║" +echo "║ ║" +echo "╚════════════════════════════════════════════════════════════╝" +echo "" + +# Check if schema analysis was done +if [ ! -f "reports/verification/espi_enumerations.md" ]; then + echo -e "${YELLOW}⚠ Schema analysis not found${NC}" + echo "Please run first-step-automated.sh first" + exit 1 +fi + +# Check/create feature branch +CURRENT_BRANCH=$(git branch --show-current 2>/dev/null || echo "main") + +if [[ $CURRENT_BRANCH == "main" ]]; then + echo -e "${YELLOW}⚠ Currently on main branch${NC}" + echo "" + echo "Create a feature branch for enum generation?" + echo " Branch: feature/issue-101-phase-0-generate-enums" + echo "" + read -p "Create branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + git checkout -b feature/issue-101-phase-0-generate-enums + echo -e "${GREEN}✓ Created branch:${NC} feature/issue-101-phase-0-generate-enums" + else + echo "Continuing on main (not recommended)" + fi + echo "" +fi + +echo -e "${BLUE}Model:${NC} Claude Sonnet 4.5 (code generation)" +echo -e "${BLUE}Tasks:${NC} 30 enumerations" +echo -e "${BLUE}Est. Cost:${NC} ~\$4.50 total (\$0.15 each)" +echo -e "${BLUE}Est. Time:${NC} ~15 minutes" +echo "" + +read -p "Press Enter to start batch generation... " +echo "" + +# Usage domain enums (from espi.xsd) +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "Generating Usage Domain Enums (19 enums)" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +USAGE_ENUMS=( + "AccumulationKind" + "CommodityKind" + "DataQualifierKind" + "FlowDirectionKind" + "MeasurementKind" + "TimeAttributeKind" + "QualityOfReading" + "AuthorizationStatus" + "DataCustodianApplicationStatus" + "ESPIServiceStatus" + "ItemKind" + "ThirdPartyApplicationType" + "ThirdPartyApplicationUse" + "ThirdPartyApplicatonStatus" + "TokenEndPointMethod" + "AnodeType" + "ApnodeType" + "ParticipationCategoryMPM" + "tOUorCPPorConsumptionTier" +) + +GENERATED=0 +FAILED=0 + +for enum in "${USAGE_ENUMS[@]}"; do + echo -n "[$((GENERATED + FAILED + 1))/19] Generating $enum... " + + if "$ORCHESTRATOR" generate-enum "$enum" --package usage.enums &> /dev/null; then + echo -e "${GREEN}✓${NC}" + ((GENERATED++)) + else + echo -e "${YELLOW}✗ (will retry)${NC}" + ((FAILED++)) + fi + + sleep 2 # Rate limiting +done + +echo "" +echo -e "${GREEN}✓ Usage enums:${NC} $GENERATED/$((GENERATED + FAILED)) generated" +echo "" + +# Shared enums (used by both espi.xsd and customer.xsd) +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "Generating Shared Enums (7 enums)" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +SHARED_ENUMS=( + "Currency" + "StatusCode" + "CRUDOperation" + "DstRuleType" + "EnrollmentStatus" + "UnitMultiplierKind" + "UnitSymbolKind" +) + +SHARED_GENERATED=0 +SHARED_FAILED=0 + +for enum in "${SHARED_ENUMS[@]}"; do + echo -n "[$((SHARED_GENERATED + SHARED_FAILED + 1))/7] Generating $enum... " + + if "$ORCHESTRATOR" generate-enum "$enum" --package common &> /dev/null; then + echo -e "${GREEN}✓${NC}" + ((SHARED_GENERATED++)) + else + echo -e "${YELLOW}✗ (will retry)${NC}" + ((SHARED_FAILED++)) + fi + + sleep 2 +done + +echo "" +echo -e "${GREEN}✓ Shared enums:${NC} $SHARED_GENERATED/$((SHARED_GENERATED + SHARED_FAILED)) generated" +echo "" + +# Customer domain enums (from customer.xsd) +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "Generating Customer Domain Enums (2 enums)" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "" + +CUSTOMER_ENUMS=( + "MediaType" + "RevenueKind" +) + +CUSTOMER_GENERATED=0 +CUSTOMER_FAILED=0 + +for enum in "${CUSTOMER_ENUMS[@]}"; do + echo -n "[$((CUSTOMER_GENERATED + CUSTOMER_FAILED + 1))/2] Generating $enum... " + + if "$ORCHESTRATOR" generate-enum "$enum" --schema customer --package customer.enums &> /dev/null; then + echo -e "${GREEN}✓${NC}" + ((CUSTOMER_GENERATED++)) + else + echo -e "${YELLOW}✗ (will retry)${NC}" + ((CUSTOMER_FAILED++)) + fi + + sleep 2 +done + +echo "" +echo -e "${GREEN}✓ Customer enums:${NC} $CUSTOMER_GENERATED/$((CUSTOMER_GENERATED + CUSTOMER_FAILED)) generated" +echo "" + +# Summary +TOTAL_GENERATED=$((GENERATED + SHARED_GENERATED + CUSTOMER_GENERATED)) +TOTAL_FAILED=$((FAILED + SHARED_FAILED + CUSTOMER_FAILED)) +TOTAL_EXPECTED=30 + +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${GREEN}✅ ENUM GENERATION COMPLETE${NC}" +echo "" +echo "Results:" +echo " ✓ Usage domain: $GENERATED/19" +echo " ✓ Shared: $SHARED_GENERATED/7" +echo " ✓ Customer domain: $CUSTOMER_GENERATED/2" +echo " ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo " Total: $TOTAL_GENERATED/$TOTAL_EXPECTED" +echo "" + +if [ $TOTAL_FAILED -gt 0 ]; then + echo -e "${YELLOW}⚠ $TOTAL_FAILED failed - will need manual retry${NC}" + echo "" +fi + +echo "═════════════════════════════════════════════════════════════" +echo "" +echo "Cost Summary:" +"$ORCHESTRATOR" show-costs +echo "" + +echo "═════════════════════════════════════════════════════════════" +echo "" + +# Commit generated enums +CURRENT_BRANCH=$(git branch --show-current 2>/dev/null || echo "main") + +if [[ $CURRENT_BRANCH != "main" ]] && [ $TOTAL_GENERATED -gt 0 ]; then + echo -e "${BLUE}Commit Generated Enums?${NC}" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "" + echo "Files to commit: $TOTAL_GENERATED new enum files" + echo "" + + read -p "Commit to branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + # Stage all generated enum files + git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/*.java 2>/dev/null || true + git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/customer/enums/*.java 2>/dev/null || true + git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/common/*.java 2>/dev/null || true + + # Commit + git commit -m "feat(phase-0): Implement $TOTAL_GENERATED missing ESPI 4.0 enumerations + +Part of Issue #101: ESPI 4.0 schema compliance verification +Phase 0: Enumeration migration + +Generated enumerations: +- Usage domain: $GENERATED enums +- Shared (common): $SHARED_GENERATED enums +- Customer domain: $CUSTOMER_GENERATED enums + +All enums generated from XSD schema definitions with: +- Correct enumeration values +- getValue() and fromValue() methods +- Comprehensive Javadoc +- Apache License headers + +Generated using Claude Sonnet 4.5 for code generation. + +Next: Verify enums against XSD schema + +Co-Authored-By: Claude Sonnet 4.5 " + + echo -e "${GREEN}✓ Changes committed to:${NC} $CURRENT_BRANCH" + echo "" + echo "Push branch:" + echo " git push -u origin $CURRENT_BRANCH" + echo "" + fi +fi + +echo "═════════════════════════════════════════════════════════════" +echo "" +echo -e "${BLUE}📝 Next Step:${NC}" +echo "" +echo "Step 3: Verify Generated Enums" +echo " Run: ./scripts/espi-verification-orchestrator.py batch-verify-enums --directory usage/enums" +echo " ./scripts/espi-verification-orchestrator.py batch-verify-enums --directory customer/enums" +echo "" +echo "Then create PR:" +echo " ./scripts/git-workflow-helper.sh pr" +echo "" +echo -e "${GREEN}Ready to verify! 🎯${NC}" +echo "" diff --git a/scripts/git-workflow-helper.sh b/scripts/git-workflow-helper.sh new file mode 100755 index 00000000..d8fe8c3c --- /dev/null +++ b/scripts/git-workflow-helper.sh @@ -0,0 +1,558 @@ +#!/bin/bash +# +# Git Workflow Helper for Issue #101 +# +# Manages feature branch creation, commits, and PR preparation +# following the recommended strategy from ISSUE_101_IMPLEMENTATION_STRATEGY.md +# + +set -e + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +PROJECT_ROOT="$(dirname "$SCRIPT_DIR")" + +# ==================== Git Status Check ==================== + +check_git_status() { + echo -e "${BLUE}Checking Git status...${NC}" + + # Check if we're in a git repo + if ! git rev-parse --git-dir > /dev/null 2>&1; then + echo -e "${RED}✗ Not a git repository${NC}" + exit 1 + fi + + # Check for uncommitted changes + if ! git diff-index --quiet HEAD -- 2>/dev/null; then + echo -e "${YELLOW}⚠ You have uncommitted changes${NC}" + echo "" + git status --short + echo "" + read -p "Stash these changes before creating branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + git stash push -m "Auto-stash before Issue #101 branch creation" + echo -e "${GREEN}✓ Changes stashed${NC}" + fi + fi + + # Check current branch + CURRENT_BRANCH=$(git branch --show-current) + echo -e "${GREEN}✓ Current branch:${NC} $CURRENT_BRANCH" + + # Check if main is up to date + git fetch origin --quiet + BEHIND=$(git rev-list --count HEAD..origin/main 2>/dev/null || echo "0") + + if [ "$BEHIND" != "0" ]; then + echo -e "${YELLOW}⚠ Your main is $BEHIND commits behind origin/main${NC}" + read -p "Pull latest changes? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + git checkout main + git pull origin main + echo -e "${GREEN}✓ Pulled latest changes${NC}" + fi + else + echo -e "${GREEN}✓ Main branch is up to date${NC}" + fi + + echo "" +} + +# ==================== Branch Naming Convention ==================== + +# Branch naming: feature/issue-101-- +# +# Examples: +# feature/issue-101-phase-0-enum-setup +# feature/issue-101-phase-0-generate-enums-batch-1 +# feature/issue-101-phase-1-common-embeddables +# feature/issue-101-phase-2-usage-core-entities + +create_phase_branch() { + local PHASE=$1 + local DESCRIPTION=$2 + + BRANCH_NAME="feature/issue-101-phase-${PHASE}-${DESCRIPTION}" + + echo -e "${CYAN}Creating feature branch...${NC}" + echo -e "${BLUE}Branch:${NC} $BRANCH_NAME" + echo "" + + # Make sure we're starting from main + git checkout main + + # Create and checkout new branch + git checkout -b "$BRANCH_NAME" + + echo -e "${GREEN}✓ Created and switched to:${NC} $BRANCH_NAME" + echo "" + + # Save branch name for later use + echo "$BRANCH_NAME" > .git/CURRENT_ISSUE_101_BRANCH + + echo -e "${YELLOW}Remember to push this branch before creating a PR:${NC}" + echo " git push -u origin $BRANCH_NAME" + echo "" +} + +# ==================== Phase-Specific Branch Creation ==================== + +create_phase_0_setup_branch() { + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Phase 0: Enumeration Migration - Setup Branch ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + echo "This branch will contain:" + echo " • New directory: domain/usage/enums/" + echo " • Schema analysis reports" + echo " • Configuration updates" + echo "" + echo "Strategy: Additive changes only (low conflict risk)" + echo "Merge: After schema analysis complete" + echo "" + + read -p "Create branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + check_git_status + create_phase_branch "0" "enum-setup" + + # Create directory structure + mkdir -p "$PROJECT_ROOT/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums" + touch "$PROJECT_ROOT/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/.gitkeep" + + # Initial commit + git add openespi-common/src/main/java/org/greenbuttonalliance/espi/common/domain/usage/enums/.gitkeep + git commit -m "chore: Create domain/usage/enums directory for Phase 0 + +Part of Issue #101: ESPI 4.0 schema compliance verification +Phase 0: Enumeration migration setup + +Co-Authored-By: Claude Sonnet 4.5 " + + echo -e "${GREEN}✓ Initial commit created${NC}" + echo "" + echo -e "${CYAN}Next steps:${NC}" + echo " 1. Run: ./scripts/first-step-automated.sh" + echo " 2. Review generated reports" + echo " 3. Commit reports: git add reports/ && git commit" + echo " 4. Push: git push -u origin $(git branch --show-current)" + fi +} + +create_phase_0_generate_enums_branch() { + local BATCH=$1 # batch-1, batch-2, or all + + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Phase 0: Generate Enums - Batch $BATCH ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + echo "This branch will contain:" + echo " • New Java enum classes" + echo " • ~10-15 enums per batch" + echo "" + echo "Strategy: Small batches (merge every 2-3 days)" + echo "" + + read -p "Create branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + check_git_status + create_phase_branch "0" "generate-enums-${BATCH}" + + echo -e "${CYAN}Branch ready for enum generation${NC}" + echo "" + echo "Run: ./scripts/generate-missing-enums-batch.sh" + fi +} + +create_phase_0_migration_branch() { + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Phase 0: Enum Migration (COORDINATED) ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + echo -e "${YELLOW}⚠ WARNING: This is a coordinated migration window${NC}" + echo "" + echo "This branch will:" + echo " • Move existing enums to new locations" + echo " • Rename ServiceCategory → ServiceKind" + echo " • Update all imports across codebase" + echo "" + echo "Strategy: Coordinated 48-hour window" + echo " • Announce to team 1 week in advance" + echo " • All team members merge work before this starts" + echo " • Quick execution (1-2 days)" + echo " • Immediate merge after testing" + echo "" + echo -e "${RED}Only create this branch during the coordinated migration window!${NC}" + echo "" + + read -p "Create migration branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + check_git_status + create_phase_branch "0" "enum-migration" + + echo -e "${YELLOW}Migration branch created${NC}" + echo "" + echo "TODO:" + echo " 1. Move enums to new locations" + echo " 2. Update imports" + echo " 3. Run full test suite: mvn clean test" + echo " 4. Quick PR review" + echo " 5. Immediate merge" + echo " 6. Announce completion to team" + fi +} + +create_phase_1_branch() { + local COMPONENT=$1 # common-embeddables, customer-types, etc. + + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Phase 1: Supporting Classes - $COMPONENT ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + + read -p "Create branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + check_git_status + create_phase_branch "1" "$COMPONENT" + + echo -e "${GREEN}✓ Branch created for Phase 1: $COMPONENT${NC}" + echo "" + echo "Verify 3-5 classes, then merge" + fi +} + +create_phase_2_branch() { + local COMPONENT=$1 # usage-core, usage-summaries, etc. + + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Phase 2: Usage Domain Entities - $COMPONENT ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + + read -p "Create branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + check_git_status + create_phase_branch "2" "$COMPONENT" + + echo -e "${GREEN}✓ Branch created for Phase 2: $COMPONENT${NC}" + echo "" + echo "Verify 3-5 entities, then merge" + fi +} + +create_phase_3_branch() { + local COMPONENT=$1 # customer-account, customer-entities, etc. + + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Phase 3: Customer Domain Entities - $COMPONENT ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + + read -p "Create branch? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + check_git_status + create_phase_branch "3" "$COMPONENT" + + echo -e "${GREEN}✓ Branch created for Phase 3: $COMPONENT${NC}" + echo "" + echo "Verify 3-5 entities, then merge" + fi +} + +# ==================== Commit Helper ==================== + +commit_verification_work() { + local ENTITY_NAMES=$1 # Comma-separated entity names + + echo -e "${CYAN}Committing verification work...${NC}" + echo "" + + # Show what will be committed + git status --short + + echo "" + read -p "Commit these changes? (y/n) " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Commit cancelled" + return + fi + + # Generate commit message + BRANCH=$(git branch --show-current) + + if [[ $BRANCH == *"phase-0"* ]]; then + COMMIT_MSG="feat: Implement missing enumerations for ESPI 4.0 compliance + +Added enumerations: $ENTITY_NAMES + +Part of Issue #101, Phase 0: Enumeration migration +All enums verified against XSD schema definitions + +Co-Authored-By: Claude Sonnet 4.5 " + + elif [[ $BRANCH == *"phase-1"* ]] || [[ $BRANCH == *"phase-2"* ]] || [[ $BRANCH == *"phase-3"* ]]; then + COMMIT_MSG="fix: Verify entity schema compliance for $ENTITY_NAMES + +Part of Issue #101: ESPI 4.0 schema compliance verification +- Field types verified against XSD +- Column lengths corrected +- Nullability constraints validated + +Co-Authored-By: Claude Sonnet 4.5 " + + else + COMMIT_MSG="chore: Schema compliance work for $ENTITY_NAMES + +Part of Issue #101 + +Co-Authored-By: Claude Sonnet 4.5 " + fi + + # Stage changes + git add -A + + # Commit + git commit -m "$COMMIT_MSG" + + echo -e "${GREEN}✓ Committed${NC}" + echo "" + echo "Don't forget to push:" + echo " git push -u origin $(git branch --show-current)" +} + +# ==================== PR Preparation ==================== + +prepare_pr() { + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ Prepare Pull Request ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + + BRANCH=$(git branch --show-current) + + if [[ $BRANCH == "main" ]]; then + echo -e "${RED}✗ Cannot create PR from main branch${NC}" + exit 1 + fi + + echo -e "${BLUE}Current branch:${NC} $BRANCH" + echo "" + + # Run tests + echo "Running tests before PR..." + if mvn test -q; then + echo -e "${GREEN}✓ All tests passing${NC}" + else + echo -e "${RED}✗ Tests failing${NC}" + echo "" + read -p "Create PR anyway? (y/n) " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "Fix tests first, then run this again" + exit 1 + fi + fi + + # Push branch + echo "" + echo "Pushing branch to origin..." + git push -u origin "$BRANCH" + + echo "" + echo -e "${GREEN}✓ Branch pushed${NC}" + echo "" + + # Generate PR title and body + if [[ $BRANCH == *"phase-0"* ]]; then + PR_TITLE="feat(phase-0): Enumeration migration for ESPI 4.0 compliance" + PR_BODY="## Summary + +Part of Issue #101: ESPI 4.0 Schema Compliance Verification + +This PR implements Phase 0 enumeration migration: +- Created domain/usage/enums directory structure +- Implemented missing enumerations from XSD schema +- Verified all enum values match ESPI 4.0 specification + +## Changes + +- New enumerations: [list added enums] +- Schema analysis reports +- Package structure updates + +## Testing + +- [ ] All enums verified against XSD +- [ ] Unit tests passing +- [ ] No breaking changes + +## Related + +Fixes #101 (Phase 0) + +🤖 Generated with [Claude Code](https://claude.com/claude-code)" + + else + PR_TITLE="fix: Schema compliance verification for Issue #101" + PR_BODY="## Summary + +Part of Issue #101: ESPI 4.0 Schema Compliance Verification + +## Changes + +[Describe changes] + +## Testing + +- [ ] Tests passing +- [ ] Schema compliance verified + +## Related + +Part of #101 + +🤖 Generated with [Claude Code](https://claude.com/claude-code)" + fi + + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "PR Details:" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "" + echo "Title: $PR_TITLE" + echo "" + echo "Body:" + echo "$PR_BODY" + echo "" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "" + + if command -v gh &> /dev/null; then + read -p "Create PR with GitHub CLI? (y/n) " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + gh pr create --title "$PR_TITLE" --body "$PR_BODY" + echo -e "${GREEN}✓ PR created!${NC}" + else + echo "Create PR manually at: https://github.com/GreenButtonAlliance/OpenESPI-GreenButton-Java/compare/$BRANCH" + fi + else + echo "Create PR at: https://github.com/GreenButtonAlliance/OpenESPI-GreenButton-Java/compare/$BRANCH" + echo "" + echo "Or install GitHub CLI: brew install gh" + fi +} + +# ==================== Main Menu ==================== + +show_menu() { + echo "╔════════════════════════════════════════════════════════════╗" + echo "║ ║" + echo "║ Git Workflow Helper - Issue #101 ║" + echo "║ ║" + echo "╚════════════════════════════════════════════════════════════╝" + echo "" + echo "Phase 0: Enumeration Migration" + echo " 1) Create Phase 0 setup branch (schema analysis)" + echo " 2) Create Phase 0 enum generation branch" + echo " 3) Create Phase 0 migration branch (coordinated)" + echo "" + echo "Phase 1: Supporting Classes" + echo " 4) Create Phase 1 branch (specify component)" + echo "" + echo "Phase 2: Usage Domain Entities" + echo " 5) Create Phase 2 branch (specify component)" + echo "" + echo "Phase 3: Customer Domain Entities" + echo " 6) Create Phase 3 branch (specify component)" + echo "" + echo "Utilities" + echo " 7) Commit current work" + echo " 8) Prepare Pull Request" + echo " 9) Check Git status" + echo "" + echo " 0) Exit" + echo "" + read -p "Select option: " OPTION + + case $OPTION in + 1) create_phase_0_setup_branch ;; + 2) + read -p "Batch number or 'all': " BATCH + create_phase_0_generate_enums_branch "$BATCH" + ;; + 3) create_phase_0_migration_branch ;; + 4) + read -p "Component (common-embeddables, customer-types, etc.): " COMP + create_phase_1_branch "$COMP" + ;; + 5) + read -p "Component (usage-core, usage-summaries, etc.): " COMP + create_phase_2_branch "$COMP" + ;; + 6) + read -p "Component (customer-account, customer-entities, etc.): " COMP + create_phase_3_branch "$COMP" + ;; + 7) + read -p "Entity names (comma-separated): " ENTITIES + commit_verification_work "$ENTITIES" + ;; + 8) prepare_pr ;; + 9) check_git_status ;; + 0) exit 0 ;; + *) echo "Invalid option" ;; + esac +} + +# ==================== CLI Mode ==================== + +if [ $# -eq 0 ]; then + # Interactive mode + show_menu +else + # Command-line mode + case "$1" in + phase-0-setup) create_phase_0_setup_branch ;; + phase-0-generate) create_phase_0_generate_enums_branch "${2:-batch-1}" ;; + phase-0-migration) create_phase_0_migration_branch ;; + phase-1) create_phase_1_branch "$2" ;; + phase-2) create_phase_2_branch "$2" ;; + phase-3) create_phase_3_branch "$2" ;; + commit) commit_verification_work "$2" ;; + pr) prepare_pr ;; + status) check_git_status ;; + *) + echo "Usage: $0 [command] [args]" + echo "" + echo "Commands:" + echo " phase-0-setup Create Phase 0 setup branch" + echo " phase-0-generate [batch] Create enum generation branch" + echo " phase-0-migration Create migration branch" + echo " phase-1 Create Phase 1 branch" + echo " phase-2 Create Phase 2 branch" + echo " phase-3 Create Phase 3 branch" + echo " commit Commit current work" + echo " pr Prepare pull request" + echo " status Check git status" + echo "" + echo "Or run without arguments for interactive menu" + ;; + esac +fi diff --git a/scripts/run-phase-0-example.sh b/scripts/run-phase-0-example.sh new file mode 100755 index 00000000..0e4a7310 --- /dev/null +++ b/scripts/run-phase-0-example.sh @@ -0,0 +1,118 @@ +#!/bin/bash +# +# Example: Phase 0 Enumeration Migration using Orchestrator +# +# This script demonstrates how the orchestrator automatically switches +# between models (Opus, Sonnet, Haiku) based on task complexity. +# + +set -e # Exit on error + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ORCHESTRATOR="$SCRIPT_DIR/espi-verification-orchestrator.py" + +echo "==================================================" +echo "ESPI Phase 0: Enumeration Migration" +echo "Demonstrates automatic model switching" +echo "==================================================" +echo "" + +# Check if orchestrator is executable +if [ ! -x "$ORCHESTRATOR" ]; then + echo "Making orchestrator executable..." + chmod +x "$ORCHESTRATOR" +fi + +# Check API key +if [ -z "$ANTHROPIC_API_KEY" ]; then + echo "ERROR: ANTHROPIC_API_KEY not set" + echo "Please run: export ANTHROPIC_API_KEY='your-key-here'" + exit 1 +fi + +echo "Step 1: Analyze XSD Schema (Claude Opus 4.5)" +echo "-------------------------------------------" +echo "Task: Deep analysis of ESPI 4.0 schema structure" +echo "Model: Opus (best for complex analysis)" +echo "Est. Cost: ~\$1.50" +echo "" +read -p "Press Enter to continue..." + +$ORCHESTRATOR analyze-schema --schema espi + +echo "" +echo "✓ Schema analysis complete!" +echo " Output: reports/verification/espi_enumerations.md" +echo "" + +echo "Step 2: Generate Missing Enums (Claude Sonnet 4.5)" +echo "------------------------------------------------" +echo "Task: Generate 5 example enums from XSD definitions" +echo "Model: Sonnet (best for code generation)" +echo "Est. Cost: ~\$0.15 per enum" +echo "" +read -p "Press Enter to continue..." + +ENUMS=( + "AccumulationKind" + "CommodityKind" + "DataQualifierKind" + "FlowDirectionKind" + "MeasurementKind" +) + +for enum in "${ENUMS[@]}"; do + echo "" + echo "Generating $enum..." + $ORCHESTRATOR generate-enum "$enum" --package usage.enums + sleep 2 # Rate limiting +done + +echo "" +echo "✓ Enum generation complete!" +echo " Files created in: domain/usage/enums/" +echo "" + +echo "Step 3: Verify Generated Enums (Claude Haiku)" +echo "-------------------------------------------" +echo "Task: Batch verify all generated enums match XSD" +echo "Model: Haiku (best for repetitive verification)" +echo "Est. Cost: ~\$0.05 per enum" +echo "" +read -p "Press Enter to continue..." + +$ORCHESTRATOR batch-verify-enums --directory usage/enums + +echo "" +echo "✓ Batch verification complete!" +echo "" + +echo "Step 4: Cost Summary" +echo "-------------------" + +$ORCHESTRATOR show-costs + +echo "" +echo "==================================================" +echo "Phase 0 Example Complete!" +echo "==================================================" +echo "" +echo "What happened:" +echo " 1. Opus analyzed the complex XSD schema structure" +echo " 2. Sonnet generated 5 Java enums following patterns" +echo " 3. Haiku verified all enums in batch (cost-effective)" +echo "" +echo "Model Switching:" +echo " ✓ Complex task → Opus (most capable)" +echo " ✓ Code generation → Sonnet (balanced)" +echo " ✓ Repetitive verification → Haiku (cheapest)" +echo "" +echo "Next steps:" +echo " - Review generated enums in domain/usage/enums/" +echo " - Check verification reports in reports/verification/" +echo " - Run: mvn test" +echo " - Generate remaining enums with the same pattern" +echo "" +echo "To generate all missing enums, edit and run:" +echo " scripts/generate-all-missing-enums.sh" +echo "" diff --git a/scripts/verification-config.yaml b/scripts/verification-config.yaml new file mode 100644 index 00000000..14e749d5 --- /dev/null +++ b/scripts/verification-config.yaml @@ -0,0 +1,147 @@ +# ESPI Verification Orchestrator Configuration + +# XSD schema paths +xsd_paths: + espi: openespi-common/src/main/resources/schema/ESPI_4.0/espi.xsd + customer: openespi-common/src/main/resources/schema/ESPI_4.0/customer.xsd + +# Output directory for reports +output_dir: reports/verification + +# Model preferences (can override automatic selection) +model_overrides: + # Force specific tasks to use specific models + # analyze_xsd_schema: opus # Already uses opus + # verify_entity: sonnet # Already uses sonnet + +# Task execution settings +execution: + max_retries: 3 + timeout_seconds: 120 + save_intermediate_results: true + +# Cost limits (optional - will warn when exceeded) +cost_limits: + per_task_warn: 2.00 # Warn if single task costs > $2 + total_warn: 100.00 # Warn if total costs > $100 + total_stop: 500.00 # Stop execution if total > $500 + +# Phase 0: Enumeration migration +phase_0: + # Missing enums to generate (from espi.xsd) + missing_enums_usage: + - AccumulationKind + - CommodityKind + - DataQualifierKind + - FlowDirectionKind + - MeasurementKind + - TimeAttributeKind + - QualityOfReading + - AuthorizationStatus + - DataCustodianApplicationStatus + - ESPIServiceStatus + - ItemKind + - ThirdPartyApplicationType + - ThirdPartyApplicationUse + - ThirdPartyApplicatonStatus + - TokenEndPointMethod + - AnodeType + - ApnodeType + - ParticipationCategoryMPM + - tOUorCPPorConsumptionTier + + # Missing enums (shared between espi.xsd and customer.xsd) + missing_enums_shared: + - Currency + - StatusCode + - CRUDOperation + - DstRuleType + - EnrollmentStatus + - UnitMultiplierKind + - UnitSymbolKind + + # Missing enums (from customer.xsd) + missing_enums_customer: + - MediaType + - RevenueKind + + # Enums to move from common/ to usage/enums/ + move_to_usage_enums: + - AmiBillingReadyKind + - GrantType + - OAuthError + - PhaseCodeKind + - ResponseType + - TokenType + - UsagePointConnectedKind + +# Phase 1: Supporting classes +phase_1: + common_embeddables: + - RationalNumber + - DateTimeInterval + - SummaryMeasurement + - LinkType + - ReadingInterharmonic + - BillingChargeSource + + customer_embeddables: + - TelephoneNumber + - ElectronicAddress + - StreetAddress + - MeterMultiplier + - ProgramDateIdMapping + + customer_base_classes: + - Asset + - AssetContainer + - Agreement + - Organisation + - Location + - Document + - Status + - WorkLocation + - AccountNotification + - EndDeviceFields + +# Phase 2: Usage domain entities +phase_2: + usage_entities_core: + - UsagePoint + - MeterReading + - IntervalBlock + - ReadingType + - IntervalReading + - ReadingQuality + + usage_entities_summary: + - ElectricPowerQualitySummary + - UsageSummary + - TimeConfiguration + + usage_entities_customer: + - RetailCustomer + - ApplicationInformation + - Authorization + - Subscription + + usage_entities_misc: + - BatchList + - LineItem + - AggregatedNodeRef + - PnodeRef + - ServiceDeliveryPoint + +# Phase 3: Customer domain entities +phase_3: + customer_entities: + - CustomerAccount + - Customer + - CustomerAgreement + - ServiceSupplier + - ServiceLocation + - Meter + - EndDevice + - Statement + - StatementRef + - ProgramDateIdMappings