Merge pull request #177 from RonanB96/workstream-3/completeness-missing-features

Workstream 3/completeness missing features

Author: RonanB96

.github/ai-agent-characteristic-rules.md

@@ -26,12 +26,34 @@ SIG characteristics auto-resolve UUID. Custom require `_info = CharacteristicInf
 
 **Your `_decode_value()` only:** Parse bytes using templates, apply scaling, return typed result
 
+**`_python_type` is auto-resolved** from `BaseCharacteristic[T]` generic parameter. Never set `_python_type` on new characteristics — the generic param and template already provide the type. `dict` is banned.
+
 **Patterns:**
-- Simple value → Use template (`Uint8Template`, etc.)
+- Simple value → Use template from `templates/` package (`Uint8Template`, `ScaledUint16Template`, etc.)
 - Multi-field → Override `_decode_value()`, return `msgspec.Struct`
 - Enum/bitfield → Use `IntFlag`
+- Parsing pipeline → `pipeline/parse_pipeline.py` for multi-stage decode, `pipeline/encode_pipeline.py` for encode, `pipeline/validation.py` for range/type/length checks
+
+**Templates package** (`gatt/characteristics/templates/`):
+- `numeric.py` — `Uint8Template`, `Uint16Template`, `Sint8Template`, etc.
+- `scaled.py` — `ScaledUint8Template(d=N, b=N)`, `ScaledSint16Template`, etc.
+- `string.py` — `Utf8StringTemplate`
+- `enum.py` — `EnumTemplate`
+- `ieee_float.py` — `IEEE11073FloatTemplate`, `IEEE11073SFloatTemplate`
+- `composite.py` — `CompositeTemplate`
+- `domain.py` — domain-specific templates
+- `data_structures.py` — structured data templates
+- `base.py` — `CodingTemplate` base class
+
+**Core decomposition** (`core/`):
+- `query.py` — characteristic/service lookup
+- `parser.py` — raw bytes → Python values
+- `encoder.py` — Python values → raw bytes
+- `registration.py` — custom characteristic/service registration
+- `service_manager.py` — service lifecycle management
+- `translator.py` — public facade (`BluetoothSIGTranslator`)
 
 **Standards:**
 - Multi-byte: little-endian
 
-**Reference:** See `battery_level.py`, `heart_rate_measurement.py`, `templates.py`
+**Reference:** See `battery_level.py`, `heart_rate_measurement.py`, `templates/`

.github/instructions/bluetooth-gatt.instructions.md

@@ -7,9 +7,14 @@ applyTo: "docs/**/*.md, docs/*.md"
 ## Code Samples
 
 - Every code block must be runnable and validated
-- Use Sphinx cross-references: `:class:`CharacteristicData``, `:meth:`parse_characteristic``
+- Use Sphinx cross-references: `:class:`BluetoothSIGTranslator``, `:meth:`parse_characteristic``
 - Pair code with expected output
 
+## Architecture Diagrams
+
+- Use Mermaid for architecture and flow diagrams in Markdown docs
+- Keep diagrams close to the code they describe
+
 ## Style
 
 - Google-style docstrings (see python-implementation.instructions.md)

.github/instructions/documentation.instructions.md

@@ -9,11 +9,12 @@ applyTo: "**/*.py,**/pyproject.toml,**/requirements*.txt"
 - Hardcoded UUIDs (use registry resolution)
 - `from typing import Optional` (use `Type | None`)
 - `TYPE_CHECKING` blocks
-- Lazy/conditional imports in core logic
+- Lazy/conditional imports in core logic (deferred imports to break cycles are acceptable with a valid `# NOTE:` comment)
 - Untyped public function signatures
 - `hasattr`/`getattr` when direct access is possible
 - Bare `except:` or silent `pass`
 - Returning raw `dict` or `tuple` (use `msgspec.Struct`)
+- Setting `_python_type` on new characteristics (`BaseCharacteristic[T]` generic param auto-resolves)
 - Magic numbers without named constants
 
 ## Type Safety (MANDATORY)
@@ -26,6 +27,19 @@ applyTo: "**/*.py,**/pyproject.toml,**/requirements*.txt"
 
 - Use `msgspec.Struct` (frozen, kw_only)
 
+## Characteristic Implementation Patterns
+
+- **Simple scalars:** Use a template from `templates/` package (`Uint8Template`, `ScaledUint16Template`, etc.)
+- **Multi-field composites:** Override `_decode_value()`, use `DataParser` for field extraction, return `msgspec.Struct`
+- **Parsing pipeline:** Use `pipeline/parse_pipeline.py` for multi-stage decode with validation
+- **Core modules:** `core/encoder.py` for Python→bytes, `core/parser.py` for bytes→Python, `core/query.py` for lookups
+
+## Peripheral Device Patterns
+
+- `PeripheralDevice` (in `device/peripheral_device.py`) for server-side BLE
+- `PeripheralManagerProtocol` (in `device/peripheral.py`) for adapter abstraction
+- Fluent configuration: `device.with_name(...).with_service(...)`
+
 ## Docstrings
 
 - Google style (Args, Returns, Raises)

.github/instructions/python-implementation.instructions.md

@@ -22,12 +22,22 @@ Every new function needs: success + 2 failure cases minimum.
 - Boundaries: min, max, just outside
 - Sentinels: 0xFF, 0xFFFF, 0x8000
 
+## Test Directory Structure
+
+- `tests/gatt/` — characteristic and service tests (the primary pattern)
+- `tests/device/` — device, peripheral, advertising tests
+- `tests/core/` — translator, encoder, parser, query tests
+- `tests/benchmarks/` — performance benchmarks (`test_performance.py`, `test_comparison.py`)
+- `tests/static_analysis/` — registry completeness, code quality checks
+
 ## Commands
 
 ```bash
 python -m pytest tests/ -v
 python -m pytest -k "battery" -v
 python -m pytest --lf
+python -m pytest tests/ --cov=bluetooth_sig --cov-fail-under=85
+python -m pytest tests/benchmarks/ --benchmark-only
 ```
 
 Tests must be deterministic. See `tests/gatt/` for patterns.

.github/instructions/testing.instructions.md

@@ -55,7 +55,7 @@ jobs:
 
       - name: Run tests with coverage
         run: |
-          python -m pytest tests/ -n auto --ignore=tests/benchmarks/ --junitxml=test-results.xml --cov=src/bluetooth_sig --cov-report=html --cov-report=xml --cov-report=term-missing --cov-fail-under=70
+          python -m pytest tests/ -n auto --ignore=tests/benchmarks/ --junitxml=test-results.xml --cov=src/bluetooth_sig --cov-report=html --cov-report=xml --cov-report=term-missing --cov-fail-under=85
 
       - name: Publish JUnit test report
         if: always()

.github/workflows/test-coverage.yml

@@ -0,0 +1,30 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+
+        {
+            "name": "Python Debugger: Current File",
+            "type": "debugpy",
+            "request": "launch",
+            "program": "${file}",
+            "console": "integratedTerminal"
+        }
+        ,
+        {
+            "name": "Pytest: Current Test",
+            "type": "debugpy",
+            "request": "launch",
+            "console": "integratedTerminal",
+            "justMyCode": false,
+            "module": "pytest",
+            "args": [
+                "-q",
+                "--rootdir=/root/homeassistant/custom_components/bluetooth-sig-python",
+                "${file}::${pytestTest}"
+            ]
+        }
+    ]
+}

.vscode/launch.json

@@ -0,0 +1,6 @@
+{
+    "chat.tools.terminal.autoApprove": {
+        "bluetoothctl": true,
+        "hciconfig": true
+    }
+}

.vscode/settings.json

@@ -0,0 +1,134 @@
+# Workstream 1: God Class Decomposition Plan
+
+**Goal**: Decompose 4 God classes using **composition + delegation**. Preserve all public APIs. No logic duplication. Single source of truth per concern.
+
+**Execution order**: Step 3 → Step 1 → Step 4 → Step 2 (simplest first, most complex last).
+
+---
+
+## Step 1: Split `BluetoothSIGTranslator` (1,359 lines → ~682 line facade) — ✅ COMPLETE
+
+The class is nearly stateless — only `_services: dict` is mutable. All other methods delegate to static registries. Pattern: **Composition + Delegation Facade** (like `requests.Session`).
+
+### 1.1 New delegate modules under `src/bluetooth_sig/core/`
+
+| New file | Class | Methods moved from translator.py | Mutable state |
+|---|---|---|---|
+| `core/query.py` | `CharacteristicQueryEngine` | `supports`, `get_value_type`, all 8 `get_*_info_*`, `get_characteristics_info_by_uuids`, `list_supported_*`, `get_service_characteristics`, `get_sig_info_by_name`, `get_sig_info_by_uuid` | None |
+| `core/parser.py` | `CharacteristicParser` | `parse_characteristic` (+overloads), `parse_characteristics` (batch), 5 `_*` batch helpers | None |
+| `core/encoder.py` | `CharacteristicEncoder` | `encode_characteristic` (+overloads), `create_value`, `validate_characteristic_data`, `_get_characteristic_value_type_class` | None |
+| `core/registration.py` | `RegistrationManager` | `register_custom_characteristic_class`, `register_custom_service_class` | None (writes to registries) |
+| `core/service_manager.py` | `ServiceManager` | `process_services`, `get_service_by_uuid`, `discovered_services`, `clear_services` | `_services: dict` — **only mutable state** |
+
+### 1.2 Facade pattern
+
+`BluetoothSIGTranslator.__init__` creates 5 delegate instances eagerly. Every public method becomes a one-line delegation with identical signature and `@overload` decorators. Async wrappers stay on facade. Singleton `__new__`/`get_instance()`/global `BluetoothSIG` stay.
+
+### 1.3 Update `core/__init__.py`
+
+Re-export delegate classes alongside `BluetoothSIGTranslator` and `AsyncParsingSession`.
+
+---
+
+## Step 2: Split `BaseCharacteristic` (1,761 lines → 1,258 lines) — ✅ COMPLETE
+
+Keeps Template Method contract (`_decode_value`/`_encode_value`). Internal composition invisible to ~150 subclasses. Pattern: **Internal Composition with back-reference**.
+
+### 2.1 New `pipeline/` package under `src/bluetooth_sig/gatt/characteristics/`
+
+| New file | Class | Methods extracted | Status |
+|---|---|---|---|
+| `pipeline/__init__.py` | Re-exports | — | ✅ |
+| `pipeline/parse_pipeline.py` | `ParsePipeline` | `parse_value` orchestration, `_perform_parse_validation`, `_extract_and_check_special_value`, `_decode_and_validate_value`, `_extract_raw_int`, `_check_special_value`, `_is_parse_trace_enabled` | ✅ |
+| `pipeline/encode_pipeline.py` | `EncodePipeline` | `build_value` orchestration, `_pack_raw_int`, `encode_special`, `encode_special_by_meaning` | ✅ |
+| `pipeline/validation.py` | `CharacteristicValidator` | `_validate_range` (3-level precedence), `_validate_type`, `_validate_length` | ✅ |
+
+### 2.2 Additional extractions
+
+| New file | Class | Methods extracted | Status |
+|---|---|---|---|
+| `role_classifier.py` | `classify_role()` function | `_classify_role`, `_spec_has_unit_fields` | ✅ |
+| `descriptor_support.py` | `DescriptorSupport` | 11 descriptor methods | Deferred — low value, methods are 1-liner proxies |
+| `special_values.py` | `SpecialValueHandler` | special value methods | Deferred — already uses SpecialValueResolver |
+
+### 2.3 What stays on `BaseCharacteristic`
+
+- `__init__`/`__post_init__` (composition wiring)
+- Properties: `uuid`, `info`, `spec`, `name`, `description`, `display_name`, `unit`, `size`, `value_type_resolved`, `role`, `get_byte_order_hint`
+- Abstract: `_decode_value`, `_encode_value` (Template Method hooks for subclasses)
+- Thin delegation: `parse_value` → `ParsePipeline.run()`, `build_value` → `EncodePipeline.run()`, `encode_special*` → `EncodePipeline`
+- Class-level UUID resolution (5 classmethods)
+- Dependency resolution (5 methods)
+- YAML metadata accessors (5 methods)
+- Descriptor methods (kept in base, 1-liner proxies to descriptor_utils)
+- Special value properties (kept in base, delegate to SpecialValueResolver)
+- YAML metadata accessors (5 methods)
+- Proxy methods for backward compat (delegate to composed objects)
+
+---
+
+## Step 3: Split `templates.py` (1,488 lines → package) — ✅ COMPLETE
+
+No circular dependencies. Pure file reorganisation + re-export. Pattern: **Module → Package promotion**.
+
+### 3.1 New `templates/` package
+
+| New file | Classes | Approx lines |
+|---|---|---|
+| `templates/__init__.py` | Re-exports everything via explicit imports + `__all__` | ~60 |
+| `templates/base.py` | `CodingTemplate[T_co]` (ABC), resolution constants | ~100 |
+| `templates/data_structures.py` | `VectorData`, `Vector2DData`, `TimeData` | ~40 |
+| `templates/numeric.py` | `Uint8Template`, `Sint8Template`, `Uint16Template`, `Sint16Template`, `Uint24Template`, `Uint32Template` | ~200 |
+| `templates/scaled.py` | `ScaledTemplate` (abstract) + 8 `Scaled*Template` variants + `PercentageTemplate` | ~400 |
+| `templates/domain.py` | `TemperatureTemplate`, `ConcentrationTemplate`, `PressureTemplate` | ~200 |
+| `templates/ieee_float.py` | `IEEE11073FloatTemplate`, `Float32Template` | ~80 |
+| `templates/string.py` | `Utf8StringTemplate`, `Utf16StringTemplate` | ~150 |
+| `templates/complex.py` | `TimeDataTemplate`, `VectorTemplate`, `Vector2DTemplate` | ~200 |
+| `templates/enum.py` | `EnumTemplate[T]` | ~240 |
+
+### 3.2 Backward compat
+
+Python resolves `from .templates import X` identically whether `templates` is a module or package — as long as `X` is in the package `__init__.py`. Zero characteristic files need changing.
+
+---
+
+## Step 4: Split `Device` (1,172 lines → ~818 lines) — ✅ COMPLETE
+
+13/40 methods are pure delegation. Substantial logic in dependency resolution and characteristic I/O. Pattern: **Composition with explicit dependencies (no back-references)**.
+
+### 4.1 New modules under `src/bluetooth_sig/device/`
+
+| New file | Class | Methods extracted |
+|---|---|---|
+| `dependency_resolver.py` | `DependencyResolver` + `DependencyResolutionMode` enum | `_resolve_single_dependency`, `_ensure_dependencies_resolved` |
+| `characteristic_io.py` | `CharacteristicIO` | `read` (+overloads), `write` (+overloads), `start_notify` (+overloads), `stop_notify`, `read_multiple`, `write_multiple`, `_resolve_characteristic_name` |
+
+### 4.2 Device composes
+
+```python
+self._dep_resolver = DependencyResolver(connection_manager, translator, self.connected)
+self._char_io = CharacteristicIO(connection_manager, translator, self.connected, self._dep_resolver)
+```
+
+Remaining on Device: delegation one-liners, discovery, advertising, properties, service queries.
+
+---
+
+## Verification (after each step)
+
+1. `python -m pytest tests/ -v` — all existing tests pass
+2. `./scripts/lint.sh --all` — zero errors
+3. `./scripts/format.sh --check` — formatting valid
+4. Backward compat imports still work
+
+## Design Principles
+
+| Principle | Application |
+|---|---|
+| **Single Responsibility** | Each delegate/composed class owns one concern |
+| **DRY** | Each method exists in exactly one place; facade only delegates |
+| **Composition over Inheritance** | Translator: 5 delegates. Base: internal composition. Templates: domain grouping |
+| **Single Source of Truth** | Registry access per delegate. Validation in one validator. Pipeline in one orchestrator |
+| **Open/Closed** | BaseCharacteristic open for extension (override hooks), closed for modification (pipeline internal) |
+| **Dependency Inversion** | Delegates take abstractions (registries, protocols), not concretions |
+| **Interface Segregation** | QueryEngine separate from Parser — consumers depend only on what they use |

DECOMPOSITION_PLAN.md

@@ -136,8 +136,9 @@ for char in client.services.characteristics:
     uuid_str = str(char.uuid)
     if translator.supports(uuid_str):
         raw_data = await client.read_gatt_char(uuid_str)  # SKIP: async
-        result = translator.parse_characteristic(uuid_str, raw_data)
-        print(f"{result.info.name}: {result.value}")  # Returns Any
+        parsed = translator.parse_characteristic(uuid_str, raw_data)
+        info = translator.get_characteristic_info_by_uuid(uuid_str)
+        print(f"{info.name}: {parsed}")  # parsed is the value directly (Any)
     else:
         print(f"Unknown characteristic UUID: {uuid_str}")
 ```
@@ -215,8 +216,8 @@ battery_uuid = translator.get_characteristic_uuid_by_name(CharacteristicName.BAT
 async with BleakClient(address) as client:
     # Read: bleak handles connection, bluetooth-sig handles parsing
     raw_data = await client.read_gatt_char(str(battery_uuid))
-    result = translator.parse_characteristic(str(battery_uuid), raw_data)
-    print(f"Battery: {result.value}%")
+    level = translator.parse_characteristic(str(battery_uuid), raw_data)
+    print(f"Battery: {level}%")
 
     # Write: bluetooth-sig handles encoding, bleak handles transmission
     data = translator.encode_characteristic(str(battery_uuid), 85)