Improve CHIP SDK compatibility in Python cluster generator

[TheJulianJES]

Note: Target branch

This is a PR targeting the branch of the original PR:

• Remove chip-clusters dependency and auto generate API compatible classes  #303

This PR is to ease review of the differences compared to that PR and can later be merged into the generated-python-client branch when it's ready.

TODOs

• The comparison scripts included in this PR should be dropped before it's merged!
  • Lint checks are expected to fail because of this
• Go through all "key fixes in generator" and see if we actually want them
  • E.g. we may want to include the eventList attribute
• Go through all "intentionally not changed" notes
• Go through all "Remaining known differences"
• Test with Home Assistant Core 
• ...?

AI summary

Improves the Python cluster generator (generate-python-clusters.ts) to produce definitions that are much more closely aligned with the CHIP SDK's chip.clusters.Objects. A comparison script was used to systematically identify and fix differences across all 121 generated clusters. Fixes were verified by starting Home Assistant with the Matter integration.

Key fixes in the generator

• Fix list element optionality: The CHIP SDK uses List[Type], not List[Type | None]. List entry types are now resolved via resolveScalarType() directly, skipping optional/nullable wrappers (~183 annotations fixed).
• Use model.members for command/event fields: Commands and events now use model.members (which includes inherited fields) instead of model.children. This fixes empty WithOnOff commands in LevelControl and ~51 other missing command fields.
• Fall back to base attribute when local override is an empty stub: Derived clusters (e.g., DishwasherMode from ModeBase) can have local attribute stubs with no type or children that override the base cluster's full definition. Now falls back to the base attribute, fixing supportedModes resolving as List[uint] instead of List[ModeOptionStruct] across all mode clusters.
• Deduplicate and sort attributes: Attributes are now deduplicated by ID (fixes duplicate minLevel in LevelControl from the Matter.js model) and sorted by tag to match CHIP SDK ordering.
• Fix multi-bit bitmap masks: The generator only handled single-bit constraints ({"value": 4} → 1 << 4). Multi-bit fields use range constraints ({"min": 0, "max": 1} → bits 0-1) which fell back to 0x1 instead of the correct mask. Example from Thermostat.HVACSystemTypeBitmap:
  • CoolingStage (bits 0-1): was 0x1, now 0x3
  • HeatingStage (bits 2-3): was 0x1, now 0xC
• Use metatype as fallback when attribute type is undefined: Attributes with no explicit type but a valid metatype (e.g., float, boolean, string) were all resolving to uint. Now checks metatype first. Fixes EveCluster pressure/voltage/watt/altitude typed as uint instead of float32, and BridgedDeviceBasicInformation vendorName/reachable etc. typed as uint instead of str/bool.
• Use signed int for temperature, power, energy, voltage types: Builds a SIGNED_INT_TYPES set by walking the Matter.js datatype hierarchy (global + cluster-level) — any type whose base resolves to int8/int16/.../int64 is automatically recognized as signed. No hardcoded type aliases needed. Fixes Thermostat setpoints and calibration values, TemperatureMeasurement, DeviceEnergyManagement power values, and all other signed integer attributes that could have negative values.
• Remove redundant quotes from type annotations: The generated files use from __future__ import annotations, so explicit quoting (value: 'uint' = 0) is redundant. Now matches CHIP SDK style: value: uint = 0.
• Remove incorrect GroupId field-name override: CHIP SDK uses groupID (uppercase ID) in most command fields; the previous GroupId: "groupId" override was wrong for the majority of uses.
• Fix FIELD_NAME_OVERRIDES keys to match Matter.js model names: Override keys must use the raw model name, not the post-toChipName version. Fixes requirePINforRemoteOperation (DoorLock), IPv4Addresses/offPremiseServicesReachableIPv4 (GeneralDiagnostics), extendedPanIdPresent/panIdPresent (ThreadNetworkDiagnostics).
• Rename objects/ to cluster_defs/: The per-cluster files directory was renamed from objects to cluster_defs to avoid a macOS case-insensitivity collision (objects/ and Objects.py resolved to the same path, causing pylint to read __init__.py instead of Objects.py).
• Pass clusterName to all generators: toCamelCase and toChipClassName were only receiving clusterName in some generators (cluster-level attributes, enums). Now all generators (structs, commands, events, bitmaps) pass it through, so cluster-qualified overrides in FIELD_NAME_OVERRIDES and CLASS_NAME_OVERRIDES work in all contexts.

CHIP SDK naming compatibility

Added several override mechanisms to match CHIP SDK naming where it diverges from the Matter.js model:

• CLASS_NAME_OVERRIDES + toChipClassName(): New override map (supports both global and cluster-qualified "Cluster.Name" keys) and helper function for class names where toChipName() doesn't match the CHIP SDK. Applied to all class-name generation sites. Examples:
  • CommissioningARL, AdminVendorId, ICACCSRRequest, ACCapacityformat
  • Enum suffix stripping: WindowCovering.TypeEnum → Type, WindowCovering.EndProductTypeEnum → EndProductType, ModeTagEnum → ModeTag
  • Bitmap suffix stripping: WindowCovering.ModeBitmap → Mode, ConfigStatusBitmap → ConfigStatus, etc.
  • DoorLock legacy names: LockStateEnum → DlLockState, LockTypeEnum → DlLockType, StatusCodeEnum → DlStatus
  • Cluster-specific StatusCode handling: stripped to StatusCode in AdministratorCommissioning/RvcCleanMode/RvcRunMode/TimeSynchronization, kept as StatusCodeEnum elsewhere
  • Custom cluster acronym prevention: DraftElectricalMeasurementCluster uses Ac/Rms (not AC/RMS)
• K_VALUE_OVERRIDES: Enum/bitmap member name overrides:
  • HueSaturation → HueAndSaturation, CredentialOverTheAirAccess → CredentialsOverTheAirAccess
  • TiltBlindLift → TiltBlindLiftAndTilt, Unbolting → Unbolt
  • LedFeedback → LedFeedback (prevent LED acronym expansion in WindowCovering)
• FIELD_NAME_OVERRIDES: Additional field name fixes:
  • Lqi → lqi (CHIP SDK uses lowercase in NetworkCommissioning/ThreadNetworkDiagnostics struct fields)
• EXTRA_ENUM_MEMBERS: Inject enum members present in the CHIP SDK but missing from the Matter.js model. Used for DoorLock.DlStatus which needs kSuccess, kFailure, kInvalidField, kResourceExhausted, kNotFound.
• EXTRA_BITMAPS: Inject entire bitmap definitions missing from the Matter.js model. Used for ColorControl.Bitmaps.ColorCapabilitiesBitmap (kHueSaturation, kEnhancedHue, kColorLoop, kXy, kColorTemperature).
• Cluster-qualified FIELD_NAME_OVERRIDES: toCamelCase() now accepts optional cluster context for cluster-specific field name overrides (e.g., rmsVoltage only in DraftElectricalMeasurementCluster).

Comparison scripts

Added two utility scripts under python_client/scripts/ for verifying compatibility. These are rough developer tools used during this work and can be dropped later:

• compare_clusters.py: Detailed per-cluster structural comparison between generated and CHIP SDK clusters
• compare_summary.py: Categorized summary of systematic differences (detects case, Enum suffix, and Bitmap suffix mismatches)

Intentionally not changed

These differences from the older CHIP SDK were identified but deliberately kept as-is, since the generated output is more correct or forward-compatible:

• eventList global attribute: The CHIP SDK version doesn't include eventList (tag 0xFFFA), but it is part of the Matter spec and newer CHIP SDK versions include it. Kept.
• kUnknownEnumValue calculation: The older CHIP SDK sometimes uses 0, but the current approach (first unused value after the max defined value) avoids assigning semantically meaningful low values. Kept.
• Bitmap types in annotations: The older CHIP SDK uses plain uint for bitmap-typed fields, but using the actual bitmap type (e.g., OptionsBitmap) is more type-safe. Kept.

Issues to fix upstream in Matter.js

These are data gaps or bugs in the Matter.js model that required workarounds in the generator. Fixing them upstream would allow removing the corresponding overrides:

• Duplicate MinLevel attribute in LevelControl: The model defines MinLevel (id 2) twice — once for the Lighting conformance variant and once for non-Lighting. The generator deduplicates by ID, but the model should only expose one.
• Empty attribute stubs in derived clusters: Clusters derived from a base (e.g., DishwasherMode from ModeBase) have local attribute stubs with no type or children that shadow the base definition. This caused supportedModes to lose its List[ModeOptionStruct] type. The generator now falls back to the base, but ideally the model would inherit the full definition.
• Missing ColorCapabilitiesBitmap in ColorControl: The CHIP SDK defines this bitmap but it's absent from the Matter.js model. Currently injected via EXTRA_BITMAPS.
• Missing DlStatus enum members in DoorLock: The StatusCodeEnum only has Duplicate and Occupied, but the CHIP SDK's DlStatus also includes kSuccess, kFailure, kInvalidField, kResourceExhausted, kNotFound. Currently injected via EXTRA_ENUM_MEMBERS.
• TiltBlindLift naming in WindowCovering.TypeEnum: Matter.js uses TiltBlindLift (value 8) but the CHIP SDK calls it TiltBlindLiftAndTilt. Currently handled via K_VALUE_OVERRIDES.
• Unbolting naming in DoorLock Feature bitmap: Matter.js uses Unbolting but the CHIP SDK uses Unbolt. Currently handled via K_VALUE_OVERRIDES.
• Attributes with undefined type but valid metatype: Custom cluster attributes (e.g., EveCluster pressure, voltage, altitude) and some inherited attributes (BridgedDeviceBasicInformation vendorName, reachable) have type: undefined in the model. The generator now falls back to metatype (float → float32, string → str, boolean → bool), but ideally the model would set the type field explicitly.
• Channel.ProgramStruct.externalIDList wrong struct reference: Matter.js references ContentLauncher.AdditionalInfoStruct but CHIP SDK uses Channel.ProgramCastStruct. Cross-cluster reference issue in the model.
• Descriptor.tagList uses Globals.Structs.semtag: CHIP SDK uses Descriptor.Structs.SemanticTagStruct. The struct is defined globally in Matter.js but locally in the CHIP SDK.
• IlluminanceMeasurement.lightSensorType resolves to uint: Should be LightSensorTypeEnum but the enum reference isn't resolved. Model-level type resolution issue.
• ContentAppObserver.ContentAppMessage data/encodingHint optionality swapped: data is optional in CHIP SDK but mandatory in generated; encodingHint is the reverse. Conformance mismatch in the model.

Issues to address in HA

• Pylint E0611 (no-name-in-module) for chip.clusters.Objects: Now resolved by renaming the per-cluster directory from objects/ to cluster_defs/, which eliminates the macOS case-insensitivity collision that caused pylint to read objects/__init__.py instead of Objects.py.

Remaining known differences (not fixable at generator level)

• Module path differences (~284): Expected from the split-file architecture (chip.clusters.cluster_defs.X.X vs chip.clusters.Objects.X)
• aenum vs enum (~189): Inherent difference in MatterIntEnum base class between CHIP SDK and our implementation
• Spec version differences: Some enums/bitmaps/structs/commands exist in only one side due to different Matter spec versions between Matter.js and the CHIP SDK
• Optionality mismatches (~79 total across descriptors, commands, structs, events): Some fields are optional in CHIP SDK but mandatory in generated (or vice versa), due to conformance differences in the Matter.js model vs CHIP SDK. Notable: BridgedDeviceBasicInformation fields, mode cluster statusText, MediaPlayback trackAttributes nullability.
• Cross-cluster struct references: Matter.js resolves some structs from different clusters than the CHIP SDK (e.g., Channel.ProgramStruct.externalIDList references ContentLauncher.AdditionalInfoStruct instead of Channel.ProgramCastStruct; Descriptor.tagList uses Globals.Structs.semtag instead of Descriptor.Structs.SemanticTagStruct)
• Field name casing (a few remaining): GroupKeyManagement.groupId vs groupID, ThreadNetworkDirectory.extendedPanId vs extendedPanID — trade-offs where the override benefits the majority of uses

Test plan

• Generator runs without errors (npx tsx python_client/scripts/generate-python-clusters.ts)
• Comparison script confirms 0 duplicate fields, 0 name mismatches
• LevelControl cluster matches CHIP SDK structurally (verified manually)
• HA Matter integration loads without import errors (WindowCovering, DoorLock, DraftElectricalMeasurementCluster, ElectricalPowerMeasurement, device_types verified)
• Mode cluster supportedModes correctly typed as List[ModeOptionStruct] (DishwasherMode, RvcRunMode, etc.)
• EveCluster float attributes (pressure, voltage, etc.) correctly typed as float32
• Temperature/power/energy attributes correctly typed as int (Thermostat, TemperatureMeasurement, DeviceEnergyManagement)
• Pylint E0611 resolved after objects/ → cluster_defs/ rename
• DoorLock requirePINforRemoteOperation field name correct (lowercase for)
• GeneralDiagnostics IPv4Addresses/offPremiseServicesReachableIPv4 field names correct
• Thermostat calibration/deadband/changeAmount correctly typed as int (signed)
• NetworkCommissioning/ThreadNetworkDiagnostics lqi field name correct (lowercase)
• WindowCovering kLedFeedback bitmap member correct (not kLEDFeedback)
• Full comparison run: 0 duplicate fields, 0 name mismatches, 0 empty commands
• Filtered comparison: 284 remaining differences are all model-level (optionality, spec version, cross-cluster references)
• Full HA integration test with commissioned devices

[TheJulianJES]

HA uses DoorLock.Enums.DlStatus (here), but this does not appear to exist in Matter.js on a cluster-level, from what I can see. The generated Python definitions from the CHIP SDK for reference:

@dataclass
class DoorLock(Cluster):
    # [...]
    class Enums:

        # [...]
        class DlStatus(MatterIntEnum):
            kSuccess = 0x00
            kFailure = 0x01
            kDuplicate = 0x02
            kOccupied = 0x03
            kInvalidField = 0x85
            kResourceExhausted = 0x89
            kNotFound = 0x8B
            # All received enum values that are not listed above will be mapped
            # to kUnknownEnumValue. This is a helper enum value that should only
            # be used by code to process how it handles receiving an unknown
            # enum value. This specific value should never be transmitted.
            kUnknownEnumValue = 4

We may need to use the generic status codes in HA(?)

matterjs-server/packages/dashboard/src/util/matter-status.ts

Lines 11 to 47 in 018f8a5

	export const MatterStatusNames: Record<number, string> = {
	0: "Success",
	1: "Failure",
	125: "InvalidSubscription",
	126: "UnsupportedAccess",
	127: "UnsupportedEndpoint",
	128: "InvalidAction",
	129: "UnsupportedCommand",
	133: "InvalidCommand",
	134: "UnsupportedAttribute",
	135: "ConstraintError",
	136: "UnsupportedWrite",
	137: "ResourceExhausted",
	139: "NotFound",
	140: "UnreportableAttribute",
	141: "InvalidDataType",
	143: "UnsupportedRead",
	146: "DataVersionMismatch",
	148: "Timeout",
	155: "UnsupportedNode",
	156: "Busy",
	157: "AccessRestricted",
	195: "UnsupportedCluster",
	197: "NoUpstreamSubscription",
	198: "NeedsTimedInteraction",
	199: "UnsupportedEvent",
	200: "PathsExhausted",
	201: "TimedRequestMismatch",
	202: "FailsafeRequired",
	203: "InvalidInState",
	204: "NoCommandResponse",
	205: "TermsAndConditionsChanged",
	206: "MaintenanceRequired",
	207: "DynamicConstraintError",
	208: "AlreadyExists",
	209: "InvalidTransportType",
	};

I'll add them as hardcoded values to the generator for now, just so I can further test what else breaks HA.

[TheJulianJES]

• Remove chip-clusters dependency and auto generate API compatible classes  #303 not only modifies the CHIP cluster definitions but matter_server models as well, like device types. These are incompatible as well.
  • We should either drop them for that PR or fix them here as well.
    • I'll try to fix them here as well for now, so I can continue testing.
    • It would be cleaner to maybe have those auto-generation changes in another PR though.
  • E.g. HeatingCoolingUnit seems to be missing completely now (used by HA here)
    • Does this even exist in Matter.js?
  • There also appear to be other unrelated changes in that PR to matter_server that seem like weird AI generations, e.g. why is a separate msg str for every exception created before now?

EDIT: The HeatingCoolingUnit does not actually seem to exist, so it should be removed from HA. I'll do that later.

[TheJulianJES]

Other differences to look at later:
EDIT: Some fixed. PR description updated now.

---

Critical (would cause wrong values or crashes in HA):

1. Temperature fields as uint instead of int — Thermostat setpoints, TemperatureMeasurement. Negative temperatures would be wrong. This affects the metatype: "integer" fallback — when type is undefined, signed integers also default to uint.
2. DeviceEnergyManagement/ElectricalEnergyMeasurement signed int vs uint — Power/energy values that can be negative.
3. BridgedDeviceBasicInformation optionality inversions — Fields mandatory in Gen but optional in CHIP (would crash if device doesn't provide them).
4. Mode clusters statusText mandatory vs optional — Across all mode clusters.
5. MediaPlayback TrackStruct.trackAttributes lost nullability — Would crash on null.

Field name mismatches (could cause attribute lookup failures):

6. GeneralDiagnostics IPv4Addresses vs IPV4Addresses — We have FIELD_NAME_OVERRIDES but they may not be applied to struct field contexts.
7. GroupKeyManagement groupId vs groupID — Known trade-off.
8. DoorLock requirePINforRemoteOperation — We have override, may not apply everywhere.
9. ThreadNetworkDirectory extendedPanID vs extendedPanId

Structural issues:

10. Channel ProgramStruct.externalIDList — References wrong cluster struct (ContentLauncher.AdditionalInfoStruct instead of ProgramCastStruct)
11. Descriptor tagList — Uses Globals.Structs.semtag instead of Descriptor.Structs.SemanticTagStruct
12. WindowCovering commands have extra ignored field