fix energy mix by LKuemmel · Pull Request #3335 · openWB/core

LKuemmel · 2026-04-28T11:56:53Z

Ticket #66001913

Copilot

Pull request overview

This PR refactors measurement logging math to use centralized Decimal-based helpers, adjusts energy mix (“Strom-Mix”) calculations and derived per-source energy fields, and updates/expands tests and fixtures accordingly.

Changes:

Add precision_math utilities (decimal_add/multiply/subtract, string conversion helpers) and use them in measurement logging to reduce floating point drift.
Update energy mix analysis to compute and persist energy_imported_{grid,pv,bat,cp} per component and in totals.
Restructure tests (split unit vs integration) and update test fixtures/data to reflect new precision/units.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New shared Decimal math helpers used to standardize rounding/precision.
packages/helpermodules/measurement_logging/process_log.py	Switch totals/deltas/power math to Decimal helpers; update energy mix + per-source energy attribution.
packages/helpermodules/measurement_logging/process_log_unit_test.py	New/expanded unit coverage for energy mix edge cases and totals attribution.
packages/helpermodules/measurement_logging/process_log_integration_test.py	Integration-level test coverage for daily log processing.
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	Test fixture data for percentage totals attribution.
packages/helpermodules/measurement_logging/conftest.py	Update fixtures to match new numeric precision/units and new expected outputs.
packages/helpermodules/measurement_logging/process_log_testdata.py	Update expected processed log structures for revised math/units.
packages/helpermodules/data_migration/data_migration.py	Update imports to use string conversion helpers from `precision_math`.
packages/control/chargelog/chargelog_test.py	Update expected charged-energy-by-source values to match new precision behavior.
packages/helpermodules/measurement_logging/process_log_test.py	Removed; tests appear split into new unit/integration test modules.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    value_str = f'{decimal_value: f}'
+    return string_to_float(value_str) if "." in value_str else string_to_int(value_str)


+def calc_energy_imported_by_source(entry):
+
+    try:
        for source in ("grid", "pv", "bat", "cp"):
            if "all" in entry["hc"].keys():
-                entry["hc"]["all"][f"energy_imported_{source}"] = calc_energy_imported_by_source(
+                entry["hc"]["all"][f"energy_imported_{source}"] = decimal_multiply(
                    entry["hc"]["all"]["energy_imported"], entry["energy_source"][source])
            for key in entry["cp"].keys():
-                entry["cp"][key][f"energy_imported_{source}"] = calc_energy_imported_by_source(
+                entry["cp"][key][f"energy_imported_{source}"] = decimal_multiply(
                    entry["cp"][key]["energy_imported"], entry["energy_source"][source])


-
    except Exception:
-        log.exception(f"Fehler beim Berechnen des Strom-Mix von {entry['timestamp']}")
+        log.exception(f"Fehler beim Berechnen der Engergie-Anteile aus dem Strom-Mix von {entry['timestamp']}")


Copilot

Pull request overview

This PR updates measurement logging energy-mix calculations by introducing shared Decimal-based math helpers and expanding test coverage around energy-source attribution and totals aggregation.

Changes:

Add precision_math utilities for Decimal-based add/multiply/subtract (quantized to 0.001) and reuse them across logging computations.
Adjust measurement log processing to compute energy-source shares and derived “energy imported by source” fields per entry and for totals.
Restructure/extend tests (unit + integration) and update fixtures/test data to match the new precision and units.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New Decimal-based arithmetic + string parsing helpers reused across the codebase.
packages/helpermodules/measurement_logging/process_log.py	Switch totals/delta math to `precision_math`, update energy-mix logic, compute imported-by-source fields.
packages/helpermodules/measurement_logging/process_log_unit_test.py	New/expanded unit tests for energy mix edge cases, imported-by-source, and totals.
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	New JSON fixture for totals aggregation test coverage.
packages/helpermodules/measurement_logging/process_log_integration_test.py	Split out/get-daily-log integration test from the prior test module.
packages/helpermodules/measurement_logging/process_log_testdata.py	Update expected processed log structures and values to match new calculations.
packages/helpermodules/measurement_logging/conftest.py	Update fixtures/expected totals and processed entry shapes/values.
packages/helpermodules/data_migration/data_migration.py	Move `string_to_float/int` imports to the new `precision_math` module.
packages/control/chargelog/chargelog_test.py	Update expected charged-energy-by-source values due to precision changes.
packages/helpermodules/measurement_logging/process_log_test.py	Remove old combined test module (replaced by unit/integration split).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+def calc_energy_imported_by_source(entry):
+    try:
+        if "energy_source" not in entry.keys():
+            return
        for source in ("grid", "pv", "bat", "cp"):
            if "all" in entry["hc"].keys():
-                entry["hc"]["all"][f"energy_imported_{source}"] = calc_energy_imported_by_source(
+                entry["hc"]["all"][f"energy_imported_{source}"] = decimal_multiply(
                    entry["hc"]["all"]["energy_imported"], entry["energy_source"][source])
            for key in entry["cp"].keys():
-                entry["cp"][key][f"energy_imported_{source}"] = calc_energy_imported_by_source(
+                entry["cp"][key][f"energy_imported_{source}"] = decimal_multiply(
                    entry["cp"][key]["energy_imported"], entry["energy_source"][source])
            for counter in entry["counter"].values():
                if counter["grid"] is False:
-                    counter[f"energy_imported_{source}"] = calc_energy_imported_by_source(
+                    counter[f"energy_imported_{source}"] = decimal_multiply(
                        counter["energy_imported"], entry["energy_source"][source])


+def test_convert_value_to_kW():
+    # setup and execution
+    power = _calculate_average_power(100, 250, 300)
+
+    # evaluation
+    assert power == 1800
+


+    assert result["hc"]["all"]["energy_imported_grid"] == 7909   # (4820.5 + 3087.9) Wh = 7908.4 Wh
+    assert result["hc"]["all"]["energy_imported_pv"] == 2980     # (1734.2 + 1245.6) Wh = 2979.8 Wh
+    assert result["hc"]["all"]["energy_imported_bat"] == 912    # (287.8 + 623.7) Wh = 911.5 Wh
+    assert result["hc"]["all"]["energy_imported_cp"] == 273     # (95.3 + 178.4) Wh = 273.7 Wh
+
+    # Check cp totals (in Wh)
+    assert result["cp"]["cp1"]["energy_imported_grid"] == 22222  # (12345.7 + 9876.2) Wh = 22221.9 Wh
+    assert result["cp"]["cp1"]["energy_imported_pv"] == 8354     # (4632.1 + 3721.8) Wh = 8353.9 Wh
+    assert result["cp"]["cp1"]["energy_imported_bat"] == 3300    # (1876.4 + 1423.5) Wh = 3299.9 Wh
+    assert result["cp"]["cp1"]["energy_imported_cp"] == 802     # (234.6 + 567.1) Wh = 801.7 Wh
+
+    assert result["cp"]["cp2"]["energy_imported_grid"] == 18721  # 18721.3 Wh (only in first entry)
+    assert result["cp"]["cp2"]["energy_imported_pv"] == 7124    # 7123.8 Wh
+    assert result["cp"]["cp2"]["energy_imported_bat"] == 2955    # 2954.7 Wh
+    assert result["cp"]["cp2"]["energy_imported_cp"] == 313     # 312.9 Wh
+
+    # 11432.6 Wh (only in second entry)


Copilot

Pull request overview

This PR addresses ticket #66001913 by adjusting how the energy mix (“Strom-Mix”) and related per-source energy totals are calculated, while improving numeric precision/rounding behavior and updating the test suite accordingly.

Changes:

Introduces shared precision_math helpers (Decimal-based add/multiply/subtract + safe string parsing) and wires them into log processing.
Updates process_log energy/power calculations and energy-source attribution, including deriving energy_imported_{grid,pv,bat,cp} fields.
Refactors/expands tests (unit + integration) and updates fixtures/testdata to match the new precision and attribution behavior.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	Adds reusable Decimal-based arithmetic + safe string parsing utilities.
packages/helpermodules/measurement_logging/process_log.py	Switches key computations to shared precision helpers; updates energy mix attribution and per-source imported energy derivation.
packages/helpermodules/measurement_logging/conftest.py	Updates expected fixture values to reflect the new precision and units.
packages/helpermodules/measurement_logging/process_log_unit_test.py	Adds unit tests for totals, energy mix edge cases, per-source energy distribution, and log collection behavior.
packages/helpermodules/measurement_logging/process_log_integration_test.py	Adds an integration-style test for `get_daily_log` behavior.
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	Adds JSON test data for verifying `analyse_percentage_totals`.
packages/helpermodules/measurement_logging/process_log_testdata.py	Updates expected processed log outputs to match new calculations/precision.
packages/helpermodules/measurement_logging/process_log_test.py	Removes the previous combined test file (replaced by unit + integration tests).
packages/helpermodules/data_migration/data_migration.py	Updates imports to use the new shared string parsing helpers.
packages/control/chargelog/chargelog_test.py	Updates expected charged-energy-by-source values to reflect new rounding/precision behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

            entry["energy_source"] = {
-                "grid": grid_energy_source,
-                "pv": pv_energy_source,
-                "bat": bat_energy_source,
-                "cp": cp_energy_source}
+                "grid": format(grid_direct / consumption),
+                "pv": format(pv_direct / consumption),
+                "bat": format(bat_direct / consumption),
+                "cp": format(cp_direct / consumption)}


Copilot

Pull request overview

This PR targets ticket #66001913 (“fix energy mix”) by improving the measurement-log energy-mix calculation, introducing consistent Decimal-based arithmetic helpers, and extending log entries/migrations to include fault_state so energy-mix/cost calculations can react to component errors.

Changes:

Add precision_math helpers and refactor log processing to use Decimal-based add/multiply/subtract for stable Wh/W calculations.
Extend measurement log entries with fault_state fields and add a datastore upgrade (upgrade_datastore_122) to backfill existing logs.
Rework/expand test coverage for process_log and adjust chargelog expectations due to changed numeric precision/units.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New shared helpers for Decimal-safe arithmetic and string parsing.
packages/helpermodules/measurement_logging/process_log.py	Refactors totals/energy computations; adds fault-state aware energy-mix + per-source attribution.
packages/helpermodules/measurement_logging/write_log.py	Adds `fault_state` fields into newly written log entries.
packages/helpermodules/update_config.py	Adds datastore migration `upgrade_datastore_122` to inject `fault_state` into existing daily logs.
packages/helpermodules/update_config_test.py	Adds unit test coverage for the datastore 122 migration behavior.
packages/helpermodules/measurement_logging/conftest.py	Updates fixtures to match new Wh/W precision and added `fault_state`.
packages/helpermodules/measurement_logging/process_log_unit_test.py	New unit tests for percentage/totals/collection edge cases.
packages/helpermodules/measurement_logging/process_log_integration_test.py	New integration-style test replacing the removed older test.
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	New JSON test data for totals-based percentage analysis.
packages/helpermodules/measurement_logging/process_log_testdata.py	Updates expected processed-log structures (precision + `fault_state`).
packages/helpermodules/measurement_logging/process_log_test.py	Removes prior test file in favor of new unit/integration split.
packages/helpermodules/data_migration/data_migration.py	Switches imports to use the new shared string conversion helpers.
packages/control/chargelog/chargelog.py	Passes CP context into energy-source analysis for CP-specific attribution.
packages/control/chargelog/chargelog_test.py	Updates expected charged-energy-by-source values due to changed precision/units.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

    try:
        prices = data.data.general_data.data.prices
        try:
            grid_price = data.data.optional_data.ep_get_current_price()
+            fault_state = max(data.data.optional_data.data.electricity_pricing.flexible_tariff.get.fault_state,
+                              data.data.optional_data.data.electricity_pricing.grid_fee.get.fault_state)
        except Exception:
            grid_price = prices.grid
        prices_dict = {"grid": grid_price,
                       "pv": prices.pv,
                       "bat": prices.bat,
-                       "cp": prices.cp}
+                       "cp": prices.cp,
+                       "fault_state": fault_state}
    except Exception:


+def _analyse_energy_source(data, calc_cp: Optional[str] = None) -> Dict:
    if data and len(data["entries"]) > 0:
        try:
            for i in range(0, len(data["entries"])):
-                data["entries"][i] = analyse_percentage(data["entries"][i])
+                data["entries"][i], message_analyse = analyse_percentage(data["entries"][i])
+                if calc_cp is not None:
+                    data["entries"][i], message_calc = calc_energy_imported_by_source_cp(data["entries"][i], calc_cp)
+                else:
+                    data["entries"][i], message_calc = calc_energy_imported_by_source_all(data["entries"][i])
            data["totals"] = analyse_percentage_totals(data["entries"], data["totals"])
        except Exception:
            pub_system_message({}, "Fehler beim Berechnen des Strom-Mix", MessageType.ERROR)
+    data["message"] = message_analyse + message_calc
    return data


+        cp_section = entry.get("cp")
+        if isinstance(cp_section, dict):
+            for cp_key, cp_data in cp_section.items():
+                if isinstance(cp_data, dict):
+                    if cp_data.get("fault_state", 0) == 0 and "energy_imported" in cp_data:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            cp_data[f"energy_imported_{source}"] = decimal_multiply(
+                                cp_data["energy_imported"], energy_source[source])
+                    else:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            cp_data[f"energy_imported_{source}"] = 0
+                        message += ERROR_STATE_MESSAGE.format(f"Ladepunkt {entry['names'][cp_key]}")
+


+        else:
+            for source in ("grid", "pv", "bat", "cp"):
+                cp_data[f"energy_imported_{source}"] = 0
+            message += ERROR_STATE_MESSAGE.format(f"Ladepunkt {entry['names'][cp]}")


+                            entry["prices"]["fault_state"] = None
+                        for cp in entry["cp"].values():
+                            cp["fault_state"] = None
+                        for ev_data in entry["ev"].values():
+                            ev_data["fault_state"] = None
+                        for counter in entry["counter"].values():
+                            counter["fault_state"] = None
+                        for pv in entry["pv"].values():
+                            pv["fault_state"] = None
+                        for bat in entry["bat"].values():
+                            bat["fault_state"] = None
+                        if entry.get("hc") is not None and entry["hc"].get("all") is not None:
+                            entry["hc"]["all"]["fault_state"] = None


+        counter_section = entry.get("counter")
+        if isinstance(counter_section, dict):
+            for counter in counter_section.values():
+                if isinstance(counter, dict) and counter.get("grid") is False:
+                    if counter.get("fault_state", 0) == 0 and "energy_imported" in counter:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            counter[f"energy_imported_{source}"] = decimal_multiply(
+                                counter["energy_imported"], energy_source[source])
+                    else:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            counter[f"energy_imported_{source}"] = 0
+                        message += ERROR_STATE_MESSAGE.format(f"Zähler {entry['names'][counter]}")
+    except Exception:


+                        for cp in entry["cp"].values():
+                            cp["fault_state"] = None
+                        for ev_data in entry["ev"].values():
+                            ev_data["fault_state"] = None
+                        for counter in entry["counter"].values():
+                            counter["fault_state"] = None
+                        for pv in entry["pv"].values():
+                            pv["fault_state"] = None
+                        for bat in entry["bat"].values():


+from pprint import pprint
+from unittest.mock import Mock
+import pytest
+
+from helpermodules.measurement_logging import process_log
+
+from helpermodules.measurement_logging.process_log_testdata import (counter_jumps_forward,
+                                                                    counter_jumps_forward_processed,
+                                                                    regular_daily_log_entry,
+                                                                    regular_daily_log_entry_processed)
+
+
+@pytest.mark.parametrize("data, expected", [
+    pytest.param(counter_jumps_forward, counter_jumps_forward_processed, id="counter jumps forward"),
+    pytest.param(regular_daily_log_entry, regular_daily_log_entry_processed, id="regular daily log entry")
+])
+def test_get_daily_log(data, expected, monkeypatch):
+    # setup
+    collect_daily_log_data_mock = Mock(return_value=data)
+    monkeypatch.setattr(process_log, "_collect_daily_log_data", collect_daily_log_data_mock)
+
+    # execution
+    daily_log_processed = process_log.get_daily_log("20250616")
+    pprint(daily_log_processed)
+    # evaluation


+                if hc_all.get("fault_state", 0) == 0 and "energy_imported" in hc_all:
+                    for source in ("grid", "pv", "bat", "cp"):
+                        hc_all[f"energy_imported_{source}"] = decimal_multiply(
+                            hc_all["energy_imported"], energy_source[source])
+                else:
+                    for source in ("grid", "pv", "bat", "cp"):
+                        hc_all[f"energy_imported_{source}"] = 0
+                    message += ERROR_STATE_MESSAGE.format("den Hausverbrauch")
+
+        cp_section = entry.get("cp")
+        if isinstance(cp_section, dict):
+            for cp_key, cp_data in cp_section.items():
+                if isinstance(cp_data, dict):
+                    if cp_data.get("fault_state", 0) == 0 and "energy_imported" in cp_data:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            cp_data[f"energy_imported_{source}"] = decimal_multiply(
+                                cp_data["energy_imported"], energy_source[source])
+                    else:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            cp_data[f"energy_imported_{source}"] = 0
+                        message += ERROR_STATE_MESSAGE.format(f"Ladepunkt {entry['names'][cp_key]}")
+


Copilot

Pull request overview

This PR updates measurement logging and energy-mix (“Strom-Mix”) calculations to incorporate component fault states and improve numeric precision/rounding consistency across log processing, plus a datastore migration bump to version 122.

Changes:

Introduces precision_math helpers for Decimal-based arithmetic and safe string-to-number conversions, and applies them in log processing.
Extends measurement log entries and datastore migration to include fault_state fields across prices/CP/EV/counter/PV/battery/home-consumption sections.
Refactors and expands tests for measurement log processing and chargelog energy-by-source calculations.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New utility module for Decimal-based arithmetic and parsing helpers.
packages/helpermodules/update_config_test.py	Adds tests for datastore upgrade to version 122.
packages/helpermodules/update_config.py	Bumps `DATASTORE_VERSION` to 122 and adds `upgrade_datastore_122()` to add `fault_state` fields in logs.
packages/helpermodules/setdata.py	Adds validation for new chargepoint `fault_state` / `fault_str` set topics.
packages/helpermodules/measurement_logging/write_log.py	Writes `fault_state` into logged sections (prices/cp/ev/counter/pv/bat/hc).
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	Adds fixture data for totals-based percentage analysis testing.
packages/helpermodules/measurement_logging/process_log_unit_test.py	New unit tests for log processing, energy-source analysis, and collection behavior.
packages/helpermodules/measurement_logging/process_log_testdata.py	Updates test data to include `fault_state` and new Wh/precision expectations.
packages/helpermodules/measurement_logging/process_log_test.py	Removes older test module (split into unit/integration tests).
packages/helpermodules/measurement_logging/process_log_integration_test.py	Adds integration coverage for `get_daily_log` processing path.
packages/helpermodules/measurement_logging/process_log.py	Refactors energy-mix logic, adds fault-state handling, switches arithmetic to `precision_math`, and changes some unit assumptions to Wh.
packages/helpermodules/measurement_logging/conftest.py	Updates fixtures/expected values for new precision and fault-state fields.
packages/helpermodules/data_migration/data_migration.py	Moves `string_to_float/int` imports to `precision_math`.
packages/control/chargepoint/chargepoint_all.py	Adds aggregate `fault_state`/`fault_str` fields and updates sum logic to ignore faulty chargepoints.
packages/control/chargelog/chargelog_test.py	Updates expected charged-energy-by-source values to match new precision behavior.
packages/control/chargelog/chargelog.py	Passes chargepoint context into reference-entry processing and uses updated energy-source analysis path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

    try:
        prices = data.data.general_data.data.prices
        try:
            grid_price = data.data.optional_data.ep_get_current_price()
+            fault_state = max(data.data.optional_data.data.electricity_pricing.flexible_tariff.get.fault_state,
+                              data.data.optional_data.data.electricity_pricing.grid_fee.get.fault_state)
        except Exception:
            grid_price = prices.grid
        prices_dict = {"grid": grid_price,
                       "pv": prices.pv,
                       "bat": prices.bat,
-                       "cp": prices.cp}
+                       "cp": prices.cp,
+                       "fault_state": fault_state}
    except Exception:


+        hc_dict = {"all": {"imported": data.data.counter_all_data.data.set.imported_home_consumption,
+                           "fault_state": data.data.counter_all_data.data.set.invalid_home_consumption}}


+def _analyse_energy_source(data, calc_cp: Optional[str] = None) -> Dict:
    if data and len(data["entries"]) > 0:
        try:
            for i in range(0, len(data["entries"])):
-                data["entries"][i] = analyse_percentage(data["entries"][i])
+                data["entries"][i], message_analyse = analyse_percentage(data["entries"][i])
+                if calc_cp is not None:
+                    data["entries"][i], message_calc = calc_energy_imported_by_source_cp(
+                        data["entries"][i], calc_cp, data["names"][calc_cp])
+                else:
+                    data["entries"][i], message_calc = calc_energy_imported_by_source_all(
+                        data["entries"][i], data["names"])
+                data["message"] += message_analyse + message_calc
            data["totals"] = analyse_percentage_totals(data["entries"], data["totals"])


+        counter_section = entry.get("counter")
+        if isinstance(counter_section, dict):
+            for counter in counter_section.values():
+                if isinstance(counter, dict) and counter.get("grid") is False:
+                    if counter.get("fault_state", 0) != 2 and "energy_imported" in counter:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            counter[f"energy_imported_{source}"] = decimal_multiply(
+                                counter["energy_imported"], energy_source[source])
+                    else:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            counter[f"energy_imported_{source}"] = 0
+                        message += ERROR_STATE_MESSAGE.format(f"Zähler {names[counter]}")
+    except Exception:


+        else:
+            for source in ("grid", "pv", "bat", "cp"):
+                cp_data[f"energy_imported_{source}"] = 0
+            message += ERROR_STATE_MESSAGE.format(f"Ladepunkt {name[cp]}")


Copilot

Pull request overview

This PR updates the measurement logging “energy mix” pipeline by introducing fault-state awareness in log entries, migrating existing daily logs to include the new fields, and switching energy/power calculations to consistent, rounded Decimal-based math.

Changes:

Added shared precision math helpers and refactored measurement log processing to use Decimal-based add/subtract/multiply.
Extended daily log schema to include fault_state (and related handling) across prices/cp/ev/counter/pv/bat/hc, plus a datastore upgrade to backfill the field.
Reworked/expanded automated tests for log processing, energy mix analysis, and charge log cost calculations.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New Decimal-based helpers for consistent rounding/conversion.
packages/helpermodules/update_config_test.py	Adds tests for datastore upgrade 122 log-file conversion.
packages/helpermodules/update_config.py	Bumps datastore version to 122 and adds `upgrade_datastore_122()` to backfill `fault_state` in daily logs.
packages/helpermodules/setdata.py	Validates new `openWB/set/chargepoint/get/fault_state` and `fault_str` topics.
packages/helpermodules/measurement_logging/write_log.py	Writes `fault_state` into log entries (prices/cp/ev/counter/pv/bat/hc).
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	New JSON fixture data for totals-based percentage analysis tests.
packages/helpermodules/measurement_logging/process_log_unit_test.py	New unit test suite for processing/percentage/totals and log collection.
packages/helpermodules/measurement_logging/process_log_testdata.py	Updates expected processed log structures (incl. `fault_state`, new unit scales).
packages/helpermodules/measurement_logging/process_log_test.py	Removes prior combined tests (split into unit + integration).
packages/helpermodules/measurement_logging/process_log_integration_test.py	Adds integration-style test for `get_daily_log` using fixture data.
packages/helpermodules/measurement_logging/process_log.py	Refactors energy/power calculations, adds fault-state logic, uses precision_math helpers.
packages/helpermodules/measurement_logging/conftest.py	Updates fixtures/expected values for new units and schema additions.
packages/helpermodules/data_migration/data_migration.py	Moves `string_to_float/int` imports to the new precision_math module.
packages/control/chargepoint/chargepoint_all.py	Adds aggregated `fault_state`/`fault_str` to chargepoint-all and alters sum behavior under faults.
packages/control/chargelog/chargelog_test.py	Updates chargelog tests to include `names` and new rounded expected values.
packages/control/chargelog/chargelog.py	Passes cp context into energy-source analysis so chargelog uses per-CP source breakdown.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+                        log.exception("Fehler in der allgemeinen Ladepunkt-Klasse für Ladepunkt "+cp)
+                    try:
+                        power = power + cp.data.get.power
+                    except Exception:
+                        log.exception("Fehler in der allgemeinen Ladepunkt-Klasse für Ladepunkt "+cp)


+        grids = [counter for counter in entry["counter"].values() if counter.get("grid")]
+        if not grids:
+            raise KeyError(f"Kein Zähler für das Netz gefunden in Eintrag '{entry['timestamp']}'.")
+        max_grid = max(grids, key=lambda g: g.get("fault_state", 0))
+        return max_grid.get("fault_state", 0)


Copilot

Pull request overview

This PR updates measurement logging and processing to “fix energy mix” calculations by introducing fault-state-aware handling and more consistent Decimal-based precision arithmetic, alongside a datastore bump/migration and corresponding test updates.

Changes:

Add fault_state fields into daily-log entries (prices, cp, ev, counter, pv, bat, hc) and migrate existing logs via datastore upgrade to version 122.
Refactor energy-mix and totals calculations in process_log.py to use shared Decimal helpers and produce more stable numeric results.
Update chargepoint aggregation and charge-log cost calculations to incorporate fault state and revised log processing.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New shared helpers for safe string parsing and Decimal add/mul/sub with quantized precision.
packages/helpermodules/update_config.py	Bump datastore to 122 and add log conversion to inject `fault_state` fields.
packages/helpermodules/update_config_test.py	Tests for new datastore upgrade 122 log conversion behavior.
packages/helpermodules/setdata.py	Validate new aggregated chargepoint `fault_state` / `fault_str` set-topics.
packages/helpermodules/measurement_logging/write_log.py	Write `fault_state` into logged sections (prices/cp/ev/counter/pv/bat/hc).
packages/helpermodules/measurement_logging/process_log.py	Energy-mix calculation refactor; fault-state-aware attribution; Decimal-based totals/power/energy differences.
packages/helpermodules/measurement_logging/conftest.py	Update fixtures to reflect new units/precision and added `fault_state` fields.
packages/helpermodules/measurement_logging/process_log_unit_test.py	New/expanded unit tests for energy-mix and attribution edge cases.
packages/helpermodules/measurement_logging/process_log_integration_test.py	Integration-style test moved/split from prior test file.
packages/helpermodules/measurement_logging/process_log_testdata.py	Update test data structures/expected processed outputs for new behavior.
packages/helpermodules/measurement_logging/process_log_test.py	Removed legacy test file (replaced by unit/integration split).
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	New fixture data for percentage totals analysis.
packages/helpermodules/data_migration/data_migration.py	Import `string_to_float/int` from new `precision_math` module.
packages/control/chargepoint/chargepoint_all.py	Aggregate `fault_state`/`fault_str` and exclude faulty CPs from summed readings.
packages/control/chargelog/chargelog.py	Pass CP context into reference-entry processing to compute CP-specific energy-source attribution.
packages/control/chargelog/chargelog_test.py	Update mocks/expected values for new processed-log structure and precision.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

-        hc_dict = {"all": {"imported": data.data.counter_all_data.data.set.imported_home_consumption}}
+        hc_dict = {"all": {
+            "imported": data.data.counter_all_data.data.set.imported_home_consumption,
+            "fault_state": 3 if data.data.counter_all_data.data.set.invalid_home_consumption >= 3 else 0}}


+                    else:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            cp_data[f"energy_imported_{source}"] = 0
+                        message += ERROR_STATE_MESSAGE.format(f"Ladepunkt {names[cp_key]}")
+


+                        log.exception("Fehler in der allgemeinen Ladepunkt-Klasse für Ladepunkt "+cp.num)
+                    try:
+                        power = power + cp.data.get.power
+                    except Exception:
+                        log.exception("Fehler in der allgemeinen Ladepunkt-Klasse für Ladepunkt "+cp.num)


Copilot

Pull request overview

This PR updates the measurement logging “energy mix” computation to be more robust and precise by introducing fault-state handling across logged components and by centralizing Decimal-based arithmetic to reduce floating-point drift.

Changes:

Add fault_state/fault_str for aggregated chargepoint data and propagate fault-state fields into daily log entries + datastore upgrade to backfill fields.
Refactor process_log energy-mix computation and energy-by-source calculations, using shared precision_math helpers for consistent rounding/precision.
Rework and expand unit/integration tests for process_log, including new JSON test data.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/helpermodules/utils/precision_math.py	New shared Decimal helpers for add/multiply/subtract + string parsing.
packages/helpermodules/measurement_logging/process_log.py	Refactors energy-mix logic, adds fault-state aware splits, switches to precision helpers, adjusts log collection defaults.
packages/helpermodules/measurement_logging/write_log.py	Logs `fault_state` into prices/cp/ev/counter/pv/bat/hc sections.
packages/control/chargepoint/chargepoint_all.py	Adds aggregated CP `fault_state`/`fault_str` and modifies sum behavior under fault.
packages/helpermodules/setdata.py	Validates incoming CP aggregated `fault_state`/`fault_str` topics.
packages/helpermodules/update_config.py	Bumps datastore version to 122 and adds daily-log conversion to include `fault_state` keys.
packages/helpermodules/update_config_test.py	Adds tests for datastore upgrade 122 conversion behavior.
packages/control/chargelog/chargelog.py	Passes CP context into log processing so charged-energy-by-source can be computed per-CP.
packages/control/chargelog/chargelog_test.py	Updates mocks/expected values for new names structure + new precision behavior.
packages/helpermodules/data_migration/data_migration.py	Moves `string_to_float/int` import to the new `precision_math` module.
packages/helpermodules/measurement_logging/conftest.py	Updates fixtures for new units/precision and fault_state fields.
packages/helpermodules/measurement_logging/process_log_unit_test.py	Adds comprehensive unit tests for edge cases and new behavior.
packages/helpermodules/measurement_logging/process_log_integration_test.py	Splits former tests into integration-style test for `get_daily_log`.
packages/helpermodules/measurement_logging/process_log_testdata.py	Updates testdata to include `fault_state` fields and new computed values.
packages/helpermodules/measurement_logging/test_data_analyse_percentage_totals.json	Adds fixture data for totals percentage analysis.
packages/helpermodules/measurement_logging/process_log_test.py	Removes the previous combined test module (replaced by new unit/integration tests).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+                    else:
+                        for source in ("grid", "pv", "bat", "cp"):
+                            counter_data[f"energy_imported_{source}"] = 0
+                        message += ERROR_STATE_MESSAGE.format(f"Zähler {names[counter_key]}")


fix energy mix

ef51b64

LKuemmel added this to the 2.2.0-Patch.1 milestone Apr 28, 2026

LKuemmel requested a review from Copilot April 28, 2026 11:56

Copilot started reviewing on behalf of LKuemmel April 28, 2026 11:57 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

review

b3a374b

LKuemmel requested a review from Copilot April 28, 2026 12:51

Copilot started reviewing on behalf of LKuemmel April 28, 2026 12:52 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

review

2eb8117

LKuemmel requested a review from Copilot April 29, 2026 06:50

Copilot started reviewing on behalf of LKuemmel April 29, 2026 06:51 View session

clean up

1351444

Copilot AI reviewed Apr 29, 2026

View reviewed changes

error handling

297fd25

LKuemmel requested a review from Copilot April 29, 2026 14:02

Copilot started reviewing on behalf of LKuemmel April 29, 2026 14:02 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

LKuemmel added 2 commits April 29, 2026 16:35

review, fault state for chargepoint all

2f58515

update datastore version

eef2671

LKuemmel requested a review from Copilot April 30, 2026 05:44

Copilot started reviewing on behalf of LKuemmel April 30, 2026 05:44 View session

Copilot AI reviewed Apr 30, 2026

View reviewed changes

LKuemmel added 2 commits April 30, 2026 08:17

review

5184722

review

891c02e

LKuemmel requested a review from Copilot April 30, 2026 07:22

Copilot started reviewing on behalf of LKuemmel April 30, 2026 07:23 View session

Copilot AI reviewed Apr 30, 2026

View reviewed changes

review

65b482c

LKuemmel requested a review from Copilot April 30, 2026 07:53

Copilot started reviewing on behalf of LKuemmel April 30, 2026 07:53 View session

Copilot AI reviewed Apr 30, 2026

View reviewed changes

review

30b7115

LKuemmel requested a review from Copilot April 30, 2026 08:06

Copilot started reviewing on behalf of LKuemmel April 30, 2026 08:06 View session

Copilot AI reviewed Apr 30, 2026

View reviewed changes

LKuemmel added 4 commits April 30, 2026 10:44

error handling

e1e95ad

fixes

3770306

undo comment

0855a4c

mock path

14f6911

LKuemmel marked this pull request as ready for review April 30, 2026 13:43

LKuemmel merged commit 08f0b25 into master Apr 30, 2026
1 check passed

LKuemmel deleted the fix_energy_mix branch April 30, 2026 13:47

		value_str = f'{decimal_value: f}'
		return string_to_float(value_str) if "." in value_str else string_to_int(value_str)

		hc_dict = {"all": {"imported": data.data.counter_all_data.data.set.imported_home_consumption,
		"fault_state": data.data.counter_all_data.data.set.invalid_home_consumption}}

Conversation

LKuemmel commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

LKuemmel commented Apr 28, 2026 •

edited

Loading