Hot-swap re-solve + mutable operational Params

CHANGELOG.md

@@ -3,7 +3,9 @@
 ## [4.18.17RC] - 2026-06-13 Unreleased in PyPI
 
 - [CHANGED] fix some errors in writing H2 and heat network output results
-- [ADDED] Mode C post-build hot-swap re-solve (`openTEPES_ProblemSolvingResolve.py`). `resolve(OptModel, SolverName, overlays)` re-solves an already-built model once per parameter overlay without rebuilding it, so a sweep, sensitivity study or Monte-Carlo reuses the one slow build; `overlay_scaled` builds a scale-by-a-factor overlay. Overlays are applied relative to the baseline, not cumulatively, and the baseline is restored at the end. Only `mutable=True` Params can be hot-swapped, so the operational set is promoted to mutable: `pDemandElec`, `pENSCost`, `pLinearVarCost`, `pEFOR`, `pReserveMargin`, `pRESEnergy`. Structural and topology Params stay immutable because they are read in build-time `if` statements, where a mutable Param raises instead of returning its value (RFC Modes A / B). The six promoted Params are now read by call (`pFoo[idx]()`) in the build-time guards and in the cost and demand sums; reads inside constraints and the objective stay symbolic so the swap reaches the solver. No behaviour change: case 9n solves bit-identical (164.382043867 MEUR, `PYTHONHASHSEED=0`) and the solve suite passes with the same locked costs. New test `test_mode_c_resolve_demand_hot_swap` builds 9n once, then checks an identity overlay reproduces the cost, a +10 % demand overlay raises it, and restoring the baseline returns it.
+- [FIXED] `setup_solver` no longer crashes on Windows when an earlier in-process solve still holds the stale log file open.
+- [FIXED] deprecated `datetime.utcnow()` replaced with timezone-aware UTC; emitted timestamp unchanged.
+- [ADDED] Mode C post-build hot-swap re-solve (`openTEPES_ProblemSolvingResolve.py`): `resolve(OptModel, SolverName, overlays)` re-solves a built model once per parameter overlay without rebuilding, so a sweep reuses the single slow build; `overlay_scaled` makes a scale-factor overlay. Overlays apply relative to the baseline, which is restored at the end. Only `mutable=True` Params hot-swap, so the operational set (`pDemandElec`, `pENSCost`, `pLinearVarCost`, `pEFOR`, `pReserveMargin`, `pRESEnergy`) is promoted to mutable and read by call in the build-time guards; structural and topology Params stay immutable. No behaviour change: 9n solves bit-identical (164.382043867 MEUR, `PYTHONHASHSEED=0`). New test `test_mode_c_resolve_demand_hot_swap`.
 - [CHANGED] split the ~2900-line `openTEPES_InputData.py` into three modules at the package root, one per model-build step — the input-side counterpart of the earlier Formulation, Solver and Output splits, and the last of the three mega-file splits. `openTEPES_InputData.py` keeps `InputData`, which reads the raw sets and parameters from the case source (a CSV directory or a DuckDB file). The new `openTEPES_DataConfiguration.py` holds `DataConfiguration`, which builds the derived and instrumental sets and the flag-driven branches (hydro topology, hydrogen, heat, PTDF). The new `openTEPES_SettingUpVariables.py` holds `SettingUpVariables`, which creates the decision variables and their bounds, fixes the generators' commitment, relaxes or forbids investment conditions, zeroes out epsilon values, and screens for infeasibilities. `openTEPES.py` imports each function from its own module, and `__init__.py` re-exports all three, so a user who imported `DataConfiguration` or `SettingUpVariables` from `openTEPES.openTEPES_InputData` moves to the matching module (e.g. `...DataConfiguration`). This is a pure move — the three functions are byte-identical to before, checked function by function, and none of them calls another — so results do not change: the whole solve test suite passes with the same locked costs. The two new modules import only pandas and Pyomo (no sibling `openTEPES` modules), so they need no relative-import guard; `tests/test_direct_run.py` still runs them as scripts like every other module. This finishes the Layer-3 model-build split from the architecture RFC. Separating set construction from parameter construction inside `InputData` (groundwork for the planned overlay and mutable-parameter work) is function surgery rather than a pure move and is left for its own later change.
 - [CHANGED] document every bundled case on the Download & Installation page (`doc/md/Download.md`). The list now covers all ten cases under `openTEPES/cases/`, not just seven: the three 9-node variants that were missing are added — `9n_PTDF` (the 9-node case solved with the PTDF network formulation instead of the angle-based DC power flow), `9n_heat` (the 9-node case coupled with a heat network), and `9n_H2` (the 9-node case coupled with a hydrogen network) — and the `sSEP` entry now notes it also includes a hydrogen network. Each entry says in one line what makes the case different, so a new user can pick the right starting point.
 - [CHANGED] split the ~1770-line `openTEPES_ModelFormulation.py` into six per-concern modules at the package root — the formulation-side counterpart of the earlier Input, Solver and Output splits. The two cross-sector concerns are `openTEPES_ModelFormulationObjective.py` (the total-cost objective and the per-stage operation-cost accumulation) and `openTEPES_ModelFormulationInvestment.py` (the four investment builders plus the installed-capacity, adequacy-reserve-margin and emission / RES-energy limits). Each energy sector then gets its own module: `openTEPES_ModelFormulationElectricity.py` (demand balance, operating reserves and inertia, storage, unit commitment, ramping, line switching, DC network operation and the cycle constraints), `openTEPES_ModelFormulationHydro.py` (reservoir water balance), `openTEPES_ModelFormulationHydrogen.py` (H2 network) and `openTEPES_ModelFormulationHeat.py` (heat network). The old `openTEPES_ModelFormulation.py` is removed: `openTEPES.py` imports the objective and investment builders from their modules, `openTEPES_ProblemSolvingStageIter.py` imports the per-stage operation builders, and `__init__.py` re-exports all six (so `from openTEPES.openTEPES_ModelFormulation import ...` users move to the concern module — e.g. `...Investment`). This is a pure move — the 19 builder functions are byte-identical to before, checked function by function — so results do not change: the whole solve test suite passes with the same locked costs. Keeping each sector in one file, with its existing granular per-constraint functions inside, is the groundwork for selectable formulations later (for example a DC vs AC network build, or unit commitment on or off) chosen by which functions the stage driver calls. The new modules use the descriptive-docstring convention of the other split modules; the standalone test `tests/test_heat_investment_typo.py` now imports `InvestmentHeatModelFormulation` from `...Investment`.

openTEPES/openTEPES.py

@@ -138,7 +138,7 @@ def openTEPES_run(DirName, CaseName, SolverName, pIndOutputResults, pIndLogConso
     """
 
     InitialTime = time.time()
-    _RunStartedUtc = datetime.datetime.utcnow().isoformat(timespec="seconds") + "Z"
+    _RunStartedUtc = datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None).isoformat(timespec="seconds") + "Z"
 
     # If the caller pointed at a .duckdb file (CaseName carries the
     # suffix), open it as a DuckDBSource; InputData reads tables from
@@ -387,7 +387,7 @@ def openTEPES_run(DirName, CaseName, SolverName, pIndOutputResults, pIndLogConso
         "backend":            getattr(mTEPES, "pOutputBackend", "csv"),
         "opentepes_version":  "4.18.17RC",
         "run_started_utc":    _RunStartedUtc,
-        "run_finished_utc":   datetime.datetime.utcnow().isoformat(timespec="seconds") + "Z",
+        "run_finished_utc":   datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None).isoformat(timespec="seconds") + "Z",
         "outputs_enabled":    [k for k, v in _flags.items() if v],
         "gzip_patterns":      list(gzip_patterns) if gzip_patterns else None,
         "gzip_files":         _GzipFiles,

openTEPES/openTEPES_ProblemSolvingPersistent.py

@@ -12,6 +12,7 @@
 """
 from __future__ import annotations
 
+import gc
 import os
 
 import pyomo.environ as pyo
@@ -28,7 +29,14 @@ def setup_solver(OptModel, SolverName: str, FileName: str, ncall: int, mTEPES):
     if SolverName != "appsi_gurobi" and SolverName != "gurobi_persistent":
         Solver = SolverFactory(SolverName)
         if os.path.exists(FileName):
-            os.remove(FileName)
+            # A prior in-process solve may still hold this log open (the HiGHS interface keeps it open until
+            # GC frees the highspy object); on Windows an open file cannot be deleted. Collect first, then
+            # remove best-effort: a still-locked log is left in place, since the solver truncates it on re-open.
+            gc.collect()
+            try:
+                os.remove(FileName)
+            except OSError:
+                pass
         return Solver
 
     if SolverName == "appsi_gurobi":

openTEPES/openTEPES_ProblemSolvingResolve.py

@@ -3,9 +3,8 @@
 
 openTEPES.openTEPES_ProblemSolvingResolve — Mode C post-build hot-swap re-solve.
 
-Re-solve an already-built model once per parameter overlay without rebuilding it, so a
-sweep, sensitivity study or Monte-Carlo reuses the one slow build. Only ``mutable=True``
-Params can be hot-swapped: the operational set ``pDemandElec``, ``pENSCost``,
+Re-solve a built model once per parameter overlay without rebuilding, so a sweep reuses the
+one slow build. Only ``mutable=True`` Params hot-swap: ``pDemandElec``, ``pENSCost``,
 ``pLinearVarCost``, ``pEFOR``, ``pReserveMargin``, ``pRESEnergy``. See ``resolve`` for the
 overlay format.
 """
@@ -16,9 +15,9 @@
 
 
 def overlay_scaled(OptModel, param_name: str, factor: float) -> dict:
-    """Return an overlay that scales every current value of ``param_name`` by ``factor``.
+    """Return an overlay scaling every current value of ``param_name`` by ``factor``.
 
-    Example: ``overlay_scaled(model, "pDemandElec", 1.10)`` for a 10 % demand uplift.
+    E.g. ``overlay_scaled(model, "pDemandElec", 1.10)`` for a 10 % demand uplift.
     """
     param = getattr(OptModel, param_name)
     return {param_name: {idx: val * factor for idx, val in param.extract_values().items()}}
@@ -27,18 +26,16 @@ def overlay_scaled(OptModel, param_name: str, factor: float) -> dict:
 def resolve(OptModel, SolverName: str, overlays, *, restore: bool = True, tee: bool = False):
     """Re-solve ``OptModel`` once per overlay, swapping mutable Param values in place.
 
-    ``overlays`` is a list of dicts, each mapping a mutable Param name to new values (a
-    dict ``index -> value`` for an indexed Param, or a scalar). Each overlay is applied
-    relative to the baseline (values at call time), not cumulatively; ``{}`` re-solves the
-    baseline. With ``restore`` (default) the baseline is restored at the end. ``SolverName``
-    must be a non-persistent solver. Returns one dict per overlay with the termination
+    ``overlays`` is a list of dicts mapping a mutable Param name to new values (``index ->
+    value``, or a scalar). Each overlay applies relative to the baseline, not cumulatively;
+    ``{}`` re-solves the baseline. ``restore`` (default) restores the baseline at the end.
+    ``SolverName`` must be non-persistent. Returns one dict per overlay with the termination
     condition and total system cost (None if not optimal).
     """
     overlays = list(overlays)
 
-    # Snapshot the baseline values of every Param any overlay touches. The snapshot serves
-    # two purposes: resetting the touched Params before each overlay (so overlays are
-    # independent, not cumulative) and the optional restore at the end of the sweep.
+    # Snapshot the baseline of every touched Param: used to reset before each overlay (so
+    # overlays are independent, not cumulative) and for the optional restore at the end.
     touched = set()
     for ov in overlays:
         touched.update(ov.keys())