Skip to content

Controls: restore legacy [CONTROLS] parity for SIMULATION TIME (and 10 others) Addressing #74#76

Merged
cbuahin merged 5 commits into
developfrom
swmm6_rel
May 28, 2026
Merged

Controls: restore legacy [CONTROLS] parity for SIMULATION TIME (and 10 others) Addressing #74#76
cbuahin merged 5 commits into
developfrom
swmm6_rel

Conversation

@cbuahin
Copy link
Copy Markdown
Member

@cbuahin cbuahin commented May 28, 2026

Fixes the user-reported regression where the pump-outage rule

RULE PUMP1_outage
 IF SIMULATION TIME > 4.5
 AND SIMULATION TIME < 12
 THEN PUMP PUMP1 status = OFF
 ELSE PUMP PUMP1 status = ON
 PRIORITY 4

held PUMP1 ON for the entire run instead of going OFF between hours
4.5 and 12. The same rule passes under pyswmm / legacy controls.c.

Root cause of the reported bug

Two interacting unit mismatches:

  1. parseRuleText pushed every premise RHS literal through
    tryParseDouble. Legacy controls.c::getPremiseValue routes
    SIM_TIME / CLOCK_TIME / LINK_TIMEOPEN / LINK_TIMECLOSED through
    datetime_strToTime, which reads "4.5" as 4.5 decimal hours and
    converts to days (0.1875). The new engine stored it as the raw
    double 4.5.

  2. SIM_TIME LHS returned ctx.current_time as-is. The docstring said
    "decimal days" but TimestepController.cpp:88 increments the
    field by dt_taken seconds, so the value is actually in seconds.
    The comparison reduced to {seconds} > 4.5 (true after t=5s) AND
    {seconds} < 12 (false after t=12s), so the IF block could never
    satisfy both clauses and the ELSE branch (pump ON) always won.

Fix:

  • New parsePremiseRHS dispatches on the LHS attribute, mirroring
    legacy getPremiseValue: parseTimeToken (decimal hours OR
    HH:MM:SS -> days) for time variables, parseDateToken for
    SIM_DATE, parseDayOfYearToken (M/D or 1..365) for SIM_DAYOFYEAR,
    range checks for SIM_DAY / SIM_MONTH, parseStatusValue for
    LINK_STATUS, raw double otherwise.
  • SIM_TIME LHS now divides current_time by SEC_PER_DAY so both
    sides of the comparison are in decimal days.
  • SimulationContext::current_time docstring corrected to say
    seconds; future readers will not repeat the mistake.

Other parity gaps closed

All documented in docs/CONTROL_RULES_LEGACY_PARITY_AUDIT.md.

P0-C02 CLOCK_TIME LHS now returns the fractional-day portion of
ctx.current_date (legacy CurrentTime), not the hour-of-day.

P0-C03 dt is converted from seconds to days inside
ControlEngine::evaluate so compareTimes' half-step
tolerance is in the same unit as the LHS and EQ/NE on
time-valued attributes works.

P0-C04, P0-C05 AND/OR short-circuit now follows the rule's
declared premise order. The previous SoA batching
scattered premises by variable type, so rules with mixed
variable types or expression premises diverged from
legacy. evaluate() is now a three-phase pipeline:
- Phase 1a: SoA gather + batch compare per variable
group (SIMD-amenable, unchanged).
- Phase 1b: scalar fallback for variable-RHS and
expression premises.
- Phase 2: per-rule reducer walks each rule's premise
list in declaration order and applies legacy
AND/OR/short-circuit against a flat
premise_results_ cache.
Strategy 3A from docs/CONTROL_RULES_OPTIMIZATION_STRATEGY.md
is now the default; per-operator sub-loops and auto-vector
dispatch all still apply.

P0-C06 SIM_DAY applies the DateDelta epoch shift used by
legacy datetime_dayOfWeek; previously off by six days.

P1-C07 VARIABLE = and
EXPRESSION = are now recognised by
parseRuleText. A named-variable or named-expression
token in any premise LHS / RHS resolves through
addNamedVariable / addExpression. Previously silently
dropped.

P1-C08 CURVE / TIMESERIES / PID actions are excluded from
ctx.control_log when rpt_controls is on, matching legacy
controls.c:1770.

P1-C09 Link time_last_set storage moved to
ctx.links.time_last_set and is written by
SWMMEngine::stepRouting only when
target_setting * current == 0 (open<->closed transition),
matching legacy routing.c:295-299. ControlEngine reads
from this vector for LINK_TIMEOPEN / LINK_TIMECLOSED.

P1-C10 options.rule_step is honoured: evaluate() short-circuits
until ctx.current_date reaches next_rule_eval_time_, and
the counter advances by rule_step / SEC_PER_DAY each
fire. Matches legacy routing.c:286-290.

P1-C11 parseRuleText return code is checked at the call site in
SWMMEngine.cpp; on failure ctx.error_code is set to 217
(ERR_RULE) and a message goes into ctx.errors. Previously
every parse failure was silently swallowed.

P2-C13 The misleading "IF SIMULATION TIME > 3600" example in
ControlsHandler.cpp is replaced with two working forms
(decimal hours and HH:MM:SS).

Files changed

src/engine/controls/Controls.{hpp,cpp}
Parsers, three-phase evaluator, LHS unit fixes,
named-variable / expression support, RULE_STEP gating,
time_last_set reads from ctx.links.
src/engine/core/SWMMEngine.cpp
Open<->closed transition writes ctx.links.time_last_set;
parser errors propagate via ctx.error_code / errors.
src/engine/core/SimulationContext.hpp
current_time docstring corrected (it is in seconds).
src/engine/data/LinkData.hpp
Adds time_last_set vector and initialises in resize().
src/engine/input/handlers/ControlsHandler.cpp
Sample-comment example rewritten to a working SIMULATION
TIME form.
tests/unit/engine/test_controls_legacy_parity.cpp
11 new gtest cases, including the user's exact
RULE PUMP1_outage asserted at seven sample times.
tests/unit/engine/CMakeLists.txt
Registers the new test executable.
docs/CONTROL_RULES_LEGACY_PARITY_AUDIT.md
Audit and remediation plan, with section 13 recording the
actual landing.

Verification

Every touched translation unit passes g++ -std=c++20 -fsyntax-only
in the implementation sandbox. The new fixture
test_engine_controls_legacy_parity.PumpOutageWindow_UserReportedCase
exercises the exact rule above at t = {0, 4, 4.6, 8, 11.9, 12.5, 24}
hours; without this change the pump stays at target_setting 1.0 at
every sample inside (4.5 h, 12 h). The full build + ctest run must
happen on the macOS host:

cmake --build build-arm64-osx --target
test_engine_controls_legacy_parity
test_engine_controls
test_engine_control_rule_validate
ctest --test-dir build-arm64-osx -R controls --output-on-failure

controls: restore legacy [CONTROLS] parity for SIMULATION TIME and 10 other gaps

Fixes the reported pump-outage regression where
"IF SIMULATION TIME > 4.5 AND SIMULATION TIME < 12 THEN PUMP P1 OFF"
held the pump ON for the whole run instead of cycling between
hours 4.5 and 12. Root cause was a units mismatch on both sides of
the comparison: parseRuleText stored the RHS "4.5" as a raw double
(legacy datetime_strToTime would read it as 4.5 hours -> 0.1875
days), and the SIM_TIME LHS returned ctx.current_time unchanged
(stored in seconds despite a docstring that claimed days). RHS
parsing is now dispatched on the LHS attribute via parsePremiseRHS
(decimal hours / HH:MM:SS / M-D-Y / M/D / status words, all
matching legacy controls.c:getPremiseValue), and SIM_TIME divides
by SEC_PER_DAY so both sides are in decimal days. The same landing
also closes ten further parity gaps: CLOCK_TIME LHS units (P0-C02),
half_step in days (P0-C03), AND/OR declaration-order short-circuit
via a flat per-(rule,premise) result cache and a new three-phase
evaluator (P0-C04, P0-C05), SIM_DAY DateDelta shift (P0-C06),
VARIABLE / EXPRESSION keyword support (P1-C07), modulated actions
excluded from the control log (P1-C08), time_last_set moved to
ctx.links and updated only on open<->closed transitions (P1-C09),
RULE_STEP gating (P1-C10), and parser errors surfaced via
ctx.error_code instead of being swallowed (P1-C11). New gtest
fixture tests/unit/engine/test_controls_legacy_parity.cpp covers
the user's exact rule and each gap; the SoA batch compare and
per-operator sub-grouping still apply.
@cbuahin cbuahin requested a review from Copilot May 28, 2026 05:36
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

cbuahin added 4 commits May 28, 2026 05:51
The lids.usage_add signature was split across two lines inside an inline
literal, which broke the definition-list term and caused docutils to
emit 'Unexpected indentation' at line 117. With -W in CI, this failed
the Sphinx build.

Collapse the term onto a single line to match the style used elsewhere
in the file (e.g. inlets.set_params).
Controls.hpp uses std::vector<uint8_t> for the premise_results_ cache
but did not include <cstdint>. Older libstdc++ versions pulled it in
transitively via <vector>; GCC 13/14 on Ubuntu 24.04 no longer does,
so the Linux CI build failed with:

  error: 'uint8_t' was not declared in this scope

and a cascade of 'premise_results_ was not declared' errors from
the broken declaration. Add the include explicitly.
Two independent Cython errors surfaced on macOS and Windows wheel
builds (Linux wheels died earlier on the Controls.hpp issue fixed
in d8db311):

1. _links.pyx:632 called swmm_link_hyd_power, missing the get_
   prefix. The C function is swmm_link_get_hyd_power (declared in
   _common.pxd:337); the bulk variant on _links.pyx:981 already
   used the correct name.

2. _spatial.pyx used f"{arr.shape}" on buffer-typed
   cdef np.ndarray[double, ndim=2] variables. For buffer-typed
   ndarrays Cython exposes .shape as npy_intp*, not a Python tuple,
   so the f-string conversion failed with
   'Cannot convert npy_intp * to Python object'. Replaced with
   f"({arr.shape[0]}, {arr.shape[1]})" at the three sites
   (lines 115, 156, 197).
Signed-off-by: cbuahin <caleb.buahin@gmail.com>
@cbuahin cbuahin merged commit df7d369 into develop May 28, 2026
20 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants