Controls: restore legacy [CONTROLS] parity for SIMULATION TIME (and 10 others) Addressing #74#76
Conversation
controls: restore legacy [CONTROLS] parity for SIMULATION TIME and 10 other gaps Fixes the reported pump-outage regression where "IF SIMULATION TIME > 4.5 AND SIMULATION TIME < 12 THEN PUMP P1 OFF" held the pump ON for the whole run instead of cycling between hours 4.5 and 12. Root cause was a units mismatch on both sides of the comparison: parseRuleText stored the RHS "4.5" as a raw double (legacy datetime_strToTime would read it as 4.5 hours -> 0.1875 days), and the SIM_TIME LHS returned ctx.current_time unchanged (stored in seconds despite a docstring that claimed days). RHS parsing is now dispatched on the LHS attribute via parsePremiseRHS (decimal hours / HH:MM:SS / M-D-Y / M/D / status words, all matching legacy controls.c:getPremiseValue), and SIM_TIME divides by SEC_PER_DAY so both sides are in decimal days. The same landing also closes ten further parity gaps: CLOCK_TIME LHS units (P0-C02), half_step in days (P0-C03), AND/OR declaration-order short-circuit via a flat per-(rule,premise) result cache and a new three-phase evaluator (P0-C04, P0-C05), SIM_DAY DateDelta shift (P0-C06), VARIABLE / EXPRESSION keyword support (P1-C07), modulated actions excluded from the control log (P1-C08), time_last_set moved to ctx.links and updated only on open<->closed transitions (P1-C09), RULE_STEP gating (P1-C10), and parser errors surfaced via ctx.error_code instead of being swallowed (P1-C11). New gtest fixture tests/unit/engine/test_controls_legacy_parity.cpp covers the user's exact rule and each gap; the SoA batch compare and per-operator sub-grouping still apply.
The lids.usage_add signature was split across two lines inside an inline literal, which broke the definition-list term and caused docutils to emit 'Unexpected indentation' at line 117. With -W in CI, this failed the Sphinx build. Collapse the term onto a single line to match the style used elsewhere in the file (e.g. inlets.set_params).
Controls.hpp uses std::vector<uint8_t> for the premise_results_ cache but did not include <cstdint>. Older libstdc++ versions pulled it in transitively via <vector>; GCC 13/14 on Ubuntu 24.04 no longer does, so the Linux CI build failed with: error: 'uint8_t' was not declared in this scope and a cascade of 'premise_results_ was not declared' errors from the broken declaration. Add the include explicitly.
Two independent Cython errors surfaced on macOS and Windows wheel builds (Linux wheels died earlier on the Controls.hpp issue fixed in d8db311): 1. _links.pyx:632 called swmm_link_hyd_power, missing the get_ prefix. The C function is swmm_link_get_hyd_power (declared in _common.pxd:337); the bulk variant on _links.pyx:981 already used the correct name. 2. _spatial.pyx used f"{arr.shape}" on buffer-typed cdef np.ndarray[double, ndim=2] variables. For buffer-typed ndarrays Cython exposes .shape as npy_intp*, not a Python tuple, so the f-string conversion failed with 'Cannot convert npy_intp * to Python object'. Replaced with f"({arr.shape[0]}, {arr.shape[1]})" at the three sites (lines 115, 156, 197).
Signed-off-by: cbuahin <caleb.buahin@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the user-reported regression where the pump-outage rule
held PUMP1 ON for the entire run instead of going OFF between hours
4.5 and 12. The same rule passes under pyswmm / legacy controls.c.
Root cause of the reported bug
Two interacting unit mismatches:
parseRuleText pushed every premise RHS literal through
tryParseDouble. Legacy controls.c::getPremiseValue routes
SIM_TIME / CLOCK_TIME / LINK_TIMEOPEN / LINK_TIMECLOSED through
datetime_strToTime, which reads "4.5" as 4.5 decimal hours and
converts to days (0.1875). The new engine stored it as the raw
double 4.5.
SIM_TIME LHS returned ctx.current_time as-is. The docstring said
"decimal days" but TimestepController.cpp:88 increments the
field by dt_taken seconds, so the value is actually in seconds.
The comparison reduced to {seconds} > 4.5 (true after t=5s) AND
{seconds} < 12 (false after t=12s), so the IF block could never
satisfy both clauses and the ELSE branch (pump ON) always won.
Fix:
legacy getPremiseValue: parseTimeToken (decimal hours OR
HH:MM:SS -> days) for time variables, parseDateToken for
SIM_DATE, parseDayOfYearToken (M/D or 1..365) for SIM_DAYOFYEAR,
range checks for SIM_DAY / SIM_MONTH, parseStatusValue for
LINK_STATUS, raw double otherwise.
sides of the comparison are in decimal days.
seconds; future readers will not repeat the mistake.
Other parity gaps closed
All documented in docs/CONTROL_RULES_LEGACY_PARITY_AUDIT.md.
P0-C02 CLOCK_TIME LHS now returns the fractional-day portion of
ctx.current_date (legacy CurrentTime), not the hour-of-day.
P0-C03 dt is converted from seconds to days inside
ControlEngine::evaluate so compareTimes' half-step
tolerance is in the same unit as the LHS and EQ/NE on
time-valued attributes works.
P0-C04, P0-C05 AND/OR short-circuit now follows the rule's
declared premise order. The previous SoA batching
scattered premises by variable type, so rules with mixed
variable types or expression premises diverged from
legacy. evaluate() is now a three-phase pipeline:
- Phase 1a: SoA gather + batch compare per variable
group (SIMD-amenable, unchanged).
- Phase 1b: scalar fallback for variable-RHS and
expression premises.
- Phase 2: per-rule reducer walks each rule's premise
list in declaration order and applies legacy
AND/OR/short-circuit against a flat
premise_results_ cache.
Strategy 3A from docs/CONTROL_RULES_OPTIMIZATION_STRATEGY.md
is now the default; per-operator sub-loops and auto-vector
dispatch all still apply.
P0-C06 SIM_DAY applies the DateDelta epoch shift used by
legacy datetime_dayOfWeek; previously off by six days.
P1-C07 VARIABLE = and
EXPRESSION = are now recognised by
parseRuleText. A named-variable or named-expression
token in any premise LHS / RHS resolves through
addNamedVariable / addExpression. Previously silently
dropped.
P1-C08 CURVE / TIMESERIES / PID actions are excluded from
ctx.control_log when rpt_controls is on, matching legacy
controls.c:1770.
P1-C09 Link time_last_set storage moved to
ctx.links.time_last_set and is written by
SWMMEngine::stepRouting only when
target_setting * current == 0 (open<->closed transition),
matching legacy routing.c:295-299. ControlEngine reads
from this vector for LINK_TIMEOPEN / LINK_TIMECLOSED.
P1-C10 options.rule_step is honoured: evaluate() short-circuits
until ctx.current_date reaches next_rule_eval_time_, and
the counter advances by rule_step / SEC_PER_DAY each
fire. Matches legacy routing.c:286-290.
P1-C11 parseRuleText return code is checked at the call site in
SWMMEngine.cpp; on failure ctx.error_code is set to 217
(ERR_RULE) and a message goes into ctx.errors. Previously
every parse failure was silently swallowed.
P2-C13 The misleading "IF SIMULATION TIME > 3600" example in
ControlsHandler.cpp is replaced with two working forms
(decimal hours and HH:MM:SS).
Files changed
src/engine/controls/Controls.{hpp,cpp}
Parsers, three-phase evaluator, LHS unit fixes,
named-variable / expression support, RULE_STEP gating,
time_last_set reads from ctx.links.
src/engine/core/SWMMEngine.cpp
Open<->closed transition writes ctx.links.time_last_set;
parser errors propagate via ctx.error_code / errors.
src/engine/core/SimulationContext.hpp
current_time docstring corrected (it is in seconds).
src/engine/data/LinkData.hpp
Adds time_last_set vector and initialises in resize().
src/engine/input/handlers/ControlsHandler.cpp
Sample-comment example rewritten to a working SIMULATION
TIME form.
tests/unit/engine/test_controls_legacy_parity.cpp
11 new gtest cases, including the user's exact
RULE PUMP1_outage asserted at seven sample times.
tests/unit/engine/CMakeLists.txt
Registers the new test executable.
docs/CONTROL_RULES_LEGACY_PARITY_AUDIT.md
Audit and remediation plan, with section 13 recording the
actual landing.
Verification
Every touched translation unit passes g++ -std=c++20 -fsyntax-only
in the implementation sandbox. The new fixture
test_engine_controls_legacy_parity.PumpOutageWindow_UserReportedCase
exercises the exact rule above at t = {0, 4, 4.6, 8, 11.9, 12.5, 24}
hours; without this change the pump stays at target_setting 1.0 at
every sample inside (4.5 h, 12 h). The full build + ctest run must
happen on the macOS host:
cmake --build build-arm64-osx --target
test_engine_controls_legacy_parity
test_engine_controls
test_engine_control_rule_validate
ctest --test-dir build-arm64-osx -R controls --output-on-failure