fix: price the discharge gate off the cell the battery is actually in

[johanzander]

Summary

• The sub-period discharge gate read its value estimate from a different point on the value function than the policy walks, so it could hold the battery for a single period while the periods either side discharged.
• Adds _value_slope_below, the left one-sided derivative of the interpolant _interpolate_value defines, and routes _record_marginal_value through it.
• No planned energy moves: 142 of 2168 golden gate booleans flip, with actions, intents, soe_trajectory and cost bit-identical.

Root cause

_record_marginal_value did its own index arithmetic (dp_battery_algorithm.py:908):

i = round((soe - battery_settings.min_soe_kwh) / SOE_STEP_KWH)
shadow_price = float((V[t, i] - V[t, i - 1]) / SOE_STEP_KWH)

round() snaps to the nearest grid point. _interpolate_value (:1436) and _local_value_slope (:1461) both floor. So for any SoE in the lower half of a cell the gate was priced off the cell below — a region of V the battery is not in — while the policy walked a different curve.

Visible directly on the reporter's data, sweeping initial_soe at production resolution. The reported slope changes at frac 0.4→0.6, i.e. half-way between rungs, not at the rung boundaries:

initial_soe    idx      frac   shadow
9.8750     323.0000     1.00   0.19499
9.8800     323.2000     0.20   0.19499   <- same cell as below, different answer
9.8850     323.4000     0.40   0.19499
9.8900     323.6000     0.60   0.20806
9.8950     323.8000     0.80   0.20806
9.9000     324.0000     1.00   0.20806

9.8800 and 9.8950 are both inside cell [323, 324] and must price the same kWh identically. That is the plateau-centred-on-the-rung signature of round().

For SOLAR_STORAGE the planned rate is 0, so max(planned, gate) (battery_system_manager.py:2688) makes the gate the entire hardware decision, which is why a misread surfaces as an isolated 0% period rather than a small cost wobble.

Not reproduced: the issue's specific numbers (shadow 0.25554, gate CLOSED at 9.9000) do not reproduce in my replay — I get 0.20806 and an OPEN gate, presumably a horizon/terminal-value difference from the reporter's production run. The defect reproduces exactly; that one closed period does not, and periods 60/61 were never in the bundle. The mechanism and the corpus-wide effect below stand on their own evidence.

Fix

_value_slope_below(V_row, soe, battery_settings) -> float | None returns the left one-sided derivative of the same piecewise-linear interpolant the policy walks. Left-sided because the gate authorizes energy leaving the battery, so it must price the value given up going down.

Deliberately not _local_value_slope: that is the right-sided reading, correct for the tie detector's noise magnitude and wrong here. Measured on the corpus it flips 395 decisions against this fix's 142, nearly all opening — a systematic over-open, not a fix.

tie_detection / the PWL re-solve are deliberately not wired in. epsilon_for_period is a currency band on two candidate values, not a EUR/kWh level — converting means dividing by SOE_STEP_KWH, which is the hand-derived epsilon P5 forbids — and the PWL window's V is solved with its exit SoE pinned (:2111), so its slope prices an artificial terminal constraint.

One thing found while implementing, worth a reviewer's eye: a one-sided derivative is discontinuous at each grid point, so it needs a tolerance the continuous interpolant does not. A battery resting exactly on its reserve floor yields idx = +1e-16 rather than 0.0 in several fixtures; without _GRID_POINT_TOLERANCE that ulp prices a kWh below the floor that does not exist and reopens the ceiling #526 closed. The existing test_discharge_gate_authorization_526 caught this on the first implementation (6 floor periods wrongly authorized).

soe_levels became unused in _record_marginal_value and was dropped from it and its two call sites.

Test plan

• ./scripts/quality-check.sh passes (1853 passed, 31 skipped; Black/Ruff/frontend/TS/ESLint clean)
• .venv/bin/pytest -m slow passes (537 passed, 6 skipped, 3m47s)
• Observed end-to-end through the real hardware-write path, not just tests: took a real optimizer-produced flip (realworld_2026_04_24_090423, period 47) and drove the decision through the real BatterySystemManager._apply_period_schedule:

Gate    : False  ->  True   (pre-fix -> post-fix)
  pre-fix  gate=False -> discharge_rate = 0
  post-fix gate=True  -> discharge_rate = 100

That is the reporter's symptom — the register that was writing 0% now writes 100%.

• Full E2E stack (docker-compose.ci.yml, real backend + mock-HA) brought up and healthy; prices fetched, health checks green. The scheduler could not be driven to a fresh optimization there because the scenario pins a past mock_time and an accelerated container clock does not produce optimization runs in this project, so the observation above was taken through the real BSM path instead.

Evidence the test discriminates

• Reverted: lo = int(np.ceil(...)) - 1 → lo = min(round(idx), len(V_row) - 1) - 1, i.e. the pre-bug: discharge gate reads a raw snapped V grid difference while the policy uses _interpolate_value — shadow price 42% above the optimizer's own objective #571 nearest-grid-point snap.
• Result: test_shadow_price_is_constant_within_one_grid_cell FAILED (obtained 0.19499, expected 0.20806), and 20 of 38 test_action_selector_parity golden tests FAILED.
• Restored: tree clean, 55 passed.

Stated plainly: test_gate_slope_stays_on_the_policys_interpolant does not redden under that mutation — at the offsets it uses, round() and ceil-1 pick the same cell. It is a standing guard against the two estimators drifting apart again, not a reproduction of this bug. The real-data cell test and the goldens are what catch it.

Outcome-level coverage

• Action-selector goldens — intra_period_discharge_allowed re-pinned across 21 fixtures. Measured delta: 142 of 2168 periods flip, 141 opening a wrongly-closed gate, 1 closing. actions, intents, soe_trajectory and cost are bit-identical on all 35 existing fixtures; I verified the regeneration itself touched no other field.
• New fixture regression_2026_08_13_145213 built from the reporter's debug bundle via from_debug_log.py, plus its golden and a VPP baseline entry (--add-new only, no full re-baseline).
• No run_scenario_realized / R == P test, deliberately: the gate moves no planned energy, and inverter_simulator.py:194-196 computes deficit = max(0, home - solar) for the load_first modes SOLAR_STORAGE/SOLAR_EXPORT map to, which is structurally zero at point-forecast resolution for exactly the intents that select them (the file argues this itself at :118-126). An R == P test would pass identically with the gate open or closed and would be coverage theatre. Reviewed and agreed independently.

Scope assessment

Local — stays inside _record_marginal_value's existing contract (same inputs, same two fields written); the new helper is a peer of the two existing value estimators in their established owner file. No parameter, flag, default-fallback, second construction site, or extra trigger routing around an ordering/timing problem: the diff removes a mirrored value estimator, which is what P1 exists to prevent.

Documentation

docs/agents/bess-knowledge.md updated — it described shadow_price as a "backward difference between adjacent SoE grid levels", the exact mechanism this replaces. Added how the slope is now read and the bundle signature of the old bug. docs/SOFTWARE_DESIGN.md mentions neither shadow_price nor the gate, so nothing to change there.

Refs #571 — does not close it. The reported period (58, soe = 9.9000)
is unchanged by this PR: idx = 324.0 sits exactly on a grid point, so
round() and ceil()-1 select the same cell and the half-cell correction is
identically zero there. Verified against this branch's head; see the
reproduction in the comments. #571's own mechanism is the concavity violation
at that cell, which is a separate fix.

Blocked on Phase 4a: this adds a third dV/dSoE estimator, which the parent
plan records as a deferred behaviour change, and 4a (approved D1) relocates
intra_period_discharge_gate into core/bess/execution_model.py.

[bess-agent]

The reporter's numbers do reproduce — and this PR does not change them

Posting this so the finding isn't re-derived. It resolves the PR's first stated
divergence ("Frank's specific numbers don't reproduce ... likely a horizon or
terminal-value difference"), and it changes the disposition.

They reproduce at period 58, not 59

p58 soe=9.9000 SOLAR_STORAGE shadow=0.25553757 buy*eff=0.24515589 allowed=False

against the bundle's recorded "shadow_price": 0.2555375657938441,
"intra_period_discharge_allowed": false. Eight digits, bit-for-bit.

Inputs: horizon 134, initial_soe 9.9, initial_cost_basis
0.0363146679449133 — i.e. the 14:30 run, built by prepending period 58 onto
the from_debug_log.py fixture using the bundle's own 14:30:10 prediction
snapshot (## Prediction Snapshots, "optimization_period": 58).

This PR's sweep is on the period-59 fixture (horizon 133), which is a
different optimizer run against a different V. That is why it yields
0.19499/0.20806 and matches nothing recorded in the bundle. Not a horizon or
terminal-value mystery — just a different run.

Run against this PR's head, the reported period is unchanged

Same replay, checked out at 8ab435f9:

p58 soe=9.9000 SOLAR_STORAGE shadow=0.25553757 buy*eff=0.24515589 allowed=False

Bit-identical to pre-fix main. The gate the reporter filed is still CLOSED.

It can't be otherwise: at soe = 9.9, idx = (9.9 - 1.8)/0.025 = 324.0 sits
exactly on a grid point, so round(idx) and ceil(idx) - 1 select the same
cell [323, 324]. The half-cell correction is identically zero there.

And that generalises for this reporter. SoC arrives as whole percent, so SoE is
always an exact multiple of 0.15 kWh, and

(k * 0.15 - 1.8) / 0.025 = 6k - 72

is always an integer. Every optimization's entry SoE lands exactly on a grid
node. The offset this PR fixes can only move periods later in the horizon, at
interpolated trajectory SoEs — never period 0, which is the one that drives the
hardware write.

What is actually wrong at that state

The concavity violation #571 documents. On the production grid:

[9.8500,9.8750] -> +0.23602
[9.8750,9.9000] -> +0.25554   <- read by the gate
[9.9000,9.9250] -> +0.17978

One interior cell above both neighbours, which a concave V cannot be.
Re-solving at SOE_STEP_KWH = 0.005 gives 0.17978 at the same state,
matching the issue's independent price/physics derivation
(0.1404/0.97 + 0.035) to five digits, and opening the gate.

Scored against that 0.005 reference across all 37 fixtures, split by whether the
coarse V is locally concavity-violating at the state:

	n	current (backward)	central	concave envelope
violating	217	49.3% wrong gate, err 3.11	46.5%, 1.90	27.7%, 1.32
clean	1642	25.8%, 0.81	23.5%, 0.77	23.1%, 0.85

Reading the slope off the upper concave envelope gives 0.22889 < 0.24516 at
the reporter's state ⇒ OPEN, the correct decision.

Also worth noting: the issue's suggested metric (gate-flip rate between adjacent
same-price periods) does not discriminate — 9/702, 10/702, 9/702 for the
three estimators. It shouldn't be used as the acceptance criterion.

Two smaller points

• round() vs floor() is a real defect — this is not a "the PR is wrong"
  comment. It just isn't the defect bug: discharge gate reads a raw snapped V grid difference while the policy uses _interpolate_value — shadow price 42% above the optimizer's own objective #571 reports, and the two are independent.
• "the fix moves no energy" is true of the plan, not of execution. The
  goldens never run the inverter simulator, and the gate feeds
  max(planned, gate). This PR's own E2E evidence (register 0 → 100) is the
  proof that execution moves. So the field blast radius is ~141 periods that
  will now discharge intra-period where they previously held, under forecast
  error — not captured by "cost bit-identical".

Suggested disposition

• Drop Closes #571 — the reported period is unchanged, so this PR does not
  close it.
• Hold behind Phase 4a. The parent plan already records this class of change
  as deferred: "The grid and PWL paths compute dV/dSoE differently
  (_local_value_slope clamps a grid index; _pwl_local_value_slope takes a
  clamped central difference); unifying them would be a behavior change"
  (2026-08-09-optimizer-target-architecture.md). This PR adds
  _value_slope_below as a third peer estimator and re-pins 142 gate booleans
  to prove it — that is the deferred behaviour change. 4a (approved D1) also
  relocates intra_period_discharge_gate into core/bess/execution_model.py,
  so this code is about to move regardless.
• After 4a, measure the half-cell offset and the concavity repair together on the
  relocated code. Both are real; only the second one closes bug: discharge gate reads a raw snapped V grid difference while the policy uses _interpolate_value — shadow price 42% above the optimizer's own objective #571.

The reproduction is a ~20-line script on top of the existing
regression_2026_08_13_145213 fixture (prepend period 58, set initial_soe and
initial_cost_basis) — happy to push it as a test if that's wanted.