From 49b5915ee9523c5d1f0005997344c7b8f401aad7 Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 14:16:00 +0530 Subject: [PATCH 01/10] feat(sl-hunting): v4g knowledge from the 13 Aug live session Source: Intraday Hunter live session, 13 Aug 2026 (Vig9Kjab2T0, 10:36). A REDUCED win -- the trade reached most of its target, reversed into a loss, recovered, and was booked for about half. That arc produces the two structural ideas and the most complete discipline block in the series. Knowledge changes (all prose): - OPENING_DRIVE: A COMPLETED STOP-HUNT ENDS THAT DIRECTION. The sharpest idea here and it inverts the naive read of a bounce: a move that has just cleared one side's stops has SPENT its fuel and turns rather than continues. "Yesterday it took support EXACTLY at the 500 level and gave a retracement, so whoever was selling got chased out -- the chances of going DIRECTLY UP are LOW... if it has chased the sellers out, we follow THAT SAME DIRECTION", meaning the direction before the clearing bounce. The large-gap escape hatch is kept in the prose and asserted by a test, because unconditional it would harden into "always fade the bounce". - OPENING_DRIVE: THE ROUND NUMBER IS WHERE THE THESIS DIES, NOT JUST WHERE IT PAYS. v4d used round numbers for targets and v4c for recruitment; this adds the declared invalidation, named per index before entry. "Until the market crosses 24,500 we will not have much problem"; "when is there no danger to these buyers? If the market goes above 58,000." - RISK: FEAR IS NOT A SIGNAL -- NEVER CONVERT IT INTO AN EXIT; TIME SPENT IN THE TRADE SHRINKS THE ACHIEVABLE TARGET; NEVER EXIT AT ZERO AFTER A GOOD PROFIT HAS PRINTED. The fear rule sits directly on top of v4f's BOOK WHEN THE PROFIT STOPS GROWING and the pair is the most dangerous in the prompt: read carelessly v4g disables v4f, while v4f licenses exactly the fear-driven exit v4g forbids. The encoded distinction is MEASUREMENT versus EMOTION, and test_v4g_fear_rule_and_v4f_book_rule_do_not_cancel_each_other asserts both halves say so and that the fear rule still enumerates legitimate exit reasons rather than banning exits. Prompt 105,933 -> 111,283 chars. HEADROOM IS NOW 8,717 -- at ~5,000 per version that is roughly two more before the 120,000 cap, so the next pass should prune superseded prose rather than only append. The doc addendum also records the agent's day, which is the exact inverse of 12 Aug: SL Hunting was the BEST strategy on the board (+2,727.00 over 2 trades, cross-checked against its own Result summary) where yesterday it was the worst. Both trades were SHORT off the flat open -- the same side IH took -- and both exits keyed off the round number (profit_booking_round_number, profit_book_stall_near_round_number) rather than the premise-STALL churn that cost it yesterday. The read was short on both days; what changed is that the exits had a checkable reason, which is what v4g's fear rule demands. Two trades is a small sample and the basket was still deeply negative (-9,229.75 provisional, runner live at 14:13). Gates: SL Hunting pytest 179 passed, master 516 OK, ruff clean. Co-Authored-By: Claude Opus 5 --- .../SL Hunting AI Agent/sl_hunting_doc.md | 120 ++++++++++++++++++ .../sl_hunting_knowledge.py | 72 +++++++++++ .../test_sl_hunting_schema.py | 62 +++++++++ 3 files changed, 254 insertions(+) diff --git a/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md b/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md index 5fdb21d..efff0a4 100644 --- a/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md +++ b/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md @@ -3262,3 +3262,123 @@ Test updated: `test_shipped_note_matches_august_13_intraday_hunter_plan` replaces the 12 Aug equivalent and asserts both branch directions plus the round-number stop-location reasoning, because an inverted plan and a smoothed- away justification are the two failures a copy-forward would produce. + +--- + +## Video addendum - the 13 Aug LIVE SESSION (v4g) + +**Source:** Intraday Hunter live session, 13 Aug 2026 (`Vig9Kjab2T0`, 10:36, +published 11:36 IST). A win, but a REDUCED one - the trade reached most of its +target, reversed into a loss, recovered, and was booked for roughly half. That +arc is what makes the session valuable: it produces the two structural ideas +below and the most complete discipline block in the series. + +### A completed stop-hunt ends that direction + +The sharpest new idea, and it inverts the naive reading of a bounce: + +> "Yesterday the market gave good selling, then took support EXACTLY at the 500 +> level and gave a retracement. Because of that retracement, whoever was selling +> got chased out... so the chances of going DIRECTLY UP are LOW." + +The up-move existed *to clear the shorts*. With the shorts gone there is nobody +left to squeeze, so the bounce has spent its fuel. His operating rule: + +> "If it has chased the sellers out, we try to follow THAT SAME DIRECTION." + +- meaning the direction that preceded the clearing bounce, not the bounce. The +escape hatch is explicit and is kept in the prose: a fresh large gap recruits a +new crowd and restarts the question. + +### The round number is where the thesis DIES + +v4d used round numbers to place targets; v4c used them to explain recruitment. +This adds the third and most operational use - the declared invalidation: + +> "Until the market crosses the round number - as we see in NIFTY, the 24,500 +> level - we will not have much problem." +> "When is there no danger to these buyers? If the market goes above 58,000." + +Each index carries its own named level, decided before entry, and a decisive +cross means the read has failed even if the arithmetic stop is untouched. + +### The discipline block, and why it needed a test + +> "Before the target is hit there is a fear - should I book here, what if the +> market turns? **Fear is not a big deal, I feel it too.** But do NOT convert +> that fear into ACTION." +> "If you cut early it gradually becomes a HABIT. Then you cut small profits and +> leave, and when there is a loss you wait a long time to save the position and +> take a BIG loss. That is why most traders never become profitable." + +This sits directly on top of v4f's BOOK WHEN THE PROFIT STOPS GROWING, and the +pair is the most dangerous in the prompt: read carelessly, v4g reads as "hold +through everything" and disables v4f, while v4f licenses exactly the fear-driven +exit v4g forbids. The distinction encoded in both directions is **measurement +versus emotion** - the profit RATE falling is a measurement; "it might turn" is +not. `test_v4g_fear_rule_and_v4f_book_rule_do_not_cancel_each_other` asserts +both halves say so, and that the fear rule still enumerates the legitimate exit +reasons rather than banning exits. + +Two more from the same arc: **time spent shrinks the achievable target** ("the +market turned and wasted our TIME... only half the profit is showing, where +earlier it showed double"), and **never exit at zero after a good profit has +printed** - once real open profit has appeared, the floor stops being breakeven. + +### Knowledge changes (v4g, all prose) + +- `OPENING_DRIVE`: A COMPLETED STOP-HUNT ENDS THAT DIRECTION; THE ROUND NUMBER + IS WHERE THE THESIS DIES, NOT JUST WHERE IT PAYS. +- `RISK`: FEAR IS NOT A SIGNAL - NEVER CONVERT IT INTO AN EXIT; TIME SPENT IN + THE TRADE SHRINKS THE ACHIEVABLE TARGET; NEVER EXIT AT ZERO AFTER A GOOD + PROFIT HAS PRINTED. +- Test markers: `test_system_prompt_has_v4g_stop_hunt_completion_and_discipline_knowledge` + plus two drift guards - the fear/book pair above, and + `test_v4g_stop_hunt_rule_does_not_become_always_fade_the_bounce`, which keeps + the large-gap escape hatch alive so the rule cannot harden into "always fade". +- Prompt size 105,933 -> 111,283 chars. **Headroom is now 8,717** - at the + recent ~5,000 chars per version that is roughly two more before the 120,000 + cap, so the next pass should start pruning superseded prose rather than only + appending. + +### How our agent traded the same session + +**Provisional - the runner was still live at 14:13.** Realized so far: +**-9,229.75** across 45 legs, and the shape is the exact inverse of 12 Aug. + +| Strategy | Legs | Realized | +|---|---|---| +| SL Hunting AI | 4 | **+2,727.00** | +| Heikin Ashi | 6 | +2,431.00 | +| Supertrend Bullish | 1 | +1,043.25 | +| RSI Reversal | 1 | +994.50 | +| SMA Crossover | 2 | +539.50 | +| Long Strangle | 8 | +305.50 | +| ... | | | +| CPR Algo 3 | 1 | -2,158.00 | +| Donchian Bearish | 1 | -2,567.50 | +| Supertrend | 4 | -2,671.50 | +| Opening Strike | 1 | -2,717.00 | +| Renko | 6 | -4,127.50 | +| **Total** | **45** | **-9,229.75** | + +**SL Hunting was the best strategy on the board**, cross-checked against its own +`Result summary` (+2,727.00, Trades=2). Yesterday it was the worst. + +What changed is exactly what v4g and v4d describe. Both trades were SHORT off the +flat open - the same side IH took - and **both exits keyed off the round number**: + +| Entry | Setup | Exit | Held | +|---|---|---|---| +| 09:29 | flat_open_pivot_breakdown_bearish_engulfing (stop 24353, target 24300) | 09:35 `profit_booking_round_number` | 6 min | +| 10:06 | double_top_rejection_confirmed_bearish | 10:26 `profit_book_stall_near_round_number` | 20 min | + +Contrast with 12 Aug: four entries in 47 minutes, three released on premise-STALL +judgements, one cut by the hierarchy rule after 60 seconds. Today: two entries in +57 minutes, both booked on a NAMED level. The difference is not the read - it was +short both days - it is that the exits had a checkable reason, which is precisely +what v4g's FEAR IS NOT A SIGNAL demands and what the stall-churn lacked. + +One caveat against reading too much into it: two trades is a small sample, and +the deterministic strategies had a poor day on the same tape (Renko -4,127.50 over +six trades), so the basket is deeply negative regardless. diff --git a/Signal Generators/SL Hunting AI Agent/sl_hunting_knowledge.py b/Signal Generators/SL Hunting AI Agent/sl_hunting_knowledge.py index 24f8d1d..906d44d 100644 --- a/Signal Generators/SL Hunting AI Agent/sl_hunting_knowledge.py +++ b/Signal Generators/SL Hunting AI Agent/sl_hunting_knowledge.py @@ -620,6 +620,37 @@ target case rather than the entry case: "gradually Sensex and NIFTY will try to cover themselves, so we will get our target." Read alongside INDEX HIERARCHY: the hierarchy decides who must AGREE, this decides who to WEIGHT. +- A COMPLETED STOP-HUNT ENDS THAT DIRECTION (v4g). The sharpest single idea in + the series, and it inverts the naive reading. When a move has just finished + taking out one side's stops, that move has SPENT its fuel — it does not + continue, it turns. IH on a flat open after the prior session's late bounce: + "yesterday the market gave good selling, then took support EXACTLY at the 500 + level and gave a retracement. Because of that retracement, whoever was selling + got chased out... so the chances of going DIRECTLY UP are LOW." + The up-move existed to clear the shorts. With the shorts gone there is nobody + left to squeeze, so the path of least resistance is back down. He states the + operating rule plainly: "if it has chased the sellers out, we try to follow + THAT SAME DIRECTION" — meaning the direction the market was in BEFORE the + clearing bounce, not the bounce itself. + Practical form: after a retracement that visibly cleared one side, do NOT + chase the retracement. Trade the original direction, and treat the bounce's + end as the entry. The one thing that voids this is a fresh large gap, which + recruits a new crowd and restarts the question (see the gap branches above). +- THE ROUND NUMBER IS WHERE THE THESIS DIES, NOT JUST WHERE IT PAYS (v4g). + Earlier versions used round numbers to locate targets (v4d BOOK BEFORE THE + ROUND NUMBER) and recruitment (v4c ROUND NUMBERS AMPLIFY RECRUITMENT). This + adds the third and most operational use: the round number is the level at + which the trade is WRONG, declared before entry. IH, opening a short: "until + the market crosses the round number — as we see in NIFTY, the 24,500 level — + until Sensex crosses that resistance, we will not have much problem." And on + the BankNIFTY buyers he intended to hunt: "when is there no danger to these + buyers? If the market goes above 58,000, or gives a direct gap-up — then + whether buyers are seated or not, we cannot target them." + So each index carries its own named invalidation, and it is a ROUND number + rather than an indicator level. Name it with the entry, not after the trade + starts hurting, and treat a decisive cross as the read failing even if the + arithmetic stop has not been touched (this is the concrete form of v4d's + PRE-COMMIT THE ADVERSE MOVE YOUR THESIS TOLERATES). - SEATED-BUYER TEST — run this BEFORE the long branch fires (v3y). The whole gap-up-long premise is "a gap-up leaves nobody trapped, so there is no hunt available". That premise is FALSE when the prior session already seated a buying @@ -1061,6 +1092,47 @@ This is the general form of BOOK BEFORE THE ROUND NUMBER (v4d): that rule names WHERE the late crowd's targets sit, this one names WHEN your own edge has been spent regardless of where price is. +- FEAR IS NOT A SIGNAL — NEVER CONVERT IT INTO AN EXIT (v4g). The single most + important guard on the rule directly above, and the two must be read together: + BOOK WHEN THE PROFIT STOPS GROWING is a MEASUREMENT (the rate of accrual has + fallen); this rule forbids the same action when the input is an EMOTION. + IH, with an open winner approaching target: "before the target is hit there is + a fear — should I book here, what if the market turns? ... Fear is not a big + deal, I feel it too. The target is almost about to hit and I feel it. But do + NOT convert that fear into ACTION. Feeling fear is fine; do not make a mistake + in handling your position because of it." + He also gives the mechanism for why cutting early is corrosive rather than + merely suboptimal: "if you cut early it gradually becomes a HABIT. Then you + cut small profits and leave, and when there is a loss you wait a long time to + save the position and take a BIG loss. That is why most traders never become + profitable." So an early book is not a small cost paid once — it trains the + asymmetry that destroys the account. + Operationally: an exit needs a NAMED, checkable reason — stop, target, the + profit rate falling, the premise invalidated, the round number crossed, the + time cutoff. "It might turn" is not on that list. +- TIME SPENT IN THE TRADE SHRINKS THE ACHIEVABLE TARGET (v4g). Elapsed time is a + cost in its own right, separate from price. A trade that stalls and round-trips + does not merely return to where it started — it returns with less of the + session left to pay you. IH, after a near-target winner reversed and came back: + "the market turned and wasted our TIME... because time was spent we will have + to wait extra. Maybe we get the target only after the breakdown now." The + position that had shown a full target was booked for roughly half: "only half + the profit is showing, where earlier it showed double." + So when a trade consumes materially more time than the read assumed, SHRINK + the target rather than extending the wait — especially on an expiry session, + where premium is draining while you wait. +- NEVER EXIT AT ZERO AFTER A GOOD PROFIT HAS PRINTED (v4g). Once meaningful open + profit has actually appeared on the screen, the floor for that trade stops + being breakeven and becomes "some profit". IH, booking a reduced winner: "we + book here, because if this profit also reduces we will have to exit at + zero-zero — and we do not want to exit zero-zero after having SEEN a good + profit." Note this is NOT the fear rule above in disguise: the trigger is the + observed fact that the move has stopped working ("the momentum is not + happening, the market went a bit down, a bit up"), and the printed profit only + sets the FLOOR for what an acceptable exit looks like once that fact is + established. It pairs with v4d's YOUR ENTRY PRICE IS THE FOURTH TARGET INPUT: + that one says a poor fill shrinks the target, this one says a good unrealised + print raises the minimum acceptable outcome. - CROWD SIZE IS THE THIRD TARGET INPUT (v4c). Alongside how recently the crowd was recruited (v4a) and whether it has averaged down (v4b), HOW MANY are seated scales the move available against them — and for a reason worth knowing: a diff --git a/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_schema.py b/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_schema.py index 9e6ea14..dc32cb2 100644 --- a/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_schema.py +++ b/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_schema.py @@ -963,6 +963,68 @@ def test_v4f_exit_rule_stays_on_the_winning_side_only(): assert "DISCIPLINE IS ASYMMETRIC BETWEEN WINNERS AND LOSERS" in prompt +def test_system_prompt_has_v4g_stop_hunt_completion_and_discipline_knowledge(): + """v4g (13 Aug live session): a reduced win, and the richest discipline block. + + Two structural ideas -- a completed stop-hunt marks the END of that + direction, and the round number is the declared invalidation rather than + only a target -- plus the psychology that guards the v4f exit rule. + """ + prompt = build_system_prompt() + assert "A COMPLETED STOP-HUNT ENDS THAT DIRECTION" in prompt + assert "THE ROUND NUMBER IS WHERE THE THESIS DIES" in prompt + assert "FEAR IS NOT A SIGNAL" in prompt + assert "TIME SPENT IN THE TRADE SHRINKS THE ACHIEVABLE TARGET" in prompt + assert "NEVER EXIT AT ZERO AFTER A GOOD PROFIT HAS PRINTED" in prompt + + +def test_v4g_fear_rule_and_v4f_book_rule_do_not_cancel_each_other(): + """The most dangerous pair in the whole prompt, and the reason for this test. + + v4f says BOOK WHEN THE PROFIT STOPS GROWING. v4g says never exit because you + are afraid it might turn. Read carelessly the second reads as "hold through + everything" and would disable the first; read carelessly the first licenses + exactly the fear-driven early exit the second forbids. The distinction is + MEASUREMENT versus EMOTION, and both halves have to say so explicitly. + """ + prompt = build_system_prompt() + + fear = prompt[prompt.index("FEAR IS NOT A SIGNAL"):] + fear = fear[: fear.index("\n- ")] if "\n- " in fear else fear + # It must name the measurement rule it is guarding, not contradict it... + assert "BOOK WHEN THE PROFIT STOPS GROWING" in fear + assert "MEASUREMENT" in fear + assert "EMOTION" in fear + # ...and it must require a checkable reason rather than banning exits. + assert "NAMED, checkable reason" in fear + for allowed in ("stop", "target", "premise invalidated", "time cutoff"): + assert allowed in fear, allowed + + # The zero-zero floor must not become a second fear-driven exit: it triggers + # on an observed fact, and only then sets the floor. + floor = prompt[prompt.index("NEVER EXIT AT ZERO AFTER A GOOD PROFIT HAS PRINTED"):] + floor = floor[: floor.index("\n- ")] if "\n- " in floor else floor + assert "NOT the fear rule above in disguise" in floor + assert "the move has stopped working" in floor + + +def test_v4g_stop_hunt_rule_does_not_become_always_fade_the_bounce(): + """The completed-stop-hunt rule needs its escape hatch intact. + + "Follow the original direction after a clearing retracement" is powerful and + would be dangerous as an unconditional rule, so the gap exception that voids + it must survive any later edit. + """ + prompt = build_system_prompt() + section = prompt[prompt.index("A COMPLETED STOP-HUNT ENDS THAT DIRECTION"):] + section = section[: section.index("\n- ")] if "\n- " in section else section + assert "SPENT its fuel" in section + assert "do NOT" in section and "chase the retracement" in section + # The voiding condition. + assert "fresh large gap" in section + assert "recruits a new crowd" in section + + def test_reentry_gate_does_not_contradict_the_exit_rules(): """The re-entry gate must never be readable as a reason to delay an EXIT. From cd4b53734b39a8d9cf7477217b79df2eded614ea Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 15:51:08 +0530 Subject: [PATCH 02/10] feat(session-state): supervisor heartbeat and a stale-marks guard The durable/marks split's first live session exposed a second hole. On 2026-08-12 the supervisor stopped writing marks at 12:30 while workers traded on until 15:10 -- no exception, no partial file, no log line. It was found hours later by comparing file mtimes, and even then the cause could not be determined, because MainThread emits nothing during a healthy session so its silence carried no information. Worse, the marks left on disk described 11 open positions where 23 were genuinely open. With resume enabled that 2h40m-stale set would have been restored as a current book -- inventing exposure that had been closed and omitting exposure that had been opened. That is the exact failure ADR-0012 exists to prevent, arriving through the file it created. Two additions: - SUPERVISOR HEARTBEAT (SESSION_STATE_HEARTBEAT_SECONDS, default 300s, ~75 lines a session): workers alive, completed marks writes, marks age, open positions, trades recorded. Its presence proves the loop is turning; the gap between the last heartbeat and a crash localises where it stopped. SessionStateStore.health() supplies the counters and warn_if_marks_stalled() raises one ERROR per stall episode once the age exceeds MARKS_STALL_INTERVALS x the interval. The heartbeat swallows its own exceptions and still advances its stamp: a diagnostic that can take down the supervisor, or spin the log, is worse than none. - STALE-MARKS GUARD. load_session_state now records marks_age_seconds from the two documents' updated_at stamps, and resumable_open_positions refuses EVERY position when that lag exceeds MAX_RESUMABLE_MARKS_AGE_SECONDS (300s) or cannot be determined. Fail-closed on an unknown lag is deliberate: a book of unknown age is not a book. Realized P&L is explicitly unaffected by the refusal and the log line says so. What deliberately gets no guard: the durable half. Trade events are written by the trading threads themselves, so a frozen supervisor cannot touch realized P&L -- which is why the 12 Aug reconciliation was exact even though the marks were useless. Test fixtures had to be corrected as well, and the correction is the point: both the module and the runner suites built state by hand in the PRE-SPLIT single-file shape, so they carried no marks lag and the new guard refused them. They now write the real pair of files, which is what they should always have done -- a fixture that does not match the on-disk shape cannot exercise a guard that reads it. 8 new module tests (53 total) covering live/stalled health, the one-shot warning latch, the latch clearing on recovery, lag measurement, and the stale / just-inside-limit / unknown-lag resume cases; 4 new runner tests covering heartbeat throttling, stall detection, exception isolation, and a stale-marks resume refusal. Gates: master 520 OK, market-data-health 26 OK, pytest 1158 passed, ruff/mypy clean, session_state.py branch coverage 91.3% against its 90% budget. check-env reports zero undocumented keys. Co-Authored-By: Claude Opus 5 --- Dependencies/env.example | 7 ++ Dependencies/session_state.py | 119 ++++++++++++++++++ ...Multi Strategy Front Test - Master File.py | 58 +++++++++ Tests/Dependencies/test_session_state.py | 110 ++++++++++++++++ Tests/test_nifty_multi_strategy_master.py | 113 ++++++++++++++--- docs/adr/0012-crash-durable-session-state.md | 31 +++++ docs/lld/reporting-and-observability.md | 17 +++ 7 files changed, 436 insertions(+), 19 deletions(-) diff --git a/Dependencies/env.example b/Dependencies/env.example index b5c0ddd..14ce1f5 100644 --- a/Dependencies/env.example +++ b/Dependencies/env.example @@ -140,6 +140,13 @@ SESSION_STATE_SNAPSHOT_SECONDS=30.0 # authority. Every replaced state file is archived beside the configured file as # `session_state..recovery.json` before the new run writes anything. SESSION_STATE_RESUME_ENABLED=false +# How often the supervisor logs a liveness line (workers alive, marks writes, +# marks age, open positions). MainThread is otherwise silent through a healthy +# session, so when the snapshot loop stopped on 2026-08-12 its silence carried no +# information and the freeze was only found hours later from a file mtime. At the +# default that is ~75 lines a session, and a stall becomes visible within minutes +# instead of at end of day. Raise it to quieten the log; do not disable it. +SESSION_STATE_HEARTBEAT_SECONDS=300.0 # Underlying instrument symbol prefix used to filter the option chain. UNDERLYING=NIFTY diff --git a/Dependencies/session_state.py b/Dependencies/session_state.py index 79df474..08fab3c 100644 --- a/Dependencies/session_state.py +++ b/Dependencies/session_state.py @@ -94,6 +94,20 @@ # lines in one session (2026-08-11) and buried the 13 that actually mattered. SLOW_MARKS_WRITE_WARNING_SECONDS = 2.0 +# How far the marks file may lag the durable document before its open positions +# stop being trustworthy. This exists because of 2026-08-12: the snapshot loop +# silently stopped at 12:30 while trading continued to 15:10, so the marks on +# disk described a 2h40m-old book -- 11 positions where 23 were actually open. +# Nothing warned, and nothing would have stopped a resume from restoring that +# stale set as if it were current, which is precisely the "invent exposure" +# failure ADR-0012 exists to prevent. +MAX_RESUMABLE_MARKS_AGE_SECONDS = 300.0 + +# How long the supervisor may go without a successful marks write before the +# store says so. Deliberately a multiple of the snapshot interval rather than a +# fixed number, so a slow disk that merely delays a write does not cry wolf. +MARKS_STALL_INTERVALS = 4 + # Per-strategy keys that live in the MARKS file rather than the durable one. # All of them are refreshed wholesale by every snapshot, so losing the newest # 30 seconds of them in a crash is the documented trade-off (ADR-0012); none is @@ -297,6 +311,16 @@ def __init__( # permissions, missing drive) logs one error instead of one per trade. self._write_failure_logged = False + # Liveness bookkeeping for the supervisor heartbeat and the stall guard. + # `_marks_writes` counts SUCCESSFUL marks publishes; the two timestamps + # are monotonic (immune to a clock step) and wall-clock (readable in a + # log line) views of the newest one. They start at construction so a loop + # that never runs at all still reports a growing age rather than None. + self._marks_writes = 0 + self._last_marks_write_monotonic = time.monotonic() + self._last_marks_write_at = _now_ist() + self._marks_stall_logged = False + self._state: dict[str, Any] = { "schema_version": SCHEMA_VERSION, "session_date": self.session_date.isoformat(), @@ -621,6 +645,13 @@ def _flush_marks_locked(self) -> None: elapsed = self._write_document( self.marks_path, self._marks_document_locked(), durable=False ) + # Only a COMPLETED publish counts. The 2026-08-12 freeze produced no + # error and no partial file, so "did a write finish" is the only signal + # that distinguishes a healthy loop from a stopped one. + self._marks_writes += 1 + self._last_marks_write_monotonic = time.monotonic() + self._last_marks_write_at = _now_ist() + self._marks_stall_logged = False if elapsed >= SLOW_MARKS_WRITE_WARNING_SECONDS: self.log.warning( "Session state marks write took %.3fs (path=%s); this delays " @@ -649,6 +680,56 @@ def snapshot(self) -> dict[str, Any]: with self._lock: return json.loads(json.dumps(self._state)) + def health(self) -> dict[str, Any]: + """Liveness of the snapshot loop, for the supervisor heartbeat. + + `marks_stalled` is the question 2026-08-12 could not answer: the loop + stopped writing at 12:30 while trading ran to 15:10, with no exception, + no partial file and no log line. Counting completed publishes and how + long ago the newest one was makes a stopped loop observable within a few + minutes instead of at end-of-day. + """ + with self._lock: + age = time.monotonic() - self._last_marks_write_monotonic + open_positions = sum( + 1 + for entry in self._state.get("strategies", {}).values() + if isinstance(entry, Mapping) and entry.get("open_position") + ) + return { + "marks_writes": self._marks_writes, + "marks_age_seconds": round(age, 1), + "marks_last_write_at": self._last_marks_write_at.isoformat(), + "marks_stalled": age > (self.snapshot_interval_seconds * MARKS_STALL_INTERVALS), + "trades_recorded": len(self._state.get("trades", [])), + "open_positions": open_positions, + "write_failures_logged": self._write_failure_logged, + } + + def warn_if_marks_stalled(self) -> bool: + """Log ONCE per stall episode that the snapshot loop has gone quiet. + + Returns True when a stall is currently detected (whether or not this + call did the logging). Kept here rather than in the runner so the + threshold and the one-shot latch live beside the counters they read. + """ + report = self.health() + if not report["marks_stalled"]: + return False + if not self._marks_stall_logged: + self._marks_stall_logged = True + self.log.error( + "Session state marks have not been written for %.0fs (expected every " + "%.0fs, last at %s, %d writes so far). Open-position recovery for this " + "session is DEGRADED: the marks file describes an older book than the " + "durable trade log. Realized P&L is unaffected.", + report["marks_age_seconds"], + self.snapshot_interval_seconds, + report["marks_last_write_at"], + report["marks_writes"], + ) + return True + def _marks_path_for(path: str | Path) -> Path: """Sibling marks path for a durable state path (``x.json`` -> ``x.marks.json``).""" @@ -714,6 +795,11 @@ def load_session_state(path: str | Path) -> dict[str, Any] | None: ) return state + # How far the marks lag the durable document. The durable file is stamped on + # every trade event, so this is a direct measure of "how much trading + # happened after the last snapshot" -- which on 2026-08-12 was 2h40m. + state["marks_age_seconds"] = _document_lag_seconds(state, marks) + mark_strategies = marks.get("strategies") if not isinstance(mark_strategies, Mapping): return state @@ -729,6 +815,22 @@ def load_session_state(path: str | Path) -> dict[str, Any] | None: return state +def _document_lag_seconds( + state: Mapping[str, Any], marks: Mapping[str, Any] +) -> float | None: + """Seconds by which the marks document trails the durable one. + + ``None`` when either timestamp is missing or unparseable — the caller treats + an unknown lag as untrustworthy rather than as zero. + """ + try: + durable_at = datetime.fromisoformat(str(state.get("updated_at", ""))) + marks_at = datetime.fromisoformat(str(marks.get("updated_at", ""))) + except ValueError: + return None + return round((durable_at - marks_at).total_seconds(), 1) + + def resumable_open_positions( state: Mapping[str, Any] | None, *, @@ -766,6 +868,23 @@ def resumable_open_positions( if bool(state.get("clean_shutdown", False)): return {} + # STALE MARKS ARE NOT A BOOK. The positions live in the best-effort marks + # file; if the snapshot loop stopped while trading continued, that file + # describes an older set of positions than actually existed. Restoring it + # would invent exposure that was closed and omit exposure that was opened -- + # exactly what happened on 2026-08-12, where a 2h40m-stale file held 11 + # positions against 23 genuinely open. Refuse rather than half-restore. + lag = state.get("marks_age_seconds") + if lag is None or float(lag) > MAX_RESUMABLE_MARKS_AGE_SECONDS: + logger.warning( + "Not resuming any position: the marks file lags the durable trade log " + "by %s (limit %.0fs), so its open positions are not a current book. " + "Realized P&L is unaffected; square off manually against the broker.", + "an unknown amount" if lag is None else f"{float(lag):.0f}s", + MAX_RESUMABLE_MARKS_AGE_SECONDS, + ) + return {} + resumable: dict[str, dict[str, Any]] = {} strategies = state.get("strategies") if not isinstance(strategies, Mapping): diff --git a/Nifty Multi Strategy Front Test - Master File.py b/Nifty Multi Strategy Front Test - Master File.py index 3b40326..4fbda7e 100644 --- a/Nifty Multi Strategy Front Test - Master File.py +++ b/Nifty Multi Strategy Front Test - Master File.py @@ -652,6 +652,12 @@ def _scaled_float(prefix: str, name: str, default: float) -> float: # and the runner already reconciles against it; a JSON file that disagrees with # the account is worse than no file at all. See `_resume_open_positions`. SESSION_STATE_RESUME_ENABLED = _env_bool("SESSION_STATE_RESUME_ENABLED", False) +# How often the supervisor logs a liveness line. MainThread is otherwise silent +# through a healthy session, so when the snapshot loop stopped on 2026-08-12 its +# silence carried no information and the freeze was only found hours later from +# a file mtime. Five minutes is ~75 lines a session -- cheap enough to keep on +# always, frequent enough to localise a stall to a few minutes. +SESSION_STATE_HEARTBEAT_SECONDS = _env_float("SESSION_STATE_HEARTBEAT_SECONDS", 300.0) # Telegram trade-notification settings. See Dependencies/.env for the one-time # bot/channel setup. When disabled (or token/chat blank) the notifier thread is @@ -17335,6 +17341,51 @@ def _paper_position_from_record(record: dict) -> PaperPosition: ) +def _emit_supervisor_heartbeat( + session_state: SessionStateStore, + started_workers: list[BasePaperStrategyWorker], + last_heartbeat_at: float, +) -> float: + """Log one supervisor liveness line per interval; return the new stamp. + + Why this exists: on 2026-08-12 the snapshot loop stopped writing marks at + 12:30 while workers traded on until 15:10. There was no exception, no + partial file and no log line, so the only evidence was a file mtime found + hours later -- and it was impossible to tell afterwards whether MainThread + had blocked, the loop had exited, or writes were failing silently. + + MainThread otherwise logs nothing during a healthy session, so its silence + carried no information. A periodic line fixes that in both directions: its + presence shows the loop is turning, and the gap between the last heartbeat + and the crash localises where it stopped. + + Never raises: a diagnostic that can take the supervisor down is worse than + no diagnostic at all. + """ + now = time.monotonic() + if (now - last_heartbeat_at) < SESSION_STATE_HEARTBEAT_SECONDS: + return last_heartbeat_at + try: + report = session_state.health() + # An actual stall is an ERROR in its own right, logged once per episode + # by the store rather than repeated on every heartbeat. + session_state.warn_if_marks_stalled() + logger.info( + "Supervisor heartbeat | workers_alive=%d/%d | marks_writes=%d | " + "marks_age=%.0fs | open_positions=%d | trades_recorded=%d%s", + sum(1 for worker in started_workers if worker.is_alive()), + len(started_workers), + report["marks_writes"], + report["marks_age_seconds"], + report["open_positions"], + report["trades_recorded"], + " | PERSISTENCE DEGRADED" if report["write_failures_logged"] else "", + ) + except Exception: # noqa: BLE001 - a heartbeat must never stop supervision + logger.exception("Supervisor heartbeat failed; supervision continues.") + return now + + def _start_and_supervise_runtime_threads( fetcher: CentralMarketDataFetcher, telegram_worker: TelegramMessageWorker | None, @@ -17352,6 +17403,10 @@ def _start_and_supervise_runtime_threads( started_workers: list[BasePaperStrategyWorker] = [] shutdown_reason = "" + # Monotonic so an NTP step cannot silence or spam the heartbeat. Starting at + # 0.0 makes the first supervised tick emit one, which proves the loop was + # entered at all -- the 2026-08-12 freeze left no such evidence. + last_heartbeat_at = 0.0 try: fetcher.start() if telegram_worker is not None: @@ -17371,6 +17426,9 @@ def _start_and_supervise_runtime_threads( # immediately by publish_trade_event. if session_state is not None: session_state.update_worker_snapshot(_session_state_snapshots(started_workers)) + last_heartbeat_at = _emit_supervisor_heartbeat( + session_state, started_workers, last_heartbeat_at + ) return True except KeyboardInterrupt: shutdown_reason = "KEYBOARD_INTERRUPT" diff --git a/Tests/Dependencies/test_session_state.py b/Tests/Dependencies/test_session_state.py index 7d1fffe..bd7ae64 100644 --- a/Tests/Dependencies/test_session_state.py +++ b/Tests/Dependencies/test_session_state.py @@ -15,6 +15,7 @@ import json import os import threading +import time from dataclasses import dataclass from datetime import date, timedelta from pathlib import Path @@ -24,6 +25,7 @@ # Bare import: this folder's conftest.py puts the SOURCE `Dependencies/` on # sys.path, which is the same resolution the runtime performs. from session_state import ( + MAX_RESUMABLE_MARKS_AGE_SECONDS, SCHEMA_VERSION, SessionStateStore, _marks_path_for, @@ -387,6 +389,9 @@ def _crashed_state(**overrides) -> dict: "schema_version": SCHEMA_VERSION, "session_date": "2026-08-10", "clean_shutdown": False, + # A real merged document always carries this (load_session_state computes + # it from the two files' timestamps). Fresh marks = no lag. + "marks_age_seconds": 0.0, "strategies": { "Renko": { "live_trading": False, @@ -687,3 +692,108 @@ def test_slow_marks_write_warns_about_supervision_not_trading(state_path: Path, assert any("marks write took" in m and "not a trading decision" in m for m in messages) # And it must not blame the trading loop, which was the old message's error. assert not any("delaying the caller's trading loop" in m for m in messages) + + +# --------------------------------------------------------------------------- +# Snapshot-loop liveness and the stale-marks guard +# --------------------------------------------------------------------------- +# On 2026-08-12 the supervisor stopped writing marks at 12:30 while trading ran +# to 15:10. No exception, no partial file, no log line -- the freeze was only +# found by comparing file mtimes hours later. The marks on disk described 11 +# positions where 23 were genuinely open, and nothing would have stopped a +# resume from restoring that stale set as a current book. + + +def test_health_reports_a_live_snapshot_loop(state_path: Path): + store = _store(state_path, snapshot_interval_seconds=1.0) + before = store.health() + assert before["marks_writes"] == 0 + + store.update_worker_snapshot( + [{"strategy": "Renko", "open_position": serialize_position(_FakePosition())}], + force=True, + ) + after = store.health() + assert after["marks_writes"] == 1 + assert after["marks_age_seconds"] < 5 + assert after["marks_stalled"] is False + assert after["open_positions"] == 1 + + +def test_health_flags_a_stalled_snapshot_loop(state_path: Path, monkeypatch): + """The exact 2026-08-12 signature: writes simply stop, silently.""" + store = _store(state_path, snapshot_interval_seconds=30.0) + store.update_worker_snapshot([{"strategy": "Renko"}], force=True) + assert store.health()["marks_stalled"] is False + + # Jump the monotonic clock past MARKS_STALL_INTERVALS * interval. + import session_state as module + + base = time.monotonic() + monkeypatch.setattr(module.time, "monotonic", lambda: base + 30.0 * 4 + 1) + assert store.health()["marks_stalled"] is True + + +def test_marks_stall_warns_once_per_episode(state_path: Path, monkeypatch, caplog): + import session_state as module + + store = _store(state_path, snapshot_interval_seconds=30.0) + store.update_worker_snapshot([{"strategy": "Renko"}], force=True) + + base = time.monotonic() + monkeypatch.setattr(module.time, "monotonic", lambda: base + 200.0) + with caplog.at_level("ERROR"): + assert store.warn_if_marks_stalled() is True + assert store.warn_if_marks_stalled() is True # still stalled... + messages = [r.getMessage() for r in caplog.records] + assert sum("have not been written" in m for m in messages) == 1, "must log once" + # The message must be clear about what IS still safe. + assert any("Realized P&L is unaffected" in m for m in messages) + + +def test_a_successful_write_clears_the_stall_latch(state_path: Path, monkeypatch, caplog): + import session_state as module + + store = _store(state_path, snapshot_interval_seconds=30.0) + store.update_worker_snapshot([{"strategy": "Renko"}], force=True) + base = time.monotonic() + monkeypatch.setattr(module.time, "monotonic", lambda: base + 200.0) + store.warn_if_marks_stalled() + + monkeypatch.undo() + store.update_worker_snapshot([{"strategy": "Renko"}], force=True) + assert store.health()["marks_stalled"] is False + with caplog.at_level("ERROR"): + assert store.warn_if_marks_stalled() is False + + +def test_load_records_how_far_the_marks_lag_the_durable_file(state_path: Path): + store = _store(state_path) + store.record_trade_event({"action": "EXIT", "strategy": "Renko", "pnl": 5.0}) + store.update_worker_snapshot( + [{"strategy": "Renko", "open_position": serialize_position(_FakePosition())}], + force=True, + ) + merged = load_session_state(state_path) + assert merged is not None + assert merged["marks_age_seconds"] is not None + assert abs(float(merged["marks_age_seconds"])) < 30 + + +def test_stale_marks_are_never_resumable(state_path: Path): + """The 2026-08-12 scenario end to end: a book that is hours out of date.""" + state = _crashed_state(marks_age_seconds=2.0 * 60 * 60 + 40 * 60) # 2h40m + assert resumable_open_positions(state, session_date=TODAY) == {} + + +def test_marks_lag_just_inside_the_limit_still_resumes(): + state = _crashed_state(marks_age_seconds=MAX_RESUMABLE_MARKS_AGE_SECONDS - 1) + assert set(resumable_open_positions(state, session_date=TODAY)) == {"Renko"} + + +def test_unknown_marks_lag_fails_closed(): + """An unparseable or absent timestamp must refuse, not assume zero.""" + state = _crashed_state(marks_age_seconds=None) + assert resumable_open_positions(state, session_date=TODAY) == {} + del state["marks_age_seconds"] + assert resumable_open_positions(state, session_date=TODAY) == {} diff --git a/Tests/test_nifty_multi_strategy_master.py b/Tests/test_nifty_multi_strategy_master.py index 51f2459..0e68873 100644 --- a/Tests/test_nifty_multi_strategy_master.py +++ b/Tests/test_nifty_multi_strategy_master.py @@ -11611,6 +11611,44 @@ def test_snapshot_of_a_broken_worker_does_not_blind_the_others(self): self.assertFalse(snapshots[0]["snapshot_valid"]) self.assertTrue(snapshots[1]["snapshot_valid"]) + # -- supervisor heartbeat ------------------------------------------- + def test_heartbeat_is_throttled_to_its_interval(self): + """~75 lines a session, not one per supervised tick.""" + state = MagicMock() + state.health.return_value = { + "marks_writes": 3, "marks_age_seconds": 4.0, "open_positions": 1, + "trades_recorded": 7, "write_failures_logged": False, + } + now = time.monotonic() + # A heartbeat emitted moments ago must NOT produce another... + fresh = master_file._emit_supervisor_heartbeat(state, [self.worker], now) + self.assertEqual(fresh, now, "must not re-emit inside the interval") + state.health.assert_not_called() + + # ...but one from long ago must. + stale = now - master_file.SESSION_STATE_HEARTBEAT_SECONDS - 1 + emitted = master_file._emit_supervisor_heartbeat(state, [self.worker], stale) + self.assertGreater(emitted, stale) + state.health.assert_called_once() + + def test_heartbeat_checks_for_a_stalled_snapshot_loop(self): + """The heartbeat is also where the 2026-08-12 stall would surface.""" + state = MagicMock() + state.health.return_value = { + "marks_writes": 12, "marks_age_seconds": 9000.0, "open_positions": 11, + "trades_recorded": 108, "write_failures_logged": False, + } + master_file._emit_supervisor_heartbeat(state, [self.worker], 0.0) + state.warn_if_marks_stalled.assert_called_once() + + def test_heartbeat_never_breaks_supervision(self): + """A diagnostic that can kill the supervisor is worse than none.""" + state = MagicMock() + state.health.side_effect = RuntimeError("boom") + stamp = master_file._emit_supervisor_heartbeat(state, [self.worker], 0.0) + # It still advances the stamp, so a broken health() cannot spin the log. + self.assertGreater(stamp, 0.0) + # -- resume ---------------------------------------------------------- def test_position_record_round_trips_through_the_file(self): original = self._open_position() @@ -11679,16 +11717,38 @@ def test_nonfunctional_position_contract_is_rejected_before_resume(self): with self.assertRaises(ValueError): master_file._paper_position_from_record(broken) - def _write_state(self, tmpdir, *, live=False, clean=False, session_date=None): + def _write_state( + self, tmpdir, *, live=False, clean=False, session_date=None, marks_lag_seconds=0.0 + ): + """Write the PAIR of files the store really produces. + + The state is split across a durable document and a `.marks.` sibling, and + `load_session_state` derives the marks lag from their two `updated_at` + stamps. Writing only the durable half here would leave the lag unknown, + which the stale-marks guard correctly refuses -- so the fixture has to + mirror the real on-disk shape. `marks_lag_seconds` ages the marks file to + exercise that guard. + """ record = master_file.serialize_position( self._open_position(), leg_marks={"option": 98.1} ) - state = { + day = ( + session_date or datetime.now(master_file.IST_TIMEZONE).date() + ).isoformat() + durable_at = datetime.now(master_file.IST_TIMEZONE) + marks_at = durable_at - timedelta(seconds=float(marks_lag_seconds)) + durable = { "schema_version": 1, - "session_date": ( - session_date or datetime.now(master_file.IST_TIMEZONE).date() - ).isoformat(), + "session_date": day, + "updated_at": durable_at.isoformat(), "clean_shutdown": clean, + "strategies": {"Renko": {"recorded_pnl": -929.5, "recorded_trades": 2}}, + "trades": [], + } + marks = { + "schema_version": 1, + "session_date": day, + "updated_at": marks_at.isoformat(), "strategies": { "Renko": { "live_trading": live, @@ -11698,12 +11758,26 @@ def _write_state(self, tmpdir, *, live=False, clean=False, session_date=None): "open_position": record, } }, - "trades": [], } path = Path(tmpdir) / "session_state.json" - path.write_text(json.dumps(state), encoding="utf-8") + path.write_text(json.dumps(durable), encoding="utf-8") + Path(str(path).replace(".json", ".marks.json")).write_text( + json.dumps(marks), encoding="utf-8" + ) return str(path) + def test_resume_refuses_a_stale_marks_file(self): + """The 2026-08-12 freeze: marks 2h40m behind the durable trade log. + + Restoring that set would invent exposure that had been closed and omit + exposure that had been opened -- 11 positions against 23 truly open. + """ + with tempfile.TemporaryDirectory() as tmpdir: + path = self._write_state(tmpdir, marks_lag_seconds=2 * 3600 + 40 * 60) + restored = master_file._resume_open_positions([self.worker], path) + self.assertEqual(restored, 0) + self.assertFalse(self.worker.pos.active) + def test_resume_restores_position_pnl_and_the_ltp_subscription(self): with tempfile.TemporaryDirectory() as tmpdir: path = self._write_state(tmpdir) @@ -11798,27 +11872,28 @@ def test_resume_skips_a_strategy_that_is_not_running(self): path = self._write_state(tmpdir) self.assertEqual(master_file._resume_open_positions([], path), 0) + def _patch_marks(self, path, **fields): + """Open positions live in the `.marks.` sibling, not the durable file.""" + marks_path = Path(str(path).replace(".json", ".marks.json")) + marks = json.loads(marks_path.read_text(encoding="utf-8")) + marks["strategies"]["Renko"]["open_position"].update(fields) + marks_path.write_text(json.dumps(marks), encoding="utf-8") + def test_resume_leaves_the_worker_flat_when_the_record_is_unusable(self): with tempfile.TemporaryDirectory() as tmpdir: - path = Path(tmpdir) / "session_state.json" - state = json.loads(Path(self._write_state(tmpdir)).read_text(encoding="utf-8")) - state["strategies"]["Renko"]["open_position"]["quantity"] = 0 - path.write_text(json.dumps(state), encoding="utf-8") - restored = master_file._resume_open_positions([self.worker], str(path)) + path = self._write_state(tmpdir) + self._patch_marks(path, quantity=0) + restored = master_file._resume_open_positions([self.worker], path) self.assertEqual(restored, 0) self.assertFalse(self.worker.pos.active) def test_resume_refuses_an_unsupported_position_shape(self): with tempfile.TemporaryDirectory() as tmpdir: - path = Path(tmpdir) / "session_state.json" - state = json.loads(Path(self._write_state(tmpdir)).read_text(encoding="utf-8")) - state["strategies"]["Renko"]["open_position"]["position_type"] = ( - "HedgedPaperPosition" - ) - path.write_text(json.dumps(state), encoding="utf-8") + path = self._write_state(tmpdir) + self._patch_marks(path, position_type="HedgedPaperPosition") self.assertEqual( - master_file._resume_open_positions([self.worker], str(path)), 0 + master_file._resume_open_positions([self.worker], path), 0 ) self.assertFalse(self.worker.pos.active) diff --git a/docs/adr/0012-crash-durable-session-state.md b/docs/adr/0012-crash-durable-session-state.md index a430f5e..d0432ab 100644 --- a/docs/adr/0012-crash-durable-session-state.md +++ b/docs/adr/0012-crash-durable-session-state.md @@ -123,6 +123,37 @@ durability from "guaranteed before the call returns" to a sub-second window — which is the guarantee this ADR was written to provide. At a 0.414 s median, 13 times a session, that trade is not worth making. +#### Amendment (2026-08-13): the snapshot loop must be observable + +The split's first session exposed a second, independent hole. The supervisor +stopped writing marks at **12:30** while workers traded on to **15:10** — no +exception, no partial file, no log line. It was found hours later by comparing +file mtimes, and even then the cause could not be determined: MainThread emits +nothing during a healthy session, so its silence carried no information. + +Worse, the marks left on disk described **11 open positions where 23 were +genuinely open**. Had resume been enabled, it would have restored that +2h40m-stale set as a current book — inventing exposure that had been closed and +omitting exposure that had been opened, which is the exact failure this ADR +exists to prevent, arriving through the file it created. + +Two additions: + +- **A supervisor heartbeat** (`SESSION_STATE_HEARTBEAT_SECONDS`, default 300s) + logging workers alive, completed marks writes, marks age, open positions and + trades recorded. Its presence proves the loop is turning; the gap between the + last heartbeat and a crash localises where it stopped. `health()` supplies the + counters and `warn_if_marks_stalled()` raises one ERROR per stall episode. +- **A stale-marks guard.** `load_session_state` records `marks_age_seconds` from + the two documents' `updated_at` stamps, and `resumable_open_positions` refuses + every position when that lag exceeds `MAX_RESUMABLE_MARKS_AGE_SECONDS` (300s) + or cannot be determined. Fail-closed on an unknown lag is deliberate: a book + of unknown age is not a book. + +Note what is NOT guarded: the durable half needs none of this. Trade events are +written by the trading threads themselves, so a frozen supervisor cannot affect +realized P&L — which is why the 12 Aug reconciliation was still exact. + ##### Measured after the change (2026-08-12, first session on the split) | | warnings | median | max | >1s | diff --git a/docs/lld/reporting-and-observability.md b/docs/lld/reporting-and-observability.md index d01a9e5..e493bdb 100644 --- a/docs/lld/reporting-and-observability.md +++ b/docs/lld/reporting-and-observability.md @@ -152,6 +152,21 @@ Properties that make it trustworthy: a reporting problem; the first failure logs loudly, then stays quiet. - **Cache-only marks** — `_position_leg_marks` reads the shared LTP cache and never the broker, because it runs on the supervisor thread. +- **Observable liveness** — the supervisor logs a heartbeat every + `SESSION_STATE_HEARTBEAT_SECONDS` (default 300) with workers alive, completed + marks writes, marks age, open positions and trades recorded. MainThread is + otherwise silent through a healthy session, which is why the 2026-08-12 + snapshot freeze (writes stopped at 12:30, trading continued to 15:10) left no + evidence beyond a file mtime found hours later. `SessionStateStore.health()` + supplies the numbers; `warn_if_marks_stalled()` raises one ERROR per stall + episode once the marks age exceeds `MARKS_STALL_INTERVALS` × the interval. +- **Stale marks are never resumable** — `load_session_state` records + `marks_age_seconds` from the two documents' timestamps, and + `resumable_open_positions` refuses everything when that lag exceeds + `MAX_RESUMABLE_MARKS_AGE_SECONDS` (300s) **or cannot be determined**. On + 2026-08-12 the marks described 11 positions where 23 were genuinely open; + restoring them would have invented exposure that was closed and omitted + exposure that was opened. Realized P&L is unaffected by this refusal. - **Resume is opt-in and narrow** — `SESSION_STATE_RESUME_ENABLED` (default false), and a record is offered back only if it is from **today**, from an **unclean** shutdown, **paper**, and a single-leg `PaperPosition`. Live @@ -180,6 +195,8 @@ gitignored — it holds live position and P&L detail. | Session state durable write exceeds 250ms | Trading still waits for that `fsync` so the event is genuinely durable; a warning identifies local-disk latency for operator action. | | Session state marks write exceeds 2s | Warned separately, and the message says so: this delays supervision, not a trading decision. | | Session state marks file corrupt or missing | P&L is still recovered from the durable file; only open positions are lost, so nothing is offered for resume. | +| Snapshot loop stops writing marks | One ERROR per episode naming the age and last write; the heartbeat keeps reporting it. Trading and realized P&L are unaffected; open-position recovery is degraded and resume refuses. | +| Marks lag the durable file by >300s | No position is offered for resume, logged with the measured lag. An unknown lag is treated the same way — it fails closed. | | Session state durable file corrupt on read | Ignored, logged; the run starts with no recovery rather than refusing to start. | --- From 5839559ecdd4345ee63440a98336aebff9ab885c Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 15:55:05 +0530 Subject: [PATCH 03/10] fix(session-state): heartbeat must not seed its stamp with 0.0 CI caught a platform-dependent bug that passed cleanly on the local box. `time.monotonic()` has an ARBITRARY origin. On Windows it is machine uptime, so a seed of 0.0 reads as "long ago" and the first supervised tick emits a heartbeat. In a freshly booted Linux container the counter starts near zero, so the same 0.0 reads as "just now" and the first heartbeat is suppressed for a full interval -- five minutes during which the loop's liveness is exactly as unobservable as it was on 2026-08-12, which is the whole thing this heartbeat exists to fix. The seed is now `None`, meaning "never emitted", so the first call always logs regardless of the platform's monotonic origin. The docstring and the supervisor comment both said the first tick would emit; that claim is now actually true rather than true only on Windows. test_first_heartbeat_always_fires_regardless_of_the_monotonic_origin is the regression guard, and test_heartbeat_never_breaks_supervision now asserts against a captured baseline instead of the literal 0.0 that made it platform-dependent too. Gates: master 521 OK, ruff/mypy clean. Co-Authored-By: Claude Opus 5 --- ...Multi Strategy Front Test - Master File.py | 23 +++++++++++++----- Tests/test_nifty_multi_strategy_master.py | 24 ++++++++++++++++--- 2 files changed, 38 insertions(+), 9 deletions(-) diff --git a/Nifty Multi Strategy Front Test - Master File.py b/Nifty Multi Strategy Front Test - Master File.py index 4fbda7e..cb86e96 100644 --- a/Nifty Multi Strategy Front Test - Master File.py +++ b/Nifty Multi Strategy Front Test - Master File.py @@ -17344,7 +17344,7 @@ def _paper_position_from_record(record: dict) -> PaperPosition: def _emit_supervisor_heartbeat( session_state: SessionStateStore, started_workers: list[BasePaperStrategyWorker], - last_heartbeat_at: float, + last_heartbeat_at: float | None, ) -> float: """Log one supervisor liveness line per interval; return the new stamp. @@ -17359,11 +17359,20 @@ def _emit_supervisor_heartbeat( presence shows the loop is turning, and the gap between the last heartbeat and the crash localises where it stopped. + ``last_heartbeat_at`` is ``None`` until the first line is emitted, and that + sentinel matters: `time.monotonic()` has an arbitrary origin, so a plain 0.0 + means "long ago" on a Windows box whose counter is machine uptime and "just + now" in a freshly booted Linux container. CI caught exactly that -- the first + heartbeat fired locally and was suppressed for five minutes on the runner. + Never raises: a diagnostic that can take the supervisor down is worse than no diagnostic at all. """ now = time.monotonic() - if (now - last_heartbeat_at) < SESSION_STATE_HEARTBEAT_SECONDS: + if ( + last_heartbeat_at is not None + and (now - last_heartbeat_at) < SESSION_STATE_HEARTBEAT_SECONDS + ): return last_heartbeat_at try: report = session_state.health() @@ -17403,10 +17412,12 @@ def _start_and_supervise_runtime_threads( started_workers: list[BasePaperStrategyWorker] = [] shutdown_reason = "" - # Monotonic so an NTP step cannot silence or spam the heartbeat. Starting at - # 0.0 makes the first supervised tick emit one, which proves the loop was - # entered at all -- the 2026-08-12 freeze left no such evidence. - last_heartbeat_at = 0.0 + # Monotonic so an NTP step cannot silence or spam the heartbeat. None means + # "never emitted", so the first supervised tick always logs one and proves + # the loop was entered at all -- the 2026-08-12 freeze left no such evidence. + # It must NOT be 0.0: monotonic()'s origin is arbitrary, so 0.0 reads as + # "long ago" on Windows and "just now" in a fresh Linux container. + last_heartbeat_at: float | None = None try: fetcher.start() if telegram_worker is not None: diff --git a/Tests/test_nifty_multi_strategy_master.py b/Tests/test_nifty_multi_strategy_master.py index 0e68873..e59250d 100644 --- a/Tests/test_nifty_multi_strategy_master.py +++ b/Tests/test_nifty_multi_strategy_master.py @@ -11638,16 +11638,34 @@ def test_heartbeat_checks_for_a_stalled_snapshot_loop(self): "marks_writes": 12, "marks_age_seconds": 9000.0, "open_positions": 11, "trades_recorded": 108, "write_failures_logged": False, } - master_file._emit_supervisor_heartbeat(state, [self.worker], 0.0) + master_file._emit_supervisor_heartbeat(state, [self.worker], None) state.warn_if_marks_stalled.assert_called_once() def test_heartbeat_never_breaks_supervision(self): """A diagnostic that can kill the supervisor is worse than none.""" state = MagicMock() state.health.side_effect = RuntimeError("boom") - stamp = master_file._emit_supervisor_heartbeat(state, [self.worker], 0.0) + before = time.monotonic() + stamp = master_file._emit_supervisor_heartbeat(state, [self.worker], None) # It still advances the stamp, so a broken health() cannot spin the log. - self.assertGreater(stamp, 0.0) + self.assertGreaterEqual(stamp, before) + + def test_first_heartbeat_always_fires_regardless_of_the_monotonic_origin(self): + """`None` means "never emitted"; 0.0 would be platform-dependent. + + `time.monotonic()` has an arbitrary origin -- machine uptime on Windows, + near zero in a freshly booted Linux container. Seeding the stamp with + 0.0 therefore reads as "long ago" on one and "just now" on the other, + which is exactly how this slipped through locally and failed in CI: the + first heartbeat fired here and was suppressed for five minutes there. + """ + state = MagicMock() + state.health.return_value = { + "marks_writes": 0, "marks_age_seconds": 0.0, "open_positions": 0, + "trades_recorded": 0, "write_failures_logged": False, + } + master_file._emit_supervisor_heartbeat(state, [self.worker], None) + state.health.assert_called_once() # -- resume ---------------------------------------------------------- def test_position_record_round_trips_through_the_file(self): From 09cf07444e23be1a97c9b3b8941c9ec772324e60 Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 17:18:37 +0530 Subject: [PATCH 04/10] feat(session-state): optional off-thread durable writer Reverses a decision this ADR made two days ago. The durable/marks split rejected a writer thread because a 0.414s median stall did not seem to justify weakening durability. That reasoning was incomplete: the stalls do not merely add latency, THEY DROP THE MARKET FEED. On 2026-08-11, 86% of the websocket disconnects (25 of 29) landed within 20 seconds of a session-state write stall, and `keepalive ping timeout` is the dominant feed error across the whole log -- the signature of a blocked event loop. Pre-feature days show 1-18 reconnects with zero write stalls. The split already cut correlated errors from 25 to 3; what remains are the ~15 trade-event writes still on trading threads. Two things I previously mis-stated about the trade-off: - The data does NOT land later. The write takes the same time either way, so it reaches the platter at the same wall-clock moment. What changes is that the publishing thread is not frozen meanwhile -- and it was frozen for exactly that interval before, unable to act on anything. - A lost queued event still has its log line. publish_trade_event is called AFTER the EXIT log record is emitted, and that log is what _parse_eod_pnl_by_day reads for the Sheet, so the fallback is the same one the system used before this module existed. What genuinely changes: a hard kill can lose events queued in the last write cycle rather than only the one in flight. Coalescing bounds that -- the document is a full rewrite, so a burst becomes ONE write, which also reduces total fsyncs rather than merely relocating them. DEFAULT OFF (SESSION_STATE_ASYNC_WRITES). This is a real reduction of the guarantee ADR-0012 exists to provide, so it is the operator's call rather than a silent default change -- the same treatment SESSION_STATE_RESUME_ENABLED got. Recommended ON for this hardware, where the feed cost is measured and recurring. stop_durable_writer() drains before the clean-shutdown flag is written, so an orderly end of day is exactly as durable as the synchronous path, and a failed drain falls back to a synchronous write rather than leaving a recorded trade unwritten. A write that raises re-marks the state dirty so the next cycle retries it instead of losing the event. 7 new tests (60 total): the caller not blocking, the synchronous path staying unchanged when no writer runs, burst coalescing into fewer writes, shutdown draining everything, idempotent stop, no double writer, and a failing write being retried rather than swallowed. Gates: master 521 OK, market-data-health 26 OK, pytest 1165 passed, ruff/mypy clean, session_state.py branch coverage 90.9% against its 90% budget, check-env zero undocumented keys. Co-Authored-By: Claude Opus 5 --- Dependencies/env.example | 12 ++ Dependencies/session_state.py | 110 +++++++++++++- ...Multi Strategy Front Test - Master File.py | 33 +++- Tests/Dependencies/test_session_state.py | 141 ++++++++++++++++++ docs/adr/0012-crash-durable-session-state.md | 37 +++++ 5 files changed, 330 insertions(+), 3 deletions(-) diff --git a/Dependencies/env.example b/Dependencies/env.example index 14ce1f5..bb9d37a 100644 --- a/Dependencies/env.example +++ b/Dependencies/env.example @@ -147,6 +147,18 @@ SESSION_STATE_RESUME_ENABLED=false # default that is ~75 lines a session, and a stall becomes visible within minutes # instead of at end of day. Raise it to quieten the log; do not disable it. SESSION_STATE_HEARTBEAT_SECONDS=300.0 +# Persist trade events from a BACKGROUND writer instead of on the trading thread +# that published them. OFF by default: it trades a small amount of the durability +# ADR-0012 exists to provide -- a hard kill can lose events queued in the last +# write cycle, where the synchronous path loses only the one in flight. +# +# Turn it ON if disk stalls are costing you the market feed. On 2026-08-11, 86% +# of websocket disconnects (25 of 29) landed within 20s of a session-state write +# stall, with 'keepalive ping timeout' dominant -- a blocked event loop drops the +# feed. The data reaches the platter at the same moment either way; what changes +# is that no trading thread waits for it. Lost queued events still have their +# EXIT log line, which is the source the EOD Sheet parses anyway. +SESSION_STATE_ASYNC_WRITES=false # Underlying instrument symbol prefix used to filter the option chain. UNDERLYING=NIFTY diff --git a/Dependencies/session_state.py b/Dependencies/session_state.py index 08fab3c..f28137a 100644 --- a/Dependencies/session_state.py +++ b/Dependencies/session_state.py @@ -321,6 +321,14 @@ def __init__( self._last_marks_write_at = _now_ist() self._marks_stall_logged = False + # Optional off-thread durable writer (see start_durable_writer). While + # `_writer` is None every trade event is persisted synchronously by the + # calling trading thread, which is the original ADR-0012 contract. + self._writer: threading.Thread | None = None + self._writer_stopping = False + self._durable_dirty = False + self._durable_wakeup = threading.Event() + self._state: dict[str, Any] = { "schema_version": SCHEMA_VERSION, "session_date": self.session_date.isoformat(), @@ -446,10 +454,110 @@ def record_trade_event(self, event: Mapping[str, Any]) -> None: # Drop oldest first -- a recovery cares about the newest. del trades[: len(trades) - self.max_trade_records] self._apply_pnl_bearing_event_locked(record) - self._flush_durable_locked() + if self._writer is None: + # Synchronous: the caller does not continue until the event + # is on the platter. Costs the caller the whole fsync. + self._flush_durable_locked() + return + # Asynchronous: the in-memory state is already updated, so the + # writer thread will persist THIS event and any that arrive + # while it works, in one coalesced write. + self._durable_dirty = True + self._durable_wakeup.set() except Exception: # noqa: BLE001 - reporting must never break trading self._log_write_failure("record trade event") + # ------------------------------------------------------------------ + # Optional off-thread durable writer + # ------------------------------------------------------------------ + # Why this exists: the synchronous write blocks the TRADING thread that + # published the event. On the operator's hardware that is a median 0.65s + # and up to 2.3s per event, and the stalls are not merely latency -- on + # 2026-08-11, 86% of the websocket feed disconnects (25 of 29) landed within + # 20s of a session-state write stall, with `keepalive ping timeout` the + # dominant error. A blocked event loop drops the market feed. + # + # What it costs: a hard kill can lose events that are queued but not yet + # written, where the synchronous path loses only the one in flight. Note + # the data does NOT land later -- the write takes the same time either way, + # so it reaches the platter at the same wall-clock moment; the difference is + # that the trading thread is not frozen meanwhile. The `EXIT` log line is + # also emitted BEFORE publish_trade_event, so a lost queued event still has + # a log record, which is the same source the EOD Sheet parses. + # + # Coalescing makes the exposure smaller than it first appears: the document + # is a full rewrite, so a burst of queued events becomes ONE write. That + # reduces total fsyncs as well as moving them off the trading path. + def start_durable_writer(self) -> None: + """Begin persisting trade events on a background thread.""" + with self._lock: + if self._writer is not None: + return + self._writer_stopping = False + self._writer = threading.Thread( + target=self._durable_writer_loop, + name="SessionStateWriter", + daemon=True, + ) + self._writer.start() + self.log.info( + "Session state durable writes are ASYNCHRONOUS: trade events are " + "persisted by a background writer, so a trading thread is never " + "blocked by fsync. A hard kill can lose events queued in the last " + "write cycle; their log lines survive." + ) + + def stop_durable_writer(self, timeout: float = 10.0) -> bool: + """Drain and stop the writer. Returns True when everything was written. + + Called at shutdown BEFORE results are published, so an orderly end of + day is exactly as durable as the synchronous path. + """ + with self._lock: + writer = self._writer + if writer is None: + return True + self._writer_stopping = True + self._durable_wakeup.set() + writer.join(timeout=timeout) + drained = not writer.is_alive() + with self._lock: + self._writer = None + pending = self._durable_dirty + if pending: + # Last resort: persist synchronously on the caller's thread rather + # than leave a recorded trade unwritten. + try: + with self._lock: + self._flush_durable_locked() + self._durable_dirty = False + except Exception: # noqa: BLE001 - shutdown must still complete + self._log_write_failure("drain the durable writer") + return False + return drained + + def _durable_writer_loop(self) -> None: + """Coalesce pending trade events into one atomic write at a time.""" + while True: + self._durable_wakeup.wait(timeout=1.0) + self._durable_wakeup.clear() + try: + with self._lock: + stopping = self._writer_stopping + dirty = self._durable_dirty + if dirty: + # Clear BEFORE writing: events arriving during the write + # re-set the flag and earn their own next cycle, so none + # is silently folded into a write that already started. + self._durable_dirty = False + self._flush_durable_locked() + except Exception: # noqa: BLE001 - the writer must never die quietly + self._log_write_failure("write session state from the writer thread") + with self._lock: + self._durable_dirty = True + if stopping and not dirty: + return + def _apply_pnl_bearing_event_locked(self, record: Mapping[str, Any]) -> None: """Fold a realized-P&L event into that strategy's running totals. diff --git a/Nifty Multi Strategy Front Test - Master File.py b/Nifty Multi Strategy Front Test - Master File.py index cb86e96..1cbcb2c 100644 --- a/Nifty Multi Strategy Front Test - Master File.py +++ b/Nifty Multi Strategy Front Test - Master File.py @@ -658,6 +658,20 @@ def _scaled_float(prefix: str, name: str, default: float) -> float: # a file mtime. Five minutes is ~75 lines a session -- cheap enough to keep on # always, frequent enough to localise a stall to a few minutes. SESSION_STATE_HEARTBEAT_SECONDS = _env_float("SESSION_STATE_HEARTBEAT_SECONDS", 300.0) +# Persist trade events from a background writer instead of on the trading thread +# that published them. OFF by default because it trades a small amount of the +# durability ADR-0012 exists to provide: a hard kill can lose events queued in +# the last write cycle, where the synchronous path loses only the one in flight. +# +# Turn it ON if disk stalls are costing you the market feed. On this operator's +# hardware they are: on 2026-08-11, 86% of websocket disconnects (25 of 29) +# landed within 20 seconds of a session-state write stall, with `keepalive ping +# timeout` the dominant error -- a blocked event loop drops the feed. The data +# still reaches the platter at the same moment either way (the write takes the +# same time); what changes is that no trading thread waits for it. Lost queued +# events also still have their `EXIT` log line, which is what the EOD Sheet +# parses, so the fallback is the same one used before this module existed. +SESSION_STATE_ASYNC_WRITES = _env_bool("SESSION_STATE_ASYNC_WRITES", False) # Telegram trade-notification settings. See Dependencies/.env for the one-time # bot/channel setup. When disabled (or token/chat blank) the notifier thread is @@ -18045,9 +18059,15 @@ def main() -> None: # One immediate write so the file exists (and is stamped with this # session's date) even if the process dies before the first trade. session_state.update_worker_snapshot(_session_state_snapshots(workers), force=True) + if SESSION_STATE_ASYNC_WRITES: + # Started AFTER the first snapshot so the file exists before any + # background writing begins. + session_state.start_durable_writer() logger.info( - "Session state persistence ENABLED -> %s (snapshot every %.0fs, resume=%s).", - SESSION_STATE_FILE, SESSION_STATE_SNAPSHOT_SECONDS, SESSION_STATE_RESUME_ENABLED, + "Session state persistence ENABLED -> %s (snapshot every %.0fs, " + "resume=%s, async_writes=%s).", + SESSION_STATE_FILE, SESSION_STATE_SNAPSHOT_SECONDS, + SESSION_STATE_RESUME_ENABLED, SESSION_STATE_ASYNC_WRITES, ) except Exception: # noqa: BLE001 - reporting must never stop a session session_state = None @@ -18100,6 +18120,15 @@ def main() -> None: # Sheet publication is a separate flag: a Ctrl+C shutdown or Google outage # can be locally clean while its figures still need export/reconciliation. if session_state is not None: + # Drain the background writer FIRST so an orderly end of day is exactly + # as durable as the synchronous path: every recorded trade must be on + # disk before the clean-shutdown flag claims the session finished well. + if not session_state.stop_durable_writer(): + logger.error( + "Session state writer did not drain cleanly; the durable file may be " + "missing the last trade event(s). Their log lines remain, and the " + "EOD Sheet parses those." + ) session_state.update_worker_snapshot(_session_state_snapshots(workers), force=True) session_state.mark_clean_shutdown( results_published=finalization.results_published, diff --git a/Tests/Dependencies/test_session_state.py b/Tests/Dependencies/test_session_state.py index bd7ae64..26cded6 100644 --- a/Tests/Dependencies/test_session_state.py +++ b/Tests/Dependencies/test_session_state.py @@ -797,3 +797,144 @@ def test_unknown_marks_lag_fails_closed(): assert resumable_open_positions(state, session_date=TODAY) == {} del state["marks_age_seconds"] assert resumable_open_positions(state, session_date=TODAY) == {} + + +# --------------------------------------------------------------------------- +# Optional off-thread durable writer +# --------------------------------------------------------------------------- +# The synchronous write blocks the trading thread that published the event. On +# 2026-08-11, 86% of websocket feed disconnects landed within 20s of a write +# stall (`keepalive ping timeout` dominant) -- a blocked event loop drops the +# market feed. These tests pin the contract of moving it off-thread. + + +def _slow_store(state_path: Path, delay: float = 0.25): + """A store whose durable write is artificially slow, like the real disk.""" + store = _store(state_path) + real = store._write_document + + def slow(target, document, *, durable): + if durable: + time.sleep(delay) + return real(target, document, durable=durable) + + store._write_document = slow # type: ignore[method-assign] + return store + + +def test_record_returns_immediately_once_the_writer_is_running(state_path: Path): + """The whole point: a trading thread must not wait on fsync.""" + store = _slow_store(state_path, delay=0.4) + store.start_durable_writer() + try: + began = time.monotonic() + store.record_trade_event({"action": "EXIT", "strategy": "Renko", "pnl": 1.0}) + elapsed = time.monotonic() - began + assert elapsed < 0.2, f"caller blocked for {elapsed:.3f}s" + finally: + assert store.stop_durable_writer(timeout=10.0) is True + + on_disk = load_session_state(state_path) + assert on_disk is not None + assert on_disk["strategies"]["Renko"]["recorded_pnl"] == 1.0 + + +def test_synchronous_path_is_unchanged_without_a_writer(state_path: Path): + """Default behaviour must remain the original ADR-0012 contract.""" + store = _slow_store(state_path, delay=0.3) + began = time.monotonic() + store.record_trade_event({"action": "EXIT", "strategy": "Renko", "pnl": 1.0}) + elapsed = time.monotonic() - began + assert elapsed >= 0.3, "without a writer the caller must pay the fsync" + assert load_session_state(state_path)["strategies"]["Renko"]["recorded_pnl"] == 1.0 + + +def test_a_burst_is_coalesced_into_fewer_writes(state_path: Path): + """Coalescing is what makes the queued-loss window small AND cuts fsyncs.""" + store = _slow_store(state_path, delay=0.15) + writes = [] + real = store._write_document + + def counting(target, document, *, durable): + if durable: + writes.append(1) + return real(target, document, durable=durable) + + store._write_document = counting # type: ignore[method-assign] + store.start_durable_writer() + try: + for i in range(12): + store.record_trade_event({"action": "EXIT", "strategy": "Renko", "pnl": 1.0, "n": i}) + finally: + assert store.stop_durable_writer(timeout=10.0) is True + + # Every event must be present... + state = load_session_state(state_path) + assert len(state["trades"]) == 12 + assert state["strategies"]["Renko"]["recorded_trades"] == 12 + # ...but they must NOT have cost 12 separate durable writes. + assert len(writes) < 12, f"no coalescing happened ({len(writes)} writes)" + + +def test_stop_drains_everything_recorded(state_path: Path): + """An orderly shutdown must be exactly as durable as the sync path.""" + store = _slow_store(state_path, delay=0.05) + store.start_durable_writer() + for i in range(6): + store.record_trade_event({"action": "EXIT", "strategy": "Renko", "pnl": 2.0, "n": i}) + assert store.stop_durable_writer(timeout=10.0) is True + + state = load_session_state(state_path) + assert len(state["trades"]) == 6 + assert state["strategies"]["Renko"]["recorded_pnl"] == 12.0 + + +def test_stop_is_idempotent_and_safe_without_a_writer(state_path: Path): + store = _store(state_path) + assert store.stop_durable_writer() is True # never started + store.start_durable_writer() + assert store.stop_durable_writer(timeout=10.0) is True + assert store.stop_durable_writer() is True # already stopped + + +def test_starting_twice_does_not_spawn_a_second_writer(state_path: Path): + store = _store(state_path) + store.start_durable_writer() + first = store._writer + store.start_durable_writer() + try: + assert store._writer is first + finally: + store.stop_durable_writer(timeout=10.0) + + +def test_a_failing_write_does_not_kill_the_writer_or_lose_the_event(state_path: Path): + """A transient disk error must be retried, not swallowed silently.""" + store = _store(state_path) + real = store._write_document + calls = {"n": 0} + + def flaky(target, document, *, durable): + if durable: + calls["n"] += 1 + if calls["n"] == 1: + raise OSError("disk hiccup") + return real(target, document, durable=durable) + + store._write_document = flaky # type: ignore[method-assign] + store.start_durable_writer() + store.record_trade_event({"action": "EXIT", "strategy": "Renko", "pnl": 7.0}) + try: + # The writer re-marks the state dirty, so a later cycle persists it. + deadline = time.monotonic() + 10.0 + while time.monotonic() < deadline: + state = load_session_state(state_path) + if state and state.get("strategies", {}).get("Renko", {}).get("recorded_pnl") == 7.0: + break + time.sleep(0.05) + finally: + store.stop_durable_writer(timeout=10.0) + + state = load_session_state(state_path) + assert state["strategies"]["Renko"]["recorded_pnl"] == 7.0 + assert calls["n"] >= 2, "the failed write must have been retried" diff --git a/docs/adr/0012-crash-durable-session-state.md b/docs/adr/0012-crash-durable-session-state.md index d0432ab..61fa4af 100644 --- a/docs/adr/0012-crash-durable-session-state.md +++ b/docs/adr/0012-crash-durable-session-state.md @@ -123,6 +123,43 @@ durability from "guaranteed before the call returns" to a sub-second window — which is the guarantee this ADR was written to provide. At a 0.414 s median, 13 times a session, that trade is not worth making. +#### Amendment (2026-08-13): off-thread durable writes, offered but not imposed + +The split amendment above **rejected** moving trade-event writes to a writer +thread, on the grounds that a 0.414 s median stall did not justify weakening +durability. That reasoning was incomplete, and the correction is worth stating +plainly: the stalls do not merely add latency, **they drop the market feed**. + +On 2026-08-11, **86% of the websocket disconnects (25 of 29) landed within 20 +seconds of a session-state write stall**, and `keepalive ping timeout` is the +dominant feed error across the whole log — the signature of a blocked event +loop. Pre-feature days show 1–18 reconnects with zero write stalls. After the +durable/marks split cut stalls from 281 to 36, correlated errors fell from 25 to +3 and total reconnects halved. The remaining stalls are the ~15 trade-event +writes still on trading threads. + +Two things about the trade that were previously mis-stated: + +- **The data does not land later.** The write takes the same time either way, so + it reaches the platter at the same wall-clock moment. What changes is that the + publishing thread is not frozen meanwhile — and it was frozen for precisely + that interval before, unable to act on anything. +- **A lost queued event still has its log line.** `publish_trade_event` is called + *after* the `EXIT` log record is emitted, and that log is what + `_parse_eod_pnl_by_day` reads for the Sheet. The fallback is the same one the + system used before this module existed. + +What genuinely changes: a hard kill can lose events queued in the last write +cycle rather than only the one in flight. Coalescing bounds that — the document +is a full rewrite, so a burst becomes ONE write, which also reduces total fsyncs. + +**Decision: implement it, default OFF** (`SESSION_STATE_ASYNC_WRITES`). This is +a genuine reduction of the guarantee this ADR was written to provide, so it is +the operator's call rather than a silent default change — the same treatment +`SESSION_STATE_RESUME_ENABLED` got. `stop_durable_writer()` drains before the +clean-shutdown flag is written, so an orderly end of day is exactly as durable +as the synchronous path. + #### Amendment (2026-08-13): the snapshot loop must be observable The split's first session exposed a second, independent hole. The supervisor From e8cc1edfec338ffdf327c201de886a4fcb4dcce0 Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 17:42:48 +0530 Subject: [PATCH 05/10] Fix CPR official minute coverage Co-authored-by: Codex --- Dependencies/market_data_health.py | 34 ++++ ...Multi Strategy Front Test - Master File.py | 166 ++++++++++++------ Tests/test_market_data_health.py | 56 ++++++ Tests/test_nifty_multi_strategy_master.py | 162 ++++++++++++++++- 4 files changed, 365 insertions(+), 53 deletions(-) diff --git a/Dependencies/market_data_health.py b/Dependencies/market_data_health.py index 8d32935..e0c2db7 100644 --- a/Dependencies/market_data_health.py +++ b/Dependencies/market_data_health.py @@ -237,6 +237,40 @@ def newest_completed_minute_timestamp( return max(completed, default=None) +def stable_official_minutes( + validated_frame: pd.DataFrame, + *, + request_started_at: datetime, + grace_seconds: float, +) -> frozenset[pd.Timestamp]: + """Return REST minute stamps safely final at the request's start time. + + Dhan may include the currently forming candle in a REST response. A slow + HTTP response must not turn that provisional row into official evidence, + because the request might have started before the candle had closed. The + request-start clock is therefore authoritative: a row is usable only when + its start stamp is strictly before ``floor(request_started_at - grace)``. + + ``validated_frame`` is the normalized output of :func:`validate_ohlc_frame`. + Its timestamps are usually naive IST, but this helper deliberately accepts + aware values too so future adapters keep the same safety boundary. Empty + frames remain empty because callers may use this pure helper before the + store's non-empty publication validation. + """ + + if validated_frame is None or validated_frame.empty or "timestamp" not in validated_frame: + return frozenset() + request_ist = _as_aware_ist(request_started_at) + boundary = pd.Timestamp(request_ist - timedelta(seconds=max(0.0, float(grace_seconds)))) + boundary = boundary.floor("min").tz_localize(None) + stable: set[pd.Timestamp] = set() + for value in validated_frame["timestamp"]: + timestamp = pd.Timestamp(_as_aware_ist(value)).tz_localize(None) + if timestamp < boundary: + stable.add(timestamp) + return frozenset(stable) + + @dataclass(frozen=True) class MarketDataHealthSnapshot: """Immutable worker-facing view of the current feed safety state. diff --git a/Nifty Multi Strategy Front Test - Master File.py b/Nifty Multi Strategy Front Test - Master File.py index 3b40326..610e4dc 100644 --- a/Nifty Multi Strategy Front Test - Master File.py +++ b/Nifty Multi Strategy Front Test - Master File.py @@ -253,6 +253,7 @@ import time import uuid import warnings +from collections.abc import Iterable from dataclasses import dataclass from datetime import date, datetime, timedelta from datetime import time as dt_time @@ -289,6 +290,7 @@ MarketDataValidationError, complete_minute_bucket_mask, newest_completed_minute_timestamp, + stable_official_minutes, validate_ohlc_frame, ) from Dependencies.next_open_entry import PendingNextOpenEntry @@ -1904,8 +1906,10 @@ class MarketSnapshot: - `source_candle_ts` : timestamp of the latest candle in `frame`. - `candle_signature` : lightweight fingerprint of the latest row's state. - `fetched_at` : wall-clock time when the fetch completed. - - `official_candle_ts`: newest row supplied by the REST/official source, or - ``None`` when the publisher has not proved any official coverage. + - `official_completed_minutes`: exact immutable REST minute stamps proven + final for this generation. + - `official_candle_ts`: compatibility maximum derived from that exact set, + or ``None`` when the publisher has not proved any official coverage. Why we keep both `source_candle_ts` and `candle_signature`: - During a live 1-minute candle, the timestamp does not change but the @@ -1922,8 +1926,12 @@ class MarketSnapshot: candle_signature: tuple | None fetched_at: datetime # Websocket frames can contain a mix of official REST history and newer - # tick-built rows. This watermark tells a five-minute consumer exactly how - # far the official portion reaches without changing the canonical OHLC data. + # tick-built rows. CPR needs the exact REST minute identities, not merely a + # maximum: a missing 09:57 must block the 09:55 five-minute bucket even when + # official data already reaches 09:59. + official_completed_minutes: frozenset[pd.Timestamp] = frozenset() + # Compatibility view for existing non-CPR consumers. This is always derived + # from ``official_completed_minutes`` by SharedMarketDataStore.update(). official_candle_ts: pd.Timestamp | None = None @@ -2227,6 +2235,7 @@ def update( timeframe: str, frame: pd.DataFrame, *, + official_completed_minutes: Iterable[pd.Timestamp | datetime] | None = None, official_candle_ts: pd.Timestamp | datetime | None = None, ) -> MarketSnapshot: """ @@ -2235,13 +2244,29 @@ def update( The new snapshot is built BEFORE the lock is taken; only the swap happens under lock so the critical section stays small. - ``official_candle_ts`` is optional because a pure tick publisher has no - official coverage to claim. REST-backed publishers pass their newest - source timestamp so conservative consumers can wait for official data. + ``official_completed_minutes`` is optional because a pure tick publisher + has no official coverage to claim. REST-backed publishers pass every + proven-final minute in the same update as the matching frame. The legacy + ``official_candle_ts`` argument remains a compatibility input for older + callers; it is converted to a one-element exact collection. """ validated = validate_ohlc_frame(frame) source_candle_ts = pd.to_datetime(validated.iloc[-1]["timestamp"]) candle_signature = build_last_row_signature(validated) + if official_completed_minutes is None: + raw_official_minutes = () if official_candle_ts is None else (official_candle_ts,) + else: + raw_official_minutes = official_completed_minutes + normalized_official_values: set[pd.Timestamp] = set() + for value in raw_official_minutes: + timestamp = pd.Timestamp(value) + if pd.isna(timestamp): + continue + if timestamp.tzinfo is not None: + timestamp = timestamp.tz_convert(ZoneInfo("Asia/Kolkata")).tz_localize(None) + normalized_official_values.add(timestamp) + normalized_official_minutes = frozenset(normalized_official_values) + derived_official_candle_ts = max(normalized_official_minutes, default=None) snapshot = MarketSnapshot( timeframe=str(timeframe), @@ -2249,11 +2274,8 @@ def update( source_candle_ts=source_candle_ts, candle_signature=candle_signature, fetched_at=datetime.now(ZoneInfo("Asia/Kolkata")), - official_candle_ts=( - None - if official_candle_ts is None - else pd.Timestamp(official_candle_ts) - ), + official_completed_minutes=normalized_official_minutes, + official_candle_ts=derived_official_candle_ts, ) with self._lock: self._snapshots[str(timeframe)] = snapshot @@ -2277,6 +2299,7 @@ def get(self, timeframe: str) -> MarketSnapshot | None: source_candle_ts=snapshot.source_candle_ts, candle_signature=snapshot.candle_signature, fetched_at=snapshot.fetched_at, + official_completed_minutes=snapshot.official_completed_minutes, official_candle_ts=snapshot.official_candle_ts, ) @@ -3817,14 +3840,20 @@ def run(self) -> None: if self.stop_event.is_set(): break try: + # The request-start time, not the response arrival time, + # decides whether Dhan's final row was still forming. + request_started_at = _ist_now() frame = self.fetch_ohlc(timeframe) - # This producer's entire frame came directly from REST, so - # its newest timestamp is also the official-data watermark. - # Websocket mode publishes the same metadata after true-up. + validated = validate_ohlc_frame(frame) + completed_minutes = stable_official_minutes( + validated, + request_started_at=request_started_at, + grace_seconds=WS_TRUEUP_DELAY_SECONDS, + ) snapshot = self.store.update( timeframe, - frame, - official_candle_ts=pd.Timestamp(frame["timestamp"].max()), + validated, + official_completed_minutes=completed_minutes, ) if self.last_logged_candle_ts.get(timeframe) != snapshot.source_candle_ts: self.last_logged_candle_ts[timeframe] = snapshot.source_candle_ts @@ -3921,10 +3950,10 @@ def __init__( # Latest REST history: warmup seed, then refreshed by every true-up. # Supervisor-owned; the pump never touches it. self.official_frame: pd.DataFrame = pd.DataFrame() - # The frame may later be merged with newer tick-built rows. Keep the - # newest timestamp that came from REST as a separate watermark so a - # strategy can prove its completed bucket has been officially trued up. - self._official_candle_ts: pd.Timestamp | None = None + # The frame may later be merged with newer tick-built rows. Keep the + # exact REST minutes proved final for this generation so CPR can reject + # a five-minute bucket with an intermediate official-data hole. + self._official_completed_minutes: frozenset[pd.Timestamp] = frozenset() # Connection state shared between pump and supervisor. self._feed_lock = threading.Lock() @@ -4015,17 +4044,28 @@ def _warmup_official_history(self) -> bool: index_key = (NIFTY_INDEX_EXCHANGE_SEGMENT, NIFTY_INDEX_SECURITY_ID) while not self.stop_event.is_set(): try: + request_started_at = _ist_now() frame = self.broker.fetch_index_1m_ohlc( security_id=NIFTY_INDEX_SECURITY_ID, exchange_segment=NIFTY_INDEX_EXCHANGE_SEGMENT, instrument_type=NIFTY_INDEX_INSTRUMENT_TYPE, ) - self.official_frame = frame - self._official_candle_ts = pd.Timestamp(frame["timestamp"].max()) + validated = validate_ohlc_frame(frame) + completed_minutes = stable_official_minutes( + validated, + request_started_at=request_started_at, + grace_seconds=WS_TRUEUP_DELAY_SECONDS, + ) + # REST may include the live or grace-period minute. Leave those + # rows entirely tick-owned until a later request proves them final. + self.official_frame = validated.loc[ + validated["timestamp"].isin(completed_minutes) + ].reset_index(drop=True) + self._official_completed_minutes = completed_minutes self.store.update( "1", - frame, - official_candle_ts=self._official_candle_ts, + self.official_frame, + official_completed_minutes=completed_minutes, ) self.log.info("Warmup history loaded | Rows=%s", len(frame)) return True @@ -4156,7 +4196,7 @@ def _publish_frame_locked(self, force: bool) -> None: snapshot = self.store.update( "1", frame, - official_candle_ts=self._official_candle_ts, + official_completed_minutes=self._official_completed_minutes, ) self._ohlc_ok = True if self.last_logged_candle_ts != snapshot.source_candle_ts: @@ -4214,6 +4254,13 @@ def _run_true_up(self, reason: str, now_ist: datetime | None = None) -> None: """ if now_ist is None: now_ist = datetime.now(ZoneInfo("Asia/Kolkata")).replace(tzinfo=None) + # Record the actual request boundary before any blocking HTTP work. + # Test callers pass ``now_ist`` as that deterministic request-start clock. + request_started_at = ( + now_ist.replace(tzinfo=IST_TIMEZONE) + if now_ist is not None and now_ist.tzinfo is None + else now_ist or _ist_now() + ) try: official = self.broker.fetch_index_1m_ohlc( security_id=NIFTY_INDEX_SECURITY_ID, @@ -4228,25 +4275,34 @@ def _run_true_up(self, reason: str, now_ist: datetime | None = None) -> None: if official is None or official.empty: self.log.warning("True-up (%s) returned no official candles.", reason) return + validated = validate_ohlc_frame(official) + completed_minutes = stable_official_minutes( + validated, + request_started_at=request_started_at, + grace_seconds=WS_TRUEUP_DELAY_SECONDS, + ) + stable_official = validated.loc[ + validated["timestamp"].isin(completed_minutes) + ].reset_index(drop=True) stats = divergence_stats( - official, + stable_official, self.aggregator.tick_bars_frame(), forming_minute=pd.Timestamp(now_ist).floor("min"), ) - newest_official = pd.Timestamp(official["timestamp"].max()) # Update the official frame, its watermark, and the published merged # snapshot under one lock. Without this atomic boundary, the supervisor # could publish new official OHLC with an older watermark (or vice versa) # and make a waiting CPR worker observe a mixed generation. with self._publish_lock: - self.official_frame = official - self._official_candle_ts = newest_official + self.official_frame = stable_official + self._official_completed_minutes = completed_minutes self._publish_frame_locked(force=True) - # Drop every tick bar the official history now covers. The merge would - # ignore them anyway (official wins), but keeping them makes the NEXT - # divergence report re-count this cycle's mismatches forever -- the - # stats above must describe only the minutes trued-up right now. - self.aggregator.prune_older_than(newest_official + pd.Timedelta(minutes=1)) + newest_official = max(completed_minutes, default=None) + if newest_official is not None: + # Prune only through the newest stable official minute. A REST row + # still inside the grace period must remain tick-owned and available + # for the later, final true-up. + self.aggregator.prune_older_than(newest_official + pd.Timedelta(minutes=1)) log_fn = ( self.log.warning if stats.mismatched and stats.max_abs_delta > self.TRUEUP_DIVERGENCE_WARN_POINTS @@ -4255,7 +4311,7 @@ def _run_true_up(self, reason: str, now_ist: datetime | None = None) -> None: log_fn( "True-up (%s) | OfficialRows=%s | Overlap=%s | Mismatched=%s | MaxAbsDelta=%.2f", reason, - len(official), + len(stable_official), stats.overlapping, stats.mismatched, stats.max_abs_delta, @@ -9674,25 +9730,33 @@ def _official_snapshot_covers_completed_bar( snapshot: MarketSnapshot, strategy_frame: pd.DataFrame, ) -> bool: - """Return true only when REST covers the bucket's final source minute. + """Return true only when REST covers every exact source minute. Five-minute candles are start-stamped. A 09:55 candle therefore needs - the official one-minute source to cover 09:59 before Codex may freeze - it. This condition-based gate naturally tolerates a slow Dhan response: - the normal worker poll simply checks the next atomic snapshot instead - of guessing how many seconds the REST true-up will take. + every official one-minute source from 09:55 through 09:59 before Codex + may freeze it. A maximum timestamp alone is unsafe because a REST hole + (for example missing 09:57) would otherwise authorize invented OHLC. + This exact-set gate naturally tolerates a slow Dhan response: the normal + worker poll simply checks the next atomic snapshot instead of guessing + how many seconds the REST true-up will take. """ if strategy_frame.empty or "timestamp" not in strategy_frame.columns: return False - official_timestamp = self._naive_ist_timestamp(snapshot.official_candle_ts) bucket_start = self._naive_ist_timestamp( strategy_frame.iloc[-1]["timestamp"] ) - if official_timestamp is None or bucket_start is None: + if bucket_start is None: return False - bucket_final_minute = bucket_start + pd.Timedelta(minutes=4) - return official_timestamp >= bucket_final_minute + required_minutes = frozenset( + bucket_start + pd.Timedelta(minutes=offset) for offset in range(5) + ) + official_minutes = frozenset( + timestamp + for value in snapshot.official_completed_minutes + if (timestamp := self._naive_ist_timestamp(value)) is not None + ) + return required_minutes.issubset(official_minutes) def _position_state_payload(self) -> dict[str, object]: """Expose allowlisted premise/risk facts, never execution capabilities. @@ -10631,10 +10695,10 @@ def run(self) -> None: The worker starts decisions at 09:30 but safety runs even before then. Shared one-minute data is resampled with the current IST clock so forming - websocket minutes are excluded. A clock-complete bucket then waits until - the atomic REST watermark covers its fifth source minute. Any single poll - failure is logged and retried; it does not terminate future hard-stop or - square-off handling. + websocket minutes are excluded. A clock-complete bucket then waits for + all five exact official REST source minutes; an intermediate REST hole + keeps the bucket blocked. Any single poll failure is logged and retried; + it does not terminate future hard-stop or square-off handling. """ self.log.info("Starting %s strategy worker.", self.strategy_name) @@ -10669,8 +10733,8 @@ def run(self) -> None: if waiting_identity != self._waiting_for_official_bar_identity: self._waiting_for_official_bar_identity = waiting_identity self.log.info( - "Completed CPR bucket %s is waiting for its final " - "one-minute REST true-up.", + "Completed CPR bucket %s is waiting for all five " + "official source minutes (including any REST hole).", waiting_identity, ) self.wait_for_next_poll() diff --git a/Tests/test_market_data_health.py b/Tests/test_market_data_health.py index 3e74395..496e68f 100644 --- a/Tests/test_market_data_health.py +++ b/Tests/test_market_data_health.py @@ -15,6 +15,7 @@ complete_minute_bucket_mask, market_hours_between, newest_completed_minute_timestamp, + stable_official_minutes, validate_ohlc_frame, ) @@ -185,6 +186,61 @@ def test_excludes_current_minute_and_returns_latest_completed_bar(self) -> None: ) +class TestStableOfficialMinutes(unittest.TestCase): + """REST rows become official only after the request-start grace boundary.""" + + def test_uses_request_start_and_normalizes_timestamp_representations(self) -> None: + """A slow response cannot certify a row that was forming when requested.""" + + frame = validate_ohlc_frame( + _frame( + [ + pd.Timestamp("2026-08-13 03:58:00+00:00"), + pd.Timestamp("2026-08-13 03:59:00+00:00"), + ] + ), + now=datetime(2026, 8, 13, 15, 0, tzinfo=IST), + ) + + stable = stable_official_minutes( + frame, + request_started_at=datetime(2026, 8, 13, 9, 29, 59), + grace_seconds=5.0, + ) + + self.assertEqual(stable, frozenset({pd.Timestamp("2026-08-13 09:28:00")})) + + def test_certifies_just_closed_minute_after_grace_and_keeps_empty_empty(self) -> None: + """Strictly-before boundary admits 09:29 only after 09:30:05 IST.""" + + frame = validate_ohlc_frame( + _frame([datetime(2026, 8, 13, 9, 28), datetime(2026, 8, 13, 9, 29)]), + now=datetime(2026, 8, 13, 10, 0, tzinfo=IST), + ) + + self.assertEqual( + stable_official_minutes( + frame, + request_started_at=datetime(2026, 8, 13, 9, 30, 7, tzinfo=IST), + grace_seconds=5.0, + ), + frozenset( + { + pd.Timestamp("2026-08-13 09:28:00"), + pd.Timestamp("2026-08-13 09:29:00"), + } + ), + ) + self.assertEqual( + stable_official_minutes( + pd.DataFrame(columns=["timestamp", "open", "high", "low", "close"]), + request_started_at=datetime(2026, 8, 13, 9, 30, 7), + grace_seconds=5.0, + ), + frozenset(), + ) + + class TestMarketHoursElapsed(unittest.TestCase): """Bar staleness is counted in MARKET time, not wall-clock time. diff --git a/Tests/test_nifty_multi_strategy_master.py b/Tests/test_nifty_multi_strategy_master.py index 51f2459..277c9b5 100644 --- a/Tests/test_nifty_multi_strategy_master.py +++ b/Tests/test_nifty_multi_strategy_master.py @@ -1380,6 +1380,32 @@ def test_rest_fetcher_marks_its_newest_candle_as_official(self): self.assertEqual(self.store.get("1").official_candle_ts, latest) + def test_rest_publication_does_not_certify_a_provisional_final_minute(self): + """A 09:29 request cannot claim its 09:29 REST row is final evidence.""" + + request_started_at = datetime(2026, 8, 13, 9, 29, 59) + self.broker.fetch_index_1m_ohlc.return_value = pd.DataFrame( + { + "timestamp": [pd.Timestamp("2026-08-13 09:28"), pd.Timestamp("2026-08-13 09:29")], + "open": [100.0, 101.0], + "high": [101.0, 102.0], + "low": [99.0, 100.0], + "close": [100.5, 101.5], + } + ) + self.broker.fetch_ltp_map.return_value = {} + self.stop_event.wait = MagicMock(side_effect=lambda _seconds: self.stop_event.set()) + + request_started_ist = request_started_at.replace( + tzinfo=master_file.IST_TIMEZONE + ) + with patch.object(master_file, "_ist_now", return_value=request_started_ist): + self.fetcher.run() + + snapshot = self.store.get("1") + self.assertEqual(snapshot.official_completed_minutes, frozenset({pd.Timestamp("2026-08-13 09:28")})) + self.assertEqual(snapshot.official_candle_ts, pd.Timestamp("2026-08-13 09:28")) + # ============================================================================= # TEST SUITE: MARKET DATA SOURCE SELECTOR @@ -1668,6 +1694,50 @@ def test_true_up_overwrites_completed_bar_keeps_forming(self): # ...while the forming minute keeps its tick-built values. self.assertEqual(by_ts.loc[forming]["close"], 100.6) + def test_true_up_before_grace_keeps_just_closed_minute_tick_owned(self): + """A reconnect before grace cannot let REST replace a forming tick bar.""" + + stable = pd.Timestamp("2026-05-15 10:15:00") + just_closed = pd.Timestamp("2026-05-15 10:16:00") + self.fetcher.aggregator.add_tick(just_closed, 100.6) + self.broker.fetch_index_1m_ohlc.return_value = pd.DataFrame( + { + "timestamp": [stable, just_closed], + "open": [99.8, 100.0], "high": [100.4, 101.0], + "low": [99.6, 99.5], "close": [100.1, 100.5], + } + ) + + self.fetcher._run_true_up("reconnect", now_ist=datetime(2026, 5, 15, 10, 17, 3)) + + snapshot = self.store.get("1") + self.assertEqual(snapshot.official_completed_minutes, frozenset({stable})) + self.assertEqual(snapshot.frame.set_index("timestamp").loc[just_closed, "close"], 100.6) + + def test_true_up_after_grace_uses_final_ohlc_and_prunes_only_stable_minutes(self): + """At 10:17:07 the 10:16 REST candle is final and may replace ticks.""" + + stable = pd.Timestamp("2026-05-15 10:15:00") + just_closed = pd.Timestamp("2026-05-15 10:16:00") + forming = pd.Timestamp("2026-05-15 10:17:00") + self.fetcher.aggregator.add_tick(just_closed, 100.6) + self.fetcher.aggregator.add_tick(forming, 100.8) + self.broker.fetch_index_1m_ohlc.return_value = pd.DataFrame( + { + "timestamp": [stable, just_closed, forming], + "open": [99.8, 100.0, 100.1], "high": [100.4, 101.0, 101.1], + "low": [99.6, 99.5, 99.7], "close": [100.1, 100.5, 100.2], + } + ) + + self.fetcher._run_true_up("minute-close", now_ist=datetime(2026, 5, 15, 10, 17, 7)) + + snapshot = self.store.get("1") + self.assertEqual(snapshot.official_completed_minutes, frozenset({stable, just_closed})) + self.assertEqual(snapshot.frame.set_index("timestamp").loc[just_closed, "close"], 100.5) + self.assertEqual(snapshot.frame.set_index("timestamp").loc[forming, "close"], 100.8) + self.assertEqual(self.fetcher.aggregator.tick_bars_frame()["timestamp"].tolist(), [forming]) + def test_true_up_prunes_trued_minutes_so_divergence_is_per_cycle(self): """Once official candles cover a minute, its tick bar must leave the aggregator -- otherwise every later true-up re-reports the same old @@ -2075,6 +2145,36 @@ def test_shared_store_snapshot_carries_an_optional_official_candle_watermark(sel store.update("1", frame, official_candle_ts=watermark) self.assertEqual(store.get("1").official_candle_ts, watermark) + def test_shared_store_publishes_frame_exact_official_set_and_watermark_together(self): + """CPR readers must receive one immutable official-data generation.""" + + frame = pd.DataFrame( + { + "timestamp": pd.date_range("2026-08-13 09:25", periods=5, freq="1min"), + "open": [100.0] * 5, + "high": [101.0] * 5, + "low": [99.0] * 5, + "close": [100.5] * 5, + } + ) + exact = frozenset( + { + pd.Timestamp("2026-08-13 09:25"), + pd.Timestamp("2026-08-13 09:27"), + pd.Timestamp("2026-08-13 09:29"), + } + ) + store = master_file.SharedMarketDataStore() + + snapshot = store.update("1", frame, official_completed_minutes=exact) + copied = store.get("1") + + self.assertEqual(snapshot.official_completed_minutes, exact) + self.assertEqual(copied.official_completed_minutes, exact) + self.assertEqual(copied.official_candle_ts, pd.Timestamp("2026-08-13 09:29")) + with self.assertRaises(AttributeError): + copied.official_completed_minutes.add(pd.Timestamp("2026-08-13 09:26")) + def test_ltp_snapshot(self): """LTPSnapshot identifies a leg and its latest price + fetched time.""" snap = master_file.LTPSnapshot( @@ -9971,7 +10071,10 @@ def test_run_waits_for_the_bucket_final_minute_to_be_official_before_inference(s worker.store.update( "1", minutes, - official_candle_ts=pd.Timestamp("2026-08-03 09:58:00"), + official_completed_minutes=frozenset( + pd.Timestamp("2026-08-03 09:55:00") + pd.Timedelta(minutes=offset) + for offset in range(4) + ), ) poll_count = 0 @@ -9984,7 +10087,10 @@ def advance_true_up_then_stop(): worker.store.update( "1", minutes, - official_candle_ts=pd.Timestamp("2026-08-03 09:59:00"), + official_completed_minutes=frozenset( + pd.Timestamp("2026-08-03 09:55:00") + pd.Timedelta(minutes=offset) + for offset in range(5) + ), ) return self.assertEqual(agent.decide.call_count, 1) @@ -10024,6 +10130,58 @@ def test_bucket_identity_is_stable_while_content_signature_detects_true_up(self) worker._completed_bar_signature(corrected), ) + def test_official_gate_requires_every_exact_source_minute_in_bucket(self): + """A final-minute watermark cannot hide an intermediate REST hole.""" + + worker, _agent, _logger = self._worker() + minutes = pd.DataFrame( + { + "timestamp": pd.date_range("2026-08-03 09:55", periods=5, freq="1min"), + "open": [100.0] * 5, + "high": [101.0] * 5, + "low": [99.0] * 5, + "close": [100.5] * 5, + } + ) + completed = pd.DataFrame( + [{"timestamp": pd.Timestamp("2026-08-03 09:55"), "open": 100.0, "high": 101.0, "low": 99.0, "close": 100.5}] + ) + missing_intermediate = frozenset( + pd.Timestamp("2026-08-03 09:55") + pd.Timedelta(minutes=offset) + for offset in (0, 1, 3, 4) + ) + worker.store.update("1", minutes, official_completed_minutes=missing_intermediate) + + self.assertFalse(worker._official_snapshot_covers_completed_bar(worker.store.get("1"), completed)) + + all_five = frozenset( + pd.Timestamp("2026-08-03 09:55") + pd.Timedelta(minutes=offset) + for offset in range(5) + ) + worker.store.update("1", minutes, official_completed_minutes=all_five) + self.assertTrue(worker._official_snapshot_covers_completed_bar(worker.store.get("1"), completed)) + + def test_later_official_correction_stays_stale_and_does_not_repeat_inference(self): + """One nominal bucket consumes one turn even when final OHLC is corrected.""" + + worker, agent, _logger = self._worker() + worker._latest_frozen_context = lambda: { + "session_levels": {"prior_accepted_regime": None}, + "momentum_vwap": {}, + "market_structure": {}, + "position_state": {"is_flat": True}, + } + original = pd.DataFrame( + [{"timestamp": pd.Timestamp("2026-08-03 09:55"), "open": 100.0, "high": 101.0, "low": 99.0, "close": 100.5}] + ) + corrected = original.copy(deep=True) + corrected.loc[0, "close"] = 100.75 + + worker.process_strategy_frame(original) + worker.process_strategy_frame(corrected) + + self.assertEqual(agent.decide.call_count, 1) + def test_final_entry_audit_uses_actual_broker_and_position_provenance(self): """Paper fallback and indeterminate live exposure cannot inherit the configured LIVE tag.""" From ef8e6b89599919f7c30cc81ae7acfda7cd111a17 Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 18:02:12 +0530 Subject: [PATCH 06/10] fix(cpr-ai): repair incomplete MCP evidence Co-authored-by: Codex --- .../CPR AI Agent/cpr_ai_agent.py | 240 +++++++++++- .../CPR AI Agent/cpr_ai_codex_runner.py | 9 +- .../CPR AI Agent/cpr_ai_codex_subprocess.py | 23 +- .../CPR AI Agent/test_cpr_ai_runtime.py | 366 +++++++++++++++++- 4 files changed, 617 insertions(+), 21 deletions(-) diff --git a/Signal Generators/CPR AI Agent/cpr_ai_agent.py b/Signal Generators/CPR AI Agent/cpr_ai_agent.py index 57109a2..c761305 100644 --- a/Signal Generators/CPR AI Agent/cpr_ai_agent.py +++ b/Signal Generators/CPR AI Agent/cpr_ai_agent.py @@ -19,6 +19,7 @@ from collections.abc import Callable, Mapping from concurrent.futures import ThreadPoolExecutor, TimeoutError from dataclasses import dataclass, field +from enum import StrEnum from time import monotonic from typing import Any @@ -30,6 +31,18 @@ _RETRIABLE_TOOL_EVIDENCE_CODES = frozenset({"missing_tool_call", "failed_tool_call"}) +class CPRTurnRequestKind(StrEnum): + """Name the only two fixed turn requests allowed across the child boundary. + + The parent controls this enum. In particular, no model response or caller + string can become a follow-up instruction, which keeps the corrective turn + limited to re-reading the four already-frozen facts. + """ + + NORMAL = "normal" + TOOL_REPAIR = "tool_repair" + + @dataclass(frozen=True) class CPRToolCallRecord: """One SDK-observed MCP read, retained only to prove tool coverage. @@ -57,6 +70,22 @@ class CPRAgentRunResult: unexpected_actions: tuple[str, ...] = () +@dataclass(frozen=True) +class CPRAttemptEvidence: + """Auditable result of one isolated model attempt within a bar deadline. + + This records only request mode, tool coverage, and numeric usage. It does + not keep chain-of-thought, returned MCP payloads, authentication data, or + any execution capability. + """ + + attempt_number: int + request_kind: CPRTurnRequestKind + evidence_code: str | None + tool_records: tuple[CPRToolCallRecord, ...] + token_usage: dict[str, int] + + @dataclass class CPRAgentOutcome: """Host-owned outcome separating advice from executable geometry. @@ -83,6 +112,7 @@ class CPRAgentOutcome: token_usage: dict[str, int] = field(default_factory=dict) tool_evidence: tuple[CPRToolCallRecord, ...] = () inference_attempts: int = 1 + attempt_evidence: tuple[CPRAttemptEvidence, ...] = () def _hold(code: str, reason: str, proposal: Any | None = None, *, regime: str | None = None) -> CPRAgentOutcome: @@ -442,14 +472,21 @@ def decide( started = monotonic() executor = ThreadPoolExecutor(max_workers=1) release_when_finished = False + # The parent may reach its wall-clock deadline while the child thread is + # still unwinding. This shared, append-only audit lets that fail-closed + # path retain every completed attempt rather than returning a blank HOLD. + attempt_evidence: list[CPRAttemptEvidence] = [] try: future = executor.submit( self._run_turn_with_tool_retry, context, bar_signature, + started + self.timeout_seconds, + current_signature, + attempt_evidence, ) try: - result, inference_attempts, combined_usage = future.result( + result, attempt_evidence, combined_usage, terminal_code = future.result( timeout=self.timeout_seconds ) except TimeoutError: @@ -459,26 +496,39 @@ def decide( future.cancel() release_when_finished = True future.add_done_callback(lambda _future: self._inference_lock.release()) - return _hold("timeout", "Codex did not finish inside the configured deadline.") + return self._hold_with_attempt_evidence("timeout", attempt_evidence) except Exception as error: # optional SDK failure must disable this agent only + # Timeout-class errors retain their conservative pre-launch audit. + # Other first-turn runtime errors keep the established generic + # runtime outcome and never expose an exception message. + if self._is_timeout_error(error): + return self._hold_with_attempt_evidence("timeout", attempt_evidence) return _hold("runtime_error", f"Optional Codex runtime failed: {type(error).__name__}.") finally: executor.shutdown(wait=False, cancel_futures=True) if not release_when_finished: self._inference_lock.release() latency_ms = int((monotonic() - started) * 1000) - outcome = self._validate_run(result, context, bar_signature, current_signature) + outcome = ( + _hold(terminal_code, self._terminal_failure_reason(terminal_code)) + if terminal_code is not None + else self._validate_run(result, context, bar_signature, current_signature) + ) outcome.latency_ms = latency_ms outcome.token_usage = combined_usage outcome.tool_evidence = result.tool_calls - outcome.inference_attempts = inference_attempts + outcome.inference_attempts = len(attempt_evidence) + outcome.attempt_evidence = attempt_evidence return outcome def _run_turn_with_tool_retry( self, context: Mapping[str, Any], bar_signature: str, - ) -> tuple[CPRAgentRunResult, int, dict[str, int]]: + deadline: float, + current_signature: Callable[[], str] | None, + attempt_evidence: list[CPRAttemptEvidence], + ) -> tuple[CPRAgentRunResult, tuple[CPRAttemptEvidence, ...], dict[str, int], str | None]: """Retry incomplete frozen-tool evidence once inside one total deadline. Both attempts receive the same in-memory context and bar signature. A @@ -487,7 +537,6 @@ def _run_turn_with_tool_retry( so this recovery path never doubles the configured SDK timeout. """ - deadline = monotonic() + self.timeout_seconds results: list[CPRAgentRunResult] = [] for attempt_index in range(2): if attempt_index == 0: @@ -498,17 +547,78 @@ def _run_turn_with_tool_retry( else: attempt_timeout = deadline - monotonic() if attempt_timeout <= 0: - break - result = self._run_turn( - context, - bar_signature, - timeout_seconds=attempt_timeout, + # The repair was selected but no wall-clock budget remains. + # Record that fact explicitly without inventing a tool call + # or token count for a child process that never launched. + attempt_evidence.append( + CPRAttemptEvidence( + attempt_number=2, + request_kind=CPRTurnRequestKind.TOOL_REPAIR, + evidence_code="timeout", + tool_records=(), + token_usage={}, + ) + ) + return results[-1], tuple(attempt_evidence), self._combined_token_usage(results), "timeout" + request_kind = ( + CPRTurnRequestKind.NORMAL + if attempt_index == 0 + else CPRTurnRequestKind.TOOL_REPAIR + ) + # Install a conservative no-evidence timeout diagnostic before the + # optional runtime starts. If the parent deadline wins the race, + # this proves which turn began while never inventing tool calls or + # usage that the child did not return. + diagnostic_index = len(attempt_evidence) + attempt_evidence.append( + CPRAttemptEvidence( + attempt_number=attempt_index + 1, + request_kind=request_kind, + evidence_code="timeout", + tool_records=(), + token_usage={}, + ) ) + try: + result = self._run_turn( + context, + bar_signature, + request_kind=request_kind, + timeout_seconds=attempt_timeout, + ) + except Exception as error: + # Preserve the usual first-turn exception behavior. Once the + # normal result proved a repair was needed, however, a failed + # repair must retain that completed first audit trail. + if attempt_index == 0: + raise + terminal_code = "timeout" if self._is_timeout_error(error) else "runtime_error" + attempt_evidence[diagnostic_index] = ( + CPRAttemptEvidence( + attempt_number=2, + request_kind=CPRTurnRequestKind.TOOL_REPAIR, + evidence_code=terminal_code, + tool_records=(), + token_usage={}, + ) + ) + return results[-1], tuple(attempt_evidence), self._combined_token_usage(results), terminal_code results.append(result) evidence_error = self._tool_evidence_error(result) - if ( - evidence_error is None - or evidence_error[0] not in _RETRIABLE_TOOL_EVIDENCE_CODES + completed_evidence = CPRAttemptEvidence( + attempt_number=attempt_index + 1, + request_kind=request_kind, + evidence_code=None if evidence_error is None else evidence_error[0], + tool_records=result.tool_calls, + token_usage=dict(result.token_usage), + ) + attempt_evidence[diagnostic_index] = completed_evidence + if not self._retry_is_allowed( + result, + context, + bar_signature, + current_signature, + evidence_error, ): break @@ -516,7 +626,95 @@ def _run_turn_with_tool_retry( # this list cannot be empty. Keeping the assertion documents that local # invariant without converting an SDK exception into trusted evidence. assert results - return results[-1], len(results), self._combined_token_usage(results) + return results[-1], tuple(attempt_evidence), self._combined_token_usage(results), None + + @staticmethod + def _is_timeout_error(error: Exception) -> bool: + """Recognize local deadline exceptions without retaining their details. + + The optional adapter may surface either the standard library's + ``TimeoutExpired`` or the executor's ``TimeoutError``. The host stores + only the safe classification, never a potentially sensitive message. + """ + + return isinstance(error, TimeoutError) or type(error).__name__ == "TimeoutExpired" + + @staticmethod + def _terminal_failure_reason(code: str) -> str: + """Return a safe generic reason for a repair failure audit outcome.""" + + if code == "timeout": + return "The corrective Codex turn exhausted the original deadline." + return "The optional corrective Codex runtime failed." + + def _hold_with_attempt_evidence( + self, + code: str, + attempt_evidence: list[CPRAttemptEvidence], + ) -> CPRAgentOutcome: + """Build a terminal HOLD while retaining only safe completed-attempt data.""" + + recorded_attempts = tuple(attempt_evidence) + outcome = _hold(code, self._terminal_failure_reason(code)) + outcome.inference_attempts = len(recorded_attempts) + outcome.attempt_evidence = recorded_attempts + outcome.token_usage = self._combined_attempt_token_usage(recorded_attempts) + outcome.tool_evidence = next( + (record.tool_records for record in reversed(recorded_attempts) if record.tool_records), + (), + ) + return outcome + + @staticmethod + def _combined_attempt_token_usage( + attempts: tuple[CPRAttemptEvidence, ...], + ) -> dict[str, int]: + """Combine retained attempt counters with the same context-window rule.""" + + combined: dict[str, int] = {} + for attempt in attempts: + for key, value in attempt.token_usage.items(): + if key == "model_context_window": + combined[key] = max(combined.get(key, 0), int(value)) + else: + combined[key] = combined.get(key, 0) + int(value) + return combined + + def _retry_is_allowed( + self, + result: CPRAgentRunResult, + context: Mapping[str, Any], + bar_signature: str, + current_signature: Callable[[], str] | None, + evidence_error: tuple[str, str] | None, + ) -> bool: + """Allow the one repair only for otherwise-valid missing/failed reads. + + A repair exists to recover an incomplete observation of immutable MCP + facts. It must not hide a stale bar, bad schema/model/prompt echo, or + deterministic host-policy rejection behind a second model attempt. + """ + + if evidence_error is None or evidence_error[0] not in _RETRIABLE_TOOL_EVIDENCE_CODES: + return False + if current_signature is not None and current_signature() != bar_signature: + return False + try: + from cpr_ai_schema import CPRAgentDecision + + proposal = CPRAgentDecision.model_validate_json(result.final_response) + except Exception: + return False + if proposal.model_used != self.model: + return False + expected_prompt = self.prompt_version + if expected_prompt is None: + from cpr_ai_prompt import CPR_AI_PROMPT_VERSION + + expected_prompt = CPR_AI_PROMPT_VERSION + if proposal.prompt_version != expected_prompt: + return False + return self.policy.validate(context, proposal).accepted @staticmethod def _combined_token_usage( @@ -543,6 +741,7 @@ def _run_turn( context: Mapping[str, Any], bar_signature: str, *, + request_kind: CPRTurnRequestKind = CPRTurnRequestKind.NORMAL, timeout_seconds: float | None = None, ) -> CPRAgentRunResult: """Build prompt/schema lazily and pass only advisory inputs to the child. @@ -566,6 +765,7 @@ def _run_turn( reasoning_effort=self.reasoning_effort, prompt_version=self.prompt_version or CPR_AI_PROMPT_VERSION, output_schema=CPRAgentDecision.model_json_schema(), + request_kind=request_kind, # Direct diagnostic callers historically used this helper without # an explicit deadline. Normal and retry paths pass their exact # per-attempt budget; the fallback preserves that diagnostic API. @@ -647,4 +847,12 @@ def _tool_evidence_error(result: CPRAgentRunResult) -> tuple[str, str] | None: return None -__all__ = ["CPRAgent", "CPRAgentOutcome", "CPRAgentRunResult", "CPRHostPolicy", "CPRToolCallRecord"] +__all__ = [ + "CPRAgent", + "CPRAgentOutcome", + "CPRAgentRunResult", + "CPRAttemptEvidence", + "CPRHostPolicy", + "CPRToolCallRecord", + "CPRTurnRequestKind", +] diff --git a/Signal Generators/CPR AI Agent/cpr_ai_codex_runner.py b/Signal Generators/CPR AI Agent/cpr_ai_codex_runner.py index 38f7fa6..00d9609 100644 --- a/Signal Generators/CPR AI Agent/cpr_ai_codex_runner.py +++ b/Signal Generators/CPR AI Agent/cpr_ai_codex_runner.py @@ -29,7 +29,7 @@ from pathlib import Path from typing import Any -from cpr_ai_agent import CPRAgentRunResult, CPRToolCallRecord +from cpr_ai_agent import CPRAgentRunResult, CPRToolCallRecord, CPRTurnRequestKind from cpr_ai_codex_subprocess import build_isolated_thread_config _PROCESS_CODEX_HOME_LOCK = threading.Lock() @@ -171,6 +171,9 @@ def run_codex_turn(**kwargs: Any) -> CPRAgentRunResult: context = kwargs.get("context") if not isinstance(context, Mapping): raise ValueError("Codex turn requires a frozen CPR context mapping.") + request_kind = kwargs.get("request_kind", CPRTurnRequestKind.NORMAL) + if not isinstance(request_kind, CPRTurnRequestKind): + raise ValueError("Codex turn request kind must be a CPRTurnRequestKind enum value.") timeout_seconds = float(kwargs.get("timeout_seconds", 90.0)) if not math.isfinite(timeout_seconds) or timeout_seconds <= 0.0: raise ValueError("Codex subprocess timeout must be a positive finite number.") @@ -198,6 +201,10 @@ def run_codex_turn(**kwargs: Any) -> CPRAgentRunResult: "reasoning_effort": kwargs.get("reasoning_effort"), "prompt": kwargs.get("prompt"), "output_schema": kwargs.get("output_schema"), + # This enum is the only parent-to-child authority for selecting a + # turn request. The child maps it to one constant; arbitrary text + # can never become a repair prompt. + "request_kind": request_kind.value, } completed = subprocess.run( [sys.executable, str(script)], diff --git a/Signal Generators/CPR AI Agent/cpr_ai_codex_subprocess.py b/Signal Generators/CPR AI Agent/cpr_ai_codex_subprocess.py index ae4cd1d..ae11ce4 100644 --- a/Signal Generators/CPR AI Agent/cpr_ai_codex_subprocess.py +++ b/Signal Generators/CPR AI Agent/cpr_ai_codex_subprocess.py @@ -19,7 +19,7 @@ from pathlib import Path from typing import Any -_REQUEST_KEYS = {"snapshot_path", "model", "reasoning_effort", "prompt", "output_schema"} +_REQUEST_KEYS = {"snapshot_path", "model", "reasoning_effort", "prompt", "output_schema", "request_kind"} _EXPECTED_TOOLS = ("session_levels", "momentum_vwap", "market_structure", "position_state") # ``build_system_prompt()`` remains the durable policy authority. Repeating the # four reads in the immediate turn request is intentional defense in depth: a @@ -30,6 +30,16 @@ "momentum_vwap, market_structure, and position_state. Wait for all four " "calls to complete, then evaluate the frozen CPR context and return one decision." ) +# This deliberately constant repair request is selected solely by the parent +# enum. It supplies feedback about evidence rejection without allowing either +# a caller or the first model response to inject a new instruction. +_TOOL_REPAIR_TURN_REQUEST = ( + "The prior attempt was rejected because required frozen-tool evidence was missing or failed. " + "Before returning JSON, call each frozen MCP tool exactly once: session_levels, momentum_vwap, " + "market_structure, and position_state. Wait for all four calls to complete, then evaluate the same frozen " + "CPR context and return one decision." +) +_TURN_REQUESTS = {"normal": _TURN_REQUEST, "tool_repair": _TOOL_REPAIR_TURN_REQUEST} _ALLOWED_TURN_ITEM_TYPES = frozenset( { # The SDK records the prompt submitted through ``Thread.run`` as a @@ -166,6 +176,9 @@ def _run_request(request: Mapping[str, Any]) -> dict[str, Any]: from openai_codex import ApprovalMode, Codex, Sandbox + request_kind = request.get("request_kind") + if not isinstance(request_kind, str) or request_kind not in _TURN_REQUESTS: + raise ValueError("Invalid isolated Codex request kind.") snapshot_path = str(request["snapshot_path"]) runtime_directory = str(Path(snapshot_path).parent) config = build_isolated_thread_config(snapshot_path) @@ -183,7 +196,7 @@ def _run_request(request: Mapping[str, Any]) -> dict[str, Any]: approval_mode=ApprovalMode.deny_all, ) result = thread.run( - _TURN_REQUEST, + _TURN_REQUESTS[request_kind], approval_mode=ApprovalMode.deny_all, output_schema=request["output_schema"], effort=request["reasoning_effort"], @@ -213,7 +226,11 @@ def main() -> int: try: request = json.load(sys.stdin) - if not isinstance(request, Mapping) or set(request) != _REQUEST_KEYS: + if ( + not isinstance(request, Mapping) + or set(request) != _REQUEST_KEYS + or request.get("request_kind") not in _TURN_REQUESTS + ): raise ValueError("Invalid isolated Codex request.") response = _run_request(request) except (ImportError, KeyError, TypeError, ValueError, RuntimeError) as error: diff --git a/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py b/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py index a4ef6e4..7c7c85e 100644 --- a/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py +++ b/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py @@ -13,6 +13,7 @@ from __future__ import annotations +import itertools import json import subprocess # nosec B404 import sys @@ -22,11 +23,18 @@ from pathlib import Path from types import SimpleNamespace +import cpr_ai_agent import cpr_ai_codex_runner as codex_runner import cpr_ai_codex_subprocess as codex_child import cpr_ai_mcp_server as mcp_server import pytest -from cpr_ai_agent import CPRAgent, CPRAgentRunResult, CPRHostPolicy, CPRToolCallRecord +from cpr_ai_agent import ( + CPRAgent, + CPRAgentRunResult, + CPRHostPolicy, + CPRToolCallRecord, + CPRTurnRequestKind, +) from cpr_ai_codex_runner import build_codex_thread_config, safe_subprocess_environment from cpr_ai_decision_log import CPRDecisionLogger from cpr_ai_prompt import CPR_AI_PROMPT_VERSION @@ -204,7 +212,219 @@ def runner(**kwargs): assert attempts[1]["context"] is frozen assert attempts[0]["bar_signature"] == "same-frozen-bar" assert attempts[1]["bar_signature"] == "same-frozen-bar" + assert attempts[0]["request_kind"] is CPRTurnRequestKind.NORMAL + assert attempts[1]["request_kind"] is CPRTurnRequestKind.TOOL_REPAIR assert 0 < attempts[1]["timeout_seconds"] <= attempts[0]["timeout_seconds"] <= 17.5 + assert [(item.attempt_number, item.request_kind, item.evidence_code) for item in outcome.attempt_evidence] == [ + (1, "normal", "missing_tool_call" if not first_attempt_calls else "failed_tool_call"), + (2, "tool_repair", None), + ] + + +def test_agent_keeps_frozen_context_signature_and_remaining_deadline_for_failed_tool_repair(): + """A failed frozen read gets one repair turn against precisely the original bar.""" + + attempts = [] + + def runner(**kwargs): + attempts.append(kwargs) + return CPRAgentRunResult( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + tool_calls=_calls(failed="position_state") if len(attempts) == 1 else _calls(), + token_usage={"input_tokens": 7, "total_tokens": 11, "model_context_window": 128000}, + ) + + frozen = _context() + outcome = CPRAgent(runner=runner, timeout_seconds=13.0).decide(frozen, bar_signature="failed-read-bar") + + assert outcome.validation_code == "accepted_hold" + assert [attempt["request_kind"] for attempt in attempts] == [ + CPRTurnRequestKind.NORMAL, + CPRTurnRequestKind.TOOL_REPAIR, + ] + assert all(attempt["context"] is frozen for attempt in attempts) + assert [attempt["bar_signature"] for attempt in attempts] == ["failed-read-bar", "failed-read-bar"] + assert 0 < attempts[1]["timeout_seconds"] <= attempts[0]["timeout_seconds"] <= 13.0 + assert outcome.token_usage == {"input_tokens": 14, "total_tokens": 22, "model_context_window": 128000} + assert [item.tool_records for item in outcome.attempt_evidence] == [ + _calls(failed="position_state"), + _calls(), + ] + + +def test_repair_runtime_error_preserves_completed_first_attempt_audit_without_exception_text(): + """A second-turn adapter failure must not erase the first rejected evidence.""" + + calls = [] + + def runner(**kwargs): + calls.append(kwargs) + if len(calls) == 2: + raise RuntimeError("operator secret must never reach an audit row") + return CPRAgentRunResult( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + tool_calls=_calls(missing="market_structure"), + token_usage={"input_tokens": 5, "total_tokens": 8, "model_context_window": 128000}, + ) + + outcome = CPRAgent(runner=runner).decide(_context(), bar_signature="repair-runtime-error") + + assert outcome.action == "HOLD" + assert outcome.validation_code == "runtime_error" + assert "secret" not in outcome.validation_reason + assert outcome.inference_attempts == 2 + assert outcome.tool_evidence == _calls(missing="market_structure") + assert outcome.token_usage == {"input_tokens": 5, "total_tokens": 8, "model_context_window": 128000} + recorded_attempts = [ + (item.request_kind, item.evidence_code, item.tool_records, item.token_usage) + for item in outcome.attempt_evidence + ] + assert recorded_attempts == [ + (CPRTurnRequestKind.NORMAL, "missing_tool_call", _calls(missing="market_structure"), outcome.token_usage), + (CPRTurnRequestKind.TOOL_REPAIR, "runtime_error", (), {}), + ] + assert len(calls) == 2 + + +def test_repair_deadline_exhaustion_preserves_completed_first_attempt_audit(monkeypatch): + """A spent total deadline records a second repair timeout without inventing evidence.""" + + clock = itertools.chain((100.0, 105.0), itertools.repeat(105.0)) + monkeypatch.setattr(cpr_ai_agent, "monotonic", lambda: next(clock)) + calls = [] + + def runner(**kwargs): + calls.append(kwargs) + return CPRAgentRunResult( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + tool_calls=_calls(failed="position_state"), + token_usage={"total_tokens": 13}, + ) + + outcome = CPRAgent(runner=runner, timeout_seconds=5.0).decide(_context(), bar_signature="repair-deadline") + + assert outcome.action == "HOLD" + assert outcome.validation_code == "timeout" + assert outcome.inference_attempts == 2 + assert outcome.tool_evidence == _calls(failed="position_state") + assert outcome.token_usage == {"total_tokens": 13} + recorded_attempts = [ + (item.request_kind, item.evidence_code, item.tool_records, item.token_usage) + for item in outcome.attempt_evidence + ] + assert recorded_attempts == [ + (CPRTurnRequestKind.NORMAL, "failed_tool_call", _calls(failed="position_state"), {"total_tokens": 13}), + (CPRTurnRequestKind.TOOL_REPAIR, "timeout", (), {}), + ] + assert len(calls) == 1 + + +def test_outer_normal_deadline_timeout_records_started_normal_attempt_without_invented_evidence(): + """The parent deadline must retain a normal-turn diagnostic while its thread unwinds.""" + + release = threading.Event() + started = threading.Event() + + def runner(**_kwargs): + started.set() + release.wait(timeout=0.3) + return CPRAgentRunResult( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + tool_calls=_calls(), + token_usage={"total_tokens": 99}, + ) + + try: + outcome = CPRAgent(runner=runner, timeout_seconds=0.01).decide( + _context(), bar_signature="normal-outer-timeout" + ) + finally: + release.set() + + assert started.is_set() + assert outcome.action == "HOLD" + assert outcome.validation_code == "timeout" + assert outcome.inference_attempts == 1 + recorded_attempts = [ + (item.request_kind, item.evidence_code, item.tool_records, item.token_usage) + for item in outcome.attempt_evidence + ] + assert recorded_attempts == [(CPRTurnRequestKind.NORMAL, "timeout", (), {})] + assert outcome.tool_evidence == () + assert outcome.token_usage == {} + + +def test_normal_runner_timeout_error_records_typed_timeout_without_raw_exception_text(): + """A child timeout before its first response stays fail-closed and auditable.""" + + def runner(**_kwargs): + raise subprocess.TimeoutExpired(cmd="private child command", timeout=1.0) + + outcome = CPRAgent(runner=runner).decide(_context(), bar_signature="normal-inner-timeout") + + assert outcome.action == "HOLD" + assert outcome.validation_code == "timeout" + assert "private" not in outcome.validation_reason + assert outcome.inference_attempts == 1 + recorded_attempts = [ + (item.request_kind, item.evidence_code, item.tool_records, item.token_usage) + for item in outcome.attempt_evidence + ] + assert recorded_attempts == [(CPRTurnRequestKind.NORMAL, "timeout", (), {})] + assert outcome.tool_evidence == () + assert outcome.token_usage == {} + + +@pytest.mark.parametrize("first_calls", [_calls(missing="market_structure"), _calls(failed="position_state")]) +def test_second_incomplete_tool_repair_remains_hold_after_exactly_two_attempts(first_calls): + """A repair may not cascade into a third inference or a permissive result.""" + + attempts = [] + + def runner(**kwargs): + attempts.append(kwargs) + return CPRAgentRunResult( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + tool_calls=first_calls, + ) + + outcome = CPRAgent(runner=runner).decide(_context(), bar_signature="second-failure") + + assert outcome.action == "HOLD" + assert outcome.validation_code in {"missing_tool_call", "failed_tool_call"} + assert outcome.inference_attempts == 2 + assert len(attempts) == 2 + assert [item.request_kind for item in outcome.attempt_evidence] == ["normal", "tool_repair"] + + +@pytest.mark.parametrize( + "defective_calls", + [ + (*_calls(), CPRToolCallRecord(tool="session_levels", status="completed")), + _calls(unexpected=True), + ], + ids=["duplicate-tool", "unexpected-capability"], +) +def test_duplicate_or_unexpected_evidence_never_starts_corrective_retry(defective_calls): + """Capability-contract violations are final HOLDs, not repairable omissions.""" + + attempts = [] + + def runner(**kwargs): + attempts.append(kwargs) + return CPRAgentRunResult( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + tool_calls=defective_calls, + ) + + outcome = CPRAgent(runner=runner).decide(_context(), bar_signature="capability-violation") + + assert outcome.validation_code == "unexpected_agent_action" + assert len(attempts) == 1 + assert outcome.inference_attempts == 1 + assert [(item.request_kind, item.evidence_code) for item in outcome.attempt_evidence] == [ + ("normal", "unexpected_agent_action") + ] def test_runtime_configuration_is_read_only_and_sanitizes_credentials(tmp_path): @@ -416,6 +636,62 @@ def fake_run(command, **kwargs): assert tuple(call.tool for call in result.tool_calls) == EXPECTED_TOOL_NAMES +def test_parent_serializes_fixed_normal_or_tool_repair_kind_without_arbitrary_request_text(monkeypatch, tmp_path): + """The parent can select only the enum-backed repair mode for the child boundary.""" + + requests = [] + + def fake_run(command, **kwargs): + requests.append(json.loads(kwargs["input"])) + return subprocess.CompletedProcess( + command, + 0, + stdout=json.dumps( + { + "ok": True, + "final_response": _proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + "tool_calls": [{"tool": name, "status": "completed"} for name in EXPECTED_TOOL_NAMES], + "token_usage": {}, + "unexpected_actions": [], + } + ), + stderr="", + ) + + isolated_home = tmp_path / "codex-home" + isolated_home.mkdir() + (isolated_home / "auth.json").write_text("{}", encoding="utf-8") + monkeypatch.setattr(codex_runner.subprocess, "run", fake_run) + monkeypatch.setattr(codex_runner, "process_isolated_codex_home", lambda: isolated_home) + + codex_runner.run_codex_turn( + context=_context(), + prompt="unchanged prompt", + model="gpt-5.6-terra", + reasoning_effort="medium", + output_schema={}, + request_kind=CPRTurnRequestKind.TOOL_REPAIR, + ) + + assert requests[0]["request_kind"] == "tool_repair" + assert set(requests[0]) == { + "snapshot_path", + "model", + "reasoning_effort", + "prompt", + "output_schema", + "request_kind", + } + + +@pytest.mark.parametrize("invalid_kind", ["repair with these words", "unknown", 1, None]) +def test_parent_rejects_non_enum_request_kind_before_child_launch(invalid_kind): + """Caller/model text must never become a child repair instruction.""" + + with pytest.raises(ValueError, match="request kind"): + codex_runner.run_codex_turn(context=_context(), request_kind=invalid_kind) + + @pytest.mark.parametrize("configured_timeout", [0.25, 135.0]) def test_real_runner_uses_the_configured_subprocess_timeout(configured_timeout, monkeypatch, tmp_path): """The child deadline follows the validated agent setting at lower and upper values.""" @@ -802,6 +1078,7 @@ def thread_start(self, **kwargs): "reasoning_effort": "medium", "prompt": "prompt", "output_schema": {"type": "object"}, + "request_kind": "normal", } response = codex_child._run_request(request) @@ -845,3 +1122,90 @@ def thread_start(self, **kwargs): ) ) assert host.decide(_context(), bar_signature="enum-status").validation_code == "accepted_hold" + + +def test_child_repair_kind_uses_constant_corrective_request_with_unchanged_read_only_config(monkeypatch, tmp_path): + """The repair is a new read-only thread, not model-supplied corrective text.""" + + observed = {} + + class Thread: + def run(self, prompt, *, approval_mode, output_schema, effort): + observed["run"] = (prompt, approval_mode, output_schema, effort) + return SimpleNamespace( + final_response=_proposal("HOLD", "UNDECIDED", "NONE").model_dump_json(), + items=[ + SimpleNamespace(root=SimpleNamespace(type="mcpToolCall", tool=name, status="completed")) + for name in EXPECTED_TOOL_NAMES + ], + usage={}, + ) + + class Codex: + def __enter__(self): + return self + + def __exit__(self, *_args): + return None + + def thread_start(self, **kwargs): + observed["start"] = kwargs + return Thread() + + monkeypatch.setitem( + sys.modules, + "openai_codex", + SimpleNamespace( + Codex=Codex, + Sandbox=SimpleNamespace(read_only="read"), + ApprovalMode=SimpleNamespace(deny_all="deny"), + ), + ) + + response = codex_child._run_request( + { + "snapshot_path": str(tmp_path / "snapshot.json"), + "model": "gpt-5.6-terra", + "reasoning_effort": "medium", + "prompt": "same system prompt", + "output_schema": {"type": "object"}, + "request_kind": "tool_repair", + } + ) + + assert observed["run"][0] == codex_child._TOOL_REPAIR_TURN_REQUEST + assert "prior attempt" in observed["run"][0].lower() + assert all(observed["run"][0].count(name) == 1 for name in EXPECTED_TOOL_NAMES) + assert observed["start"]["developer_instructions"] == "same system prompt" + assert observed["start"]["config"] == codex_child.build_isolated_thread_config(str(tmp_path / "snapshot.json")) + assert observed["start"]["config"]["mcp_servers"]["cpr_ai"]["enabled_tools"] == list(EXPECTED_TOOL_NAMES) + assert response["unexpected_actions"] == [] + + +@pytest.mark.parametrize( + "request_kind", + [ + pytest.param(None, id="missing"), + pytest.param("unknown", id="unknown"), + pytest.param("arbitrary repair", id="arbitrary"), + ], +) +def test_child_request_shape_requires_known_request_kind(request_kind, monkeypatch): + """The JSON boundary rejects absent/unknown kinds before the SDK import.""" + + request = { + "snapshot_path": "snapshot.json", + "model": "gpt-5.6-terra", + "reasoning_effort": "medium", + "prompt": "prompt", + "output_schema": {}, + } + if request_kind is not None: + request["request_kind"] = request_kind + monkeypatch.setattr(codex_child, "_run_request", lambda _request: pytest.fail("SDK path must not run")) + monkeypatch.setattr(sys, "stdin", SimpleNamespace(read=lambda: json.dumps(request))) + captured = [] + monkeypatch.setattr(codex_child.json, "dump", lambda payload, _stream: captured.append(payload)) + + assert codex_child.main() == 2 + assert captured == [{"ok": False, "error": "ValueError"}] From 00e5040a14bc73f3417c7f4f0cc413cffe1973d4 Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 18:42:22 +0530 Subject: [PATCH 07/10] feat(cpr-ai): add decision audit diagnostics Co-authored-by: Codex --- ...Multi Strategy Front Test - Master File.py | 84 +++- Signal Generators/CPR AI Agent/README.md | 45 ++- .../CPR AI Agent/cpr_ai_agent.py | 99 ++++- .../CPR AI Agent/cpr_ai_decision_log.py | 90 ++++- .../CPR AI Agent/test_cpr_ai_runtime.py | 368 ++++++++++++++++++ Tests/test_nifty_multi_strategy_master.py | 44 +++ 6 files changed, 695 insertions(+), 35 deletions(-) diff --git a/Nifty Multi Strategy Front Test - Master File.py b/Nifty Multi Strategy Front Test - Master File.py index 610e4dc..585a7ab 100644 --- a/Nifty Multi Strategy Front Test - Master File.py +++ b/Nifty Multi Strategy Front Test - Master File.py @@ -9758,6 +9758,59 @@ def _official_snapshot_covers_completed_bar( ) return required_minutes.issubset(official_minutes) + def _completed_bar_audit_metadata( + self, + snapshot: MarketSnapshot | None, + strategy_frame: pd.DataFrame, + frozen_signature: str, + ) -> dict[str, object]: + """Capture exact official-minute coverage once for both audit records. + + The worker already proved this coverage before inference in ``run``. + Keeping the same small, JSON-ready mapping through pre- and post-action + logging makes the two rows comparable without re-reading a market store + that may have advanced while Codex or execution was running. Direct + unit/diagnostic callers may not have a snapshot; they receive an empty, + explicitly false coverage record rather than invented evidence. + """ + + bucket_start = ( + None + if strategy_frame.empty or "timestamp" not in strategy_frame.columns + else self._naive_ist_timestamp(strategy_frame.iloc[-1]["timestamp"]) + ) + required_minutes = frozenset( + bucket_start + pd.Timedelta(minutes=offset) + for offset in range(5) + ) if bucket_start is not None else frozenset() + official_minutes = frozenset( + timestamp + for value in ( + () if snapshot is None else snapshot.official_completed_minutes + ) + if (timestamp := self._naive_ist_timestamp(value)) is not None + ) + + def ist_iso(timestamp: pd.Timestamp) -> str: + """Represent internal naive-IST timestamps unambiguously in JSONL.""" + + return timestamp.tz_localize(IST_TIMEZONE).isoformat() + + return { + "bar_timestamp": None if bucket_start is None else ist_iso(bucket_start), + "frozen_signature": frozen_signature, + "required_official_minutes": [ + ist_iso(timestamp) for timestamp in sorted(required_minutes) + ], + "present_official_minutes": [ + ist_iso(timestamp) + for timestamp in sorted(required_minutes & official_minutes) + ], + "official_coverage": bool(required_minutes) and required_minutes.issubset( + official_minutes + ), + } + def _position_state_payload(self) -> dict[str, object]: """Expose allowlisted premise/risk facts, never execution capabilities. @@ -10436,6 +10489,7 @@ def _write_final_execution( frozen_context: dict[str, object], outcome, execution: dict[str, object], + bar_metadata: dict[str, object], ) -> None: """Best-effort append actual post-action provenance to the decision log. @@ -10456,6 +10510,8 @@ def _write_final_execution( token_usage=outcome.token_usage, tool_evidence=self._tool_log_payload(outcome.tool_evidence), execution=execution, + audit_stage="POST_ACTION", + bar_metadata=bar_metadata, ) except Exception as exc: # noqa: BLE001 - the pre-action audit still exists self.log.error( @@ -10487,7 +10543,12 @@ def _post_inference_exposure_block_reason(self) -> str: return "entry_cutoff" return "" - def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: + def process_strategy_frame( + self, + strategy_frame: pd.DataFrame, + *, + audit_metadata: dict[str, object] | None = None, + ) -> None: """Evaluate one completed bucket through mechanics, Codex, and host gates. Flat workers skip new turns after 15:00; open workers continue for @@ -10512,6 +10573,13 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: bar_signature = self._completed_bar_signature(strategy_frame) if not bar_signature: return + if audit_metadata is None: + # Tests and direct diagnostics can invoke this method outside the + # normal ``run`` loop. Capture a conservative local snapshot once + # so every row still has explicit, non-invented coverage facts. + audit_metadata = self._completed_bar_audit_metadata( + self.store.get(self.timeframe), strategy_frame, bar_signature + ) frozen_context = self._latest_frozen_context() if self.pos.active and self._manage_completed_bar(frozen_context): return @@ -10560,6 +10628,8 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: else "AUDITED_BEFORE_EXECUTION" ), }, + audit_stage="PRE_ACTION", + bar_metadata=audit_metadata, ) except Exception as exc: # noqa: BLE001 - entries fail closed; exits continue audit_ok = False @@ -10603,6 +10673,7 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: "submitted": True, "status": "EXIT_CONFIRMED" if not self.pos.active else "EXIT_UNCONFIRMED", }, + audit_metadata, ) return if outcome.action == "SCALE_IN" and audit_ok: @@ -10617,6 +10688,7 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: "status": "SCALE_IN_BLOCKED", "blocked_reason": blocked_reason, }, + audit_metadata, ) # A closed exposure gate prevents the add immediately. The # normal safety pass also performs any associated lifecycle, @@ -10644,6 +10716,7 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: else "SCALE_IN_UNCONFIRMED" ), }, + audit_metadata, ) return if not audit_ok or outcome.action not in {"ENTER_LONG", "ENTER_SHORT"}: @@ -10662,6 +10735,7 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: "status": "ENTRY_BLOCKED", "blocked_reason": blocked_reason, }, + audit_metadata, ) return direction = "LONG" if outcome.action == "ENTER_LONG" else "SHORT" @@ -10688,6 +10762,7 @@ def process_strategy_frame(self, strategy_frame: pd.DataFrame) -> None: else "ENTRY_BLOCKED" ), }, + audit_metadata, ) def run(self) -> None: @@ -10740,7 +10815,12 @@ def run(self) -> None: self.wait_for_next_poll() continue self._waiting_for_official_bar_identity = None - self.process_strategy_frame(completed) + audit_metadata = self._completed_bar_audit_metadata( + snapshot, + completed, + self._completed_bar_signature(completed), + ) + self.process_strategy_frame(completed, audit_metadata=audit_metadata) except Exception as exc: # noqa: BLE001 - one turn must not kill safety self.log.exception("CPR AI worker poll failed: %s", exc) self.wait_for_next_poll() diff --git a/Signal Generators/CPR AI Agent/README.md b/Signal Generators/CPR AI Agent/README.md index bc12487..5126997 100644 --- a/Signal Generators/CPR AI Agent/README.md +++ b/Signal Generators/CPR AI Agent/README.md @@ -79,13 +79,16 @@ host then enforces the selected framework: each newly completed five-minute candle. A start-stamped one-minute candle is not complete until the next minute begins, and all five exact minute slots must exist once. -- In websocket mode, clock completeness alone is not enough. Every shared OHLC - snapshot carries an atomic `official_candle_ts` watermark. A 09:55 five-minute - bucket waits until the REST source covers its final 09:59 one-minute candle, - even when Dhan's true-up takes longer than its normal five-second delay. This - is a condition check rather than a hard-coded sleep. A later official revision - can still invalidate an in-flight result, but it cannot create a second model - call for an already-consumed bucket. +- In websocket mode, clock completeness alone is not enough. The REST producer + records only a stable generation: a source minute is eligible exactly when + its timestamp is strictly before `floor(request_started_at - true_up_delay)`. + Every shared OHLC snapshot atomically carries that exact immutable set as + `official_completed_minutes`. A 09:55 five-minute bucket needs all five exact + source minutes, 09:55 through 09:59; the final watermark alone is insufficient + because an intermediate REST hole must still block inference. This is a + condition check rather than a hard-coded sleep. A later official revision can + still invalidate an in-flight result, but it cannot create a second model call + for an already-consumed bucket. - At 15:00 IST new entries and adds stop; management and exits continue. - At 15:15 IST the host square-off closes exposure and stops the worker. @@ -144,13 +147,27 @@ missing row with a warning; the labels remain separate from legacy CPR workers. ## Decision audit With `CPR_AI_DECISION_LOGGING_ENABLED=true`, the host appends sanitized JSONL to -`Backtest Outputs/cpr_ai_decisions.jsonl` by default. Each row records the frozen -context, proposal, accepted regime, validation code/reason, authoritative host -geometry, execution outcome, latency, inference-attempt count, aggregate token -usage across a retry, and final tool-call evidence. -Credential-like mapping fields are removed recursively before serialization. -Logging never makes a proposal executable, and an enabled log must succeed -before an entry or add may be submitted. +`Backtest Outputs/cpr_ai_decisions.jsonl` by default. Each row has an IST +`recorded_at` timestamp and an `audit_stage`: `PRE_ACTION` is the host record +before an entry/add may increase exposure, and `POST_ACTION` records the actual +submission/confirmation result afterward. Direct diagnostic callers retain the +safe `DIRECT` stage. + +The `bar` object records the start timestamp, frozen signature, the one +validation-time current signature retained by the outcome, all five required +official minute stamps, the required stamps present in the inference snapshot, +and the resulting exact coverage boolean. Each `attempt_evidence` item records +only its request kind (`normal` or the fixed `tool_repair`), typed evidence +result, safe tool name/status records, and token usage. The corrective retry is +allowed once only for missing/failed frozen-tool evidence, uses the same frozen +snapshot, and receives only the time remaining from the original total deadline; +any terminal failure remains a fail-closed HOLD. + +Credential-like mapping fields are removed recursively before serialization, and +the logger deliberately omits model reasoning/final responses, auth data, local +paths, broker/order/venue details, symbols, quantities, and SDK error text. +Logging never makes a proposal executable, and an enabled log must succeed before +an entry or add may be submitted. ## Zero-order smoke commands diff --git a/Signal Generators/CPR AI Agent/cpr_ai_agent.py b/Signal Generators/CPR AI Agent/cpr_ai_agent.py index c761305..6263de9 100644 --- a/Signal Generators/CPR AI Agent/cpr_ai_agent.py +++ b/Signal Generators/CPR AI Agent/cpr_ai_agent.py @@ -113,6 +113,10 @@ class CPRAgentOutcome: tool_evidence: tuple[CPRToolCallRecord, ...] = () inference_attempts: int = 1 attempt_evidence: tuple[CPRAttemptEvidence, ...] = () + # This is the single signature sample the host used while validating the + # child result. Keeping it on the outcome prevents a later audit writer + # from accidentally observing a newer shared-market generation. + validation_current_signature: str | None = None def _hold(code: str, reason: str, proposal: Any | None = None, *, regime: str | None = None) -> CPRAgentOutcome: @@ -486,7 +490,7 @@ def decide( attempt_evidence, ) try: - result, attempt_evidence, combined_usage, terminal_code = future.result( + result, recorded_attempts, combined_usage, terminal_code = future.result( timeout=self.timeout_seconds ) except TimeoutError: @@ -496,29 +500,37 @@ def decide( future.cancel() release_when_finished = True future.add_done_callback(lambda _future: self._inference_lock.release()) - return self._hold_with_attempt_evidence("timeout", attempt_evidence) + return self._terminal_hold_with_attempt_evidence( + "timeout", attempt_evidence, current_signature + ) except Exception as error: # optional SDK failure must disable this agent only # Timeout-class errors retain their conservative pre-launch audit. # Other first-turn runtime errors keep the established generic # runtime outcome and never expose an exception message. if self._is_timeout_error(error): - return self._hold_with_attempt_evidence("timeout", attempt_evidence) - return _hold("runtime_error", f"Optional Codex runtime failed: {type(error).__name__}.") + return self._terminal_hold_with_attempt_evidence( + "timeout", attempt_evidence, current_signature + ) + return self._terminal_hold_with_attempt_evidence( + "runtime_error", attempt_evidence, current_signature + ) finally: executor.shutdown(wait=False, cancel_futures=True) if not release_when_finished: self._inference_lock.release() latency_ms = int((monotonic() - started) * 1000) outcome = ( - _hold(terminal_code, self._terminal_failure_reason(terminal_code)) + self._terminal_hold_with_attempt_evidence( + terminal_code, list(recorded_attempts), current_signature + ) if terminal_code is not None else self._validate_run(result, context, bar_signature, current_signature) ) outcome.latency_ms = latency_ms outcome.token_usage = combined_usage outcome.tool_evidence = result.tool_calls - outcome.inference_attempts = len(attempt_evidence) - outcome.attempt_evidence = attempt_evidence + outcome.inference_attempts = len(recorded_attempts) + outcome.attempt_evidence = recorded_attempts return outcome def _run_turn_with_tool_retry( @@ -591,6 +603,17 @@ def _run_turn_with_tool_retry( # normal result proved a repair was needed, however, a failed # repair must retain that completed first audit trail. if attempt_index == 0: + if not self._is_timeout_error(error): + # A normal child failed before returning any evidence. + # Keep one typed record without inventing tools, usage, + # or an exception string for the operator audit. + attempt_evidence[diagnostic_index] = CPRAttemptEvidence( + attempt_number=1, + request_kind=CPRTurnRequestKind.NORMAL, + evidence_code="runtime_error", + tool_records=(), + token_usage={}, + ) raise terminal_code = "timeout" if self._is_timeout_error(error) else "runtime_error" attempt_evidence[diagnostic_index] = ( @@ -647,12 +670,18 @@ def _terminal_failure_reason(code: str) -> str: return "The corrective Codex turn exhausted the original deadline." return "The optional corrective Codex runtime failed." - def _hold_with_attempt_evidence( + def _terminal_hold_with_attempt_evidence( self, code: str, attempt_evidence: list[CPRAttemptEvidence], + current_signature: Callable[[], str] | None, ) -> CPRAgentOutcome: - """Build a terminal HOLD while retaining only safe completed-attempt data.""" + """Build a terminal HOLD with its one retained host signature sample. + + These paths never reach ``_validate_run`` because the optional child + timed out or failed. Capture one signature here rather than leaving a + JSONL row ambiguous or calling the mutable shared-data getter later. + """ recorded_attempts = tuple(attempt_evidence) outcome = _hold(code, self._terminal_failure_reason(code)) @@ -663,7 +692,10 @@ def _hold_with_attempt_evidence( (record.tool_records for record in reversed(recorded_attempts) if record.tool_records), (), ) - return outcome + return self._with_validation_current_signature( + outcome, + current_signature() if current_signature is not None else None, + ) @staticmethod def _combined_attempt_token_usage( @@ -792,26 +824,49 @@ def _validate_run( to place a trade. """ + # Sample mutable market identity exactly once for this validation. The + # audit row must describe this host decision, not a later poll that may + # have received an official-candle correction in the meantime. + validation_current_signature = ( + current_signature() if current_signature is not None else None + ) evidence_error = self._tool_evidence_error(result) if evidence_error is not None: - return _hold(*evidence_error) - if current_signature is not None and current_signature() != bar_signature: - return _hold("stale_bar_signature", "The frozen completed bar is no longer current.") + return self._with_validation_current_signature( + _hold(*evidence_error), validation_current_signature + ) + if ( + current_signature is not None + and validation_current_signature != bar_signature + ): + return self._with_validation_current_signature( + _hold("stale_bar_signature", "The frozen completed bar is no longer current."), + validation_current_signature, + ) try: from cpr_ai_schema import CPRAgentDecision proposal = CPRAgentDecision.model_validate_json(result.final_response) except Exception: - return _hold("malformed_output", "Codex output did not match the strict decision schema.") + return self._with_validation_current_signature( + _hold("malformed_output", "Codex output did not match the strict decision schema."), + validation_current_signature, + ) if proposal.model_used != self.model: - return _hold("model_mismatch", "Model echo does not match the configured model.", proposal) + return self._with_validation_current_signature( + _hold("model_mismatch", "Model echo does not match the configured model.", proposal), + validation_current_signature, + ) expected_prompt = self.prompt_version if expected_prompt is None: from cpr_ai_prompt import CPR_AI_PROMPT_VERSION expected_prompt = CPR_AI_PROMPT_VERSION if proposal.prompt_version != expected_prompt: - return _hold("prompt_version_mismatch", "Prompt-version echo does not match the host prompt.", proposal) + return self._with_validation_current_signature( + _hold("prompt_version_mismatch", "Prompt-version echo does not match the host prompt.", proposal), + validation_current_signature, + ) outcome = self.policy.validate(context, proposal) # The SDK boundary has proved that this was a contemporaneous, pinned # regime classification. Preserve it even when hard execution gates @@ -821,6 +876,18 @@ def _validate_run( "invalid_frozen_context", }: outcome.accepted_regime = proposal.regime + return self._with_validation_current_signature( + outcome, validation_current_signature + ) + + @staticmethod + def _with_validation_current_signature( + outcome: CPRAgentOutcome, + validation_current_signature: str | None, + ) -> CPRAgentOutcome: + """Retain the one validation-time signature on every audited outcome.""" + + outcome.validation_current_signature = validation_current_signature return outcome @staticmethod diff --git a/Signal Generators/CPR AI Agent/cpr_ai_decision_log.py b/Signal Generators/CPR AI Agent/cpr_ai_decision_log.py index 1b46dc1..4c3a090 100644 --- a/Signal Generators/CPR AI Agent/cpr_ai_decision_log.py +++ b/Signal Generators/CPR AI Agent/cpr_ai_decision_log.py @@ -16,8 +16,16 @@ import json import re from collections.abc import Mapping +from datetime import datetime from pathlib import Path from typing import Any +from zoneinfo import ZoneInfo + +from cpr_ai_tools import EXPECTED_TOOL_NAMES + +# The isolated adapter reports terminal evidence only for these statuses. Keep +# the serializer closed so untrusted child text cannot become a JSONL disclosure. +_SAFE_TOOL_STATUSES = frozenset({"completed", "failed"}) _SENSITIVE_KEY_TOKENS = frozenset( { @@ -30,13 +38,21 @@ "brokers", "credential", "credentials", + "path", + "paths", "order", "orders", "password", "passwords", "secret", "secrets", + "symbol", + "symbols", "token", + "quantity", + "quantities", + "reasoning", + "response", "venue", "venues", } @@ -130,18 +146,29 @@ def write( token_usage: Mapping[str, Any], tool_evidence: list[Mapping[str, Any]], execution: Mapping[str, Any] | None = None, + audit_stage: str = "DIRECT", + bar_metadata: Mapping[str, Any] | None = None, ) -> None: """Append one complete sanitized record after a host decision. - The default execution object explicitly says no order was submitted, - which is safer than leaving an absent field open to interpretation. + ``DIRECT`` is a safe default for older diagnostics that call this + logger outside the worker. The CPR worker labels its first record + ``PRE_ACTION`` and its follow-up provenance record ``POST_ACTION``. Parent directories are created only after the enabled guard passes. """ if not self.enabled: return + bar = dict(bar_metadata or {}) + # The outcome is the authority for this value: it was sampled once at + # validation and must not be rebuilt from mutable shared market data. + bar["validation_current_signature"] = getattr( + outcome, "validation_current_signature", None + ) row = _sanitized( { + "recorded_at": datetime.now(ZoneInfo("Asia/Kolkata")).isoformat(), + "audit_stage": audit_stage, "frozen_context": frozen_context, "proposal": proposal, "accepted_regime": outcome.accepted_regime, @@ -167,12 +194,69 @@ def write( # totals understandable during later operational review. "inference_attempts": int(getattr(outcome, "inference_attempts", 1)), "token_usage": token_usage, - "tool_evidence": tool_evidence, + "tool_evidence": self._safe_tool_records(tool_evidence), + "attempt_evidence": self._safe_attempt_evidence( + getattr(outcome, "attempt_evidence", ()) + ), + "bar": bar, } ) self.path.parent.mkdir(parents=True, exist_ok=True) with self.path.open("a", encoding="utf-8") as handle: handle.write(json.dumps(row, sort_keys=True, separators=(",", ":")) + "\n") + @staticmethod + def _safe_tool_records(records: Any) -> list[dict[str, str]]: + """Keep only canonical MCP tool names and terminal statuses. + + SDK error text can contain a local path, a response body, or credentials. + The typed evidence code already says whether a tool failed, so retaining + opaque error text is unnecessary and unsafe for an operational audit. + """ + + safe_records: list[dict[str, str]] = [] + for record in records: + if isinstance(record, Mapping): + tool = record.get("tool") + status = record.get("status") + else: + tool = getattr(record, "tool", None) + status = getattr(record, "status", None) + if tool not in EXPECTED_TOOL_NAMES or status not in _SAFE_TOOL_STATUSES: + # The typed evidence code still records why this attempt was + # rejected, so retain no arbitrary tool/status payload at all. + continue + safe_records.append({"tool": tool, "status": status}) + return safe_records + + @classmethod + def _safe_attempt_evidence(cls, attempts: Any) -> list[dict[str, Any]]: + """Serialize Task 2 attempt facts without model text or SDK errors.""" + + safe_attempts: list[dict[str, Any]] = [] + for attempt in attempts: + if isinstance(attempt, Mapping): + attempt_number = attempt.get("attempt_number", 0) + request_kind = attempt.get("request_kind") + evidence_code = attempt.get("evidence_code") + tool_records = attempt.get("tool_records", ()) + token_usage = attempt.get("token_usage", {}) + else: + attempt_number = getattr(attempt, "attempt_number", 0) + request_kind = getattr(attempt, "request_kind", None) + evidence_code = getattr(attempt, "evidence_code", None) + tool_records = getattr(attempt, "tool_records", ()) + token_usage = getattr(attempt, "token_usage", {}) + safe_attempts.append( + { + "attempt_number": int(attempt_number), + "request_kind": str(request_kind), + "evidence_code": evidence_code, + "tool_records": cls._safe_tool_records(tool_records), + "token_usage": dict(token_usage), + } + ) + return safe_attempts + __all__ = ["CPRDecisionLogger"] diff --git a/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py b/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py index 7c7c85e..28cb374 100644 --- a/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py +++ b/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py @@ -375,6 +375,136 @@ def runner(**_kwargs): assert outcome.token_usage == {} +@pytest.mark.parametrize( + ("kind", "runner", "expected_code", "expected_attempts"), + [ + pytest.param( + "outer_timeout", + None, + "timeout", + [(CPRTurnRequestKind.NORMAL, "timeout", (), {})], + id="outer-timeout", + ), + pytest.param( + "inner_timeout", + lambda **_kwargs: (_ for _ in ()).throw( + subprocess.TimeoutExpired(cmd="private timeout path", timeout=1.0) + ), + "timeout", + [(CPRTurnRequestKind.NORMAL, "timeout", (), {})], + id="inner-timeout", + ), + pytest.param( + "first_runtime", + lambda **_kwargs: (_ for _ in ()).throw(RuntimeError("credential private response")), + "runtime_error", + [(CPRTurnRequestKind.NORMAL, "runtime_error", (), {})], + id="first-runtime", + ), + ], +) +def test_terminal_first_attempt_outcomes_capture_one_validation_signature( + kind, runner, expected_code, expected_attempts +): + """Terminal first-turn outcomes audit one host signature without raw errors.""" + + release = threading.Event() + started = threading.Event() + if kind == "outer_timeout": + + def runner(**_kwargs): + started.set() + release.wait(timeout=0.3) + return CPRAgentRunResult("{}", (), {}) + + agent = CPRAgent(runner=runner, timeout_seconds=0.01) + else: + agent = CPRAgent(runner=runner) + signature_samples = [] + + try: + outcome = agent.decide( + _context(), + bar_signature="frozen", + current_signature=lambda: signature_samples.append("terminal-current") or "terminal-current", + ) + finally: + release.set() + + assert outcome.validation_code == expected_code + assert outcome.validation_current_signature == "terminal-current" + assert signature_samples == ["terminal-current"] + assert [ + (item.request_kind, item.evidence_code, item.tool_records, item.token_usage) + for item in outcome.attempt_evidence + ] == expected_attempts + assert "private" not in outcome.validation_reason + + +@pytest.mark.parametrize("repair_error", [TimeoutError(), RuntimeError("broker secret path")]) +def test_terminal_repair_failures_capture_one_validation_signature(repair_error): + """A repair terminal result retains its one final host-generation sample.""" + + calls = [] + + def runner(**_kwargs): + calls.append("attempt") + if len(calls) == 1: + return CPRAgentRunResult( + _proposal("HOLD", "SIDEWAYS", "NONE").model_dump_json(), + _calls(missing="market_structure"), + {"total_tokens": 3}, + ) + raise repair_error + + samples = [] + current_signatures = iter(("frozen", "repair-current")) + outcome = CPRAgent(runner=runner).decide( + _context(), + bar_signature="frozen", + current_signature=lambda: samples.append(next(current_signatures)) or samples[-1], + ) + + assert outcome.validation_code == ("timeout" if isinstance(repair_error, TimeoutError) else "runtime_error") + assert outcome.validation_current_signature == "repair-current" + assert samples == ["frozen", "repair-current"] + assert [(item.request_kind, item.evidence_code) for item in outcome.attempt_evidence] == [ + (CPRTurnRequestKind.NORMAL, "missing_tool_call"), + (CPRTurnRequestKind.TOOL_REPAIR, outcome.validation_code), + ] + + +def test_decision_log_drops_untrusted_tool_and_status_text_from_every_evidence_shape(tmp_path): + """Only canonical tool/status pairs may enter final or per-attempt JSONL.""" + + unsafe = "C:/private auth broker order venue symbol quantity response" + outcome = CPRHostPolicy().validate(_context(), _proposal("HOLD", "SIDEWAYS", "NONE")) + outcome.attempt_evidence = ( + cpr_ai_agent.CPRAttemptEvidence( + attempt_number=1, + request_kind=CPRTurnRequestKind.NORMAL, + evidence_code="unexpected_agent_action", + tool_records=(CPRToolCallRecord(tool=unsafe, status=unsafe),), + token_usage={}, + ), + ) + path = tmp_path / "decisions.jsonl" + CPRDecisionLogger(str(path)).write( + frozen_context=_context(), + proposal=None, + outcome=outcome, + latency_ms=1, + token_usage={}, + tool_evidence=[{"tool": unsafe, "status": unsafe}], + ) + + raw = path.read_text(encoding="utf-8") + row = json.loads(raw) + assert unsafe not in raw + assert row["tool_evidence"] == [] + assert row["attempt_evidence"][0]["tool_records"] == [] + + @pytest.mark.parametrize("first_calls", [_calls(missing="market_structure"), _calls(failed="position_state")]) def test_second_incomplete_tool_repair_remains_hold_after_exactly_two_attempts(first_calls): """A repair may not cascade into a third inference or a permissive result.""" @@ -940,6 +1070,244 @@ def test_decision_log_removes_plural_sensitive_keys_but_keeps_near_matches(tmp_p } +def test_decision_log_records_ist_bar_coverage_and_typed_attempt_evidence_without_secrets(tmp_path): + """An audit row must preserve host facts without retaining unsafe model text. + + Removing the timestamp, a bar-generation field, retry evidence, or the + recursive redaction of a nested diagnostic must make this test fail. The + expected values are hand-authored so this does not mirror logger helpers. + """ + + calls = iter( + [ + CPRAgentRunResult( + _proposal("HOLD", "SIDEWAYS", "NONE").model_dump_json(), + _calls(missing="market_structure"), + {"input_tokens": 3, "total_tokens": 5}, + ), + CPRAgentRunResult( + _proposal("HOLD", "SIDEWAYS", "NONE").model_dump_json(), + _calls(), + {"output_tokens": 2, "total_tokens": 4}, + ), + ] + ) + validation_signatures = iter(["frozen-0930", "frozen-0930"]) + outcome = CPRAgent(runner=lambda **_kwargs: next(calls)).decide( + _context(), + bar_signature="frozen-0930", + current_signature=lambda: next(validation_signatures), + ) + path = tmp_path / "decisions.jsonl" + metadata = { + "bar_timestamp": "2026-08-13T09:30:00+05:30", + "frozen_signature": "frozen-0930", + "required_official_minutes": [ + "2026-08-13T09:30:00+05:30", + "2026-08-13T09:31:00+05:30", + "2026-08-13T09:32:00+05:30", + "2026-08-13T09:33:00+05:30", + "2026-08-13T09:34:00+05:30", + ], + "present_official_minutes": [ + "2026-08-13T09:30:00+05:30", + "2026-08-13T09:31:00+05:30", + "2026-08-13T09:32:00+05:30", + "2026-08-13T09:33:00+05:30", + "2026-08-13T09:34:00+05:30", + ], + "official_coverage": True, + "nested": { + "auth": "DO-NOT-LOG-AUTH", + "local_path": "C:/private/decision.jsonl", + "broker": "DO-NOT-LOG-BROKER", + "order": "DO-NOT-LOG-ORDER", + "venue": "DO-NOT-LOG-VENUE", + "symbol": "DO-NOT-LOG-SYMBOL", + "quantity": "DO-NOT-LOG-QUANTITY", + }, + } + logger = CPRDecisionLogger(str(path)) + logger.write( + frozen_context=_context(), + proposal=_proposal("HOLD", "SIDEWAYS", "NONE"), + outcome=outcome, + latency_ms=12, + token_usage=outcome.token_usage, + tool_evidence=[record.__dict__ for record in outcome.tool_evidence], + audit_stage="PRE_ACTION", + bar_metadata=metadata, + ) + logger.write( + frozen_context=_context(), + proposal=_proposal("HOLD", "SIDEWAYS", "NONE"), + outcome=outcome, + latency_ms=12, + token_usage=outcome.token_usage, + tool_evidence=[record.__dict__ for record in outcome.tool_evidence], + audit_stage="POST_ACTION", + bar_metadata=metadata, + ) + + raw = path.read_text(encoding="utf-8") + rows = [json.loads(line) for line in raw.splitlines()] + assert outcome.validation_current_signature == "frozen-0930" + assert [row["audit_stage"] for row in rows] == ["PRE_ACTION", "POST_ACTION"] + assert all( + __import__("datetime").datetime.fromisoformat(row["recorded_at"]).utcoffset().total_seconds() == 19800 + for row in rows + ) + assert rows[0]["bar"] == { + "bar_timestamp": "2026-08-13T09:30:00+05:30", + "frozen_signature": "frozen-0930", + "validation_current_signature": "frozen-0930", + "required_official_minutes": metadata["required_official_minutes"], + "present_official_minutes": metadata["present_official_minutes"], + "official_coverage": True, + "nested": {}, + } + assert rows[0]["attempt_evidence"] == [ + { + "attempt_number": 1, + "request_kind": "normal", + "evidence_code": "missing_tool_call", + "tool_records": [ + {"tool": "session_levels", "status": "completed"}, + {"tool": "momentum_vwap", "status": "completed"}, + {"tool": "position_state", "status": "completed"}, + ], + "token_usage": {"input_tokens": 3, "total_tokens": 5}, + }, + { + "attempt_number": 2, + "request_kind": "tool_repair", + "evidence_code": None, + "tool_records": [ + {"tool": name, "status": "completed"} for name in EXPECTED_TOOL_NAMES + ], + "token_usage": {"output_tokens": 2, "total_tokens": 4}, + }, + ] + assert "Synthetic test proposal." not in raw + assert "DO-NOT-LOG" not in raw + + +def test_decision_log_direct_call_keeps_a_safe_default_stage_and_disabled_logging_is_a_noop(tmp_path): + """Legacy direct callers must remain append-compatible and optional logging inert.""" + + path = tmp_path / "decisions.jsonl" + outcome = CPRHostPolicy().validate(_context(), _proposal("HOLD", "SIDEWAYS", "NONE")) + CPRDecisionLogger(str(path)).write( + frozen_context=_context(), + proposal=_proposal("HOLD", "SIDEWAYS", "NONE"), + outcome=outcome, + latency_ms=1, + token_usage={}, + tool_evidence=[], + ) + CPRDecisionLogger(str(tmp_path / "disabled.jsonl"), enabled=False).write( + frozen_context=_context(), + proposal=None, + outcome=outcome, + latency_ms=1, + token_usage={}, + tool_evidence=[], + ) + + assert json.loads(path.read_text(encoding="utf-8"))["audit_stage"] == "DIRECT" + assert not (tmp_path / "disabled.jsonl").exists() + + +@pytest.mark.parametrize( + ("calls", "signatures", "code", "expected_signature"), + [ + pytest.param( + _calls(missing="position_state"), + ("new-before-retry", "new-at-validation"), + "missing_tool_call", + "new-at-validation", + id="missing-tool", + ), + pytest.param( + _calls(), + ("new-at-validation",), + "stale_bar_signature", + "new-at-validation", + id="stale-bar", + ), + ], +) +def test_agent_retains_the_exact_validation_signature_for_nonexecuting_outcomes( + calls, signatures, code, expected_signature +): + """The audit must use the validation sample, not an earlier retry check.""" + + samples = iter(signatures) + outcome = CPRAgent( + runner=_runner(_proposal("HOLD", "SIDEWAYS", "NONE"), calls=calls) + ).decide( + _context(), + bar_signature="frozen", + current_signature=lambda: next(samples), + ) + + assert outcome.validation_code == code + assert outcome.validation_current_signature == expected_signature + + +def test_decision_log_keeps_normal_and_failed_repair_attempt_evidence_parseable(tmp_path): + """Every attempt shape remains useful when recovery still fails closed.""" + + normal = CPRAgent(runner=_runner(_proposal("HOLD", "SIDEWAYS", "NONE"))).decide( + _context(), bar_signature="normal-only" + ) + attempts = iter( + [ + CPRAgentRunResult( + _proposal("HOLD", "SIDEWAYS", "NONE").model_dump_json(), + _calls(missing="market_structure"), + {"total_tokens": 3}, + ), + CPRAgentRunResult( + _proposal("HOLD", "SIDEWAYS", "NONE").model_dump_json(), + _calls(failed="position_state"), + {"total_tokens": 4}, + ), + ] + ) + failed_repair = CPRAgent(runner=lambda **_kwargs: next(attempts)).decide( + _context(), bar_signature="failed-repair", current_signature=lambda: "failed-repair" + ) + path = tmp_path / "decisions.jsonl" + logger = CPRDecisionLogger(str(path)) + for outcome in (normal, failed_repair): + logger.write( + frozen_context=_context(), + proposal=_proposal("HOLD", "SIDEWAYS", "NONE"), + outcome=outcome, + latency_ms=1, + token_usage=outcome.token_usage, + tool_evidence=[], + ) + + rows = [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines()] + assert rows[0]["attempt_evidence"] == [ + { + "attempt_number": 1, + "request_kind": "normal", + "evidence_code": None, + "tool_records": [ + {"tool": name, "status": "completed"} for name in EXPECTED_TOOL_NAMES + ], + "token_usage": {"total_tokens": 17}, + } + ] + assert [(item["request_kind"], item["evidence_code"]) for item in rows[1]["attempt_evidence"]] == [ + ("normal", "missing_tool_call"), + ("tool_repair", "failed_tool_call"), + ] + + def test_geometry_uses_next_directional_level_not_hard_coded_r1_s1(): """Already beyond R1/S1, the next frozen milestone must be used instead.""" diff --git a/Tests/test_nifty_multi_strategy_master.py b/Tests/test_nifty_multi_strategy_master.py index 277c9b5..c2075c7 100644 --- a/Tests/test_nifty_multi_strategy_master.py +++ b/Tests/test_nifty_multi_strategy_master.py @@ -9987,6 +9987,50 @@ def test_worker_runs_one_agent_turn_per_completed_five_minute_signature(self): self.assertEqual(worker._prior_accepted_regime, "SIDEWAYS") logger.write.assert_called_once() + def test_worker_marks_pre_and_post_audits_with_one_frozen_coverage_snapshot(self): + """An entry blocked after inference must still retain both distinct audit stages. + + The bar/coverage evidence belongs to the inference snapshot, so a later + execution outcome must not rebuild or silently alter it. + """ + + worker, agent, logger = self._worker() + outcome = agent.decide.return_value + outcome.action = "ENTER_LONG" + outcome.accepted = True + outcome.entry_price = 100.0 + outcome.stop_price = 95.0 + outcome.final_target_price = 118.0 + outcome.validation_current_signature = "validated-0930" + worker._latest_frozen_context = lambda: { + "session_levels": {}, + "momentum_vwap": {}, + "market_structure": {}, + "position_state": {"is_flat": True}, + } + worker._completed_bar_signature = lambda _frame: "frozen-0930" + worker._current_completed_spot_signature = lambda: "validated-0930" + worker._post_inference_exposure_block_reason = lambda: "entry_cutoff" + metadata = { + "bar_timestamp": "2026-08-13T09:30:00+05:30", + "frozen_signature": "frozen-0930", + "required_official_minutes": ["one", "two", "three", "four", "five"], + "present_official_minutes": ["one", "two", "three", "four", "five"], + "official_coverage": True, + } + + worker.process_strategy_frame( + pd.DataFrame([{"timestamp": pd.Timestamp("2026-08-13 09:30"), "close": 100.0}]), + audit_metadata=metadata, + ) + + self.assertEqual(logger.write.call_count, 2) + pre_action, post_action = logger.write.call_args_list + self.assertEqual(pre_action.kwargs["audit_stage"], "PRE_ACTION") + self.assertEqual(post_action.kwargs["audit_stage"], "POST_ACTION") + self.assertEqual(pre_action.kwargs["bar_metadata"], metadata) + self.assertEqual(post_action.kwargs["bar_metadata"], metadata) + def test_forming_websocket_minute_waits_for_close_and_true_up_never_repeats_bucket(self): """Model forming websocket revisions and an official REST correction. From 437f21badde2503777c42ff7bc35440d252aeffd Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 19:50:33 +0530 Subject: [PATCH 08/10] docs(cpr-ai): clarify official-candle recovery Co-authored-by: Codex --- ...Multi Strategy Front Test - Master File.py | 25 ++++++--- Signal Generators/CPR AI Agent/README.md | 32 ++++++----- .../CPR AI Agent/cpr_ai_agent.py | 53 ++++++++++++------- .../CPR AI Agent/test_cpr_ai_runtime.py | 6 ++- Tests/test_market_data_health.py | 4 +- Tests/test_nifty_multi_strategy_master.py | 16 +++++- 6 files changed, 91 insertions(+), 45 deletions(-) diff --git a/Nifty Multi Strategy Front Test - Master File.py b/Nifty Multi Strategy Front Test - Master File.py index 0292d1b..560412e 100644 --- a/Nifty Multi Strategy Front Test - Master File.py +++ b/Nifty Multi Strategy Front Test - Master File.py @@ -3929,9 +3929,11 @@ class WebSocketMarketDataFetcher(threading.Thread): the full desired instrument set by construction. Bar semantics: the published frame is always - ``merge_official_and_tick_frames(official REST history, tick bars)`` -- - official candles win for completed minutes, the forming minute is always - tick-built, and every publish still goes through `store.update()` and its + ``merge_official_and_tick_frames(official REST history, tick bars)``. + Official REST candles win only for minutes proved final at the REST + request's start. The forming minute and a newly closed minute still inside + the grace period remain tick-built until a later true-up proves them final. + Every publish still goes through `store.update()` and its `validate_ohlc_frame` net. Consumers are untouched: same store, same snapshot shape, same health gates. """ @@ -4267,10 +4269,13 @@ def _maybe_run_true_up(self, now_ist: datetime | None = None) -> None: def _run_true_up(self, reason: str, now_ist: datetime | None = None) -> None: """ - Replace completed candles with Dhan's official ones (tick bars stay - for anything REST does not cover, most importantly the forming - minute). A REST failure keeps serving tick bars and retries on the - next minute -- the tick feed remains the live source of truth. + Replace only REST minutes proved final at the request-start boundary. + + Tick bars remain in charge of the forming minute and any newer row that + is still inside the grace period. An older hole in stable REST history + stays fail-closed until a later REST response fills it; we do not keep a + provisional tick candle as if it were official. After a REST failure, + the next active minute with fresh ticks creates another true-up chance. """ if now_ist is None: now_ist = datetime.now(ZoneInfo("Asia/Kolkata")).replace(tzinfo=None) @@ -9792,6 +9797,12 @@ def _completed_bar_audit_metadata( that may have advanced while Codex or execution was running. Direct unit/diagnostic callers may not have a snapshot; they receive an empty, explicitly false coverage record rather than invented evidence. + + ``required_official_minutes`` lists the five one-minute source rows + needed to build this bucket. ``present_official_minutes`` is only the + intersection of those five rows with the frozen official set -- not all + REST history in the store. ``official_coverage`` is true only when that + intersection contains all five required rows. """ bucket_start = ( diff --git a/Signal Generators/CPR AI Agent/README.md b/Signal Generators/CPR AI Agent/README.md index 5126997..7a05954 100644 --- a/Signal Generators/CPR AI Agent/README.md +++ b/Signal Generators/CPR AI Agent/README.md @@ -29,11 +29,13 @@ deep-copy views of the same frozen completed-bar context: The snapshot contains no order surface, account, credential, broker, venue, or execution object. The immediate turn request repeats the four exact tool names -in addition to the developer prompt. If the first isolated turn still omits a -tool, or one tool reports failure, the host permits one retry against the same -immutable snapshot using only the time left in the original SDK deadline. A -second incomplete result, a duplicate/unapproved tool, or any unexpected agent -action invalidates the turn and produces `HOLD`. +in addition to the developer prompt. A repair is considered only when a missing +or failed required tool is the sole remaining defect: the first response must +also still match the current bar, strict schema, configured model/prompt, and +deterministic host policy. The repair uses the same immutable snapshot and only +the time left in the original CPR turn wall-clock deadline, including isolated +child-process overhead. A second incomplete result, a duplicate/unapproved +tool, or any unexpected agent action invalidates the turn and produces `HOLD`. ## Decision contract and host gates @@ -153,15 +155,17 @@ before an entry/add may increase exposure, and `POST_ACTION` records the actual submission/confirmation result afterward. Direct diagnostic callers retain the safe `DIRECT` stage. -The `bar` object records the start timestamp, frozen signature, the one -validation-time current signature retained by the outcome, all five required -official minute stamps, the required stamps present in the inference snapshot, -and the resulting exact coverage boolean. Each `attempt_evidence` item records -only its request kind (`normal` or the fixed `tool_repair`), typed evidence -result, safe tool name/status records, and token usage. The corrective retry is -allowed once only for missing/failed frozen-tool evidence, uses the same frozen -snapshot, and receives only the time remaining from the original total deadline; -any terminal failure remains a fail-closed HOLD. +The `bar` object records the start timestamp, frozen signature, the one current +signature captured when the host finalizes validation or a terminal fail-closed +outcome, all five required official minute stamps, the required stamps present +in the inference snapshot, and the resulting exact coverage boolean. + +Each `attempt_evidence` item records only its request kind (`normal` or the fixed +`tool_repair`), typed evidence result, safe tool name/status records, and token +usage. A provisional timeout marker can appear when a turn was selected but no +child evidence returned before the shared deadline; empty tool/usage fields do +not claim that a launched child consumed zero tokens. Any terminal failure +remains a fail-closed HOLD. Credential-like mapping fields are removed recursively before serialization, and the logger deliberately omits model reasoning/final responses, auth data, local diff --git a/Signal Generators/CPR AI Agent/cpr_ai_agent.py b/Signal Generators/CPR AI Agent/cpr_ai_agent.py index 6263de9..7b4aac4 100644 --- a/Signal Generators/CPR AI Agent/cpr_ai_agent.py +++ b/Signal Generators/CPR AI Agent/cpr_ai_agent.py @@ -72,11 +72,14 @@ class CPRAgentRunResult: @dataclass(frozen=True) class CPRAttemptEvidence: - """Auditable result of one isolated model attempt within a bar deadline. - - This records only request mode, tool coverage, and numeric usage. It does - not keep chain-of-thought, returned MCP payloads, authentication data, or - any execution capability. + """Auditable record for one isolated turn slot within a bar deadline. + + Most records describe a child turn that returned. A record can instead be + a conservative pre-launch/timeout marker when the shared deadline expires + before usable child evidence comes back. Empty tool records and token usage + then mean "nothing was returned and proved," not "the child proved it used + zero tokens." The audit never keeps chain-of-thought, returned MCP payloads, + authentication data, or any execution capability. """ attempt_number: int @@ -113,9 +116,10 @@ class CPRAgentOutcome: tool_evidence: tuple[CPRToolCallRecord, ...] = () inference_attempts: int = 1 attempt_evidence: tuple[CPRAttemptEvidence, ...] = () - # This is the single signature sample the host used while validating the - # child result. Keeping it on the outcome prevents a later audit writer - # from accidentally observing a newer shared-market generation. + # This is the single current-signature sample captured when the host + # finalizes either normal validation or a terminal fail-closed outcome. + # Keeping it on the outcome prevents a later audit writer from accidentally + # observing a newer shared-market generation. validation_current_signature: str | None = None @@ -545,8 +549,10 @@ def _run_turn_with_tool_retry( Both attempts receive the same in-memory context and bar signature. A retry therefore cannot silently move to newer market facts. The second - attempt receives only the wall-clock budget left after the first one, - so this recovery path never doubles the configured SDK timeout. + attempt receives only the wall-clock budget left after the first one. + That shared budget includes parent/child process setup as well as the + SDK turn itself, so this recovery path never doubles the configured + CPR inference timeout. """ results: list[CPRAgentRunResult] = [] @@ -577,10 +583,11 @@ def _run_turn_with_tool_retry( if attempt_index == 0 else CPRTurnRequestKind.TOOL_REPAIR ) - # Install a conservative no-evidence timeout diagnostic before the - # optional runtime starts. If the parent deadline wins the race, - # this proves which turn began while never inventing tool calls or - # usage that the child did not return. + # Install a conservative no-evidence marker before the optional + # runtime starts. If the parent deadline wins the race, this still + # records which turn slot was selected. Empty tool/usage fields mean + # the child returned no provable evidence; they are not an assertion + # that a launched child consumed exactly zero tokens. diagnostic_index = len(attempt_evidence) attempt_evidence.append( CPRAttemptEvidence( @@ -664,7 +671,15 @@ def _is_timeout_error(error: Exception) -> bool: @staticmethod def _terminal_failure_reason(code: str) -> str: - """Return a safe generic reason for a repair failure audit outcome.""" + """Return a credential-safe reason for a terminal Codex-turn outcome. + + Both the initial turn and an optional repair use this helper. The + historical human-readable text uses the word ``corrective`` for either + path, so ``attempt_evidence.request_kind`` is the authoritative field + when an operator needs to distinguish ``normal`` from ``tool_repair``. + The reason itself never includes exception text, child output, command + arguments, or local paths. + """ if code == "timeout": return "The corrective Codex turn exhausted the original deadline." @@ -678,9 +693,11 @@ def _terminal_hold_with_attempt_evidence( ) -> CPRAgentOutcome: """Build a terminal HOLD with its one retained host signature sample. - These paths never reach ``_validate_run`` because the optional child - timed out or failed. Capture one signature here rather than leaving a - JSONL row ambiguous or calling the mutable shared-data getter later. + These paths never reach ``_validate_run`` because the initial or repair + child timed out or failed. Capture one signature here rather than + leaving a JSONL row ambiguous or calling the mutable shared-data getter + later. The call happens once during finalization, so the audit describes + this decision even if the market store advances immediately afterward. """ recorded_attempts = tuple(attempt_evidence) diff --git a/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py b/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py index 28cb374..266e127 100644 --- a/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py +++ b/Tests/Signal Generators/CPR AI Agent/test_cpr_ai_runtime.py @@ -289,6 +289,8 @@ def runner(**kwargs): def test_repair_deadline_exhaustion_preserves_completed_first_attempt_audit(monkeypatch): """A spent total deadline records a second repair timeout without inventing evidence.""" + # The first sample starts the shared five-second budget. The second shows + # that budget fully consumed before a repair child can be launched. clock = itertools.chain((100.0, 105.0), itertools.repeat(105.0)) monkeypatch.setattr(cpr_ai_agent, "monotonic", lambda: next(clock)) calls = [] @@ -766,8 +768,8 @@ def fake_run(command, **kwargs): assert tuple(call.tool for call in result.tool_calls) == EXPECTED_TOOL_NAMES -def test_parent_serializes_fixed_normal_or_tool_repair_kind_without_arbitrary_request_text(monkeypatch, tmp_path): - """The parent can select only the enum-backed repair mode for the child boundary.""" +def test_parent_serializes_enum_backed_tool_repair_kind_without_arbitrary_request_text(monkeypatch, tmp_path): + """A closed enum -- never caller prose -- selects the fixed repair request.""" requests = [] diff --git a/Tests/test_market_data_health.py b/Tests/test_market_data_health.py index 496e68f..baea7ad 100644 --- a/Tests/test_market_data_health.py +++ b/Tests/test_market_data_health.py @@ -210,8 +210,8 @@ def test_uses_request_start_and_normalizes_timestamp_representations(self) -> No self.assertEqual(stable, frozenset({pd.Timestamp("2026-08-13 09:28:00")})) - def test_certifies_just_closed_minute_after_grace_and_keeps_empty_empty(self) -> None: - """Strictly-before boundary admits 09:29 only after 09:30:05 IST.""" + def test_certifies_just_closed_minute_after_grace_and_returns_empty_set_for_empty_frame(self) -> None: + """After grace, 09:29 is official; an empty REST frame proves no minutes.""" frame = validate_ohlc_frame( _frame([datetime(2026, 8, 13, 9, 28), datetime(2026, 8, 13, 9, 29)]), diff --git a/Tests/test_nifty_multi_strategy_master.py b/Tests/test_nifty_multi_strategy_master.py index fc5727f..968d59e 100644 --- a/Tests/test_nifty_multi_strategy_master.py +++ b/Tests/test_nifty_multi_strategy_master.py @@ -1695,7 +1695,11 @@ def test_true_up_overwrites_completed_bar_keeps_forming(self): self.assertEqual(by_ts.loc[forming]["close"], 100.6) def test_true_up_before_grace_keeps_just_closed_minute_tick_owned(self): - """A reconnect before grace cannot let REST replace a forming tick bar.""" + """A clock-closed REST row stays provisional until the grace boundary. + + Although 10:16 has closed by the clock, its REST row must not overwrite + the tick-built candle until a later request proves that row final. + """ stable = pd.Timestamp("2026-05-15 10:15:00") just_closed = pd.Timestamp("2026-05-15 10:16:00") @@ -2146,7 +2150,12 @@ def test_shared_store_snapshot_carries_an_optional_official_candle_watermark(sel self.assertEqual(store.get("1").official_candle_ts, watermark) def test_shared_store_publishes_frame_exact_official_set_and_watermark_together(self): - """CPR readers must receive one immutable official-data generation.""" + """Publish the immutable official-minute set and its watermark together. + + The immutable object is the exact minute-identity set. The DataFrame is + still defensively copied for readers; this test does not claim that a + pandas frame itself is immutable. + """ frame = pd.DataFrame( { @@ -10011,6 +10020,9 @@ def test_worker_marks_pre_and_post_audits_with_one_frozen_coverage_snapshot(self worker._completed_bar_signature = lambda _frame: "frozen-0930" worker._current_completed_spot_signature = lambda: "validated-0930" worker._post_inference_exposure_block_reason = lambda: "entry_cutoff" + # These sentinels stand for the exact five one-minute REST rows that + # produced the decided 09:30 bucket. PRE_ACTION and POST_ACTION must + # retain this same frozen list instead of resampling a newer store. metadata = { "bar_timestamp": "2026-08-13T09:30:00+05:30", "frozen_signature": "frozen-0930", From f0334618906b79bf8540049a16acaa027b98cbc9 Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 22:13:53 +0530 Subject: [PATCH 09/10] feat(sl-hunting): pre-open note for 2026-08-14 from the IH analysis video Source: Intraday Hunter, "Prediction For 14 AUG 2026" (MhmlrlUEUGI, uploaded 2026-08-13, 2:14). Neither side is seated, and he says so outright for two of the three indices: BankNIFTY "buyers and sellers would be sitting at the SAME price level", Sensex "to say more sellers are seated, or more buyers are seated, would be WRONG". That is the empty-book condition v4f describes, stated more plainly than the series has had it -- and it is the exact condition v4e's expensive loss came from forecasting a crowd into. The note records the absence rather than smoothing it into a direction. His mechanism for trading a thin book is the line worth keeping: "where the crowd is THIN, that is where the market tries to make momentum." Thin is not dead -- the move goes where there is least resistance, which sits alongside v4f's AN EMPTY BOOK MEANS A TRAP IS COMING rather than against it. He also states a preference the series had only inferred: "better that it is NOT flat -- either a gap-up or a gap-down", because a flat open "keeps extracting a small momentum and going away". That is v4d's participation reading turned into an operational preference. Plan: SELL-side on flat-to-gap-down (on Sensex explicitly to target the buyers sitting on support since yesterday), BUY-side and follow on a gap-up where "the buyer becomes safe". On BankNIFTY there is little pressure on the sellers, so that branch is recorded as a FOLLOW of the drift rather than a seller hunt. Transcription caveat, and a repeat: NIFTY's second support arrived as the same unresolvable "2476" token as the 13 Aug video, so it is omitted again. Twice in two days makes it a consistent ASR failure on one number rather than noise; the test asserts NIFTY carries exactly one support so a tidy-up cannot invent it. Sensex's levels are unchanged from yesterday, consistent with a session that went nowhere. test_shipped_note_matches_august_14_intraday_hunter_plan replaces the 13 Aug equivalent and asserts both branch directions plus the thin-crowd mechanism and the gap-over-flat preference, which are what a summarising edit would drop first. Gates: SL Hunting pytest 179 passed, ruff clean. Co-Authored-By: Claude Opus 5 --- .../SL Hunting AI Agent/premarket_note.json | 27 ++++----- .../SL Hunting AI Agent/sl_hunting_doc.md | 59 +++++++++++++++++++ .../test_sl_hunting_premarket.py | 50 ++++++++-------- 3 files changed, 98 insertions(+), 38 deletions(-) diff --git a/Signal Generators/SL Hunting AI Agent/premarket_note.json b/Signal Generators/SL Hunting AI Agent/premarket_note.json index 2fa6065..07a70fd 100644 --- a/Signal Generators/SL Hunting AI Agent/premarket_note.json +++ b/Signal Generators/SL Hunting AI Agent/premarket_note.json @@ -1,26 +1,25 @@ { - "for_date": "2026-08-13", - "source": "Intraday Hunter, 'Prediction For 13 AUG 2026' (PfthlsdW2E8, uploaded 2026-08-12)", - "context": "Sensex sold hard, then took support EXACTLY at the round number and rallied, chasing out everyone who was short. BankNIFTY has been positive from the start. NIFTY sold but is recovering. BUYERS, not sellers, are now the seated crowd.", + "for_date": "2026-08-14", + "source": "Intraday Hunter, 'Prediction For 14 AUG 2026' (MhmlrlUEUGI, uploaded 2026-08-13)", + "context": "No decisive crowd anywhere. BankNIFTY made little momentum with buyers and sellers at the SAME price level; Sensex ran both ways. The market held its round number and recovered, so sellers are unlikely to have carried overnight.", "plan": [ - "FLAT to GAP-DOWN: identify SELL-side setups. Stated for all three indices, and on BankNIFTY the reason is explicit -- target the BUYERS who are now seated.", - "GAP-UP above the round number: go WITH the market and identify BUY-side setups. Do NOT try to hunt the buyers there.", - "The reason is WHERE the stops sit: 'buyers' SLs should be BELOW the round number.' A gap-up above it leaves their stops far away, so they are not huntable and following beats fading.", - "The seller crowd is SPENT. Sensex took support at the round number and the recovery chased shorts out on the retracement -- the same spent-seller condition as 12 Aug.", - "BankNIFTY is the index where buyers are genuinely seated: positive from the open, a retracement, then more upside. That is the crowd a gap-down would hunt.", - "SENSEX EXPIRY tomorrow, flagged explicitly. Expect expiry pinning and premium distortion on the Sensex read specifically.", - "NIFTY is the least committed of the three: selling, but 'some recovery is also visible'. Treat its conditional as weaker than BankNIFTY's." + "FLAT to GAP-DOWN: identify SELL-side setups. On Sensex the reason is explicit -- target the BUYERS, since the market has sat on support since yesterday and the sellers did not hold.", + "GAP-UP: go WITH the market and identify BUY-side setups. On BankNIFTY 'the buyer becomes safe' there, so there is nothing left to hunt on that side.", + "NOBODY IS DECISIVELY SEATED -- on BankNIFTY 'buyers and sellers would be sitting at the SAME price level', and on Sensex claiming either side is seated 'would be WRONG'. This is the empty-book condition.", + "His mechanism for that condition, worth keeping: 'where the crowd is THIN, that is where the market tries to make momentum.' Thin is not dead -- the move goes where there is least resistance.", + "He PREFERS a gap to a flat open: 'better it is not flat -- either a gap-up or a gap-down.' A flat open 'extracts a small momentum and goes away', producing nothing worth trading.", + "There is little pressure on BankNIFTY's sellers, so the market cannot target them directly. The flat/gap-down branch is therefore a FOLLOW of the drift, not a seller hunt." ], "levels": [ { "index": "NIFTY", - "resistance": [24500, 24600], - "support": [24260] + "resistance": [24440, 24540], + "support": [24275] }, { "index": "BANKNIFTY", - "resistance": [58000, 58300], - "support": [57500, 57310] + "resistance": [57890, 58000], + "support": [57500, 57320] }, { "index": "SENSEX", diff --git a/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md b/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md index efff0a4..bf81b66 100644 --- a/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md +++ b/Signal Generators/SL Hunting AI Agent/sl_hunting_doc.md @@ -3382,3 +3382,62 @@ what v4g's FEAR IS NOT A SIGNAL demands and what the stall-churn lacked. One caveat against reading too much into it: two trades is a small sample, and the deterministic strategies had a poor day on the same tape (Renko -4,127.50 over six trades), so the basket is deeply negative regardless. + +--- + +### Pre-open note for 2026-08-14 (Friday) + +**Source:** Intraday Hunter, "Prediction For 14 AUG 2026" (`MhmlrlUEUGI`, +uploaded 2026-08-13, 2:14). Note-only; no knowledge version attached. + +**Neither side is seated, and he says so for two of the three indices.** This is +the empty-book condition v4f describes, stated more plainly than the series has +had it before: + +> BankNIFTY: "we did not see much momentum... buyers and sellers would be sitting +> at the SAME price level. Not many people held their positions." +> Sensex: "momentum on both sides remained -- a bit of rejection, a bit of +> buying. So to say more sellers are seated, or more buyers are seated, would be +> WRONG." + +That matters because v4e's expensive lesson was forecasting a crowd into +existence on exactly this condition, and v4f's answer was to wait for +confirmation. The note records the absence rather than smoothing it into a +direction. + +**His mechanism for trading a thin book** is the line worth keeping: + +> "If they did not hold positions, then WHERE THE CROWD IS THIN, THAT IS WHERE +> THE MARKET TRIES TO MAKE MOMENTUM." + +Thin is not dead: the move goes where there is least resistance, which is a +different claim from "no crowd means no trade" and sits alongside v4f's +AN EMPTY BOOK MEANS A TRAP IS COMING rather than against it. + +**He prefers a gap to a flat open, explicitly.** New for the series as a stated +preference rather than an inference: + +> "Better that it is NOT flat -- either a gap-up or a gap-down would be better, +> because in flat it keeps extracting a small momentum and going away. It does +> not make any special momentum." + +That is v4d's participation reading turned into an operational preference: a +flat open grants everyone entry, so it produces chop rather than a move. + +Plan: SELL-side on flat-to-gap-down (on Sensex explicitly to target the buyers +sitting on support since yesterday), BUY-side and follow on a gap-up, where "the +buyer becomes safe" and there is nothing left to hunt. On BankNIFTY he notes +there is little pressure on the sellers, so that branch is a **follow of the +drift rather than a seller hunt** -- a distinction the note keeps. + +**Transcription caveat, and a repeat one.** NIFTY's second support arrived as the +same unresolvable "2476" token as the 13 Aug video, so it is **omitted** again. +Twice in two days makes it a consistent ASR failure on one specific number +rather than random noise; the test asserts NIFTY carries exactly one support so +a later tidy-up cannot invent it. Sensex's levels are unchanged from yesterday +(78500/78145, 77500/77200), which is consistent with a session that went nowhere. + +Test updated: `test_shipped_note_matches_august_14_intraday_hunter_plan` +replaces the 13 Aug equivalent and asserts both branch directions plus the +thin-crowd mechanism and the gap-over-flat preference, because those two are +what a summarising edit would drop first. diff --git a/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_premarket.py b/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_premarket.py index 5ff2eb7..c92655b 100644 --- a/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_premarket.py +++ b/Tests/Signal Generators/SL Hunting AI Agent/test_sl_hunting_premarket.py @@ -201,16 +201,17 @@ def test_shipped_note_targets_the_next_TRADING_day_not_the_next_calendar_day(): ) -def test_shipped_note_matches_august_13_intraday_hunter_plan(): - """The committed advisory must match the hand-checked 12 Aug transcript. +def test_shipped_note_matches_august_14_intraday_hunter_plan(): + """The committed advisory must match the hand-checked 13 Aug transcript. This catches a stale prior-session note, an inverted gap plan, or a mistyped chart level before the dated note is injected into the live prompt. - What distinguishes this one is that the seated crowd has FLIPPED to buyers, - and the gap-up branch is a follow rather than a fade -- justified by where - the stops sit relative to the round number. Both directions are asserted - explicitly because a copy-forward would silently keep the previous day's. + What distinguishes this one is that NEITHER side is seated -- he says so for + both BankNIFTY ("the same price level") and Sensex ("would be WRONG" to + claim either). That is the empty-book condition v4f describes, so the note + carries his own mechanism for it (thin crowd -> that is where momentum + goes) rather than letting the agent infer a seated crowd that is not there. """ import os @@ -219,31 +220,32 @@ def test_shipped_note_matches_august_13_intraday_hunter_plan(): note = load_premarket_note(shipped) assert note is not None - assert note.for_date == "2026-08-13" - assert "PfthlsdW2E8" in note.source - # The distinguishing fact: buyers, not sellers, are the seated crowd now. - assert "BUYERS, not sellers, are now the seated crowd" in note.context + assert note.for_date == "2026-08-14" + assert "MhmlrlUEUGI" in note.source + # The distinguishing fact: no decisive crowd on either side. + assert "SAME price level" in note.context assert note.plan[0].startswith("FLAT to GAP-DOWN: identify SELL-side setups") - assert note.plan[1].startswith("GAP-UP above the round number") - # The round-number stop-location reasoning is the whole justification for - # following a gap-up instead of fading it; it must survive verbatim. - assert any("BELOW the round number" in line for line in note.plan) - assert any("not huntable" in line for line in note.plan) - assert any("SENSEX EXPIRY" in line for line in note.plan) - assert len(note.plan) == 7 + assert note.plan[1].startswith("GAP-UP: go WITH the market") + assert any("NOBODY IS DECISIVELY SEATED" in line for line in note.plan) + # The thin-crowd mechanism and the explicit preference for a gap over a flat + # open are the two ideas a summarising edit would drop first. + assert any("where the crowd is THIN" in line for line in note.plan) + assert any("PREFERS a gap to a flat open" in line for line in note.plan) + assert len(note.plan) == 6 assert [level.model_dump() for level in note.levels] == [ { "index": "NIFTY", - # Only ONE support: the transcript's second NIFTY support arrived as - # "2476" and could not be resolved, so it is omitted rather than - # guessed. A missing advisory level is safer than a wrong one. - "resistance": [24500.0, 24600.0], - "support": [24260.0], + # One support again: the transcript's second NIFTY support arrived as + # the same unresolvable "2476" token as 13 Aug, so it is omitted + # rather than guessed. Twice in two days makes it a consistent ASR + # failure on one number, not random noise. + "resistance": [24440.0, 24540.0], + "support": [24275.0], }, { "index": "BANKNIFTY", - "resistance": [58000.0, 58300.0], - "support": [57500.0, 57310.0], + "resistance": [57890.0, 58000.0], + "support": [57500.0, 57320.0], }, { "index": "SENSEX", From 809bb290f6f58ee24ad011ed70409109adcb8ebd Mon Sep 17 00:00:00 2001 From: DoRmAmMu1997 Date: Thu, 13 Aug 2026 22:27:32 +0530 Subject: [PATCH 10/10] fix(sl-hunting): recover the garbled NIFTY support from the video frame The "2476" token that the auto-caption produced on both 13 and 14 Aug is 24,176. It was read directly off the chart rather than inferred. Method, which is now the standard step whenever a level does not parse: force the player to 1080p with setPlaybackQualityRange (it defaults to 854x480, where the axis labels are illegible), seek to the moment he reads the levels, draw the