feat: dead-man gap-day monitor + preflight scripts (S6-004)

docs/paper-validation-runbook.md

@@ -30,7 +30,10 @@ The Azure Functions isolated worker runs **locally on the dev box**, not in Azur
 ## 2. Daily preflight (before the pre-market timer)
 
 Run this check each trading morning **before 5:00 AM PT in winter (PST) / 6:00 AM PT in summer
-(PDT)** — the winter deadline is the earlier one (see section 3):
+(PDT)** — the winter deadline is the earlier one (see section 3) — or run
+`tools/ops/preflight.ps1` (S6-004), which scripts the same checks with bounded timeouts:
+TWS port probe (step 1), gateway health (step 2 — gateway-down is a WARN, not a FAIL),
+worker admin-endpoint 4-function list (steps 3–5), and the snapshots data directory:
 
 1. **TWS paper** is running and logged in to the paper account; API enabled with
    socket port **7497** and trusted IP **127.0.0.1** (Configure → API → Settings).
@@ -174,7 +177,8 @@ KD-005/KD-006 — so its absence is by design, not a gap to fix.)
 
 | Symptom | Likely cause | Action |
 |---|---|---|
-| **Nothing at all** — no report, no alert, on a trading day | Worker (or the whole box) was not running — the dead-man case | **Check this first each morning.** Confirm the worker process is up and the startup banner shows all 4 functions (section 2 step 5). Distinguish crash from never-started: startup banner present in the console = the worker ran and then exited (check the exit reason in the console tail / `%LOCALAPPDATA%\TradingSystem\logs`); no banner at all = it was never started. If the box slept through a timer, that run is gone (no catch-up); log the gap day in the Run Log |
+| **Nothing at all** — no report, no alert, on a trading day | Worker (or the whole box) was not running — the dead-man case | **Check this first each morning.** Confirm the worker process is up and the startup banner shows all 4 functions (section 2 step 5). Distinguish crash from never-started: startup banner present in the console = the worker ran and then exited (check the exit reason in the console tail / `%LOCALAPPDATA%\TradingSystem\logs`); no banner at all = it was never started. If the box slept through a timer, that run is gone (no catch-up); log the gap day in the Run Log. The `TradingSystem-GapDayMonitor` task (2:30 PM weekdays — see the gap-day monitor subsection below) automates this check and alerts when the day's snapshot is missing |
+| `Dead-Man Alert — No EOD Snapshot for <date>` (orange; sent by the **gap-day monitor task**, not the worker) | The 2:30 PM `TradingSystem-GapDayMonitor` task found no entry for today in `data/snapshots.json` — the EOD path never ran (worker down, box asleep, or EOD failure: the dead-man case above). **On a market holiday this is an expected false positive** — the monitor is weekday-only with no holiday calendar (~9/year, accepted by design); ignore it | Holiday → ignore. Otherwise triage the dead-man row above (worker up? banner? local logs), then record the gap day in the Run Log |
 | **Worker IS running, timers fired (visible in log), but no Discord message arrived** | Discord unreachable (or webhook broken) at the moment an ops/risk alert was sent — the alert was dropped (dead-man gap on the alert path) | The ONLY signal is the local worker log (console, or the files under `%LOCALAPPDATA%\TradingSystem\logs`): the signature is a `LogError` line from `DiscordRiskAlertService` matching `"… alert NOT delivered and will not be retried — … AlertDropped=True …"` (rendered console form — capital `T`); search the log files for `AlertDropped=True`. Operator action: treat the logged title as the alert you never received — triage its underlying failure first, then fix Discord delivery (webhook URL valid? Discord up?), and log the incident in the Run Log |
 | No daily report by ~15 min after the EOD timer, but no orange alert either | Report send failed silently is *not* possible without a log — check worker log for report-path warnings; also re-check the dead-man cases above | Worker log around 20:30 UTC (12:30 PM PT winter / 1:30 PM PT summer); `data/snapshots.json` tells you whether the EOD run itself happened |
 | `Broker Connect Failure — …` alert | TWS not running, not logged in, API disabled, or port/client-id mismatch | Match TWS API settings against `IBKR:Host/Port/ClientId` in `local.settings.json`; restart TWS; pre-market failure skips the options sleeve, EOD failure skips the snapshot |
@@ -183,6 +187,45 @@ KD-005/KD-006 — so its absence is by design, not a gap to fix.)
 | Gateway down (health check fails, or log line `Direct API fallback disabled; gateway miss → deterministic rules`) | claude-gateway process not running or CLI session expired | **Expected degrade, not an incident:** regime classification falls back to deterministic rules with zero metered spend. Restart the gateway; check `GET /health/cli` for credential status. Do NOT enable the metered fallback as a remediation |
 | Snapshot missing for a day (`data/snapshots.json`) | Any of the above on the EOD path | Identify which from logs/alerts; record the gap day in the Run Log |
 
+### Gap-day monitor (dead-man) — one-time manual registration
+
+`tools/ops/Check-DailySnapshot.ps1` (S6-004) closes the dead-man blind spot in the first
+triage row: a down worker can never tell you it is down, so an independent scheduled task
+checks each weekday at **2:30 PM local** (≥45 min after the EOD timer in both DST regimes)
+that today's entry exists in `data/snapshots.json`, and posts an orange
+`Dead-Man Alert — No EOD Snapshot for <date>` embed when it does not. The webhook URL is
+read from the gitignored `local.settings.json` (`Discord:WebhookUrl`) and is never logged.
+
+**Registration is a one-time manual operator step** — it is never performed by CI, tests,
+hooks, or agent pipelines:
+
+```powershell
+pwsh -NoProfile -File D:\Source\TradingSystem\tools\ops\Register-GapDayMonitorTask.ps1
+```
+
+Run this as the normal dev-box user — no elevation required; the task is registered to
+run as the current user.
+
+The script is idempotent (re-running replaces the task). Verify with
+`schtasks /query /tn TradingSystem-GapDayMonitor /v /fo LIST`.
+
+At the end of the 12-week run, decommission the task:
+`Unregister-ScheduledTask -TaskName TradingSystem-GapDayMonitor -Confirm:$false`.
+
+**Exit-code legend** (visible in Task Scheduler history — the code tells you what happened
+without opening logs):
+
+| Exit code | Meaning |
+|---|---|
+| 0 | Snapshot present (or weekend — nothing expected) |
+| 1 | Snapshot MISSING; alert skipped (Discord disabled or webhook not configured) — still investigate |
+| 2 | Snapshot MISSING; alert delivery FAILED (Discord unreachable) — investigate both |
+| 3 | Snapshot MISSING; alert delivered — triage per the table above |
+
+Optional controlled drill (renders a real alert without waiting for a genuine gap day):
+`Check-DailySnapshot.ps1 -Date <past-weekday-with-no-snapshot>` — note it in the Run Log as
+a drill. Use `-WhatIf` to evaluate without posting.
+
 ## 6. Locked posture (ops policy)
 
 Restating the repo trading rules as standing ops policy for the entire validation window:

tools/ops/Check-DailySnapshot.ps1

@@ -0,0 +1,163 @@
+<#
+.SYNOPSIS
+    Dead-man gap-day monitor: verifies today's DailySnapshot entry exists (S6-004).
+
+.DESCRIPTION
+    Observability only — reads a JSON file and (on a gap) posts one Discord embed.
+    It never touches TWS, the worker, or any trading state, and is safe to run while
+    the live host is up.
+
+    Snapshot location of record: JsonSnapshotRepository writes
+    Path.Combine(LocalStorage:DataDirectory, "snapshots.json") with DataDirectory="data"
+    (relative), resolved against the worker's working directory. The
+    TradingSystem-FuncHost scheduled task action is
+    `cmd /c cd /d D:\Source\TradingSystem\src\TradingSystem.Functions && func start ...`,
+    so the worker cwd — verified against the registered task on 2026-06-10 — is
+    D:\Source\TradingSystem\src\TradingSystem.Functions and the file of record is
+    D:\Source\TradingSystem\src\TradingSystem.Functions\data\snapshots.json (the
+    -SnapshotPath default). JSON is camelCase with ISO-8601 dates ("date" property).
+
+    Trading-day logic is weekday-only (Mon-Fri). Market-holiday false positives
+    (~9/year) are ACCEPTED and benign: "no snapshot today" on a holiday is expected —
+    the operator ignores the alert (see docs/paper-validation-runbook.md section 5).
+    No holiday calendar, no Polygon call from PowerShell — by design (spec D11).
+
+    Secrets: the Discord webhook URL is read from the gitignored local.settings.json
+    and flows file -> variable -> Invoke-RestMethod -Uri ONLY. It is never written to
+    console, logs, errors, or transcript, and never interpolated into any string.
+
+.PARAMETER Date
+    The trading date to check (default: today). Override for fixture-driven testing
+    and for controlled drills against past dates.
+
+.PARAMETER SnapshotPath
+    Path to snapshots.json (default: the live worker's file of record, see above).
+
+.PARAMETER SettingsPath
+    Path to the gitignored local.settings.json holding Discord:Enabled /
+    Discord:WebhookUrl (default: the live worker's settings file).
+
+.PARAMETER WhatIf
+    Evaluate and print the verdict, but never POST. Exits with the code the real run
+    would produce assuming delivery success (so fixtures are deterministic).
+
+.NOTES
+    Exit codes (diagnosable from scheduled-task history):
+      0 = snapshot present (or weekend — nothing expected)
+      1 = snapshot missing, alert skipped (Discord disabled, settings/webhook absent)
+      2 = snapshot missing, alert delivery FAILED
+      3 = snapshot missing, alert delivered (or would be, under -WhatIf)
+
+    Never auto-registered or run by CI, tests, hooks, or agent pipelines.
+    Registration is an explicit operator action via Register-GapDayMonitorTask.ps1.
+    Requires PowerShell 7 (pwsh).
+#>
+[CmdletBinding()]
+param(
+    [DateTime]$Date = (Get-Date),
+    [string]$SnapshotPath = 'D:\Source\TradingSystem\src\TradingSystem.Functions\data\snapshots.json',
+    [string]$SettingsPath = 'D:\Source\TradingSystem\src\TradingSystem.Functions\local.settings.json',
+    [switch]$WhatIf
+)
+
+Set-StrictMode -Version Latest
+$ErrorActionPreference = 'Stop'
+
+$dateLabel = $Date.ToString('yyyy-MM-dd')
+
+# --- 1. Trading-day check (weekday-only; holiday false positives accepted, see header) ---
+if ($Date.DayOfWeek -in @([DayOfWeek]::Saturday, [DayOfWeek]::Sunday)) {
+    exit 0
+}
+
+# --- 2. Look for a snapshot entry whose date part equals -Date ---
+$found = $false
+if (Test-Path -LiteralPath $SnapshotPath) {
+    try {
+        $entries = Get-Content -LiteralPath $SnapshotPath -Raw | ConvertFrom-Json
+        foreach ($entry in @($entries)) {
+            if ($null -eq $entry -or -not ($entry.PSObject.Properties.Name -contains 'date')) { continue }
+            $entryDate = if ($entry.date -is [DateTime]) {
+                $entry.date
+            } else {
+                [DateTime]::Parse([string]$entry.date,
+                    [System.Globalization.CultureInfo]::InvariantCulture,
+                    [System.Globalization.DateTimeStyles]::RoundtripKind)
+            }
+            if ($entryDate.Date -eq $Date.Date) { $found = $true; break }
+        }
+    }
+    catch {
+        # Unreadable/malformed file counts as missing — still a useful dead-man signal.
+        Write-Warning "Could not parse snapshot file ($($_.Exception.GetType().Name)); treating as missing."
+    }
+}
+
+if ($found) {
+    Write-Host "OK: DailySnapshot entry present for $dateLabel."
+    exit 0
+}
+
+Write-Warning "GAP: no DailySnapshot entry for $dateLabel (weekday). If today is a market holiday this is a benign false positive — see runbook section 5."
+
+# --- 3. Read Discord settings from the gitignored local.settings.json ---
+$discordEnabled = $false
+$webhookUrl = $null   # flows file -> this variable -> -Uri only; NEVER logged or interpolated
+if (Test-Path -LiteralPath $SettingsPath) {
+    try {
+        $cfg = Get-Content -LiteralPath $SettingsPath -Raw | ConvertFrom-Json
+        $discordEnabled = ([string]$cfg.Values.'Discord:Enabled') -eq 'true'
+        $webhookUrl = [string]$cfg.Values.'Discord:WebhookUrl'
+    }
+    catch {
+        Write-Warning "Could not parse settings file ($($_.Exception.GetType().Name))."
+    }
+}
+else {
+    Write-Warning 'Settings file not found.'
+}
+
+if (-not $discordEnabled -or [string]::IsNullOrWhiteSpace($webhookUrl) -or $webhookUrl -eq 'YOUR_DISCORD_WEBHOOK_URL') {
+    # Name the exact skip condition so the operator knows what to fix (message-level only; URL value never echoed).
+    $skipReason = if (-not $discordEnabled) {
+        "Discord is disabled in config — set Discord:Enabled to 'true' in local.settings.json"
+    }
+    elseif ([string]::IsNullOrWhiteSpace($webhookUrl)) {
+        'webhook URL is blank — set Discord:WebhookUrl in local.settings.json'
+    }
+    else {
+        'webhook URL is still the placeholder — replace YOUR_DISCORD_WEBHOOK_URL with the real webhook in local.settings.json'
+    }
+    Write-Warning "Alert skipped: $skipReason. Snapshot is still MISSING — investigate per runbook section 5."
+    exit 1
+}
+
+# --- 4. Alert (orange embed; date + runbook pointer only — no paths, no config values) ---
+$title = "Dead-Man Alert — No EOD Snapshot for $dateLabel"
+if ($WhatIf) {
+    Write-Host "WhatIf: would POST '$title' to the configured Discord webhook (not sent)."
+    exit 3
+}
+
+$payload = @{
+    username = 'TradingSystem Ops'
+    embeds   = @(
+        @{
+            title       = $title
+            description = "No DailySnapshot entry was written for $dateLabel. The worker may be down (dead-man case) — triage per docs/paper-validation-runbook.md section 5, first row. If $dateLabel was a market holiday, this is an expected false positive: ignore."
+            color       = 15105570   # orange — operational failure class
+        }
+    )
+} | ConvertTo-Json -Depth 5
+
+try {
+    Invoke-RestMethod -Uri $webhookUrl -Method Post -ContentType 'application/json' `
+        -Body $payload -TimeoutSec 10 | Out-Null
+    Write-Host 'Alert delivered.'
+    exit 3
+}
+catch {
+    # Exception text could echo the request URI — report the type name only.
+    Write-Error -ErrorAction Continue "Alert delivery FAILED ($($_.Exception.GetType().Name)). Snapshot is still MISSING for $dateLabel."
+    exit 2
+}

tools/ops/Register-GapDayMonitorTask.ps1

@@ -0,0 +1,69 @@
+<#
+.SYNOPSIS
+    Registers the TradingSystem-GapDayMonitor scheduled task (one-time MANUAL operator step).
+
+.DESCRIPTION
+    Registers a Windows scheduled task that runs Check-DailySnapshot.ps1 on weekdays at
+    2:30 PM local time — at least 45 minutes after the EOD timer in BOTH DST regimes
+    (EOD = 1:30 PM PDT / 12:30 PM PST), so the snapshot has had time to land.
+
+    Idempotent: re-running unregisters the existing task and registers it again
+    (useful after moving the repo or changing the schedule).
+
+    Quoting note: the -File path is wrapped in literal double quotes inside the
+    argument string (the S5 TradingSystem-Azurite cmd-quoting bug is the cautionary
+    precedent). Verify after registration with:
+        schtasks /query /tn TradingSystem-GapDayMonitor /v /fo LIST
+
+.PARAMETER TaskName
+    Scheduled task name (default: TradingSystem-GapDayMonitor — repo task convention).
+
+.PARAMETER At
+    Local wall-clock run time (default 2:30 PM). Do not move earlier than 2:15 PM:
+    the EOD timer fires at 1:30 PM local in PDT (12:30 PM in PST), and the monitor
+    must stay >=45 minutes behind it in both DST regimes.
+
+.PARAMETER ScriptPath
+    Absolute path to Check-DailySnapshot.ps1 (default: sibling of this script).
+
+.NOTES
+    NEVER invoked by CI, tests, hooks, or any agent pipeline — registration is a
+    human/runbook action (docs/paper-validation-runbook.md, gap-day monitor subsection).
+    Requires PowerShell 7 (pwsh) and rights to register scheduled tasks.
+#>
+[CmdletBinding()]
+param(
+    [string]$TaskName = 'TradingSystem-GapDayMonitor',
+    [string]$At = '14:30',
+    [string]$ScriptPath = (Join-Path $PSScriptRoot 'Check-DailySnapshot.ps1')
+)
+
+Set-StrictMode -Version Latest
+$ErrorActionPreference = 'Stop'
+
+$ScriptPath = (Resolve-Path -LiteralPath $ScriptPath).Path
+$repoRoot = Split-Path (Split-Path (Split-Path $ScriptPath))   # tools/ops/.. /.. = repo root
+$pwshPath = (Get-Command pwsh).Source
+
+# Literal quotes around the -File path — the Azurite task's quoting bug is the precedent.
+$arguments = "-NoProfile -ExecutionPolicy Bypass -File `"$ScriptPath`""
+
+$action = New-ScheduledTaskAction -Execute $pwshPath -Argument $arguments -WorkingDirectory $repoRoot
+$trigger = New-ScheduledTaskTrigger -Weekly -At $At -DaysOfWeek Monday, Tuesday, Wednesday, Thursday, Friday
+$settings = New-ScheduledTaskSettingsSet -StartWhenAvailable -ExecutionTimeLimit (New-TimeSpan -Minutes 5)
+
+$existing = Get-ScheduledTask -TaskName $TaskName -ErrorAction SilentlyContinue
+if ($existing) {
+    Write-Host "Task '$TaskName' already exists — unregistering for idempotent re-register."
+    Unregister-ScheduledTask -TaskName $TaskName -Confirm:$false
+}
+
+Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Settings $settings `
+    -Description 'TradingSystem dead-man gap-day monitor: checks for today''s DailySnapshot entry, weekdays 2:30 PM local (S6-004). Exit codes: 0 present/weekend, 1 missing+skipped, 2 missing+alert-failed, 3 missing+alert-delivered.' | Out-Null
+
+Write-Host "Registered scheduled task '$TaskName':"
+Get-ScheduledTask -TaskName $TaskName | Select-Object -ExpandProperty Actions |
+    Format-List Execute, Arguments, WorkingDirectory
+Get-ScheduledTask -TaskName $TaskName | Select-Object -ExpandProperty Triggers |
+    Format-List StartBoundary, DaysOfWeek
+Write-Host "Verify quoting with: schtasks /query /tn $TaskName /v /fo LIST"