fix(statements): correct + unpublish the 2026-07-28 weekly (unresolved window, not a 0% hit rate) - #58
Merged
Conversation
added 2 commits
July 29, 2026 12:36
…ot a 0% hit rate This statement is live at `draft: false` and reads "263 alerts, hit rate 0.00". To a reader that says we made 263 calls and got every one wrong. What it actually means is that none of the 263 has resolved yet: measured on the producer host, 111 of 111 alerted markets join `markets` and none carries an outcome, against 97.3% outcome coverage warehouse-wide. Not a bad week and not a data gap. Every statement back to May carried 0.35-0.71, all of them tagged `[politics, macro-finance]`. This is the first Macro-only one — Politics was retired 2026-07-24, and Politics markets (elections, daily events) were supplying essentially the whole resolved sample, while Macro markets resolve months out. `hypothetical_pnl_usd: 0.0` and `top_wallets: []` had a separate cause: both are derived from tables that live only in the slim warehouse, and the producer was reading a warehouse that structurally cannot hold them. Regenerated from the corrected producer against the same 2026-07-22 → 2026-07-28 window. `alert_count`, `period_start`, `period_end`, `hit_rate`, `title` and `draft` are unchanged — 263 alerts reproduces exactly, which is what confirms the window is identical and only the reporting was wrong. summary "hit rate 0.00." → "outcomes pending (0 of 263 resolved)." hypothetical_pnl_usd 0.0 → 199.11 top_wallets [] → 3 entries `hit_rate: 0.0` deliberately stays. It is a required non-nullable number in the schema, and the honest fix — reporting on markets RESOLVED in the window rather than alerted in it — needs a resolution-time backfill that does not exist yet. The summary now carries the denominator so the figure cannot be misread alone. Producer: auditmos/ogsfrompoly 554f537 · refs auditmos/ogsfrompoly#236, #234
… express "0 of 0" CI caught what the correction could not fix, and it was right to. With `hypothetical_pnl_usd` restored to its true 199.11, `findPublishedInconsistencies` fires: alert_count=263 with hit_rate=0 but hypothetical_pnl_usd=199.11 > 0 (0 resolved alerts collapsing 0/0 → 0?) That guard's own comment already names the diagnosis — "the tell of a `0/0 → 0` resolution collapse" — and that is exactly what this window is. It could not fire before only because `hypothetical_pnl_usd` was ALSO 0, from the upstream bug that left the producer reading a warehouse without the derived tables. Fixing that made the pathology visible, which is the guard working as designed. The guard cannot be satisfied honestly under `schema_version: 1`. There is no field that separates "0 of 0 resolved" from "0 of 240 resolved", so a truthful statement for an unresolved window is unpublishable: either `hit_rate: 0` asserts a 0% success rate that did not happen, or the PnL is falsified back to 0 to match. Neither is acceptable, and weakening the guard would discard the one check that caught this. So the statement goes to `draft: true`. Drafts are exempt from the guard and excluded by every consumer (`excludeDrafts` in the index, feeds and statements.md — there is no per-statement route), which takes the false "263 alerts, hit rate 0.00" claim off the live site immediately. The corrected numbers stay in the file so republishing is a one-line flip once the schema can carry a resolved count. Unblocking work: add `resolved_count` and bump to `schema_version: 2` across the producer and this repo, then teach the sanity check that `resolved_count === 0` makes hit_rate vacuous rather than inconsistent. Tracked as auditmos/ogsfrompoly#236. Refs auditmos/ogsfrompoly#234, auditmos/ogsfrompoly#236
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 29, 2026
## [1.17.1](v1.17.0...v1.17.1) (2026-07-29) ### Bug Fixes * **statements:** correct + unpublish the 2026-07-28 weekly (unresolved window, not a 0% hit rate) ([#58](#58)) ([fc5c9a7](fc5c9a7)), closes [auditmos/ogsfrompoly#236](https://github.com/auditmos/ogsfrompoly/issues/236) [#234](https://github.com/auditmos/ogsfrompoly-lp/issues/234) [auditmos/ogsfrompoly#234](https://github.com/auditmos/ogsfrompoly/issues/234) [auditmos/ogsfrompoly#236](https://github.com/auditmos/ogsfrompoly/issues/236)
tkowalczyk
added a commit
that referenced
this pull request
Jul 30, 2026
…minator (#60) `resolved_count` landed in #59, so an unresolved window is expressible: adding `resolved_count: 0` clears `findHitRateInconsistencies`, which is what held this file at `draft: true`. The numbers are unchanged from #58 — regenerated from the warehouse at auditmos/ogsfrompoly@554f537 over the identical window — and its summary's "0 of 263 resolved" is the count itself, so the field states what the prose already did. Publishing it would otherwise have put `Hit rate 0%` back on the live page beside this stat's own caption, "0.50 ≈ a coin flip; above 0.50 is signal" — the exact misreading auditmos/ogsfrompoly#236 exists to stop, on the most-skimmed surface. So the stat now reads `pending` when the denominator is known-empty, with wording that replaces the standing caption rather than appending a caveat to it. `formatHitRate` / `describeHitRate` key off the denominator, never the rate, so a genuine 0-in-favour-of-240 week still reads as a real 0%. Absent stays unknown, never zero: every statement published before the field existed renders exactly as before, verified across all twelve. A known non-zero denominator is now named too ("Measured over the 240 of 263 alerts that have resolved") — publishing a rate without one is what let the 0.00 pass unnoticed for a week. Refs auditmos/ogsfrompoly#236 Co-authored-by: Tomasz Kowalczyk <tkow@Mac-Studio-Tomasz.local>
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 30, 2026
## [1.18.1](v1.18.0...v1.18.1) (2026-07-30) ### Bug Fixes * **statements:** republish the 2026-07-28 weekly, and render the denominator ([#60](#60)) ([792ecdd](792ecdd)), closes [#58](#58) [auditmos/ogsfrompoly#236](https://github.com/auditmos/ogsfrompoly/issues/236) [auditmos/ogsfrompoly#236](https://github.com/auditmos/ogsfrompoly/issues/236)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
2026-07-28-weekly.mdis live atdraft: falseclaiming:To a reader that says 263 calls, all wrong. In fact none of the 263 had
resolved. This PR corrects the numbers and takes the claim off the live site.
What changed
summary… 263 alerts, hit rate 0.00.… 263 alerts, outcomes pending (0 of 263 resolved).hypothetical_pnl_usd0.0199.11top_wallets[]draftfalsetrue— see belowalert_count·period_*·hit_rate·titlealert_count: 263reproduces exactly against the same2026-07-22 → 2026-07-28window, which is what confirms the window is identicaland only the reporting of it was wrong.
Why it lands as a draft rather than a corrected publication
CI caught what the correction could not fix, and it was right to. With
hypothetical_pnl_usdrestored to its true value,findPublishedInconsistenciesfires:
That guard's own comment already names the diagnosis — "the tell of a
0/0 → 0resolution collapse" — and that is precisely this window. It could not fire
before only because
hypothetical_pnl_usdwas also 0, from the upstream bugthat had the producer reading a warehouse without the derived tables
(auditmos/ogsfrompoly#234). Fixing that made the pathology detectable. The guard
worked as designed.
It cannot be satisfied honestly under
schema_version: 1. No field separates"0 of 0 resolved" from "0 of 240 resolved", so a truthful statement for an
unresolved window is unpublishable: either
hit_rate: 0asserts a 0 % successrate that did not happen, or the PnL gets falsified back to 0 to match. Neither is
acceptable, and weakening the guard would throw away the one check that caught it.
Drafts are exempt from the guard and excluded by every consumer (
excludeDraftsin the index, the feeds and
statements.md.ts; there is no per-statement route),so
draft: trueremoves the false claim from the live site immediately. Thecorrected numbers stay in the file, making republication a one-line flip.
Root cause, for the record
hit_rate = in_favor / resolved, scored over alerts emitted in the window.Measured on the producer host: 111 of 111 alerted markets join
markets, 0carry an outcome — against 97.3 % coverage warehouse-wide. Not the join, not
enrichment; the markets genuinely have not settled.
Every statement back to May carried 0.35–0.71, all tagged
[politics, macro-finance]. This is the first Macro-only statement — Politicswas retired 2026-07-24, and Politics markets (elections, daily events) supplied
essentially the whole resolved sample, while Macro markets resolve months out.
So this recurs every week until the schema can express it.
Follow-up that unblocks republishing
Add
resolved_count, bump toschema_version: 2across the producer and thisrepo, and teach the sanity check that
resolved_count === 0makeshit_ratevacuous rather than inconsistent. Then flip this file back to
draft: false.Tracked as auditmos/ogsfrompoly#236.
Verification
poly-track report public weeklyat auditmos/ogsfrompoly@554f537over the identical window
validate_statement(schema_version: 1)Verify (types, test, lint, knip)greenRefs auditmos/ogsfrompoly#234, auditmos/ogsfrompoly#236