Skip to content

Part 3b Phase 1: build the audited MHS harness#15

Merged
AMBRA7592 merged 1 commit into
mainfrom
codex/mhs-phase-1-build
Jul 18, 2026
Merged

Part 3b Phase 1: build the audited MHS harness#15
AMBRA7592 merged 1 commit into
mainfrom
codex/mhs-phase-1-build

Conversation

@AMBRA7592

Copy link
Copy Markdown
Collaborator

Summary

  • add a stdlib MHS converter implementing the committed floor-20, exact-ideology, 2+2 primary filter, categorical-label, and 50-item halt rules
  • add pure Wilson, median/IQR, and deterministic 10,000-resample item-bootstrap statistics fixed at seed 20260718
  • add a dormant Phase-2 harness that verifies the frozen source hash before a guarded pyarrow load, runs the existing instruments unchanged, and emits only a claim-bounded aggregate report/manifest
  • add a 245-row wholly synthetic fixture exercising the ready/halt paths, included/excluded cohorts, label preservation, a known fork, a tied cohort, and undefined geometry
  • CI-pin both schema-valid outputs and all four metric branches without downloading or opening real MHS data

Binding protocol

Implements reports/part-3b-dataset-selection-audit.md as merged at 404bf89. No protocol parameter or claim boundary is changed.

geometry.py is intentionally demo-pinned and has no --data/--out parser. The harness therefore imports its existing arithmetic_mean, geometric_mean, and tv functions directly; disagreement.py and soft_labels.py are called unchanged via --data/--out. No instrument is modified.

Verification

  • bare Python: 37 tests, 34 pass + 3 expected schema skips
  • jsonschema 4.23.0: 37/37 pass
  • synthetic conversion: 50 primary items, 51 reliability-corpus items; 1 known fork, 1 manufactured-consensus case, tied cohort is not a fork, 2 disjoint-support cases remain undefined
  • default artifact byte gate against 404bf89: triage.json, soft-label JSONL/CSV, and governance.jsonl byte-identical; resolution records identical after removing the intentionally dynamic timestamp, with identical replay hashes
  • all eight existing operational/proof scripts pass
  • wrong source checksum halts before creating work/report artifacts

Scope and phase boundary

Exactly seven files: .gitignore, adapters/mhs.py, reports/mhs/{README.md,metrics.py,run_study.py,fixtures/mhs_sample.jsonl}, and test_claims.py.

No real MHS rows or outcomes; no parquet, report.md, or manifest.json; no change to instruments, schemas, essays, demo data, README, or the pre-registration. CI imports no pyarrow. Phase 2 remains unauthorized.

Audit request

Do not merge or run Phase 2. Claude should independently reconstruct fixture eligibility/cohorts/primary items, re-derive the four synthetic metrics, inspect the hash/pyarrow/claim gates, and confirm exact scope plus both CI legs.

@AMBRA7592
AMBRA7592 merged commit af34ebd into main Jul 18, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant