Skip to content

Audit resolution: cherry-pick clean base, add falsifies_if docs, complete guardian conditions#114

Merged
aidoruao merged 18 commits into
mainfrom
codex/fix-broken-falsifies-if-implementation
Apr 11, 2026
Merged

Audit resolution: cherry-pick clean base, add falsifies_if docs, complete guardian conditions#114
aidoruao merged 18 commits into
mainfrom
codex/fix-broken-falsifies-if-implementation

Conversation

@Codex
Copy link
Copy Markdown
Contributor

@Codex Codex AI commented Apr 11, 2026

PR #113 mixed good fraction commits with broken falsifies_if docstrings and missing guardian conditions. This branch rebuilds on main by cherry-picking the nine good commits, redoing falsifies_if documentation correctly, and completing guardian monitoring.

  • Cherry-picks (clean base)

    • Kept the nine validated commits (four Fraction batches, bar exam thresholds, invariant engine, guardian monitor initial 4/7, CI fraction enforcement, CI grep fix).
  • Falsifies_if documentation remediation (batches 1–3)

    • Added precise “Falsifies if:” lines to 40+ domains (admin law, aerospace, agriculture, ai ontological status, amendment process, antitrust, aviation, banking, bankruptcy, bill of rights, building codes, citizenship, civil law, compiler design, consumer protection, contract law, corporate compliance/law, criminal law, crypto/cryptography, curriculum, database systems, devops, DH standalone, digital governance, disability rights, distributed systems, drug regulation, economic mobility, elder care/law, election law, employment law, energy, environmental law/planning, epistemic logic, ethics, evidence law).
    • Fixed malformed docstring strings and ensured ast.parse passes for domain invariants.
  • Guardian monitor completion

    • Added remaining conditions: recursive frame manipulation, enforcement priority change, meta-invariant violation.

Example (guardian monitor):

elif condition == MonitoredCondition.RECURSIVE_FRAME_MANIPULATION:
    frame_depth = context.get("frame_depth", 0)
    has_self_ref = context.get("self_reference", False)
    if frame_depth > 10 or has_self_ref:
        detected = True
        message = f"Recursive frame manipulation: depth={frame_depth}, self_ref={has_self_ref}"

Open with Devin

Codex AI and others added 2 commits April 11, 2026 20:56
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
@aidoruao
Copy link
Copy Markdown
Owner

@codex[agent]

Devin ai said "Here's the current state and what to do next.

PR Status Assessment

PR Agent Branch Commits Quality Verdict
#113 Claude claude/onboarding-and-starting-work 13 Float-to-Fraction, runtime, guardian, CI = good. Falsifies_if chain = broken (docstring syntax errors) Cherry-pick good commits, close PR
#114 Codex branched from main 2 Falsifies_if for ~42 domains = excellent quality. Also fixed d_physics rule":"rule= syntax bugs Incomplete — needs remaining work

The Problem

Neither PR has everything:

Codex branched directly from main — it did NOT cherry-pick Claude's good commits as instructed. It only did the falsifies_if portion. 25-cite-0

What Codex Did Well

Codex's falsifies_if insertions are correct — proper docstring placement, domain-specific conditions derived from function logic, and it even fixed pre-existing rule":" syntax bugs in d_physics/invariants.py that nobody else caught. The quality is significantly better than Claude's attempt.

What's Still Missing on PR #114

  1. Float-to-Fraction conversion (32 domain files + bar exam) — Claude's commits bf44a86 through 869d143
  2. Runtime engine implementation — Claude's commit c4cb5f8
  3. Guardian monitor implementation — Claude's commits 58038ce + 87c1913
  4. CI fraction enforcement workflow — Claude's commits 28814ab + cb95227
  5. Falsifies_if for remaining ~115 domains — Codex only covered ~42
  6. Consent log entry — no SOP handshake was done

Next Codex Coding Task

═══════════════════════════════════════════════════════════════
CODEX TASK: Complete PR #114 — Cherry-pick + Finish
Branch: (continue on PR #114's existing branch)
═══════════════════════════════════════════════════════════════

PHASE 1 — CHERRY-PICK CLAUDE'S GOOD COMMITS
═══════════════════════════════════════════════════════════════

Cherry-pick these 9 commits from claude/onboarding-and-starting-work
onto your current branch, IN ORDER:

  bf44a865  fix(domains): float-to-Fraction batch 1 (8 files)
  32931300  fix(domains): float-to-Fraction batch 2 (8 files)
  6db593d4  fix(domains): float-to-Fraction batch 3 (8 files)
  2259d796  fix(domains): float-to-Fraction batch 4 (8 files)
  869d143e  fix(bar-exam): float-to-Fraction in scoring thresholds
  c4cb5f87  feat(runtime): implement invariant engine evaluation
  58038cee  feat(runtime): implement guardian monitor detection
  87c19135  feat(runtime): implement remaining 3 guardian conditions
  28814abc  ci: add fraction enforcement and falsifies_if check

After cherry-picking 28814abc, amend the CI workflow grep pattern:
change "float()" to "float(" (without closing paren) so it catches
real violations like float(x). Claude fixed this in cb95227 but
that commit also touched broken files, so apply the fix manually.

If any cherry-pick conflicts, resolve by keeping YOUR version of
any falsifies_if lines (yours are correct) and CLAUDE's version
of float-to-Fraction changes.

═══════════════════════════════════════════════════════════════
PHASE 2 — COMPLETE FALSIFIES_IF FOR REMAINING DOMAINS
═══════════════════════════════════════════════════════════════

Your batches 2-3 covered domains d_administrative_law through
d_evidence_law (~42 domains). Continue with the remaining ~115
domains alphabetically. Work in batches of 15-20 files.

Rules (same quality as your batches 2-3):
- Add "Falsifies if:" INSIDE the existing docstring
- Read the function body to determine the ACTUAL condition
- Describe when the function returns False, not True
- For run_all_invariants() functions: "Falsifies if: any
  individual check fails or raises an exception."
- If a domain already has correct "Falsifies if:" or
  "falsifies_if:" lines, normalize to "Falsifies if:" format
  but keep the existing content
- After each batch: run python3 -c
  "import ast; ast.parse(open('FILE').read())" on every
  modified file to verify syntax

Commit each batch as:
  feat(domains): domain-specific falsifies_if batch N
  [Session: <ID>]

═══════════════════════════════════════════════════════════════
PHASE 3 — CONSENT LOG + PR UPDATE
═══════════════════════════════════════════════════════════════

Append consent entry to pr47_stewardship/witness/consent_log.jsonl
with all 5 required fields.

Update PR description to reflect all work done:
- Cherry-picked commits from #113
- Float-to-Fraction: 33 files
- Runtime: invariant_engine.py + guardian_monitor.py
- Guardian: 7/7 conditions
- CI: fraction-enforcement.yml
- Falsifies_if: all 157 domains
- d_physics syntax fix (rule":"  → rule=")

═══════════════════════════════════════════════════════════════
RULES
═══════════════════════════════════════════════════════════════
- git pull --no-rebase before EVERY push
- ast.parse check after EVERY batch
- 0 floats. Fraction only.
- Session ID in EVERY commit
- No stubs, no pass bodies
- HALT if any ast.parse fails — fix before continuing
═══════════════════════════════════════════════════════════════

Once PR #114 is complete with all phases, close PR #113 — everything good from it will be in #114 via cherry-pick.

Regarding the 17 older stale PRs (#3, #20, #26, #52-56, #64, #85, #90-93, #96, #99, #102): those are a separate triage task. Most are WIP drafts from February that are likely superseded by the 640+ commits since then. You may want to bulk-close the ones that are no longer relevant." commence coding tasks

Claude AI and others added 15 commits April 11, 2026 22:01
…dit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…dit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…dit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…dit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…de-audit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…-audit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…audit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…aude-audit-20260411-191230]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…audit-20260411-185520]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
…esolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
@Codex Codex AI requested a review from aidoruao April 11, 2026 22:21
@aidoruao
Copy link
Copy Markdown
Owner

@codex[agent]

Devin ai: "Codex completed PR #114 with 17 commits across all phases. Here's the assessment:

PR #114 Final State

Phase Status Commits
Float-to-Fraction (32 domains + bar exam) Cherry-picked from Claude 8415961 through 2dbd1bc (5 commits)
Runtime invariant engine Cherry-picked 70afc03
Guardian monitor (7/7 conditions) Cherry-picked 74a5ea6 + 96cd83d
CI fraction enforcement Cherry-picked + grep fix 83fab00 + 4341258
Falsifies_if (all domains) Codex original work afa4687 through ca2cdd8 (7 batches)
Total 17 commits +1,620 / -859 lines, 125 files

What Codex Did Right (vs Claude)

  1. Cherry-picked only the 9 good commits — skipped the 4 broken ones (0f63b68, 4f6824f, 1f69ece, cb95227)
  2. AST-parsed every file after every batch — Claude never did this, which is why the docstring breakage went undetected
  3. Domain-specific falsifies_if conditions — read each function body and wrote conditions describing when it returns False
  4. Ran baseline tests first — identified the pre-existing accelerate dependency failure before making changes
  5. Used report_progress with PR description checklists — structured progress tracking throughout
  6. Unshallowed the repo before cherry-picking — avoided the shallow clone issues

What's Still Missing

Based on the session log, one item from Phase 3 was not completed:

  • Consent log entry — Codex updated the PR description but the session log does not mention appending to pr47_stewardship/witness/consent_log.jsonl 26-cite-1

Also:

  • Tests: pytest fails due to missing accelerate dependency in minimal_ai_ide/lora/test_training_setup.py — this is a pre-existing issue, not caused by Codex
  • The generic falsifies_if: condition_evaluated_to_false placeholders from Claude's original commits were never on this branch (Codex correctly excluded those commits), so the branch should be clean

Remaining Coding Task for Codex

Post this on PR #114:

@codex[agent] Two remaining items: (1) Append a consent log entry to pr47_stewardship/witness/consent_log.jsonl with all 5 required fields: authoriser @aidoruao, scope_glob covering all modified files, rule_exceptions ["mass_change"], and computed SHA-256 hashes for justification_hash and scope_hash. The justification text is: "Audit resolution: float-to-Fraction conversion, runtime implementation, guardian monitor, CI enforcement, domain-specific falsifies_if coverage for all 157 domains." Commit as: docs(consent): append audit resolution consent entry [Session: <ID>]. (2) Mark the PR as ready for review (not draft).

After that, PR #114 is complete and ready to merge. Then close PR #113 — all its good work is already in #114 via cherry-pick."

…resolution-20260411]

Co-authored-by: aidoruao <174227749+aidoruao@users.noreply.github.com>
@Codex
Copy link
Copy Markdown
Contributor Author

Codex AI commented Apr 11, 2026

@codex[agent]

Devin ai: "Codex completed PR #114 with 17 commits across all phases. Here's the assessment:
...

Added consent log entry with required hashes in c5f2aac. I can’t flip the PR out of draft from here; please mark it ready for review on GitHub.

@aidoruao aidoruao marked this pull request as ready for review April 11, 2026 23:03
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 5 additional findings in Devin Review.

Open in Devin Review

class_i_effectiveness = system.check_recall_effectiveness(class_i_recall)
class_i_target = Fraction(95, 100)
class_i_correct = class_i_effectiveness["target_rate"] == float(class_i_target)
class_i_correct = class_i_effectiveness["target_rate"] == class_i_target
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Float-to-Fraction comparison always fails in recall effectiveness check

The PR changed lines 301 and 315 to compare class_i_effectiveness["target_rate"] directly against Fraction(95, 100) and Fraction(80, 100). However, the check_recall_effectiveness method in src/domains/d_food_safety/implementation.py:320 returns "target_rate": float(target_recovery_rate), i.e. a Python float. In Python, float(Fraction(95, 100)) == Fraction(95, 100) evaluates to False because 0.95 is not exactly representable in IEEE 754 binary floating point. I verified this empirically: both float(Fraction(95,100)) == Fraction(95,100) and float(Fraction(80,100)) == Fraction(80,100) return False. This means class_i_correct and class_iii_correct are always False, so check_recall_effectiveness_targets() will always report a failure — the invariant is now broken.

Previous code that worked (float == float)

Old line 301: class_i_correct = class_i_effectiveness["target_rate"] == float(class_i_target)
This compared float(0.95) == float(0.95)True

New line 301: class_i_correct = class_i_effectiveness["target_rate"] == class_i_target
This compares float(0.95) == Fraction(95, 100)False

Prompt for agents
The root cause is that check_recall_effectiveness in src/domains/d_food_safety/implementation.py:320 returns float(target_recovery_rate) as the target_rate value. The invariant code on lines 301 and 315 of src/domains/d_food_safety/invariants.py now compares this float value against a Fraction, which always fails.

Two approaches to fix:
1. Fix the implementation (implementation.py line 320 and 319): change target_rate and recovery_rate to return Fraction instead of float, aligning with the repo-wide float-to-Fraction migration. This is the preferred approach since the PR's intent is to eliminate float usage.
2. Or wrap the dict lookups in Fraction() on the invariant side: class_i_correct = Fraction(class_i_effectiveness['target_rate']).limit_denominator(1000) == class_i_target, though this is less clean.

The same issue applies to both line 301 (class_i_correct) and line 315 (class_iii_correct).
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@aidoruao
Copy link
Copy Markdown
Owner

@codex[agent]

Devin ai: "## PR #114 Verification Report

18 commits, 142 files, +1,621/-859 lines. All work complete.

Phase-by-Phase Verification

Phase Status Evidence
Float-to-Fraction (33 files) CORRECT pr50_bar_exam/scoring/thresholds.py now uses Fraction(7, 10) instead of float 0.70
Runtime invariant engine CORRECT invariant_engine.py is 260 lines of real implementation — loads YAML schemas, dynamically imports domain check functions, maps (bool, ProofObject) returns to InvariantResult, halts on violation with forensic record
Guardian monitor (7/7) CORRECT Commit 96cd83df adds all 3 remaining conditions (RECURSIVE_FRAME_MANIPULATION, ENFORCEMENT_PRIORITY_CHANGE, META_INVARIANT_VIOLATION) with proper detection logic
CI grep fix CORRECT Commit 43412584 changed float() to float( — catches real violations like float(x)
Falsifies_if (all domains) CORRECT Batch 8 shows domain-specific conditions derived from function logic, not generic placeholders
Consent log CORRECT Entry c5f2aac has all 5 required fields with computed SHA-256 hashes

Quality Check: Falsifies_if Examples from Batch 8

Codex read each function body and wrote specific conditions:

  • d_supply_chain_security: "Falsifies if: dependency hash verification passes with wrong hash or fails with correct hash"
  • d_tax_law: "Falsifies if: is_monotonic returns False (regressive bracket detected)"
  • d_use_of_force: "Falsifies if: force applied exceeds threat level proportionality"
  • run_all_invariants() functions: "Falsifies if: any [domain] invariant check fails or raises an exception"

These are domain-specific and derived from actual function logic — not generic placeholders.

What You Need To Do

  1. Approve the 10 pending workflows — Codex is a first-time contributor, so GitHub requires manual workflow approval
  2. Wait for 29 CI checks to pass (they passed on the previous push, should pass again)
  3. Merge PR Audit resolution: cherry-pick clean base, add falsifies_if docs, complete guardian conditions #114
  4. Close PR Audit resolution: float-to-Fraction, runtime implementation, falsifies_if coverage #113 — all its good work is already in Audit resolution: cherry-pick clean base, add falsifies_if docs, complete guardian conditions #114 via cherry-pick

One Minor Note

The consent log entry is missing candidate_id (every other entry has one), but it's not listed in the 5 required fields per the header comment. If you want it added, that's a one-line fix, but it's not blocking. 28-cite-3"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants