Skip to content

Recover empty contract criteria safely#55

Merged
chriswu727 merged 1 commit into
mainfrom
agent/empty-criteria-recovery
Jul 22, 2026
Merged

Recover empty contract criteria safely#55
chriswu727 merged 1 commit into
mainfrom
agent/empty-criteria-recovery

Conversation

@chriswu727

Copy link
Copy Markdown
Owner

Summary

  • recover an empty compiler criteria list from the bounded explicit user instruction
  • retain the independent critic, capability policy, and criterion-to-execution evidence gates
  • persist recovery provenance in the hashed contract and Receipt, and expose it in the task UI
  • preserve the frozen DeepSeek report without retroactively changing its score

Root cause

An otherwise usable compiler response with criteria: [] failed schema validation before repository evidence or the independent critic could participate, causing an unnecessary step-zero clarification in one accessible-dialog evaluation run.

Safety behavior

Recovery does not auto-accept the instruction. The accessible-dialog fixture locks only because its discovered test source substantiates the recovered criterion. An unrelated no-evidence goal still fails closed. Recovery is bounded to 4,000 characters and malformed non-criteria fields remain invalid.

Verification

  • backend: ruff, format, mypy, 545 tests
  • web/desktop: lint, typecheck, 19 Web tests, 14 Desktop tests
  • production Web and Desktop builds
  • production dependency audit: no known vulnerabilities
  • legacy contract hash compatibility and recovered-provenance tamper tests
  • runtime identity drift against the frozen report is detected as designed

@chriswu727
chriswu727 marked this pull request as ready for review July 22, 2026 03:32
@chriswu727
chriswu727 merged commit 479811d into main Jul 22, 2026
13 checks passed
@chriswu727
chriswu727 deleted the agent/empty-criteria-recovery branch July 22, 2026 03:32

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d554379ff8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

)[:12]
recovered = False
if not criteria:
criteria = _bounded_fallback_criteria(fallback_criteria or [])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not treat test-pass goals as substantiated criteria

When the compiler returns criteria: [], this falls back to the entire user goal as one criterion; for common goals such as Implement CSV export and make all tests pass, _test_previews_cover_contract() skips marker matching for any criterion containing test and pass, so _effective_checks() can promote an unrelated discovered test command as contract evidence even when the tests do not mention the requested behavior. If the critic accepts, the recovered contract can lock on placeholder/stale tests and later verify by passing them, creating a false acceptance for the actual task.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant