Skip to content

[codex] add Self-Harness autoresearch implementation#606

Merged
tractorjuice merged 2 commits into
mainfrom
feat/selfharness-pr
Jun 18, 2026
Merged

[codex] add Self-Harness autoresearch implementation#606
tractorjuice merged 2 commits into
mainfrom
feat/selfharness-pr

Conversation

@tractorjuice

@tractorjuice tractorjuice commented Jun 18, 2026

Copy link
Copy Markdown
Owner

Summary

Adds the Self-Harness autoresearch implementation from the local feat/selfharness-implementation branch, rebuilt onto current main to avoid carrying stale Vibe and OKF reversions.

  • Adds execution trace capture for autoresearch runs.
  • Adds weakness mining over trace/verifier output.
  • Adds harness proposal generation for prompts, agents, hooks, runtime settings, and verifier criteria.
  • Adds harness validation with held-in/held-out scoring and conservative acceptance criteria.
  • Adds the Self-Harness autoresearch program document based on arXiv:2606.09498v1.

Validation

  • node --check plugins/arckit-claude/hooks/autoresearch-tracer.mjs
  • node --check plugins/arckit-claude/hooks/harness-proposer.mjs
  • node --check plugins/arckit-claude/hooks/harness-validator.mjs
  • node --check plugins/arckit-claude/hooks/weakness-miner.mjs
  • git diff --check
  • npx markdownlint-cli2 scripts/autoresearch/program-selfharness.md

tractorjuice and others added 2 commits June 18, 2026 16:59
Implements Zhang et al. (2026) Self-Harness paradigm for ArcKit autoresearch:
- Weakness Mining (Section 3.2): Verifier-grounded failure pattern clustering
- Harness Proposal (Section 3.3): Diverse, minimal candidate generation
- Proposal Validation (Section 3.4, Algorithm 1): Conservative acceptance rule

New files:
- plugins/arckit-claude/hooks/autoresearch-tracer.mjs
- plugins/arckit-claude/hooks/weakness-miner.mjs
- plugins/arckit-claude/hooks/harness-proposer.mjs
- plugins/arckit-claude/hooks/harness-validator.mjs
- scripts/autoresearch/program-selfharness.md

Modified files:
- scripts/converter.py (minor compatibility adjustments)
- extensions/arckit-vibe/scripts/bash/create-project.sh (minor fix)

Expected improvements: 2-3x optimization effectiveness, 15-50% performance gains

Paper: Self-Harness: Harnesses That Improve Themselves (arXiv:2606.09498v1)
https://arxiv.org/abs/2606.09498v1

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
@tractorjuice tractorjuice marked this pull request as ready for review June 18, 2026 17:16
@tractorjuice tractorjuice merged commit 0b14f51 into main Jun 18, 2026
2 checks passed
@tractorjuice tractorjuice deleted the feat/selfharness-pr branch June 18, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant