SAGE

Self-correcting · Autonomous · Grounded · Experimenter

One Reflection Is Not Enough:
Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution

When an autonomous experiment fails, don't guess once. Diagnose many causes, verify the most critical, and route the fix to the right level of abstraction.

---

Overview

Autonomous research agents can draft hypotheses, write code, run experiments, and produce papers, but they remain brittle when experiments fail. In current pipelines, recovery is delegated to a single free-form reflection: a rich trajectory of metrics, logs, and design choices is compressed into one verbal critique, which leads either to localized trial-and-error or to hard pivots that discard useful context.

SAGE (Self-correcting, Autonomous, Grounded Experimenter) tackles this failure-recovery bottleneck. Its core mechanism, Multi-Hypothesis Failure Attribution (MHFA), treats recovery as structured causal diagnosis: a divergent generator proposes several evidence-grounded explanations for a failed run, an independent critic scores their severity and support, and a deterministic router maps the top verified cause to a hierarchy-aware intervention (method refinement, protocol redesign, or hypothesis pivot). A two-stage numeric-grounding mechanism then checks every reported table value against measured results and redacts unverified cells.

Monolithic reflection compresses a failure into one guess and gets trapped in local trial-and-error (left). MHFA generates and ranks multiple causal hypotheses and routes the top cause to the right abstraction level (right).

Key ideas

• Multi-hypothesis attribution, not one critique. Recovery preserves competing, evidence-grounded explanations before committing to a repair, operationalizing the method of multiple working hypotheses for autonomous agents.
• Diagnosis decoupled from action. A deterministic router maps a verified structural cause to an intervention at the appropriate level; the same diagnosis always yields the same action, so recovery is fully auditable.
• Honest by design. A data-sufficiency verdict clamp protects valid-but-weak results from false pivots, and the grounded-reporting layer redacts any number it cannot trace to a measurement. When no real metric can be obtained, SAGE abstains from writing a paper rather than fabricating one.

SAGE runs eight phases (A–H). At Analysis (F), zero-metric execution failures take an explicit repair path; structural failures enter MHFA, which generates, ranks, and data-grounds hypotheses before a deterministic router selects the repair. A grounding manifest and sanitizer constrain the reported numbers.

Highlights

Evaluated on a 12-topic, five-domain subset of ARC-Bench (machine learning, statistics, quantum computing, biology, high-energy physics):

	SAGE	reflection baseline
Metric-bearing recovery	11 / 12 (92%)	5 / 12 (42%)
Calibrated AR-Eval (/10)	6.75	5.00
Blind human PhD review, ML topics (Overall /10)	5.67	3.00

• In a blind, uniform evaluation of complete artifacts, SAGE is competitive with AI-Scientist-v2 while significantly outperforming the reflection baseline, with gains concentrated in code development and execution.
• Blind human experts (PhDs) independently rank SAGE first on every ML topic, and their scores agree with the automated judge about as well as two strong LLM judges agree with each other.

Installation

git clone https://github.com/JieMaMagic/SAGE.git
cd SAGE
pip install -e .          # core; use  pip install -e ".[all]"  for web search, figures, and PDF export

Python 3.11+. Generated experiments run in a sandbox (optional GPU via CUDA_VISIBLE_DEVICES); point experiment.sandbox.python_path in the config at the interpreter that should run them.

Environment variables

All keys are read from the environment; none are stored in the config.

Variable	Used for	Required
OPENAI_API_KEY	main LLM calls (endpoint set by llm.base_url; the exact variable name is whatever llm.api_key_env is set to)	yes
TAVILY_API_KEY	web / literature search	optional
GEMINI_API_KEY (or GOOGLE_API_KEY)	figure generation	optional
HF_TOKEN	Hugging Face datasets / models	optional
GITHUB_TOKEN	code search	optional

export OPENAI_API_KEY=sk-...
export TAVILY_API_KEY=...     # optional
export GEMINI_API_KEY=...     # optional

Quickstart

# self-correcting run on a single topic
researchclaw run \
  -c config.example.yaml \
  -t "Compare dropout strategies for shallow tabular MLPs" \
  -o runs/my_first_run \
  --auto-approve

# environment / config health check
researchclaw doctor

Each run emits experiment code, machine-readable results, a per-topic self-correction trace, and a grounded paper draft.

Reproducing the paper

CUDA_VISIBLE_DEVICES=0 researchclaw run -c config.example.yaml -o runs/mhfa_ML01 --auto-approve

Each run writes its experiment code, results, self-correction trace, and grounded paper draft to the output directory. The evaluation artifacts reported in the paper (per-topic recovery traces, AR-Eval scores, the human-review workbook, and the threshold-sensitivity replay) are provided with the paper's supplementary material.

Citation

@article{ma2026sage,
  title   = {One Reflection Is Not Enough: Self-Correcting Autonomous Research
             via Multi-Hypothesis Failure Attribution},
  author  = {Ma, Jie and Chu, Binfei and Gao, Jie and Zhang, Jinlu and Ma, Yiwei
             and Tan, Yi and Ji, Jiayi and Sun, Xiaoshuai and Ji, Rongrong},
  journal = {arXiv preprint arXiv:2606.31478},
  year    = {2026}
}

Acknowledgements

SAGE builds on AutoResearchClaw; we thank its authors and the ARC-Bench benchmark, a subset of which we use for evaluation. Released under the MIT License.