Executive Summary

Key figures

Source: arXiv:2607.27191 Table 1, §4, §5, and Nature (2026-08-13)

Every existing way of measuring AI research automation carries its own blind spot. Automatically graded tasks such as RE-Bench and MLE-Bench need a fixed answer to exist in advance, which writes off open-ended research, where the work starts with deciding what to ask. Submitting the output to a conference does cover open-ended research, but there the grader wobbles. In the two self-review experiments NeurIPS ran, in 2014 and 2021, the same papers went back out to a different review panel and roughly a quarter of the accept and reject decisions flipped, with about half the variance in scores traced to reviewer subjectivity. Take conference acceptance as your evidence and you end up measuring ability with a ruler that reads differently each time you pick it up.

Some systems have already cleared that ruler. The AI Scientist, built by Japan's Sakana AI, went public in 2024 and put three papers through workshop peer review, one of which passed the acceptance threshold. An improved version appeared in Nature in March 2026. Zochi, an agent from Intology, made it through the ACL main track. Sayash Kapoor, a co-author of this paper, argues that such records are hard to treat as evidence about the quality of AI research. If acceptance itself moves depending on who reviews, then what the acceptance tells you moves by just as much.

What the CRUX researchers changed instead was the grader. Working with the authors of two NeurIPS 2026 submissions that had not been made public, they handed the agent only the research questions those papers were built around. One dealt with the structure and controllability of personas in large language models; the other was the problem of designing a distribution-shift detector for tabular foundation models. The agent had no access to the original papers, and since those papers were unpublished anywhere, there was no chance of them sitting in the training data either. The authors call this design shadow evaluation.

Grading followed the conference reviewer's format. An overall rating from 1 to 6, strong reject through strong accept, plus separate marks out of 4 for quality, clarity, significance and originality. The one difference is that the grader had spent months digging into that exact problem. Unlike a conference reviewer who reads a paper for the first time and rules on it within days, these readers came to the output already knowing which approaches are dead ends and why.

The agent was given generous resources. Six days per paper, 144 hours, along with $3,000 in Anthropic API credits, GPU credits, full permissions on an AWS Linux virtual machine and open web access. The harness put Claude Opus 4.8 at its highest reasoning-effort setting on top of an OpenClaw scaffold, with tools for spinning up subagents to divide the work and for checking its own progress. Human intervention happened three times: fixing a bug in the harness itself, extending the deadline by 24 hours at a point where the self-review was stuck at weak reject, and asking for another pass on the readability of the output.

The agent pulled off more than anyone expected. Nine of the 11 CRUX co-authors who answered the pre-registration survey expected the agent to get stuck in an error loop it could not escape, and that never happened. Hundreds of hours of GPU experiments ran without trouble, and the literature review was solid. Both original authors judged the initial hypotheses the agent formed on its own to be reasonable and interesting. Five cases in which its own subagents fabricated or distorted results were all caught by the orchestrator before they reached the final draft.

The behaviour evaluation designers worry about most did not show up either. There was no reward hacking, no bending of metrics to make the numbers look good. If anything it went the other way. The agent started out with claims that would have sold more easily, then backed off toward negative results on its own as the evidence weakened. The persona experiments also produced an incidental finding, that narrowly fine-tuning for style does not spill over into broad misalignment.

Then the original authors' scoresheets came back. The overall ratings were 2 and 1 out of 6.

Look at the individual criteria and the deductions are not spread evenly. Among the marks out of 4, originality was the highest for both papers, at 3 and 2, while the criterion that hit bottom differed: clarity scored 1 for the personas paper and quality scored 1 for the TabPFN paper. Building a new dataset and trying a new method earned credit on its own terms. The points fell away at the point where that work has to be turned into research someone can read. The graders rated their own confidence at 4 and 5 out of 5, so these were not low scores handed down out of uncertainty.

The reason for the deductions was not that the experiments failed to run. It was that nothing explained why those particular experiments had been chosen. David Africa, the author of the personas paper, wrote that the experiments and methodological choices were bizarre and hard to follow, and that the results looked like the product of choices made after the fact. His reaction to reading the progress reports is more concrete still: every time the agent dropped another hypothesis, the research got narrower and less interesting.

Viet Nguyen, the author of the TabPFN paper, pointed at a leap in the reasoning. Going from a result in which a handful of signals drawn from the model's internals failed, to the conclusion that no usable signal exists inside the model, is proof by example and a wildly unscientific leap, in his words. The experience of reading the output was a problem in itself. It was impossible to tell quickly what was noise and what mattered, and that assessment landed directly on the clarity score.

The authors sorted the failures common to both runs into five kinds. None of them is an individual bug. Each points at a part of research work that was missing.

• Poor judgment about research quality standards. It dismissed hypotheses prematurely on the strength of underpowered experiments and hand-picked synthetic datasets, and engaged only shallowly with prior work.
• No creative reframing of the problem. Faced with negative feedback, it never restated the question or redesigned the experiment, and instead piled caveats and conditions onto the claim it already had.
• No effective backtracking. It made local course corrections, rerunning experiments and adding robustness checks, but it never stepped back at the project level to reconsider the approach itself.
• No awareness of resources or context. Both runs spent less than half the API budget even though remaining budget and time were visible in real time. The personas run used roughly a third and the TabPFN run 40 percent, then declared themselves done a few hours before the deadline.
• Drift away from instructions. Directions such as a 48-hour minimum exploration period, the page limit and the use of external review credits were acknowledged early on and then let go of during execution.

The third item weighs the most. In the TabPFN run the agent rejected all six of its approaches within the first 14 hours. A human researcher blocked six times in a row would start to suspect the framing of the problem. The agent spent most of the remaining time on the direction it had begun with. It kept changing what to try. It did not change what it was asking.

Instruction drift showed up most plainly in the formal requirements. Both papers came in at 10 pages against the NeurIPS 9-page limit, which at a real conference means a desk reject before anyone reads the content. The body of the personas paper contained no visualisations at all. The original author's paper on the same question had 15. The reference lists ran to 16 entries where the original had 52, and 36 where the original had 69.

For anyone who designs evaluations, the most useful part of this paper sits elsewhere. The agent reviewed its own output throughout the work, 10 rounds in the TabPFN run and 15 in the personas run, and not once did a self-review return an accept. That looks like honest self-assessment, except the reviews stopped at weak reject and the agent locked that state in as the finished product. On the same output the original authors gave 2 and 1. The self-grading had the direction right and the magnitude wrong.

The researchers also checked whether this was a property of one model. They reran the TabPFN task with a GPT-5.6 Sol and Codex combination. Different model, different scaffold, and the failure modes reproduced almost exactly. Only the pattern of resource use went wrong in the opposite direction, burning through the token budget in two days and leaving time unspent. Using a real distribution-shift dataset instead of synthetic data was an improvement over the earlier run, but the list of failure modes did not get any shorter for it. A preliminary run at lower reasoning effort added degraded literature review and writing quality on top of the same failures. The authors expect results to improve as reasoning effort rises, while adding time and resources alone will not close this gap.

One more run fed the failure logs to Claude Fable 5 and had it improve the scaffold itself. That improvement work exhibited the same failures, including putting excessive weight on a single sample. An AI fixing an AI agent fell into the same traps.

Anthropic published a post in June 2026 on how AI is coming to build itself, and OpenAI said in July that GPT-5.6 Sol had helped with post-training a smaller model and saved researchers weeks of work. The story coming out of the industry runs toward automation already being present in research. This paper is close to the first measured data pushing back on that story directly. One caveat: Anthropic's top model has its R&D capabilities deliberately restricted and could not be tested here, and the researchers have signalled follow-up experiments.

The authors also wrote down the limits of their own design. Original-author grading is not blind. The grader already knows the answer they arrived at and knows the output was written by an AI, which leaves room for bias. The sample is two papers, and even among the co-authors the reading of the cause split between a lack of creativity and a fixation in the agent's perception. The paper goes as far as stating that the team's own scepticism about recursive self-improvement may have shaped the design and the interpretation. In the Nature interview, Sayash Kapoor said he does not see full automation of open-ended research as close at hand.

What changed in this experiment was not the agent but the grader. On the same output, the self-review returned weak reject, and the people who had spent months on the problem returned 2 and 1 out of 6. What produced the score was not the quality of the output alone but the choice of who was asked to judge. Put in terms of evaluation data, the source of the label changed and the same system's results turned over.

Organisations introducing agents internally stand in the same place. Once output volume grows past what people can review, an automated grader or a model judge gets attached, and at that moment the credibility of the score transfers to the credibility of the grader. The following three checks hold regardless of domain.

• Who applied the evaluation labels you use now? If a model applied them, has anyone checked how far those labels diverge from a domain expert's judgment?
• Where is the passing bar set? The self-review here stopped at weak reject and locked that in as done, so it is worth checking whether the bar itself has been fixed low.
• Did the run finish with budget and time spent? Declaring completion with resources left over shows up not as a performance problem but as a problem of judging when to stop.

Earlier pieces along the same axis include The Era of AI Writing Scientific Papers on the movement toward AI-authored research, the ICLR 2026 peer review crisis on the review side coming loose, and the data boundary facing AI research agents. The paper itself is at arXiv:2607.27191, and the Nature coverage at d41586-026-02494-5.

Academic papers

• 1.Kirgis, P. et al. (2026). "Can AI agents conduct open-ended AI research? Early evidence from two case studies." arXiv:2607.27191.
• 2.Lu, C. et al. (2026). "Towards end-to-end automation of AI research." Nature 651, 914–919.

News and industry sources

• 3.Hutson, M. (2026). "AI isn't ready to research itself." Nature. DOI: 10.1038/d41586-026-02494-5.
• 4.Anthropic. (2026). "When AI Builds Itself: Our Progress Toward Recursive Self-Improvement and Its Implications."