Executive Summary

Key Figures

Source: Ross et al., Why Better Models Can Create Riskier Systems, arXiv:2609.04373 (2026)

Choosing a model usually means reading a scoreboard. Swap in the model with the higher score and the service built on it is expected to improve by about as much. For one model in one seat, that arithmetic holds. This study takes aim at the other situation, where hundreds or thousands of models of much the same kind sit in one market, or at one review desk, at the same time. The authors set that scene in their opening line: "Large language models (LLMs) are being deployed at scale in consequential real-world systems, from financial markets to content moderation to hiring."

In that setting there is a second thing to watch besides individual accuracy, which is how closely the participants' errors move together. Errors that scatter average out as you add more participants, and errors that lean the same way do not. The team's hypothesis starts here: models that share much of their training data and their architecture extract the same patterns in the same way as they grow more capable, and so come to act more like one another. The word hypothesis should stay attached. The authors write it as one rather than as an established mechanism, and measure only the part where the overlap grows: "We hypothesize that shared training and architectures can lead more capable LLMs to behave more similarly, creating correlated actions that do not diversify away."

The named phenomenon appears when you lay those two trends on top of each other. A more capable model misses by less on its own task. At the same time it overlaps more with its neighbors. Where the second grows faster than the first shrinks, each participant gets more accurate while the risk the whole system carries goes up. The authors call that reversal the capability paradox, and they check it in three steps inside a simulated stock market.

Of the two, the market stage holds a single asset. Over 100 rounds the fundamental value of that asset is shocked twice: at round 33 it jumps from 100 to 120, and at round 66 it falls from 120 to 90. Three kinds of participant take part. LLM traders place orders after looking at the price, the recent history, their own inventory and a fair-value signal. Noise traders supply liquidity with random orders that average to zero. A market maker quotes with a fixed 0.5% spread. The price updates by taking net order flow divided by the number of participants, multiplying it by a depth coefficient, and adding the result to the previous price. A small coefficient means a deep market that absorbs orders, and a large one means a shallow market where the same orders swing the price hard.

No financial knowledge enters the way capability is measured. The two yardsticks the team uses are the MMLU-Pro score and the LMSYS Chatbot Arena ELO rating, and both measure general reasoning and knowledge. So the capability in this experiment means scoring highly on general tasks rather than trading well. The model table in the appendix runs from Llama-3.2-3B at 34.7% on MMLU-Pro up to Claude Sonnet 4.6 at 88.0%. Models from Meta, OpenAI, Google and Anthropic are mixed together, and seven of them go into the panel experiment that measures correlation, giving twenty-one pairs.

There are three questions. The first of them never runs the market. Sixty-four scenarios, with the gap to fundamental value, the momentum, the volatility and the analyst view varied a little at a time, are put to each model separately, and the five responses collected per scenario are averaged before pairs are compared. The two later questions are the ones that actually run the 100-round market. The LLMs that go into it are one kind at a time. Each condition fills the population with a single model and runs five independent seeds, dropping to three seeds for the condition with planted commentary. So the overlap between models is measured outside the market, and the market numbers show where the price goes once one model becomes many.

The three research questions of the study, set side by side. Source: arXiv:2609.04373, sections 3 and 4.

Two models placing the same order is not a problem in itself. When the price sits below fundamental value, buying is the right move for both, and agreement of that sort is a sign the market is working. So the team splits each model's behavior in two. A corrective component, the part explained by the gap between price and fundamental value, is estimated by a separate regression for every model and taken out, and what remains is treated as the non-corrective component. Pairwise correlation is measured between those residuals. The design keeps only the part where two models go wrong in the same place and in the same direction.

Both measures pointed the same way. Pairs with higher average capability shared more of their residuals, with t=3.94, p=0.001 and R² of 0.475 on Arena ELO, and t=4.88, p<0.001 and R² of 0.580 on MMLU-Pro. The variable for coming from the same company reached significance in neither setting, at p=0.39 on ELO and p=0.28 on MMLU. The authors call this the most uncomfortable part of what they found, and write it like this: "The most uncomfortable implication of our results is that the natural response to correlated failures — deploying more diverse models from more providers — does not address the dominant driver." Their reading of the coefficient follows in the results: "two high-capability models from different companies are more correlated with each other than two low-capability models from the same company."

One line of variance algebra explains why that overlap sets a floor under the risk. The variance of the average residual across N participants is a sum of two terms. The first is divided by N and heads toward zero as participants are added, while the second is the average correlation multiplied by the residual variance and stays where it is however large N grows. The floor the team measured is 0.010, roughly half of 0.021, the residual variance of a single model. Even if individual models improve and 0.021 comes down, the floor rises whenever the average correlation climbs faster.

The paper attaches its own caveats to that floor, listing three conditions under which the number does not transfer cleanly. Across a regime shift, where fundamental value jumps in one step, transients specific to that moment sit on top, so the floor describes ordinary noise rather than those transients. Where residuals run the same way over time, long-run variance comes out larger than a calculation assuming independence. Where cash and inventory limits bind often, a linear prediction of variance is only an approximation. The second condition points the other way. In a market where residuals persist, the real floor sits above 0.010.

Up to this point overlap looks like something to get rid of. The second finding runs the other way. As the LLM share of market participants was raised from 0 to 1 in steps of 0.2, the distance between the price and fundamental value fell steeply and monotonically. In a market where every trader was an LLM, the remaining distance was smallest for Gemini 2.0 Flash Lite at around 1.33, with Claude Haiku 4.5 at 1.69 and Claude Sonnet 4.6 near 2.10. When the judgment points at the right answer, many participants moving the same way is price discovery. Overlap, in other words, is not the flaw.

The third experiment plants false information in the same market. Eight pieces of market commentary arrive each round, all of them insisting in unison that the fair-value signal on screen overstates the true value by 25 to 30%. If the participants were wrong in different ways, claims like that would cancel. Where they overlap, nothing cancels. This time the price deviation grew as the LLM share grew, reaching five to seven times the baseline in the market made entirely of LLMs. The authors describe the state as price discovery close to inverted, and say such a market is "strictly worse than one populated by uninformed noise traders alone."

Directional accuracy, taken on its own, shows the paradox once more. Under the planted commentary, Gemini 2.0 Flash Lite got the direction right 68% of the time, Claude Haiku 4.5 54%, and Claude Sonnet 4.6 42%. Of the three, the one with the highest general-capability score came last, and its figure sits under the 50% a coin flip would deliver. If a model accepts a false premise and reasons consistently from it, the better its reasoning the further it travels.

All of the evidence here comes from simulation. The authors say as much in the closing line of their abstract: "Whether the same dynamics arise in other domains is an open empirical question." Two results of a similar shape have already arrived from other teams using other methods.

One is a record of money moving in a real market. The InvestLogicBench team had models trade against live prices for seven weeks, from November 19, 2025 to January 9, 2026. Their baseline capability scores put Claude Sonnet 4.5 at 77.2 and GPT-5 at 73.5, near the top, and seven weeks later the portfolios stood at $9,901.1 for GPT-5, a loss of 1.0%, and $10,072.8 for Claude Sonnet 4.5, a gain of 0.7%. The two models rotated into risk-off positions prematurely at much the same time. The team puts that down to consensus bias and reasoning fragmentation. The first of those is a tendency to converge on the statistical average of the training data, which in a market amounts to reciting a consensus the price already holds.

The other comes from outside finance. A team working on multi-agent security reported that raising the capability of worker agents lifted the attack success rate across the system from 18.4% to 63.9%, peaking at 94.4%. Their name for it is the capability paradox as well. The cause was linguistic confidence. A more capable worker agent interprets a malicious request as a legitimate piece of business with more assurance and passes it up, and the manager agent takes that assurance as grounds for execution. Mediation analysis attributed 74% of the effect to that path. The fix is suggestive too: pairing workers of asymmetric capability, so that their vulnerabilities differ, dropped the attack success rate from 52.8% to 2.0%.

There are doubts about the first finding too. A review posted on the paper-review platform Pith calls the framework itself simple and correct while entering a reservation about how that finding was measured. The team regressed each model's behavior on the gap between price and fundamental value alone to strip out the corrective component, and the sixty-four scenarios from Section 2 carried other signals too. If a capable model read those signals better, that would be correct corrective behavior counted as non-corrective correlation. The review adds that seven models and twenty-one pairs give little statistical power, which makes the null result on the shared-company variable weak evidence as well. The relationship between rising capability and overlapping behavior is better handled as early evidence than as settled fact.

What follows is not in the paper. The most widely used way of splitting AI risk is to add suppliers. Teams attach models from two or three places so they are not tied to one vendor, and expect one to hold things up if another goes down. That expectation is what this measurement shakes. Even with the companies mixed, if the models were comparably capable, the overlap did not shrink, at least not in this experiment. None of which makes vendor diversification useless. Contract risk, pricing risk and availability risk really are divided by it. The risk of behavior going wrong in the same direction, though, has to be managed on a different axis.

Where that axis lies is where the paper's hypothesis points. The reasons the authors give for models resembling each other are overlapping training data and a shared tuning process that fits human preferences. If that is right, mixing at the top helps little when nothing is separated underneath. Their own recommendations go toward training aimed at behavioral diversity, designs that perturb inputs, and rules requiring disclosure of model provenance, rather than toward buying more models. They acknowledge one more thing on top: "There is also a collective action problem: individual deployers are incentivized to use the most capable available model, even as aggregate welfare would be better served by a more heterogeneous population."

One more recommendation concerns measurement. The paper argues that "AI governance needs population-level correlation audits alongside individual model evaluations," and points to the way financial regulators look at exposures running across institutions alongside the soundness of each one. Every bank can look fine on its own while all of them hold the same asset, and that concentration has to be counted separately. The proposal is to look at a population of models the same way.

There is something countable right now on the data side too. In Section 4 the two experiments diverged over the information fed into the models rather than over the models themselves. The more agents share one misinformed environment, the less that misinformation cancels and the more it amplifies. That leaves three questions to ask.

• How many separate inputs do our agents see? Three models are attached, but are all three being fed the same search index, the same news feed and the same bundle of internal documents?
• When contamination enters those inputs, who notices first? If every agent trusts the same source, none of them can act as a check on the others.
• Are we watching only the average accuracy of our models, or also how often they go wrong in the same place and the same direction? The second does not show up in an evaluation of one model.

Academic Papers

• 1.Ross, J., So, E., De Simone, Z., Pozniak, C., Lo, A. W. (2026). "Why Better Models Can Create Riskier Systems: Evidence from LLM Agents in Financial Markets." arXiv:2609.04373.
• 2.Jiang, Y., Zou, J., Jiang, R., Lin, Z., Yu, X., Huang, Q., Jia, S., Dai, S. (2026). "Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalized Financial Agents." arXiv:2608.06108.
• 3.Liu, Q., Song, R., Ye, S. (2026). "The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure." arXiv:2605.17480.

Review

• 4.Pith.science. (2026). "Why Better Models Can Create Riskier Systems — Review."