Executive Summary

What companies bought was a smarter model. Yet 88% of the pilots built on those models never clear the threshold into production — a figure from IDC and Lenovo's joint research. Swapping in GPT-5 didn't change the outcome. The demo ran smoothly, then collapsed the moment it moved to real operations, and what collapsed in between was never the model's intelligence. This article traces where that 88% leaked out, and why, using 2025 data from IDC, MIT, and Gartner.

The hardest evidence came from MIT: 80% of the work involved in moving a pilot to production is data engineering, governance, and integration. In other words, it's data readiness, not model selection, that decides success or failure. Gartner went a step further, attributing 85% of failed AI projects to data quality problems. The model was fine; what wasn't ready was the data.

So the question to ask isn't "which model should we buy?" but "is our data ready to go to production?" This article translates the places where failure leaked into money, and closes with a five-question self-assessment your organization can answer directly.

The scale of the wreckage and its cause sit together in four numbers: the share of pilots that never reached production, the share of generative-AI pilots that produced no ROI, the investment that vanished in a single year without results, and the one thing most often named as the root of that failure.

88%

never reach production

Most AI pilots fail the move to operations (IDC·Lenovo)

95%

generative AI miss ROI

Share producing no measurable return (MIT GenAI Divide)

$547B

spent without results

Of 2025's $684B AI investment, the part that led nowhere

85%

root cause is data

Share of failed AI projects undone by data quality (Gartner)

1

Seven of Every Eight Vanish

The number 88% is abstract. Put another way: of every eight AI pilots that drew applause in the boardroom, seven disappear without ever entering an operational system. The same IDC analysis reports that the few that did land in production returned an average ROI of 171%. The problem isn't that the model is incompetent. With the very same model, someone makes 171% while most make zero. The fork in the road came after the model.

Convert the scale into money and it gets sharper. Pulling industry analyses together, the world spent roughly $684B on AI in 2025 alone, and $547B of that never translated into measurable results. S&P Global counted the share of companies abandoning AI initiatives at 42% in 2025, a sharp jump from 17% the year before. Large enterprises scrapped an average of 2.3 initiatives each, with an average sunk cost of $7.2M per cancellation.

The interesting part is the shape of the failure. Most didn't blow up spectacularly. Neither cancelled nor successful, projects piled up frozen at the pilot stage. The industry calls this "pilot purgatory" — the demo works, so it's too valuable to kill, but it can't be promoted to operations, so it can't be saved either. It's a zombie state. Budget keeps flowing while value never comes out. A large portion of that 88% is trapped right here, in purgatory.

It's worth noting this isn't one institution's pessimistic tally. A 2025 RAND study analyzing more than 2,400 companies found that 80.3% of AI projects failed to deliver business value. IDC's 88%, MIT's 95%, and RAND's 80% rest on different samples and definitions, yet they point in the same direction. And the trend shows no sign of letting up. Gartner projects that over 40% of agentic AI projects will be cancelled by 2027, and warns that without AI-Ready data, a substantial number of projects risk being scrapped by 2026. The models get better every year, while the scale of failure barely shrinks.

Here's the crux: the 88% failure rate, the 42% abandonment, and the $547B that evaporated all moved almost independently of the model-performance curve. Models improved year after year, but the production-conversion rate didn't climb with them. That's the signal that the bottleneck sits outside the model, not inside it.

2

The Culprit Was Never the Model

If the bottleneck is outside the model, where exactly is it? One sentence from MIT's 2025 report "The GenAI Divide" compresses the answer: 80% of the work needed to move a pilot into production is data engineering, governance, workflow integration, and measurement infrastructure. Choosing the model and tuning the prompt fall into the remaining 20%. The part we talked about most turned out to carry the least weight.

MLOps Venn diagram — ML Ops sits at the intersection of Machine Learning, DevOps, and Data Engineering. The model (ML) is only one of three circles.
▲ Production AI (ML Ops) sits at the intersection of Machine Learning, DevOps, and Data Engineering — the model is just one part | Source: Wikimedia Commons (CC BY-SA 4.0)

Gartner's diagnosis is blunter still. In 85% of failed AI projects, the root cause was data quality. In the same analysis, only 12% of organizations were confident they held data of sufficient quality for AI. That means roughly nine in ten bought the model first, while the data sat unprepared.

So why did everyone invest in the model first? Because the model is visible and easy to measure. Benchmark scores rise, demos run smoothly, and it all shows well to executives. Data infrastructure, by contrast, is dull and invisible. Fixing pipelines and standing up governance is hard to boast about on a single slide. IDC described this dynamic as "panic-driven thinking born at the board level" — competitors are doing it, we can't fall behind, so pilots get pushed through even if the very definition of ROI has to be bent to make them look good.

A single line from Composio's 2025 agent report sums up the paradox most concisely: "Even the best model is useless if it's fed bad data or can't execute actions reliably." The era when the model's ceiling was the problem is over. The place that breaks now is below the model — the pathway by which data reaches it.

Flip the perspective for a moment. Frame AI adoption as a question of "which model should we buy," and you've already started by skipping 80% of the work. The real outcome is decided by "is the data we'll feed that model trustworthy in production?"

4

The Rest Is a Data Problem Too

If access to data and tools is one axis when failure is split in three, are the other two axes any different? Look closely and they're branches from the same root. The remaining two categories Anar Solutions lists are governance gaps and poor use-case selection.

4.1The Governance Gap

Fewer than one in five companies have a formal data-governance framework in place. No governance means no mechanism to control the quality, provenance, and permissions of data. The model doesn't question the data that comes in — it just uses it. When unvalidated data flows in, you get plausible but wrong answers, and the moment those go into operations, trust collapses. This is exactly why agentic content verification matters.

4.2The Wrong Use Case, and Poisoned RAG

Why did organizations pick impressive demo-friendly tasks first? Because demo data comes ready and clean. The operational data, meanwhile, is messy — and so use cases that succeed in a demo collapse in production. The criterion for selection wasn't "which problem matters most" but "which data is already clean."

The trap Composio calls "dumb RAG" runs along the same grain. Dump internal documents into a vector DB indiscriminately, with no preprocessing, and the context flows into the model already poisoned. The result is the "high-confidence hallucination" — wrong, but stated with total assurance. The model isn't lying; poisoned data has simply borrowed its mouth.

Retrieval-Augmented Generation (RAG) architecture — documents are chunked and indexed into a vector DB; at query time, relevant chunks are retrieved and injected as context into the LLM
▲ Full RAG pipeline. Feed documents in without preprocessing and the context is already poisoned — producing high-confidence hallucinations | Source: Wikimedia Commons (CC BY-SA 4.0)

The three categories differ only in name; they point to the same place — data that can't be accessed, can't be controlled, or has been poisoned. Swapping the model won't fix it; making the data production-ready will. It wasn't a third of failures that were a data problem — it was effectively all of them.

5

What the Winners Did Differently

So what did the few that landed in production and returned 171% ROI do differently? McKinsey's 2025 analysis names the clearest distinction: organizations that produced meaningful financial results were twice as likely to have redesigned their end-to-end data workflows before choosing modeling techniques. The order was reversed. Instead of picking the model first and fitting the data to it, they fixed the data first and laid the model on top.

The difference shows up in budget allocation, too. Organizations that delivered results put 50–70% of their AI budget into data preparation. That means money went first to data pipelines, evaluation infrastructure, monitoring, and operations staff — not model licenses or fine-tuning. More was spent keeping data alive in production than on flashy model demos.

Supervised machine learning pipeline in a nutshell — training data flows through an ML algorithm to produce a model, which then takes new input and generates predictions
▲ The ML pipeline starts with training data. Winning organizations finished this first box before touching the model | Source: Wikimedia Commons (CC BY-SA 4.0)

Success tends to look modest. One case that automated back-office document review saved $2–10M a year — not from a headline-grabbing giant agent, but from a narrow task where the data was well prepared. Another striking point in MIT's analysis: success rates were 33% for systems built in-house, versus 67% (double) for those built with specialized vendors. The know-how to make data production-ready is scarcer than the model itself, so those who brought that know-how in from outside succeeded more often.

  • → Design with deployment as the starting assumption — treat production, not the demo, as a given from day one.
  • → Redesign the data workflow before selecting the model.
  • → Allocate more than half the budget to data preparation, evaluation, and monitoring.
  • → Win first on a narrow task where the data is ready, rather than a sprawling one.

If you want a more concrete view of what "AI-Ready Data" actually is, Pebblous's conditions for AI-Ready Data and 5 signals that separate AI-Ready Data are a starting point. They translate what the winning 5% did by instinct into items you can actually diagnose.

6

Is Your Organization Ready?

Everything so far converges on a single question: before you buy the next model, is your data ready to go to production? Answer the five questions below honestly and you'll get a rough fix on where you stand. They ask about data, not the model.

1. Same source? Is the data you used in the pilot the same source as the data you'll use in production — or is it data refined separately just for the demo?

2. Can you reach it? Are the interfaces of the systems the model must connect to (CRM, ERP, DB, external APIs) documented and reliably accessible?

3. Is it fresh? Do you have a production pipeline that accesses real-time, or sufficiently up-to-date, data?

4. Do you watch it? Is there a mechanism that continuously monitors data quality in production and catches anomalies when they appear?

5. Do you control it? Do you have data governance and a hallucination-mitigation system that can stop unvalidated data from flowing into the model?

If three or more of the five draw a "no," that's a signal your next investment should go to data, not the model. Upgrading the model a notch will barely move the production-conversion rate. The place to plug first is the pathway where data is leaking out.

Editor's Note. Pebblous has been doing the work of translating these five questions into diagnosable metrics — measuring what data lives where, how fresh it is, who accesses it, and how its quality changes over time. Whether it's Physical AI or an in-house agent, confirming your data is production-ready before laying the model on top is the starting point the $547B lesson points to. One example of the related work can be seen in our Physical AI data pipeline.

R

References

Industry & Press

Official Sources