Executive Summary
The AI infrastructure story of summer 2026 is fixed on GPUs and power. Just outside that frame, another axis has quietly moved. On Intel's April earnings call, CEO Lip-Bu Tan said the ratio of CPUs to GPUs used to be one to eight, is now one to four, and could move toward parity or beyond. In July the CFO added that it is almost at parity and could skew further toward CPUs on a unit basis. In the same window, Amazon CEO Andy Jassy wrote in his shareholder letter that two large customers had asked to buy all of AWS's 2026 Graviton capacity, and that Amazon had turned them down. Graviton is a CPU.
Why CPUs, specifically? Researchers from Georgia Tech and Intel profiled five agent workloads and found that in the tool-dominated ones, up to 88 percent of runtime drained into CPU-side tool processing. Open that segment and you find retrieval, summarization, file I/O and preprocessing. Separate work shows that on long-sequence inference, tokenization alone can consume up to half of total latency, and that giving a serving stack enough CPU improves time to first token several times over without adding a single GPU. What the agent waits on, in other words, is not the model. It is data.
Summarizing that as "we are short on CPUs" gets it wrong, though. In the same measurements, average CPU utilization during agent execution sits around 10 percent. The cores are not scarce; the data path is narrow. Cloud price lists do not back the shortage story either. The visible pressure is still on GPUs, while general-purpose instance pricing has held steady or fallen. So the conclusion here is not "buy more CPUs." It is a question: has anyone in your organization ever counted, in seconds and in dollars, the data-preparation share of your agent budget?
Four numbers carry this report. The first three point toward a bottleneck that has moved outside the accelerator. The fourth says that movement must not be read as a CPU shortage. Where those four appear to contradict each other is exactly what this piece is about.
CPUs per GPU, as stated by Intel
(Apr 2026 → Jul 2026)
of runtime spent on CPU-side tool
processing in tool-dominated workloads
time-to-first-token improvement from
adequate CPU alone, with no added GPU
and yet: average CPU utilization
during agent execution
This is the third piece in a series on AI compute resources. The first, "The Bill for Spare Servers," located the bottleneck in overestimated demand; the second, "AI Hit a Wall This Summer," located it in electricity. This one is not about the size of the bottleneck but about its location.
Some Documents Are Firmer Than Hearsay
This story began with a single scoop. On August 7, 2026, The Information reported that at an internal meeting in May, AWS leadership told engineers to conserve capacity by every available means. Teams were given deadlines to cut usage before year-end, and were asked to tear down idle EC2 instances they no longer needed so that the capacity could go to customers. Both AI-chip servers and CPU servers were reportedly in scope. One engineer said CPU server capacity that used to arrive within hours now takes days.
The more appealing a story is, the more honest it is to show its sourcing first. This report sits behind a paywall, and its core evidence is an account from anonymous engineers. Everything that followed, including the Tom's Hardware write-up, is a re-report of that same original, so no one should inflate the source count by saying "three outlets reported it." As of this report's writing on August 11, 2026, no independent confirmation from a Reuters- or Bloomberg-tier newsroom had appeared. The Information carried its own caveats: the shortfall was largely a spot-instance problem, and one consultant said there was no shortage in contracted capacity.
AWS pushed back. Nothing has changed in how it manages compute resources, the company said, and it continues to meet the "overwhelming majority" of demand from both internal teams and customers. A spokesperson described demand for its services, EC2 included, as "incredibly strong and continuing to grow," and added that reclaiming idle instances and rightsizing are long-standing practices, so reading such efficiency measures as evidence of a new capacity constraint is "incorrect."
What AWS denied is the causal reading that its operating posture fundamentally changed, not the existence of the efficiency measures. That idle instances were reclaimed and that they were reclaimed because of a capacity crunch are two separate claims, and the company contested the second. Blur that distinction and you end up misquoting both sides.
1.1Why anonymous accounts are not the anchor here
Adjudicating whether that report is true is not what this piece is for, because it does not need to be. That AWS faces CPU capacity constraints stands without leaning on anonymous sourcing at all. CEO Andy Jassy wrote it down himself in his 2025 letter to shareholders, published April 9, 2026.
Two large customers asked to buy "all" of AWS's 2026 Graviton capacity, and Amazon declined out of fairness to other customers. AWS added 3.9 GW of new power capacity in 2025 alone, plans to double its total power capacity by the end of 2027, and spends capital 6 to 24 months before the meter starts running.
Graviton is a CPU that AWS designs itself. A statement that two customers wanted to buy a full year of CPU capacity outright, and that the cloud provider had to say no, is far firmer than one engineer's sense of how long queues have gotten. It is a written statement from a chief executive to shareholders. So the anchor changes here. The wait-time anecdotes quoted below stay qualitative and never make it into a table or a metric; the load-bearing weight is split between Jassy's sentence and Intel's earnings calls in the next section.
The Ratio Intel Put on the Record
If the AWS side straddles hearsay and a shareholder letter, the Intel side is much simpler. Named executives said numbers out loud on earnings calls, and the transcripts exist. On the first-quarter call of April 23, 2026, CEO Lip-Bu Tan summarized the CPU-to-GPU ratio this way.
"The ratio of CPU to GPUs used to be 1-to-8, and now it is 1-to-4, and I think it could move towards parity or even better."
Nail down the reading direction first. The notation runs CPU count to GPU count. A configuration where eight GPUs shared a single CPU became four GPUs per CPU, and then roughly one to one, which means this shift is a story about CPUs increasing. Flip the order and read it as "four to one became one to one" and it looks like CPUs shrank. The opposite is true.
On the same call, CFO David Zinsner broke the ratio out by workload. Training runs about seven or eight GPUs per CPU, inference three or four, and once you get to agentic and multi-agent work the direction "may invert somewhat." That is the most important sentence here, because a named CFO attributed the ratio shift to agents. The three months of remarks that followed point the same way. On the second-quarter call of July 23, Zinsner said the ratio was "almost in parity" and could "skew more to CPUs on a unit basis," declining to give a precise figure and citing customer spending inputs and visibility from long-term supply agreements as his basis.
Line the four statements up chronologically and the direction of travel becomes visible. The column to watch is the last one. The left-hand number is the CPU count and the right-hand number is the GPU count, so a larger left-hand share means more CPUs attached to the same number of accelerators. Moving from training to inference to agents, that share grew nearly eightfold.
| When | Statement | CPU : GPU (units) |
|---|---|---|
| Past (training) | "used to be 1-to-8" / training runs 7–8 GPUs per CPU | 1 : 8 |
| 2026-04 (inference) | "now it is 1-to-4" / inference is 3–4 to 1 | 1 : 4 |
| 2026-05 (some agentic customers) | "four CPU to one GPU" — notation reversed | 4 : 1 |
| 2026-07 | "almost in parity … could skew more to CPUs on a unit basis" | ~1 : 1 |
The CPU-to-GPU unit ratio as described by Intel executives. The third row is written in the opposite order from the rows around it, so the numbers cannot be compared directly.
How many CPUs per GPU
Each square is one unit. Orange is CPU, gray is GPU. CPU is held fixed at one; only the GPU count next to it shrinks, tracing the shift in the ratio.
Original Pebblous diagram — the number of GPUs (gray) attached to one CPU (orange), per Intel executive statements. The same one CPU handled fewer and fewer GPUs, from eight down to one, moving from training to inference to agentic work. Read the other way, that means each GPU now needs that many more CPUs alongside it.
That third row needs care. At the JPMorgan Global TMC Conference in May 2026, Tan said some customers now tell him it is four to one: four CPUs to one GPU. The "1-to-4" of the earlier rows and the "four to one" here are written in opposite orders, and this one is a secondhand account of specific customer cases. Set them side by side and read only the digits, and a reader will be confused every time. Tan attributed the remark to frontier model companies and startups telling him CPUs were more useful for reinforcement learning, orchestration across multiple agents, and workload optimization.
2.1Intel's CEO named data as the reason
More important to this piece than the ratio itself is the reason Tan gave. Asked why CPUs matter more as you move to inference, he named three things.
"On the inference side, in terms of orchestration, control plane, and also managing all the different agents with data, CPU is much more efficient."
The last of those three is where this report stands. The CEO of a company that sells CPUs named handling data as a driver of CPU demand. On the same call he redefined the CPU as "the orchestration layer and critical control plane for the entire AI stack." Section 4 returns to that sentence.
2.2The people saying this are the people selling it
Better to fold vendor incentives into the arithmetic now rather than defer the judgment. The loudest voices on the CPU renaissance are precisely the companies that sell CPUs. Intel published its own white paper in March 2026 on "the rising CPU-to-GPU ratio in AI infrastructure," raised server CPU prices by 10 to 20 percent after March, and is reported to be planning another 8 to 10 percent in the second half. Some Xeon SKUs got more than a thousand dollars more expensive. AMD projects a $220 billion server CPU market by 2030 and uses that as its growth thesis. A scarcity narrative justifies a price increase all by itself.
What is interesting is that Intel also offered a different explanation. Intel VP Tasha Chuang told TrendForce that the price adjustments primarily reflected changes in manufacturing costs across the supply chain. That frames it as costs rising, not demand exploding. A demand story and a cost story coexist inside one company, which makes it premature to convert a price increase directly into evidence of a CPU shortage. What this piece takes from Intel's remarks is the direction of the ratio and the fact that data was named as the reason, not their conclusion.
What the CPU Does While an Agent Takes One Lap
So far this is what companies say. Where the time actually goes is something you have to measure. Fortunately, people have. arXiv:2511.00739, in which Ritik Raj and colleagues from Georgia Tech and Intel profiled five agent workloads from a CPU-centric perspective, is currently the paper that answers this question most directly. That two of the five authors are at Intel is worth keeping in mind throughout.
First, the citation conditions. The paper has been revised three times since its November 2025 first version, and the "up to 90.6% of end-to-end latency" line in the original abstract became up to 88 percent in the current v3 (April 16, 2026), where the percentage was dropped from the abstract entirely. Every figure below comes from the v3 body. And 88 percent is not an average across the five workloads; it is an upper bound in tool-dominated workloads. Strip that condition off in a citation and the sentence becomes false immediately.
The team ran the same workloads on two systems. Sys 1 paired a high-end CPU (Intel 6th-gen Xeon Granite Rapids) with a relatively modest GPU (RTX Pro 6000 Blackwell); Sys 2 paired a high-end CPU (NVIDIA Grace) with a high-end GPU (H200). Two things are worth reading in the table: which stage held the clock longest in each workload, and what share that stage took on each of the two systems. The workload-level results follow.
| Workload | Stage that dominated the time | Sys 1 | Sys 2 | Nature |
|---|---|---|---|---|
| Haystack RAG | ENNS vector search | 81–83% | up to 89% | Data work |
| ChemCrow | RDKit conformer generation | 85% | 88% | Data work |
| SWE-Agent | Bash and Python execution | 25–38% | up to 65% | Data work |
| LangChain (web-augmented) | LexRank summarization | 48–55% | — | Data work |
| Toolformer | LLM inference | ~88% | — | Model compute (counterexample) |
Per-workload latency breakdown from Raj et al. (arXiv:2511.00739 v3). The bottom row is a counterexample to this article's thesis: in Toolformer, the 88 percent belongs to LLM inference, not to tools.
There is a temptation to delete that last row, and deleting it would make this article false. The same paper contains a case in which the agent is not CPU-bound. Toolformer's 88 percent is model compute, not tool processing. So the accurate sentence is not "agents are CPU-bound" but "in tool-dominated workloads, most of the runtime drains through the CPU side." Which category a given workload falls into is something you have to measure, and that is arguably the most practical implication of the table.
3.1Upgrading the GPU made the bottleneck easier to see
Reading the Sys 1 and Sys 2 columns side by side produces the most interesting result in this piece. Swapping in the better GPU made the tool segment's share grow, not shrink. RAG went from 83 to 89 percent; SWE-Agent went from 38 to 65 percent. This is obvious and important at the same time. As the GPU's slice of total runtime shrank, the CPU segment that stayed exactly where it was stood out proportionally more.
That observation answers the natural objection, "won't this resolve itself once GPU supply loosens up?" As accelerators get cheaper and faster, the effect does not ease. It intensifies. One paper showed the direction with nothing more than a two-system comparison.
3.2The same thing happens one layer down, in serving
If that paper works at the agent layer, the same phenomenon shows up a layer below, in inference serving infrastructure. Chung et al. (arXiv:2603.22774) catalogued three mechanisms by which the CPU induces slowdowns in multi-GPU LLM inference. The first is tokenization. Serving Llama 3.1 8B across four H200s with 16 CPU cores allocated, the single step of chopping text into tokens consumed up to half of total latency on long-sequence requests. The second is collective communication sync: when the CPU is oversubscribed, one rank arriving a millisecond late at an NCCL barrier puts all eight GPUs into a wait. The third is shared-memory contention, where they reported dequeue latency in the vLLM V1 broadcast queue degrading from 12 milliseconds to 228, a factor of nineteen.
The paper's practical conclusion is that this is an allocation problem. With sufficient CPU, time to first token improved 1.36 to 5.40 times without adding a single GPU. And real clusters are mostly allocated the other way. Across 4.65 million job scheduler records the authors analyzed, the median CPU cores per GPU on educational clusters was one to two, and on research clusters roughly 60 percent of jobs ran with fewer than eight cores per GPU.
3.3What one lap looks like on a timeline
The third measurement comes from an operating-systems angle. AgentCgroup (arXiv:2602.09345 v3), from researchers at UC Santa Cruz, decomposed execution time while running 144 SWE-rebench tasks through the Claude Code harness. A single task took five to eleven minutes. Of that, LLM inference took 40 to 45 percent, tool execution 20 to 35 percent of active time, and initialization 31 to 48 percent. Grouping initialization and tool execution together, OS-level overhead accounted for 55 to 60 percent of the completion time a user actually experiences. Below, that loop is stretched out along a timeline.
The orange blocks are the data work running on the CPU. Calling tools, searching, reading files, parsing what comes back and packing it into the next prompt, and summarizing the result at the end all live there. The model itself runs in only two gray blocks: planning and replanning. Bar lengths represent the order and relative weight of the stages conceptually, not measured proportions from any one workload.
The breakdown itself depends heavily on the environment. Where AgentCgroup put initialization at 31 to 48 percent, PASTE (arXiv:2603.18897), which parallelizes tool execution with LLM generation, reported initialization at under 20 percent of tool latency in its own traces. Different harnesses and different traces produce different decompositions. No single number here works as a universal constant.
Most of That Time Is Data Work
The question narrows to one thing here. What exactly is happening inside those CPU segments? Write the stages that held the clock in the previous section back under the names of the work they actually do, and the answer appears.
| Stage the measurements named | What it actually does | Name of the data task |
|---|---|---|
| ENNS search (Haystack RAG) | Finds nearest neighbors in an embedding index | Vector search |
| LexRank summarization (LangChain) | Picks the key sentences out of scraped web documents | Text cleaning |
| Bash and Python execution (SWE-Agent) | Walks the repository, reads and writes files | File I/O |
| Conformer generation (ChemCrow) | Turns molecular structures into a computable form | Preprocessing |
| Tokenization (serving layer) | Converts text into the numbers a model consumes | Encoding |
| Parsing and serialization (agent loop) | Structures tool responses and loads them into the next prompt | Format conversion |
The time-dominating stages from section 3, rewritten under the names of data tasks. None of them is model compute.
There is not a single model operation in the right-hand column. It is all moving, selecting, cleaning and reshaping data. The claim that a large share of what gets called agent infrastructure cost is really data pipeline cost stops being rhetoric here and becomes an inventory. The sentence in section 2 where Intel's CEO named "managing all the different agents with data" as a driver of CPU demand and this table are saying the same thing from two angles. One is the observation of someone selling chips, the other is the output of a profiler, and the stages they point at overlap.
4.1The money goes to the GPU; the time goes to the CPU
Two apparently contradictory pieces of evidence need to sit side by side here. A study decomposing the cost of MCP-based agentic workflows (arXiv:2601.14735) reported that across two applications, the LLM accounted for 61 to 94 percent of total cost, agent execution about 13 percent, and MCP function execution around 3 percent. That looks like a head-on collision with section 3's finding that tools eat 88 percent of the time.
It is not a collision. The two numbers measure different things. One is money and the other is time. Token charges flow to the inference servers running on GPUs, while orchestration and tool execution run long and slow on far cheaper CPUs. Look only at the invoice and this is a GPU problem; look at the clock and it is a data problem. Same system, two different instruments.
Original Pebblous diagram — the cost share is the LLM's share of total cost from arXiv:2601.14735 (two applications); the time share is the tool-dominated upper bound from arXiv:2511.00739. The two figures come from different studies and workloads, so they are not a single execution split exactly in half — this is a conceptual diagram showing only which side dominates when the same system is read through a different instrument.
That distinction is also the relationship between this piece and an earlier one. In June 2026, "The Cheaper Tokens Get, the Bigger the Bill" examined how 73 percent of enterprises overrun their AI budgets even as token prices fell to a third of what they were. The material there was one analysis citing a thirtyfold increase from $0.04 to $1.20 per run, and retry loops inflating invoices as much as fiftyfold. That article followed the axis of the token bill; this one follows the axis of CPU time. Where the money goes and where the time goes are different questions, and unless you count both, half of your agent budget stays invisible.
4.2And yet CPU utilization is 10 percent
Everything so far leads easily to "so we are short on CPUs." The same AgentCgroup measurements block that conclusion head-on. Average CPU utilization during agent execution was 13.2 percent on a Claude Haiku 4.5 backend and 7.6 percent on a GLM backend. The authors' conclusion is even sharper: what constrains multi-tenancy density is not CPU but memory, with a 185 MB framework baseline on top of which tool calls push peak usage to 15.4 times the average.
Hiding that number would make this article false, and not hiding it makes the argument more precise. The 88 percent does not flow while the cores burn. Time accumulates on top of cores that are mostly idle. It is the result of serial stretches stacked end to end: waiting on a search result, waiting for a file to open, waiting for a full response before parsing can start. So the bottleneck is not compute volume but waiting and serialization, and the prescription changes accordingly. Buying more cores does not shorten a wait. What shortens it is fixing the path.
Agents do not wait on the GPU. They wait on data. Most of the runtime drains through CPU-side stages that search, parse, summarize and read files, and in those stages CPU utilization sits near 10 percent. Nothing is burning cores; time is piling up on a serialized data path. Which is why swapping in a better GPU does not remove the bottleneck. It makes it more visible.
4.3Where this argument has to stop
Push the argument one step further and it goes like this: the dirtier the data, the more parsing fails, the more schema mismatches trigger retries, and the more CPU time it costs. Plausible, and probably right. But this report does not present that as a measured fact, because no study was found that connects "dirty data" and "extra CPU seconds" inside a single experiment.
What exists are fragments of the chain. There are benchmarks observing tool-call schema error rates of 6.4 to 9.8 percent in multi-turn conversations. There is a case study in which an agent loop raised API calls from 308 to 504 against a zero-shot baseline and pushed cost up 52 percent, driven mainly by 196 retries. There are observations that missing required parameters spike as schema complexity rises. The fragments point the same way, but it is this report that connected them, not an experiment. Read it as circumstantial, not causal.
One fragment is operationally certain, though. A call that returned a 400 or 422 because of a missing required field or a type mismatch will return the same error if you resend it with the same parameters. Those retries are not partially recovered cost; they are pure waste. In organizations that aggregate retry metrics into a single number, that waste disappears into transient errors and rate-limit retries.
The Price List Only Half Agrees
If demand really has shifted toward CPUs, prices should have risen or supply should have tightened. Check, and it turns out to be half true. At the manufacturing stage the pressure is unmistakable. On cloud price lists it is barely visible.
5.1Manufacturing: the pressure is real
In an April 15, 2026 release, the research firm TrendForce cut its forecast for total server shipment growth that year from 20 percent to 13 percent. The reason was components, not demand. Lead times for PCBs and CPUs stretched to roughly a year; power management ICs went from 21–26 weeks to 35–40, and baseboard management controllers from 11–16 weeks to 21–26. In an August 5 release, the same firm raised its growth forecast for AI server shipments from 28 percent to nearly 31 percent, and projected that combined 2026 capital expenditure at the top nine cloud providers would rise about 90 percent year over year.
The picture only resolves when you read the two numbers together. The overall server market's forecast came down because components are scarce; the AI server forecast went up because demand is strong. Components are being allocated to AI servers first, and general-purpose servers are queued behind them. That pressure sits across the whole supply chain regardless of what happened inside AWS, which is why no single company's statement erases it.
5.2Cloud price lists: not there yet
On the screen where cloud customers actually pay, though, that pressure has not arrived. No basis was found for 2026 increases in on-demand pricing for AWS general-purpose instances (M7i, C7i, R7i), and older generations such as m5, c5 and r5 came down 3 to 7 percent. The Graviton4 families (r8g, c8g, m8g) run 10 to 15 percent cheaper than the prior generation. The GPU side is unambiguous in the other direction: EC2 GPU Capacity Block prices rose 15 percent in January and another 20 percent in July.
Reduce it to directions and the split is clean. Exactly one line item rose in 2026, and it was GPU capacity; the general-purpose instances that contain CPUs held flat or fell. Had the demand shift reached consumer pricing, the picture would look the other way around.
Direction of 2026 AWS instance price changes, on-demand. The pressure showed up as pricing only on the GPU side.
So this report does not say CPUs have become expensive in the cloud. Writing that would contradict the evidence. The accurate sentence is that manufacturing-stage pressure has not yet propagated to cloud price lists. Lead-time signals suggest it may show up with a lag, but as of August 2026 that transmission is not observable.
Not every cloud is alike, however. Azure is close to an exception. Bloomberg reported that new subscription restrictions in core regions including Northern Virginia and Texas were expected to run through the first half of 2026, attributed to shortages of physical space and servers, and explicitly affecting traditional CPU-based workloads, not only GPUs. Morgan Stanley expects relief starting in the second half of 2026. No comparable signal was found for GCP or OCI. That asymmetry stands as written: two of four providers confirmed, two not. Generalizing to "CPUs are short across every cloud" is wrong, and shrinking it to "an AWS-specific quirk" is also wrong.
5.3Six ways this article could be wrong
Rather than tuck these in defensively, this report stands them up as a list of their own. The six below weaken this article's thesis or attach conditions to it, and whatever survives a full reading of them is the report's actual claim.
- The most visible bottleneck is still the GPU. NVIDIA's most recent quarterly data center revenue was $51.2 billion, and Jensen Huang has said cloud GPUs are sold out. The CPU story sits on top of that one; it does not replace it.
- The real constraint is connectivity, not compute. Futurum argues the binding constraint on agentic inference is scale-up networking data movement for mixture-of-experts models rather than CPU compute, and Marvell's CEO has framed AI scaling as fundamentally a connectivity problem. This objection partly favors the argument here, though: saying data movement is the bottleneck is still saying the data path is the bottleneck.
- The bottleneck is power. Gartner expects 40 percent of AI data centers to face operational constraints from power availability by 2027. That axis got its own treatment in an earlier report.
- The people saying this are the people selling it. The vendor incentives from section 2 apply here too, and two of the five authors of the core paper this article relies on are at Intel.
- Scheduling recovers a good deal of the bottleneck. Techniques proposed in that same paper improved P50 latency by 1.7 times and end-to-end latency for certain request types by 2.37 to 2.49 times. Structural is not the same as inevitable.
- Both camps cite the same data. In a SambaNova demo, a 45-second agentic task used the GPU for 2 seconds. The CPU camp cites this as evidence the bottleneck has moved; the GPU camp cites the fact that those 2 seconds produced most of the tokens. It stands here as an illustration of how far the interpretations diverge.
How to Count the Data-Prep Share
After all of that, one question remains: what about us? Most organizations do not know, because the cost dashboard shows only tokens. The six items below use nothing but figures already introduced above, and each can be checked in your own environment. There are no new numbers here.
6.1Split the time axis first
Measure a single agent run in five segments: LLM wait, tool execution, serialization and parsing, retrieval and preprocessing, and initialization. All three papers in section 3 started from this decomposition, and without it there is no way to know which segment dominates in your environment. If yours is a workload like Toolformer, where model inference dominates, the prescriptions in this article do not apply to you. There is no reason to leave that determination to anyone else.
6.2Watch wait time, not utilization
CPU utilization of 10 percent reads as "plenty of headroom," but it can also mean "currently waiting." The 7.6 to 13.2 percent in section 4 was exactly that situation. Drop the CPU side from your investigation because the utilization graph looks flat, and you skip the segment where time accumulates most. What to look at instead is I/O wait, queue delay, and the length of the serial stretches.
6.3Check your CPU cores per GPU
As section 3 showed, real clusters commonly run one or two CPU cores per GPU, and that is under-provisioning. Given the result that adequate allocation improved time to first token by 1.36 to 5.40 times, this is a cheap thing to verify before buying more GPUs. Pulling the ratio out of your own cluster's job specs takes less than a day.
6.4Time tokenization and retrieval separately
For services handling long sequences, tokenization can reach half of total latency. The same goes for the retrieval stage. Both are usually buried inside a single "model call" bucket in application metrics, and they do not announce themselves until you pull them out and time them on their own.
6.5Break retries out by type
This is where the distinction from section 4.3 becomes a metric. A retry after a transient error or a rate limit has a chance of succeeding on resend; a retry after a 400 or 422 caused by a schema mismatch will fail again. Aggregate the two into one line and the retry rate can look perfectly normal while pure waste hides inside it. Benchmarks put multi-turn tool-call schema error rates around 6.4 to 9.8 percent. If your environment runs higher than that, that is your priority by itself.
6.6Split the budget into three lines
The last item is this article's actual request. Account for your agent budget in three lines: token spend, accelerator time, and data path time. Most organizations already watch the first two. The third is usually absent from every dashboard, which means that whether it is large or small, it never enters the conversation. Count it and you at least learn its size, and once you know the size you can judge whether it deserves priority. That is as far as this report asks the reader to go.
Why Pebblous Watches This Subject
Every product Pebblous builds sits outside the model. DataClinic diagnoses the condition of a dataset, AI-Ready Data covers the stage before anything reaches training, and DataGreenhouse and PebbloSim are the layer where data gets generated and validated. That is why this subject held our attention for a long time. The signals observable at both Intel and AWS say the center of gravity for AI system resources is moving into precisely that layer.
7.1Data quality shows up as time before it shows up as accuracy
Data quality problems are usually described as degraded model performance: accuracy drops, hallucinations rise, benchmark scores fall. In an agent architecture they are likely to appear first as time. Parsing fails, schema mismatches attach retries, and the clock runs while preprocessing is redone. As noted in section 4.3, no study has yet measured that connection inside a single experiment. So this is a Pebblous claim, not a fact this report has proven, and the proving is better done in your own environment with the instrumentation in section 6.
7.2The line missing from the quote
Organizations evaluating agent adoption generally calculate token costs and accelerator costs. What this report points at is the third line that is not on that quote. Cloud capacity planning, data pipeline redesign and tool-call failure rate management are all directly coupled to the condition of the data, and all three are usually not line items at all. Without a line item, improvements cannot be measured, and what cannot be measured does not attract investment.
7.3Why this piece is about the clock, not the ledger
Back in June, Pebblous published a piece treating the agent invoice as an accounting problem. This is not a repeat of that story but a reading from a different instrument. That one followed the flow of money; this one follows the flow of time. Back when the bottleneck sat on the GPU, data preparation never even registered as a cost line. As the center of gravity moves outward, that layer starts to show up in seconds, and after that in dollars.
Editor's Note. The conclusion of this article is not "buy more CPUs," and it is not "buy a particular product." What the measurements say is that a substantial share of agent runtime drains through stages that move, select and clean data, and that the CPU during those stages is mostly idle. If so, the thing to fix is the path, not the hardware. That is why Pebblous reads this issue as a data path problem rather than a semiconductor news story, and this article only proposes that lens. It does not substitute for your own measurements.
The 2026 conversation about AI infrastructure still revolves around how many of the biggest accelerators you can secure. Not because that question is unimportant, but because it alone cannot tell you what an agent is actually waiting on. Before the next quarterly plan gets written, the clock is worth a look. If what your agent waits on is not the model, buying a better model will not shorten the wait.
References
Academic (arXiv / papers)
- 1.Raj, R., Kundu, S., Vohra, I., Wang, H., Krishna, T., "Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective," 2025–2026 (v1 2025-11-01, v3 2026-04-16). Georgia Tech / Intel. arXiv:2511.00739 — source of the 88 percent figure, the per-workload breakdown, and the two-system comparison
- 2.Chung, E., Jia, Y., Jezghani, A., Kim, H., "Characterizing CPU-Induced Slowdowns in Multi-GPU LLM Inference," 2026-03-24. arXiv:2603.22774 — TTFT 1.36–5.40×, tokenization at 50 percent, real-world core allocation
- 3.Zheng, Y., Fan, J., Fu, Q., Yang, Y., Zhang, W., Quinn, A., "AgentCgroup: Understanding and Controlling OS Resources of AI Agents," v3 2026-07-23. UC Santa Cruz. arXiv:2602.09345 — the time decomposition and CPU utilization of 7.6–13.2 percent
- 4."Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving" (PASTE), 2026-03. arXiv:2603.18897 — the conflicting observation on initialization share
- 5."Schema First Tool APIs for LLM Agents," 2026. arXiv:2603.13404 — a taxonomy of tool-call failures
- 6.Cost decomposition of MCP-based agentic workflows (FaaS environment, two applications), 2026. arXiv:2601.14735 — LLM at 61–94 percent of cost
Corporate primary sources (earnings calls, shareholder letters, IR)
- 7.Intel, Q1 2026 earnings call transcript, 2026-04-23 — remarks by Lip-Bu Tan and David Zinsner. fool.com (via transcript service)
- 8.Intel, Q2 2026 earnings call, 2026-07-23 — the "almost in parity" remark. intc.com
- 9.Amazon, "CEO Andy Jassy's 2025 Letter to Shareholders," 2026-04-09 — the Graviton capacity request and refusal, 3.9 GW, capex lead time. aboutamazon.com
- 10.AMD, Q2 2026 earnings press release, 2026-08-04 — server CPU growth and the 2030 market projection. ir.amd.com
- 11.J.P. Morgan 54th Annual Global TMC Conference, Intel fireside chat, 2026-05-19 — the "four CPU to one GPU" remark (the reversed-notation case)
Research firms and statistics
- 12.TrendForce, press release on the lowered server shipment forecast and component lead times, 2026-04-15. trendforce.com
- 13.TrendForce, press release raising the AI server shipment forecast, 2026-08-05. trendforce.com
- 14.Gartner, "Lack of AI-Ready Data Puts AI Projects at Risk," press release, 2025-02-26 (N=248, Q3 2024). gartner.com
Press (secondary, traced back to primary sources)
- 15.The Information, exclusive report on AWS internal capacity-conservation instructions, 2026-08-07 (paywalled). theinformation.com
- 16.Tom's Hardware, re-report of the AWS story with the AWS statement. tomshardware.com
- 17.Tom's Hardware, confirmation of Intel server and consumer CPU price increases, 2026-04. tomshardware.com
- 18.AI Times, report on the AWS internal CPU usage crackdown. aitimes.com
Adjacent Pebblous work (cross-links)
- 19.Pebblous, "The Cheaper Tokens Get, the Bigger the Bill" (2026-06-13) — the axis of the token bill. blog/ai-agent-token-cost-retry-loop
- 20.Pebblous, "AI Hit a Wall This Summer, and It Wasn't Chips or Money" (2026-08-01). report/ai-datacenter-power-wall-2026
- 21.Pebblous, "The Bill for Spare Servers" (2026-07-03). report/meta-compute-semiconductor-shock
- 22.Pebblous, "The GPU Was Never the Bottleneck — Data Was" (2026-05-12). report/nvidia-virtual-cell-challenge-2026-05