For most of the AI buildout, the GPU owned the narrative. Frontier clusters, token throughput, Nvidia accelerators — that was the story of the first wave. Then two earnings calls, one day apart, quietly moved the spotlight to a different part of the machine. Microsoft's Satya Nadella compressed the shift into a single sentence: “When running agent workloads, CPU is as important as GPU.” That is not a talking point. It is a billing reality.
The Agent Wave Changed What the Computer Does
Agentic workloads do not look like classic inference. An agent retrieves data, calls tools, executes code, runs policies, and verifies results — a far broader task set than generating tokens. The GPU still carries the bulk of model training and inference, but the CPU is doing everything around the model: request routing, data pre-processing, tool orchestration, memory and storage management, and keeping thousands of concurrent interactions stable.
This is why the first AI wave measured success in tokens per second, while the agent wave measures it in coordination. When your infrastructure's job is to keep a swarm of parallel tasks alive and coherent, the chip that decides who runs next and what they get becomes as valuable as the chip that does the heavy math.
Microsoft and AWS Are Voting with Silicon
Microsoft's Cobalt 200 is the company's second-generation Arm cloud CPU, built on Arm Neoverse CSS V3. It delivers up to 50 percent higher performance than its predecessor, scales to 128 vCPUs, and is purpose-built for cloud-native, data-intensive and agent workloads. Released in November 2025 with VM previews in June 2026, Cobalt-powered racks reached more than 25 data centers in under two months — and they already run Microsoft's own services plus Adobe, Arm, Elastic, OpenAI, Sprinklr and TomTom.
AWS is moving even faster. Its chip business — Graviton, Trainium and Nitro — has crossed $25 billion in annualized revenue, growing triple digits year over year. Adoption is the real signal: 98 percent of AWS's top 1,000 EC2 customers use Graviton, more than 120,000 customers build on it, and for the third straight year more than half of new AWS CPU capacity is Graviton. The fifth-generation Graviton5 powers the M9g instances, delivering up to 25 percent more compute and 35 percent faster machine-learning inference than Graviton4 — with 30 to 40 percent better price-performance across the line. This mirrors a trend we flagged earlier: the CPU-to-GPU ratio is marching toward 1:1.
Efficiency Is the New Revenue Lever
For hyperscalers, CPU efficiency is no longer a cost-saving metric — it is a capacity multiplier. Microsoft's Azure grew 43 percent year over year with demand still outstripping supply. CFO Amy Hood's explanation was direct: higher infrastructure efficiency and faster activation release compute that gets consumed by demand almost immediately and converts into revenue.
The pressure is structural. The IEA says AI data center electricity consumption grew 50 percent in 2025, and Dell'Oro puts hyperscaler capex up 78 percent year over year in Q1 2026. When every watt and every rack is constrained, a chip that does the same job with better utilization is not a nice-to-have — it is how you free up revenue that would otherwise stay locked in power bills. Infrastructure is quietly entering a self-evolving era where this kind of efficiency becomes the competitive edge.
The Whole Industry Is Converging on Arm
Microsoft and AWS are the loudest examples, but not the only ones. Google Cloud's Axion is an Arm-based CPU family that also serves as the AI head-node CPU inside its latest TPU systems, handling orchestration, networking and infrastructure services. NVIDIA's Vera is an Arm-based CPU that will anchor Rubin-generation AI factories, coordinating memory, networking and accelerators at rack scale.
Four hyperscalers, one architecture conclusion: as AI infrastructure moves from model inference to autonomous agent execution, Arm-based CPUs are becoming the general-purpose compute substrate of the modern data center — the layer that routes, coordinates and keeps thousands of concurrent agent tasks running.
What Infra Teams Should Do Now
- Rebalance capacity planning. Model your CPU:GPU mix for agent workloads, not just training and inference tokens. Orchestration-heavy paths change the ratio faster than most budgets expect.
- Benchmark Arm instances. Graviton5's M9g and Cobalt-based VMs are worth measuring on your orchestration and inference paths — the 30 to 40 percent price-performance advantage compounds at fleet scale.
- Measure cost per completed task, not per token. Agent economics reward coordination efficiency. IDC expects 2026 global AI infrastructure spending to hit $497 billion, up 56 percent, with demand extending to orchestration platforms, data pipelines and CPU-based inference clusters.
The GPU built the first AI wave. The CPU is quietly deciding who wins the second one.
