Two worlds in one keynote. On July 24, AMD Advancing AI 2026 opened without theatrics — CEO Lisa Su walked out and dropped a single judgment: “In 2026, roughly 60% of global AI compute will be spent on inference.”
That line set the keynote’s thesis. AMD is not competing with NVIDIA on single-chip benchmark runs; it is redefining how AI infrastructure gets measured. NVIDIA has Vera Rubin NVL72 — AMD answers with Helios. NVIDIA has CUDA — AMD answers with ROCm.AI. NVIDIA builds everything in-house — AMD chooses open architecture. This was not a chip launch. It was a manifesto about how AI infrastructure should be built.
Helios and MI455X: the Rack Is the System
The centerpiece is Helios, a rack-scale AI platform whose strategic meaning fits in one line: the rack becomes the new system. Helios links 72 MI455X GPUs through a UALoE (Unified All-to-All Lossless Ethernet) scale-up fabric into one unified compute domain. Instead of nine 8-GPU servers stitched across multi-hop networks, the 72 GPUs live in a single rack — one-hop interconnect, uniform bandwidth.
The architecture attacks today’s inference bottleneck directly. Models keep growing, contexts keep lengthening, KV caches devour memory — and traditional servers’ independent-memory design means communication latency, data-locality constraints, and congestion risk all worsen exponentially with scale. Helios’s answer: design the whole rack as one very large computer.
MI455X hard specs
-
Memory: 432GB HBM4 — up to 1.5× the MI355X.
-
Memory bandwidth: 23.3 TB/s — up to 2.9× the MI355X.
-
Compute: 20 PFLOPS MXFP8 / 40 PFLOPS MXFP4 — low-precision throughput up to 4×.
-
System total: ~31TB of HBM4 across the rack’s 72 GPUs.
-
Aggregate scale-up bandwidth: 260 TB/s.
AMD’s own numbers: in DeepSeek V4 Flash FP4 inference, MI455X token throughput reaches up to 34× the MI355X, with per-token cost down 18×. Caveat attached — those figures come from pre-production hardware and specific configurations; production verification is still pending.
A three-tier network fabric
-
Salina DPU: offloads front-end networking, storage, and security so CPUs stop burning cycles on infrastructure chores.
-
UALoE fabric: the in-rack scale-up interconnect for all 72 GPUs, delivering 260TB/s aggregate.
-
Vulcano AI NIC: 800G plus PCIe 6.0, up to 2.4Tbps of scale-out bandwidth per GPU for multi-rack clusters.
Together the three form a complete data path from server ingress to in-rack fabric to cross-rack clusters. Pensando — the networking silicon company AMD acquired in 2024 — has graduated from standalone business unit to core of the AI platform.
Venice EPYC: CPUs Return to Center Stage in the Agent Era
Su’s sharpest hidden insight: agents’ incremental compute does not all land on GPUs. An agent task executes code, queries databases, calls tools, and coordinates parallel steps — much of it runs on CPUs. AMD accordingly launched the sixth-generation EPYC 9006 series “Venice”: up to 256 cores and 512 threads on Zen 6 with PCIe 6.0, spanning four product lines across general compute, enterprise, HPC, and GPU host nodes.
One number stands out: EPYC now holds 46% of the server CPU market, and AMD says a Forbes Global 2000 company switches to AMD every week. When agents scale from millions to billions, the server CPU market gets redefined.
ROCm.AI: Solving GPU Programming With AI
The software stack has always been AMD’s weak flank in AI. CUDA’s moat is not just performance — it is ecosystem habit. ROCm.AI exists to close that gap: not a new low-level framework but an AI-assisted layer over the existing ROCm stack — GPU code generation, CUDA-to-HIP migration assistance, bottleneck profiling, and optimization suggestions — cooperating with mainstream tools like Cursor, Claude, Codex, and Gemini. AMD also introduced FlyDSL, a Pythonic domain-specific language that lets Python developers spend less time on threads, memory, and scheduling and more on the algorithm itself.
ROCm.AI is slated for availability in August 2026. Meanwhile over 3 million Hugging Face models already run on AMD out of the box, the top ten open-source AI projects natively support AMD, and community contributions have grown more than 10×. But software competitiveness ultimately settles on model adaptation turnaround, kernel performance, and production stability — the promises ROCm.AI must keep delivering on.
Local and Edge: From Cloud to Everywhere
-
Ryzen AI 400 / AI MAX: local inference for roughly 24B and larger parameter models respectively.
-
Gorgon Halo: next-gen platform with unified memory raised to 192GB, supporting ~300B-parameter models.
-
Kria AI robotics platform: silicon → production modules → dev platform → software → ecosystem, all five layers in place.
Worth watching: the joint AMD–Cisco solution combines Ryzen AI Halo, the Lemonade inference engine, and agent sandboxes with Cisco’s network-security policy and observability — a full local agent governance stack for enterprises. It hints that the “last mile” of enterprise AI infrastructure is shifting from cloud-only to cloud-edge cooperation.
Strategic Read: AMD’s Offense and Defense
1. Inference is the main battlefield
As training saturates — or at least hits diminishing returns — inference is the ongoing operating cost. AMD answered “token cost,” the metric that matters, rather than competing on training FLOPS peaks.
2. System competition replaces chip competition
The market’s yardstick for AI infrastructure is changing: from single-GPU performance to per-rack throughput, per-token cost, cluster utilization, and fault tolerance. Helios benchmarks against NVIDIA’s whole system play — not against an H100 or a B200.
3. Open architecture versus vertical integration
NVIDIA controls every layer — GPUs, CPUs, interconnect, NICs, DPUs, switches, CUDA. AMD chose openness: joint delivery with silicon, server, network, cloud, and software partners. Openness buys customization room and bargaining power at the price of harder execution — compatibility validation and tuning grow more complex with every participant.
4. Benchmarked against Vera Rubin
AMD’s modeling claims Helios delivers up to 30% more tokens per dollar than NVIDIA’s Vera Rubin NVL72. If production verifies that number, AMD earns its ticket onto the big buyers’ shortlists.
What to Watch Next
Today’s announcement was comprehensive, but the real test is not on slides:
-
When ROCm.AI lands in August, watch developers’ actual migration cost and first-run experience — that is where the CUDA moat is genuinely thinnest.
-
Helios’s production timeline and early customer deployments decide whether AMD moves from “second choice” to “credible choice.”
-
MI500 and successors’ interfaces and architecture — whether Helios protects customers’ long-term hardware investment determines if it evolves from “modular rack” into “upgradeable platform.”
-
Whether agent workloads’ CPU demand explodes as Su predicts, validating Venice EPYC’s positioning.
AI infrastructure is crossing from “which chip is faster” to “which system is cheaper.” AMD bet the right direction at Advancing AI 2026 — but the execution road has only begun.
