Imagine you are a cloud vendor's procurement lead selecting hardware for an inference cluster. Two options: an NVIDIA B200 or an AMD MI355X. NVIDIA's per-card price is nearly 2.75× AMD's, but its software ecosystem is more mature and deployment easier — "buying NVIDIA is buying peace of mind" has been the industry's strongest consensus for five years.
But what if I told you that consensus is crumbling fast? This week, a company with only $4 million in seed funding — Wafer — ran Zhipu's latest open-source model GLM 5.2 on AMD MI355X. The result: single-node aggregate throughput at 80% of B200, at half the cost. It hit number two on Hacker News, the comment section packed with peers asking for technical details — not because they disbelieved it, but because they desperately wanted to know "how."
On the surface, this is an "AMD is getting competitive" story. Look deeper and it reveals something more fundamental: NVIDIA's moat is being eroded from three directions simultaneously.
A Moat Melting in Real Time
The data first. Wafer compressed GLM 5.2's weights from BF16 to MXFP4 format (using AMD's own Quark tool) with almost no precision loss — GSM8K dropped from 96.5% to 95.5%, GPQA-Diamond from 92.2% to 90.3%, and tau2 actually rose from 81.9% to 83.4%. Then they used sglang as the inference engine, hitting two problems: MTP-head weight naming caused a loading crash, and a fused kernel hardcoded CUDA headers without a ROCm branch. The fixes were trivial: re-register the shared-expert layers under sglang's actual naming, and add one #ifdef USE_ROCM compile guard. Two code-level changes tripled single-stream throughput to 213 tok/s. In the aggregate-throughput scenario (20k input, 1k output, 60% cache hit), they further switched parallelism from TP8 to TP4×DP2 and hand-specified AMD kernels for GLM 5.2's expert-layer shapes, reaching 2,626 tok/s/node — versus B200's 3,192 on the same workload. Eighty percent of the performance at less than half the cost.
But what deserves attention is not the number itself. It is what Wafer wrote in their own blog: "This time, we didn't need to write any custom kernels." Consider: just months earlier, to run Qwen3.5 397B on AMD, they had to hand-forge fused kernels — merging six GPU calls into one — to barely reach the top of the AA leaderboard. Now, adapting GLM 5.2 had degraded to "debug a few framework compatibility issues, add two lines of code." What drove the change? Not AMD's marketing team — it was the chemical reaction of two forces: AI-assisted engineering capability improving, and AMD's official toolchain accelerating to catch up. Wafer's own positioning is using AI to optimize AI infrastructure; their few-dozen-person team can track model-release cadence for adaptation because agents' kernel and model optimization capabilities keep strengthening. On AMD's side, the ROCm ecosystem is rapidly converging — from "rewrite kernels every time" to "toolchain mostly works, just a few potholes left." The intersection of these two trends is the premise for companies like Wafer: doing the adaptation work NVIDIA and AMD "should have finished but haven't."
The "Three-Layer Moat" Framework
How deep is NVIDIA's moat really? Decomposed, it has three layers. Layer one: the hardware moat (shallowest). Compute, memory, bandwidth. AMD's MI355X already approaches or partially exceeds same-generation B200 on paper specs. This gap is narrowing fast, but it was never NVIDIA's real barrier. Layer two: the software moat (thinning). CUDA, cuDNN, TensorRT — fifteen years of accumulation considered "irreplicable." But AMD's ROCm is catching up, and interestingly, not by AMD filling every pothole itself — it is third-party optimization companies (like Wafer) and AI agents jointly completing the "last mile" of adaptation. Layer three: the speed moat (deepest). Day-0 support — NVIDIA can ship optimized inference solutions the day a new model launches, while AMD often waits weeks or months. That is why Wafer exists: compressing AMD's speed gap from "months" to "days."
The layers differ in thickness, but the key change is: a crack has appeared between layers two and three. When AI agents can fill AMD's compatibility gaps in days, "Day-0 support" stops being NVIDIA's exclusive capability. As long as the model is open-source and AMD's chips exist, someone — large companies, startups, the open-source community — will fill the adaptation time gap.
Who Fills the Gap, Who Benefits
Apply the frame to the players and the picture sharpens. For cloud vendors — inference prices will likely keep falling. AMD hardware is cheaper, and adaptation costs are dropping fast. Cloud vendors that can run more open-source models have grounds to price inference more competitively than NVIDIA. This directly changes model vendors' deployment decisions. For open-source models (GLM, Qwen, etc.) — being procurable at low cost by more cloud vendors determines whether they can compete with closed models for inference users. GLM 5.2's price-performance on AMD means Zhipu's models can appear on more cloud vendors' shelves behind a lower "invisible threshold." For NVIDIA — they have not weakened, but the rules have changed. Their competitiveness was "every dollar you spend is worth it — because peace of mind." When the peace-of-mind premium grows more expensive (B200 prices soaring on supply constraints) while AMD-plus-AI-agents becomes increasingly "good enough," the premium's justification erodes.
Independent testing firm SemiAnalysis recorded a similar trajectory in May: AMD's team spent 14 weeks closing the software gap on Qwen3.5, pulling the cost curve past NVIDIA — $0.22 per million tokens versus B200's $0.30. Two important caveats: first, single-node deployment only; multi-node, large-scale clusters still favor NVIDIA. Second, "cost" here means hardware TCO plus adaptation cost, excluding the opportunity cost of migrating team expertise. For enterprises already built on CUDA, migration cost far exceeds the chip-price difference. So NVIDIA's moat is not vanishing overnight — it is thinning from the weakest edge inward: single-node inference, open-model adaptation, price-sensitive incremental customers. Then, from edge toward core.
If You Are Choosing Inference Hardware Now
Small-to-mid AI application developers: AMD's value proposition has reached the "seriously evaluate" stage. Find a Wafer-like partner (or run their public playbook yourself) and test your actual workload. Do not look at paper throughput — look at your scenario's first-token latency, cache-hit rate, and cost curve. Large cloud vendors: keeping NVIDIA for core clusters is correct. But actively deploy AMD nodes on edge-inference clusters. This is not "switching vendors" — it is "adding a procurement dimension." AMD's existence is itself your bargaining chip against NVIDIA. Technical decision-makers: stop asking the binary "can AMD replace NVIDIA." The real question: in your specific scenario, does AMD's "cost advantage" cover "adaptation cost plus performance gap"? The answer to that question changes every six months — and every change moves in AMD's favor.
References: Wafer, GLM 5.2 on AMD MI355X (wafer.ai/blog/glm52-amd) · Wafer, Kernels Are Still the Moat (wafer.ai/blog/kernels-are-still-the-moat) · SemiAnalysis AMD vs NVIDIA inference cost comparison (May 2026)