There is a particular irony running through the biggest chip story of August: Nvidia spent nearly twenty years cultivating the world's richest AI software ecosystem, and the AI agent that just replicated the core layer of CUDA in ten hours grew up inside that very ecosystem. The moat is not being stormed from outside. It is being digested by something the industry — Nvidia loudest of all — spent two decades feeding.
The facts on the table: Infinity, a startup barely a year old, used its in-house coding agent Ignition to build a complete low-level software stack for the inference chips of AI silicon company d-Matrix. Kernel generation, compilation, testing, performance optimization — the entire pipeline ran AI-driven, and it took ten hours. On first access to the hardware, Ignition reached 92 percent of the chip's theoretical peak; within ten days, Qwen3, Qwen3.5, and Gemma4 were running end to end, every layer of code generated from scratch by the agent. The same workflow has historically consumed engineering teams for months, sometimes years.
This is not alarmism. It is a documented result — and the interesting question is what it actually proves.
What Ignition Did — and What It Left Alone
CUDA — Compute Unified Device Architecture — is not a programming language. It is a three-story software platform built around Nvidia GPUs:
- Ground floor (the kernel layer): kernel generation, compilers, runtimes — the instruction layer that makes silicon compute.
- Middle floor (the library layer): heavily tuned libraries like cuBLAS and cuDNN, with two decades of debugging, verification, and profiling tooling.
- Top floor (the ecosystem layer): PyTorch and TensorFlow integration, mountains of enterprise code and workflow, and a community of millions of developers.
Ignition went through the ground floor. It auto-generates and optimizes inference kernels for specific chips and builds low-level software capability around them, running a classic agent loop: write code, compile, run, test correctness and performance, iterate on the feedback. The task suits agents almost suspiciously well because the feedback signals are unambiguous — does it compile, is the result right, is it faster? No fuzzy judgment required.
What Ignition did not do is copy CUDA's full stack. The twenty-year polished library floor was untouched; the ecosystem of millions of developers even more so. What it demonstrated is something more consequential: a new path is viable.
Why This Time Is Different
Stories about CUDA being replaced surface every few months. AMD's ROCm, Intel's oneAPI, Google's TPU — each challenger arrived to analyst conclusions of "it will not move." Those conclusions were correct, and the reason they were correct explains why the current wave is different.
Every previous challenge to CUDA was a headcount game. Dethroning it meant assembling an engineering organization resembling Nvidia's own, then grinding through years of adaptation, optimization, and validation — a contest Nvidia wins on scale, every time.
Ignition's path is a compute game. AI generates the code, AI tests and optimizes it, and the binding constraint on software construction shifts from how many elite engineers you can hire to how much compute you can rent. Compute is a commodity, priced and purchasable by anyone. The software stack's cost structure just decoupled from headcount — and no incumbent has ever defended a moat built on headcount against an opponent whose costs scale with commodity hardware.
DeepSeek is running the same play from another direction, open-sourcing its TileKernels operator library written in the self-developed TileLang language to slash hand-written CUDA. Founder Liang Wenfeng said recently that coding agents plus TileLang make building AI software substantially easier.
The Battlefield Is Inference
The second difference is where the ground was broken. AI compute splits into training and inference, and on the training side CUDA remains close to unanswerable — massive runs punish instability, punish interconnect jitter, punish everything, and enterprises buy training platforms accordingly.
Inference runs on different physics. Marshall Choy, chief commercial officer at Korean chip company Rebellions, is blunt about it: "On the inference side, CUDA is no longer the deciding factor. This will be a competition of open-source software." Inference buyers do not worship peak performance; they count cost per answer. Good enough and cheap beats perfect and expensive, which means they will try anything that runs.
Crucially, inference software can run across chips. A framework that supports many accelerators lets users switch hardware without rewriting a line — and vendor lock-in, the largest single rivet holding CUDA in place, simply stops engaging on that side of the market.
The Moat Is Relocating in Three Timeframes
Short term: Nvidia's training-side dominance does not move. Cluster-scale stability, inter-chip communication, fault recovery — none of that materializes from a ten-hour agent run. But inference becomes the second battlefield immediately. d-Matrix, Rebellions, Cerebras, AMD, Amazon's Inferentia, Google's TPU are already deployed across it, and none of them needs to dethrone Nvidia outright. They only need to prove that on specific inference workloads, life without CUDA is faster or cheaper.
Medium term: the sharpest observation in this entire episode belongs to Bing Xu — founder of INT21, whose earlier chip software company HippoML was itself acquired by Nvidia. Agents can flood a repo with code in an hour, he notes, but verification is the bottleneck: generating ten thousand lines is fast, while proving those lines are correct and stable across boundary conditions is brutally slow. CUDA's deepest asset is its verification toolchain. The moat is migrating from stockpiled code to a verification and optimization ecosystem — and code generation is what AI does best, while verification in niche silicon domains, dense with domain knowledge and scarce test data, remains what it does worst. Whoever plants a flag on verification first wins the next era.
Long term: the defender accelerates too. Nvidia's VP of developer ecosystem Ankit Patel says plainly that Nvidia uses AI agents to develop CUDA faster and validate it at greater scale. Attack with the attacker's own spear, and the attacker's relative advantage discounts. Xu's framing stands as the cleanest forecast available: the fight turns on whether challengers can close the gap faster than Nvidia widens it.
Four Judgments to Act On
- If you buy inference hardware: look past the spec sheet. Ask whether the chip's software ecosystem supports cross-chip inference frameworks — multi-accelerator compatibility in vLLM, TensorRT-LLM, and their peers — because that compatibility is your future bargaining power against every vendor.
- If you are an AI infrastructure engineer: spend real hours mapping what AI coding agents can and cannot do in GPU kernel generation. Track Ignition and DeepSeek's TileKernels as two live paths — this is plausibly the highest-leverage skill bet you can make for the next twelve months.
- If you run a chip startup: your core competency is no longer "how close our silicon gets to Nvidia's." It is "how fast our software enablement runs." Compress adaptation from twelve months to one with agents, and hardware at 70 percent of Nvidia's performance still carves out a survivable market.
- If you invest: watch the AI-native chip software stack category. Infinity's $15 million seed at a $100 million valuation is the opening print, not the ceiling. When AI can generate low-level compute software on demand, chip entrepreneurship gets cheaper to enter — and the investing logic migrates from betting on hardware to betting on software ecosystems.
The thesis in one line: Nvidia's twenty-year CUDA moat was not breached by a competitor — it was redefined by a new mode of producing software, in which the moat's substance shifts from stockpiled code to a verification ecosystem, and the shift begins on the inference side.
The thing feeding on the moat was raised inside it. That is not a metaphor worth fearing; it is a mechanism worth studying.