Large-model inference is hitting two walls at once: a capacity wall and an I/O wall.
Model parameters keep growing, while long contexts, multi-turn conversations, and agent workloads make the KV cache balloon. MoE architectures slash the number of activated parameters per forward pass but must store expert weights far exceeding the capacity of VRAM or system memory. In the cloud, scarce HBM sits occupied by weights and caches while GPU utilization is throttled by data movement; on devices, limited DRAM or unified memory caps the size of any locally runnable model.
As VRAM and DRAM struggle to carry inference alone, a long-underestimated player is stepping onto the stage: the SSD. It is moving from a static warehouse for model files into the real-time data path of inference — a formal storage tier after HBM, VRAM, and DRAM.
Why the SSD Has to Enter the Inference Path
This is not concept-chasing; the infrastructure shift has already happened. Moonshot AI’s Mooncake runs a KV-cache-centric disaggregated architecture that organizes idle CPUs, DRAM, and SSDs across a GPU cluster into a distributed cache, letting cold data reside long-term on cheap, capacious flash. In 2026, NVIDIA launched the CMX context memory storage platform, adding an Ethernet-attached, KV-cache-optimized flash tier to Vera Rubin infrastructure — NVIDIA positions it as a pod-level context layer between GPU memory and shared storage.
The industry signal is explicit: SSDs are becoming a resource inference systems schedule directly, no longer a peripheral that loads models before the fun starts.
But being called into the path is not the same as being fit for the job. Traditional SSDs were designed around file and block storage, judged on sequential bandwidth, random IOPS, reliability, and cost per terabyte. LLM inference needs data delivery under hard timing constraints: the KV cache is generated in bursts during prefill and then re-read by session, layer, and token position; MoE inference must fetch specific expert weights ahead of every layer based on routing results. One read that misses its compute window leaves the GPU idling — and an idle GPU is money on fire.
The deeper conflict lives inside the SSD itself. NAND flash reads in pages and erases in blocks; the FTL must juggle address mapping, garbage collection, and wear leveling, breeding write amplification and tail latency. Sustained KV-cache writing wears flash far faster than consumer workloads ever did. And the traditional LBA layout knows nothing about the semantic relationships between model layers, MoE experts, and KV blocks — related data gets scattered across die and channel, making predictable parallel reads nearly impossible.
Two Technical Routes: Reinforced I/O, or Inference-Aware Storage
Products marketed as “AI SSDs” now span several approaches, with no unified definition yet. They fall into two broad camps.
Route One: AI-Reinforced Enterprise SSDs
These remain block devices, but their design metrics have been reordered around the AI cluster’s data path. The representative is InnoGrit’s Dongting N3X series, built on XL-Flash and SLC-class NAND for low latency and high endurance, targeting KV-cache offload and high-concurrency temporary reads and writes. Against conventional TLC SSDs, public materials cite access latency down to roughly one-third, write throughput up about three times, and DWPD raised 17 to 33 times. The value is less about peak numbers than about flattening the performance wobble that sustained writes and garbage collection inject into serving.
Huawei’s OceanDisk line covers different AI loads as a family: the EX 560 pushes extreme random-write performance and microsecond-level write latency for fine-tuning and hot cache writes; the SP 560 balances performance, endurance, and cost for KV-cache and long-context serving in clusters; the LC 560 tops out at 245TB per drive for training corpora, vector stores, and model files. AI-reinforced, in other words, is not one SKU — it is splitting into ultra-performance, value-inference, and capacity product lines.
Route Two: Inference-Participating AI SSDs
The second camp goes further and changes what an SSD does in the machine: through co-design of controller, firmware, middleware, and OS, it builds a near “NAND-plus-DRAM/VRAM joint virtualization” hierarchy, letting model weights, MoE experts, and KV cache reside on the SSD during inference and stream into memory as computation demands.
Phison’s aiDAPTIV scheme partitions model weights that exceed VRAM capacity into blocks streamed between VRAM and SSD according to the model’s execution: active weights stay in VRAM, inactive weights sink to flash, and the loadable model size grows without buying more GPUs. It also saves KV cache that would otherwise be evicted, avoiding prefill re-execution. Its dedicated cache SSDs are rated up to 100 DWPD.
Longsys takes a “hardware SPU plus software iSA” route: the WM8500 SPU, built on 5nm, integrates in-storage lossless compression, an HLC advanced cache, and hybrid NAND scheduling — compression raises effective capacity and bandwidth utilization, HLC lets the SSD absorb warm-cold data that used to live in DRAM, and the hybrid mechanism moves data between media zones by temperature. Where Phison sells VRAM extension, Longsys is upgrading the SSD controller from data courier to storage-side intelligent scheduler.
The Yinspect–Maxio alliance attempts the deepest integration, stitching the compute side’s cache hierarchy through middleware, firmware, controller, and NAND media management: data segmentation, hot-cold tiering, and predictive prefetch around three dimensions — MoE experts, KV cache, and numerical precision — splitting model files into hotter low-precision and colder high-precision pieces. The stated goal is to adapt across processor platforms without modifying model files or invading mainstream inference frameworks.
A Framework for Understanding AI SSDs
Read the whole shift as a triple promotion of storage’s role.
First, from warehouse to assembly line. A traditional SSD is a warehouse for model files — move the weights into memory before inference and its job is done. An AI SSD is a workbench on the production line: data supply must keep the beat of computation, and a slow read idles the GPU, which is burned money.
Second, from capacity arms race to latency arms race. Traditional storage competes on sequential bandwidth and raw capacity; AI inference competes on stable latency at low queue depths and fine-grained access. Peak IOPS mean nothing if a single read misses its compute window.
Third, from peripheral to cost variable. As the industry’s metrics move from peak compute toward cost per token, throughput per watt, and effective GPU utilization, the SSD’s per-terabyte price advantage over HBM makes it the control valve of token economics — every piece of cold data moved from HBM to flash lowers marginal inference cost.
Outlook: The Competition Has Only Started
Three trends look locked in. First, AI SSDs will not replace HBM, VRAM, or DRAM; they add a tier, reshaping the stack by capacity, performance, and cost into four or five layers across cloud, edge, and device — each with its own job. Second, the bar to compete is rising from “build a fast SSD” to “operate a full collaboration stack from NAND control through inference scheduling to system integration.” Phison and the Yinspect–Maxio alliance are past concept demos, validating simultaneously in consumer AI PCs, workstations, edge platforms, and enterprise inference nodes — early movers are banking controller, firmware, runtime, and model-ecosystem data that functions as a moat. Third, the biggest beneficiaries are local MoE deployments and long-context applications: inference-participating SSDs let end devices run MoE models far beyond their DRAM, and let long-context sessions stop paying HBM prices for every additional page of history.
What This Means for Developers
Three practical takeaways. When you estimate inference cost, stop staring at GPU pricing alone — price the KV cache’s residency tier, because the same long context costs an order of magnitude more in HBM than in flash. When you build long-context or multi-agent applications, judge inference frameworks by their multi-tier cache scheduling — Mooncake-style KV-cache disaggregation is the pattern — rather than reflexively stacking VRAM. And when you spec an AI PC or workstation for local MoE models, treat the AI SSD as a configuration parameter as load-bearing as memory capacity; Phison, Longsys, and Maxio-based products are already shipping into consumer hardware.
As the industry’s attention shifts from “how much compute” to “how much per token,” storage stops being a supporting act. Whatever HBM can no longer hold has become the AI SSD’s opportunity.
