On August 5, the Tsinghua-spunout team Sand.ai released MAGI-2 Preview — the first open-source video generation model built on a hundred-billion-parameter mixture-of-experts architecture. The numbers are striking: 114 billion total parameters with only about 6 billion activated per forward pass, generating a 10-second 1080p clip on eight H100s for roughly 0.5 RMB (about seven US cents) — a tenth of what mainstream models charge. On the Artificial Analysis video leaderboard, it sits sixth overall, outranking a row of closed-source models that activate more than a hundred billion parameters per token.
But the release matters for a harder reason than price: it is the first working answer to a question the open-source community has failed at for two years — why hasn’t video generation been able to scale?
Why Video Models Stayed Small
Over the past two years, open-source video generation models have mostly topped out in the low tens of billions of parameters, with the largest reaching perhaps thirty billion. Language models, by contrast, rode mixture-of-experts into the hundreds of billions — even trillions — of open weights. The two tasks differ, and parameter counts don’t translate directly across modalities. But the gap still says something: scaling a video model costs fundamentally more than scaling an LLM, for three compounding reasons.
- The sequences are enormous. A clip decomposes into vast numbers of spatial patches across continuous frames, and adding audio and text inflates sequence length and information density exponentially.
- The communication is brutal. MoE routing is dynamic — the number of activated experts fluctuates every step, so cross-device traffic jitters with it. Long video sequences turn that jitter into the fatal bottleneck.
- Training is fragile. Dynamic routing, load imbalance across experts, and multimodal data on top multiply every pathology LLM scaling already struggles with.
The conclusion is unavoidable: video scaling cannot copy the LLM playbook. Bolt-a-MoE-on-top does not work; the stack has to be rebuilt from the substrate up. Sand.ai is the first team to push a video MoE from whitepaper to hundred-billion parameters that actually runs.
Three Layers of Engineering: Architecture, Efficiency, System
The technical report — MAGI-2 Preview: Scaling Video Generation Models Efficiently — organizes the answer into three layers.
Architecture: One Stream for Audio and Video
Traditional pipelines run video and audio through separate backbones, then glue them together with cross-attention or post-processing. Every extra hop adds latency and compounds error — and that stitching is exactly where lip-sync failures come from. MAGI-2 Preview extends the single-stream architecture Sand.ai validated in its earlier daVinci-MagiHuman work: text, video, and audio tokens enter the same Transformer and exchange information directly in every self-attention layer, so picture and sound are generated by one shared backbone.
Inside that backbone, experts divide along two axes: shared experts that capture what the modalities have in common, and modality-specific experts that handle what is unique to language, image, or sound. The payoff is structural rather than cosmetic. When a character speaks, the lip shape, facial expression, and body motion are generated in the same pass as the voice. When an impact lands, objects on screen react in the same frame. When the music shifts, performance and camera movement adjust with it. Audio-visual sync stops being an engineering patch and becomes a native capability of the model.
Efficiency: 3,072 Experts per Layer
MAGI-2 Preview splits its 3,072-dimensional feature representation into twelve 256-dimensional subspaces. Each subspace independently selects 6 experts, so a single video token dynamically composes 72 of them — and each layer holds 3,072 experts in total. For comparison, mainstream MoE language models like DeepSeek run a few hundred experts per layer. Finer-grained experts mean finer-grained divisions of labor, and far more capability combinations the model can assemble. The price is scheduling pressure: at thousands of experts, conventional routing hits its efficiency ceiling fast.
System: Head Parallel Turns Jitter into a Constant
This is the most interesting move in the report. Conventional Expert Parallel routes first and communicates second: each token picks its experts, then gets shipped across devices to wherever those experts live, so traffic volume swings with every routing decision. Head Parallel inverts the order — token representations are distributed by attention head first, and each device performs expert selection and computation locally. Because the distributed data shapes are fixed, communication depends only on the input representation and no longer scales with the number of activated experts. The bottleneck that made long video sequences lethal simply gets routed around.
Around that core sit two in-house components: the MagiMoE kernel library, which fuses routing, sorting, and expert computation into a tight pipeline that avoids shuffling intermediate results through GPU memory, and the MagiMuon optimizer. Together they lift training efficiency by roughly 300%. The data side got equal rigor: rather than crude filtering, the team built a fine-grained annotation system covering 2,000 hours of professional video across more than 200 label categories — motion trajectories, sound characteristics, camera language — so that 3,072 experts genuinely differentiate instead of collapsing into redundancy.
Reading the Signal: The LLM MoE Playbook, Replayed for Video
Placed in context, MAGI-2 Preview hits three structural marks at once. First, it is a declaration that hundred-billion MoE no longer belongs to language models alone — the first time open weights, MoE, and hundred-billion scale have been combined for video. Second, it is the open-source twin of the same wave MiniMax H3 rode: H3 fired the price signal from the closed-source side, MAGI-2 fires the architecture manifesto from the open side, and both point the same direction — video generation is entering an industrial cost band where per-minute generation drops from thousands of RMB to tens. Third, it is another validation of a deliberately non-consensus route. Founder Cao Yue — a Tsinghua top-scholarship winner and co-first author of the Swin Transformer paper who co-founded Guangnian Zhiwai and led multimodal and vision research at BAAI — has built a track record of unfashionable bets: the autoregressive MAGI-1 (open-sourced April 2025 at 24B), then joint audio-video generation, now hundred-billion MoE. Kai-fu Lee has called Sand.ai “the DeepSeek of AI video generation,” and the analogy holds: skip the hype cycle, bet on contrarian architecture calls, then win through systems engineering.
Three downstream shifts are already visible. Content workflows get rewritten: when per-clip cost approaches rounding error, polish-one-shot gives way to generate-many-and-select, and advertising, short drama, animation, and game asset pipelines gain their first economically viable automation. The open-versus-closed comparison gets new axes: quality and price converge, so evaluation extends to whether a model can be fine-tuned further and whether its interface is hostage to one platform — the same split that defined LLMs, with closed vendors selling product experience and open weights competing on deployment and ecosystem. And the video infrastructure stack starts to layer: Head Parallel’s communication pattern poses new requirements for inference clusters, and ultra-fine-grained experts stress kernel libraries and routing algorithms in ways the public stack has not yet answered.
The Fine Print
Boundaries matter: this is still a preview. Clips max out at 10 seconds, generated through a two-stage pipeline of low-resolution preview denoising followed by a refiner that upscales to 1080p. The system assembles open components — the Qwen3.5-27B text encoder, the Wan2.2 VAE, and stable-audio-open-1.0 for audio. Sixth place on the Artificial Analysis board means genuinely close to the first tier, not on top of it. The official release and community validation are the real exam for this architecture.
What Each Audience Should Do Now
- Creators and studios: start prototyping generate-many-select-few pipelines. At tens-of-RMB-per-minute costs, AI generation belongs in pre-visualization now, replacing expensive live test shoots with cheap iterations.
- Developers: read the report for Head Parallel alone — the communication-reordering idea generalizes to any long-sequence MoE inference problem. Watch whether MagiMoE and MagiMuon ship with the official release.
- Technical decision-makers: add per-clip cost and audio-visual sync quality to vendor scorecards. For data-sensitive verticals — advertising, gaming, virtual humans — open weights plus local deployment is now an option worth serious evaluation.
The signals to watch next: the official non-preview release, support for longer durations, adaptation by mainstream inference frameworks such as vLLM, and the count of community fine-tunes that appear. The relay baton for hundred-billion-parameter video MoE has left Sand.ai’s hands and landed with the open-source community.
