The large model that can watch a video and reason across modalities — Keye-VL 1.5 — has been open-sourced by Kuaishou. Against its preview version, Keye-VL 1.5 upgrades temporal grounding and adds cross-modal reasoning, and it introduces a novel Slow-Fast dual-path encoding mechanism that delivers a 128K context window while balancing speed and detail. On scores: 73.0 on the Video-MME short-video benchmark, leading same-size models across a broad spread of general, video, and reasoning benchmarks.
Three Headline Abilities, One Demo
The team names three highlights: 128K context, standout short-video understanding, and stronger reasoning — and all three show at once in video understanding. Temporal grounding: Keye-VL 1.5 pinpoints when a specific object appears in a video, precise to 0.1 seconds. In one 26-second e-commerce clip introducing a power bank — including a segment showing it packed into a bag to demonstrate portability — Keye-VL 1.5 answered correctly that the bag appears at 22.3-23.8 seconds. Competing models either timed to the second and got it wrong, or skipped timestamps entirely and counted shots instead.
Description: the model describes video scenes and details in depth, producing a faithful accounting of the clip’s content. Video reasoning: it infers why later events happen from earlier footage. In a pet video where a big dog bites a small dog’s ear, Keye-VL 1.5 was asked to analyze why — the answer was written on-screen in the video, but Keye’s explanation went further, reinforcing its reading with the dog’s subsequent behavior in the footage. On benchmarks, Keye-VL 1.5 posts the highest scores among same-size models across public benchmarks and internal evaluations: beating Qwen2.5-VL 7B on general suites (MMBench, OpenCompass) with multiple SOTAs, exceeding peers on vision-reasoning sets (AI2D, OCRBench), and leading on video (Video-MME, TempCompass, LongVideoBench) plus visual math and logic. The team also built 200 internal multi-dimension evaluations targeting short-video applications, where Keye-VL-1.5-8B scored 3.53 of 5 across five human-labeled metrics (correctness, completeness, relevance, fluency, creativity) — up 0.51 from the preview and ahead of the comparison competitors.
Video Understanding Gets Fast-Slow Thinking Too
The architecture is a three-part stack: a vision transformer (ViT), an MLP projector, and a language decoder. The ViT cuts input images or video frames into 14×14 patch sequences and captures global spatial relations with self-attention — initialized from SigLIP-400M, whose image-text contrastive pretraining gives visual features native semantic alignment. To handle arbitrary resolutions without cropping, the team added native-resolution support: absolute position vectors are interpolated to any size, and 2D RoPE (rotary position encoding) strengthens high-resolution extrapolation. Patch features pass through a two-layer MLP projector into the language decoder — Qwen3-8B — whose positional encoding is extended to 3D RoPE: the conventional two rotational angles plus a time/depth angle, so text tokens and visual tokens sort into one unified timeline.
The centerpiece is the Slow-Fast encoding strategy, aimed at video’s fundamental conflict between frame rate and resolution. The model computes patch-level cosine similarity between consecutive frames: if similarity to the most recent “slow frame” (a change frame — low count, high resolution) exceeds 95 percent, the frame is classed a “fast frame” (a static frame — high count, low resolution); otherwise it becomes a new slow frame. Slow frames keep full resolution; fast frames get 30 percent of a slow frame’s token budget; binary search pins total spend precisely to the limit; and timestamp special tokens mark frame boundaries in the sequence. The result: performance and compute cost balanced by design, which is what makes a 128K video context economically possible.
Four-Stage Pretraining and the Weight-Merging Trick
Pretraining runs a four-stage progressive pipeline ordered “unimodal before multimodal, alignment before extension”: Stage 0 — vision-encoder pretraining, continuing the ViT on SigLIP contrastive loss alone to strengthen visual semantics on internal data; Stage 1 — cross-modal alignment, freezing ViT and Qwen while training only the MLP projector; Stage 2 — multi-task pretraining, unfreezing the full network for end-to-end optimization at 8K context, building baseline visual understanding; Stage 3 — annealing, fine-tuning on curated high-quality data while introducing long-context modality data to stretch context to 128K. The pretraining corpus exceeds one trillion tokens, drawing on public multilingual image-text sets (LAION, DataComp, CC12M) plus large-scale self-built images, video, and text. After the four stages, the team merges “homogeneous” weights — same architecture, similar task goals, varied data mixes, sample difficulty, and seeds — with “heterogeneous” expert weights fine-tuned on targeted domains (an OCR-Expert trained on extra-collected and synthesized license-plate, receipt, and street-sign data, for example). Because the architectures match, merging is direct weight interpolation: zero inference-time overhead, with the experts’ local strengths injected into the general model and bias reduced.
Post-Training: Four Stages From Baseline to Alignment
Post-training runs four phases: supervised fine-tuning combined with multi-preference optimization (MPO) to set the output-quality baseline; a five-step pipeline of large-scale chain-of-thought data as a cold start, supplying reliable reasoning demonstrations; multi-turn reinforcement learning under verifiable rewards using the GSPO algorithm with progressive prompt sampling, systematically lifting general reasoning; and a final alignment RL stage with rule-based, generative, and model-based triple-source rewards, emphasizing instruction following, format consistency, and user-preference alignment. In the SFT stage, the team built a candidate pool of 7.5 million multimodal Q&As, mapped samples onto seventy thousand task labels via TaskGalaxy, and deliberately raised the share of hard types. MPO then used 250,000 open-source, 150,000 text-only, and 26,000 human-annotated samples, constructing high-low quality pairs from Keye-Reward model scores and human ratings, with a preference loss teaching the model to favor the higher-scoring answer to the same question — lifting response quality further.
Why It Matters
Keye-VL 1.5 matters for what it proves about video’s economics. The Slow-Fast strategy is a systems answer to the question every video LLM faces — how to afford a million tokens of footage — and its answer (spend where things change, skim where they don’t) mirrors how humans watch. The 0.1-second temporal grounding is the capability that converts video understanding from description into utility: e-commerce indexing, content moderation, ad placement, highlight extraction all need timestamps, not summaries. And the open-source release, coming from a company whose core business is short video, puts a production-grade video brain into everyone’s hands — with the internal 200-case short-video benchmark a hint of where the team’s own roadmap points. For developers building on video, Keye-VL 1.5 just became the default first test.
