You have seen the robot fails: a plug rubbed against an outlet edge over and over, never seating; a plastic bottle crushed by a grip that misjudged force; a neatly folded towel collapsing the moment it’s released. The vision system judged everything correctly — and the physical contact still failed.
This is the embodied-AI industry’s acknowledged “last centimeter” problem: cameras see spatial position, but nothing senses whether contact happened, whether the object hit an edge, whether grip force is overloading, or whether the surface is slipping. Vision drives 99% of the path planning — and goes blind at the instant of touch.
On July 25, Shanghai-based NeoteAI, together with Fudan University’s Institute of Trustworthy Embodied Intelligence, released three technical reports that promote touch from “auxiliary modality” to core infrastructure. Together they form a complete blueprint: one tactile data foundation plus two technology routes with different emphases. And both data and models are open source.
First, Fix the Dialect Problem: a 30,000-Hour Tactile Data Foundation
The biggest obstacle to scaling tactile data has long been sensor incompatibility. Gel deformation maps, capacitance matrices, six-axis force vectors — each vendor’s sensors speak a different dialect, and none fuses in one model. NeoteAI’s NeoData dataset attacks that wall directly:
-
Over 30,000 hours of vision-tactile interaction video;
-
Roughly 1.4 million manipulation segments, 3.3 billion timesteps;
-
8 billion RGB frames paired with 10 billion tactile frames;
-
Six robot platforms, including Franka, Piper, and UR5e;
-
450 real long-horizon tasks — folding laundry, plugging connectors, pouring liquid — collected by 90 operators;
-
5,000 hours of vision-tactile video already open-sourced.
Alongside the dataset comes NeoForce, a unified tactile representation model. Whatever the underlying sensor hardware, it learns transferable, temporally structured tactile representations — where was pressure applied, how hard, is there lateral shear? This “tactile lingua franca” cracks the cross-device fusion problem.
VTLA: Tactile Prediction, So VLAs Stop Being Half a Beat Late
With the data foundation laid, the harder question follows: how does touch actually enter a VLA (Vision-Language-Action) architecture? Naively concatenating tactile images with RGB performs poorly, for two reasons. Tactile signals spike only at the instant of physical contact and stay silent most of the time, so they drown in the flood of visual tokens. Worse, even a well-extracted tactile feature is lagging feedback about an action already taken — rearview-mirror perception, forever half a beat behind in dynamic interaction.
NeoteAI’s VTLA takes a different road: instead of relying on belated current touch, it predicts tactile evolution over the next 50 steps. The model encodes current tactile input into compact latent tokens and, combined with visual context, anticipates slip and force trends so the action module adjusts in advance.
The results are vivid:
-
Plug insertion: VTLA 85% success versus 60% for vision-only counterparts;
-
Key extraction: VTLA 99% versus 35%.
VTLA also breaks imitation learning’s dependence on expert success trajectories. Combining post-deployment data with tactile information and human-in-the-loop training, the model learned to recognize execution phases — even complex behaviors like “regression” and “recovery.” With offline reinforcement learning on top, long-horizon success rates jumped: towel folding 50% → 95%, backpack packing 35% → 80%, carton folding 20% → 75%.
TWAM: Rebuilding “Tactile Imagination” Inside a World Model
If VTLA is a prediction radar, TWAM rebuilds the robot’s cognitive core — placing touch inside the world model so the robot can rehearse an operation’s viewpoint and feel in its own imagination before acting. Existing world models mostly generate future visual frames, and driving real robots with them hits a brutal physical-alignment gap: the picture shows the cup grasped, but did the fingers hold it? The plug aligned, but will it jam next second? That information is simply missing.
TWAM uses a mixture-of-experts architecture with three asynchronous expert networks — video, tactile, action. The video expert warm-starts from a pretrained model while the tactile and action experts use narrower structures, bringing total parameters to just 7.2 billion — half of a full-width design — with startling results:
-
Average simulation success rate 84.5%, against 36% for the strongest world-model baseline;
-
Average success on 8 real-robot tasks 46.3%, against 21.9% for contemporaries.
Ablations confirm the design: removing either “future tactile prediction” or “current tactile conditioning” causes significant performance drops. Touch’s indispensability inside world models is now empirically settled.
From Seeing the World to Feeling It Out
Read together, the three reports send a clear industry signal: 𝒩₀-Foundation validates that the tactile modality has vision-like scaling potential; VTLA proves touch integrates naturally into existing VLA architectures; TWAM establishes touch as a first-class citizen at the top of world-model design, truly on par with vision.
The cognitive paradigm of robots is evolving from vision-driven to vision-touch fusion. Robots are no longer understanding the world only by looking — they are starting to verify hypotheses and plan next actions through active tactile exploration, the way infants do. “Seeing the cup is not holding it steady; recognizing the outlet is not seating the plug” — that piece of physical common sense has finally been written into the base logic of robots.
A long road remains before general embodied intelligence can “touch once and know the heft”: real-world data collection costs, cross-platform generalization, and inference latency are all open engineering problems. But this body of work has substantively moved touch from niche research topic to core embodied-AI infrastructure. The next chapter of embodied intelligence may be less about teaching robots to see the world — and more about teaching them to feel it out.
Project: research.neoteai.com · Company: neoteai.com
