If you understand how a language model works, you are most of the way to understanding the technology that changed robotics this year. The jump is smaller than it looks.
A language model takes text and produces text. A vision-language-action model takes images and an instruction, and produces movement.
What Changed, Concretely
Teaching a robot a task used to mean explicit programming. Define the positions, script the motion, handle the edge cases, repeat for every new task. Expensive, brittle, and it required a specialist for each job.
VLA models replaced that with generalisation. Show the robot a scene, tell it in ordinary language what you want, and the model produces the actions. Systems like Figure Helix, Physical Intelligence π0, NVIDIA GR00T N1 and DeepMind Gemini Robotics collapsed work that used to take a team months.
The transition mirrors what happened in software when models replaced hand-written parsers. You stop enumerating cases and start relying on generalisation, with all the benefits and all the unpredictability that brings.
The Three Parts of the Name
| Part | What it does | Why it is hard |
|---|---|---|
| Vision | Interprets the scene from cameras | Lighting, occlusion, novel objects |
| Language | Turns an instruction into intent | Ambiguity, implied context |
| Action | Produces motor commands | Physics is unforgiving and continuous |
The action part is what makes this genuinely different from the models you know. Text can be wrong and get corrected on the next line. A wrong action puts a mug through a table, and there is no retry.
Why Robotics Is Harder Than Text
- No undo. Actions are irreversible in a way tokens never are.
- Latency is physical. A thoughtful two-second pause is a stumble when the robot is mid-step.
- Training data is scarce. The internet is full of text and thin on robot motion recordings.
- Reality does not repeat. The same instruction meets slightly different physics every time.
- Failures are visible. A model hallucinating in a document is a bad paragraph. A robot doing it is a video everyone shares.
Open Versus Vertical Approaches
Two strategies are competing, and both are legitimate.
Vertical integration means one company builds the body, the silicon and the model together. Figure runs Helix in-house, and Tesla takes a similar route with networks derived from its driving stack and its own chips. Tight control, potentially better performance, closed ecosystem.
The open approach is the opposite bet. Unitree open-sourced UnifoLM-VLA-0 in March 2026, letting robots perform household tasks from natural language commands. Commodity hardware plus shared models could commoditise the brain the way open models did for text.
Which wins depends on whether robot intelligence turns out to be a hardware-coupled problem or a general one. Nobody knows yet.
Conclusion
VLA models are the reason robotics moved from programming to instruction, and that is a genuine step change rather than a better demo. The remaining hard parts are reliability over long runs, manipulation of varied objects, and the fact that physical mistakes cannot be retried. Watch intervention rates rather than capability videos, because generalisation that works 95% of the time is a research triumph and a production problem.
Frequently Asked Questions
Are VLA models just language models with cameras?
Related architecture, different output space. Producing continuous motor commands that respect physics is a genuinely different problem from producing the next token.
Can I fine-tune a VLA model for my own task?
Increasingly yes, especially with open models, though you need demonstration data from your own environment. Collecting that data is usually the real project.
Do these models run on the robot or in the cloud?
On the robot for anything time critical, because a network round trip is unacceptable mid-motion. That constraint is a major driver of custom silicon in this field.