What is it?
A vision-language model can describe a scene. A VLA goes further: it sees through a robot's camera, reads your instruction and outputs the movements to carry it out. It reuses what large models know about the world so the robot can handle instructions it was not explicitly trained on.
Also known as: VLA, VLA models, RT-2, OpenVLA, pi0, π0, vision language action, robot foundation model.
Technical definition
RT-2 (Brohan et al., 2023) co-trains a vision-language model on web data and robot trajectories, representing actions as text tokens; OpenVLA (Kim et al., 2024) is an open 7B-parameter VLA trained on the Open X-Embodiment dataset; π0 (Black et al., 2024) combines a vision-language backbone with a flow-matching action head for dexterous, high-frequency control. Related: RT-1 (Brohan et al., 2022), Octo (Octo Model Team, 2024). Design choices: action tokenisation vs continuous heads (diffusion/flow), history/proprioception inputs, action chunking, and cross-embodiment training. Reported success rates depend on specific robots, tasks and evaluation protocols and are difficult to compare across papers.
Entity type: model · Freshness class: volatile.
When should I use it?
Relevant to teams building language-driven manipulation and general-purpose robot policies.
Implementation patterns
- Fine-tune a pretrained VLA on your robot's demonstrations (LoRA where possible).
- Action chunking for smoother control.
- Add a classical safety controller beneath the policy.
- Evaluate over many trials with varied scenes.
Technologies and frameworks
Commonly associated: RT-2, OpenVLA, π0, Octo, LeRobot, Open X-Embodiment dataset. Names are examples of what exists, not recommendations, and projects change quickly.
Practical guide
- Choose an open VLA base.
- Collect teleoperated demonstrations.
- Fine-tune.
- Evaluate across many trials and conditions.
- Wrap with safety limits.
Common failure modes
- Poor generalisation to new embodiments or scenes.
- Latency too high for dynamic tasks.
- Imprecise manipulation.
- Evaluation on too few trials.
- Unsafe actions without low-level constraints.
Troubleshooting
- Policy hesitates or jitters — Use action chunking/smoothing; check control frequency and latency.
- Fails in new lighting/scene — Add diverse demonstrations and augmentation; fine-tune.
Contrasted with
- Vision-Language Models (VLMs) — VLMs output text; VLAs output robot actions.
- Imitation Learning (Behaviour Cloning and DAgger) — Imitation learning is how VLAs are typically trained (behaviour cloning on demonstrations), at foundation-model scale.
Questions this page answers
- What is a vision-language-action model?
- How does RT-2 work?
- What is OpenVLA?
- How are VLAs different from VLMs?
- What is action chunking?
Typical goals: understand how language models control robots; compare VLA architectures.
Sources and verification
Status: not independently verified. This entry was authored on 2026-10-06 from general technical knowledge, without live checking of the sources below. Treat every source as a lead to confirm (identifier, title, authorship, current version) before publication or citation.
Freshness class: volatile — Products, versions, protocols, benchmarks, leaderboards, regulations and dates. Verify against the primary source before relying on specifics; re-review every 3 months.
Time-sensitive claims to verify before relying on them:
- Model names, sizes, success-rate claims and release status.
- Which VLAs are open-weights and under what licence.
- Any statement about commercial deployments.
Canonical sources (unverified leads):
- Brohan et al., RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (2023) — arXiv:2307.15818
- Kim et al., OpenVLA: An Open-Source Vision-Language-Action Model (2024) — arXiv:2406.09246
- Black et al., π0: A Vision-Language-Action Flow Model for General Robot Control (2024) — arXiv:2410.24164
- Open X-Embodiment Collaboration, Open X-Embodiment: Robotic Learning Datasets and RT-X Models (2023) — arXiv:2310.08864