EXPLAINER

NVIDIA Cosmos (World Foundation Models for Physical AI)

platformfreshness: volatilekey claims checked 2026-10-09

NVIDIA Cosmos is an open platform of world foundation models for physical AI — robots, autonomous vehicles and smart spaces. Cosmos 3 combines a reasoner and a generator in one omnimodal family released in three sizes.

Short answer: NVIDIA Cosmos is an open platform of world foundation models for physical AI. Cosmos 3 (released May 2026 per the repository) is a family of omnimodal world models with a Reasoner (text/vision → text) and a Generator (text, vision, sound, action → vision, sound, action) in three sizes — Super 64B, Nano 16B, Edge 4B. Use it for synthetic data, simulation and robot policy research; outputs still need validation for safety-critical use.

What is it?

Robots and self-driving cars need huge amounts of realistic training data, and real-world data is slow and risky to collect. A world model predicts what happens next in a scene, so it can generate realistic video and sensor-like data, simulate "what if" situations, and reason about physical situations ("a rolling ball may mean a child will follow"). Cosmos is NVIDIA's open set of such models, plus tools for curating data, training and evaluating.

Also known as: NVIDIA Cosmos, Cosmos 3, Cosmos world foundation models, Cosmos WFM, Cosmos Predict, Cosmos Transfer, Cosmos Reason, Cosmos3-Super.

Technical definition

The repository README (checked on the date below) describes NVIDIA Cosmos as an open platform for building physical AI applications and Cosmos 3 as its current model family: omnimodal world models built on a unified Mixture-of-Transformers architecture, with two surfaces — a Reasoner (inputs text and vision, outputs text; world understanding, grounding, task planning, embodied reasoning) and a Generator (inputs text, vision, sound and action; outputs vision, sound and action; world simulation, future prediction, synthetic data, policy learning). Three base models are listed: Cosmos3-Super (64B; H200/B200/GB200 class GPUs), Cosmos3-Nano (16B; RTX Pro 6000/H100/B200) and Cosmos3-Edge (4B; Jetson AGX Orin/Thor/RTX Pro 6000, released July 2026). Example post-trained checkpoints are labelled capability demonstrations, not products. Training (SFT, LoRA, RL, distillation) is in the separate Cosmos Framework; data curation and evaluation have separate Curator and Evaluator repositories; evaluation suites named include PAIBench, Physics-IQ and VLMEvalKit; serving options include an OpenAI-compatible API through vLLM, vLLM-Omni or NVIDIA NIM. A guardrail model is gated behind access conditions on Hugging Face and generation ships with guardrails on by default. Source code and models are released under OpenMDW-1.1. The README's limitations section says outputs can show artifacts in long, high-resolution or physically complex scenes (temporal inconsistency, object morphing, implausible dynamics) and that safety-critical applications need additional validation and system-level safety analysis. Earlier Cosmos families (Predict2.5, Transfer2.5, Reason1) have repositories that encourage migrating to Cosmos 3; the Predict2.5 and Transfer2.5 pages cite arXiv:2511.00062 and the Reason1 page cites arXiv:2503.15558.

Entity type: platform · Freshness class: volatile.

When should I use it?

Consider when you need open world models for robotics, driving or video-based physical reasoning and have suitable GPUs. Not a drop-in safety-validated simulator.

Implementation patterns

  • Generate varied synthetic scenes (weather, lighting, rare events) to augment real driving or robot data.
  • Post-train a base model on your own robot data, then evaluate before use.
  • Use the Reasoner to label or critique video clips for task planning.
  • Distil a large teacher into an edge-size model for on-device policies.

Technologies and frameworks

Commonly associated: Cosmos 3 (Super/Nano/Edge), Cosmos Framework, Cosmos Curator, Cosmos Evaluator, Hugging Face Diffusers, vLLM / vLLM-Omni / NIM, PAIBench, Physics-IQ. Names are examples of what exists, not recommendations, and projects change quickly.

Practical guide

  1. Decide the job: synthetic data, simulation, reasoning or policy.
  2. Pick the smallest model size that meets quality needs on your hardware.
  3. Run the repository quick-start with guardrails on.
  4. Evaluate outputs with physics and task metrics, not only visual inspection.
  5. Review the OpenMDW-1.1 licence and any gated-model terms before shipping.

Common failure modes

  • Treating generated video as physically accurate ground truth.
  • Training a perception model only on synthetic data and skipping real-world validation (Sim-to-Real Transfer).
  • Ignoring licence and gated-model conditions.
  • Choosing the largest model when hardware cannot serve it.
  • Using unvalidated outputs in safety-critical decisions.

Troubleshooting

  • First run is very slow — The README warns the first run downloads a large checkpoint and diffusion steps are compute-heavy.
  • Cannot download the guardrail model — Request access on Hugging Face and log in with a token from the account that was granted access.
  • Videos show morphing or impossible motion — Shorten clips or lower resolution; the README lists these artifacts as known limitations.

Contrasted with

Questions this page answers

  • What is NVIDIA Cosmos?
  • What are world foundation models?
  • What is the difference between Cosmos Reasoner and Generator?
  • Which Cosmos 3 model size should I use?
  • Is Cosmos open source?
  • Can Cosmos be used for autonomous vehicles?

Typical goals: understand what Cosmos is and which model to use; generate synthetic data for robotics or driving; evaluate whether a world model is safe to rely on.

Sources and verification

Status: key claims checked on 2026-10-09 against the primary sources listed below; everything else is not independently verified. The claims under "Checked against primary sources" were compared with the linked pages on that date. Pages change, so re-check anything time-sensitive before relying on it.

Checked against primary sources (2026-10-09):

  • Cosmos described as an open platform for physical AI; Cosmos 3 is the current family, omnimodal world models on a unified Mixture-of-Transformers architecture; Reasoner and Generator input/output modalities as listed.
  • Three base models: Super 64B, Nano 16B, Edge 4B with the hardware tiers named in the README; Cosmos 3 released May 2026 and Cosmos3-Edge July 2026 per the "What's new" list.
  • Training, curation and evaluation live in separate repositories; serving via vLLM, vLLM-Omni or NIM; guardrail model is gated.
  • Code and models released under OpenMDW-1.1; limitations and safety paragraph as summarised.
  • The Cosmos-Predict2.5 README encourages migrating to Cosmos 3 and cites arXiv:2511.00062; the Reason1 page cites arXiv:2503.15558.

Primary sources consulted:

Not independently verified:

  • The Cosmos 3 technical report itself was not read; its benchmark claims are not repeated.
  • The original January 2025 Cosmos platform paper and launch details.
  • Independent evaluations of Cosmos output quality or physical accuracy.
  • Whether the OpenMDW-1.1 licence is "open source" in the OSI sense (the README calls it a licence for source code and models).
  • Commercial availability, pricing and NIM packaging.

Freshness class: volatile — Products, versions, protocols, benchmarks, leaderboards, regulations and dates. Verify against the primary source before relying on specifics; re-review every 3 months.

Time-sensitive claims to verify before relying on them:

  • Model names, sizes, hardware targets and release dates.
  • Licence terms (OpenMDW-1.1 per the README).
  • Benchmark claims.
  • Which earlier Cosmos families remain supported.

Further reading and canonical sources (not all checked):

  • NVIDIA Cosmos repository README — github.com/NVIDIA/cosmos
  • Cosmos 3 technical report — research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf
  • Cosmos-Predict2.5 / Transfer2.5 paper — arXiv:2511.00062
  • Cosmos-Reason1 — arXiv:2503.15558