At SIGGRAPH 2026 (July 19–23, Los Angeles) NVIDIA unveiled Cosmos 3 — a foundation model for physical AI that fuses scene understanding, world generation and action prediction into a single model. Alongside it the company showed the edge variant Cosmos 3H and the Cosmos Dreams simulator. The talk was led by Ming-Yu Liu, VP of Cosmos Lab.
Key takeaways
- Cosmos 3 merges understanding, world generation and action prediction into one model — previously three separate components.
- Cosmos 3H has 4 billion parameters and runs on edge devices: Jetson, RTX and DGX Spark.
- Cosmos Dreams generates continuous future world states from a single image, following the laws of physics.
- A simulation that needed 64 GB300 GPUs now runs on a single RTX Pro 6000 workstation.
- The architecture is a Mixture of Transformers combining an autoregressive model with a generative diffusion model.
Three pieces of the suite
The previous Cosmos generation split tasks across separate models: understanding, generation and reasoning. Cosmos 3 merges them into a single omnimodal world model handling language, image, audio and robot actions. NVIDIA says the model achieves top results across eight benchmarks.
| Model | Role | Runs on |
|---|---|---|
| Cosmos 3 | Understanding, world generation and action in one model | Open on Hugging Face |
| Cosmos 3H | Control on edge devices (4B parameters) | Jetson, RTX, DGX Spark |
| Cosmos Dreams | Neural closed-loop simulator | 1× RTX Pro 6000 (was 64× GB300) |
One architecture
Under the hood is a Mixture of Transformers — an architecture pairing an autoregressive reasoning model with a generative diffusion model. That lets one system handle perception and planning as well as synthesizing an image of the future scene. NVIDIA released the model openly on Hugging Face.
Cosmos 3H: inference on the robot itself
The most interesting variant for robotics is the 4-billion-parameter Cosmos 3H, built for edge devices — Jetson, RTX and DGX Spark. It moves inference from the cloud onto the robot itself, cutting latency in the control loop. In the demo NVIDIA controlled a robot arm in real time using the model.
That is a meaningful shift. Controlling a robot from the cloud means latency and dependence on the link. A model small enough to fit an onboard Jetson removes both limits — a precondition for work in a real, unpredictable environment.
Cosmos Dreams and "compute is data"
Cosmos Dreams is a neural closed-loop simulator. From a single image it generates a continuous stream of future world states consistent with physics, including dangerous scenarios for training autonomous vehicles.
Behind it is NVIDIA’s "compute is data" thesis — instead of collecting data from the real world, you generate it with GPU compute. The company promises "better data", "better environments" (a thousand kitchen variants, warehouse types, weather conditions) and a "better starting point" via a pre-trained Cosmos.
Competition and context
Cosmos is NVIDIA’s answer to the same challenge VLA robots try to solve: a lack of physical-world training data. The difference is the approach — instead of collecting teleoperation data, NVIDIA generates synthetic worlds from a model. Knowledge is meant to be shared across different robot body types, from humanoids to autonomous vehicles and industrial machinery.
Why it matters
The bottleneck in robotics today is not hardware but data to train control policies. If a world model generates realistic, physics-consistent environments more cheaply than real-world collection, it changes the economics of the whole field. Moving inference to the edge is equally key — without it, world models stay a lab tool, not a production one.
What's next?
- NVIDIA announced Cosmos 3.5 with multi-view support.
- Cosmos 3 and 3H are available on Hugging Face — the real test will be adoption by third-party robot makers.
- Sim-to-real transfer remains to be verified: whether policies trained in Cosmos Dreams work on physical robots.
