Nvidia unveils new Cosmos world models, infra for robotics and physical uses

Nvidia on Monday unveiled a set of new world AI models, libraries and other infrastructure for robotics developers, most notable of which is Cosmos Reason, a 7-billion-parameter “reasoning” vision language model for physical AI applications and robots.

Also joining the existing batch of Cosmos world models are Cosmos Transfer-2, which can accelerate synthetic data generation from 3D simulation scenes or spatial control inputs, and a distilled version of Cosmos Transfers that is more optimized for speed.

During its announcement at the SIGGRAPH conference on Monday, Nvidia noted that these models are meant to be used to create synthetic text, image and video data sets for training robots and AI agents.

Cosmos Reason, per Nvidia, allows robots and AI agents to “reason” thanks to its memory and physics understanding, which lets it “serve as a planning model to reason what steps an embodied agent might take next.” The company says it can be used for data curation, robot planning, and video analytics.

The company also unveiled new neural reconstruction libraries, which includes one for a rendering technique that lets developers simulate the real world in 3D using sensor data. This rendering capability is also being integrated into open source simulator CARLA, a popular developer platform. There’s even an update to the Omniverse software development kit.

There’s new servers for robotics workflows, too. The Nvidia RTX Pro Blackwell Servers offers a single architecture for robotic development workloads, while Nvidia DGX Cloud is a cloud-based management platform. 

These announcements come as the semiconductor giant is pushing further into robotics as it looks toward the next big use case for its AI GPUs beyond AI data centers.