Nvidia’s Cosmos Models and Omniverse tools aim to accelerate robotics development with high-fidelity simulation, AI reasoning, and scalable synthetic data generation. Image Source: ChatGPT-5
Nvidia Unveils Cosmos Reason, Omniverse Libraries, and AI Infrastructure for Robotics
Key Takeaways:
Nvidia introduced Cosmos Reason, a 7B-parameter reasoning vision language model for robotics and physical AI.
Cosmos Transfer-2 and a distilled Cosmos Transfer dramatically speed synthetic data generation from 3D simulations.
New Omniverse SDKs and NuRec libraries improve world capture, reconstruction, and simulation realism.
Nvidia RTX PRO Blackwell Servers and DGX Cloud provide infrastructure for training, simulation, and deployment at scale.
Early adopters include Boston Dynamics, Amazon Devices, Magna, Uber, and Accenture.
Cosmos Models: Teaching Robots to Reason and Adapt
At SIGGRAPH 2025, Nvidia announced major updates to its Cosmos world foundation models (WFMs) — AI models designed to help robots and physical AI agents perceive, reason, and act in the real world. Since the debut of OpenAI’s CLIP model, vision language models (VLMs) have advanced computer vision by excelling at tasks such as object and pattern recognition. However, these models still struggle with multistep tasks, and they lack the ability to manage ambiguous instructions or adapt to novel situations.
The highlight is Cosmos Reason, an open, customizable 7-billion-parameter reasoning vision language model (VLM) designed to overcome these limitations by combining prior knowledge, physics understanding, and common sense to reason and act like a human in the real world.
Cosmos Reason can be used for robotics and physical AI applications including:
Data curation and annotation, enabling developers to automate the high-quality curation and labeling of massive, diverse training datasets.
Robot planning and reasoning, acting as the brain for deliberate, methodical decision-making in a robot vision language action (VLA) model. Cosmos Reason lets robots interpret environments and, given complex commands, break them down into tasks and execute them using common sense — even in unfamiliar environments.
Video analytics AI agents built on the Nvidia Blueprint for video search and summarization, capable of extracting valuable insights and performing root-cause analysis on massive volumes of video data.
The technology is already being used by:
Magna, integrating Cosmos Reason into its City Delivery platform — a fully autonomous, low-cost solution for instant delivery — to help vehicles adapt more quickly to new cities. Cosmos Reason adds world understanding to the vehicles’ long-term trajectory planner, improving route decision-making and adaptability.
Uber — annotating and captioning autonomous vehicle (AV) training data.
VAST Data, Milestone Systems, and Linker Vision — enhancing city traffic monitoring, safety, and industrial visual inspection.
Cosmos Transfer-2: Fast, Photorealistic Synthetic Data
Cosmos world foundation models (WFMs) — downloaded over 2 million times — allow developers to generate diverse datasets for training robots at scale using text, image, and video prompts.
Synthetic data is essential for training physical AI, especially in scenarios where collecting real-world data is expensive or impractical. Cosmos Transfer-2, coming soon, generates photorealistic training data from ground-truth 3D simulations or spatial control inputs like depth maps, segmentation, edge detection, and HD maps.
A distilled version reduces what was previously a 70-step process into a single step, enabling developers to run the model at high speed on Nvidia RTX PRO Servers.
Adopters such as Lightwheel, Moon Surgical, and Skild AI are already using Cosmos Transfer to train AI in a wide range of simulated conditions, allowing robots to handle more diverse, unpredictable real-world scenarios.
New Omniverse Libraries: Building Blocks for Robotics Simulation
Nvidia is rolling out new Omniverse software development kits (SDKs) and libraries designed to make it easier for developers to create and deploy industrial AI and robotics simulation applications with high physical accuracy. These tools are aimed at connecting different simulation platforms, capturing the real world in detail, and generating synthetic data for AI training.
Nvidia says the new Omniverse libraries and Cosmos models are powered by NVIDIA RTX PRO™ Servers and NVIDIA DGX™ Cloud, enabling developers worldwide to create physically accurate digital twins, capture and reconstruct the real world in simulation, generate synthetic data for training physical AI models, and build AI agents with a deep understanding of the physical world.
“Computer graphics and AI are converging to fundamentally transform robotics,” said Rev Lebaredian, vice president of Omniverse and simulation technologies at Nvidia. “By combining AI reasoning with scalable, physically accurate simulation, we’re enabling developers to build tomorrow’s robots and autonomous vehicles that will transform trillions of dollars in industries.”
Key capabilities include:
New Omniverse SDKs — Now available, they introduce data interoperability between MuJoCo (MJCF) and Universal Scene Description (OpenUSD), enabling more than 250,000 MuJoCo robot learning developers to run their simulations in Omniverse and exchange data seamlessly across platforms.
Omniverse NuRec libraries — advanced neural rendering capabilities featuring RTX ray-traced 3D Gaussian splatting, which reconstructs physical environments in 3D from sensor data for use in simulation.
Nvidia Isaac Sim™ 5.0 and Isaac Lab 2.2 — open-source robot simulation and learning frameworks now available on GitHub. Isaac Sim includes NuRec neural rendering and new OpenUSD-based robot and sensor schemas designed to help developers close the simulation-to-reality gap in robotics.
Integration with leading simulators and tools — NuRec rendering is now built into CARLA, the open-source autonomous vehicle simulator used by over 150,000 developers. In the AV sector, Foretellix is combining NuRec with Nvidia Omniverse Sensor RTX™ and Cosmos Transfer to boost synthetic data generation, enabling highly detailed and physically accurate driving scenarios. Meanwhile, Voxel51’s FiftyOne — a data engine for visual and multimodal AI — supports NuRec to speed up reconstruction dataset preparation. The platform is already in use at companies such as Ford and Porsche.
Industry adoption is already underway:
Boston Dynamics, Figure AI, Hexagon, RAI Institute, Lightwheel, and Skild AI are using Omniverse libraries, Isaac Sim, and Isaac Lab to accelerate AI robotics development, while Amazon Devices & Services is applying the same tools to power a new manufacturing solution.
By combining cross-platform interoperability, physically accurate reconstruction, and synthetic data generation, these libraries help robotics developers test and refine AI systems in rich, realistic environments — before those systems ever touch real-world hardware.
AI Infrastructure to Power Robotics Anywhere
To ensure these tools can be deployed at scale, Nvidia introduced RTX PRO Blackwell Servers, a single architecture for training, synthetic data generation, robot learning, and simulation workloads.
Additionally, DGX Cloud — now on Microsoft Azure Marketplace — gives developers a fully managed platform for streaming OpenUSD- and RTX-based applications without managing complex infrastructure. Early adopters include Accenture and Hexagon.
Training the Workforce and Expanding the Ecosystem
Recognizing the need for skilled developers, Nvidia also announced:
An OpenUSD Curriculum and Certification program, supported by Adobe, Amazon Robotics, Autodesk, Pixar, Siemens, Hexagon, and others.
An open-source partnership with Lightwheel to integrate advanced robot policy training into Isaac Lab, including parallel reinforcement learning and benchmark-ready assets for manipulation and locomotion tasks.
You can watch the full Nvidia Research special address at SIGGRAPH here.
Q&A: Nvidia’s Robotics AI Announcements
Q: What is Cosmos Reason?
A: A 7B-parameter reasoning vision language model that helps robots and AI agents understand environments, plan multi-step actions, and adapt using physics and common sense.
Q: What is Cosmos Transfer-2?
A: A synthetic data generation model that creates photorealistic training data from 3D simulations and spatial inputs, with a distilled version that runs up to 70x faster.
Q: What are the new Omniverse libraries for?
A: To provide SDKs and neural rendering tools for building industrial AI and robotics simulations, with features like MuJoCo–OpenUSD interoperability and 3D Gaussian splatting.
Q: What does Omniverse NuRec do?
A: It enables 3D reconstruction of real-world environments from sensor data for use in simulations, improving the realism and accuracy of robotics testing.
Q: Who is already using these tools?
A: Companies including Boston Dynamics, Amazon Devices, Magna, Uber, Accenture, and Hexagon.
What This Means
By integrating reasoning AI, synthetic data generation, and high-fidelity simulation into a single ecosystem, Nvidia is equipping robotics developers with the tools to build and deploy smarter, more adaptable physical AI systems.
The combination of Cosmos Reason’s cognitive capabilities, Omniverse libraries and NuRec’s simulation realism, and RTX-powered infrastructure represents a step toward closing the simulation-to-reality gap — a key challenge in robotics.
As industries from autonomous vehicles to manufacturing adopt these tools, Nvidia is positioning itself at the center of a robotics and physical AI boom that could rival the impact of its AI data center business into real-world physical automation.
Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.