Nvidia announced Nvidia Omniverse libraries — a collection of software components that give AI agents tools and skills to add physical AI capabilities to existing applications and prepare 3D content for simulation.
Robots, factories and autonomous systems need to be designed, tested and trained in simulation before they operate in the real world. Preparing 3D content for simulation takes more than realistic visuals — assets need the right structure, materials, scale, labels, sensors and physical properties.
Nvidia Omniverse libraries give AI agents tools and skills to build workflows, inspect scenes, flag issues and prepare assets, helping developers move faster from 3D content to simulation-ready environments.
“The physical AI era will be built in simulation first,” said Jensen Huang, CEO of Nvidia, in a statement. “With Omniverse libraries, AI agents become collaborators inside the 3D tools developers already use, building the simulation-ready worlds where robots, factories and autonomous systems are trained and tested long before they reach the real world.”
Software makers including SideFX and PTC are integrating Omniverse libraries for agent-ready sensor simulation, physics and asset validation, helping bring agentic AI into the applications and workflows developers and technical artists already use to prepare 3D
content.
“The most consequential platform that’s at the intersection of of computer graphics, physics simulation, and AI is our Nvidia Omniverse,” said Rev Lebaredian, vice president of physical AI simulation technology at Nvidia, said in a press briefing. “We started on Omniverse almost 10 years ago with the purpose of building a platform that allows the simulation of the physical world accurately enough, so we could build AIs that will operate in the real world, essentially robots.”
He noted it started in 2017 with Nvidia’s first robotic simulators. Omniverse continued to evolve.
“In the early days, we had big visions and dreams of what we would like it to do, but we were constrained by the technologies that were available to us. AI wasn’t quite developed yet,” Lebaredian said. “We didn’t have RTX in the very early days. We knew it was coming, but the simulation of light wasn’t accurate enough in in the GPUs we had at the time, and the various algorithms we needed to accelerate physics simulation weren’t quite quite ready. But we worked towards that. And and over the years, it kept evolving.”
He said Nvidia hit a problem with how to deploy these technologies and make it available to as many people as possible, because the the compute requirements needed needed to have very specialized Nvidia computers that were only available in Nvidia workstations, and that wasn’t always practical to have everywhere.
“So a few years ago, we started working towards disaggregating Omniverse, the whole platform, into microservices and making it available through the cloud. And that’s been a great boon for us. It made it easier to integrate into existing applications through our partners and make it available to more users everywhere,” he said.
Lebaredian added, “But today we we have the opportunity to make it even more accessible and make it more available through through our partners. What we are announcing is that we are disaggregating Omniverse even further into more atomic libraries that are agent friendly, making it extremely easy and almost automatic to integrate these technologies into existing applications.”
He said at Siggraph, Nvidia will be showing a few examples of integrations that are already starting to happen with the new Omniverse libraries, with PTC and SideFX’s Houdini tool.
Omniverse Libraries Add Simulation Capabilities to 3D Applications
The new Omniverse libraries, such as ovrtx, ovphysx and CAD-to-SimReady skills, are openly available on GitHub and give AI agents tools to build workflows for inspecting scenes, testing changes and preparing 3D assets for simulation. A new blueprint for integrating Omniverse libraries in Blender is also now available.
The libraries’ key capabilities include:
● Nvidia RTX sensor simulation: ovrtx helps applications generate camera, lidar, radar and other sensor outputs from 3D scenes, so developers and AI agents can test how physical AI systems may perceive virtual environments.
● Physical behavior: ovphysx uses GPU-accelerated physics to bring realistic behavior to 3D scenes using properties such as collisions, mass, friction and motion, so teams can first test how objects and systems interact in simulation.
● Simulation-ready 3D objects: CAD-to-SimReady skills help convert computer-aided design (CAD) data to SimReady assets built on OpenUSD, giving 3D content the properties needed for physical AI simulation and virtual testing.
Software Makers Build With Omniverse Libraries
Software makers including SideFX and PTC as well as startups Palatial, ForgeCAD and MoonlakeAI, are among the first to adopt and build with Omniverse libraries for agents.
SideFX is using OpenUSD workflows, as well as ovrtx and ovphysx libraries, to explore how
agents can help integrate Omniverse libraries into its Houdini procedural 3D content creation workflows, giving technical artists a path to generate, test physics and prepare content for simulation.
“Procedural 3D creation is essential to building the complex, controllable worlds needed for
simulation, robotics and industrial AI,” said Kim Davidson, president and CEO of SideFX, in a statement. “With Nvidia Omniverse libraries and OpenUSD, SideFX is exploring how agent-ready tools can support Houdini workflows, helping technical artists review, test and prepare
procedural content for simulation while staying in control of the creative process.”
The PTC Onshape CAD and product data management (PDM) platform is using OpenUSD
and ovrtx to connect cloud-native design workflows with physical simulation, helping product design content stay connected with CAD, PDM, collaboration and simulation workflows.
“Engineering teams are seeking more connected ways to design, collaborate and simulate
throughout the development process,” said Neil Barua, president and CEO of PTC, in a statement. “PTC’s work with Nvidia supports that broader vision, while Nvidia Omniverse libraries help enable simulation-ready workflows that bring validation and testing closer to where products are designed.”
On display at Siggraph, “SimReady” Blender is a sample workflow built in Blender with Nvidia Omniverse libraries and the Nvidia Nemotron Ultra open model, showing how software makers can add agent-ready simulation capabilities — including Nvidia RTX sensor simulation, physics and validation — into existing 3D applications while keeping creators in control. This is now openly available as a blueprint for integrating Omniverse libraries in Blender.
The demo also previews how these workflows can run locally, from compact RTX-powered
systems with On display at Siggraph, “SimReady” Blender is a sample workflow built in Blender with Nvidia Omniverse libraries and the Nvidia Nemotron Ultra RTX Spark.
On display at Siggraph, “SimReady” Blender is a sample workflow built in Blender with Nvidia Omniverse libraries and the Nvidia Nemotron Ultra GB300-powered systems with On display at Siggraph, “SimReady” Blender is a sample workflow built in Blender with Nvidia Omniverse libraries and the Nvidia Nemotron Ultra DGX Station.
RTX Spark systems will be available this fall from ASUS, Dell Technologies, HP, Lenovo, Microsoft Surface and MSI, with models from Acer and Gigabyte to follow. DGX Station systems are available to order from ASUS, Dell, Gigabyte, HP, MSI, Supermicro and
Exxact.
Startups are also using Omniverse libraries and skills to add agent-assisted asset and scene preparation workflows. Palatial is using Omniverse CAD-to-SimReady skills to automate the creation and validation of SimReady assets at scale from CAD inputs.
Lightwheel is using Omniverse Content Agents powered by OpenUSD in its SimReadyGen
technology to generate physically accurate SimReady assets from text prompts.
ForgeCAD and MoonlakeAI are exploring agent-driven 3D content workflows that use Omniverse capabilities to help generate, augment and prepare assets for physical AI simulation.