At SIGGRAPH 2024, NVIDIA unveiled significant advancements in Universal Scene Description (OpenUSD). This expands the application of this universal 3D data exchange framework in robotics, industrial design, and engineering, enhancing developers' ability to construct high-precision virtual worlds for the next wave of AI development.
Newly launched features include NVIDIA NIM™ microservices tailored for AI models. These microservices generate OpenUSD language responses to user queries, produce OpenUSD Python code, apply materials to 3D objects, understand 3D space and physics to accelerate digital twin development, among others.
Additionally, there are new USD connectors designed for robot and industrial simulation data formats, along with various developer tools allowing massive NVIDIA RTX™ ray-tracing datasets to be streamed to Apple Vision Pro.
| Applying Generative AI to USD through NVIDIA NIM
NVIDIA introduces the world's first generative AI models for OpenUSD development in the form of NVIDIA NIM microservices. With these models, developers integrate generative AI copilots and agents into USD workflows, expanding possibilities in the 3D world and accelerating USD's application in new industrial sectors like manufacturing, automotive, and robotics.
Preview microservices include:
USD Code NIM microservice: Answers common OpenUSD queries and auto-generates OpenUSD-Python code based on textual prompts, feeding into applications like Pixar's USDView or NVIDIA Omniverse Kit for 3D data visualization.
USD Search NIM microservice: Enables developers to search vast OpenUSD, 3D, and image databases using natural language or image inputs.
USD Validate NIM microservice: Checks file compatibility with OpenUSD release versions and generates RTX-rendered path-traced images using NVIDIA Omniverse Cloud API.
Upcoming microservices include:
USD Layout NIM microservice: Allows users to assemble OpenUSD scenes based on spatial intelligence and a series of textual prompts.
USD SmartMaterial NIM microservice: Predicts and applies realistic materials to computer-aided design objects.
fVDB Mesh Generation NIM microservice: Generates OpenUSD-based networks from point cloud data, rendered via Omniverse Cloud API.
fVDB Physics Super-Res NIM microservice: Performs AI super-resolution on frames or frame sequences, generating high-resolution physical simulations based on OpenUSD.
fVDB NeRF-XL NIM microservice: Generates large-scale neural radiance fields in OpenUSD using Omniverse Cloud API.
| USD Connectors Bringing Generative AI to More Industries
New USD connectors for robot data formats and Apple Vision Pro streaming bring OpenUSD interoperability and advanced authoring capabilities to more industries.
To support industrial workloads using OpenUSD, NVIDIA expands its collaboration with Siemens, a global leader in industrial automation and software. Siemens integrates OpenUSD pipelines with its Simcenter simulation technology to support evidence-based decision-making and collaboration among key stakeholders.
This integration enables high-fidelity, real-time, and realistic visualization of complex simulation data, allowing enterprises to gain deeper insights into product performance in real operational environments. Siemens incorporates Omniverse into its Teamcenter product lifecycle management software suite, laying a foundation for this work.
NVIDIA also released a connector from Unified Robotics Description Format to OpenUSD, enabling robotics experts to seamlessly import their robot data into various applications for design, simulation, and reinforcement learning purposes.
To further expand the OpenUSD ecosystem, NVIDIA launched the OpenUSD Exchange Software Development Kit, assisting developers in creating robust OpenUSD data connectors of their own.
New developer tools and APIs are available through the NVIDIA Graphics Delivery Network (GDN), enabling the streaming of large-scale OpenUSD scenes from applications built on the Omniverse platform to Apple Vision Pro. Currently, these new tools and APIs are in the early access phase.
