NVIDIA Accelerates Development of Humanoid Robotics

SIGGRAPH—To accelerate humanoid development globally, NVIDIA today announced it will provide the world’s leading robot makers, AI model developers, and software companies with a suite of services, models, and computing platforms to develop, train, and build the next generation of humanoid robotics.

Among the offers there are new ones NVIDIA NIM™ Microservices and frameworks for robot simulation and learning, NVIDIA OSMO an orchestration service for running multi-stage robotics workloads and an AI and simulation-based teleoperation workflow that allows developers to train robots using small amounts of human demo data.

“The next wave of AI is robotics, and one of the most exciting developments is humanoid robots,” said Jensen Huang, NVIDIA founder and CEO. “We’re advancing the entire NVIDIA robotics stack, opening up access for humanoid developers and enterprises around the world to use the platforms, acceleration libraries, and AI models that best fit their needs.”

Accelerating Development with NVIDIA NIM and OSMO

NIM microservices provide pre-built containers powered by NVIDIA inference software that enable developers to reduce deployment times from weeks to minutes. Two new AI microservices will enable roboticists to improve simulation workflows for Generative physics AI In NVIDIA Isaac SIM Card™, a reference application for robotics simulation built on the NVIDIA Omniverse™ platform.

The NIM MimicGen microservice generates synthetic motion data based on teleoperated data recorded from spatial computing devices like Apple Vision Pro. The NIM Robocasa microservice generates simulation-ready robot tasks and environments in OpenUSDa universal framework for developing and collaborating within 3D worlds.

NVIDIA OSMO, available now, is a cloud-native managed service that enables users to orchestrate and scale complex robotics development workflows across distributed computing resources, whether on-premises or in the cloud.

OSMO dramatically simplifies robot training and simulation workflows, reducing deployment and development times from months to less than a week. Users can visualize and manage a range of tasks, such as generating synthetic datatraining models, driving reinforcement learning and the implementation of large-scale in-the-loop software testing for humanoids, autonomous mobile robots and industrial manipulators.

Improving data capture workflows for humanoid robot developers

Training basic models of humanoid robots requires an incredible amount of data. One way to collect human demonstration data is to use teleoperation, but this process is becoming increasingly expensive and time-consuming.

An NVIDIA and Omniverse AI-enabled teleoperation reference workflow, presented at the SIGGRAPH computer graphics conference, enables AI researchers and developers to generate massive amounts of synthetic motion and perception data from a minimal amount of remotely captured human demonstrations.

First, developers use Apple Vision Pro to capture a small number of teleoperated demos. Then, they simulate the recordings in NVIDIA Isaac Sim and use the MimicGen NIM microservice to generate synthetic datasets from the recordings.

Developers train the GR00T Project humanoid foundation model with real and synthetic data, saving developers time and reducing costs. They then use the Robocasa NIM microservice in Isaac Laboratorya robotics learning framework, to generate experiments to retrain the robot model. Throughout the workflow, NVIDIA OSMO transparently assigns compute tasks to different resources, saving developers weeks of administrative tasks.

Fourier, a general-purpose robotics platform company, sees the benefit of using simulation technology to synthetically generate training data.

“Developing humanoid robots is extremely complex and requires an incredible amount of real-world data, painstakingly collected from the real world,” said Alex Gu, CEO of Fourier. “NVIDIA’s new generative AI and simulation development tools will help bootstrap and accelerate our model development workflows.”

Expanding Access to NVIDIA Humanoid Development Technologies

NVIDIA offers three computing platforms to facilitate the development of humanoid robotics: NVIDIA AI supercomputers to train models; NVIDIA Isaac Sim powered by Omniverse, where robots can learn and hone their skills in simulated worlds; and NVIDIA Jetson™ Thor humanoid robot computers to run models. Developers can access and use all platforms based on their specific needs.

Thanks to a new NVIDIA Humanoid Robot Development ProgramDevelopers can get early access to new offerings as well as the latest versions of NVIDIA Isaac Sim, NVIDIA Isaac LabJetson Thor and Project GR00T general purpose humanoid foundation models.

1x, Boston Dynamics, ByteDance Research, Field AIFigure, Fourier, Galbot, LimX Dynamics, MentoredNeura Robotics, RobotEra and Skild AI are among the first to join the early access program.

“Boston Dynamics and NVIDIA have a long history of working together to push the boundaries of what’s possible in robotics,” said Aaron Saunders, CTO of Boston Dynamics. “We’re excited to see the fruits of this work accelerate the industry as a whole, and the Early Access program is a fantastic way to get access to best-in-class technology.”

Availability

Developers can now join the NVIDIA Humanoid Robot Developer Program to access NVIDIA OSMO and Isaac Lab, and will soon have access to NVIDIA NIM microservices.

Learn about the latest advances in generative AI and accelerated computing by Listen to Huang’s Fireside Chats at SIGGRAPH, the premier computer graphics conference, running through August 1 in Denver.