Nvidia Uses Apple Vision Pro for Robotics Training

One of the major announcements that came out of the recent CES2025 was that Nvidia – the computing tech giant from Santa Clara – is looking to use the Apple Vision Pro in some of its robotics works, which is a massive boost in confidence for the Apple device.

Nvidia always have lots to reveal at shows like CES, and we think that the work that they are now doing with Apple is so significant that it’s worthy of its own article.

Nvidia and Apple; What’s the Deal?

Not very long ago, Nvidia was simply a chip manufacturer who has since expanded into several other technological branches. The company effectively dominated CES25 with its latest developments in cutting-edge technologies. The company showcased its robotics and artificial intelligence initiatives, which the corporation hopes to catapult them into many popular technological spheres. For Nvidia, the two go hand in hand; while the company keeps researching robots and self-driving vehicles, it is building the required AI foundation to run this next generation of technologies and become a major driving and supporting force in the next generation of connected devices.

Nvidia’s determination to be a major part of future technology is now driving its focus on training their robotics systems, environmental artificial intelligence concepts, and self-driving vehicle frameworks. This last sector will need to routinely analyse and interpret real-world environments in order to run properly, and will depend much on spatial computing and Nvidia’s Omniverse platform. To enable these systems to understand their world, Nvidia are partnering with Apple and specifically with the Vision Pro headset to help train future robot systems.

How Does GR00T Help?

In recent years, robotics has been an essential component in Nvidia’s massive growth, which has become a core factor. The former chip manufacturer made the announcement of Issac GR00T which they have set up as a platform for general-purpose robot foundation models and data pipelines to accelerate humanoid robotics and AI interactions with the real world. 

This project has been hailed as a transformative step in the field of humanoid robots, and a major resource in the humanisation of robotic movements. By using their Omniverse workflow, Nvidia developers can now use Apple Vision Pro devices to emulate and record human movements as 3D spatial data, which effectively creates a digital twin of the person that a robot can imitate in a game or in real life applications. But how does this work?

As its name suggests, imitation learning is precisely that. An individual executes an action, and the automaton replicates it. It is a successful approach to educating systems that are intended to automate existing tasks, such as those in warehouses and factories, where the initial iterations of humanoids are being deployed.

This type of teleoperation is essential in all of this sequencing. Through a detailed analysis of a real-world environment and movements; and it can be used to remotely instruct robots, instantly digitising one’s actions.

Users can generate these actions via Apple’s Vision Pro with the Issac GR00T Blueprint. The robot can subsequently implement the same procedure repeatedly in any simulation after capturing it as a digital twin. Once learned, it is captured forever and can be built upon to create increasingly complex routines. Movements can be added together and edited to mimic everything that humans do.

Using the Apple technology for this kind of work significantly shortens the process of training robots and getting new routines established in an AI library. Nvidia CEO and founder Jensen Huang said “The next wave of AI is robotics and one of the most exciting developments is humanoid robots. We’re advancing the entire Nvidia robotics stack, opening access for worldwide humanoid developers and companies to use the platforms, acceleration libraries and AI models best suited for their needs.”

Nvidia NIM

Central to the whole process is the Nvidia NIM microservices, which act as a central point for the transfer of the data. Nvidia NIM makes it easier for AI supporters’ developers, and AI builders to go from experimenting to launching AI apps.  

By giving users pre-optimised models and industry-standard APIs for making powerful AI agents, co-pilots, chatbots, and helpers. NIM is designed to make it easy for the latest AI base models to be inferred on Nvidia GPUs from the cloud or server to a PC. It is built on strong foundations like the TensorRT, TensorRT-LLM, and PyTorch inference engines.

By combining the Nvidia tech systems with the Apple Vision Pro, the way has been opened up to positively impact machine development and accelerate the introduction of realistic robotic movements. We expect that work in this field will now become a major area of development and growth.  Keep checking back to see all of the latest developments.

 

Unity Developers are ready to help you drive your AI or XR project forward. Contact us and see how we can accelerate your project.