Nvidia announced that its CloudXR technology will bring the power of GeForce RTX graphics to the Apple Vision Pro and other devices.
The tech streams better visuals into the experience you see on the Vision Pro XR headset. It’s targeted at the enterprise metaverse, where the virtual world is designed to be the same as what you experience in something like a factory in the real world.
The tech includes foveated streaming, which allows for the viewing of XR imagery with as much as 85% less pixels than normal.
Nvidia announced that Nvidia CloudXR 6.0 is now natively integrated with Apple’s VisionOS, enabling RTX-accelerated technologies to stream high-fidelity, 4K, low-latency immersive content directly to Apple Vision Pro.
A growing ecosystem of partners is already harnessing CloudXR for visionOS, including leading software providers as well as enterprise and automotive leaders.
Richard Kerris, vice president of media and entertainment at Nvidia, said in a press briefing that the project was in the works for two years.
He said being able to deploy once and run on all XR devices is more convenient for developers. You can see in a demo at Nvidia’s GTC event in San Jose, California, that details on a car can be much better than XR on standalone devices without streaming.
“Spatial computing is becoming more and more a reality for people to use in their day-to-day work,” Kerris said. “They can use it right out of the box.”
He added, “When you see it, you get it. You understand the impact you will see across industries.”
Kerris said the key is for the developer to get better visuals on a variety of XR platforms. The handshake between two devices establishes a secure connection between them. The aim is to secure the connection from one end the other, he said.
The amount of bandwidth you need is based on the experience you’re viewing. For a full VR experience, the downstream bandwidth needs to be about 80 megabits a second, lower than previous solutions in the streaming stack.
At GTC, I saw a demo of the tech, wearing an Apple Vision Pro and staring at a McLaren car on the show floor. As you can see in the image, I saw the foveated streaming tech in action, showing the wind flow over the car. The wind in the yellow and blue colors showed the air flow in a virtual way, overlaid on the actual car. It looked pretty precisely calibrated between the virtual wind and the car itself, and it worked well on wireless, though there was a bit of pixelation when the network was weak.
Creating digital twins of AI factories and healthcare labs. Designing sleek car exteriors in
extended reality (XR) with physically accurate color and lighting. Fully immersing in
high-resolution racing games or other simulation titles.
Kerris said these are just some of the applications that professionals, developers and enthusiasts can tap into through spatial computing — seamlessly blending digital content with the physical world — using the Nvidia CloudXR 6.0 suite of streaming technologies, now natively integrated with Apple Vision Pro.
Nvidia has a similar demo of a Kia car, viewed through the Apple Vision Pro.
Nvidia and Apple showed how visionOS now supports NVIDIA CloudXR with foveated streaming, enabling apps to display high-resolution, low-latency immersive content on Apple Vision Pro. This feature intelligently optimizes rendering resolution based on approximately where the user is looking, while strictly protecting user gaze data.
“Apple Vision Pro is redefining what professionals can do with spatial computing, enabling
teams to visualize, collaborate and work with extraordinary fidelity in entirely new ways,”
said Jeff Norris, senior director of the vision products group at Apple, in a statement. “With Nvidia, we’ve brought together the powerful capabilities of visionOS with CloudXR streaming technology to deliver high-fidelity experiences to accelerate work across industries ranging from automotive design to healthcare, aviation and beyond.”
Demanding XR and spatial computing workloads share three critical requirements:
uncompromised high-fidelity visuals at 4K resolution, real-time response rates and full
immersion. CloudXR for visionOS enables all three.
With this new integration, users can put on Apple Vision Pro and augment their spatial
computing workflow with apps like Immersive for Autodesk VRED, enabled by Innoactive’s
XR streaming solutions, automatically and natively streaming from Nvidia RTX technologies — ranging from NVIDIA RTX PRO workstations to GeForce RTX GPUs — on
PCs or in the cloud.
In XR, Volvo Group designers step inside the design to perfect the windscreen to front transition, shaping a clean and aerodynamic expression before anything is built. Image courtesy of Volvo Group.
The spatial computing capabilities of Apple Vision Pro, together with the ultimate graphics
engine provided by Nvidia RTX technology, deliver high-fidelity, immersive visuals in real
time.
A growing ecosystem of partners is already harnessing CloudXR for visionOS to transform
their workflows. Leading software providers — including Autodesk, Innoactive, Synopsys,
Trifork, X-Plane and iRacing — are using the integration to deliver high-fidelity applications
natively to Apple Vision Pro. This provides enterprises such as Roche, Foxconn and Switch,
and automotive leaders like Kia, BMW Group, Rivian, and Volvo Group with the ability to
visualize massive 3D assets and complex simulations with uncompromised photorealism.
Users can now play the RTX-powered title “X-Plane” on Apple Vision Pro. Video courtesy of Apple and X-Plane.
Performance Meets Immersion
Previously, bringing high-fidelity 3D assets to untethered devices often required developers to degrade, or dramatically simplify, their datasets. CloudXR for visionOS lets
professionals review designs and digital twins in full fidelity, preserving quality while
reducing extraneous work.
Dynamic foveated streaming — a capability enabled by CloudXR 6.0 and visionOS 26.4 —
ensures the highest possible visual fidelity at focus points while maximizing performance
and bandwidth efficiency.
Approximate gaze data is never exposed to the application, strictly protecting user privacy
while delivering the ultralow latency required for comfortable, photorealistic immersion.
Industry Leaders Boost Graphics Workflows
Major partners are already harnessing the integration to pioneer innovation in design,
manufacturing and simulation.
Immersive for Autodesk VRED, coming later this spring, is enabled by Autodesk with
Innoactive, to transform the design review process for automotive giants including BMW
Group, Kia, Rivian and Volvo Group. Designers using Autodesk VRED can now visualize
massive models with RTX-powered ray tracing at a 1:1 scale in Apple Vision Pro.
“By combining Autodesk VRED with Nvidia CloudXR, Apple Vision Pro spatial computing
and Innoactive’s app expertise, we’re enabling immersive, real-time collaboration without
the constraints of traditional high-end setups,” said Thomas Heermann, vice president of
automotive design at Autodesk, in a statement. “Automotive design reviews are high-stakes moments where teams need to iterate quickly and make confident decisions. By making the review process more seamless, companies can spend less time on infrastructure and more time on bringing products to life.”
“Enterprises are moving beyond single-user XR pilots toward collaborative, multi-location
deployments that require performance, control and scalability across locations — and
Innoactive’s enterprise XR streaming solutions deliver workstation-class visualization
streamed directly to spatial devices, including Apple Vision Pro, for product showrooms,
design reviews and digital twins,” said Daniel Seidl, CEO of Innoactive, in a statement.
“Integrating immersive spatial computing into our workflow with NVIDIA CloudXR for
visionOS allows us to evaluate our designs at full size with greater clarity and speed on
Apple Vision Pro,” said Karim Habib, executive vice president and head of Kia Global Design, in a statement.
“We can experience proportions, surfaces, colors and materials together in a shared
real-world environment and collaborate in real time across our global teams. Through these collaborations, we can continually refine our process, maximize efficiency and remain focused on the creative intent behind every decision.”
“At Volvo Group Design, we work digitally first, building physical prototypes only when
essential,” said Mikael Gordh, head of Volvo Group Design at Volvo Group, in a statement. “Spatial computing, Apple Vision Pro and CloudXR let us experience everything users see and touch — years sooner. This tech delivers realism never possible before: premium experiences through seamless collaboration. Shaping transport’s future, beyond vehicles.”
In XR, proportions are experienced rather than guessed as cab side split lines and stance are tuned in a lifelike virtual model from every angle. Image courtesy of Volvo Group.
Pharmaceutical company Roche, in collaboration with Innoactive, is using Autodesk Revit
software, NVIDIA Omniverse libraries and CloudXR for visionOS to simulate layouts for its
biofluid analysis labs with spatial computing before building them in the real world.
Manufacturer Foxconn is using CloudXR for visionOS to visualize factory-floor
walkthroughs so designers can explore and optimize facilities before even stepping foot
inside.
MHP, management and IT consultancy for mobility and manufacturing customers, is using
Synopsys Ansys Discovery 3D product simulation software, Omniverse and Innoactive
Spatial, tapping into CloudXR for visionOS, to simulate real-time aerodynamics airflow.
Data center platform provider Switch, in collaboration with Trifork, is demoing an Omniverse digital twin of its EVO AI Factories. By tapping into CloudXR for visionOS, Switch
enables infrastructure optimization within a fully immersive spatial environment, allowing
operators to monitor and manage complex data center layouts from anywhere.
Untethering Production and Play for Developers
Beyond enterprise use cases, CloudXR for visionOS lets simulation enthusiasts connect
high-fidelity, RTX-powered titles like iRacing and X-Plane to Apple Vision Pro, enabling an
ultimate simulation rig with uncompromised 4K resolution.
User wearing Apple Vision Pro plays “iRacing” powered by NVIDIA RTX. Video courtesy of Apple and iRacing.
CloudXR 6.0 is now available to developers as a native streaming framework for Swift, the
programming language for Apple platforms including visionOS, iOS, iPadOS and more.
This means Swift developers can stream and build high-fidelity, consumer and enterprise
apps right out of the box within Xcode, Apple’s integrated development environment for
developing, testing and distributing apps across Apple platforms. visionOS 26.4 and these
apps will be available to users this spring.