Apple unveils visionOS 2 at WWDC 2024 bringing new spatial computing features to Vision Pro

In Virtual Reality and Mixed Reality News

June 10, 2024 – At its WWDC 2024 event today, Apple has announced a range of new capabilities for Apple Vision Pro, just months after the device’s U.S. release. With the launch of visionOS 2 an integral part of the announcement, Vision Pro users can now expect a range of new features on the headset, including a new way to turn regular photos into spatial photos, new gestures for navigating visionOS, and more ways for developers to take advantage of the device’s spatial computing capabilities.

What New Features Does visionOS 2 Bring to Apple Vision Pro?

visionOS 2 marks a significant update to Vision Pro and introduces new spatial computing experiences. The main aspects of the update include:

• Machine learning that can turn regular 2D photos into spatial photos;
• New hand gestures for navigation and enhanced productivity tools;
• New APIs and frameworks to make it easier for developers to create more volumetric apps and games for Vision Pro;
• Additional features for Safari, Apple TV, systemwide live captions, and more.

“Apple Vision Pro delivers revolutionary experiences for users and developers, and we’re excited to advance spatial computing even further with visionOS 2 just months after its initial release,” said Mike Rockwell, Apple’s vice president of the Vision Products Group. “From navigating visionOS more seamlessly to pushing the boundaries of how users connect with memories, and enhancements to key Vision Pro apps, we can’t wait for users to experience the range of new capabilities with visionOS 2. And with a robust set of tools that enable developers to create richer spatial apps, visionOS 2 makes Vision Pro even more powerful.”

How Does visionOS 2 Enhance Spatial Photo Creation?

With visionOS 2, users can create spatial photos from their existing image libraries using advanced machine learning, transforming 2D images into spatial photos on Apple Vision Pro. These photos can be shared to view on Vision Pro, or shared from the Photos app using SharePlay – Apple’s feature that allows for the sharing of movies, TV, music, and more with friends over FaceTime calls.

What Platforms Support Spatial Videos?

Currently, spatial videos can only be captured using the Vision Pro itself, or using an iPhone 15 Pro. However, according to Apple, later this year Canon will introduce a new “spatial lens” for its EOS R7 digital camera that will allow for the capture of spatial videos, even in challenging lighting conditions. Furthermore, an update to Final Cut Pro will allow creators to edit spatial videos on Mac, adding immersive titles and effects. Plus, with the Vimeo app designed for Vision Pro, users will be able to upload and share spatial videos for others to discover and enjoy.

What Are the New Hand Gestures and Productivity Tools in visionOS 2?

visionOS 2 enhances navigation on Apple Vision Pro with new hand gestures for accessing Home View and Control Center, as well as for checking time, battery levels and adjusting the volume. Apple also stated that the Mac Virtual Display functionality will soon feature a higher resolution and larger size to create an ultra-wide display that is equivalent to two 4K monitors side by side.

visionOS 2 also adds mouse support for additional workflow options, and Vision Pro will now reveal the user’s physical Magic Keyboard, even when the user is fully immersed in an Environment or app.

Other productivity features of visionOS 2 allow users to personalize their Home View by rearranging apps and placing them wherever they want, including compatible iPhone and iPad apps. Plus, visionOS 2’s Travel Mode adds support for trains, allowing travelers to access apps and environments on the go.

How Are Developers and Creators Benefiting from visionOS 2?

With more than 2,000 spatial apps designed for Apple Vision Pro, and more than 1.5 million compatible iOS and iPadOS apps available on the device, visionOS 2 introduces new APIs and frameworks that enable developers to create more volumetric and shareable app experiences.

HealthKit is now also available on Vision Pro, providing developers with new ways to create health and fitness experiences. Other new tools like TabletopKit will enable developers to build shared and collaborative app experiences that are centered around a table, such as board games.

In addition to developer tools, Apple also stated that later this year, Blackmagic Design will launch the first-ever commercially available camera system, and an update to its post-production software DaVinci Resolve Studio, to support Apple Immersive Video.

Apple Immersive Video is the company’s storytelling format that leverages 3D 8K video with a 180-degree field of view and Spatial Audio. Today’s announcement will mean professional filmmakers will soon have access to a camera system specifically tailored to Immersive Video. 

Additional features in visionOS 2

• Safari – While in Apple’s Safari browser on Vision Pro, users can watch videos in an Environment, including on popular sites such as YouTube, Netflix, and Amazon. While scrolling through webpages, users can tap on panoramic photos and immerse themselves as the image wraps around them. Siri can also read and speak the content of a webpage while users multitask.
• Apple TV – visionOS 2 brings support for multiview to Apple Vision Pro’s Apple TV app. Later this year, users will be able to watch up to five simultaneous streams so they can keep track of all their favorite sports and teams.
• Mindfulness app – To help users achieve calm and focus, the Mindfulness app on Vision Pro will include a new capability called ‘Follow Your Breathing’ that presents dynamic visual animations and sounds that respond to the user’s breathing patterns.
• Live Captions – System-wide live captions will provide assistive features to users to help follow along with spoken dialogue in live conversations and in audio from apps.
• AirPlay Viewing – Users can view content on their Vision Pro with AirPlay from their iPhone, iPad, or Mac.

When Is visionOS 2 Available?

The developer preview of visionOS 2 is available today for Apple Developer Program members. Apple noted that visionOS 2 will be coming this fall. For more information, please visit the company’s website.

Image / video credit: Apple