Snap Spectacles 5 Unveiled with Enhanced AR Capabilities and OpenAI Integration

In Augmented Reality News 

September 18, 2024 – Snap Inc. has this week announced the release of the fifth generation of Spectacles – the company’s new see-through, standalone augmented reality (AR) glasses designed to work with Snap’s Lense technology. The glasses, powered by Snap OS, are designed to enhance how people naturally interact with the world using AR, according to Snap. Spectacles are available now as part of the Spectacles Developer Program.

The new Spectacles integrate with mobile devices through the new Spectacles app, allowing users to control Lenses, mirror their phone screens, use their phone as a gaming controller with Lenses, and enable a spectator mode that allows people without their own Spectacles to follow along.

What are the hardware specifications of Snap’s new Spectacles?

Snap stated that Spectacles are the result of a decade of research and development, and pack some serious AR technology into a glasses form factor that weighs less than half the weight of a typical VR headset at only 226 grams. 

Spectacles are equipped with four cameras for hand tracking, powered by the Snap Spatial Engine. The Optical Engine, built in-house, leverages Snap’s proprietary technology to enable a see-through AR display, and includes Liquid Crystal on Silicon (LCoS) micro-projectors to create sharp images.

Snap also stated that the waveguides used on Spectacles feature billions of nanostructures that enable the projection of images into the user’s field of view without requiring lengthy calibrations or fittings.

The device offers a 46-degree diagonal field of view with a resolution of 37 pixels per degree, equivalent to viewing a 100-inch display from 10 feet away. The glasses also adjust to environmental lighting thanks to their ‘auto tint’ capabilities, providing clear visuals even in direct sunlight.

Finally, Spectacles are powered by Snap’s dual system-on-a-chip architecture that features two Snapdragon processors from Qualcomm. This infrastructure splits the compute workload across the two processors, allowing Spectacles to provide up to 45 minutes of continuous runtime, according to Snap.

Snap OS features

Snap OS, the operating system for Spectacles, includes an intuitive interface designed for natural interactions using hand and voice commands. The Snap Spatial Engine powers realistic AR experiences on Spectacles, and according to Snap is able to understand the world around the user so that Lenses appear realistically in three dimensions.

Lenses are rendered with low motion-to-photon latency of 13 milliseconds, ensuring accurate integration into the physical environment. Snap OS also allows developers to build and share AR experiences with groups of users, enabling collaborative experiences.

Developer support and tools

Snap Inc. is also enhancing its tools for developers through the newly rebuilt Lens Studio 5.0, which offers improved features for creating Lenses. Developers can now build Lenses more efficiently and deploy them to Spectacles directly, eliminating the need for complex compiling processes, according to Snap. The company has also introduced a new ‘Spectacles Interaction Kit,’ which allows developers to design Lenses without having to develop their own interaction system from scratch.

Multimodal AI model integration on Spectacles

Snap Inc. has also announced a partnership with OpenAI, which will allow developers to integrate cloud-hosted multimodal AI models into their Spectacles experiences. This integration is expected to provide developers with new ways to contextualize what users see, say, or hear in AR environments. Spectacles feature a suite of cameras and sensors that power multi-modal AI, contextual understanding, and 6DoF tracking on the device.

Snap Spectacles Availability

Spectacles are available to developers through the Spectacles Developer Program for USD $99 per month, with a one-year commitment. The subscription includes access to Spectacles hardware and developer support from Snap Inc.

What experiences are there on Snap’s new Spectacles?

Snap stated that several companies have already been using Lens Studio and Snap OS to create new Lenses for Spectacles, including: 

• LEGO Group is launching BRICKTACULAR, an interactive AR game controlled entirely by hand and voice. The experience will unlock endless LEGO building possibilities to challenge players and see how fast they can build.
• ILM Immersive, Lucasfilm’s award-winning interactive studio, is developing new experiences that connect users and their friends with the Star Wars Galaxy.
• Snap has also partnered with Niantic to bring some of their AR experiences to Spectacles, including Peridot and Scaniverse.
• Finally, Wabisabi Games has created a new way to play capture the flag on Spectacles.

For more information or to join the Spectacles Developer Program, visit Snap’s website.

Image / video credit: Snap Inc.