Brilliant Labs announces new ‘Frame’ Multimodal AI glasses

In General XR News

February 8, 2024 – Brilliant Labs, a developer of AI glasses, has announced the launch of its ‘Frame’ glasses that come with an integrated multimodal AI assistant. The announcement is followed by support from the company’s latest notable investor and AR pioneer, Niantic CEO John Hanke.

Frame offers hands-free access to next-generation AI, which enables users to amplify their human intelligence all within a lightweight package that comes in a slim form factor similar to that of stylish eyeglasses.

Brilliant Labs stated that Frame introduces a new, alternative way of interacting with the physical world, which is enhanced by an always-on AI assistant called “Noa.” Powered by an integrated multimodal generative AI system that can run multiple AI models at the same time, Noa is capable of conducting real-world visual processing, novel image generation, and real-time speech recognition and translation, according to the company.

Noa – Frame’s Multimodal AI Assistant

Noa uses a “privacy-preserving knowledge graph” to analyze compounded interactions between the user and AI assistant, enabling Noa to not only offer an individually tailored user experience but also take on a personality of its own.

For example, tasks and requests could include summarizing the page of a book into bullet points and then translating the book’s foreign language text into a preferred language; structuring a white boarding session into a shareable document for colleagues; identifying a skin condition to recommend off-the-shelf ointments; and even suggesting a healthy amount of coffee to drink daily, based on the nutritional value or caffeine content in a single cup.

Users can even fully leverage the glasses’ built-in multimodal generative AI capabilities, meaning that  Frame can orchestrate multiple AI systems to work together to complete more advanced tasks. For instance, by combining GPT4 with Frame’s front-facing sensors, Inertial Measurement Unit (IMU), microphone, and knowledge graph, these systems can join forces to visually and acoustically measure emotional sentiment and facial cues, according to Brilliant Labs.

“Frame truly offers a new paradigm in the way humans interface with the digital world, and with each other. Noa, a totally new kind of AI virtual assistant, empowers users to engage more with the world around them, not less. Our hope is that Frame enables people to connect with each other in entirely new ways, and opens up new experiences in everyday life, the workplace, the classroom, and more. This is the unique promise of bringing generative AI in front of the human eye,” said Bobak Tavangar, CEO and Co-founder of Brilliant Labs.

Real-Time Search in Partnership with Perplexity

Brilliant Labs has also announced a partnership with Perplexity to rethink and redefine the traditional notion of a search engine in the age of AI-enhanced glasses. Powered by online Large Language Models (LLMs), Perplexity references the most recent web data (unlike other GPT models).

Its conversational AI-powered answer engine can then respond to spoken queries “near-instantaneously,” and is apparently able to provide information based on even the most recent events, allowing Frame users to access real-time, reliable information from the web on-the-go or hands-free within any setting.

Brilliant Labs used the example of using Frame glasses with Noa’s multimodal capabilities to search for price checks on in-store items by simply eyeballing an article of clothing, which Perplexity will then instantly compare to prices on Amazon.

“The future of human/AI interaction will come to life in innovative wearables and new devices, and I’m so excited to be bringing Perplexity’s real-time answer engine to Brilliant Labs’ Frame,” said Aravind Srinivas, CEO and Founder of Perplexity.

Frame’s Features: Designed to Be Indistinguishable From Eyeglasses

Frame itself is a pair of prescription eyeglasses, which can completely replace a user’s daily worn glasses, but also offers the perks of a baked-in personal assistant, with a lightweight form factor and stylish design that’s indistinguishable from everyday eyewear, according to the company. Frame’s features include:

• Lightweight AI Glasses – Frame weighs only 39 grams, making the device no heavier than a standard pair of eyeglasses but durable and comfortable enough to be worn all-day;
• Vivid Display – Frame’s high resolution vivid display boasts 3,000 nits of brightness;
• Front-Facing Spatial Camera – Frame integrates a low-power machine vision camera that accurately captures and interprets the world around the user;
• Integrated Microphone – Users can control or speak with their AI assistant through voice inputs for a hands-free experience;
• Multimodal AI – Wirelessly connects to a phone through Bluetooth 5 to support simultaneous generative AI inputs from Perplexity, Stability AI, GPT4, and Whisper to offer advanced solutions to real-world problems;
• 6DoF IMU – Centrally located within the nose bridge, an advanced 6-axis IMU (Inertial Measurement Unit) provides Frame with a three acceleration axis and three compass axis, which can be used for menu navigation, step counting, directions and more;
• Custom Prescription Optics – Prescription lenses, in partnership with AddOptics, can be ‘Precision Bonded’ to Frame using a unique optical lens casting process.

Frame will follow the footsteps of its predecessor, Monocle, which was the company’s previous AI-embedded device. Monocle sold out completely of its first orders and as a result, Brilliant Labs stated that it will focus solely on the sales and availability of Frame for both consumers and developers.

Frame will be available for pre-order starting today, and will retail for USD $349. Brilliant Labs stated that shipping will begin in April 2024.

Finally, Brilliant Labs stated that the latest round of funding from Niantic CEO John Hanke puts its total amount of funding commitments to-date at USD $6 million. 

To learn more about Brilliant Labs and its Frame AI glasses, please visit the company’s website.

Image credit: Brilliant Labs