Despite their science fiction premise, augmented reality (AR) glasses have been available for years — from the public failure that was Google Glass, to the recent, and far more modest, Meta Ray-Ban Smart Glasses. Augmenting what you see with contextual information, visual or otherwise, is a fundamental tech industry goal at this point that, thanks to advancements in computer vision and other AI techniques, is totally possible. If you ignore the actual “overlaying visuals over the world around you” part.
Besides the philosophical problems of altering what your eyes naturally see, it’s very difficult to do well on a technical level. Despite that, AR glasses remain the long-term goal of giant companies like Meta and Apple, and current products like the Quest 3 or Vision Pro are attempts to realize some of the dream of AR with mixed reality experiences (virtual objects overlaid on a video feed of the world around you) while we wait for hardware development to catch up with ambition.
One month with Apple Vision Pro: I get it now. I’m obsessed
Despite my initial skepticism of Spacial Computing, and the hurdles to get eye lenses, Apple Vision Pro is wildly impressive.
Brilliant Labs, a small AR startup started in 2019, has opted to skip the wait and sell a deliberately open-source, AI-powered version of AR glasses that is possible today. They’re called the Frame, are available to pre-order for $349, and might be a first taste of the AR future people have long imagined — or at the very least, a promising first sketch.
From a hacker’s dream to a real product
From a hacker’s dream to a real product Frame started from modest beginnings, but backed by admittedly high-minded beliefs. “I’ve always believed that AI is the ultimate tool to amplify our latent human powers,” Brilliant Labs CEO and co-founder Bobak Tavangar tells Pocket-lint. Frame is the more consumer-friendly version of that idea, but it all really started in 2023 with Monocle, a single AR lens designed to be attached to your existing glasses and endlessly tinkered with. “We saw lots of companies pursuing glasses or goggles, but we didn’t know a thing about glasses or prescriptions,” Tavangar explains. Monocle was a less risky way of figuring out some of those unknowns, and seeing what developers interested in augmented reality might build if they had an open-source AR tool to play with.
“I’ve always believed that AI is the ultimate tool to amplify our latent human powers.”
One notable (and goofy) example was “rizzGPT,” an application designed to be used during dates or job interviews to generate responses with GPT-4 and project them on the Monocle. As TechCrunch reported in 2023, Brilliant Labs was working on something similar internally called arGPT that was “able to receive sound natively through Monocle’s mic and other sensors, enabling low-latency dialogue with GPT-4.”
The growing popularity of generative AI, and dedicated hardware for AI experiences, arrived at a perfect time for Frame’s launch. It’s the more refined version of Monocle in terms of features, and it actually looks like the kind of thing normal people would wear, with black, gray, and clear color options and even support for prescription lenses. In fact, if you looked at the Frame quickly, you wouldn’t necessarily even know they weren’t traditional glasses. It’s really the details that give them away, like the round batteries at the ends of the temples, the small 720p camera on the bridge, and the flat tops of the otherwise round lenses that hide the tech that creates the Frame’s augmented reality effect.
What the Frame does
The AI abilities Brilliant Labs is promising for the Frame so far include access to Perplexity for questions and basic web searches, the ability to translate text that’s in front of you with OpenAI’s Whisper model, identify objects you hold in front of it, and generate images, all projected on the right Frame lens just below eye level.
Brilliant Labs is focused on visuals, but specifically not attempting to create the illusion of digital information existing in the world around you. Instead, like the Focals from Google-acquired company North, Frame displays two-dimensional images and text.
Brilliant Labs is focused on visuals, but specifically not attempting to create the illusion of digital information existing in the world around you. Instead, like the Focals from Google-acquired company North, Frame displays two-dimensional images and text. “We wanted to tap into the idea that most human beings are visual learners,” Tavangar explains. And as he later noted, most people carry wireless earbuds they can connect to if they want to hear an audio response from Brilliant Labs’ assistant Noa.
The hope
The real hope, though, is that developers leverage the open-source nature of the Frame to make their own applications and experiences. Tavangar stressed that Brilliant Labs has no desire to run a platform in the way Apple or Google do, but would like to feature apps and other ideas developed by the open-source community in some kind of storefront to make them easier to access. Moving away from the traditional app store model is a common refrain from other AI hardware companies, too. Humane, creators of the AI Pin, are doing away with the notion of apps entirely and turning AI software into a subscription that’s improved over time, and intelligently calls up different models and experiences depending on what you need.
Brilliant Labs / Pocket-lint
Making AR now
Hang out around enough tech trade shows or read through enough press releases for AR products, and you’ll eventually hear about waveguide optics, the preferred method for realizing augmented reality in glasses small enough to wear. Techniques vary, but in general, waveguide displays shoot images from a small, bright display through a textured, layered lens that then directs the images towards your eyes.
The Frame notably doesn’t use holographic waveguides, instead opting for a combination of a microOLED display shot into a geometric prism that then bounces the image off the Frame’s lenses and into your eye. The reason is primarily price, and the difficulty of making waveguides at scale. According to Tavangar, “there’s not one waveguide design house that can do that at high yield.” So Brilliant Labs is opting out.
Trade-offs
The company performed a similar calculus with speakers on the Frame. There are no onboard speakers because they were deemed too power-hungry for the Frame’s 210mAh battery. Similarly, the Frame doesn’t operate on its own (other than as prescription glasses), requiring a companion app to connect to the internet and access Brilliant Labs’ Noa and associated AI models. Processing AI requests is just too demanding from both a financial and processing perspective. “We will always have a free tier,” Tavangar says, but Brilliant Labs will cap the number of requests you can make once it learns how often owners use the Frame’s skills. It also plans on offering a premium subscription for other features.
What is generative AI and what can it do?
The world is abuzz about AI technology, but what exactly is generative AI? And how can you use it? We explain.
All of these decisions are the reality of building an AI wearable, and they’re only complicated by the addition of AR. But they’re the kinds of trade-offs you consider to be small enough to wear on your face. According to Tavangar, “there are advantages to being eye-level, at the level you see your life.” Computer vision might work better when it’s clear where you’re looking, for one. But that doesn’t mean he isn’t a fan of other AI gadgets, like the Rabbit R1 or the Humane AI Pin. “We think there’s room for multiple devices,” Tavangar says.
Brilliant Labs Frame
You can preorder Frames today.
As Brilliant Labs gets ready to ship the first Frames in April, that really will be the test.
