“AR is going to change the way we use technology forever.” For the past five years, Tim Cook has used every possible opportunity to tell the world how incredibly excited he is about AR, and how AR glasses could be the next iPhone. “I’m AR fan, No. 1,” Cook recently told an interviewer. “I think it’s that big.”
By all accounts, Cook has put his money where his mouth is. The company reportedly has a team of 1,000 engineers working on AR glasses, with plans to release a still-unannounced mixed-reality device as early as this year. Apple was also first to introduce a dedicated AR developer framework for mobile apps in 2017, leading Cook to proclaim that “AR is going to change everything.”
Cook’s enthusiasm, and the company’s massive investments into its own AR hardware, stand in stark contrast to a notable lack of commitment on another front: web-based AR. While ostensibly part of an industrywide effort to standardize AR for modern browsers, Apple has yet to add support for web-based AR to the iPhone’s Safari browser — an omission that has severely hampered adoption of augmented reality, according to industry insiders.
“Apple has been a drag on innovation in WebAR,” said immersive computing specialist Christopher Lepkowski.
An Apple spokesperson declined to comment when contacted for this story.
The promise and perils of mobile AR
Lepkowski is a technical director and partner at Pretty Big Monster, a Los Angeles-based marketing agency that has produced AR experiences for brands like Netflix, Warner Bros. Discovery and American Airlines. With consumer AR glasses still years away, those types of AR marketing campaigns nowadays all focus on phones. “There is no more ubiquitous device in the world to reach consumers than the mobile phone,” Lepkowski said.
In the early days of phone-based AR, some brands experimented with dedicated apps to distribute their experiences. However, few if any consumers were willing to download yet another app just to be able to to see a movie character come alive in their living room.
“Every time you make someone click, you potentially lose 50% of your audience,” said Pretty Big Monster managing partner Jason Steinberg. The drop-off is exponentially higher when you force your audience to download and install an app. “For 95% of our clients, that’s a nonstarter,” Steinberg said.
The good news is that AR can work outside of native apps, thanks to the WebXR standard developed by a broad coalition of browser- and device-makers including Google, Meta, Samsung, Mozilla and Magic Leap. WebXR makes it possible to run AR experiences in a browser, allowing people to launch immersive experiences directly from a website.
Apple has been a drag on innovation in WebAR.
Google’s Chrome browser for Android began supporting WebXR in beta in 2018; Samsung and Opera added support to their respective mobile browsers in 2020. Apple, on the other hand, has yet to add WebXR support to Safari on iOS.
At the same time, there is no avenue for other browser-makers to bring WebXR to the iPhone, as Apple forces them to use Safari’s WebKit rendering engine to build iOS versions of their browsers. Under the hood, the iOS version of Chrome is using Safari code — which means that Chrome on iOS isn’t able to support WebXR. This arrangement means WebXR is effectively out of reach on the iPhone.
Consequently, that means that in the U.S., it’s not an option for the majority of smartphone users. “They have blocked off a huge portion of our consumers from the most successful way to view AR experiences,” Steinberg said. “It’s a shame that all these great experiences could be two clicks away.”
As a result of Apple’s lack of commitment, the web is playing a relatively minor role in AR these days. In a recent survey, only 15% of respondents said they had used a web-based AR experience, while more than 50% had used AR in an app like Snapchat or Instagram.
“We would have seen a lot more everyday usage of AR on the web [if it wasn’t] for this reluctance of theirs to step in, to support at all,” Lepkowski said.
Workarounds can be costly and inadequate
Without official support from Apple, a few companies have been filling the void with workarounds to run AR on the iPhone’s Safari browser.
The most popular solution comes from 8th Wall, an AR startup that was recently acquired by Niantic. 8th Wall’s solution is essentially a full-blown AR engine that enables key features like SLAM, a technique for understanding and tracking 3D real-world environments, without making use of Apple’s ARKit. “They do a great job,” acknowledges Lepkowski. “It’s a great product.”
However, by effectively piggybacking on other browser APIs, 8th Wall’s technology isn’t able to take advantage of the deep hardware integration that Apple is making available to apps via ARKit. “8th Wall is amazing for what it does,” said Georgia Tech professor and WebXR pioneer Blair MacIntyre. But without access to the sensor data that ARKit can tap into, a browser-based workaround can never work as well as app-based AR, he cautioned. “And it’s incredibly taxing on the battery and power,” MacIntyre said.
Using 8th Wall’s browser-based AR engine also comes with recurring monthly licensing costs. That may not be a huge deal for brands looking to launch massive marketing campaigns, but it’s enough to give some companies a pause, said Steinberg. “That is very problematic for many of our clients,” he said. “They’re just not prepared to pay an ongoing fee.”
And while big companies may be able to justify paying a fee to run their AR experience on iPhones, that’s simply not an option for many educators, artists and other independent developers looking to experiment with mobile AR.
Why did we build this entire open web if we weren’t going to use it?
For many of the same developers, building full-fledged AR apps for the iPhone is equally not an option. During a recent school year, one of MacIntyre’s students at Georgia Tech was looking to address the growing issue of sexually transmitted diseases among middle schoolers in a way that the generation understands best: with an AR game designed to teach them about safe sex. “It was a cute, little fun game,” said MacIntyre. Distributing it, however, was challenging. “There’s zero chance that [a game like this] could ever make it into any app store,” MacIntyre said.
Even for less-controversial AR content that wouldn’t get flagged by moderators, building native apps is often too high of a barrier. “In order to ship something in an app store, it has to be polished and substantial,” MacIntyre said. “Doing that is time-consuming, [and requires] lots of money.”
The web, on the other hand, has a long history of experimentation, of people trying out ideas in public. Everyone’s favorite brain-teaser Wordle, for instance, started as a personal pet project accessed best by clicking a link through a search engine. Many believe the same could be true for web-based AR, if only Apple were to support it on iPhones.
Or as Lepkowski put it: “Why did we build this entire open web if we weren’t going to use it?”
A seat at the table, but no word on its plans
Why Apple hasn’t implemented WebXR is a mystery to many industry insiders. The company has for years been part of the World Wide Web Consortium’s Immersive Web Group, which has been hashing out the WebXR standard, and Apple’s head of WebKit Engineering Maciej Stachowiak has gone on the record saying the company is “enthusiastic about WebXR.” In September 2020, an Apple employee who has been taking part in the Immersive Web Group even revealed that WebXR code had found its way into WebKit.
Those statements were celebrated by developers, and the following silence on the subject has led to increasingly desperate pleas on Apple’s forums. “Everyone and their mother is waiting eagerly for this,” wrote one developer last year. “Very interested in the progress of this issue as clients ‘can’t believe’ that WebXR is not yet supported on iOS,” added another. “Please try to complete this … before 2050,” wrote one commenter, while another on a different thread concluded: “Safari is the new Internet Explorer.”
MacIntyre is among those who didn’t think it would take Apple this long. In 2017, he was working as a principal research scientist of Mozilla’s Mixed Reality team, looking for ways to experiment with web-based AR on the iPhone. Absent official support from Apple, his group ended up building a WebXR Viewer iOS app capable of rendering web-based AR experiences. The idea at the time was to sunset the project as soon as Apple added WebXR support to iOS. Five years later, the WebXR Viewer app remains available for download on the App Store.
Some have speculated that Apple has a general animosity to the open web, and that manifests in its unwillingness to support anything that could help consumers bypass the company’s lucrative App Store. MacIntyre doesn’t buy into these theories. Instead, he pointed to other web standards, including WebGL and WebRTC. Apple was slow to adopt both, only to one day unceremoniously flip the switch.
“It wouldn’t surprise me if they have [WebXR] working [internally],” he mused. “But if they’re working on a [headset] as everyone assumes they are, then they’re probably waiting for that.”
As Apple gets closer to releasing its own headset, there are signs it is also warming to WebXR. Recent beta versions of iOS have included code snippets hinting at WebXR support, and company representatives began more actively participating in the Immersive Web Group. However, with Apple’s roadmap closely guarded, many industry insiders are still unsure what Apple’s ultimate goals are.
If they’re working on a [headset] as everyone assumes they are, then they’re probably waiting for that.
Case in point: The company recently proposed a new HTML element to display 3D content in browsers. Apple representatives used a meeting of the W3C Immersive Web Group in San Francisco last month to push for this new element to become part of the WebXR spec, and publicly available meeting notes indicate the company regards it as a way to better display 3D objects embedded in 2D websites when viewed with AR glasses.
At the meeting, representatives from other companies seemed to be at a loss about why Apple was focusing on this relatively narrow use case, as opposed to just implementing the entire WebXR spec. “People [are] frustrated by gaps of mobile AR today: Closing those is a priority,” said Microsoft HoloLens principal program manager Alex Turner, according to the meeting minutes.
Still, Apple actively participating in these discussions is a change for the company. A W3C Immersive Web Group member, who spoke to Protocol on condition of anonymity, said the company had largely remained on the sidelines in the past, with representatives not participating in discussions or giving any indications for Apple’s plans to support WebXR. That person credited Apple’s culture of secrecy for this. “It’s the way they do business,” the source said.
Apple’s support would make web-based AR real
While AR insiders have been aware of the WebXR issue for some time, there are some signs Apple could face regulatory pressure over its restrictions on third-party developers. The company has only been able to effectively bar WebXR from the iPhone because of its tight control of the phone’s browser engine.
That type of lockdown, a hallmark of Apple’s closed ecosystem, has come under increasing scrutiny from regulators both in the U.S. and overseas. In a recent draft of the Digital Markets Act, European regulators specifically called out restricting browser engine choices as anticompetitive. “When gatekeepers operate and impose browser engines, they are in a position to determine the functionality and standards that will apply not only to their own web browsers, but also to competing web browsers and, in turn, to web software applications,” the draft reads, according to The Register.
Whether by choice or under pressure, if Apple does change its heart on supporting WebXR, industry insiders predict a massive shift for the entire industry, with a new wave of innovation even before consumer-grade AR glasses are market-ready. “It would be a new dawn for AR,” said Steinberg. “You’d see a lot of investment going into this area.”
“It would be a big deal,” agreed MacIntyre. Web-based AR is simply not getting enough traction without support on iOS devices, he argued, which would make Apple stepping up to the table a game-changer.
“It just makes it real,” MacIntyre said.
