META QUEST 3 AND APPLE VISION PRO SPARK SURGE IN VR/AR HEADSETS

By CHRIS McGOWAN

While sales of VR and AR headsets were gloomy in 2023, this year has seen a solid recovery, with growth in standalone headsets powered by Meta Quest 3 and excitement stoked by the launch of the flashy Apple Vision Pro in February.

Global shipments for augmented reality and virtual reality (AR/VR) headsets declined 23.5% in 2023, but 2024 shipments “are forecast to surge 44.2% to 9.7 million units,” according to the International Data Corporation (IDC)’s “Worldwide Quarterly Augmented and Virtual Reality Headset Tracker.” VR headsets are forecast to reach 24.7 million units by the end of 2028 with a five-year CAGR of 29.2%, and AR headsets will grow to 10.9 million in 2028, representing an 87.1% CAGR over the same period, according to IDC. In addition, Meta, which introduced Quest 3 in October 2023, thoroughly dominates the VR headset market, with an 86% market share in 2023, according to Zreality.com. Its distant rivals include Sony PlayStation VR2, ByteDance’s Pico, Valve Index and the Apple Vision Pro. Oculus VR’s Beat Saber, officially launched in 2019, is most likely the biggest VR title of all time, having earned $255 million by October 2022, according to the Wall Street Journal. VR and AR have shown their many uses – from home entertainment to LBE (location-based entertainment) to enterprise applications, to usage of AR outdoors or at big public events. Now the Apple Vision Pro is another reason to get into AR/MR/VR.

“Overall, I’m optimistic about the future of immersive and HMDs in general,” says Tuong Huy Nguyen, Director Analyst at research firm Gartner Inc. “The future success of a specific device or solution will come down to a combination of price and value. Value depends on a variety of factors. Finding the correct balance while evolving the marketing is what vendors, including Apple, are trying to figure out.”

There are many reasons why VR adoption slowed prior to 2024. One is that “more content [volume and variety] is needed,” Nguyen says. In addition, more compatibility is needed. “Currently, instead of an ecosystem of VR, it’s more like a medley of standalone apps. The analogy I like to use here is to imagine what the web would be like if instead of a common interface [web browsers] to access the internet, we had apps. As in, every time you want to share something, the recipient is required to download a different program [or custom-made browser]? Also, certain browsers only work with certain hardware, similar to what we have with console gaming. Without a common delivery mechanism, VR adoption will continue to be hindered,” he adds.

Nguyen continues, “AR growth is still siloed and sporadic. One of the reasons is the same as I mentioned for VR. The majority of AR experiences are apps in the sense that they do a very specific thing involving a specific organization, brand or product and can’t be applied to anything else. There are other challenges like creating standards in order so that more AR content can share a browser. For example, two competing brands can be AR-enabled within a single view; so, a user can pan from one product to another and get relevant overlay information on both.”

Nguyen praises Apple Vision Pro, saying, “Apple’s official entry into the market [immersive and HMD] has validated the market by bringing needed hype and exposure at a time when both were on the decline. On the other hand, the introductory price point makes it nearly impossible for any [regular consumer] to justify adoption.”

WHAT IF?

Marvel Studios and ILM Immersive have teamed to release the immersive experience What If…? – An Immersive Story for its Vision Pro. Pat Conran, Supervisor of VFX and Tech Art/Visual Effects Supervisor at ILM, comments, “Our teams [at Marvel and ILM] are taking full advantage of all the spatial computing tools that Apple has created for its Vision Pro. Some parts of the experience take place within your own room, meaning that you’ll see your real-world environment and digital elements at the same time, and at a very high fidelity. Characters materialize or step out of portals, and at other times, you will transition through various portals from your room into fully VR environments.”

Conran continues, “We were really lucky to have a great foundation with Marvel Studios’ series What If…? which has two successful seasons of traditional linear episodes. Working with our director Dave Bushore and the series’ showrunners, we created a story in this animated universe where you get to interact and participate in the progression of the story. This experience is unique in that it places you in the same space as these amazing characters, and you’re a part of the story.”

The team wanted the shift from AR to VR and back out again to be a key part of the experience. Conran explains, “At certain points, you will see stereo vignettes playing on large multiversal ‘shards’ in your room, with the impression that you are looking through the shard and into the scene that is playing out. These then transition to actual environments that, as the shard you’ve been watching from glides forward, brings you into a fully immersive environment in full VR.”

Regarding the platforms, he says, “Each VR/AR/MR has its strengths and weaknesses. It’s great to be able to combine them and transition between mixed reality and full VR so you keep grounded in your own reality, but that has to make sense within the narrative. Successful VR has to be as compelling a story as we would expect from linear, more traditional content, but should also be one that can be told uniquely on that platform.” Conran calls What If…? “a new type of blend in narrative and interaction.”

BAOBAB STUDIOS

“We are excited about Apple Vision Pro, but it’s not going to be until V2 or V3 that it will make major consumer headway, due to the high price right now. The potential is there to disrupt the VR/AR industry, but it still needs to figure out the killer apps that make it a must-own,” comments David Kahn, Head of Product and Games at Baobab Studios.

Baobab is a VR animation studio and creator of titles such as the Emmy-winning Baba Yaga. Kahn notes, “The biggest change for animation technology, especially for VR and AR, is the use of 3D game engines like Unity and Unreal to create real-time animation pipelines. Developers need to rethink how to tell a story when you don’t have control over the frame and need to do it on a mobile chipset. Part of our studio focus has been to develop new tools and processes to work in a 360 real-time animated environment.”

In terms of the consumer market, Kahn notes, “VR needs a [greater number] of developers in the space creating unique content, and it still needs to have cheaper entry points for regular use. There is a growing trend to both mixed-reality experiences thanks to Apple entering the space and more social interactive experiences that appeal to a younger generation of users. These users have grown up on Roblox, TikTok and YouTube, so it’s more about quicker gratification and compelling player-driven experiences than the highest fidelity.”

FRAMESTORE VR

“Without a doubt, I think XR has greatly influenced both storytelling and filmmaking,” says Brian Solomon, Creative Technology Director for Framestore, which has worked on immersive and VR experiences such as Warner Bros.’ Fantastic Beasts and Where to Find Them and (with Thinkwell Group) several LBVR experiences for Lionsgate Entertainment World theme park in China.

Solomon comments, “Within the last 10 years, we’ve seen a sharp jump in the amount of creative and engineering talent entering the scene with knowledge of tracking systems and real-time concepts. Everyone from indie creators to major studios has received a big boost from all the advancements and research aimed at advancing spatial technology.

“On the consumer side, it feels like it’s also brought on a new expectation that the stories we produce can exist in multitudes, and that our interactions with our stories and concepts benefit from new ways of inviting the audience to play and explore. With this new expectation comes a ton of creative opportunities where we have a chance to realign and play with creator vs. audience roles. We’re getting more used to this idea of moving participants into the driver’s seat, making them into creative partners in a way,” Solomon continues.

“I fell in love with VR around the first Vive/Oculus wave. I was obsessed with the idea that you could all of a sudden ‘enter a world,’ and with this intermediary digital layer of fiction, people could walk around and you could take them somewhere new, or that their body could matter in how we tell stories and remember them. Powerful stuff. For consumers, I feel like those types of experiences can be fully transformative, and in some ways have the potential to make even real places that aren’t ‘active’ valuable again.”

LBVR

Solomon comments, “For specific venues, I do feel VR makes total sense, and people are always excited about it. Themed entertainment and rides are a wonderful starting place where you have a dedicated audience who’s so ready to have their mind blown and wants to use this technology. Location-based is so powerful for pushing to the bleeding edge and unleashing what you can do with HMDs. That space is still rich with life for audiences, and there’s a mechanism there for commerce. Framestore is working in this space frequently.”

The Location-based entertainment market was estimated at USD 3.5 billion in 2023 and is projected to reach USD 11.8 billion by 2028, at a CAGR of 27.3% from 2023 to 2028, according to a recent report by MarketsandMarkets.

“With the current crop of hardware, we’re seeing the need for much more lifelike fully immersive environments that run with very little memory usage or computational overhead. As these devices are starting to become full-fledged computers, the need to operate with utmost quality while leaving room for all the applications people want to use is paramount,” Solomon says. “Framestore is a leader in this space, having created some of the most efficient yet lifelike environments for many generations of these devices.”

Solomon adds, “For the greater public and indie spectrum, there is, however, very little educational material on how to properly configure and design for these strict requirements while maintaining quality. So, it can feel unattainable to some to convert their products and ideas into a ‘VR-ready’ format. I suppose the greatest trend is that we’re moving toward the ease of creation being most important, so more can take advantage of the tech and bend it to fit their needs.”

SMART GLASSES

AR and VR smart glasses are also generating excitement. The AR and VR smart glasses market will grow from $15.93 billion in 2023 to $18.58 billion in 2024 at a compound annual growth rate (CAGR) of 16.6%, and to $34.62 billion in 2028 at a compound annual growth rate (CAGR) of 16.8%, according to The Business Research Company.

HYPE CYCLE

“The hard part is that tech companies have geared audiences into this hype-cycle mentality, and new devices are only big news for a short time nowadays with a consumer base with short attention spans. It’s getting even harder for consumers lately who are in many ways overwhelmed with technology and its rapid advancement in general,” Solomon remarks. “I don’t think they want to feel like they’re wearing a tech company on their face; they want to experience something that enables them to go beyond that feeling and, like all good tech, let it fade into the background.”

Solomon continues, “For it to catch on publicly, there must be more to do that is so cool that it draws the audience’s attention [as must-see], and the fact it’s in VR or AR is less of the story. You almost don’t even want to make that a big deal because the public at large isn’t as blown away by the novelty of the medium itself anymore. They get it; they just need more to do with the devices.”

PEAKS AND TROUGHS

Nguyen says, “There have been many peaks and troughs in the past decade and there will be many more in the coming 10 years. We can count the peaks as milestones towards mass adoption. We can look at the troughs as lessons learned and moments when the industry can focus on quietly making progress without the distraction of the hype, rather than considering them as setbacks.”