"Wow, Alicia Keys!" my friend Jessica exclaimed.

Since unboxing my Apple Vision Pro on launch day this past February, I've been eager to let various people try it out—friends, family, neighbors, tech enthusiasts, and the less tech-savvy alike. I aimed to observe their reactions without letting my own enthusiasm overshadow the experience.

I’m Anthony Maës, and I have spent nearly 15 years in the Augmented Reality (AR) field, most recently as a lead engineer for Pokémon Go. I have always viewed the launch of Apple's inaugural AR headset as a pivotal moment destined to bring this medium into the mainstream, much like Tesla’s Model 3 did for electric cars.

Now, in 2024, the Apple Vision Pro is a prominent feature in Apple Stores, with staff eager to showcase its capabilities. However, despite a cautiously optimistic response, the long-anticipated breakthrough seems more like a minor ripple rather than a significant wave.

Fast-forward three months, and a sense of pessimism has emerged. The release of new Vision Pro apps has sharply declined since February, speculation about reduced production goals is causing concern among developers, and Tim Cook's acknowledgment of early users during this week's keynote felt lackluster and uninspired.

How did we arrive at this situation?

Apple is well-known for its cautious approach to product innovation, frequently refining ideas that originate outside the company. Products like the iPhone, iPad, and Apple Watch underwent numerous iterations to achieve their current market dominance. The initial versions of Apple’s products are often beautifully crafted and costly but tend to lack essential features—who recalls that the original iPhone had no apps, no 3G, and no copy-and-paste?

In his review of the Vision Pro, Marques Brownlee pointed out that users have previously overlooked early shortcomings if the device was otherwise appealing. Historically, AR, now termed spatial computing, has struggled to move past the novelty phase. So, what is the Vision Pro's standout application? I believe I have identified it.

In comparison to Meta's Quest 3, demonstrating the Vision Pro can be quite cumbersome. The visionOS does not facilitate multiple user accounts on a single device, so everyone except me must enter guest mode and undergo an eye calibration process that takes a few minutes each time. Additionally, the device lacks accommodation for prescription glasses; most users need to remove their glasses or wear contact lenses, or they end up squinting. Apple offers $99 prescription magnetic optical inserts, which feel unnecessary for competing headsets.

On a positive note, it’s straightforward to assist guests in navigating the system. AirPlay allows the Vision Pro's displays to be mirrored on a TV or MacBook without any performance issues. This enables me to guide users on where to look and how to interact with the device (a user experience that is less intuitive than one might expect).

After they’re finally set up, and their initial excitement fades, they discover that most applications resemble iPad-style rectangular windows floating in space. At this point, every application is inferior to its screen-bound equivalent. Despite Apple's marketing efforts to promote productivity applications, much like the iPad, input constraints render the Vision Pro primarily a tool for media consumption.

One of my favorite initial demonstrations is Encounter Dinosaurs, an engaging interactive 3D animation developed in collaboration with Jon Favreau (of Star Wars: The Mandalorian fame) and Unity. The app transports users back to prehistoric times, encouraging them to approach and interact with a life-sized, albeit virtual, T-Rex.

"This is incredible! Just think about all the other experiences they could create with portals like this," my neighbor John remarked.

And herein lies the crux of the issue: producing high-quality immersive 3D content with detailed models, textures, and animations is extremely costly. The experience itself lasts only a few minutes. While limited interactions add some vitality to the animations (like a butterfly landing on your hand), this initial wonder doesn’t encourage repeated engagement.

By its very nature, it remains a novelty, making it challenging to envision a sustainable flow of content that would keep audiences returning. Despite its strengths, this is not the defining application.

As I continued to refine my demonstration script, I stumbled upon another experience that resonated even more: immersive videos.

Apple categorizes two types of 3D videos on the Vision Pro: spatial and immersive. Both types are stereoscopic, with each frame comprising two images viewed from slightly different angles to create an illusion of depth. The main distinction lies in size and quality: spatial videos are smaller and of lower quality but can be easily produced with the Vision Pro or an iPhone 15 Pro. Conversely, immersive videos are crafted for full 180-degree immersion and boast high resolution, appearing more vivid than reality as perceived through the camera.

Currently, immersive videos are scarce. Apple TV has introduced several shows, including adventure and wildlife documentaries, music videos, 3D animations (more dinosaurs!), and sports content. As of now, Apple has less than an hour of total content available, with only 10 minutes of new episodes released since the February launch. Yet, as my mother Blandine put it, "Now I get it."

"I never watch sports, but I would definitely watch more soccer like this," my friend Ben reflected while enjoying a beautifully produced highlight reel of the 2023 MLS Cup. My colleague Emily shared a similar thought: "I'm secretly a huge UFC fan, but as a small Asian woman, I feel too out of place to attend a fight alone. Experiencing it from home would be amazing!"

In the parlance of Silicon Valley, this is a clear signal.

Immersive video is the standout application.

By immersing you within the video at crystal-clear 8K+ quality while focusing your gaze ahead, immersive videos dismantle the fourth wall. They offer a far more visceral and intimate experience than an IMAX film, conveying emotions that a flat screen could never replicate.

This is why I chose to invest in this domain. I’ve seen immersive video captivate everyone who has tried my Vision Pro, sparking a desire for the device—if only Apple could address the content shortage.

Why is there so little content available?

Immersive videos are not a novel concept. VR veterans refer to them as VR180 videos. Thousands of such videos already exist on platforms like YouTube and Vimeo or on dedicated services like DeoVR. However, the quality of those offerings typically doesn’t match Apple’s standards. Their resolutions were designed for the smaller displays of Meta's Quest series, and their production values are significantly lower, encompassing everything from capturing to editing. Furthermore, many of these platforms are not accessible via the App Store, forcing users to navigate complex WebXR settings in Safari or rely on unreliable third-party apps.

The Vision Pro sets a new standard but creates a chicken-and-egg scenario: consumers need a substantial volume of high-quality immersive videos released regularly to justify the hefty price tag. Meanwhile, content creators require a large enough audience to see a return on their investment.

Even for established studios, producing immersive videos remains a trial-and-error process. Apple has not disclosed any behind-the-scenes technical information about their production techniques. Their unique video encoding is closely guarded, and details about Apple's custom camera hardware are speculative. Canon, which sells a popular $5,000 VR camera kit designed for Meta Quest content, has openly stated that no current camera on the market meets the specifications required for the Vision Pro.

Compounding the issue, unlike Meta, Apple is not financing third-party initiatives to jumpstart content for their platform, leaving developers and filmmakers to fund their own experiments. At most, Apple may commission content through its TV or Music divisions and occasionally acquire startups in the space. In venture capital circles, VR has become a taboo term, with only cautious but commendable efforts supporting new entrepreneurs.

Consequently, it’s not surprising that, despite the tremendous potential, only a handful of non-Apple productions have launched on the Vision Pro.

Indie New Zealand filmmaker James Hustler recently released Explore POV, an app designed to showcase his stunning videos. However, each video necessitates a download in advance, often amounting to one gigabyte per minute of footage.

Veteran VR videographer Mike Swanson developed an open-source player capable of reading local immersive video files because visionOS didn’t come with one.

Clearly, this community of immersive video trailblazers is eagerly awaiting this year’s WWDC (Apple's developer conference) for potential announcements regarding new tools and APIs.

As for me, having transitioned to another role in a Vision Pro startup, I'm excited to be part of the early developments in immersive video. The user reactions I've observed are overwhelmingly positive, dispelling any surrounding skepticism.

In upcoming posts, I will delve into the production pipeline, from camera hardware to streaming formats, highlighting the current challenges and peculiarities.

In the meantime, if you're interested in learning more, I highly recommend the following resources:

• Hugh Hou, a VR video creation influencer, whose channel is filled with tutorials and technical insights into VR video production hardware and software.
• Mike Swanson, an expert in VR video engineering, who maintains the aforementioned open-source player and frequently shares his findings.
• r/AppleImmersiveVideo, a subreddit for professionals and enthusiasts discussing the technology and its gradual content release.

And if there’s an Apple Store nearby, I encourage you to experience it for yourself. I’d love to hear your thoughts!