VP Land
Posts
What is Apple Immersive Video?

What is Apple Immersive Video?

🌐 Midjourney Style Reference Tool, SIGGRAPH Talks

February 06, 2024

Together with

Welcome to VP Land 🌐

The Vision Pro is out in the wild. We’ve got the ridiculous memes, from Vision Pro in a Cyber Truck to eating a meal without a phone in sight.

But what does this mean for the future of computing? And more specifically - what does it mean for creating videos and new video formats?

We’ve known about Spatial Video, but Apple dropped a new label about a week ago.

Apple Immersive Video.

What does it mean? Let’s dig in.

STAGE A

2D Video vs Spatial Video vs Apple Immersive Video

We’re still waiting to see what the ‘killer app’ will be for the Vision Pro.

But one of the most marketed use cases (with good reason) is as an entertainment device. Specifically watching movies/videos.

But what type of movies? And what format?

From what I can tell, we’ve got 3, maybe 4 different media types.

2D Video. 3D Environment

This is the viewing experience that’s been getting the most attention - watch a regular video on a giant screen in your living room. Or on a plane. Or on Tattoine.

Video content-wise, nothing is different. It’s a regular video file played back through the Vision Pro.

Notably Netflix and YouTube are not on the Vision Pro (they specifically opted out since by default all iPad apps were ported over).

You can still access YouTube via the Safari browser (but no VR videos, more on that below) and yesterday YouTube announced that a Vision Pro app is on their roadmap.

Spatial Video & 3D Movies

Spatial Video is the 3D video format Apple announced with the first Vision Pro announcement. You can record Spatial Videos with the Vision Pro or iPhone 15 Pro.

The file format is a new 3D version of HEVC, MV-HEVC. We have an extensive breakdown of the format in a past article here.

This is presumably the format that 3D movies are being delivered in. Apple, Disney, and some other studios have made over 150 3D movies available at launch on the Vision Pro with more coming.

As one user pointed out - the Vision Pro is the only way to watch Avatar: The Way of Water in its original 3D, high frame rate version.

Apple Immersive Video

Then there’s Apple Immersive Video. What exactly is that? Good question.

Here’s Apple’s official description:

Apple Immersive Video features 180-degree 3D 8K recordings captured with Spatial Audio

The Vision Pro launched with four Apple Immersive originals available:

Alicia Keys: Rehearsal Room
Adventure (produced by Atlantic Productions)
Wild Life
Prehistoric Planet Immersive

Best guess on AIV - a quality stamp of approval from Apple for productions that meet their 180-degree, 8K delivery with Spatial Audio.

OG 360 VR Video - 4th Format?

And what about the original 360-degree VR video? YouTube supports this format, but as noted above you can’t access it via Safari (assuming this will change when YouTube launches a native app).

But The Verge’s Nilay Patel asked the same question to Apple:

I asked Apple why the huge library of VR video on YouTube doesn’t work at all on the Vision Pro, and the company basically told me it wasn’t good enough to support, saying that “much of this content was created for devices that do not deliver a high-quality spatial experience” and that the company’s efforts were instead focused on “delivering the best spatial media experience possible including spatial photos and videos, Apple Immersive Video, and 3D movies available on Apple TV.”

This also supports the Apple Immersive Video quality stamp theory.

Closing Thoughts

This article was focused on the video aspect of the Vision Pro because, well, this is a video-centric newsletter.

But for thoughts about the device in general and how this offers a glimpse into the future of computing, the two (technically three) best videos I’ve seen are MKBHD’s (this one goes over the tech and features, this one is more about the pros and cons of spatial computing), and Casey Neistat’s video which has him wearing the Vision Pro around NYC but really captures the potential future of what computing can be like.

My brain clicked and forgot I was looking through cameras and screens and just saw reality. This is the future of computing that people have been promising for the last 15 years…This wil be the worse Vision Pro that Apple ever ships. It will get so much better.

Casey Neistat

SPONSOR MESSAGE

Stay up-to-date with AI.

AI won’t replace you, but a person using AI might. That’s why 500,000+ professionals read The Rundown– the free newsletter that keeps you updated on the latest AI news, tools, and tutorials in 5 minutes a day.

SECOND UNIT

AI Updates

🌆 Midjourney has a new Style Reference tool to try to match the vibe and aesthetics of a provided image.

🗂️ AI model platform Hugging Face launched its own open-source AI assistant maker

👤 Synthetic human generator Synthesia launched a new tool to just feed it a document or website and it automatically creates a video with an AI human presenter.

🔐 WME partners with Vermillio to protect actors from deepfakes

CASTING CALL