What is 3D Gaussian Splatting?

Welcome to another edition of VP Land.

Two things are all over my feed right now:

• 3D Gaussian Splatting vs NeRF

• DALL-E 3 vs Midjourney

So let’s jump into both (and more)!

MAIN STAGE

3D Gaussian Splatting: A New Way to Capture Photorealistic Scenes

Watch out NeRF, there’s a new way to scan and digitize the world: 3D Gaussian Splatting.

As a refresher, NeRFs (neural radiance field) are a way to generate photorealistic 3D scenes from a partial set of 2D images.

Corridor Digital has a really great video demoing NeRFs and tools have been adapting it for use in virtual production, such as Volinga's integration with disguise, which we covered previously.

What’s 3D Gaussian Splatting?

Traditional computer graphics rasterize images by drawing lots and lots of triangles. Now, replace the triangle with a Gaussian.

A Gaussian has data such as

• Position: XYZ data

• Covariance: how it’s scaled in a 3x3 matrix

• Color: RGB data

• Alpha: Transparency

Now combine millions of Gaussians with all their data, generated from a sample of images. That’s 3D Gaussian Splatting.

This simple explanation comes from a post from Dylan Ebert on Hugging Face, which also goes into detail about how to make one.

He also created a great 2-minute explanation and demo:

What are the benefits?

• A lot faster than NeRFs. In this demo from Bilawal Sidhu, he was able to fly around a scene at about 100fps. The same scene as a NeRF would take about 10 seconds per frame.

• You can still get photorealistic effects, such as surfaces changing color depending on the angle viewed at. According to Sidhu, “It doesn't use a neural network, but the training loop is similar to deep learning.”

• Direct Editing: Select, move, delete, and relight the scene.

This LinkedIn post from Tristan Cezair has an extensive comparison of 3DGS vs NeRF.

One thing - it can be resource-intensive.

• High VRAM usage (4GB to view, 12GB to train)

• Large scene size (more than 1GB)

Keep an eye out for easier workflows and existing (or new) tools to adapt this model. Jonathan Stephens has a tutorial on how to create a 3D Gaussian Splat in Unity.

Random: Is it a coincidence that all these 3D rendering models have names that sound like they came from the 90s? NeRF and Nerf. Splatting and Nickelodeon. Gaussian Splat sounds like a Double Dare or Guts challenge.

SPONSOR MESSAGE

MASV is a faster, easier way to transfer large files to anyone, anywhere in the world—with just an internet connection.

Trusted by media and entertainment organizations across the globe, MASV plays an essential role in thousands of video workflows, giving production professionals absolute confidence in media delivery.

Unlike other file transfer services, MASV has no size limits and always delivers blazing-fast transfer speeds.

Try MASV for free

SECOND UNIT

DALL-E 3 is coming. And it’ll be inside ChatGPT.

Sam Altman dropped the news that DALL-E 3 is coming in the next few weeks to ChatGPT Plus and enterprise users.

Altman’s tweet had a comic that he says all came from one prompt.

What really stood out was the prompt had some text that should be in the image…and the output rendered the text! Getting text to display has always been a huge challenge when working in Midjourney.

For a side-by-side comparison of DALL-E 3 & Midjourney with the same prompts, check out this thread:

Besides being (what looks like) a much-improved model, one of the biggest improvements will be how you use it: right inside ChatGPT.

In the same conversation, you can generate images and text outputs. There’s a preview of what this looks like on the DALL-E 3 landing page.

But don’t count Midjourney out just yet. According to Nick St. Pierre, “v6 is apparently going to bring a HUGE leap in natural language understanding.”

VIDEO VILLAGE

AKS

🌐 Around the Web

🖼️ LimeWire, the infamous file-sharing tool for pirated materials, rebrands as a creative studio and acquires BlueWillow, a Midjourney rival

🖥️ Amazon releases EC2 N2 Pro Mac instances - Now you can run Apple M2 Pro Mac Mini virtual machines

🎨 Blackmagic debuts DaVinci Resolve 18.6

🏆 SIGGRAPH 2023 awards: recognizing excellence in computer graphics and interactive techniques

👩🏻‍🏫 Master's degree in virtual production now offered at NYU Tish School of the Arts

🌐 Schedule posts and track your social media with Metricool*

* Sponsored Link

CALL SHEET

📆 Upcoming Events

September 
Studio Arts - Technology for Entertainment Creatives
Unreal Engine and virtual production online courses

October 3
XR, Virtual Production & Beyond
St Martins Courtyard, Coney St, York
Engage with experts and industry leaders on the latest trends in film and TV production, games, and software.

October 10 to 11 (other dates and cities available)
Sony’s Professional Display Solutions Roadshow
Sony Corporation, New York City
Meet Sony’s sales and engineering teams to experience hands-on demonstrations of unrivaled display and AV technologies and solutions

October 23 to 27 (other dates available)
Planar Studios Virtual Production Workshop
Planar Headquarters, Hillsboro, OR
Learn about VPXR technologies from industry experts.

October 24 to 26
10th Edition Augmented Enterprise Summit
Houston, TX
World's biggest companies leveraging XR/Metaverse today and preparing for the future of work

—

Istituto Marangoni Visual Design Courses

Visual Design
Multimedia design for integrated communication
Milano - Sept 2023 to Jul 2024
London - Oct 2023 to Jun 2024

Digital Design for Immersive Experiences
Bridging physical and digital worlds and leveraging the power of immersive installations
Milano - Sep 2023 to Dec 2024

Have a film-related event to share? Just reply back to this email and let us know the details.

ABBY SINGER

Thanks for reading VP Land!

Have a link to share or a story idea? Send it here.

Interested in reaching media industry professionals? Advertise with us.