VP Land
Posts
A Busy Week for AI Video Tools

A Busy Week for AI Video Tools

Runway's Character Control, Midjourney Retexturing, Claude Takes Over Your Computer

October 25, 2024

In partnership with

Remember the 30 for 30 doc June 17th, 1994 that documented all the crazy sports related events that happened on that single day?

This week feels like that for AI.

New video control methods, new generative models, new image controls, new voice controls and a whole bunch more.

Let’s break it down in this big, giant roundup of video AI news.

AI’s Crazy Week of Updates

Better Video (and Audio) Control

On my radar, the most exciting update for AI filmmaking is Runway's Act-One. This new tool, which they’re slowly rolling out, lets you record a character performance video of a human and combine it with a still image to animate that character, matching facial expressions and mouth movements. We put out a video (above) breaking it down, highlighting what we know, and raising a few questions.

Kaiber rolled out an exciting UI take for AI - Superstudio. It’s a node-based interface that lets you run multiple models from a single spot, including Runway, Luma, Kling, and Minimax.

And for audio, ElevenLabs expanded their synthetic audio generation tools with a new text to voice design feature. Type out a prompt for what your character should sound like, and ElevenLabs will generate three options.

New Models on the Block

Source: Stability AI

Stable Diffusion released version 3.5. It offers high-quality AI image generation with 8 billion parameters, operating at 1 megapixel resolution on consumer-grade computers.

Genmo's Mochi 1, a new open-source AI video generator, offers free access to advanced video creation capabilities for both personal and commercial use.

Haiper AI's 2.0 model introduces 4K resolution support and a template system.

Image Editing Upgrades

Source: Midjourney

Midjourney released an editor tool - upload any image to use MJ’s powerful editing tools, like selective generating, upscaling, and expanding. Plus, they added a re-texture feature - keep the shape and framing of your image but change the lighting, look, or texture.

Ideogram also launched a similar feature, Canvas. Offering advanced AI image editing capabilities, including precise text rendering and the ability to edit specific regions.

For a more technical route, once you go down the AI rabbit hole, all roads lead to ComfyUI to create the most complex and precise workflow models. However, installation is a bit tricky if you’re not a developer. ComfyUI just announced a Mac and PC desktop installer that will make setup much, much easier (plus, it will come with ComfyUI Manager pre-installed.

And a few more

Source: Anthropic

Anthropic rolled out an early beta of a new tool - Computer Use. Now, you can prompt Claude to control a computer to perform tasks.

OpenAI's new sCM model generates high-quality AI images in just two steps, potentially making the process 50 times faster than current methods, like diffusion.

SPONSOR MESSAGE

Learn AI in 5 minutes a day.

The Rundown is the world’s most trusted AI newsletter, with over 700,000+ readers staying up-to-date with the latest AI news, understanding why it matters, and learning how to apply it in their work.

Their expert research team spends all day learning what’s new in AI, then distills the most important developments into one free email every morning.

How I created the love letter to the garden and bashed together 30 different Gaussian splats into a single scene
— Paul Trillo (@paultrillo)
1:38 PM • Oct 18, 2024

🍿 Better than the Dune worm: Gladiator 2’s popcorn bucket is an expected coliseum…But using your phone, you can view an interactive AR fight at the bottom of the arena (after you finish your popcorn).

👨🏻‍💻 The Depth Any Video model offers high-quality and consistent depth estimation for various video scenarios

🔲 ARwall launches the ARFX App for Windows, winner of this years NAB Show Product of the Year, which aims to simplify virtual production for indie creators

🕹️ Epic Games launches Fab, a unified digital asset marketplace supporting multiple game engines and offering free access to Quixel Megascans through 2024