Meta Unveils Movie Gen AI Mannequin Able to Generating Video With Sound to Rival OpenAI

Facebook proprietor Meta introduced on Friday it had constructed a brand new AI mannequin referred to as Movie Gen that may create realistic-seeming video and audio clips in response to person prompts, claiming it will possibly rival instruments from main media era startups like OpenAI and ElevenLabs.

Samples of Movie Gen’s creations offered by Meta confirmed movies of animals swimming and browsing, in addition to movies utilizing individuals’s actual images to depict them performing actions like portray on a canvas.

Movie Gen can also generate background music and sound results synced to the content material of the movies, Meta stated in a weblog publish, and use the device to edit present movies.

In one such video, Meta had the device insert pom-poms into the arms of a person working by himself within the desert, whereas in one other it modified a parking zone the place a person was skateboarding from dry floor into one coated by a splashing puddle.

Meta’s Movie Gen AI mannequin lets customers edit movies utilizing textual content prompts
Photo Credit: Meta

Videos created by Movie Gen may be as much as 16 seconds lengthy, whereas audio may be as much as 45 seconds lengthy, Meta stated. It shared knowledge displaying blind checks indicating that the mannequin performs favorably in contrast with choices from startups together with Runway, OpenAI, ElevenLabs and Kling.

The announcement comes as Hollywood has been wrestling with harness generative AI video know-how this yr, after Microsoft-backed OpenAI in February first confirmed off how its product Sora may create characteristic film-like movies in response to textual content prompts.

Technologists within the leisure trade are keen to make use of such instruments to reinforce and expedite filmmaking, whereas others fear about embracing programs that seem to have been educated on copyright works with out permission.

Lawmakers even have highlighted issues about how AI-generated fakes, or deepfakes, are being utilized in elections around the globe, together with within the U.S., Pakistan, India and Indonesia.

Meta spokespeople stated the corporate was unlikely to launch Movie Gen for open use by builders, because it has with its Llama sequence of large-language fashions, saying it considers the dangers individually for every mannequin. They declined to touch upon Meta’s evaluation for Movie Gen particularly.

Instead, they stated, Meta was working straight with the leisure group and different content material creators on makes use of of Movie Gen and would incorporate it into Meta’s personal merchandise someday subsequent yr.

According to the weblog publish and a analysis paper concerning the device launched by Meta, the corporate used a mixture of licensed and publicly accessible datasets to construct Movie Gen.

OpenAI has been assembly with Hollywood executives and brokers this yr to debate doable partnerships involving Sora, though no offers have been reported to have come out of these talks but. Anxieties over the corporate’s strategy elevated in May when actor Scarlett Johansson accused the ChatGPT maker of imitating her voice with out permission for its chatbot.

Lions Gate Entertainment, the corporate behind “The Hunger Games” and “Twilight,” introduced in September that it was giving AI startup Runway entry to its movie and tv library to coach an AI mannequin. In return, it stated, the studio and its filmmakers can use the mannequin to reinforce their work.

© Thomson Reuters 2024

(This story has not been edited by NDTV workers and is auto-generated from a syndicated feed.)