Facebook’s parent company Meta announced on Friday the development of a new AI model named Movie Gen, capable of generating realistic video and audio clips based on user prompts.
The company claims this technology can compete with offerings from leading media generation startups like OpenAI and ElevenLabs.
Samples shared by Meta featured videos of animals swimming and surfing, as well as clips using real photographs of individuals engaged in activities like painting. Movie Gen can also create background music and sound effects that sync with the visual content, and it allows for the editing of existing videos.
In one demonstration, the model added pom-poms to a man’s hands as he ran in a desert while another video transformed a dry parking lot into one with splashing puddles as a man skateboarded.
According to Meta, videos generated by Movie Gen can last up to 16 seconds, and audio clips can be as long as 45 seconds. Blind tests indicated that Movie Gen performs well compared to tools from startups such as Runway, OpenAI, ElevenLabs, and Kling.
This announcement comes as Hollywood grapples with how to utilise generative AI technology for video production. Microsoft-backed OpenAI previously showcased its product Sora, which creates feature film-like videos from text prompts.
While many in the entertainment industry are eager to leverage such tools to enhance filmmaking, there are concerns about the technology being trained on copyrighted materials without permission. Lawmakers have also raised alarms about the use of AI-generated deepfakes in elections worldwide, including in the US, Pakistan, India, and Indonesia.
Meta representatives indicated that the company does not plan to make Movie Gen available for open use by developers, unlike its Llama series of large-language models. They stated that they are assessing the risks of each model individually and declined to provide specifics about Movie Gen.
Instead, Meta is collaborating directly with the entertainment industry and content creators to explore applications for Movie Gen, with plans to integrate it into Meta’s own products next year.
To develop Movie Gen, Meta utilised a combination of licensed and publicly available datasets.
Also, read this
WhatsApp to offer customizable Meta AI voices in upcoming update
Meta states new AI cannot be turned off on WhatsApp, Facebook and Instagram
The rise of artificial intelligence: A double-edged sword for humanity
Meanwhile, OpenAI has been in discussions with Hollywood executives regarding potential partnerships involving Sora, though no agreements have been reached. Concerns over OpenAI’s practices heightened in May when actress Scarlett Johansson accused the company of using her voice without permission for its chatbot.
Additionally, Lions Gate Entertainment, known for franchises like “The Hunger Games” and “Twilight,” announced in September that it would allow AI startup Runway access to its film and television library for training an AI model, enabling filmmakers to enhance their projects with this technology.
