Meta announced their latest generative artificial intelligence (AI) that is not only capable of generating video based on user text input but can also produce audio that matches the generated visuals. What makes Movie Gen unique compared to other video generating AIs is its ability to be personalized by the user. This latest model from Meta is trained using 30 billion parameters with the ability to generate 16 seconds of video at a rate of 16 frames per second.
The first is that the user can provide a picture as a reference. Then the generated video will use the reference image as the basis for the generated video. You might want to produce a video of yourself riding a bike for example. Upload a photo to Movie Gen and a video of you riding a bike will be generated.
The second feature that Movie Gen can do is that editing is done in more detail. The bike riding video can be edited by changing the color of the shirt, adding/removing background objects and changing to night time for example. With more detailed editing, the generated video output will be better controlled to meet user needs.
Finally, the model for audio generation is trained using 13 billion parameters. This allows Movie Gen to generate background music with audio for videos up to 45 seconds. Meta sees Movie Gen to be used in WhatsApp to generate videos for birthdays or Reels content on Instagram.