NVIDIA has announced its latest AI model, the Foundational Generative Audio Transformer Opus 1, or Fugatto for short. This model can generate music, background audio, and even voice based on prompts entered by the user.
What makes Fugatto interesting is its ability to re-compose the audio that has been generated based on additional prompts. For example, users can ask for a guitar instrument to be replaced with a violin. It can also ask the generated voice to sound angry, happy, or sad.
In addition, users can also instruct it to separate the singer’s voice from the background music based on the uploaded file. In addition, new musical instruments can also be created, such as a saxophone that sounds like a cat meowing.
Fugatto was trained using 2.5 billion parameters and audio from open sources. At this time, NVIDIA does not plan to make this model publicly available for security reasons.