Stable Audio Open Released by Stability AI as an Open-Source Text-to-Audio Generator

Stability AI has released an open source artificial intelligence (AI) model called Stable Audio Open. Users can leverage the model to generate up to 47 seconds of samples and sound effects. Users can use it to create samples of musical instruments or ambient sounds. The AI ​​model also allows users to generate different variations and styles from a pre-generated sample. The open-source model is separate from the AI ​​company's Stable Audio platform, which allows users to generate tracks up to three minutes long and is only available as part of a subscription.

Stability AI launches Stable Audio Open

Stable Audio Open works similarly to many AI models available on the market. Users can type a text message for a sample or sound effect and it will generate up to 47 seconds of audio. Stability AI mentioned in a press release that the AI ​​model was released in open source to empower sound designers, musicians and creative communities.

However, it has limited the use of Stable Audio Open to research and non-commercial use. To obtain commercial rights, users will need to purchase a subscription to Stability AI.

In terms of features, it can generate drum beats, instrument riffs, ambient sounds, foley recordings and other audio samples. In addition, users can also fine-tune the model using their custom audio data.

Highlighting one example, the AI ​​company says a drummer can train the AI ​​on recordings of his drumming sessions and use the model to generate new beats. Although the model can generate short audio samples, it is not optimized for songs, melodies, or full voices.

To train Stable Audio Open, the company used a dataset of 4,86,492 audio recordings sourced from FreeSound and the Free Music Archive. He added: “We performed an in-depth analysis to ensure that there was no unauthorized copyrighted music in our training data before we started training.”

However, Stability AI also said that the dataset lacked diversity and that not all cultures were equally represented. As a result, the generated samples will reflect biases in the training data. To access the AI ​​model, users can go to the company's Hugging Face list, where the open model weights are currently located.