As AI continues to grow, it is our duty as producers to grow and adapt with new technological avenues available to us to produce the best content for audiences. The latest advancement involves AI voice, also known as vocal transformers, vocal synthesis, or vocal cloning. You may be thinking of a pitch shifter or auto-tune, but the neural network and generative AI used for these applications is a bit more nuanced and involved than your typical plug-in. AI voice generators, originally used for TTS (Text-to-Speech), are just as ready to tackle something as complicated as singing and some do quite a good job at fooling the unsuspecting listener. Here are a few of the top AI voice swaps and generators available to us today.
First on the list is www.voiceswap.ai. Co-founded by Dan Stein, also known as DJ Fresh, it claims to be an “AI that works for artists,” hoping to give more options for musicians and producers to use in their upcoming hits. Voiceswap.ai uses real artists as the basis for the AI models, having them come in studio to record demos to train the neural net on the different intricacies and styles of their voice. The user then uploads their own mp3, chooses the artist, and then out comes the original recording in the chosen artist’s voice. While it does a good job, it isn’t perfect. Going from female to male, or male to female vocalists distorts the sound, but it may still be useful as a vocal chop in a track you’re working on. Additionally, any pitch issues only seem to be exacerbated after the swap.
Here's the Promo Video:
Below is the Raw Audio followed by the processed files.
The promo video above appears far more promising than the tests that we ran. It looks like the raw audio needs to be auto tuned and possibly worked before its processed Future versions may produce better results.
Both subsequent processed audio sound as if the formants are shifted from the original with additional harmonic overtones added. The results are not as convincing as the promo video.
The FAQ also states that these models are exclusively trained on singing and not spoken word. Be sure to only use dry vocals and the best possible original recording.
Much of it depends on the skill of the vocalist and the ability of the producer to correct any issues, while still making the voice sound authentic. With that in mind the results for voiceswap.ai are mediocre at best.
Up next is a web app called app.kits.ai. This web app has a few more choices, including a vocal separator and the ability to train your own AI voice.
Just upload some dry vocal tracks like you normally would and the integrated neural net will create an AI artist of your own voice for your use. Kits.AI has some official artists they partner with, but the interesting options are the free voices. They range in character from things like male strained rock to female slavic and male Hindi to female afro pop, with all the options in between. The vocal separator is top notch as well for a free tool, but as usual there is a paid version as well that gives you unlimited and commercial use.
Listen to the same original track from above that has been voice swapped with two of the free to use AI vocals. Kits.AI seems to handle pitch discrepancies better and has a cleaner, smoother feel to the voice swaps. The free starter plan gives you about 15 minutes of audio and 300 characters to swap before needing to upgrade to the paid version, but the minutes and characters replenish once a month. Kits.AI is a great tool to play around with for different vocal styles, and AI instruments you can voice swap with as well.
Last on our list is www.revocalize.ai. Possibly the best web based AI voice tool available today, revocalize’s AI and neural network surpasses others available on the market with its ability to capture the subtle nuances such as the accent and emotions in your voice and adapt it to the new AI voice you are converting to. The free version offers you four voices, some more suitable for speech, singing, or rapping, while the advanced options dropdown menu has a pitch shifter, various pre and post processing effects, as well as AI key detection, auto tune, and a harmony generator. Revocalize does not offer a free trial unfortunately, but listen below to revocalize.ai’s ability to create an AI voice of a singer with greater accuracy, making it easier for the artist to create unique tracks.
Singing was once considered a huge hurdle for AI to overcome, However, it is well on its way to nailing down the finer details that are unique to each individuals voice. Compared to some of the other AI tools we have available, AI voice still seems to have a way to go as far as singing is concerned. Each of the websites detailed here promise to have ethics at the forefront of their mission statement, claiming to protect individuals rights to their own voice and partnering with artists to give them a fair payout. They have an affiliate program as well as developer API tools to keep things as open and fair as possible. Things like these will be crucial as we move forward into the future of AI and navigate our way through what is a human voice and a human voice that has been generated by a neural network. In any case, whether we use these tools now as they are or in the future when they have improved, it is the role of the producer to mix and master the vocals appropriately.