There’s a whole lot of money in voice cloning.
Working example: ElevenLabs, a startup developing AI-powered tools to create and edit synthetic voices, today announced that it closed an $80 million Series B round co-led by outstanding investors including Andreessen Horowitz, former GitHub CEO Nat Friedman and entrepreneur Daniel Gross.
The round, which also had participation from Sequoia Capital, Smash Capital, SV Angel, BroadLight Capital and Credo Ventures, brings ElevenLabs’ total raised to $101 million and values the corporate at over $1 billion (up from ~$100 million last June). CEO Mati Staniszewski says the brand new money might be put toward product development, expanding ElevenLabs’ infrastructure and team, AI research and “enhancing safety measures to make sure responsible and ethical development of AI technology.”
“We raised the brand new money to cement ElevenLabs’ position as the worldwide leader in voice AI research and product deployment,” Staniszewski told TechCrunch in an email interview.
Co-founded in 2022 by Piotr Dabkowski, an ex-Google machine learning engineer, and Staniszewski, a former Palantir deployment strategist, ElevenLabs launched in beta around a yr ago. Staniszewski says that he and Dabkowski, who grew up in Poland, were inspired to create voice cloning tools by poorly dubbed American movies. AI could do higher, they thought.
Today, ElevenLabs is probably best known for its browser-based speech generation app that may create lifelike voices with adjustable toggles for intonation, emotion, cadence and other key vocal characteristics. Without cost, users can enter text and get a recording of that text read aloud by considered one of several default voices. Paying customers can upload voice samples to craft recent styles using ElevenLabs’ voice cloning.
Increasingly, ElevenLabs is investing in versions of its speech-generating tech aimed toward creating audiobooks and dubbing movies and TV shows, in addition to generating character voices for games and marketing activations.
Last yr, the corporate released a “speech to speech” tool that attempts to preserve a speaker’s voice, prosody and intonation while routinely removing background noise, and — within the case of flicks and TV shows — translates and synchronizes speech with the source material. On the roadmap for the approaching weeks is a brand new dubbing studio workflow with tools to generate and edit transcripts and translations and a subscription-based mobile app that narrates webpages and text using ElevenLabs voices.
ElevenLabs’ innovations have won the startup customers in Paradox Interactive, the sport developer whose recent projects include Cities: Skylines 2 and Stellaris, and The Washington Post — amongst other publishing, media and entertainment firms. Staniszewski claims that ElevenLab users have generated the equivalent of greater than 100 years of audio and that the platform is getting used by employees at 41% of Fortune 500 firms.
However the publicity hasn’t been totally positive.
The infamous message board 4chan, known for its conspiratorial content, used ElevenLabs’ tools to share hateful messages mimicking celebrities like actress Emma Watson. The Verge’s James Vincent was in a position to tap ElevenLabs to maliciously clone voices in a matter of seconds, generating samples containing the whole lot from threats of violence to racist and transphobic remarks. And over at Vox, reporter Joseph Cox documented generating a clone convincing enough to idiot a bank’s authentication system.
In response, ElevenLabs has attempted to root out users repeatedly violating its terms of service, which prohibits abuse, and rolled out a tool to detect speech created by its platform. This yr, ElevenLabs plans to enhance the detection tool to flag audio from other voice-generating AI models and partner with unnamed “distribution players” to make the tool available on third-party platforms, Staniszewski says.
ElevenLabs offers an array of various voices, some synthetic, some cloned from voice actors.
ElevenLabs has also faced criticism from voice actors who claim that the corporate uses samples of their voices without their consent — samples that could possibly be leveraged to advertise content they don’t endorse or spread mis- and dis-information. In a recent Vice article, victims recount how ElevenLabs was utilized in harassment campaigns against them, in a single example to share an actor’s private information — their home address — using a cloned voice.
Then there’s the elephant within the room: the existential threat platforms like ElevenLabs pose to the voice acting industry.
Motherboard writes about how voice actors are increasingly being asked to sign away rights to their voices in order that clients can use AI to generate synthetic versions that might eventually replace them — sometimes without commensurate compensation. The fear is that voice work — particularly low cost, entry-level work — will eventually get replaced by AI-generated vocals, and that actors could have no recourse.
Some platforms try to strike a balance. Earlier this month, Replica Studios, an ElevenLabs competitor, signed a take care of SAG-AFTRA to create and license digital replicas of the media artist union members’ voices. In a press release, the organizations said that the arrangement established “fair” and “ethical” terms and conditions to make sure performer consent — and negotiating terms for uses of digital voice doubles in recent works.
Even this didn’t please some voice actors, nonetheless — including SAG-AFTRA’s own members.
ElevenLabs’ solution is a marketplace for voices. Currently in alpha and set to change into more widely available in the subsequent several weeks, the marketplace allows users to create a voice, confirm and share it. When others use a voice, the unique creators receive compensation, Staniszewski says.
“Users all the time retain control over their voice’s availability and compensation terms,” he added. “The marketplace is designed as a step towards harmonizing AI advancements with established industry practices, while also bringing a various set of voices to ElevenLabs’ platform.”
Voice actors may take issue with the indisputable fact that ElevenLabs isn’t paying in money, though — at the least not at present. The present setup has creators receiving credit toward ElevenLabs’ premium services (which some find ironic, I’d wager).
Perhaps that’ll change in the long run as ElevenLabs — which is now among the many best-funded synthetic voice startups — attempts to beat back upstart competition like Papercup, Deepdub, ElevenLabs, Acapela, Respeecher and Voice.ai in addition to Big Tech incumbents equivalent to Amazon, Microsoft and Google. In any case, ElevenLabs, which plans to grow its headcount from 40 people to 100 by the top of the yr, intends on sticking around — and making waves — within the fast-growing synthetic voice market.
