I Tried Prime Video’s New AI Dub and I Have Some Thoughts
When you watch media created in a language you don’t speak, you have several options. You can, of course, turn on subtitles and read on. But in some cases, you can switch the audio track to whatever language you speak. This is called “dubbing,” and usually voice artists record the lines in this new language so that they match the original tone of the show or movie.
This requires time and money, which studios are not always willing to part with. So not everything is dubbed. But what if it were possible?
On Wednesday , Amazon announced “AI-powered dubbing” for Prime Video . For some films, viewers can choose to play an AI-generated dub of the script in English or Latin American Spanish. Amazon says there are currently 12 games that support the feature, but only three are named: El Cid: La Leyenda; Mi Mama Laura ; and Long Lost . The company also says that “localization specialists” work with AI to “ensure quality control.” It’s unclear what this means, other than that people check these AI dubs to make sure they don’t mistranslate or make other mistakes due to hallucinations .
I have Prime Video, so I was interested in watching these dubs myself to see how Amazon’s AI compares to humans. In my search, I couldn’t find Mi Mama Laura , but I was able to find two other titles. To use this feature, you need to go to the compatible title’s language switcher and select “AI Beta.” Then you can experience what I experienced.
El Cid: La Leyenda
There are several movies called El Cid on Prime Video, but the only option that supports AI dubbing is this particular one : a 63-minute documentary about Rodrigo Diaz de Vivar, aka El Cid . I didn’t watch El Cid: La Layenda to learn more about this ruler of medieval Spain—I wanted to hear how well an AI bot would handle the English dub of a film shot in Spanish.
The results are interesting, to say the least. Most of the conversations in the documentary are done through voiceover, which is extremely unfortunate. (The “punch” may be too generous.) When you know the voice is AI-generated, you choose soft or strange intonations. This is classic AI-generated speech : items in a list are spoken in an inconsistent tone; some words are slurred or artificially slowed down, as if the bot had a couple of drinks. But to be honest, I expected the worst. It’s not like watching a documentary narrated by TikTok’s AI voice. It’s not great (not even good), but it’s a little more emotional than I expected.
The problems also extend to the dubbing of documentary film interlocutors using artificial intelligence. In many cases the voice is quite awkward, unnatural and full of the inconsistencies mentioned above. Sometimes the voice is completely devoid of any emotion and looks more like the voiceover tool of yesteryear than the “high quality” AI voices we’ve come to expect.
To Amazon’s credit, the AI appears to use a different voice for each person it duplicates. This makes it a little easier to distinguish between different speakers and saves you from having to listen to the same mediocre AI voice throughout the entire movie.
Long lost
Long Lost is a little more difficult to judge: The film, originally shot in English, has a faux Spanish dub. Since I don’t speak Spanish, it’s not so easy to pick up on the robotic quirks that your ear will pick up on when you know the language.
However, Long Lost offers a different experience than El Cid because it’s a film: you can hear how the AI handles dubbing the actors. Like the AI voiceover in the documentary, these AI actors aren’t as bad as I expected. I expected the AI voices to deliver their lines in a completely flat manner, but at times there is some emotion here. Note that no robot here will win an Oscar: in fact, the voices can be quite awkward and unclear if the script is read too loudly, harshly or overtly. But for some of the lines, I’m not sure I would have assumed they were AI voices if I didn’t know ahead of time.
Even in more intense scenes, the AI tries hard. When a character screams, the AI raises its voice. It’s far from perfect, but it’s an interesting experiment to see what AI can do:
In one scene, two characters compete in a game of “Plump Bunny”, in which each must keep adding marshmallows to their mouth while still being able to say “chubby bunny”. The AI does sound muffled at times, and while it’s not “realistic” it’s nice. (Perhaps this is where human intervention comes into play.) At other times, however, the AI sounds completely normal and the actors’ mouths are clearly full of marshmallows.
Another interesting feature is that one of the characters starts speaking French, so the AI track is cut off and the original audio appears. It’s frustrating because you can immediately hear the difference – not just the different languages, of course, but also the difference in what the actual movie audio sounds like compared to the sterile AI output.
I’d like to try this feature again on a movie shot in Spanish with an English AI dub. I think it would be easier for me to isolate the problems and know if most (or all) of the voices would sound completely false.
Is there a future for AI-powered dubbing?
Look, there is no way I genuinely enjoy watching an entire movie or TV series dubbed using AI. Personally, I’m not a big fan of human-made dubs, so I’m not interested in one that sounds like a robot hunting for EGOT at times.
But I have to admit that the technology isn’t that terrible . At some points the voices sound realistic enough for me to get the gist, especially when the voiceover is also monotone and informative, such as during a documentary. And interestingly, these AI dubs can include elements from the show or movie, such as muffled speech during a game of Chubby Bunny.
I don’t want the voice actors here to be out of work. They deserve to be paid for their work, and viewers deserve high-quality dubbing of the shows and films they pay to watch. However, I think AI dubbing could be a compromise in cases where no dubbing is prepared at all: if the studio isn’t going to pay for dubbing anyway, it would be useful to have dubbing at all, even if it’s done on the robotic side. This will certainly help visually impaired viewers who may not be able to use subtitles.
Of course, it’s naive to think that studios will only use this technology for certain purposes rather than wipe out an entire human profession to save some money. On that note, I’d be glad if the whole practice of artificial dubbing ended here and now: it’s interesting, but not good, so please don’t replace real voice actors.