The tech company Flawless has set up a new lab in Los Angeles for work on their artificial intelligence-powered film dubbing technology. Specifically, the company is working on creating artificial intelligence that can make alterations to the facial movements of actors in old films so that they match to the words of a voice actor’s dub.
The lab will be led by Ryan Black, a former Lionsgate executive. You can better understand the technology from the video below.
As you can see from the demonstration, Flawless aims to break the most staggering barrier in viewing films from different parts of the world: language. This could have massive implications for the way people view foreign language films.
On the surface, it seems like cool technology with a noble goal — trying to make all cinema more accessible to the casual viewer. However, there are some troubling implications about this type of artificial intelligence.
For starters, it seems that the technology isn’t quite there. Not only do the mouth movements not match up all that perfectly, but it seems to live in the uncanny valley that viral celebrity deepfakes operate in. It is impressively realistic, but not quite believable and a tiny bit unsettling.
One of the most glaring issues is how to justify retroactively changing a film in a way that differs from its creators’ original vision. While it may try to stay faithful to the original script and performance, the software is quite literally putting words in actors’ mouths, and undeniably altering the original performance even if only through minute details. The idea of a robot “neural net” altering a human’s performance to make it connect with more humans seems a bit ironic.
Finally, dubbing is not famously a great way to watch films in the first place. They typically involve less talented actors who are forced to match the cadence of the original footage while saying something completely different that could be shorter or longer. Subtitles, on the other hand, fit into any scene, uphold the actor’s original performance, and can even benefit the viewing at times (mumbled lines, deadpan comedy, etc.).
While subtitles are not the optimal way to enjoy any film, it is hard to say that artificially manipulated footage is better, or even artistically excusable. In a perfect world, we would all speak the native language of any given film. However, subtitles may be our only worthy substitute for the time being.
