How close are deepfakes to use in big budget films and TV shows? Pretty close when a new Disney demo has something to offer. In one Video and paper presented at a Computer graphics conference This week, House of Mouse researchers are showing what they think is the first photo-realistic deep fake with a megapixel resolution.
And the results are … pretty good! They're certainly not stunning and not good enough to be used in the next Marvel movie, but it's a solid advance over the deepfakes we've seen in the past.
As the researchers suspect, the megapixel resolution is new here. Megapixels may no longer be the epitome of the high quality images they used to be. (Your phone's camera may have a two-digit megapixel number at first.) So far, however, deepfake technology has focused on smooth face transfers rather than increasing the number of pixels.
(embed) https://www.youtube.com/watch?v=yji0t6KS7Qo (/ embed)
The deepfakes you've probably seen so far may look impressive on your phone, but their mistakes are much more apparent on a larger screen. Disney researchers find, for example, that the videos can be created with maximum resolution from the popular open source deepfake model DeepFakeLab were only 256 x 256 pixels. In comparison, your model can produce videos with a resolution of 1024 x 1024 – a considerable increase.
That being said, the functionality of the Disney deepfake model is fairly conventional: it can swap the appearance of two people while maintaining the facial expression of the target. When you watch the video, notice how technically limited the output appears to be. Only deep counterfeits of well-lit people are created who look more or less directly into the camera. Challenging angles and lighting are still not on the agenda for this technology.
However, as the researchers note, we are approaching the creation of deepfakes that are good enough for commercial projects. Right now, if a company like Disney wants to do face swapping, they'll be using traditional VFX, like the studio did when it created virtual models of late actors Peter Cushing and Carrie Fisher for the war of stars Movie Rogue One.
"Although these results are impressive, they are expensive to manufacture and typically require many months of work to get seconds of footage," the researchers write. In comparison, deep fakes require far less supervision after creating the original model and can produce videos in a few hours (with the right budget for computing power).
Sooner or later, deepfakes will no longer be a research project and will be a viable option for large studios. Some would even argue that they are already there.
