An Israeli start-up uses artificial intelligence to create digital doubles of people. This allows them to appear in an infinite number of commercials, explanatory videos and as video chatbots without having to stand in front of the camera every time. The company is currently looking for models - and promises money for every video in which the synthetic image appears.
By Michael Förtsch
Imagine that you could be omnipresent, helping millions of people with their everyday problems, giving them hints and advice. You could show up on their phones, laptops or TVs and explain how to install their newly purchased game console, help them prepare for an exam or even have a job interview with them. At least according to the Israeli start-up Hour One, this idea could well become reality. What the Tel Aviv team, currently consisting of eight people, is working on is to create digital doubles of real people. Or at least images of their bodies, which should not be distinguishable from real people.
„The idea comes from the vision that at some point in the future everyone could have a digital twin of themselves,“ says Natalie Monbiot of Hour One to 1E9. „In other words, a digital version of your own self that represents you in the digital sphere.“ This is exactly what Oren Aharon and Lior Hakim, the founders of Hour One, were researching about two years ago. However, they didn’t use 3D scanners to create the digital bodies, as a film studio did with Will Smith for the film Gemini Man. Instead, they wanted to use an artificial intelligence that simply learns how a person looks by itself.
„They thought that this would help them to overcome the Uncanny Valley“, says Monbiot, referring to the effect that digitally created characters imitating a human being often have. If they come very close to a real human, this ensures that small mistakes are unconsciously perceived particularly strongly and make the replica appear „uncanny“. These are often inconsistencies in facial expressions. For example, if there are no wrinkles when speaking, if muscles under the skin are not correctly tensed or when the eyes are too motionless or look into emptiness. Hour One demonstrates with videos on its website that AI-generated faces have fewer of these problems and are therefore more natural and real than 3D-modelled faces based on scans.
Deepfake technology, but no deepfakes?
The AI technology used by the developers of Hour One is not new or special in itself. In fact, it has already caused quite a stir and scandals. Because the basic principle is the same as with Deepfakes. Here, an AI system learns the face of a person based on thousands and thousands of images and creates a model with which the face can be exchanged for the learned face in a video. Matching the facial expressions with the ones of the original. At Hour One so-called Generative Adversarial Networks, i.e. AI systems that consist of two neural networks, learn not only a face, but also the body. A neuronal network tries to make a copy of the human being. Another network works against it. It tries to find errors. Both networks challenge each other. Thus the digital copy becomes more and more detailed and better. This is a complex and computationally intensive process, which is therefore carried out by computers in the cloud designed for AI use.
„There is a great deal in common,“ says Natalie Monbiot about the Deepfake technology. But still she doesn’t want what Hour One does to be understood as Deepfakes. Because Deepfakes would have become synonymous with manipulated videos – Hour One has nothing to do with that. Therefore, it is better to say that the start-up creates „digital twins“ that are used in „synthetic media content“. And of course, the team at Hour One works a little more professionally than the Deepfake artists: Instead of downloading celebrity videos from YouTube and social networks, the start-up invites the clone candidates to its studio in Tel-Aviv or organizes a studio and a cameraman on site, who then provides the footage that the artificial intelligence learns from.
The learned humans are then stored on a server infrastructure. There, they can be made to move and say what the customers wants using specially developed software. The voice also comes from the computer – via speech synthesis, as with audio deepfakes. In most cases it is not the original voice of the model, but of a professional speaker – although Hour One has the ability to clone the voices of the models. „We can use texts in almost any language and breathe life back into the characters,“ says the Hour One employee. "In this way, a single character can theoretically recite thousands of texts and interact with people around the world.
For Hour One this is no longer just theory, but already practice. „Today we are already replicating people and certain talents all over the world,“ says Natalie Monbiot, who says that numerous people from Israel, the USA, China and Russia have already been digitized. Likewise, thousands of small videos for advertising clips, living chat bots, e-learning and wellness offers have already been created. The calculation takes a few hours each time. If it goes to Hour One, these digital figures could soon be part of everyday life. „Yes, I think they will eventually become part of life and help us solve problems,“ says Natalie Monbiot with conviction. Just as Artificial Intelligence sneaks into all the components of life without being particularly noticeable today.
Who wants a doppelgänger?
To create the digital avatars, of course, you need volunteers to be immortalized. And that’s why anyone can apply to the Israeli company at the moment – and, if they are accepted, they can earn money with their own digital image. If the application is accepted, Hour One will organize a shooting with a cameraman in a studio nearby. „No special equipment is needed for this,“ says Natalie Monbiot. Hour One need just enough footage to create the doppelgängers – and that is video footage from all angles and as many and different facial expressions as possible. Usually 15 minutes of video would be enough. The artificial intelligence takes care of the rest.
„If you are then selected as a character by one or more of our clients, you will receive money for each video,“ says Natalie Monbiot. How much, she says, cannot be said in general terms. In addition, Hour One promises to ensure that the digital avatars are not used for borderline or illegal applications. Models can also decide whether they do not want to be used for certain products or media areas. For example, if someone doesn’t want to advertise meat products or homeopathy in a video, then that’s no problem. „In cases of certain people, for example celebrities,“ says Monbiot, „it may of course be that only they themselves are allowed to use their avatar.“
The developers at Hour One can understand why some people find digital doppelgängers quite scary. „We don’t want to pretend that the videos are real - or the people who are talking,“ says Monbiot. Viewers have a right to know what they are watching - which is why the company itself labels its videos with an „Altered Visuals“ note. Hour One also recommends that customers introduce themselves to the digital avatar with a „Hello, I’m an AI Assistant…“ or something similar. Ultimately, Monbiot believes, these are all topics that should be discussed in a public debate: „I think it’s up to us as a society to decide how and where all this will evolve.“
Images: Hour One