Researchers at Toshiba’s Cambridge Research Lab and the University of Cambridge‘s Department of Engineering have collaborated to develop a digital avatar that can express a full range of human emotions. ‘Zoe’ is a virtual “talking head,” which could be used as a digital personal assistant or for “face messaging.”
The lifelike face can appear happy, sad, angry, tender, afraid, neutral or a combination of these emotions. Users can type in a message and indicate what emotions it should have, and it will recite the text.
The face is that of Zoe Lister, a TV actress, who helped train the speech model. The researchers spent several days recording her speech and facial expressions in order to recreate her face and voice.
They collected a dataset of thousands of sentences and tracked her face while she was speaking using computer vision software. In the future the system, which is light enough to work in mobile technology, could enable people to upload their own faces and voices to personalize their own emotionally realistic digital assistants.
The researchers are currently working with a school for autistic and deaf children, where the technology could be used to help them understand emotions and lip-read. It could also have uses in gaming, audio-visual books, online lectures, and other user interfaces. You can meet Zoe in the video below: