This AI Will Bring Characters From Old TV Shows Back To Life
A team from the University of Leeds is building virtual avatars to effectively immortalize pop culture figures
Researchers from the School of Computing at the University of Leeds in the UK have detailed their proof of concept for reanimating characters from TV shows. They have shown how capturing a character’s style of speech, language, and visual appearance can be used to build an interactive avatar and effectively ‘immortalize’ them.
The team obtained audiovisual material from the TV show Friends to demonstrate their concept using the character Joey. They trained a speech synthesizer to detect spoken audio and mute background noise prior to analysis, and used time-aligned transcripts as supervisory labels for training their system. An online multi-face detection and tracking system was used to produce face tracks for all episodes, and the team’s system automatically labeled audio and video with phonetic boundaries, character face bounding boxes with their character name, and facial landmarks, in order to train their generative model.