Machine Learning Can Help Kids with Autism Recognize Emotions
Gamifying this often daunting task could prove more effective than traditional methods
Recognizing and responding to emotional and mental states of others is one of the frequently reported symptoms for children with autism. By turning input from Narrative Clip, a wearable camera, and Microsoft’s emotion recognition software into a game, Autimood can help kids enhance their social skills in a fun and effective way.
The project originated from the Narrative Hackathon, organized by Narrative, the company behind a wearable camera that takes a photo every 30 seconds, allowing its users to have a visual record of all of their encounters. At the end of the day, kids can sit down with their parents and play a game of recognizing human emotions from the photographs. The game has a potential to be more effective than a computer-generated equivalent as it is not only set in a specific context of real, everyday interactions, but it also creates a stronger bond between the child and the parents, since the game enables them to give the child feedback on his or her performance.
The emotion recognition component of the game was obtained through Microsoft Project Oxford Emotion API, which uses machine-learning techniques and cloud-based emotion recognition algorithm. For every submitted photo, the program evaluates each of the eight core emotional states that are communicated cross-culturally: happiness, sadness, surprise, anger, fear, contempt, disgust and neutral. It then returns a value between zero and one for each state. All scores sum to one and the emotion with the highest score should be interpreted as the dominant one. The bata API is available for a limited free trial.