‘SimSensei’ is a new program in development that tracks body movements and facial expressions to give medical advice.
The University of Southern California’s Institute for Creative Technologies (ICT) is developing a a virtual human platform, designed as a support tool for clinicians and healthcare providers, that can help diagnose depression. ‘SimSensei’ is based on ICT’s 10+ years of expertise with virtual human research and development.
The platform enables an engaging face-to-face interaction where the virtual human automatically reacts to the perceived user state and intent, through its own speech and gestures. The virtual human asks questions, guides conversation depending on answers, nods and “Hmms” at appropriate times.
Gizmodo notes that it also uses Microsoft Kinect sensors to track changes in body language and facial expressions. These non-verbal cues aren’t usually analyzed when making a depression diagnosis, so they could help identify more cases that would otherwise be missed. You can check out ICT’s new platform in the video below: