Play Music With Eye Movements and Facial Gestures
Musical interface enables people with physical disabilities to play any instrument
Eye Conductor is a musical interface built by Andreas Refsgaard as a final project at the Copenhagen Institute of Interaction Design. It enables those with physical disabilities to play music using their eye movements and facial gestures. They are given the opportunity to play any instrument, build beats, sequence melodies or trigger musical effects.
Refsgaard carried out user research for his project, visiting several schools and communities for people with physical disabilities. While the people he met had various levels of physical ability, he found that music was a unifying interest for all of them.
A lot of people with physical disabilities lack the fine motor skills needed to produce music on traditional instruments. Refsgaard wanted to push the boundaries of interaction design by exploring how eye and face-tracking technologies could be used for creative purposes. He set out to create a solution that operated in the same domain as traditional instruments, giving people freedom but requiring them to practice.
Eye Conductor uses a $99 eye tracker and a regular webcam to detect a person’s gaze and selected facial movements. It translates eye gaze into musical notes or beats in a drum sequencer. Different movements can signal the adjustment of different musical effects, for example, raising your eyebrows can be used to transpose all played notes up one full octave, while opening your mouth can add a delay, reverb or filter effect to the instrument. The system is open and designed for inclusion with customization options, so thresholds for facial gestures can be adjusted to support the unique abilities of each user.