How Pre-Touch Screens Try To Read Your Mind
Microsoft is developing touchscreen tech that could anticipate actions
A team from Microsoft Research has explored Pre-Touch Sensing using a touchscreen that can sense multiple fingers above a mobile device and grip around the screen’s edges. This research could have huge potential for the future of mobile interaction, leading to the development of even smarter mobile devices that can anticipate your intended actions before you even touch the screen.
Microsoft principal researcher Ken Hinckley led the project, with the research being presented in a paper titled “Pre-Touch Sensing for Mobile Interaction.” It explored using the phone’s ability to sense how it is being gripped as well as when and where fingers are approaching it.
The team notes that the problem with current mobile devices is that much of what characterizes touch starts before contact and originates from beyond the confines of the screen. Users first grip their mobile with one hand and then reach for the screen with an index finger, a thumb, or multiple digits for pinch-to-zoom. As their hand approaches, its posture hints at the user’s intended trajectory, indicating likely targets. All of this contextual detail, collectively referred to as pre-touch, is lost to mobile.
If a phone was capable of sensing your grip and approaching fingers, this would open up a lot of possibilities for mobile interaction, including an “ad-lib interface” that fades in a different UI for different scenarios. By allowing interfaces to adapt to you on the fly, depending on how you are currently using your phone (one-handed with a thumb, two-handed with an index finger, a pinch, two thumbs, etc.), they could be tailored to each specific context. For example, if you were holding your phone with your right hand while watching a video, a simple interface allowing you to pause, rewind or fast-forward could appear on the right-hand side when you moved your thumb towards the screen. You can learn more about Pre-Touch Sensing in the video below: