Steve Clayton, the editor of the Next at Microsoft blog, shares his thoughts on the growing trend of Natural User Interfaces.
Trend watching has become a profession in recent years and there are sites dedicated to monitoring trends of all kinds. I spend a good deal of my time writing about trends in technology and observing a number of key trends that Microsoft is focused on. Individual trends such as Big Data, the Internet of Things and cloud computing are fascinating on their own—but over the last year it’s become clear to me that a number of trends are maturing and colliding at the same time and helping to fuel one big trend we’re focused on—Natural User Interfaces or NUI. Allow me to explain what NUI is and some of the trends behind it.
In simple terms, NUI is concerned with making technology much more natural to interact with. That may be through voice control of a TV (rather than fumbling with remotes and on screen menus), it may be through playing a video game just using your body with no need to master a controller of any kind, or it may be logging in to a system by simply having the system recognize you. It’s no coincidence that all of these examples are features of Kinect for Xbox 360, which has taken NUI in to 18 million homes in a little over a year. Kinect is a great example of NUI in action—taking advantage of advanced computing capabilities to remove the barriers for interaction. Kinect is just one example though—the touch screen interfaces that we have become familiar with on our smartphones are examples of NUI. Behind Kinect and smartphones there are a set of underlying trends that are enabling natural interaction. Let’s take a brief look at two—the proliferation of sensors and big data.
Kinect contains a number of “sensor”—a series of cameras, including a depth camera that enable Kinect to recognize you–as well as a sophisticated audio array of microphones that can pinpoint your voice, even in a very noisy room. When you stop and think about the number of sensors (or computers of some kind) in your life you begin to notice that there are far more than the traditional devices you often associate with computing. Increasingly our homes are filled with electronics in the form of household appliances, lighting systems, heating systems, alarm systems, even electronic vacuum cleaners. Each of these systems has a degree of intelligence and over time more and more of them will become connected to the Internet and connected together. Our cars already have hundreds of computers and sensors and they’re getting smarter each year. Soon they’ll be connecting to and “talking with” smart streets and cities to help us find a parking space more quickly and pay for it automatically.
Our smartphones are already filled with sensors—GPS, accelerometers, compasses, light meters and more. Some phones already increase the ringer volume when the light sensors detect it’s dark (and assume your phone is in your pocket or bag). As we allow them to, our smartphones are set to get smarter. The GPS knows where we are and the calendar knows where we’re going so perhaps it’ll connect to our car and pre-program the route? All of these scenarios, big or small are examples of our interaction with technology becoming more natural. And all of them generate data—every click, every non-click, every movement, every gesture. Each one is data and the more there is, the more natural technology can become, as we can teach it by example using techniques known as machine learning.
Devices generate data and data enables smarter devices. All of this will lead to more natural ways to interact with technology—and there is another trend we’re very focused on at Microsoft that we believe will help fuel this shift – displays everywhere. We foresee a world where almost any surface can become an interactive display and we’re busy pushing the boundaries with transparently displays, displays that can see as well as project and technology that turns the palm of your hand in to a display.
As this mixture of hardware and software gets woven together, design becomes ever more important. We need to remain focused on designing the appropriate interaction mode. A touch screen TV is no use when you’re sitting ten feet away—but voice control is great. Voice control will become more than a little frustrating if everyone on the subway decides to use it at the same time and gesture controlled gaming isn’t going to work when you’re sitting in seat 27D on an airplane.
I’m excited about the future as sensors, displays and data start to make technology invisible—it’s an entirely new canvas for designing amazing experiences.
For more on the future of natural user interfaces head to http://aka.ms/psfknui.
PSFK is thrilled to have Microsoft on board as a sponsor of this year’s PSFK Conference Series.
Click the banner below to purchase tickets and find additional information about this year’s event.