PSFK chats with the Microsoft storyteller about the tech giant’s gesture-based gaming technology, trends in software innovation and the new era for the Natural User Interface.
Steve Clayton is the connection between Microsoft’s consumers and the brand itself. He works with people across all levels of the company in order to figure out what’s coming next in the tech world and how Microsoft can best utilize and incorporate that info. From researchers, to cinematographers, to race car drivers, he talks to and formulates updates on the ‘Next at Microsoft‘ website. Recently, Microsoft publicly unveiled the software development kit for their Kinect technology, which is a voice, facial, and motion recognition software that allows the human body or physical objects to work as a game controller, free of charge. Instead of fighting the hacking community, Microsoft opened their arms to welcome all hackers to be creative with their software. We spoke with Steve to find out more details about the code package and what has happened since the its release:
We’ve been seeing developers creating interesting “hacks” with Microsoft’s Kinect for a while now, what have you issued that is any different from what they had in the past?
Part of what we released is code that will help folks “voice-hack.”
Can you explain that a little more – and what opportunities that presents?
To date a lot of the creativity we have seen around Kinect has been gesture based (which is terrific to see) but Kinect has some amazing audio technology too. Check out Listening to Kinect to get a sense of what I mean. Right now, I can control my Xbox using voice, which works particularly well with features such as Netflix or Zune video, enabling me to choose a movie, play, pause, fast forward etc. just using my voice. Imagine driving all manner of other experiences with your voice: search, or controlling appliances in the home or office. It gets even more interesting when you combine modalities — by that I mean the combination of voice AND gesture is very useful and also very natural. Think about when you ask for something in a store and you say “that one,” you typically point at the same time. The combination of gesture and voice has a lot of untapped potential.
You launched the software update with a ‘Codecamp’ where developers have been creating their own hacks of Kinect with the official Microsoft code. What were some of your favorite projects?
My two favorite projects span the two ends of the tech spectrum. The first is a game of Pong (the classic arcade game) which two guys from Lewis and Clark college reimagined for the future with the body acting as the paddle that hits the ball. Using more contemporary technology, another group took a Quadrocopter and enabled it to be controlled using gestures – it’s impressive stuff that you can see in the video I shot here. This shows what you can do in 24 hours, imagine what we’ll see in 24 days, or 24 months?
Gesture based control seems to have captured developers imagination while 3D TV hasn’t. Both technologies were released about the same time. Is there something about natural user interface that the community embraced? Why isn’t anyone playing with 3DTV?
I’ve not looked (literally) at 3DTV that much myself and I think you’re right, there is something about the natural interface opportunities with Kinect that has captured people’s imagination. The sensor is $150 which makes it pretty affordable and with 10 million units already sold I think we’re just at the beginning of what’s possible with this kind of technology. I think the NUI era truly is upon us.