Steve Clayton: Microsoft Software Replicates Human Movement On-Screen

Steve Clayton: Microsoft Software Replicates Human Movement On-Screen

New Digits software from Microsoft blends the digital and real worlds and allows for the potential of hands-free control of your devices.

Steve Clayton, Microsoft
  • 8 october 2012

The 25th ACM Symposium on User Interface Software and Technology (UIST) is underway this week in Cambridge, Massachusetts. UIST brings together researchers and practitioners from diverse areas that include traditional graphical and web user interfaces, tangible and ubiquitous computing, virtual and augmented reality, multimedia, new input and output devices and more. That’s a good check list against some of the work that goes on in Microsoft Research and as always, MSR will be there in force.

We’re a gold sponsor of the event and Hrvoje Benko of Microsoft Research is one of the program chairs – regular readers will be familiar with Benko’s work on projects such as thewearable multi-touch projector.

A number of Microsoft Research projects will be shown at UIST and one caught my eye in particular – Digits. The project highlights the ongoing work of MSR in the field of natural user interfaces and the blending of the physical and virtual worlds. Digits employs wearable vision technology to enable freehand gesture input that could potentially be used with personal computing such as a mobile phone, laptop or gaming console. I had the chance to speak with David Kim, one of the researchers behind the project who helped me understand how it works.

David explained that Digits was inspired by a desire to enable 3D interaction that was not bound to a physical space in the way an earlier project known as Holodesk was. The team wanted to make spatial interaction mobile and turned to a combination of computer vision techniques and off the shelf hardware to build the “wristband” that you see in the video.

The combination of an infrared (IR) laser, IR camera, IR diffuser for illumination and an inertial measurement unit enables tracking of fingers. The current setup can only see five points, which normally would make it difficult to get an accurate view of the hand pose, so the vision system is allied to a biomechanical model of the hand for a full Kinematics model that delivers a very a high  degree of accuracy in modeling the hand pose. There are still some occasions when the tracking is affected by occlusion, for example when the hand is in a certain pose – though David mentioned the potential to apply a machine learning algorithm that would help alleviate these issues and further improve the accuracy.

The video already shows some pretty cool applications of the technology – and I’ve no doubt we’ll hear more ideas this week as Digits get shown at UIST and around the web. As is often the case, it’s the result of a a multidisciplinary team that has come together and built a concept in relatively short order using off the shelf components. Imagine what’s possible with more bespoke components and the inevitable miniaturization?

Originally posted on Next at Microsoft. Republished with kind permission.

Image via MSN.



DIY Kit Lets You Build Your Own Wooden Bike, Boat Or Caravan

Design & Architecture
Culture Yesterday

Messaging Add-On Helps You Correct Your Friends’ Bad Grammar

An iMessage sticker pack will help you copyedit text messages

Automotive Yesterday

Mercedes-Benz Introduces A New Electric Mobility Brand

The separate entity aims to simplify the identification of Mercedes EV products to customers


Get PSFK's Related Report: Future of Automotive

See All
Mobile Yesterday

Tinder’s New Feature Makes Swiping A Group Effort

The dating app wants to democratize its gestural interaction by buying in to the social polling trend pervasive among millennials

Related Expert

Vikram Gandhi

Film Maker, Perception Twister

Syndicated Yesterday

Autonomous Garbage Drone Prevents Trash From Reaching Deep Ocean

The solar-powered WasteShark collects refuse closer to the source: the harbor

Automotive Yesterday

Aston Martin Reveals Its Own Luxury Powerboat

The sleek AM37 echoes styling elements from the British brand's sports cars

Advertising Yesterday

An Escort Website Fights Violence Against Sex Workers

The advocacy campaign from McCann aims uncover the human toll of the exploitative industry


Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry

PSFK Op-Ed september 29, 2016

Digital Design Expert: Mobile First Is Dead, Think Mobile Native

Brian Cooper, chief creative officer of OLIVER Group UK, explains how some brands are still playing catch-up to new technology

PSFK Labs september 29, 2016

The 10 Steps To Discover, Hire, Develop Your Next Leader

PSFK's Future of Work report outlines key steps in the employee development path to empower next-gen leaders

Culture Yesterday

LIFE Magazine Relaunches In Pure VR

The general interest periodical, which ceased publication in 2000, has turned into a portal for virtual reality content

Mobile Yesterday

Reorder This Detox Drink With A Simple Text Message

Dirty Lemon is streamlining its communication by letting customers place orders, ask product questions and request help exclusively through chat

Op-Ed Yesterday

The Future Of The American Workforce Requires Unbundling College Education

President of JetBlue Technology Ventures: developing corporate education programs for non-traditional students

Retail Yesterday

Gilt’s Pop-Up House Is The Kind Of Store You’ll Want To Live In

The New York City townhouse plays host to the latest in retail inspiration, curation, and lifestyle activation (and some libations, too)


Future Of Work
Cultivating The Next Generation Of Leaders

Mobile Yesterday

Registering To Vote Is Now Just A Text Away

A new bot aims to mobilize underrepresented groups this election season through SMS and Facebook Messenger

Africa Yesterday

Virtual Reality Game Gives Lessons About Emergency Birth Care

LIFE is a serious tool that takes advantage of new technology to help save lives

Luxury Yesterday

Shoe Repair Has Moved Onto Your Phone

Cobbler Concierge is an on-demand service to get your footwear fixed online

No search results found.