Steve Clayton: Microsoft Software Replicates Human Movement On-Screen

Steve Clayton: Microsoft Software Replicates Human Movement On-Screen

New Digits software from Microsoft blends the digital and real worlds and allows for the potential of hands-free control of your devices.

Steve Clayton, Microsoft
  • 8 october 2012

The 25th ACM Symposium on User Interface Software and Technology (UIST) is underway this week in Cambridge, Massachusetts. UIST brings together researchers and practitioners from diverse areas that include traditional graphical and web user interfaces, tangible and ubiquitous computing, virtual and augmented reality, multimedia, new input and output devices and more. That’s a good check list against some of the work that goes on in Microsoft Research and as always, MSR will be there in force.

We’re a gold sponsor of the event and Hrvoje Benko of Microsoft Research is one of the program chairs – regular readers will be familiar with Benko’s work on projects such as thewearable multi-touch projector.

A number of Microsoft Research projects will be shown at UIST and one caught my eye in particular – Digits. The project highlights the ongoing work of MSR in the field of natural user interfaces and the blending of the physical and virtual worlds. Digits employs wearable vision technology to enable freehand gesture input that could potentially be used with personal computing such as a mobile phone, laptop or gaming console. I had the chance to speak with David Kim, one of the researchers behind the project who helped me understand how it works.

David explained that Digits was inspired by a desire to enable 3D interaction that was not bound to a physical space in the way an earlier project known as Holodesk was. The team wanted to make spatial interaction mobile and turned to a combination of computer vision techniques and off the shelf hardware to build the “wristband” that you see in the video.

The combination of an infrared (IR) laser, IR camera, IR diffuser for illumination and an inertial measurement unit enables tracking of fingers. The current setup can only see five points, which normally would make it difficult to get an accurate view of the hand pose, so the vision system is allied to a biomechanical model of the hand for a full Kinematics model that delivers a very a high  degree of accuracy in modeling the hand pose. There are still some occasions when the tracking is affected by occlusion, for example when the hand is in a certain pose – though David mentioned the potential to apply a machine learning algorithm that would help alleviate these issues and further improve the accuracy.

The video already shows some pretty cool applications of the technology – and I’ve no doubt we’ll hear more ideas this week as Digits get shown at UIST and around the web. As is often the case, it’s the result of a a multidisciplinary team that has come together and built a concept in relatively short order using off the shelf components. Imagine what’s possible with more bespoke components and the inevitable miniaturization?

Originally posted on Next at Microsoft. Republished with kind permission.

Image via MSN.



Dubai And The Future Of Humanitarian Design

Design & Architecture
Technology october 21, 2016

Concept Camera Designed To Only Take Unique Photos

Camera Restricta is tool that prompts photographers to only capture one-of-a-kind images

Design & Architecture october 21, 2016

Fragrance Will Release The Smell Of Data If Your Private Information Is Being Leaked

The device is designed to create a physical cue for the potential dangers lurking online


Get PSFK's Related Report: Future of Automotive

See All
Retail october 21, 2016

LYNK & CO Is A New Auto Brand That Promises Mobile Connectivity On Wheels

Online access and mobility sharing are driving the company to disrupt the auto industry

Travel october 21, 2016

Become A Citizen Of The First Nation In Space

Asgardia is a new concept for a floating society above Earth

Entertainment october 21, 2016

Speaker Displays Song Lyrics As Music Is Played

The device is able to generate the graphics on a translucent screen and retrieve the words from a connected database

AI october 21, 2016

Travel Assistant Scans Your Emails To Make Planning Easier

This AI add-on will sync with your inbox and sends reminders to make sure you don't miss anything important


Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry

PSFK Op-Ed october 20, 2016

Wearable Tech Expert: Designing Technology To Empower Connection To Ourselves

Billie Whitehouse, Founder of Wearable Experiments, shares her new vision for the quantified self

PSFK Labs october 21, 2016

PSFK Picks: Top 5 Performance-Enhancing Wearables

Our new report looks at innovations pioneering the future of performance through intelligent activewear and predictive analytics

Millennials october 21, 2016

FOMO Survival Kit Helps Millennials Cope With Social Anxieties

The satirical product is meant to be a playful diversion for people who feel like they are missing out

Food october 21, 2016

New York Restaurant Uses Tomato Sushi As Its Newest Meat Alternative

fresh&co is using sous vide Roma tomatoes to create a vegan option that has the texture and taste of tuna

Advertising october 21, 2016

Red Bull Converts Sao Paulo Payphones Into Data-Driven Bus Schedules

The booths allow city residents to check local transit times through a simple toll-free phone call

Work october 21, 2016

Health Expert: Nutritional Meal Replacements Are A Solution To Corporate Wellness

Ample Foods Founder Connor Young explains why supplements are the next food trend coming to the workplace

Retail october 21, 2016

Why Experiential Events Could Replace Trade Shows

Marketers are seeking creative and impactful new ways to connect with influencers

Children october 21, 2016

Modular Kit Teaches Kids How To Make Their Own Robots

MODI features magnetic modules and a platform for programming to encourage experimentation

Infants october 21, 2016

Work Table Doubles As A Baby Seat

Designer Kunsik Choi created the furniture to facilitate emotional communication between between parents and their children

Technology october 21, 2016

Album Turns Into Something New Each Time It’s Streamed

Bill Baird's new album explores the relationship between time and music through a website crafted by design team, One Pixel Wide

No search results found.