menu

Shapes Translated Into Sounds Help The Blind See

Shapes Translated Into Sounds Help The Blind See
technology

Synaesthesia has a very practical use for one Dutch engineer.

Rachel Pincus
  • 12 march 2014


We often think of sight and sound as two separate sensations. Though we perceive them simultaneously, it seems strange to try to relate these two important features of how we perceive the natural world. But that’s exactly what Dutch engineer Peter Meijer did in 1992 with vOICe, a groundbreaking system that converts shapes and patterns into musical notes and pitches.

This unique adaptation for the blind has now found the perfect home on Android with an additional system called EyeMusic. Though the system is at its most helpful when used with a head-mounted camera, allowing subjects to navigate around a room on sound cues alone, the phone-based system holds its own possibilities.

The visual language of vOICe sounds at first like a garbled mess, but subjects working with neuroscientist Amir Amedi and his colleagues at the Hebrew University of Jerusalem were able to use the system to spot a human figure with about 70 hours of training. The choices in creating the language sometimes seem arbitrary, but it also seems to play on all the shared words we have for musical and visual terms. For example, the pitches of vOICe’s tones get higher as the shapes they describe become higher in an image. Brightness, on the other hand, is signified through volume. EyeMusic elaborates on this system by translating colors into different musical instruments.

The app also has a talking color identifier, compass, face detector and GPS, as well as CamFind visual search and Google Goggles support. Its head-mounted functionality could bring Google Glass to the blind.

With 4500 pixels of resolution, the system has room for nuance and allows subjects to pick up on facial expressions and postures. To the delight of researchers, investigating the world in this way activated the visual cortexes of subjects – a testament to the adaptability of the human brain.

A self-proclaimed “cyborg,” Neil Harbisson was born with a rare condition that enables him to only see the world in black and white. However, an “eyeborg” device helps him “see” colors by translating the colors into sound. By turning the light frequencies into sound frequencies, Harbisson is able to perceive a full spectrum of color.

 

vOICe // EyeMusic

Sources: Current Biology, Science, Daily Mail
Image: Peter Meijer/Google Play

technology
Trending

DIY Kit Lets You Build Your Own Wooden Bike, Boat Or Caravan

Design & Architecture
Culture Yesterday

Messaging Add-On Helps You Correct Your Friends’ Bad Grammar

An iMessage sticker pack will help you copyedit text messages

Automotive Yesterday

Mercedes-Benz Introduces A New Electric Mobility Brand

The separate entity aims to simplify the identification of Mercedes EV products to customers

Trending

Get PSFK's Related Report: Future of Automotive

See All
Mobile Yesterday

Tinder’s New Feature Makes Swiping A Group Effort

The dating app wants to democratize its gestural interaction by buying in to the social polling trend pervasive among millennials

Syndicated Yesterday

Autonomous Garbage Drone Prevents Trash From Reaching Deep Ocean

The solar-powered WasteShark collects refuse closer to the source: the harbor

Automotive Yesterday

Aston Martin Reveals Its Own Luxury Powerboat

The sleek AM37 echoes styling elements from the British brand's sports cars

Advertising Yesterday

An Escort Website Fights Violence Against Sex Workers

The advocacy campaign from McCann aims uncover the human toll of the exploitative industry

PSFK LABS REPORT

Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry
NEW

PSFK Op-Ed september 29, 2016

Digital Design Expert: Mobile First Is Dead, Think Mobile Native

Brian Cooper, chief creative officer of OLIVER Group UK, explains how some brands are still playing catch-up to new technology

PSFK Labs september 29, 2016

The 10 Steps To Discover, Hire, Develop Your Next Leader

PSFK's Future of Work report outlines key steps in the employee development path to empower next-gen leaders

Culture Yesterday

LIFE Magazine Relaunches In Pure VR

The general interest periodical, which ceased publication in 2000, has turned into a portal for virtual reality content

Mobile Yesterday

Reorder This Detox Drink With A Simple Text Message

Dirty Lemon is streamlining its communication by letting customers place orders, ask product questions and request help exclusively through chat

Op-Ed Yesterday

The Future Of The American Workforce Requires Unbundling College Education

President of JetBlue Technology Ventures: developing corporate education programs for non-traditional students

Retail Yesterday

Gilt’s Pop-Up House Is The Kind Of Store You’ll Want To Live In

The New York City townhouse plays host to the latest in retail inspiration, curation, and lifestyle activation (and some libations, too)

PSFK LABS REPORT

Future Of Work
Cultivating The Next Generation Of Leaders
NEW

Mobile Yesterday

Registering To Vote Is Now Just A Text Away

A new bot aims to mobilize underrepresented groups this election season through SMS and Facebook Messenger

Africa Yesterday

Virtual Reality Game Gives Lessons About Emergency Birth Care

LIFE is a serious tool that takes advantage of new technology to help save lives

Luxury Yesterday

Shoe Repair Has Moved Onto Your Phone

Cobbler Concierge is an on-demand service to get your footwear fixed online

No search results found.