menu

Shapes Translated Into Sounds Help The Blind See

Shapes Translated Into Sounds Help The Blind See
technology

Synaesthesia has a very practical use for one Dutch engineer.

Rachel Pincus
  • 12 march 2014


We often think of sight and sound as two separate sensations. Though we perceive them simultaneously, it seems strange to try to relate these two important features of how we perceive the natural world. But that’s exactly what Dutch engineer Peter Meijer did in 1992 with vOICe, a groundbreaking system that converts shapes and patterns into musical notes and pitches.

This unique adaptation for the blind has now found the perfect home on Android with an additional system called EyeMusic. Though the system is at its most helpful when used with a head-mounted camera, allowing subjects to navigate around a room on sound cues alone, the phone-based system holds its own possibilities.

The visual language of vOICe sounds at first like a garbled mess, but subjects working with neuroscientist Amir Amedi and his colleagues at the Hebrew University of Jerusalem were able to use the system to spot a human figure with about 70 hours of training. The choices in creating the language sometimes seem arbitrary, but it also seems to play on all the shared words we have for musical and visual terms. For example, the pitches of vOICe’s tones get higher as the shapes they describe become higher in an image. Brightness, on the other hand, is signified through volume. EyeMusic elaborates on this system by translating colors into different musical instruments.

The app also has a talking color identifier, compass, face detector and GPS, as well as CamFind visual search and Google Goggles support. Its head-mounted functionality could bring Google Glass to the blind.

With 4500 pixels of resolution, the system has room for nuance and allows subjects to pick up on facial expressions and postures. To the delight of researchers, investigating the world in this way activated the visual cortexes of subjects – a testament to the adaptability of the human brain.

A self-proclaimed “cyborg,” Neil Harbisson was born with a rare condition that enables him to only see the world in black and white. However, an “eyeborg” device helps him “see” colors by translating the colors into sound. By turning the light frequencies into sound frequencies, Harbisson is able to perceive a full spectrum of color.

 

vOICe // EyeMusic

Sources: Current Biology, Science, Daily Mail
Image: Peter Meijer/Google Play

technology
Trending

Japanese Face Wash Creates A Perfect Rose Every Time

Arts & Culture
Mobile Yesterday

Get A Better Idea Of How You Are Wasting Your Time

The TouchTime app is trying to revolutionize personal task management by providing detailed insight on how to be more efficient

Culture Yesterday

London Telephone Box Repurposed As A Tiny Mobile Repair Shop

Tools and supplies to replace broken screens or damage are neatly stowed away in these micro-workrooms

Trending

Get PSFK's Latest Report: Future of Retail: Technology Primer

See All
Design Yesterday

Conceptual Sportswear Created Out Of Futuristic Condom Material

A Dutch fashion designer is experimenting with new methods and fabrics to make high performance clothing

Fashion Yesterday

Fashionable Tassel Will Ensure You Never Lose Your Valuables Again

The device is fashion meets connected tech, that will help you keep track of your belongings at all times

Syndicated Yesterday

Would You Wear Wool Shoes To Save The Environment?

As demand for wool shoes grows, a number of US footwear brands are heading directly to the source: the sheep pastures of New Zealand

Sustainability Yesterday

Self-Healing Material Is Fashioned Out Of Squid Teeth

Penn State researchers have devised a new textile that uses organic proteins

Arts & Culture Yesterday

Search Engine Turns Your Own Drawings Into Photos

This image-matching software accepts hand-made sketches instead of keywords

PSFK LABS REPORT

Future Of Work
Cultivating The Next Generation Of Leaders
NEW

PSFK Op-Ed august 23, 2016

Modern Workplace Culture: No More Fat Cats Or Kissing Ass

Samar Birwadker, CEO & Co-Founder of Good & Co, on designing shared organizational values to optimize employee happiness and success

PSFK Labs august 25, 2016

PSFK’s Workplace Vision: How The Nurturing Of Seeds Will Come To Define The Onboarding Process

Our Future of Work vision is a service that allows companies to assemble and deliver welcome packets that are uniquely focused on the concept of growth

Arts & Culture Yesterday

Illustrator Interprets The Experiences Of Blind Travelers

Artist Alby Letoy creates drawings of poignant travel memories for the visually impaired

Advertising Yesterday

Clickbait Titles Used For The Good Of Charity

An agency devised an unlikely campaign that uses clickbait as a positive force to drive awareness to nonprofit initiatives

Advertising Yesterday

The Best In Eye-Catching Olympics Campaigns

PSFK rounds out the Rio Games with our picks for the best advertising moments off the field

Work Yesterday

Editorial Roundtable: The Arrival Of The People-First Workplace

Managed By Q, Soma, Workbar, Primary and thinkPARALLAX enumerate the reasons why companies need an employee-embracing workforce in order to exist

Arts & Culture Yesterday

Transforming Light Waves Into A New Art Form

An artist uses glass treated with layers of metallic coatings to create a unique installation called lightpaintings

INSIGHTS COVERAGE

Rio Olympics
Innovation Coverage From The Rio Games
READ NOW

Design Yesterday

This Windbreaker Lets You Explore The Outdoors While Charging Your Phone

The apparel includes solar panels that allow the wearer to stay connected through the power of renewable energy

Asia Yesterday

The Goal Of This Game Is To Not Get Laid Off From Your Job

A hit mobile app has you working really, really hard to not get fired as you climb the corporate ladder

Advertising Yesterday

Movie Critic Bot Guides Viewers Through Festival Offerings

The Toronto International Film Festival has created a Facebook Messenger chatbot to help attendants curate their schedule

No search results found.