Shapes Translated Into Sounds Help The Blind See

Shapes Translated Into Sounds Help The Blind See

Synaesthesia has a very practical use for one Dutch engineer.

Rachel Pincus
  • 12 march 2014

We often think of sight and sound as two separate sensations. Though we perceive them simultaneously, it seems strange to try to relate these two important features of how we perceive the natural world. But that’s exactly what Dutch engineer Peter Meijer did in 1992 with vOICe, a groundbreaking system that converts shapes and patterns into musical notes and pitches.

This unique adaptation for the blind has now found the perfect home on Android with an additional system called EyeMusic. Though the system is at its most helpful when used with a head-mounted camera, allowing subjects to navigate around a room on sound cues alone, the phone-based system holds its own possibilities.

The visual language of vOICe sounds at first like a garbled mess, but subjects working with neuroscientist Amir Amedi and his colleagues at the Hebrew University of Jerusalem were able to use the system to spot a human figure with about 70 hours of training. The choices in creating the language sometimes seem arbitrary, but it also seems to play on all the shared words we have for musical and visual terms. For example, the pitches of vOICe’s tones get higher as the shapes they describe become higher in an image. Brightness, on the other hand, is signified through volume. EyeMusic elaborates on this system by translating colors into different musical instruments.

The app also has a talking color identifier, compass, face detector and GPS, as well as CamFind visual search and Google Goggles support. Its head-mounted functionality could bring Google Glass to the blind.

With 4500 pixels of resolution, the system has room for nuance and allows subjects to pick up on facial expressions and postures. To the delight of researchers, investigating the world in this way activated the visual cortexes of subjects – a testament to the adaptability of the human brain.

A self-proclaimed “cyborg,” Neil Harbisson was born with a rare condition that enables him to only see the world in black and white. However, an “eyeborg” device helps him “see” colors by translating the colors into sound. By turning the light frequencies into sound frequencies, Harbisson is able to perceive a full spectrum of color.


vOICe // EyeMusic

Sources: Current Biology, Science, Daily Mail
Image: Peter Meijer/Google Play


Transformable Table Maximizes Utility In Small Spaces

Design & Architecture
Syndicated Yesterday

How Fashion Became All About Fonts

Why a new wave of designs have put typography front and center on your clothing

Retail Yesterday

GM And Audi Are Experimenting With Car Rentals In SF

The automakers are trying something new to capture the segment of urban dwellers who don't want to own a car


Get PSFK's Related Report: Sports Debrief

See All
Travel Yesterday

This Vertical Forest Hotel Will Improve The Air Around It

An architectural firm is creating a lush mountain lodging in China with so much greenery that it will actually clean the atmosphere

Europe Yesterday

This Company Believes Insects Can Provide Biofuel To Power Our Future

A Polish group is testing out a new factory concept that can provide a source of renewable energy, and even serve space missions

Related Expert

Dennis Crowley

Co-Founder of Foursquare

Technology Yesterday

Music Venue Caters To Virtual Reality Events

Boiler Room has launched the first VR hub for content creators to capture special footage for the masses

Gaming & Play Yesterday

Automated Chessboard Lets You Play Against Anyone In The World

Square Off is an AI-powered board that can move the pieces on its own

Technology Yesterday

Contraband Recorder Helps Those In Need Capture Captive Conversations

Designer Marianna Mezhibovkaya created the device to help prison inmates capture crucial evidence of abuse


Future Of Automotive
Scenarios Driving The Digital Transformation Of An Industry

PSFK Op-Ed Yesterday

Health Expert: Nutritional Meal Replacements Are A Solution To Corporate Wellness

Ample Foods Founder Connor Young explains why supplements are the next food trend coming to the workplace

PSFK Labs october 25, 2016

The Keys For Exceptional Performance On And Off The Field

PSFK Labs' new report highlights five important insights for businesses to perform better than the competition

Advertising Yesterday

A Compelling Brand Purpose Is Never Born In The Boardroom

Strategist and Designer Anna-Rae Morris shares why she believes all brands exist for a reason, and how Snapchat has upended our behavior

Retail Yesterday

Creative Leadership Expert: Experiencing A Seismic Shift From Brand Loyalty To Interface Loyalty

Marc Shillum, founder of Chief Creative Office, explains why product designers must rethink the way they capture consumer attention

Fitness & Sport Yesterday

Editorial Roundtable: The Xs And Os Of Performance-Enhanced Sports

WHOOP, ShotTracker, Rithmio, PlaySight, STYR Labs, EverybodyFights and Lift / Next Level Floats explain that we're only at the iceberg's tip of performance enhancement

Design & Architecture Yesterday

Kodak’s Smartphone Is Inspired By A Retro Camera

Ektra is an Android phone with a powerful lens aimed at photographers

Sustainability Yesterday

Modern Home Created Entirely Out Of Modular Shipping Crates

A project in Ecuador used the ubiquitous material to design a spacious and industrial-chic house

Retail Yesterday

Retail Concept Offers Food Hall Inspired Space For Online Brands

The store will offer a lower cost way for internet entrepreneurs to try physical retailing

Advertising Yesterday

Disney Retells Beauty And The Beast On Snapchat

In a partnership with LACMA, the entertainment platform drew from over 130,000 works in the museum's collection

No search results found.