Using augmented reality, this engineer's nifty tool translates the names of objects with a simple point and tap

Forgot what the word for “fan” is in Chinese? No need to search an online dictionary or fumble with a translator—one engineer has developed a cool AR concept that instantly translates the names of everyday objects into several different languages.

The app is the work of front-end designer Frances Ng, who uploaded a short preview of her work on Twitter. It shows that by simply pointing a phone at an object and tapping, a user is able to receive a translation for that object in a number of different languages, with the word displayed right over top of the object itself. While the results aren’t perfect (the Korean translation for “jeans” is incorrect, for instance), it shows great potential for how AR can be used in the future.

Even more impressively, Ng developed the app in a single weekend using Apple’s ARKit and a database of about 1,000 learned objects. The tool would be ideal not only for language learners, but for travelers who find themselves in unfamiliar situations. For now though, Ng says the app is just a demo and has no immediate plans to release it.

Frances Ng

Forgot what the word for “fan” is in Chinese? No need to search an online dictionary or fumble with a translator—one engineer has developed a cool AR concept that instantly translates the names of everyday objects into several different languages.

The app is the work of front-end designer Frances Ng, who uploaded a short preview of her work on Twitter. It shows that by simply pointing a phone at an object and tapping, a user is able to receive a translation for that object in a number of different languages, with the word displayed right over top of the object itself. While the results aren’t perfect (the Korean translation for “jeans” is incorrect, for instance), it shows great potential for how AR can be used in the future.