(Video) Seeing Through Walls With Augmented Reality

A team at Carnegie Mellon University is developing an augmented reality system that would allow drivers to see through walls. Aimed at helping drivers avoid unseen hazards, the experimental project is being funded by Japanese car parts company Denso.

New Scientist reports:

The prototype uses two cameras: one that captures the driver’s view and a second that sees the scene behind a view-blocking wall. A computer takes the feed from the second camera and layers it on top of the images from the first so that the wall appears to be transparent.

This makes it simple to glance “through” a wall to see what’s going on behind it. But the techniques needed to combine them were challenging to develop, says Yaser Sheikh of Carnegie Mellon University in Pittsburgh, Pennsylvania.
Altered images

The view of the hidden scene needs to be skewed so that it looks as if it were being viewed from the position of the person using the system. The system does this by spotting landmarks seen by both cameras: the one seeing the hidden view and the one with the same view as the user.

Watch a video demonstration below:

New Scientist: “Augmented reality system lets you see through walls”