Substitutional Reality system offers an affordable way of manipulating participants' perception of reality,
Christopher Nolan’s 2010 blockbuster Inception is set in a distant future where military technology enables one to infiltrate and surreptitiously alter other people’s dreams. Leonardo Di Caprio plays Dom Cobb, an industrial spy tasked with planting an idea into the mind of a powerful businessman. The film has a complex, layered structure: Cobb and the other characters create dreams within dreams within dreams, but they cannot distinguish between reality and the dream states they fabricate.
Most of us distinguish between real and imagined events using unconscious processes to monitor the accuracy of our experiences. But these processes can break down in some psychiatric conditions. Patients with schizophrenia, for example, can experience auditory and visual hallucinations that they believe are real, while some brain damaged and delusional patients live in a world of perpetual false memories. Japanese researchers have developed an “Inception helmet” that manipulates reality to simulate such experiences, and could be used to study cognitive dysfunction in psychiatric disorders.
The Substitutional Reality (SR) system, developed by researchers at the RIKEN Brain Science Institute’s Laboratory for Adaptive Intelligence, is made of cheap, commercially available electronic components: a panoramic video camera used for recording, a computer for storing the recorded footage, and a head-mounted visual display that can switch seamlessly between the recorded footage and a live feed captured by a camera and microphone attached to it.
“In a dream, we naturally accept what is happening and hardly doubt its reality, however unrealistic it may seem on reflection.” says Keisuke Suzuki, the lead author of a recent paper describing the SR system. “Our motivation is to explore the cognitive mechanisms underlying our strong conviction in reality. How can people trust what they perceive? Answering these questions requires an experimental platform which can present scenes that participants believe are completely real, but where we are still able to manipulate the contents.”
To test the system, Suzuki and his colleagues designed a simple, yet ingenious, experiment. They recruited a group of participants, and then filmed each one as they entered a room and received instructions from one of the researchers. In turn, each participant was asked to sit in a chair in the same room, and don the head-mounted display, which then played a sequence of live and recorded scenes, and substituted one for the other without the participants’ knowledge.
The first scene was a ‘fake live’ scene – a recording of one of the researchers appearing at the door and asking if the participant felt comfortable wearing the device, and to test it by looking around the room. Next came a ‘doppelgänger’ scene, in which the participants saw the recording of themselves receiving instructions from the researcher. This was followed by a second fake live scene, in which the experimenter re-enters the room and explains how the experiment was designed. The final scene was a live feed of the researcher re-entering the room, to reveal that all the previous scenes had actually been recordings.
The doppelgänger scene contradicted reality and made the participants realize immediately that they were experiencing a recording instead of a live feed. Most of them failed, however to distinguish between the live and recorded scenes during the rest of the experiment, even after the doppelgänger scene revealed how the system works. Several noticed a small difference between the audio quality of the live and recorded scenes, and used this to establish when the switch between the two had been made, but the rest subjectively experienced the ‘fake live’ scenes as being real.
The researchers then examined several factors that might affect the performance of the system. They found that head movements reduced the likelihood that participants would detect the switch between live and recorded scenes, and that the ability to detect the switch decreased with faster head movements. In the first experiment, the researchers switched between live and recorded scenes while the participants moved their heads to look around the room, and this effectively masked the “visual slip” that occurred during the switch.
Another factor is motion parallax, a depth cue associated with movement. As we move, nearby objects seem to move faster than objects that are further away, and objects also appear to change shape with changes in head position. These cues are present in the live feed but are missing from the recorded scenes, and could potentially be used to distinguish between live and recorded scenes. The researchers tested this by asking the participants to determine whether they were experiencing a live or recorded scene by monitoring the displacement of a chair placed at varying distances from them, and found that it had no significant effect.
The SR system thus offers an affordable way of manipulating participants’ perception of reality, and could serve as a useful tool for investigating reality monitoring in psychiatric conditions.
“Psychiatric patients sometimes have delusive beliefs, as if they are in an alternative reality, and schizophrenics may also experience perceptual hallucinations – literally seeing things that are not there,” says Suzuki. “SR provides a unique opportunity to model these experiences in healthy subjects, which could be useful for investigating the cognitive mechanisms underlying hallucinations and delusions.”
“We’ve already explored the so-called ‘doppelgänger’ delusion, in which participants suddenly see themselves enter the room,” he adds. “There’s much more that can be done. For example, we can use the system to manipulate the matches between expected and actual sensory inputs in highly realistic environments, probing one current theory of schizophrenia. This might allow us to regenerate schizophrenic symptoms in a controlled fashion, perhaps providing avenues for therapy. Of course, all these potential applications require a very careful consideration of the relevant ethics.”
Suzuki, who is now at the Sackler Centre for Consciousness Science at the University of Sussex, is developing an enhanced SR system that he and his colleagues plan to use for several different projects. “A primary focus will be on the psychiatric applications, but the system could also be a powerful tool to investigate how our conscious experiences are constituted in daily natural scenes,” he says.
“It will also open a new direction in cybertherapy. Virtual reality technologies effectively treat post-traumatic stress disorder and phobias by repeatedly exposing patients to traumatic episodes in immersive devices. The SR system provides the conviction of being in the ‘real’ world, which is absent in current VR technologies.”
Reference: Suzuki, K., et al. (2012). Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality. Scientific Reports, 2: 459. DOI: 10.1038/srep00459