PREDICTIVENEUROSENS

Neural correlates of predictive mechanisms in multisensory perception

 Partecipanti

Mappa

 Word cloud

Esplora la "nuvola delle parole (Word Cloud) per avere un'idea di massima del progetto.

 Obiettivo del progetto (Objective)

'There are many situations in which what we see (hear) is strongly affected by what we hear (see). For instance, in the classic ventriloquism effect, the perceived location of a sound (ventriloquist's speech) is displaced towards the location of a concomitant visual event (puppet). In spite of the implausibility of the situation, our perception is resilient to our conscious knowledge that the puppet cannot speak. Conversely in noisy environments, seeing the interlocutor's face enhances the intelligibility of what is said. Thus, multisensory integration can improve, impair or create perception. How does the brain integrate information from different senses? Under which circumstances does multisensory integration benefit / impair /create perception? This project aims to clarify the brain mechanisms underlying the automatic integration of multisensory information. The major steps in this project are: (i) Psychophysical paradigms will be designed to specifically address automaticity and predictability in multisensory perception. (ii) Healthy population will be tested with combined magneto- and electro-encephalography (MEG-EEG) in order to functionally determine the neural dynamics of multisensory integration. (iii) Healthy population will be tested with functional magnetic resonance imaging (fMRI) in order to determine the precise anatomical structures underlying automatic multisensory integration. (iv) Patient populations (neglect and schizophrenic populations, in particular) will be tested using the paradigms developed in (i). An emphasis on the benefit of multisensory perception for neuro- and cognitive-rehabilitation will be made in adapting those paradigms. (v) Children will be tested using paradigms developed in (i) in order to comprehend how early in life multisensory interactions take effect and how learning skills such as reading could benefit from multisensory integration.'

Introduzione (Teaser)

In many situations, what people see is affected by what they hear, and vice versa. An EU-funded study explored the effects of multisensory perception.

Descrizione progetto (Article)

The project 'Neural correlates of predictive mechanisms in multisensory perception' (PREDICTIVENEUROSENS) developed a series of experiments to test how people integrate multisensory information. The study was based on the hypothesis that the brain integrates visual and auditory cues to predict future events.

In a former study, this ability was illustrated by an experiment showing that sounds synchronised with a visual target made it easier to detect that target. Other research also provided evidence for multisensory integration and functional flexibility of cortical regions. These previous findings set the stage for the PREDICTIVENEUROSENS study.

Researchers trained groups of participants to discriminate between red or green random-dot-kinematograms. One group was trained with auditory information congruent with the visual cues, one group had just the visual cues, and one group had auditory sounds unrelated to visual cues. Results showed that those trained with the congruent auditory and visual cues performed significantly better than those from the other groups.

Investigators also found that those with the congruent cues learned and retained the information differently. Based on brain activity in post-experimental recall assessments, it became clear that those in the group with congruent information used more integrated neural pathways to perform than those from the other groups. This finding shows that the brain optimises and stores information about a visual task by making use of other areas of the brain, providing further evidence for the brain's capacity to optimize its use use of the full range of sensory information.

A second study was performed with the help of patients with schizophrenia. They were asked to watch desynchronised audiovisual speech and to report what they heard and to judge the synchrony of the sound and the person speaking. Patients displayed an impairment in timing audiovisual inofrmation but were not impaired in their comprehension of audiovisual speech.

These findings suggest that while timing is essential to multisensory processing, our conscious representation of timing entails separate mechanisms. These findings can be used as a basis for further research and development on sensory substitution devices that take advantage of cross-sensory mapping in the brain.