Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - BayesianHumanCortex (Bayesian computations in the human neocortex: deciphering the neuronal mechanisms of perceptual and syntactic inferences.)

Teaser

Several lines of research suggest that a wide variety of cognitive processes (e.g. perception, motor control, decision, language) can be decomposed into a common set of elementary computations implemented by the cortex. Our goal consists in 1) identifying these elementary...

Summary

Several lines of research suggest that a wide variety of cognitive processes (e.g. perception, motor control, decision, language) can be decomposed into a common set of elementary computations implemented by the cortex. Our goal consists in 1) identifying these elementary computations, and 2) determining their neural implementations.

This research has three major lines of impacts. First, it contributes to the development of neuroimaging techniques that can be transferred to the clinics. Second, the present research improves our understanding of the elementary mechanisms of the cortex, and can thus be critical to the diagnosis, prognosis and treatment of multiple psychiatric and neural disorders. Finally, the present research aims to isolate and implement the building blocks of high-level cognition. This knowledge is invaluable for the development of artificial intelligence, and may thus widely impact society.

Overall, our research demonstrated that we can track the encoding, selection and maintenance of visual, auditory and linguistic inputs from brain recordings. These results revealed dissociable elementary mechanisms of information encoding, maintenance and retrieval in the human brain. Our research therefore provides a stepping stone to the unravel the computational architecture of human cognition.

Work performed

Two main lines of studies have been implemented. The first relates to the decomposition elementary operations of perceptual inference in the human brain. The second relates to identifying the very same components in linguistic processes.

1. Decoding elementary computations of perceptual processes

To isolate the elementary computations of perceptual processes, we first recorded the brain activity of 20 healthy subjects while they performed a visibility inference task on a stimulus presented at perceptual threshold. We show that a rich amount of sensory, perceptual and decisional information can be decoded and track from MEG activity. The present study has been published in Neuron (King et al 2016), and received the postdoctoral award from the Cognitive Neuroscience Society 2016, and the William James Prize from the Association for the Scientific Study of Consciousness. The data and code is available on https://github.com/king, partially fulfilling “WP2.3 : Datasets made publically available” and “WP1.2 : Peer-reviewed publication MEG+ECoG + Open Access codes of original analytical tools”.

Second, to identify whether the corresponding neural representations were hierarchically organized, we applied similar decoding analyses to participants’ MEG while they discriminated between ambiguous visual symbols. Our results show that a hierarchy of psychological variables can be decoded from brain activity. These computations are largely similar to those of deep neural networks subject to the same task. The present study is currently being written and has been accepted for a dynamic poster presentation at SFN and a poster presentation at Computational Cognitive Neuroscience Conference 2017 for which it received a travel award.

Finally, we investigated how this hierarchy of computations unfolded in streams of images. Fifteen subjects were presented with ~5,000 visual stimuli presented in rapid sequences while being recorded with EEG. Temporal generalization and source analyses reveal that the information contained in each stimulus is processed by a “visual pipeline”: a long cascade of transient processing stages, which can overall encode multiple stimuli at once. The corresponding study is currently under review at Science.

2. Decoding elementary computations of linguistic processes

Like perception, language can be described at many level representations, ranging from acoustic features, to phonemes, syntax and semantics. To isolate each level of representations, we developed new methods to both encode and decode linguistic representations from 24 subjects’ brain activity, while they were listened to stories. Our results show that we can isolate and track the variable maintenance of acoustic, phonetic, lexical, syntactic and semantic computations. The results suggest a predominantly hierarchical organization of elementary neural computations.


Overall, the above demonstrate that theoretically-derived computations can be isolated, and tracked in the human brain activity. Our results converge on the idea that the same computations apply at multiple level of the perceptual processing hierarchy.


3. Dissemination and exploitation

The above studies have been accepted for presentation in a number of conferences (listed in https://sites.google.com/site/jeanremiking/). This fulfills “WP1.1: Presentation of MEG results at conference” and “WP1.4, Conference Talk” Amongst these events, I have:
* organized and chaired 3 symposia (ICON 2017, ASSC 2017, Biomag 2016),
* been invited for plenary session at 2 conferences (ASSC 2017 ‘William James Prize’, CNS 2017: ‘Big Ideas’)

Additionally, 1) I have participated in open data-science competitions and ranked first and third, 2) I am an official core developer of MNE, an open-source package in python to analyze temporally resolved neuroimaging data and 3) I have been invited to present my research at Facebook Artificial Intelligence Research.

Finally, I have orga

Final results

1. Progress beyond the state of the art.
The present project makes a number of both methodological and theoretical progress. First, the development of machine learning techniques to decode brain activity demonstrate a significant gain in signal-to-noise ratio as compared to conventional analytical pipelines (e.g. King et al 2016). Second, we dissociate for the first time the neural mechanisms of information encoding and information maintenance (King et al 2016, Quentin King et al, BioRxiv 2018). Third, we show that the unfolding of perceptual processing can be tracked in rapid serial visual presentations (King & Wyart submitted). Finally, similar computations and patterns of neuronal activity are being observed in language tasks (King, et al, in prep). Overall, the present project reveals, with an unprecedented level of details, the elementary computations common to perceptual and linguistic processes.

2. Expected potential impact
The present project involves several methodological developments for the analysis of neuroimaging and clinical recordings. Beyond these proximal applications, identifying these elementary computations has important repercussions for public health. Indeed, several psychiatric conditions, such as schizophrenia and autism, cannot be related to the defect a single brain regions, and affect a wide spectrum of cognitive abilities. How can the disruption of a simple but distributed neuronal mechanisms lead to such dramatic psychiatric conditions? Most likely by the malfunctioning of an elementary brain computation. Identifying, and targeting these computations is therefore the one of the most promising direction to provide long-term solutions for psychiatric disorders.

Website & more info

More info: https://sites.google.com/site/jeanremiking/.