Reconstructing Functional Connectomes and Predicting Cognition using Contrastive Learning
Poster No:
1537
Submission Type:
Abstract Submission
Authors:
Victoria Shevchenko1,2,3,4, Malo Renaudin3,4, Robert Scholz1,2,5,6, R. Austin Benn1,2, Wei Wei1,2, Carla Pallavicini1,7,8, Ulysse Klatzmann1,2, Francesco Alberti1,2, Pierre-Louis Bazin9, Theodore Satterthwaite10, Daniel Margulies1,2, Demian Wassermann3,4
Institutions:
1Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS, Paris, France, 2Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom, 3MIND Team, Inria Saclay, Université Paris-Saclay, Palaiseau, France, 4Neurospin, CEA, Gif-sur-Yvette, France, 5Max Planck School of Cognition, Leipzig, Germany, 6Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany, 7National Scientific and Technical Research Council (CONICET), Buenos Aires, Argentina, 8Institute of Applied and Interdisciplinary Physics and Department of Physics, University of Buenos Aires, Buenos Aires, Argentina, 9Full brain picture Analytics, Leiden, Leiden, 10University of Pennsylvania Perelman School of Medicine, Philadelphia, PA
First Author:
Victoria Shevchenko
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford|MIND Team, Inria Saclay, Université Paris-Saclay|Neurospin, CEA
Paris, France|Oxford, United Kingdom|Palaiseau, France|Gif-sur-Yvette, France
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford|MIND Team, Inria Saclay, Université Paris-Saclay|Neurospin, CEA
Paris, France|Oxford, United Kingdom|Palaiseau, France|Gif-sur-Yvette, France
Co-Author(s):
Malo Renaudin
MIND Team, Inria Saclay, Université Paris-Saclay|Neurospin, CEA
Palaiseau, France|Gif-sur-Yvette, France
MIND Team, Inria Saclay, Université Paris-Saclay|Neurospin, CEA
Palaiseau, France|Gif-sur-Yvette, France
Robert Scholz
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford|Max Planck School of Cognition|Wilhelm Wundt Institute for Psychology, Leipzig University
Paris, France|Oxford, United Kingdom|Leipzig, Germany|Leipzig, Germany
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford|Max Planck School of Cognition|Wilhelm Wundt Institute for Psychology, Leipzig University
Paris, France|Oxford, United Kingdom|Leipzig, Germany|Leipzig, Germany
R. Austin Benn
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Wei Wei
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Carla Pallavicini
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|National Scientific and Technical Research Council (CONICET)|Institute of Applied and Interdisciplinary Physics and Department of Physics, University of Buenos Aires
Paris, France|Buenos Aires, Argentina|Buenos Aires, Argentina
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|National Scientific and Technical Research Council (CONICET)|Institute of Applied and Interdisciplinary Physics and Department of Physics, University of Buenos Aires
Paris, France|Buenos Aires, Argentina|Buenos Aires, Argentina
Ulysse Klatzmann
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Francesco Alberti
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Daniel Margulies
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Cognitive Neuroanatomy Lab, Université Paris Cité, INCC UMR 8002, CNRS|Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, Nuffield Department of Clinical Neurosciences, University of Oxford
Paris, France|Oxford, United Kingdom
Demian Wassermann
MIND Team, Inria Saclay, Université Paris-Saclay|Neurospin, CEA
Palaiseau, France|Gif-sur-Yvette, France
MIND Team, Inria Saclay, Université Paris-Saclay|Neurospin, CEA
Palaiseau, France|Gif-sur-Yvette, France
Introduction:
While functional connectivity (FC) correlates with cognitive measures (Chen et al., 2022; Ooi et al., 2022; Smith et al., 2015), predicting cognition from high-dimensional FC data remains challenging. To address this, we aimed to develop a supervised, generalizable method that amplifies cognition-related variance within a low-dimensional FC representation. It relies on contrastive learning (CL): a promising approach which has excelled in mapping joint text-image embeddings (Radford et al., 2021). CL consists in structuring a latent space in which similar samples cluster while dissimilar samples diverge (Wang & Liu, 2020).
However, existing CL approaches have focused on classification, not on predicting continuous outcomes. To bridge this gap, we employed the novel kernelized supervised contrastive (KSupCon) loss (Barbano et al., 2023), which adapts the supervised contrastive classification loss (Khosla et al., 2020) for regression. KSupCon leverages continuous scores as labels, enabling the creation of cognition-supervised connectivity embeddings (CSCE).
We introduce a supervised CL model trained to (1) reconstruct the original FC matrix from its reduced form, and (2) predict cognitive ability from the resulting CSCE.
However, existing CL approaches have focused on classification, not on predicting continuous outcomes. To bridge this gap, we employed the novel kernelized supervised contrastive (KSupCon) loss (Barbano et al., 2023), which adapts the supervised contrastive classification loss (Khosla et al., 2020) for regression. KSupCon leverages continuous scores as labels, enabling the creation of cognition-supervised connectivity embeddings (CSCE).
We introduce a supervised CL model trained to (1) reconstruct the original FC matrix from its reduced form, and (2) predict cognitive ability from the resulting CSCE.
Methods:
We used resting state fMRI data of 4297 adolescents from the ABCD dataset. Preprocessed data (fMRIPrep derivatives) were obtained from the ABCD community collection (Feczko et al., 2021). We applied the Schaefer parcellation (400 parcels) to the BOLD time series and correlated (Pearson) parcel-wise timeseries to produce FC matrices. As a proxy for cognition (target), we chose the NIH composite total cognitive score.
Our model consisted of a matrix autoencoder (MatAE), CSCE projector and target decoder (Figure 1A-C). We initialized the weights (W) of MatAE layers as the first n = 100 eigenvectors from the singular value decomposition (SVD) of the mean connectome of the train set (D'Souza et al., 2021) (Figure 1A, eq. 1). The output of the bottleneck of MatAE is transformed by the CSCE projector to produce a CSCE (Figure 1B). CSCE was then used for the downstream task of prediction of the cognitive score using a non-linear target decoder (Figure 1C).
We trained and tested the model 20 times for 5 different train sizes resulting in 100 training experiments. We performed this procedure twice: for an internal and external test set. In the internal test scheme, subjects from all scanning sites were present in both train and test sets. The external test scheme consisted of unseen data from three scanning sites not used in training. We report correlations and mean absolute percentage error (MAPE) for matrix reconstruction and the prediction of the cognitive score for both training schemes.
Our model consisted of a matrix autoencoder (MatAE), CSCE projector and target decoder (Figure 1A-C). We initialized the weights (W) of MatAE layers as the first n = 100 eigenvectors from the singular value decomposition (SVD) of the mean connectome of the train set (D'Souza et al., 2021) (Figure 1A, eq. 1). The output of the bottleneck of MatAE is transformed by the CSCE projector to produce a CSCE (Figure 1B). CSCE was then used for the downstream task of prediction of the cognitive score using a non-linear target decoder (Figure 1C).
We trained and tested the model 20 times for 5 different train sizes resulting in 100 training experiments. We performed this procedure twice: for an internal and external test set. In the internal test scheme, subjects from all scanning sites were present in both train and test sets. The external test scheme consisted of unseen data from three scanning sites not used in training. We report correlations and mean absolute percentage error (MAPE) for matrix reconstruction and the prediction of the cognitive score for both training schemes.
·Figure 1
Results:
Model performance improved for MAPE with increasing training size but plateaued around 750-780 individuals (Figure 2A, panels 1, 2). As expected, MAPE stabilized earlier for the internal test set (Figure 2A, panel 1). Correlations between true and predicted cognitive scores remained modest (r ~ 0.2), consistent with (Chen et al., 2022) Figure 2A, panels 3, 4). This trend was observed in both internal and external test schemes.
On a group level, our model achieved good matrix reconstruction performance overall with higher MAPE in the external test set (Figures 2A & B). Individual matrix reconstructions were characterized by good discriminability: reconstructed matrices were significantly more similar to their true counterparts than to each other (mean cos. similarity = 0.83, p → 0; Figure 2D, panel 5).
On a group level, our model achieved good matrix reconstruction performance overall with higher MAPE in the external test set (Figures 2A & B). Individual matrix reconstructions were characterized by good discriminability: reconstructed matrices were significantly more similar to their true counterparts than to each other (mean cos. similarity = 0.83, p → 0; Figure 2D, panel 5).
·Figure 2
Conclusions:
We introduced a CL-based method which distills cognition-associated information from functional connectomes. Our model allowed us to map subjects similar in their FC and cognition close to each other in a low-dimensional embedding, summarizing cognition-specific variance. Overall, work on joint brain-behavior embeddings offers a promising avenue for supervised dimensionality reduction algorithms for brain data.
Higher Cognitive Functions:
Executive Function, Cognitive Control and Decision Making
Modeling and Analysis Methods:
Classification and Predictive Modeling
Connectivity (eg. functional, effective, structural) 2
Methods Development 1
Task-Independent and Resting-State Analysis
Keywords:
Cognition
Computational Neuroscience
FUNCTIONAL MRI
Machine Learning
Other - Deep Learning
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Provide references using APA citation style.
Chen, J. et al. (2022). Shared and unique brain network features predict cognitive, personality, and mental health scores in the ABCD study. Nature Communications, 13(1), 2217.
D’Souza, N. S. et al. (2021). A matrix autoencoder framework to align the functional and structural connectivity manifolds as guided by behavioral phenotypes. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 (pp. 625–636). Springer International Publishing.
Feczko, E. et al. (2021). Adolescent brain cognitive development (ABCD) community MRI collection and utilities. In bioRxiv (p. 2021.07.09.451638). bioRxiv. https://doi.org/10.1101/2021.07.09.451638
Khosla, P. et al. (2020). Supervised Contrastive Learning. In arXiv [cs.LG]. arXiv. http://arxiv.org/abs/2004.11362
Ooi, L. Q. R. et al. (2022). Comparison of individualized behavioral predictions across anatomical, diffusion and functional connectivity MRI. NeuroImage, 263(119636), 119636.
Radford, A. et al. (2021). Learning Transferable Visual Models From Natural Language Supervision. In arXiv [cs.CV]. arXiv. http://arxiv.org/abs/2103.00020
Smith, S. M. et al. (2015). A positive-negative mode of population covariation links brain connectivity, demographics and behavior. Nature Neuroscience, 18(11), 1565–1567.
Wang, F., & Liu, H. (2020). Understanding the behaviour of contrastive loss. In arXiv [cs.LG]. arXiv. https://openaccess.thecvf.com/content/CVPR2021/papers/Wang_Understanding_the_Behaviour_of_Contrastive_Loss_CVPR_2021_paper.pdf