Extending the Krakencoder: Adding connectivity types and phenotypes to our connectome fusion model
Poster No:
1245
Submission Type:
Abstract Submission
Authors:
Keith Jamison1, Mert Sabuncu2, Amy Kuceyeski3
Institutions:
1Weill Cornell Medicine, New York, NY, 2Cornell Tech, Weill Cornell Medicine, New York, NY, 3Cornell, Ithaca, NY
First Author:
Co-Author(s):
Introduction:
Brain connectivity can be estimated in many ways, using different modalities and processing strategies. Previously, we presented the Krakencoder, a joint connectome mapping tool that bidirectionally translates between structural (SC) and functional connectivity (FC), and across different atlases and processing choices via a common latent representation (Jamison 2024). We showed that these predictions preserved inter-individual connectome variability, and that the latent space can predict genetic and phenotypic similarity, even on out-of-sample datasets. Here, we demonstrate how this model can be easily expanded to new connectivity types, and can be trained to better capture phenotypic features of interest.
Methods:
The original model included 15 connectivity types: 9 FC and 6 SC, using 3 parcellations. For each connectivity type, a linear encoder and decoder transforms a connectome to a 128-dimensional latent vector and back. To add a new type, we transform data for the original types to latent space, and average them a single "fusion" vector per training subject. For each new connectivity type (e.g. a new parcellation), we take the new connectivity data for the same subjects and train a new encoder/decoder to map this data to the pre-computed "fusion" vectors (See Fig 1a). Once trained, new data can be translated between this connectivity type and all others. This training can be done quickly (1-2 min per connectivity type, without GPU). Using this approach, we expanded the scope of the Krakencoder from 15 connectivity types to over 250, including additional filtering strategies for FC, new tractography estimates, and new parcellations.
We previously showed that phenotypic features such as age, sex, and cognitive performance can be predicted from the latent vectors from the Krakencoder model (Jamison 2024). Here, we explore refining this relationship by shaping the latent space to predict these features during training. A linear decoder predicts the phenotypic feature from the latent vector, and prediction error is added to the reconstruction error that is back-propagated through each connectivity encoder.
We previously showed that phenotypic features such as age, sex, and cognitive performance can be predicted from the latent vectors from the Krakencoder model (Jamison 2024). Here, we explore refining this relationship by shaping the latent space to predict these features during training. A linear decoder predicts the phenotypic feature from the latent vector, and prediction error is added to the reconstruction error that is back-propagated through each connectivity encoder.
Results:
Fig 1b shows the prediction identifiability (rank percentile) for each connectivity type in the new model. Fig 1c shows the median prediction performance to and from that type, separately for within and between modality. Within-modality, median identifiability averaged 0.99 (chance = 0.5), while between-modality (SC↔FC), median identifiability averaged 0.84.
To explore phenotypic predictability, we initially focus on subject age. A cross-validated grid search identified the optimal loss weight for age prediction error. Fig 2a shows that the correspondence between inter-subject similarity in latent space and age difference increased by over 100% (Spearman r increased from 0.13 from 0.27). This in turn leads to a 10% increase in age prediction accuracy (Pearson r increased from 0.57 to 0.63 in held-out subjects). Importantly, we achieve this enhanced age prediction without negatively affecting connectome reconstruction accuracy or the predictability of other phenotypic features such as cognition.
To explore phenotypic predictability, we initially focus on subject age. A cross-validated grid search identified the optimal loss weight for age prediction error. Fig 2a shows that the correspondence between inter-subject similarity in latent space and age difference increased by over 100% (Spearman r increased from 0.13 from 0.27). This in turn leads to a 10% increase in age prediction accuracy (Pearson r increased from 0.57 to 0.63 in held-out subjects). Importantly, we achieve this enhanced age prediction without negatively affecting connectome reconstruction accuracy or the predictability of other phenotypic features such as cognition.
Conclusions:
We previously demonstrated that the Krakencoder model can translate between different connectivity estimates, such as between SC and FC, and that the latent space is a useful low-dimensional representation for phenotypic prediction. To increase applicability, we have expanded the connectivity flavors for which it can be used, and we demonstrate an approach by which other users can continue to expand the model for their own needs, including on new datasets. Further, we have shown that the existing latent space can be enhanced by incorporating phenotypic information during training.
The pre-trained model can be installed and run locally or in a cloud-based environment like Google Colab, a GUI-based example of which is provided with our code (github.com/kjamison/krakencoder).
The pre-trained model can be installed and run locally or in a cloud-based environment like Google Colab, a GUI-based example of which is provided with our code (github.com/kjamison/krakencoder).
Modeling and Analysis Methods:
Connectivity (eg. functional, effective, structural) 1
fMRI Connectivity and Network Modeling
Methods Development 2
Neuroinformatics and Data Sharing:
Brain Atlases
Keywords:
Machine Learning
Other - Connectomics
1|2Indicates the priority used for review