Deep Learning in Neuroimaging

Understanding the intricate organization of the human brain, both structurally and functionally, is a pivotal pursuit in neuroscience. In recent years, graph neural networks (GNNs) have emerged as powerful tools for modeling complex relationships in various domains, including neuroscience. This tutorial aims to provide a comprehensive overview of the fundamental concepts of GNNs and their application in the analysis of brain structural and functional connectivity.
The tutorial will commence with a thorough exploration of the basics of graph neural networks, elucidating their foundational principles and mechanisms. Participants will gain insights into how GNNs operate in capturing intricate patterns within networked data. Building upon this foundational knowledge, the tutorial will then delve into our recent advancements in the development of sophisticated graph neural networks tailored for the analysis of brain connectivity. Our cutting-edge graph neural networks facilitate the examination of connectivity at multiple levels of granularity within the brain. Participants will be introduced to techniques enabling the analysis of connectivity between individual brain regions, the exploration of structural and functional connectivity, and the examination of intricate relationships among clusters of brain regions.
 

Artificial intelligence (AI) and deep learning methods are revolutionizing neuroscience, radiology and medicine, bringing powerful new approaches to analyze brain images, text, and clinical data. As the vast landscape of activity in AI makes it hard to keep up with all the key developments, we offer a guided tour for neuroimagers - summarizing several major lines of work applying AI methods to neuroimaging and population-based brain mapping datasets. The lecture will help neuroimagers get up to speed with AI methods in neuroimaging, covering the main concepts and applications. We begin by explaining 2D and 3D convolutional neural networks (CNNs), Vision Transformers (ViTs), and variational autoencoders (VAEs), which can be adapted to distill useful information from large datasets of anatomical, diffusion MRI, and functional MRI. We explain AI methods for common neuroimaging tasks including image registration, anatomical segmentation, diagnostic classification, prognostic modeling, and disease subtyping with illustrative examples from Alzheimer’s disease, Parkinson’s disease, schizophrenia, bipolar disorder, PTSD and autism. We cover (1) fine-tuning of foundation models to neuroimaging data, (2) multimodal fusion methods, such as multimodal VAEs and normative models, that combine multiple types of brain maps to make inferences (e.g., diagnosis and prognosis). Next we cover generative adversarial networks (GANs), denoising diffusion probabilistic models (DDPM), and neural style transfer, which can enhance or synthesize images (super-resolution, or PET from MRI), and harmonize data across scanners and protocols. Finally, we explain how generative AI models are beginning to create large-scale synthetic brain datasets, with applications to quality control of streamlines in tractometry, and modeling disease effects on the human brain.

*The talk includes collaborative work led by Nikhil Dhinagar, Tamoghna Chattopadhyay and many members of our AI4AD and ENIGMA Consortia.
 

Deep learning (DL) has found success in various domains like image recognition and natural language processing, yet its application in neuroimaging presents unique challenges due to the complexity of data—higher dimensionality, limited sample sizes, and heterogeneous modalities, often lacking a solid ground truth. This course addresses four crucial aspects of DL in neuroimaging. First, we delve into classification/regression, showcasing DL's superiority over standard machine learning for novel clinical and neurobiological insights. Second, we spotlight DL leveraging dynamic functional information, incorporating time series and advanced connectivity approaches. Third, we explore how DL can capitalize on complementary information from different neuroimaging modalities, enhancing prediction accuracy through cross-modality-based representations. Lastly, we examine model visualization and interpretation techniques, crucial for biomarker discovery and subtype identification. Each section summarizes research examples, outlines future directions, and discusses challenges, aiming to propel neuroimaging towards refined and personalized diagnoses and treatments through the powerful lens of DL. 

We developed a robust brain MRI-based Alzheimer’s disease (AD) diagnostic classifier using deep learning/transfer learning on an extensive dataset from over 217 sites/scanners (85,721 scans, 50,876 participants, January 2017 to August 2021). Employing the Inception-ResNet-V2 network as a base model for sex classification yielded 94.9% accuracy. Transfer learning for AD diagnosis achieved 90.9% accuracy in cross-validation on ADNI (6,857 samples) and 94.5%/93.6%/91.1% accuracy on three independent datasets (AIBL, MIRIAD, OASIS). Testing on unseen mild cognitive impairment (MCI) patients revealed 65.2% accuracy in predicting MCI to AD conversion, outperforming non-converters (20.6%). Predicted scores correlated significantly with illness severity. Our AD classifier, exhibiting high accuracy and potential for clinical integration, represents a significant advancement in neuroimaging diagnostics. 

Advances in deep learning have had transformative impacts across various scientific fields, and neuroimaging is no exception. With an ever-increasing volume of structural and functional brain imaging data, the application of deep learning techniques offers a powerful means to unlock deeper insights into brain architecture and function. As the field evolves, understanding deep learning approaches is crucial for neuroscientists, radiologists, and data scientists alike, allowing both the advancement of research and the improvement of clinical outcomes.

This tutorial scrutinizes selected approaches for semantic segmentation from structural MRI and mental disorder classification from fMRI. We will delve into distinct model architectures, their variances, training methodologies, and the appropriate application of each. For both semantic segmentation from structural data and functional data classification, we will examine the prerequisites and modes of supervised training, alongside the unique aspects of self-supervised training for static and dynamic scenarios. Additionally, we will exhibit a model for self-supervised learning in coordinated fusion. The session will culminate with the display of interpretability methods for the given tasks.
 

In this talk, we discuss the distinctions between statistical and machine learning approaches to data analysis, and the specific challenges deep learning presents. After reviewing a prediction-focused vs. inference-focused orientation to modelling, we provide a selective review of the contributions deep learning has made to neuroimaging. We then will discuss particular limitations of deep learning, including interpretability and bias. We will conclude with potential solutions to address these limitations.