Deep Learning in Neuroimaging

Anqi Qiu Organizer
the Hong Kong Polytechnic University
Hong Kong
Hong Kong
Jing Sui Co Organizer
Beijing Normal University
State Key Laboratory of Cognitive Neuroscience and Learning,
Paul Thompson Co Organizer
Marina Del Rey, CA 
United States
Chaogan Yan Co Organizer
Chinese Academy of Sciences
Sunday, Jun 23: 9:00 AM - 5:30 PM
Educational Course - Full Day (8 hours) 
Room: Conference Room E 5 
Our educational course will delve into the pivotal principles of deep learning in neuroimage analysis, covering a spectrum from fundamental concepts to advanced applications. Given the increasing integration of deep learning in neuroscience research, this topic is not only timely but also essential for researchers and neuroimagers seeking to stay at the forefront of innovative methodologies. The morning and early afternoon sessions will serve as a concentrated exploration of these core principles, emphasizing their relevance and contemporary significance.
In the afternoon, we have allotted approximately 2 hours for immersive, hands-on practical exercises. This segment is strategically designed to empower participants with tangible experience in the application of deep learning techniques to various neuroimaging domains, including brain images, as well as structural and functional networks. By actively engaging in these exercises, attendees will not only deepen their theoretical understanding but also acquire practical skills that are directly applicable to their research endeavors. As the landscape of neuroimage analysis continues to evolve, the ability to adeptly apply deep learning methodologies becomes increasingly crucial, making this course an invaluable investment in participants' skill development and research proficiency.

9:00 – 9:10: Introduction (Speaker: Anqi Qiu)
9:10 – 9:55: A Guided Tour of AI in Neuroimaging (Speaker: Paul M. Thompson)
9:55 – 10:40: Deep Learning in Neuroimaging: Approaches, Applications and Challenges in Mental Disorders (Speaker: Jing Sui)

10:40 – 11:00: Break

11:00 – 11:45: Supervised and Self-supervised Deep Learning for Structural and Functional Neuroimaging (Speaker: Sergey Plis)
11:45 – 12:30: A Practical Alzheimer Disease Classifier via Brain Imaging-Based Deep Learning on 85,721 Samples (Speaker: Chaogan Yan)

12:30 – 13:30 Lunch break

13:30 – 14:15: Advanced Graph Neural Networks for Structural and Functional Brain Networks (Speaker: Anqi Qiu)
14:15 – 15:00: Deep learning and statistics in neuroimaging research (Speaker: Thomas Nichols)

15:00 – 15:30: Break

15:30 – 16:30: Software Demonstration
16:30-17:30: Hands-on


1. To acquire a comprehensive understanding of the fundamental principles of deep learning in neuroimage analysis.
2. To explore the applications of deep learning techniques, including an examination of their advantages and disadvantages in the context of neuroimaging.
3. To gain practical, hands-on experience with the implementation of deep learning methods in neuroimage analysis through interactive sessions and exercises.

Target Audience

Neuroimagers or researchers possessing a foundational understanding of machine learning are the target audience for this educational course. The practical session is specifically tailored for individuals with basic Python programming skills. 


1. Advanced Graph Neural Networks for Structural and Functional Brain Networks

Understanding the intricate organization of the human brain, both structurally and functionally, is a pivotal pursuit in neuroscience. In recent years, graph neural networks (GNNs) have emerged as powerful tools for modeling complex relationships in various domains, including neuroscience. This tutorial aims to provide a comprehensive overview of the fundamental concepts of GNNs and their application in the analysis of brain structural and functional connectivity.
The tutorial will commence with a thorough exploration of the basics of graph neural networks, elucidating their foundational principles and mechanisms. Participants will gain insights into how GNNs operate in capturing intricate patterns within networked data. Building upon this foundational knowledge, the tutorial will then delve into our recent advancements in the development of sophisticated graph neural networks tailored for the analysis of brain connectivity. Our cutting-edge graph neural networks facilitate the examination of connectivity at multiple levels of granularity within the brain. Participants will be introduced to techniques enabling the analysis of connectivity between individual brain regions, the exploration of structural and functional connectivity, and the examination of intricate relationships among clusters of brain regions.


Anqi Qiu, the Hong Kong Polytechnic University Hong Kong
Hong Kong

2. A Guided Tour of AI in Neuroimaging

Artificial intelligence (AI) and deep learning methods are revolutionizing neuroscience, radiology and medicine, bringing powerful new approaches to analyze brain images, text, and clinical data. As the vast landscape of activity in AI makes it hard to keep up with all the key developments, we offer a guided tour for neuroimagers - summarizing several major lines of work applying AI methods to neuroimaging and population-based brain mapping datasets. The lecture will help neuroimagers get up to speed with AI methods in neuroimaging, covering the main concepts and applications. We begin by explaining 2D and 3D convolutional neural networks (CNNs), Vision Transformers (ViTs), and variational autoencoders (VAEs), which can be adapted to distill useful information from large datasets of anatomical, diffusion MRI, and functional MRI. We explain AI methods for common neuroimaging tasks including image registration, anatomical segmentation, diagnostic classification, prognostic modeling, and disease subtyping with illustrative examples from Alzheimer’s disease, Parkinson’s disease, schizophrenia, bipolar disorder, PTSD and autism. We cover (1) fine-tuning of foundation models to neuroimaging data, (2) multimodal fusion methods, such as multimodal VAEs and normative models, that combine multiple types of brain maps to make inferences (e.g., diagnosis and prognosis). Next we cover generative adversarial networks (GANs), denoising diffusion probabilistic models (DDPM), and neural style transfer, which can enhance or synthesize images (super-resolution, or PET from MRI), and harmonize data across scanners and protocols. Finally, we explain how generative AI models are beginning to create large-scale synthetic brain datasets, with applications to quality control of streamlines in tractometry, and modeling disease effects on the human brain.

*The talk includes collaborative work led by Nikhil Dhinagar, Tamoghna Chattopadhyay and many members of our AI4AD and ENIGMA Consortia.


Paul Thompson, USC Marina Del Rey, CA 
United States

3. Deep Learning in Neuroimaging: Approaches, Applications and Challenges in Mental Disorders

Deep learning (DL) has found success in various domains like image recognition and natural language processing, yet its application in neuroimaging presents unique challenges due to the complexity of data—higher dimensionality, limited sample sizes, and heterogeneous modalities, often lacking a solid ground truth. This course addresses four crucial aspects of DL in neuroimaging. First, we delve into classification/regression, showcasing DL's superiority over standard machine learning for novel clinical and neurobiological insights. Second, we spotlight DL leveraging dynamic functional information, incorporating time series and advanced connectivity approaches. Third, we explore how DL can capitalize on complementary information from different neuroimaging modalities, enhancing prediction accuracy through cross-modality-based representations. Lastly, we examine model visualization and interpretation techniques, crucial for biomarker discovery and subtype identification. Each section summarizes research examples, outlines future directions, and discusses challenges, aiming to propel neuroimaging towards refined and personalized diagnoses and treatments through the powerful lens of DL. 


Jing Sui, Beijing Normal University
State Key Laboratory of Cognitive Neuroscience and Learning,

4. A Practical Alzheimer Disease Classifier via Brain Imaging-Based Deep Learning on 85,721 Samples

We developed a robust brain MRI-based Alzheimer’s disease (AD) diagnostic classifier using deep learning/transfer learning on an extensive dataset from over 217 sites/scanners (85,721 scans, 50,876 participants, January 2017 to August 2021). Employing the Inception-ResNet-V2 network as a base model for sex classification yielded 94.9% accuracy. Transfer learning for AD diagnosis achieved 90.9% accuracy in cross-validation on ADNI (6,857 samples) and 94.5%/93.6%/91.1% accuracy on three independent datasets (AIBL, MIRIAD, OASIS). Testing on unseen mild cognitive impairment (MCI) patients revealed 65.2% accuracy in predicting MCI to AD conversion, outperforming non-converters (20.6%). Predicted scores correlated significantly with illness severity. Our AD classifier, exhibiting high accuracy and potential for clinical integration, represents a significant advancement in neuroimaging diagnostics. 


Chaogan Yan, Chinese Academy of Sciences Beijing

5. Supervised and Self-supervised Deep Learning for Structural and Functional Neuroimaging

Advances in deep learning have had transformative impacts across various scientific fields, and neuroimaging is no exception. With an ever-increasing volume of structural and functional brain imaging data, the application of deep learning techniques offers a powerful means to unlock deeper insights into brain architecture and function. As the field evolves, understanding deep learning approaches is crucial for neuroscientists, radiologists, and data scientists alike, allowing both the advancement of research and the improvement of clinical outcomes.

This tutorial scrutinizes selected approaches for semantic segmentation from structural MRI and mental disorder classification from fMRI. We will delve into distinct model architectures, their variances, training methodologies, and the appropriate application of each. For both semantic segmentation from structural data and functional data classification, we will examine the prerequisites and modes of supervised training, alongside the unique aspects of self-supervised training for static and dynamic scenarios. Additionally, we will exhibit a model for self-supervised learning in coordinated fusion. The session will culminate with the display of interpretability methods for the given tasks.


Sergey Plis, Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) Georgi Atlanta, GA 
United States

6. Deep learning and statistics in neuroimaging research

In this talk, we discuss the distinctions between statistical and machine learning approaches to data analysis, and the specific challenges deep learning presents. After reviewing a prediction-focused vs. inference-focused orientation to modelling, we provide a selective review of the contributions deep learning has made to neuroimaging. We then will discuss particular limitations of deep learning, including interpretability and bias. We will conclude with potential solutions to address these limitations.


Thomas Nichols, University of Oxford Oxford
United Kingdom