A systematic evaluation and validation of explainable deep learning methods using amyloid-PET classification in dementia as a ground truth

COEX 

One of the earliest signs of Alzheimer's disease (AD) is the accumulation of abnormal amyloid-beta protein in the brain. Imaging techniques such as positron emission tomography (PET) can detect these amyloid deposition to identify individuals with high amyloid burden at an early stage and signal those at risk of developing dementia.

Deep learning models have shown to be highly effective at pattern recognition, with high diagnostic performance across several medical tasks. However, the complex nature of these models makes it difficult to trust their predictions, which is a key obstacle to adopting these tools in clinical practice. Explainable AI (XAI) is an emerging research area aiming to produce human-interpretable, robust, and useful explanations. Validating model explanations has proved a challenge across existing studies, particularly in the medical imaging domain due to difficulties obtaining a ground-truth representation of the disease-specific pathology (Martin et al. 2023, https://doi.org/10.1002/alz.12948). Efforts to validate model explanations have so far relied on generating synthetic datasets, to control and generate sources of bias or synthetically generated features (Stanley et al. 2023 https://arxiv.org/pdf/2311.02115.pdf). However, these studies are often limited to localised features such as brain lesions and do not evaluate the model’s ability to identify sparser patterns across the images. Moreover, few studies focus on validating model explanations in the context of dementia with structural imaging, as it is difficult to define a suitable “explanation” due to patient heterogeneity as well as the widespread and global nature of typical AD disease pathology.

In this work, we utilised PET imaging and deep learning to classify amyloid-positive and amyloid-negative individuals based on visual reads from the AMYPAD PNHS dataset. We employed a 3D convolutional neural network, a powerful framework for image classification but with limited interpretability due to its black-box nature. Although identifying amyloid-positivity from PET images is not clinically challenging, this task offers a unique opportunity to validate and assess whether the important brain regions according to model explanations correlate with the underlying disease pathology. Specifically, we leveraged the fact that amyloid-uptake is highly correlated with prevalence of the disease and clearly visible via PET imaging. We compared the heatmaps produced by state-of-the-art XAI methods by correlating regional importance scores with regional visual reads, SUVR values, and centiloid quantification. In this talk, I will share the results of this project, demonstrating a systematic approach to XAI method validation in the context of neuroimaging and dementia research.