Towards associative memory in convolutional neural networks for in silico neurodegenerative diseases

200 

Abstract Submission 

Chris Kang¹, Jasmine Moore¹, Matthias Wilms¹, Nils Forkert¹

¹University of Calgary, Calgary, Alberta

Chris Kang  
University of Calgary
Calgary, Alberta

Jasmine Moore  
University of Calgary
Calgary, Alberta

Matthias Wilms  
University of Calgary
Calgary, Alberta

Nils Forkert, Ph.D.  
University of Calgary
Calgary, Alberta

Convolutional neural networks (CNNs) have emerged as a popular choice of deep learning architecture for modeling visual processing, as their hierarchical structure and flow of information processing closely resembles the human ventral stream [LeCun, Y. (1989)]. While CNNs have been used to model healthy visual cognition, there remain limitations in biologically plausible in silico modeling of cognitive decline in neurodegenerative diseases, such as Alzheimer's (AD). Previously, we developed methods to simulate neurodegeneration of the visual system through iterative synaptic injury in CNNs [Tuladhar, A. (2021), Moore, J. (2023)]. However, the limitation of CNNs lies in the lack of biologically meaningful learning mechanisms that are similar to cognitive functions, such as memory. These mechanisms are essential for accurately capturing the neuropathogenesis. For instance, the deposition of beta-amyloid peptide and neurofibrillary tangles of tau polymers in the hippocampus leads to cognitive decline in memory tasks among AD patients.

Building on our prior work, in this study, we equipped a CNN with associative memory to enhance biological plausibility, combining two critical cognitive functions of the brain: visual processing of the ventral stream and associative memory of the hippocampus. The model demonstrates intriguing and beneficial properties, including (1) robustness to noisy or occluded image queries and (2) interpretable and sparse representations in network weights. We argue that this model is an improved in silico framework for a healthy brain, as well as the cognitive profiles of AD progression.

VGG19, a CNN with a high similarity to the human brain as measured by Brain-Score [Schrimpf, M. (2018)], was equipped with a flattened layer of the modern Hopfield network [Krotov, D. (2016), Ramsauer, H. (2020)], replacing the penultimate fully-connected layer (Figure 1a). This VGG-MHN model was independently trained on two commonly used vision datasets, MNIST and CIFAR-10, for image classification.

Previously trained CIFAR-10 images and test images were injected with Gaussian noise, according to varying signal-to-signal plus noise ratios, and classification tasks were performed to measure the ability to recall noisy queries, a known ability of the human brain. The result was compared to the baseline performance of VGG19, which has been shown to perform poorly when inputs are even slightly altered [Tang, H. (2018)]. For a more challenging recall task, training and classification with MNIST images was performed on partially occluded test sets featuring images masked by 30% and 50% of the total area, and the network weights were analyzed.

VGG-MHN exhibited significantly improved robustness to noisy queries for previously seen images compared to the baseline VGG19 (Figure 1b). In case of unseen images, both VGG-MHN and VGG19 performed equally in the high SSNR domain. However, VGG-MHN underperformed when the SSNR level was below w=0.8. In the occluded MNIST experiment, VGG-MHN significantly outperformed VGG19 (Figure 1c).

Further analysis of the VGG-MHN when tested on the occluded MNIST dataset revealed that, in the Hopfield layer, the weights preserve feature and prototype representations depending on the model choice of energy function. This phenomenon was first characterized as a "feature-to-prototype" transition (Figure 1d) [Krotov, D. (2018)].

The biologically inspired CNN, equipped with associative memory, extends our existing framework for in silico neurodegeneration. VGG-HMN effectively integrates visual processing with memory systems grounded on fundamental cognitive principles (i.e., Hebbian learning) and offers many advantages, simulating a healthy brain. These include enhanced robustness against noisy and occluded queries and the production of interpretable representations. We believe that such an architecture is well suited for in silico analysis of neurodegenerative diseases in the forthcoming work.

Neurodegenerative/ Late Life (eg. Parkinson’s, Alzheimer’s) ¹

Neural Plasticity and Recovery of Function

Learning and Memory Other ²

Informatics Other

Attention: Visual

Machine Learning

Memory

STRUCTURAL MRI