Reduced learning from loss is enhanced by prioritized reactivation of past punishments experiences
Poster No:
1888
Submission Type:
Abstract Submission
Authors:
Ting Xu1, Lei Zhang2, Feng Zhou1, Xianyang Gan3, Benjamin Becker4
Institutions:
1Southwest University, Chongqing, Chongqing, 2Centre for Human Brain Healthy, School of Psychology, University of Birmingham, Birmingham, UK, Birmingham, United Kingdom, 3University of Electronic Science and Technology of China, Chengdu, Sichuan, 4The University of Hong Kong, Hong Kong, Hong Kong
First Author:
Co-Author(s):
Lei Zhang
Centre for Human Brain Healthy, School of Psychology, University of Birmingham, Birmingham, UK
Birmingham, United Kingdom
Centre for Human Brain Healthy, School of Psychology, University of Birmingham, Birmingham, UK
Birmingham, United Kingdom
Introduction:
While the offline re-integrations of past events facilitate reinforcement learning (1-3) and govern confident choices on options we have never directly experienced, reward- or punishment-based information in fact determines how effectively we can recall (4,5). Considerable evidence from animal models and human works has reported preferential reactivation of neural representations for high-reward or aversive objects (6,7), yet little is known about how rewarded and punished information reemerge differently and play dissociable roles in informing future decisions in the identical environment. The present study therefore aimed at identifying the neural reactivation of learned reward and punishment experiences during rest periods and exploring how it supports subsequent transfer learning.
Methods:
In a randomized-controlled within-subject fMRI experiment, N=46 (Mean age=21.13 years, 23 females) participants were enrolled and underwent a probabilistic reward learning (PRL) task inserted with short rest blocks during fMRI (Figure 1a). The PRL task included an initial learning stage during which subjects learned to associate certain stimuli with higher rewarding probability (Figure 1b-1c) and a subsequent transfer stage where subjects generalized the learned reward values to make optimal choices when encountering new combinations of the stimuli. Trial-wise choice data of learning stage was fitted in a computational model to estimate the hidden learning parameters including prediction errors (RPE) and expected value separated by reward and loss contexts, while the choice accuracy for approaching the best or avoiding the worst stimulus was analyzed as an exploration. The multivoxel pattern analyses were conducted to decode the multivariate neural maps predictive of stimuli, outcome, RPE and expected value. In addition several logistic regression classifiers trained on stimuli, outcome, RPE and expected value learning experiences under reward and loss situations were applied to the rest periods to check whether those learning experiences have neural reactivation and whether this can support transfer behaviors.
Results:
The choice accuracy on choosing A and avoiding B were significantly higher than the chance level (all>50%, PS<0.001) across transfer rounds (Figure 1d), which may suggest that, as transfer phases increase, individuals can effectively apply learned experiences to optimize their approach toward positive outcomes while simultaneously avoiding negative ones. While on the neural level, we identified distinct anatomical brain regions in encoding different initial learning experience such that: (1) the dmPFC, PCC, vmPFC and bilateral anterior insula (AI) incorporated stable predictive voxels processing the stimuli and outcome experience, while the VS was largely specific to outcome, (2) the SFG and right AI were preferentially important for encoding expected value, whereas the VS and bilateral AI encoded PE more (All ACC>0.84, PS<0.001). This could help find how those learning experience segregated in varied neural replay pattern and exerted dissociable contribution to guide subsequent transfer behaviours. However during the rest periods, we only observed significantly positive association between replay probabilities of loss learning experiences and choice accuracy in avoiding B condition (r=0.31, Ppermutation<0.05), and this effect was implemented through engagement of reward re-evaluation and memory integration related regions (e.g., vmPFC, VS, PCC, MTG and Parahippocampus).
Conclusions:
In conclusion, while distinct brain regions are involved in encoding different learning experiences during the initial learning stage, only punishment-related experiences are preferentially reactivated during off-task periods and subsequently enhance loss avoidance behaviors. These findings may offer a promising therapeutic pathway for depression and anxiety disorders, which are often characterized by repetitive negative thought patterns.
Disorders of the Nervous System:
Psychiatric (eg. Depression, Anxiety, Schizophrenia)
Emotion, Motivation and Social Neuroscience:
Reward and Punishment 2
Modeling and Analysis Methods:
Classification and Predictive Modeling
Multivariate Approaches
Novel Imaging Acquisition Methods:
BOLD fMRI 1
Keywords:
Computational Neuroscience
FUNCTIONAL MRI
Learning
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
evaluation in reward-based choices. Nature communications, 8(1), 15821.
2. Klein-Flügge, M. C., Wittmann, M. K., Shpektor, A., Jensen, D. E., & Rushworth, M. F. (2019). Multiple associative structures created
by reinforcement and incidental statistical learning mechanisms. Nature communications, 10(1), 4835.
3. Collins, A. G., & Cockburn, J. (2020). Beyond dichotomies in reinforcement learning. Nature Reviews Neuroscience, 21(10), 576-
586.
4. Vollberg, M. C., & Sander, D. (2024). Hidden reward: Affect and its prediction errors as windows into subjective value. Current
Directions in Psychological Science, 33(2), 93-99.
5. Schlegelmilch, R., & von Helversen, B. (2020). The influence of reward magnitude on stimulus memory and stimulus
generalization in categorization decisions. Journal of Experimental Psychology: General, 149(10), 1823.
6. Sterpenich, V., van Schie, M. K., Catsiyannis, M., Ramyead, A., Perrig, S., Yang, H. D., ... & Schwartz, S. (2021). Reward biases
spontaneous neural reactivation during sleep. Nature communications, 12(1), 4162.
7. Schuck, N. W., & Niv, Y. (2019). Sequential replay of nonspatial task states in the human hippocampus. Science, 364(6447), eaaw5181.