Print Close

Characterising Brain State Dynamics during Naturalistic Viewing fMRI using a Hidden Markov Model

Presented During: Poster Session 3
Friday, June 27, 2025: 01:45 PM - 03:45 PM

Presented During: Poster Session 4
Saturday, June 28, 2025: 01:45 PM - 03:45 PM

Poster No:

1620

Submission Type:

Abstract Submission

Authors:

Susanne Weis¹, Karolina Niessen², Yulia Nurislamova¹, Xuan Li³

Institutions:

¹Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University, Düsseldorf, Germany, ²Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands, ³Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany

First Author:

Susanne Weis
Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University
Düsseldorf, Germany

Co-Author(s):

Karolina Niessen
Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University
Maastricht, Netherlands

Yulia Nurislamova
Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University
Düsseldorf, Germany

Xuan Li, PhD
Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich
Jülich, Germany

Introduction:

Processing and integration of dynamic external and internal information is a major task of the brain in dealing with complex daily life experiences. How information is transformed and integrated into internal representations is highly variable across individuals (Baldassano et al., 2017) and is influenced by factors like culture, environment, genetics and sex. Naturalistic viewing (NV) paradigms use stimuli such as movies that resemble daily life sensory input, with no other task than watching the stimuli (Finn et al., 2022; Hasson et al., 2004). NV paradigms allow for examining the brain's everyday functioning in a novel framework, proposing that dynamic systems of brain states, rather than isolated networks, are crucial. It has been suggested that the processing of external sensory and internal input occurs in separate, distinct states of brain activity (Zagha & Mccormick, 2014) that only switch and never coexist in mixture states. So far, the relationship of brain state dynamics to individual differences during NV received little attention but may offer valuable insights (Finn et al., 2017), as NV enhances individuality while also allowing for unique brain responses. This study aimed to identify brain states during NV, examine their similarity across participants and movies, and explore how these states relate to the emotional content of the movies.

Methods:

NV fMRI data, comprising eight movie scans (7-10 minutes), was acquired on a 3T Siemens Prisma MRI scanner on 60 (27 female / 33 male) natively German-speaking subjects (mean age 23.4 years, SD = 3.6). An additional sample of 39 subjects rated the strength of six basic emotions (fear, disgust, joy, surprise, sadness, anger) depicted in the movies. fMRI data was pre-processed using fMRIprep (Esteban et al., 2019), parcellated according to the Schaefer-400 parcellation (Schaefer et al., 2018) and parcel wise BOLD time series data was extracted and averaged within 17 brain networks (Yeo et al., 2011), resulting in one time series for each network and subject. The Hidden Markov Model (HMM) analysis was conducted using the OSL Dynamics toolbox for Python (Gohil et al., 2024). Optimization of model parameters resulted in optimal model fit for 8 brain states, for which the characteristic network (de-)activation patterns were reconstructed. To assess the relationship of the brain states to the emotions depicted in the movies, the likelihood of each state was correlated with the emotion annotations across TRs.

Results:

The HMM identified 8 distinct brain states across movies and participants, each with a unique pattern of brain network coactivation (fig 1). Brain states demonstrate significant correlations with the emotion ratings of the movie scenes (fig 2), especially for negative emotions (fear, disgust, and anger), but also joy and surprise. Brain state dynamics differ significantly from each other and between different movie scenes. On average, approximately a third of the participants were synchronously in the same brain state at the same time point. At certain points, however, almost all participants adopted the same brain state.

·figure 1: Brain States and Network Coactivation Patterns. Increased (red) and decreased (blue) activation compared to the average network activity. Scale indicates strength of activation in z-scores.

·figure 2: Significant Pearson correlations of the likelihood of each state with emotion annotations across TRs (corrected for multiple comparisons, *Approaching significance)

Conclusions:

Our study shows that HMMs are an effective method for extracting and analyzing hidden brain states in NV fMRI. The 8 identified brain states are distinct in their dynamics and network activation patterns, and each correlates differently with the six basic emotions portrayed in the movies. Across the movies, at certain time points, nearly all participants synchronize in the same brain state. It is possible that specific features of the movies evoke this synchrony, while other aspects might accentuate individual differences. Our findings suggest that combining NV fMRI with HMMs offers a promising approach for investigating individual differences, as the synchrony and asynchrony of participants may be linked to behavioral outcomes. This opens exciting new avenues for exploring the relationship between brain state dynamics and behavioral phenotypes.

Higher Cognitive Functions:

Higher Cognitive Functions Other ²

Modeling and Analysis Methods:

Other Methods ¹

Perception, Attention and Motor Behavior:

Perception and Attention Other

Keywords:

ADULTS

Cognition

FUNCTIONAL MRI

Machine Learning

Other - Naturalistic Viewing

^1|2Indicates the priority used for review

Abstract Information

By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.

I accept

The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:

I do not want to participate in the reproducibility challenge.

Please indicate below if your study was a "resting state" or "task-activation” study.

Other

Healthy subjects only or patients (note that patient studies may also involve healthy subjects):

Healthy subjects

Was this research conducted in the United States?

Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.

Yes

Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.

Not applicable

Please indicate which methods were used in your research:

Functional MRI

For human MRI, what field strength scanner do you use?

3.0T

Which processing packages did you use for your study?

Other, Please list - OSL Dynamics toolbox for Python

Provide references using APA citation style.

Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017). Discovering Event Structure in Continuous Narrative Perception and Memory. Neuron, 95(3), 709-721.e705. https://doi.org/10.1016/j.neuron.2017.06.041
2. Esteban, O., Markiewicz, C. J., Blair, R. W., Moodie, C. A., Isik, A. I., Erramuzpe, A., Kent, J. D., Goncalves, M., Dupre, E., Snyder, M., Oya, H., Ghosh, S. S., Wright, J., Durnez, J., Poldrack, R. A., & Gorgolewski, K. J. (2019). fMRIPrep: a robust preprocessing pipeline for functional MRI. Nature Methods, 16(1), 111-116. https://doi.org/10.1038/s41592-018-0235-4
3. Finn, E. S., Glerean, E., Hasson, U., & Vanderwal, T. (2022). Naturalistic imaging: The use of ecologically valid conditions to study brain function. Neuroimage, 247, 118776. https://doi.org/10.1016/j.neuroimage.2021.118776
4. Finn, E. S., Scheinost, D., Finn, D. M., Shen, X., Papademetris, X., & Constable, R. T. (2017). Can brain state be manipulated to emphasize individual differences in functional connectivity? Neuroimage, 160, 140-151. https://doi.org/10.1016/j.neuroimage.2017.03.064
5. Gohil, C., Huang, R., Roberts, E., van Es, M. W. J., Quinn, A. J., Vidaurre, D., & Woolrich, M. W. (2024). osl-dynamics, a toolbox for modeling fast dynamic brain activity. Elife, 12. https://doi.org/10.7554/eLife.91949
6. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science, 303(5664), 1634-1640. https://doi.org/10.1126/science.1089506
7. Schaefer, A., Kong, R., Gordon, E. M., Laumann, T. O., Zuo, X.-N., Holmes, A. J., Eickhoff, S. B., & Yeo, B. T. T. (2018). Local-Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cerebral Cortex, 28(9), 3095-3114. https://doi.org/10.1093/cercor/bhx179
8. Yeo, B. T., Krienen, F. M., Sepulcre, J., Sabuncu, M. R., Lashkari, D., Hollinshead, M., Roffman, J. L., Smoller, J. W., Zöllei, L., Polimeni, J. R., Fischl, B., Liu, H., & Buckner, R. L. (2011). The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol, 106(3), 1125-1165. https://doi.org/10.1152/jn.00338.2011
9. Zagha, E., & Mccormick, D. A. (2014). Neural control of brain state. Current Opinion in Neurobiology, 29, 178-186. https://doi.org/10.1016/j.conb.2014.09.010

UNESCO Institute of Statistics and World Bank Waiver Form

I attest that I currently live, work, or study in a country on the UNESCO Institute of Statistics and World Bank List of Low and Middle Income Countries list provided.