Investigating higher-order signatures in the human brain using film fMRI
Poster No:
1465
Submission Type:
Abstract Submission
Authors:
Lorenzo Zaffina1, Flavia Petruso1, Maria Giulia Preti2, Giovanni Petri3, Enrico Amico4, Elenor Morgenroth1, Dimitri Van De Ville5, Andrea Santoro6
Institutions:
1Ecole Polytechnique Fédérale de Lausanne, Geneva, Switzerland, 2Ecole Polytechnique Federale de Lausanne (EPFL), Geneva, Geneva, 3Northeastern University London, London, London, 4University of Birmingham, School of Mathematics & Centre for Human Brain Health, Birmingham, United Kingdom, 5École polytechnique fédérale de Lausanne (EPFL), Geneva, Geneva, 6CENTAI Institute, Turin, Italy
First Author:
Co-Author(s):
Enrico Amico
University of Birmingham, School of Mathematics & Centre for Human Brain Health
Birmingham, United Kingdom
University of Birmingham, School of Mathematics & Centre for Human Brain Health
Birmingham, United Kingdom
Introduction:
Traditional models of human brain activity have primarily emphasized pairwise interactions, such as functional connectivity (FC). To capture the more complex interactions among multiple brain regions, recent approaches have begun to infer higher-order interactions (HOIs) involving three or more regions simultaneously. In this study, we employ a novel dynamical method [1] to quantify temporal HOI effects in multivariate time series within a naturalistic framework. Utilizing a comprehensive dataset of emotional annotations for short films and film fMRI [2], we explore the relationship between temporal fluctuations in emotions and the brain's dynamic higher-order signatures. Our goal is to elucidate the additional advantage of higher-order approaches in the context of emotional processes during movie watching.
Methods:
We utilized the Emo-FilM dataset [2], which includes fMRI recordings from 30 healthy subjects as they watched 14 emotionally charged short films, each averaging around seven minutes in length. These films were carefully selected to elicit a wide range of emotions in participants. Additionally, the dataset provides detailed annotations for each film, capturing the temporal evolution of 50 distinct emotions at every repetition time (TR), averaged from an independent group of raters (Fig. 1a-b). By leveraging these emotional annotations, we investigated how fluctuations in emotions are reflected in brain activity and captured by higher-order indicators.
For our analysis, we employed the higher-order interaction (HOI) method [3], which captures interactions among three or more brain regions simultaneously using topological data analysis (Fig. 1c). Unlike traditional approaches that focus on individual regions or pairwise interactions, the HOI method provides "instantaneous" snapshots of the brain's higher-order activity at each TR through specific topological indicators. Our objective was to determine whether HOI measures can capture more emotion-related information compared to classical pairwise approaches.
For our analysis, we employed the higher-order interaction (HOI) method [3], which captures interactions among three or more brain regions simultaneously using topological data analysis (Fig. 1c). Unlike traditional approaches that focus on individual regions or pairwise interactions, the HOI method provides "instantaneous" snapshots of the brain's higher-order activity at each TR through specific topological indicators. Our objective was to determine whether HOI measures can capture more emotion-related information compared to classical pairwise approaches.
·Figure 1
Results:
The emotional content of films fluctuates rapidly, typically on the order of tens of seconds. Utilizing the emotional annotations in our dataset, we obtained time series detailing the temporal dynamics of 50 distinct emotions throughout film scenes. We evaluated the association between emotional fluctuations and 4 methods, including low-order (BOLD and edges) and high-order (scaffold and triangles) brain measures, by comparing time-time correlation matrices, or recurrence plots, derived from emotional and brain measure time series.
The recurrence plots from emotion time series exhibited distinct temporal correlation patterns compared to those from fMRI indicators, reflecting different clusters of emotional and brain activity. To assess the overlap between emotional and brain activity clusters, we partitioned each matrix using the Louvain algorithm and quantified their similarity with the element centric similarity (ECS) metric [4], which ranges from 0 (completely dissimilar) to 1 (identical). To ensure robust partition estimation, we employed consensus clustering [5] due to the stochastic nature of the Louvain algorithm. For each film, we obtained ECS values comparing each of the four measures with the emotion correlation matrix, as illustrated in Fig. 2. The scaffold metric (red) exhibited the highest similarity to the emotional clusters, outperforming other metrics in 11 out of 14 films. Furthermore, cortical activations associated with triangles and scaffold metrics consistently differed from those derived from classical functional connectivity (not shown)
The recurrence plots from emotion time series exhibited distinct temporal correlation patterns compared to those from fMRI indicators, reflecting different clusters of emotional and brain activity. To assess the overlap between emotional and brain activity clusters, we partitioned each matrix using the Louvain algorithm and quantified their similarity with the element centric similarity (ECS) metric [4], which ranges from 0 (completely dissimilar) to 1 (identical). To ensure robust partition estimation, we employed consensus clustering [5] due to the stochastic nature of the Louvain algorithm. For each film, we obtained ECS values comparing each of the four measures with the emotion correlation matrix, as illustrated in Fig. 2. The scaffold metric (red) exhibited the highest similarity to the emotional clusters, outperforming other metrics in 11 out of 14 films. Furthermore, cortical activations associated with triangles and scaffold metrics consistently differed from those derived from classical functional connectivity (not shown)
·Figure 2
Conclusions:
Our findings demonstrate that the scaffold metric effectively captures temporal clusters of brain activity that more accurately correspond to clusters of similar emotional activity elicited by films. This supports the growing trend of higher-order methods, which have the potential to capture information beyond that obtained by traditional pairwise approaches.
Emotion, Motivation and Social Neuroscience:
Social Neuroscience Other 2
Modeling and Analysis Methods:
Activation (eg. BOLD task-fMRI)
fMRI Connectivity and Network Modeling 1
Methods Development
Multivariate Approaches
Keywords:
Computational Neuroscience
Data analysis
Emotions
FUNCTIONAL MRI
Modeling
NORMAL HUMAN
Other - Simplicial Complex; Higher-order systems; Persistent Homology; Topology; Film; Naturalistic stimuli;
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Provide references using APA citation style.
[2] Morgenroth, E., Moia, S., Vilaclara, L., Fournier, R., Muszynski, M., Ploumitsakou, M., Almató-Bellavista, M., Vuilleumier, P., & Ville, D. V. D. (2024). bioRxiv.
[3] Santoro, A., Battiston, F., Petri, G., & Amico, E. (2023). Nature Physics, 19.
[4] Gates, A. J., Wood, I. B., Hetrick, W. P., & Ahn, Y.-Y. (2019). Scientific Reports, 9. http://doi.org/10.1038/s41598-019-44892-y
[5] Lancichinetti, A., & Fortunato, S. (2012). Scientific Reports, 2.