Functional MRI timeseries prediction using sliding window-based transformer model
Poster No:
1528
Submission Type:
Abstract Submission
Authors:
Yurim Jang1, Bo-yong Park2
Institutions:
1Inha University, Incheon, Incheon, 2Korea University, Seoul, Seoul
First Author:
Co-Author:
Introduction:
Time series prediction involves forecasting future values in a sequence of data points ordered by time and can be applied across various fields, such as finance, climate science, and healthcare(Morid et al., 2021). Forecasting the temporal dynamics of brain signals measured by resting-state functional magnetic resonance imaging (rs-fMRI) holds great potential for identifying hidden brain state transitions, monitoring diseases, and predicting prognoses, thereby providing valuable insights into neural activity and connectivity(Gao et al., 2024). However, the high dimensionality of fMRI data and the associated computational complexity have posed significant challenges to understanding brain dynamics. With advancements in machine learning techniques, it has become feasible to predict future data points in complex datasets, such as fMRI time series. In this study, we employed a sliding window-based multivariate time series prediction transformer model(Zerveas et al., 2020).
Methods:
We obtained T1- and T2-weighted MRI and rs-fMRI of randomly selected 80 individuals (mean ± standard deviation (SD) age = 28.98 ± 3.85 years; male:female=46:54) from the Human Connectome Project (HCP) database(Van Essen et al., 2013). The imaging data werer preprocessed using the HCP minimal preprocessing pipeline(Glasser et al., 2013). Functional time series data mapped to the standard grayordinate space were summarized according to the Schaefer atlas with 300 parcels(Schaefer et al., 2018) and 14 individually defined subcortical regions(Patenaude et al., 2011). Age- and sex-controlled time series were used for training the transformer model. The transformer is a type of neural network that generates predictions by capturing temporal interactions between data points after extracting essential information from the input time series data(Vaswani et al., 2017). The time series of all subjects were normalized and split into training, validation, and test datasets in a 6:2:2 ratio. The training window size was set to 50, and the prediction window size varied between 1 and 10. The model was trained through a transformer encoder block followed by a linear layer, designed to predict time series data across 314 regions (Fig. 1A). With a learning rate of 0.001, the model was trained for a total of 100 epochs, and the validation data were used to assess training performance at each epoch. The model that achieved the highest validation performance was applied to the test dataset. The reliability of the predicted time series of the test dataset was evaluated by comparing the functional connectivity matrix and three gradients derived from the predicted time series with those compted from the actual data.
Results:
The transformer model showed the highest performance when the output window size was set to one (correlations of functional connectivity between the actual and predicted data=0.56±0.04 across individuals; three gradients=0.93±0.04; and mean squared error [MSE]=0.693±0.004; Fig. 1B-C). Performance decreased as the prediction window size increased. When we assessed the performance at the regional level using MSE, higher-order transmodal regions, such as the temporal and limbic areas, exhibited lower performance compared to sensory regions, indicating variability in time series prediction difficulty across brain regions.
·Figure 1. Multivariate time series prediction model.
Conclusions:
In this study, a transformer model was used to predict the next time series data based on the previous window's data. In future studies, we will expand the model's capacity to predict time points farther from the baseline. This approach has potential applications in disease populations to predict disease progression.
This work was supported by the IITP funded by the Korea Government (MSIT) (No. 2022-0-00448/RS-2022-II220448, Deep Total Recall: Continual Learning for Human-Like Recall of Artificial Neural Networks; RS-2021-II212068, Artificial Intelligence Innovation Hub), and Institute for Basic Science (IBS-R015-D1).
This work was supported by the IITP funded by the Korea Government (MSIT) (No. 2022-0-00448/RS-2022-II220448, Deep Total Recall: Continual Learning for Human-Like Recall of Artificial Neural Networks; RS-2021-II212068, Artificial Intelligence Innovation Hub), and Institute for Basic Science (IBS-R015-D1).
Modeling and Analysis Methods:
Methods Development 1
Multivariate Approaches 2
Keywords:
Computational Neuroscience
Data analysis
FUNCTIONAL MRI
Machine Learning
Modeling
Multivariate
Other - transformer; time series prediction
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Provide references using APA citation style.
Glasser, M. F., Sotiropoulos, S. N., Wilson, J. A., Coalson, T. S., Fischl, B., Andersson, J. L., Xu, J., Jbabdi, S., Webster, M., Polimeni, J. R., Van Essen, D. C., & Jenkinson, M. (2013). The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage, 80, 105–124. doi: 10.1016/j.neuroimage.2013.04.127
Morid, M. A., Sheng, O. R. L., & Dunbar, J. (2021). Time Series Prediction using Deep Learning Methods in Healthcare.
Patenaude, B., Smith, S. M., Kennedy, D. N., & Jenkinson, M. (2011). A Bayesian model of shape and appearance for subcortical brain segmentation. NeuroImage, 56(3), 907–922. doi: 10.1016/j.neuroimage.2011.02.046
Schaefer, A., Kong, R., Gordon, E. M., Laumann, T. O., Zuo, X.-N., Holmes, A. J., Eickhoff, S. B., & Yeo, B. T. T. (2018). Local-Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cerebral Cortex, 28(9), 3095–3114. doi: 10.1093/cercor/bhx179
Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E. J., Yacoub, E., & Ugurbil, K. (2013). The WU-Minn Human Connectome Project: An overview. NeuroImage, 80, 62–79. doi: 10.1016/j.neuroimage.2013.05.041
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention Is All You Need.
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2020). A TRANSFORMER-BASED FRAMEWORK FOR MULTI-VARIATE TIME SERIES REPRESENTATION LEARNING.