Language Activation from Shortened Naturalistic fMRI Time Series using Machine Learning
Poster No:
1154
Submission Type:
Abstract Submission
Authors:
Elaine Kuan1,2,3, Viktor Vegh1,2,3, John Phamnguyen2,4,1, Kieran O’Brien5, Amanda Hammond5, David Reutens1,2,4,3
Institutions:
1Australian Institute for Bioengineering and Nanotechnology, The University of Queensland, Brisbane, Australia, 2Centre of Advanced Imaging, The University of Queensland, Brisbane, Australia, 3ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland, Brisbane, Australia, 4Royal Brisbane and Women’s Hospital, Brisbane, Australia, 5Siemens Healthcare Pty Ltd, Brisbane, Australia
First Author:
Elaine Kuan
Australian Institute for Bioengineering and Nanotechnology, The University of Queensland|Centre of Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
Australian Institute for Bioengineering and Nanotechnology, The University of Queensland|Centre of Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
Co-Author(s):
Viktor Vegh
Australian Institute for Bioengineering and Nanotechnology, The University of Queensland|Centre of Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
Australian Institute for Bioengineering and Nanotechnology, The University of Queensland|Centre of Advanced Imaging, The University of Queensland|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
John Phamnguyen
Centre of Advanced Imaging, The University of Queensland|Royal Brisbane and Women’s Hospital|Australian Institute for Bioengineering and Nanotechnology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
Centre of Advanced Imaging, The University of Queensland|Royal Brisbane and Women’s Hospital|Australian Institute for Bioengineering and Nanotechnology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
David Reutens
Australian Institute for Bioengineering and Nanotechnology, The University of Queensland|Centre of Advanced Imaging, The University of Queensland|Royal Brisbane and Women’s Hospital|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
Australian Institute for Bioengineering and Nanotechnology, The University of Queensland|Centre of Advanced Imaging, The University of Queensland|Royal Brisbane and Women’s Hospital|ARC Training Centre for Innovation in Biomedical Imaging Technology, The University of Queensland
Brisbane, Australia|Brisbane, Australia|Brisbane, Australia|Brisbane, Australia
Introduction:
Naturalistic fMRI can overcome some limitations of traditional task-based fMRI such as the lack of ecological validity and engagement of participants. However, due to the dynamic nature of the naturalistic stimulus, data driven machine learning methods are required to analyse naturalistic fMRI data. Rotation Forest (RotF) (Rodriguez et al, 2006) has been shown to be effective in generating robust language activation maps, however, a long stimulus or fMRI time series amounting to 15 minutes of scan time is still required. Hence, this study aims to identify shortened segments of the naturalistic fMRI time series contributing mostly to the classification achieved by RotF using a feature selection technique, and evaluating the naturalistic stimulus segment (of an in-house created quiz show interleaved with advertisements) associated with the identified fMRI time series segment on three healthy participants.
Methods:
Sequential Forward Feature Elimination (SFFE) (Ferri et al.,1994) was performed 10 times on a previously trained RotF model (Details can be found in OHBM 2024 poster number 1438). For each run, the top five time frames were identified by SFFE and labelled with ones, while the remaining time frames were labelled with zeros, to produce a frame importance vector across the naturalistic fMRI time series data points. The 10 frame importance vectors were added over the repeated runs, resulting in an overall frame importance metric for all time points.
A contiguous shortened segment was chosen based on the importance of data points. RotF was retrained using the identified shortened naturalistic fMRI segment with the same labels as the training set used on the previously trained RotF. Subsequently, the shortened naturalistic stimuli segment corresponding to the shortened naturalistic fMRI segment was presented to three participants. Gradient echo planar imaging (EPI) fMRI data was acquired with the following parameters: matrix size = 64 x 64 x 42, TR = 2s, TE = 30ms and voxel size 3 x 3 x 3 mm3. The raw EPI data were pre-processed using the SPM12 software with slice timing correction, realignment of volumes, co-registration and smoothing. A test set comprising of shortened voxel-wise naturalistic fMRI time series of the whole brain for the three participants were extracted and evaluated using the retrained RotF model. The RotF based activation maps of each test participant were first reconstructed and then compared with the sentence completion (SC) task activation map. The structural similarity index measure (SSIM) of both activation maps were also evaluated.
A contiguous shortened segment was chosen based on the importance of data points. RotF was retrained using the identified shortened naturalistic fMRI segment with the same labels as the training set used on the previously trained RotF. Subsequently, the shortened naturalistic stimuli segment corresponding to the shortened naturalistic fMRI segment was presented to three participants. Gradient echo planar imaging (EPI) fMRI data was acquired with the following parameters: matrix size = 64 x 64 x 42, TR = 2s, TE = 30ms and voxel size 3 x 3 x 3 mm3. The raw EPI data were pre-processed using the SPM12 software with slice timing correction, realignment of volumes, co-registration and smoothing. A test set comprising of shortened voxel-wise naturalistic fMRI time series of the whole brain for the three participants were extracted and evaluated using the retrained RotF model. The RotF based activation maps of each test participant were first reconstructed and then compared with the sentence completion (SC) task activation map. The structural similarity index measure (SSIM) of both activation maps were also evaluated.
Results:
From Figure 1, we observe that over 10 runs of SFEE, there are spikes in the waveform that concentrate between the first two advertisement blocks (frames 138 to 255, corresponding to approximately 4 minutes. The stimulus was extended by 1 minute, including 30 seconds of the naturalistic stimuli at the beginning and end of the identified segment. The total stimulus length presented to participants was approximately 5 minutes), indicating that the naturalistic fMRI time series within those time points are important for the classification performed by RotF.
From Figure 2, language clusters including those in the inferior frontal gyrus and posterior superior temporal regions were captured for all three participants, with varying degrees of overlap. While bilateral activation can be observed in the temporal regions of the RotF activation map in all three participants, this was expected due to auditory components in the naturalistic fMRI paradigm. The SSIM comparing both activation maps were 0.75, 0.69 and 0.76, respectively.
From Figure 2, language clusters including those in the inferior frontal gyrus and posterior superior temporal regions were captured for all three participants, with varying degrees of overlap. While bilateral activation can be observed in the temporal regions of the RotF activation map in all three participants, this was expected due to auditory components in the naturalistic fMRI paradigm. The SSIM comparing both activation maps were 0.75, 0.69 and 0.76, respectively.
·Figure 1: The green plot shows the weighting for frame importance obtained from running the SFFE 10 times. The blue blocks show the frames corresponding to advertisement segments and the orange blocks
·Figure 2: The overlap between SC task activation and RotF activation for three participants (Blue – overlap, Red – Combined task activation, Green – RotF activation)
Conclusions:
The use of SFFE was able to identify a shortened fMRI time series segment contributing to RotF prediction. Further evaluation of the corresponding shortened stimulus with RotF training showed that language clusters corresponding to SC activation can be captured. These initial successes pave the way for new approaches to naturalistic stimulus design.
Language:
Language Acquisition 2
Modeling and Analysis Methods:
Activation (eg. BOLD task-fMRI)
Classification and Predictive Modeling 1
Methods Development
Keywords:
ADULTS
Design and Analysis
FUNCTIONAL MRI
Language
Machine Learning
Modeling
Univariate
Other - Naturalistic fMRI
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
Ferri, F. J., Pudil, P., Hatef, M., & Kittler, J. (1994). Comparative study of techniques for large-scale feature selection. In Machine intelligence and pattern recognition, vol. 16, pp. 403-413.