Calculating within- and between-pipeline variability in resting-state fMRI data analysis
Poster No:
1272
Submission Type:
Late-Breaking Abstract Submission
Authors:
Violeta Céspedes1, Lea Waller2, Tomáš Fodran1, Ilya Veer1
Institutions:
1University of Amsterdam, Amsterdam, Netherlands, 2Charité - Universitätsmedizin Berlin, Berlin, Germany
First Author:
Co-Author(s):
Late Breaking Reviewer(s):
Rosanna Olsen
Rotman Research Institute, Baycrest Academy for Research and Education
Toronto, Ontario
Rotman Research Institute, Baycrest Academy for Research and Education
Toronto, Ontario
Introduction:
Attaining similar results from repeated runs of the same processing pipeline is an important prerequisite for reliable scientific results. One barrier to achieving this form of computational reproducibility is the accumulation of numerical errors and stochastic elements from single processing steps over the course of a processing pipeline. This can lead to substantive variability in outputs for structural MRI processing across repeated runs of the sMRIPrep pipeline (Chatelain et al. 2023). Here, we investigate variability across repeated runs of calculating resting-state fMRI connectivity matrices on data minimally processed with HALFpipe (Waller et al. 2022), which is based on fMRIPrep (Esteban et al., 2019). fMRI processing typically involves many more steps than structural MRI processing, leading to an increased risk of accumulating variability.
In addition to minimal preprocessing of fMRI data, researchers often choose to use a variety of additional processing steps such as confound regression and denoising (Dafflon et al. 2022). As such, pipelines may exhibit different degrees of variability depending on which further processing steps are chosen. To understand the impact of such choices on variability in connectivity results, we also compare two additional pipelines in a multiverse approach.
In addition to minimal preprocessing of fMRI data, researchers often choose to use a variety of additional processing steps such as confound regression and denoising (Dafflon et al. 2022). As such, pipelines may exhibit different degrees of variability depending on which further processing steps are chosen. To understand the impact of such choices on variability in connectivity results, we also compare two additional pipelines in a multiverse approach.
Methods:
We selected a random participant from the Oxford-Nottingham Harmonisation dataset, available on OpenNeuro (https://openneuro.org/). First, we assessed variability across repeated runs by calculating the same minimal processing pipeline (i.e., preprocessing without confound regression or denoising; "no confounds") 100 times, each time calculating a functional connectivity (i.e., correlation) matrix based on the Schaefer 400 parcellation atlas (Schaefer et al. 2018). Second, we used two additional processing pipelines with common settings for confound regression and denoising of rs-fMRI data, one with global signal regression (GSR), and one using ICA-AROMA (Pruim et al. 2015), and also repeated the preprocessing and calculation of the connectivity matrix a 100 times for each pipeline to assess the impact on within-pipeline variability and to assess between-pipeline variability.
Results:
We calculated standard deviations across the 100 iterations for each cell of pairwise correlations in the connectivity matrix (Figure 1A). We then focused on the variability of the functional connectivity between the PCC and mPFC, which are key regions of the default mode network (Figure 1B). Within-pipeline variability is low for "no confounds" and GSR pipelines but substantially higher for ICA-AROMA. This pipeline shows the most variability, as exemplified in PCC/mPFC connectivity, where correlation values range from negative to positive. Its average standard deviation is also significantly higher, with some localized variability.For PCC/mPFC connectivity, the "no confounds" and GSR pipelines yield different averages, with "no confounds" pipeline showing an expected positive correlation (average correlation coefficient = 0.539) and the GSR pipeline a near-zero correlation (average correlation coefficient = 0.0427), even though both pipelines demonstrate low within-pipeline variability.
·Figure 1. Within-subject variability results, both within and across pipelines.
Conclusions:
Overall, within-pipeline variability was very low, with a standard deviation average of approximately 0.005 for both the minimal preprocessing pipeline and the GSR pipeline. This metric increased non-trivially for the ICA-AROMA pipeline, providing a rationale for investigating the cause for this difference. Between-pipeline variability was high, with markedly different results for the PCC/mPFC connectivity across all pipelines.
In here we looked at within- and between-pipeline variability for only one person, but how such variability influences group-level comparisons is yet to be determined. Additionally, other common choices of pipelines for confound removal or combinations of confound regression and denoising options (Van Schuerbeek, De Wandel, and Baeken 2022) could be added.
In here we looked at within- and between-pipeline variability for only one person, but how such variability influences group-level comparisons is yet to be determined. Additionally, other common choices of pipelines for confound removal or combinations of confound regression and denoising options (Van Schuerbeek, De Wandel, and Baeken 2022) could be added.
Modeling and Analysis Methods:
Connectivity (eg. functional, effective, structural) 1
Methods Development
Motion Correction and Preprocessing 2
Neuroinformatics and Data Sharing:
Informatics Other
Novel Imaging Acquisition Methods:
BOLD fMRI
Keywords:
FUNCTIONAL MRI
Other - multiverse; within-subject variability; connectomics; resting-state
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
Chatelain, Yohan, Loïc Tetrel, Christopher J. Markiewicz, Mathias Goncalves, Gregory Kiar, Oscar Esteban, Pierre Bellec, and Tristan Glatard. 2023. “A Numerical Variability Approach to Results Stability Tests and Its Application to Neuroimaging.” arXiv. https://doi.org/10.48550/arXiv.2307.01373.
Dafflon, Jessica, Pedro F. Da Costa, František Váša, Ricardo Pio Monti, Danilo Bzdok, Peter J. Hellyer, Federico Turkheimer, Jonathan Smallwood, Emily Jones, and Robert Leech. 2022. “A Guided Multiverse Study of Neuroimaging Analyses.” Nature Communications 13 (1): 3758. https://doi.org/10.1038/s41467-022-31347-8.
Esteban, O., Markiewicz, C. J., Blair, R. W., Moodie, C. A., Isik, A. I., Erramuzpe, A., ... & Gorgolewski, K. J. (2019). fMRIPrep: a robust preprocessing pipeline for functional MRI. Nature methods, 16(1), 111-116.
Pruim, Raimon H.R., Maarten Mennes, Daan Van Rooij, Alberto Llera, Jan K. Buitelaar, and Christian F. Beckmann. 2015. “ICA-AROMA: A Robust ICA-Based Strategy for Removing Motion Artifacts from fMRI Data.” NeuroImage 112 (May):267–77. https://doi.org/10.1016/j.neuroimage.2015.02.064.
Schaefer, Alexander, Ru Kong, Evan M Gordon, Timothy O Laumann, Xi-Nian Zuo, Avram J Holmes, Simon B Eickhoff, and B T Thomas Yeo. 2018. “Local-Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI.” Cerebral Cortex 28 (9): 3095–3114. https://doi.org/10.1093/cercor/bhx179.
Van Schuerbeek, P., L. De Wandel, and C. Baeken. 2022. “The Optimized Combination of aCompCor and ICA-AROMA to Reduce Motion and Physiologic Noise in Task fMRI Data.” Biomedical Physics & Engineering Express 8 (5). https://doi.org/10.1088/2057-1976/ac63f0.
Waller, Lea, Susanne Erk, Elena Pozzi, Yara J. Toenders, Courtney C. Haswell, Marc Büttner, Paul M. Thompson, et al. 2022. “ENIGMA HALFpipe: Interactive, Reproducible, and Efficient Analysis for Resting-State and Task-Based fMRI Data.” Human Brain Mapping 43 (9): 2727–42. https://doi.org/10.1002/hbm.25829.