The impact of numerical variability in fMRI preprocessing on regional and network statistics
Poster No:
1857
Submission Type:
Abstract Submission
Authors:
Mina Alizadeh1, Yohan Chatelain1, Gregory Kiar2, Tristan Glatard3
Institutions:
1Concordia University, Montreal, Quebec, 2Center for Data Analytics, Innovation, and Rigor, Child Mind Institute, NYC, NY, 3Krembil Centre for Neuroinformatics, Toronto, Ontario
First Author:
Co-Author(s):
Introduction:
The complexity and volume of neuroimaging data necessitate the use of sophisticated computational tools. However, reliance on automated workflows raises concerns about numerical stability, where cascading floating-point errors can significantly impact results. Sequential floating-point operations may amplify small variations, undermining reproducibility and validity. Stability tests are crucial to ensure results remain within acceptable variation limits. Prior studies revealed that structural connectome estimation pipelines are sensitive to machine-level noise, leading to variability that challenges the reliability of individual network features (Kiar, 2021). However, numerical instabilities in functional connectome analyses remain underexplored.
This study evaluates the numerical reliability of fMRI analyses using Monte Carlo Arithmetic (MCA) (Parker, 1997). By introducing controlled numerical perturbations into the fMRIPrep pipeline (Esteban, 2019), we assess the effects of these perturbations on functional connectomes and derived graph metrics.
This study evaluates the numerical reliability of fMRI analyses using Monte Carlo Arithmetic (MCA) (Parker, 1997). By introducing controlled numerical perturbations into the fMRIPrep pipeline (Esteban, 2019), we assess the effects of these perturbations on functional connectomes and derived graph metrics.
Methods:
MCA can be used to quantify numerical uncertainty by performing numerical perturbations to floating-point operations.
We integrated Fuzzy-libmath, a Verificarlo-compiled version of the libmath library (Denis, 2016), with the fMRIPrep pipeline. The perturbed pipeline was executed 10 times for each of 10 publicly available subjects (OpenNeuro, 2024), generating floating-point error distributions that can reveal sensitivity to numerical perturbations. The virtual precision was set to simulate frequent machine-level floating-point errors. Graph-based metrics, including local (degree centrality, betweenness centrality, eigenvector centrality, clustering coefficient) and global metrics (small-worldness, average shortest path length), were derived from functional connectomes. The standard deviation across subjects and MCA repetitions was computed for each metric.
We integrated Fuzzy-libmath, a Verificarlo-compiled version of the libmath library (Denis, 2016), with the fMRIPrep pipeline. The perturbed pipeline was executed 10 times for each of 10 publicly available subjects (OpenNeuro, 2024), generating floating-point error distributions that can reveal sensitivity to numerical perturbations. The virtual precision was set to simulate frequent machine-level floating-point errors. Graph-based metrics, including local (degree centrality, betweenness centrality, eigenvector centrality, clustering coefficient) and global metrics (small-worldness, average shortest path length), were derived from functional connectomes. The standard deviation across subjects and MCA repetitions was computed for each metric.
Results:
Considerable variability was observed in coregistration and N4BiasFieldCorrection steps across the 10 MCA repetitions. Inverse transformations from atlas to subject space, extended to BOLD space, assessed variability in transformation matrices. Dice coefficients were computed for each region across MCA runs. Regions with lower average Dice coefficients (Figure 1, indicated by dark red) were more frequent in BOLD space than in subject space, highlighting increased variability in coregistration than template normalization. Additionally, variability in derived functional connectomes was measured for each MCA run. Numerical variability was generally lower than cross-subject variability, indicating the pipeline preserved individual differences. However, certain subjects exhibited numerical variability approaching cross-subject levels, indicating that numerical instability can undermine the reliability of individual functional connectomes (Figure 2). Variability in graph metrics supported these findings, where across-subject variability consistently exceeded numerical variability, with some subjects as outliers to this trend.
·Figure 1. The Minimum Dice scores across MCA run pairs, averaged across subject for each region, for the registered atlas in subject space (top) and BOLD space (bottom).
· Figure 2. The standard deviation of functional connectome across subjects (top) and MCA runs (bottom) was computed, highlighting between subject and numerical variability.
Conclusions:
Introducing elementary numerical perturbations in the fMRIPrep pipeline introduced variability in functional connectomes and graph metrics, ranging from negligible to significant. Consistent with prior studies, this variability depends on the dataset and tools used, underscoring the need for tailored stability analyses (Kiar, 2021). Variability in ROI definitions revealed more pronounced fluctuations in the functional domain than in the structural domain. Further validation across diverse datasets is necessary to ensure generalizability.
The several outliers observed in our results suggest that automated quality control (QC) could be a potential application of MCA perturbations, wherein subjects with high variability could be flagged for QC failure.
The several outliers observed in our results suggest that automated quality control (QC) could be a potential application of MCA perturbations, wherein subjects with high variability could be flagged for QC failure.
Modeling and Analysis Methods:
fMRI Connectivity and Network Modeling 2
Neuroinformatics and Data Sharing:
Workflows 1
Keywords:
Computational Neuroscience
FUNCTIONAL MRI
Workflows
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
Provide references using APA citation style.
Esteban, O., Markiewicz, C. J., Blair, R. W., et al. (2019). fMRIPrep: A robust preprocessing pipeline for functional MRI. Nature Methods, 16(2), 111–116. https://doi.org/10.1038/s41592-018-0235-4
Kiar, G., Chatelain, Y., de Oliveira Castro, P., Petit, E., Rokem, A., Varoquaux, G., et al. (2021). Numerical uncertainty in analytical pipelines leads to impactful variability in brain networks. PLoS ONE, 16(11), e0250755. https://doi.org/10.1371/journal.pone.0250755
Parker, D. S. (1997). Monte Carlo Arithmetic: Exploiting Randomness in Floating-Point Arithmetic. Computer Science Department, University of California.
Perchtold-Stefan, C., Rominger, C., Koschutnig, K., & Fink, A. (2024). Truecrime. OpenNeuro. [Dataset] https://doi.org/10.18112/openneuro.ds004965.v1.0.1