A comparative analysis of dMRI processing pipelines using a custom benchmark
Poster No:
1312
Submission Type:
Late-Breaking Abstract Submission
Authors:
Jelle Veraart1, Arnaud Bore2, Daan Christiaens3, Stefan Winzeck4, Kurt Schilling5, Luke Edwards6, Erpeng Dai7, Siawoosh Mohammadi8, Vladimir Golkov9, Jennifer McNab7, Bennett Landman5, Nikolaus Weiskopf6, Maxime Descoteaux10, Franco Pestilli11
Institutions:
1NYU Grossman School of Medicine, New York, NY, 2Sherbrooke University, Sherbrooke, Quebec, 3KU Leuven, Leuven, 4University of Cambridge, Cambridge, 5Vanderbilt University, Nashville, TN, 6Department of Neurophysics, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Saxony, 7Stanford University, Stanford, CA, 8Department of Systems Neuroscience, University Medical Center, Hamburg, Hamburg, 9Technical University of Munich, Munich, Germany, 10University of Sherbrooke, Sherbrooke, Quebec, 11The University of Texas at Austin, Austin, TX
First Author:
Co-Author(s):
Luke Edwards
Department of Neurophysics, Max Planck Institute for Human Cognitive and Brain Sciences
Leipzig, Saxony
Department of Neurophysics, Max Planck Institute for Human Cognitive and Brain Sciences
Leipzig, Saxony
Nikolaus Weiskopf
Department of Neurophysics, Max Planck Institute for Human Cognitive and Brain Sciences
Leipzig, Saxony
Department of Neurophysics, Max Planck Institute for Human Cognitive and Brain Sciences
Leipzig, Saxony
Late Breaking Reviewer(s):
Introduction:
Diffusion MRI (dMRI) is a neuroimaging modality to map the microstructure and structural connectivity of the living human brain. The current wealth of publicly-available dMRI data poses great promise for the study of the structure and function of the brain, across genders and in health and disease. However, making dMRI data usable for scientific inquiry requires processing the raw data to remove artifacts and distortions. The flexibility of researchers in the design, implementation, and use of processing pipelines for neuroimaging data has been shown to be a barrier to the validity and reproducibility of research findings. Today, benchmarks are needed to (a) advance the rigor and reproducibility of software tools for the analysis of neuroimaging data efficiently, and (b) accelerate and streamline the development of the new generation of tools. Here, we present a first-of-its-kind benchmark, specifically tailored to the data-driven evaluation of the reliability of image processing pipelines for dMRI data and present a comparative study of nine community-facing pipelines.
Methods:
Data: 13 research volunteers were scanned on one of three MRI scanners (GE, Siemens, and Philips) using an harmonized protocol, including: (a) T1-weighted MPRAGE image (b) field map (c) four sessions of multi-shell dMRI images and (d) a b=0 image with reversed phase encoding for each session. Diffusion was applied with b = 1000 and 2000 s/mm2 for 60 gradient directions. Imaging parameters (e.g. bandwidth and the polarity of gradients) were varied between the sessions to change the shape, magnitude, or direction of 12 imaging artifacts while preserving the encoded diffusion information.
Preprocessing pipelines: We processed all dMRI data, independently for each subject and session using 9 published and containerized pipelines using their default settings(Cai et al., 2021; Chen et al., 2024; Cieslak et al., 2021; Cruces et al., 2022; Dhiman et al., 2024; Glasser et al., 2013; Pierpaoli et al., 2010; Theaud et al., 2020; Tournier et al., 2019).
Test metrics: While no single session of the raw data is distortion-free, the agreement (or lack of variability) across subject-matched image series, before or after processing, provides a metric to assess the impact of imaging artifacts on common dMRI analysis endpoints. Therefore, in addition to image blur, we computed per subject the Coefficient of Variation (CoV) of commonly used diffusion metrics, coherence of fiber peaks, and anatomical alignment across the sessions.
Statistics: We quantified the performance of each pipeline relative to the minimal preprocessing pipeline of the HCP (HCP-MPP) using the hedges' g effect size.
Preprocessing pipelines: We processed all dMRI data, independently for each subject and session using 9 published and containerized pipelines using their default settings(Cai et al., 2021; Chen et al., 2024; Cieslak et al., 2021; Cruces et al., 2022; Dhiman et al., 2024; Glasser et al., 2013; Pierpaoli et al., 2010; Theaud et al., 2020; Tournier et al., 2019).
Test metrics: While no single session of the raw data is distortion-free, the agreement (or lack of variability) across subject-matched image series, before or after processing, provides a metric to assess the impact of imaging artifacts on common dMRI analysis endpoints. Therefore, in addition to image blur, we computed per subject the Coefficient of Variation (CoV) of commonly used diffusion metrics, coherence of fiber peaks, and anatomical alignment across the sessions.
Statistics: We quantified the performance of each pipeline relative to the minimal preprocessing pipeline of the HCP (HCP-MPP) using the hedges' g effect size.
Results:
Preprocessing pipelines reduce the CoV of common diffusion metrics across sessions, highlighting that such methods make the dMRI analysis more robust to imaging artifacts and more precise. A similar observation was made for peak divergence, which is critical for tractography. Brute-force approaches such as "smoothing" are outperformed by more tailored correction approaches. Compared to the HCP-MPP, recently developed pipelines show improved performance in terms of CoV and peak divergence. However, the performance varies across pipelines and, at times, at the expense of geometric consistency across sessions.
Conclusions:
Recent technical developments such as denoising and Gibbs ringing corrections are improving the reliability of dMRI data analysis and connectivity to imaging artifacts, when compared to the HCP-MPP. Moreover, advances in neuroinformatics (Containerization), standards (BIDS), and community guidelines (FAIR) promote the reliable use of advanced processing pipelines without the need for expert knowledge. However, the pipelines still vary significantly in their design, which results in variability in performance. Future work will include the development of recommendations for pipeline design following the interpretation of the benchmark comparisons.
Modeling and Analysis Methods:
Diffusion MRI Modeling and Analysis 1
Neuroinformatics and Data Sharing:
Workflows 2
Keywords:
Data analysis
Design and Analysis
Open-Source Software
WHITE MATTER IMAGING - DTI, HARDI, DSI, ETC
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
Provide references using APA citation style.
Chen, J. et al. (2024). Optimization and validation of the DESIGNER preprocessing pipeline for clinical diffusion MRI in white matter aging. Imaging Neuroscience, 2, 1–17.
Cieslak, M. et al (2021). QSIPrep: an integrative platform for preprocessing and reconstructing diffusion MRI data. Nature Methods, 18(7), 775–778.
Cruces, R. R. et al. (2022). Micapipe: A pipeline for multimodal neuroimaging and connectome analysis. NeuroImage, 263(119612), 119612.
Dhiman, S. et al. (2024). PyDesigner v1.0: A pythonic implementation of the DESIGNER pipeline for diffusion magnetic resonance imaging. Journal of Visualized Experiments: JoVE, 207. https://doi.org/10.3791/66397
Glasser, M. F. et al. (2013). The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage, 80, 105–124.
Pierpaoli, C. et al. (2010). TORTOISE: an integrated software package for processing of diffusion MRI data. ISMRM 18th Annual Meeting, 1597.