Print Close

Parameter Reproducibility and Agreement Across Large Multi-center Study diffusion MR Protocols

Poster No:

1305

Submission Type:

Abstract Submission

Authors:

Jamie Wren-Jarvis¹^,2^,3, Valentin Stepanov¹^,3, Benjamin Ades-Aron⁴^,3, Jenny Chen¹^,3, Subah Mehrin¹^,3, Santiago Coelho¹^,3, Ricardo Coronado Leija¹^,3, Dmitry Novikov¹^,3, Jelle Veraart¹^,3, Els Fieremans¹^,3

Institutions:

¹Department of Radiology, NYU Grossman School of Medicine, New York, NY, ²Vilcek Institute of Graduate Biomedical Sciences, New York University Grossman School of Medicine, New York, NY, ³Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine, New York, NY, ⁴New York University Langone Health, New York, NY

First Author:

Jamie Wren-Jarvis, MSc
Department of Radiology, NYU Grossman School of Medicine|Vilcek Institute of Graduate Biomedical Sciences, New York University Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY|New York, NY

Co-Author(s):

Valentin Stepanov, MD
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Benjamin Ades-Aron, PhD
New York University Langone Health|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Jenny Chen, MS
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Subah Mehrin
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Santiago Coelho, PhD
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Ricardo Coronado Leija, PhD
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Dmitry Novikov, PhD
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Jelle Veraart
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Els Fieremans, PhD
Department of Radiology, NYU Grossman School of Medicine|Center for Advanced Imaging Innovation and Research (CAI2R), Department of Radiology, New York University Grossman School of Medicine
New York, NY|New York, NY

Introduction:

Pooling multi-center diffusion MRI (dMRI) datasets is a practical way to increase power and improve cohort diversity (Oh, et al., 2015). However, non-biological variability, driven by cross-scanner and cross-protocol differences, reduces sensitivity of pooled analyses (Ning et al., 2020). This study introduces a unique dataset consisting of healthy volunteers scanned with a variety of multishell dMRI protocols that match publicly available large-scale studies and are compatible with many modeling strategies. This dataset provides a unique resource to (a) quantify experimental variability of diffusion metrics relative to protocol design, (b) train and validate harmonization tools, and (c) identify critical scan parameters for harmonization of prospective data collections. As a preliminary analysis of this dataset, we quantify here the signal-to-noise ratio (SNR) of each protocol, and the between-protocol variability, relative to test/retest variability, of DTI metrics.

Methods:

Data: Eighteen healthy volunteers (11 Male, average age 32.3 ± 11.8 years) underwent brain dMRI on a 3T Siemens Prisma MR scanner after signing consent. Each scan included 7 dMRI protocols, all listed in Table 1. MESO refers to an in-house protocol that was used to scan 8000 clinical patients and controls. The MESO protocol (TE=96ms) was repeated twice to assess test/test variability.
Preprocessing: All data were preprocessed using the DESIGNER pipeline (Chen et al., 2024). As part of the pipeline, the noise map of the data is computed using MPPCA. The SNR is defined as the ratio between the non-diffusion-weighted image and the noise map. DTI estimation using b=0 and b=1000 s/mm² (1500 s/mm² for the HCPa protocol) was performed to extract MD and FA maps.
Analysis: Per subject and per protocol, we perform long-tract profiling of the corresponding MD and FA using TractSeg; N=100 segments per tract. Analysis is performed in the standard MNI space to minimize variability due to misalignment of tract segments.
Statistics: We compute (a) the agreement across protocols using the concordance correlation coefficient (CCC) and (b) variability across subjects, sessions, or protocols using the coefficient of variation (CoV).

Supporting Image: table_1_JWJ_ohbm_2025_v2.png

Results:

Table 1 lists the median SNR within the white matter alongside all acquisition settings for each protocol. The SNR varies from 20.6 to 40.3, with voxel size, echo time, image acceleration, and head coil selection being critical factors in such differences. In Figure 1(a,c), we show inter-subject, test-retest, and inter-protocol variability across the corticospinal tract (CST). Figure 1(b,d) shows heatmaps of the pairwise CoV and CCC values between protocols and the test/retest measure of FA and MD within the right CST, limited to the 25-75 segments. The results are replicated in the left CSF, not shown.

Supporting Image: figure_1_JWJ_ohbm_2025_v2.png

Conclusions:

By scanning volunteers with various commonly used dMRI protocols, we created a benchmark for assessing consistency in diffusion metrics across dMRI protocols. Inter-protocol variability exceeds test/retest variability for both DTI metrics evaluated, and is thus a notable source of variability in pooled studies, though to a varying extent. While test/retest CoV is below 1%, the CoV between protocols is < 2.6% for all paired protocols except for HCPa. HCPa does not include the commonly used b-value of 1000 s/mm², resulting in CoVs as high as 15% for MD, and up to 3.5% for FA. Beyond b-value, we observe that TE is a driving factor in inter-protocol variability, but further analysis is warranted to quantify the impact of all acquisition settings listed in Table 1. Overall, we conclude that advanced preprocessing lowers inter-protocol variability, but further data harmonization might be necessary to mitigate batch effects. Future work aims to further explore inter-protocol differences and use this dataset to develop guidelines and standards for data pooling and harmonization in dMRI studies.

Modeling and Analysis Methods:

Diffusion MRI Modeling and Analysis ¹

Methods Development ²

Neuroinformatics and Data Sharing:

Workflows

Novel Imaging Acquisition Methods:

Diffusion MRI

Keywords:

Acquisition

Data analysis

Open Data

White Matter

Workflows

^1|2Indicates the priority used for review

Abstract Information

By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.

I accept

The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:

I do not want to participate in the reproducibility challenge.

Please indicate below if your study was a "resting state" or "task-activation” study.

Other

Healthy subjects only or patients (note that patient studies may also involve healthy subjects):

Healthy subjects

Was this research conducted in the United States?

Yes

Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.

Yes, I have IRB or AUCC approval

Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.

Yes

Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.

Not applicable

Please indicate which methods were used in your research:

Diffusion MRI

For human MRI, what field strength scanner do you use?

3.0T

Which processing packages did you use for your study?

FSL

Other, Please list - ANTs, TractSeg, MRtrix3

Provide references using APA citation style.

1. Oh, S.S., Galanter, J., Thakur, N., Pino-Yanes, M., Barcelo, N.E., White, M.J., De Bruin, D.M., Greenblatt, R.M., Bibbins-Domingo, K., Wu, A.H. and Borrell, L.N. (2015). Diversity in clinical and biomedical research: a promise yet to be fulfilled. PLoS medicine, 12(12), e1001918.
2. Ning, L., Bonet-Carne, E., Grussu, F., Sepehrband, F., Kaden, E., Veraart, J., Blumberg, S.B., Khoo, C.S., Palombo, M., Kokkinos, I. and Alexander, D.C. (2020). Cross-scanner and cross-protocol multi-shell diffusion MRI data harmonization: Algorithms and results. Neuroimage, 221, 117128.
3. Chen, J., Ades-Aron, B., Lee, H.H., Mehrin, S., Pang, M., Novikov, D.S., Veraart, J. and Fieremans, E. (2024). Optimization and validation of the DESIGNER preprocessing pipeline for clinical diffusion MRI in white matter aging. Imaging Neuroscience, 2, 1-17.
4. Miller, K.L., Alfaro-Almagro, F., Bangerter, N.K., Thomas, D.L., Yacoub, E., Xu, J., Bartsch, A.J., Jbabdi, S., Sotiropoulos, S.N., Andersson, J.L. and Griffanti, L. (2016). Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nature neuroscience, 19(11), 1523-1536.
5. Weiner, M.W., Veitch, D.P., Aisen, P.S., Beckett, L.A., Cairns, N.J., Green, R.C., Harvey, D., Jack Jr, C.R., Jagust, W., Morris, J.C. and Petersen, R.C. (2017). The Alzheimer's Disease Neuroimaging Initiative 3: Continued innovation for clinical trial improvement. Alzheimer's & Dementia, 13(5), 561-571.
6. Casey, B.J., Cannonier, T., Conley, M.I., Cohen, A.O., Barch, D.M., Heitzeg, M.M., Soules, M.E., Teslovich, T., Dellarco, D.V., Garavan, H. and Orr, C.A. (2018). The adolescent brain cognitive development (ABCD) study: imaging acquisition across 21 sites. Developmental cognitive neuroscience, 32, 43-54.
7. Bookheimer S.Y., Salat D.H., Terpstra M., Ances B.M., Barch D.M., Buckner R.L., Burgess G.C., Curtiss S.W., Diaz-Santos M., Elam J.S., Fischl B., Greve D.N., Hagy H.A., Harms M.P., Hatch O.M., Hedden T., Hodge C., Japardi K.C., Kuhn T.P., Ly T.K., Smith S.M., Somerville L.H., Uğurbil K., van der Kouwe A., Van Essen D., Woods R.P., Yacoub E. (2019). The lifespan human connectome project in aging: an overview. Neuroimage, 185, 335-348.

UNESCO Institute of Statistics and World Bank Waiver Form

I attest that I currently live, work, or study in a country on the UNESCO Institute of Statistics and World Bank List of Low and Middle Income Countries list provided.