Digging Deeper into the Pervasive Problem of Non-Compliance in MR datasets
Presented During:
Monday, June 24, 2024: 5:45 PM - 7:00 PM
COEX
Room:
Grand Ballroom 103
Poster No:
2230
Submission Type:
Abstract Submission
Authors:
Harsh Sinha1, Pradeep Raamana1
Institutions:
1University of Pittsburgh, Pittsburgh, PA
First Author:
Co-Author:
Introduction:
Large-scale neuroimaging datasets are vital for brain-behavior studies, but the reliability of statistical results depends on its data quality. Therefore, protocol compliance becomes indispensable, emphasizing the need for accurate data acquisition for each subject across sites and scanners. Manual protocol compliance is impractical especially for massive datasets, necessitating an automated approach for minimizing non-compliance.
We have demonstrated the pervasive lack of compliance in large-scale datasets [1] using our open-source tool mrQA, revealing a substantial non-compliance rate of up to 60%, even though the initial exploration focused on a limited subset of parameters.
mrQA can now inspect many more parameters to generate a comprehensive compliance report. Apart from ensuring that all subjects were acquired accurately for each sequence (horizontal audit), mrQA also checks if related sequences acquired within a session are compatible with each other (vertical audit) as shown in Figure 1. With the integration of deeper checks with additional parameters, it becomes apparent that more issues may emerge, emphasizing the need for rigorous monitoring practices. We also explore patterns of non-compliance across scanner vendors, models, and sites such that appropriate strategies can be adopted to minimize such issues at MR imaging centers.
We have demonstrated the pervasive lack of compliance in large-scale datasets [1] using our open-source tool mrQA, revealing a substantial non-compliance rate of up to 60%, even though the initial exploration focused on a limited subset of parameters.
mrQA can now inspect many more parameters to generate a comprehensive compliance report. Apart from ensuring that all subjects were acquired accurately for each sequence (horizontal audit), mrQA also checks if related sequences acquired within a session are compatible with each other (vertical audit) as shown in Figure 1. With the integration of deeper checks with additional parameters, it becomes apparent that more issues may emerge, emphasizing the need for rigorous monitoring practices. We also explore patterns of non-compliance across scanner vendors, models, and sites such that appropriate strategies can be adopted to minimize such issues at MR imaging centers.
Methods:
mrQA parses DICOM files from the input dataset to store the most comprehensive acquisition information, and then summarizes issues of non-compliance in a user-friendly report. We assess the protocol compliance and patterns of non-compliance, in the large open Adolescent Brain Cognitive Development (ABCD) dataset [2,4]. For the vertical audit, we focus on field maps for DWI as they play a crucial role in distortion correction [3].
Results:
The results demonstrate a lack of compliance in coil and pixel spacing as shown in Table 1. We also observe significant differences in non-compliance rates across vendors, scanner models & sites.
We observed that scans in the ABCD dataset have been acquired with different coils as the choice of using either a 32-ch (HEA, HEP) or a 64-ch coil (HC) was determined by their availability at respective sites [2]. A few subjects were also scanned using body coil (BC) and spine coil (SP). Identifying scans with 32-ch/64-ch coils is important, so that appropriate measures are taken to adjust for coil differences.
A total of 217 subjects had a non-compliant shim setting, meaning that the shim values were not identical for the field map and the corresponding DWI, which can lead to suboptimal distortion correction [3]. Some subjects had a non-complaint PED for both the field map and the DWI, which we speculate may have been due to manual propagation of the acquisition information from prior sequences (field map) to the latter ones (DWI) at scanning interface. Although, this doesn't impede distortion correction, it may cause differences in fractional anisotropy estimates.
Certain scanner models (Figure 2) and acquisition sites (Figure 3) exhibited higher levels of non-compliance rates, indicating the influence of scanner & site-specific factors. These patterns highlight the need for automated tools that can identify non-compliance across vendors and sites. mrQA can be used as a continuous monitoring tool e.g., hourly or daily, to promptly catch non-compliance and minimize the number of non-compliant sessions for a given project.
We observed that scans in the ABCD dataset have been acquired with different coils as the choice of using either a 32-ch (HEA, HEP) or a 64-ch coil (HC) was determined by their availability at respective sites [2]. A few subjects were also scanned using body coil (BC) and spine coil (SP). Identifying scans with 32-ch/64-ch coils is important, so that appropriate measures are taken to adjust for coil differences.
A total of 217 subjects had a non-compliant shim setting, meaning that the shim values were not identical for the field map and the corresponding DWI, which can lead to suboptimal distortion correction [3]. Some subjects had a non-complaint PED for both the field map and the DWI, which we speculate may have been due to manual propagation of the acquisition information from prior sequences (field map) to the latter ones (DWI) at scanning interface. Although, this doesn't impede distortion correction, it may cause differences in fractional anisotropy estimates.
Certain scanner models (Figure 2) and acquisition sites (Figure 3) exhibited higher levels of non-compliance rates, indicating the influence of scanner & site-specific factors. These patterns highlight the need for automated tools that can identify non-compliance across vendors and sites. mrQA can be used as a continuous monitoring tool e.g., hourly or daily, to promptly catch non-compliance and minimize the number of non-compliant sessions for a given project.
Conclusions:
We have demonstrated that the problem of non-compliance is pervasive in MR imaging and report deeper patterns of non-compliance through vertical audits. Automated tools (such as mrQA) are required to minimize non-compliance that can directly interface with DICOM files from a scanner and conduct comprehensive horizontal and vertical audits. Furthermore, acquired images should be monitored for compliance on a daily basis, rather than the current practice of producing years' worth of non-compliant data, that is a very costly reacquire.
Neuroinformatics and Data Sharing:
Databasing and Data Sharing 1
Informatics Other 2
Keywords:
Acquisition
Data analysis
Informatics
MRI
Open-Source Software
Other - Protocol Compliance
1|2Indicates the priority used for review
Provide references using author date format
2. Hagler Jr, D. J., Hatton, S., Cornejo, M. D., Makowski, C., Fair, D. A., Dick, A. S., ... & Dale, A. M. (2019). Image processing and analysis methods for the Adolescent Brain Cognitive Development Study. Neuroimage, 202, 116091.
3. Wang, S., Peterson, D. J., Gatenby, J. C., Li, W., Grabowski, T. J., & Madhyastha, T. M. (2017). Evaluation of field map and nonlinear registration methods for correction of susceptibility artifacts in diffusion MRI. Frontiers in neuroinformatics, 11, 17.
4. Casey, B. J., Cannonier, T., Conley, M. I., Cohen, A. O., Barch, D. M., Heitzeg, M. M., ... & Dale, A. M. (2018). The adolescent brain cognitive development (ABCD) study: imaging acquisition across 21 sites. Developmental cognitive neuroscience, 32, 43-54.