Saturday, Jul 22: 8:00 AM - 5:00 PM
1257
Educational Course - Full Day (8 hours)
Palais
Room: 510
The ever-increasing number of public datasets makes this a timely topic, given the wide variety of QC procedures applied to datasets before release, and lack of a standard way to describe any procedures that were performed. fMRI researchers generally agree that QC is important, and that reproducibility and reliability of study findings is unlikely if low-quality images are included in the analyses. But there is much less agreement on what criteria to use to determine the quality of a particular image or dataset, and how these criteria should vary across different types of analyses. Further, the terminology used to describe QC procedures and criteria varies across groups and software, and is often altogether omitted from manuscripts or released datasets. The FMRI Open QC Project and this educational course helps to fill this important gap by providing a common baseline for discussion: teams used their typical software tools to run QC on the same datasets, describing their procedures and criteria in detail, giving examples of acceptable images, and, importantly, examples of problematic ones. The goal was to create an open resource for discussing what are good, bad or questionable datasets, as well as procedures for performing QC in many currently available software packages.
Become comfortable assessing data quality from different FMRI processing pipelines and for different target analyses. This will be done by looking at and discussing examples of good, bad, and questionable data from the Open QC Project datasets.
Become familiar with some of the most commonly used FMRI QC tools currently available (including SPM, pyfMRIqc, MRIQC, fMRIprep, DPABI, Conn and AFNI).
Identify characteristics and descriptions of different quality control approaches, and how to clearly present them and discuss them with others.
Our target audience is anyone involved in fMRI studies (both task and resting state), trainee or experienced researcher, whether they are reading the literature, analyzing datasets from public repositories, or conducting complete studies. The QC Project examples are human datasets with single-echo acquisitions, but the techniques and principles apply directly to non-human data and other acquisition paradigms, as well.
Presentations
The monitoring and assessment of data quality is an essential step in the acquisition and analysis of functional MRI (fMRI) data. Various quality control (QC) metrics can determine what subjects to exclude from the group analyses, and can also guide additional processing steps that may be necessary. This presentation describes a combination of qualitative and quantitative assessments to determine the quality of fMRI data, particularly resting-state fMRI data used to estimate functional connectivity. Processing is performed using the AFNI data analysis package, but can in principle be implemented using any fMRI processing package. QC measures are evaluated at different steps in the processing pipeline to catch gross abnormalities in the data, determine deviations in acquisition parameters, evaluate the alignment to template space, determine the level of head motion, and detect other sources of noise. This presentation also shows the effect of different quantitative QC cutoffs, specifically the motion censoring threshold, and the impact of bandpass filtering. This analysis shows that while motion censoring reduces artifacts, overly stringent censoring can result in more noisy functional connectivity estimates particularly when combined with bandpass filtering. The qualitative and quantitative metrics presented here can provide information about what subjects to exclude and what subjects to examine more closely in the analysis of large datasets.
Presenter
Rasmus Birn, Ph.D., University of Wisconsin
Psychiatry
Madison, WI
United States
Several major software programs are commonly used for the processing and statistical analysis of fMRI data. Here, I describe QC procedures for resting-state and task fMRI data processing statistical parametric mapping (SPM). The goal is not only to identify and remove any issues or anomalies in the data, but also to ensure that the processing has been carried out correctly. I will illustrate each step using the data from fMRI Open quality control (QC) Project. I also show that simple steps such as skull stripping can improve coregistration between the functional and anatomical images.
Presenter
Xin Di, New Jersey Institute of Technology Newark, NJ
United States
This presentation will demonstrate QC for task datasets, with a particular focus on evaluating task presentation and performance. Many things can go wrong during task experiments, from an implementation error causing an unexpected trial randomization to confused participants not responding to the intended features of the stimuli, and identifying such situations as quickly as possible is essential for success. I will present open-source reports, created with R, AFNI, and knitr, which summarize quality-related features of both the task and fMRI aspects of the Open QC Project task dataset. The reports and their underlying tests are designed to let the reader quickly highlight potential issues, are pdf files for easy archiving, and require relatively little experience to use and adapt. Example reports showing various bugs or unusual participant response patterns will be shown, and the audience challenged to describe what happened.
Presenter
Jo Etzel, PhD, Washington University in St. Louis Saint Louis, MO
United States
Designing and executing a good quality control (QC) process is vital to robust and reproducible science and is often taught through hands on training. As FMRI research trends towards studies with larger sample sizes and highly automated processing pipelines, the people who analyze data are often distinct from those who collect and preprocess the data. While there are good reasons for this trend, it also means that important information about how data were acquired, and their quality, may be missed by those working at later stages of these workflows. Similarly, an abundance of publicly available datasets, where people (not always correctly) assume others already validated data quality, makes it easier for trainees to advance in the field without learning how to identify problematic data.
This presentation is designed as an introduction for researchers who are already familiar with fMRI, but who did not get hands on QC training or who want to think more deeply about QC. This could be someone who has analyzed fMRI data but is planning to personally acquire data for the first time, or someone who regularly uses openly shared data and wants to learn how to better assess data quality. We describe why good quality control (QC) processes are important, explain key priorities and steps for fMRI QC, and demonstrate some of these steps.
Background: Reliability of resting-state functional magnetic resonance imaging (rs-fMRI) can be improved by censoring or 'scrubbing' volumes affected by motion. While censoring preserves the integrity of participant-level data, including excessively censored data sets in group analyses may add noise. A variety of quality control (QC) approaches are employed to determine data quality and ultimately inclusion or exclusion of a fMRI data set in group analysis. In this talk, we describe our methods for performing QC of rs-fMRI data using software-generated quantitative and qualitative output and trained visual inspection. We will review our step-by-step process for quantitative data review of files and file structure as well as qualitative data review including documentation of visual inspection. Results: The data provided for this QC paper had relatively low motion-censoring, thus quantitative QC resulted in no exclusions. Qualitative checks of the data resulted in limited exclusions due to potential incidental findings and failed pre-processing scripts. Conclusion: Visual inspection in addition to the review of quantitative QC metrics improves the rigor, reliability, and accuracy of rs-fMRI data analysis.
Presenter
Rebecca Lepping, PhD, University of Kansas Medical Center
Neurology
Kansas City, KS
United States
During this presentation, we will provide a walkthrough of a quality control (QC) protocol using the CONN toolbox for the assessment of MRI and fMRI resting state data intended for functional connectivity analysis. We will demonstrate visual and automated QC procedures that can be employed for the evaluation of the data at different stages of the data life cycle including raw-level data, preprocessed, and denoised form. Discussions on how several of the modular and mutually non-exclusive QC procedures could be complemented and combined will be encouraged and supported with examples from the FMRI Open QC Project rest data collection. Altogether, QC testing will be discussed as a tool to become familiar with the data and aid the interpretation of the generated results. With this talk, we aim to contribute towards the dissemination of QC testing and QC reporting, with the hope that both would become an integral part of neuroimaging studies.
This presentation will demonstrate a protocol for the quality control (QC) of resting state and task functional MRI studies involving two checkpoints. The first checkpoint entails assessing the quality of the unprocessed data using MRIQC visual report. Second, for the data that surpassed this first checkpoint, the second checkpoint entails assessing the results of minimal preprocessing using the fMRIPrep visual reports. We will discuss why setting QC checkpoints at several steps of the preprocessing is important. Moreover, we will describe how the overall application scope (that is, the intended use of the data) determines how QC is carried out and defines the exclusion criteria for anatomical (T1-weighted; T1w) and functional (blood-oxygen dependent-level; BOLD) images at the two QC checkpoints accordingly. With this talk, we aim to promote best practices in QA/QC and help researchers implement their protocols for functional MRI more effectively.
Presenter
Céline Provins, Lausanne University Hospital and University of Lausanne
Radiology
Lausanne, Vaud
Switzerland
Quality control (QC) is a necessary, but often an under-appreciated, part of FMRI processing. Here we describe procedures for performing quality control on acquired or publicly available FMRI datasets using the widely used AFNI software package. This work is part of the Research Topic, "Demonstrating Quality Control (QC) Procedures in fMRI." We used a sequential, hierarchical approach that contained the following major stages: 1) getting to know your data, esp. its basic acquisition properties (GTKYD, 2) examining quantifiable measures, with thresholds (APQUANT), 3) viewing qualitative images, graphs and other information in systematic HTML reports (APQUAL) and 4) checking features interactively with a graphical user interface (GUI); and for task data, 5) checking stimulus event timing statistics (STIM). We describe how these are complementary and reinforce each other to help researchers stay close to their data. We show how to incorporate these steps within FMRI processing, evaluating the provided, publicly available resting state data collections (7 groups, 139 total subjects) and task-based data collection (1 group, 30 subjects). As specified within the Topic guidelines, each subject's dataset is placed into one of three categories: include, exclude or uncertain. The main focus of this work, however, is presenting a detailed description of QC procedures: how to understand the contents of an FMRI dataset, to check its contents for appropriateness, to verify processing steps, and to examine potential quality issues. Scripts for the processing and analysis are freely available.
The quality control (QC) of functional magnetic resonance imaging (fMRI) data is widely performed using a range of automated tools developed to aid data quality assessments. Yet, ultimately these tools still require one or more raters to make subjective decisions about the overall quality of a subject’s data, and these decisions can vary both within and between raters. Furthermore, there is little consensus about what constitutes good or poor quality data. One way this variability in decision making can be mitigated is by using a predefined QC protocol. To illustrate this point this lecture will present results from our fMRI Open QC Project, where multiple raters reviewed the same QC reports and were asked to make decisions about data quality using a predefined QC protocol. This lecture will include exemplar QC outputs that attendees will be asked to assess with and without a QC protocol for guidance (the results of these group assessments will be shared at the end of the lecture). Lastly, this lecture will highlight common QC issues and suggest how users could generate their own QC protocols for their preferred tool.
Presenter
Brendan Williams, University of Reading
School of Psychology and Clinical Language Sciences
Reading, Berkshire
United Kingdom
Quality control (QC) is an important stage for functional magnetic resonance imaging (fMRI) studies. The methods for fMRI QC vary in different fMRI preprocessing pipelines. The increasing sample size and the number of scanning sites for fMRI studies further add to the difficulty and working load of the QC procedure. In this lecture, we would like to illustrate the fMRI QC procedure in DPABI by preprocessing a well-organized open-available dataset. Six categories of DPABI-derived reports were used to eliminate participants without adequate imaging quality and the representative images with bad quality would be displayed. More validated automatic QC tools were needed in the big-data era while visually inspecting images was still indispensable now.
Presenter
Chao-Gan Yan, CAS Key Laboratory of Behavioral Science, Institute of Psychology Beijing, Beijing
China