3. Cross-ethnicity/race generalization failure of RSFC-based behavioral prediction and potential downstream consequences

COEX 

In neuroimaging, a recent, important line of research is to predict behavioral phenotypes from neuroimaging data, e.g. resting-state functional connectivity (RSFC). However, algorithmic unfairness that favors certain subpopulations over others was uncovered in many other machine learning applications but not yet in the application to the neuroimaging field. The risk is high because the predictive models in this field were typically built using large cohorts with mixed ethnic groups, where certain groups, e.g. African Americans (AA), only occupied a very limited proportion. Here, we investigated the cross-ethnicity/race generalizability of the current, field-standard behavioral prediction approach using two large-scale public datasets from the United States: the Human Connectome Project – Young Adults and the Adolescent Brain Cognitive Development cohort. Specifically, prediction errors in AA were much larger than in white Americans (WA) for most behavioral measures. Concerns were raised when looking into the direction of prediction errors. For example, African pre-adolescent participants were more easily overpredicted in social problems, rule-breaking, and aggressive behaviors compared to white participants, leading to a higher false positive rate for AA if such models were directly deployed to diagnose mental disorders. Furthermore, we wondered if training population composition was the main reason for the bias. Therefore, we compared predictive models trained specifically on AA, specifically on WA, or on a mixture of AA and WA with equal sizes. Specific training on AA only helped to slightly reduce the biases against AA, but most of the biases remained. Other possible sources of the biases such as neuroimaging preprocessing (e.g., brain templates and functional atlases) and the design of behavioral measures need to be examined in the future. Our recent follow-up study further discovered a broad association between prediction error amplitudes with all ethnic groups in the datasets beyond the WA-AA comparison, highlighting the severity of this issue.