Big Data, Small Bias: Harmonizing Structural Connectomes to Mitigate Site Bias in Data Integration
Poster No:
3274
Submission Type:
Abstract Submission
Authors:
Rui Shen1, Drew Parker1, Andrew Chen2, Benjamin Yerys3, Birkan Tunç3, Timothy Roberts3, Russell Shinohara1, Ragini Verma1
Institutions:
1University of Pennsylvania, Philadelphia, PA, 2Medical University of South Carolina, Charleston, SC, 3Children’s Hospital of Philadelphia, Philadelphia, PA
First Author:
Co-Author(s):
Introduction:
Structural connectomes are commonly used to investigate connectivity changes related to various disorders. However, small sample sizes in individual studies and highly heterogeneous disorder-related manifestations underscore the need to pool datasets across multiple studies to identify coherent and generalizable patterns linked to disorders. Yet, combining datasets introduces site bias due to variations in scanner hardware or acquisitions. This highlights the necessity for data harmonization to mitigate site bias while preserving the biological integrity associated with participant demographics and the disorders. While several paradigms exist for harmonizing normally distributed imaging data, this study represents the first effort to establish a harmonization framework specifically for structural connectomes.
Methods:
Common harmonization methods such as ComBat and CovBat assume a normal distribution and are therefore unsuitable for structural connectomes, where most edges (defined by streamline counts) are highly skewed. We explored several statistical models to develop a tailored framework specifically for structural connectomes. We pooled structural connectomes from 6 datasets and created 4 data configurations by combining the cohorts in various ways. A total of 1503 participants (890 males, 613 females) were involved, comprising 1194 neurotypicals (NT) and 309 with autism. Each dataset is detailed in Fig.1A.
Firstly, we applied a logarithmic transformation to skewed edges before ComBat/CovBat and restored harmonized weights with exponentiation, namely log-ComBat and log-CovBat. Alternatively, we modeled edge values using a gamma-distributed generalized linear model (gamma-GLM) with a log link, incorporating site, sex, age, and age^2 as covariates.
Fig.1B shows the overview of the evaluation framework. Harmonization is considered successful if it removed site effects at edge-, node-, and global levels while preserving the biological variability. We used the Kruskal-Wallis test to identify edgewise site effects before and after harmonization. One-way ANOVA test was used to evaluate site effects in 6 global graph measures and 4 nodal measures. Sex, age, and age^2 were controlled. Moreover, we assessed the replicability of Spearman correlations between age and edge values before and after harmonization. We also evaluated the case in presence of substantial confounds between age and sites. Finally, we assessed the ability of harmonization in enhancing the generalizability of machine learning models to new sites and increasing statistical power for detecting group differences between autism and NT.
Firstly, we applied a logarithmic transformation to skewed edges before ComBat/CovBat and restored harmonized weights with exponentiation, namely log-ComBat and log-CovBat. Alternatively, we modeled edge values using a gamma-distributed generalized linear model (gamma-GLM) with a log link, incorporating site, sex, age, and age^2 as covariates.
Fig.1B shows the overview of the evaluation framework. Harmonization is considered successful if it removed site effects at edge-, node-, and global levels while preserving the biological variability. We used the Kruskal-Wallis test to identify edgewise site effects before and after harmonization. One-way ANOVA test was used to evaluate site effects in 6 global graph measures and 4 nodal measures. Sex, age, and age^2 were controlled. Moreover, we assessed the replicability of Spearman correlations between age and edge values before and after harmonization. We also evaluated the case in presence of substantial confounds between age and sites. Finally, we assessed the ability of harmonization in enhancing the generalizability of machine learning models to new sites and increasing statistical power for detecting group differences between autism and NT.
·Overview of the harmonization and evaluation framework for structural connectomes
Results:
We observed striking site effects in edgewise connectivity values before harmonization, which were largely reduced after harmonization. The gamma-GLM model outperformed other methods (Fig.2A). Specifically, 1035 (70%) edges initially showed significant site effects pre-harmonization. None of them remained significant after gamma-GLM.
All 6 global measures showed significant site effects pre-harmonization. After ComBat, site effects on characteristic path length and global efficiency remained significant. All other methods addressed site effects on tested global measures, while gamma-GLM showed the best performance, yielding the smallest effect sizes (Fig.2B).
The gamma-GLM performed the best in the replicability of age associations at each site, showing an almost flat CAT curve near 1 and recovered all correlations between age and edgewise connectivity. In the presence of confounds, the performance of gamma-GLM remained robust (Fig.2C).
In two use cases, we showed gamma-GLM effectively enhanced the generalizability of predictive models to unseen data at new sites and resolved significant differences in structural connectomes between diagnostic groups that were previously undetectable.
All 6 global measures showed significant site effects pre-harmonization. After ComBat, site effects on characteristic path length and global efficiency remained significant. All other methods addressed site effects on tested global measures, while gamma-GLM showed the best performance, yielding the smallest effect sizes (Fig.2B).
The gamma-GLM performed the best in the replicability of age associations at each site, showing an almost flat CAT curve near 1 and recovered all correlations between age and edgewise connectivity. In the presence of confounds, the performance of gamma-GLM remained robust (Fig.2C).
In two use cases, we showed gamma-GLM effectively enhanced the generalizability of predictive models to unseen data at new sites and resolved significant differences in structural connectomes between diagnostic groups that were previously undetectable.
·Performance of harmonization models
Conclusions:
We recommend the gamma-GLM to harmonize structural connectomes, as it outperformed other models in reducing site bias while preserving biological integrity.
Modeling and Analysis Methods:
Connectivity (eg. functional, effective, structural) 1
Diffusion MRI Modeling and Analysis
Methods Development 2
Keywords:
Autism
Computational Neuroscience
Data analysis
Machine Learning
Modeling
MRI
Tractography
WHITE MATTER IMAGING - DTI, HARDI, DSI, ETC
1|2Indicates the priority used for review