Network Enrichment Testing using Ordinal Dominance Curves
Poster No:
1679
Submission Type:
Abstract Submission
Authors:
Noah Hillman1, Sarah Weinstein2, Joëlle Bagautdinova3,4, Kevin Sun3,4, Arielle Keller5,6, Aaron Alexander-Bloch3,7, Theodore Satterthwaite3,4, Haochang Shou1, Russell Shinohara1
Institutions:
1Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania, Philadelphia, PA, 2Department of Epidemiology and Biostatistics, Temple University, Philadelphia, PA, 3Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, 4Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania, Philadelphia, PA, 5Department of Psychological Sciences, University of Connecticut, Storrs, CT, 6Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, 7Department of Child and Adolescent Psychiatry and Behavioral Science, Children's Hospital of Philadelphia, Philadelphia, PA
First Author:
Noah Hillman
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
Philadelphia, PA
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
Philadelphia, PA
Co-Author(s):
Joëlle Bagautdinova
Department of Psychiatry, University of Pennsylvania|Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania
Philadelphia, PA|Philadelphia, PA
Department of Psychiatry, University of Pennsylvania|Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania
Philadelphia, PA|Philadelphia, PA
Kevin Sun
Department of Psychiatry, University of Pennsylvania|Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania
Philadelphia, PA|Philadelphia, PA
Department of Psychiatry, University of Pennsylvania|Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania
Philadelphia, PA|Philadelphia, PA
Arielle Keller, PhD
Department of Psychological Sciences, University of Connecticut|Institute for the Brain and Cognitive Sciences, University of Connecticut
Storrs, CT|Storrs, CT
Department of Psychological Sciences, University of Connecticut|Institute for the Brain and Cognitive Sciences, University of Connecticut
Storrs, CT|Storrs, CT
Aaron Alexander-Bloch
Department of Psychiatry, University of Pennsylvania|Department of Child and Adolescent Psychiatry and Behavioral Science, Children's Hospital of Philadelphia
Philadelphia, PA|Philadelphia, PA
Department of Psychiatry, University of Pennsylvania|Department of Child and Adolescent Psychiatry and Behavioral Science, Children's Hospital of Philadelphia
Philadelphia, PA|Philadelphia, PA
Theodore Satterthwaite
Department of Psychiatry, University of Pennsylvania|Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania
Philadelphia, PA|Philadelphia, PA
Department of Psychiatry, University of Pennsylvania|Penn Lifespan Informatics and Neuroimaging Center, University of Pennsylvania
Philadelphia, PA|Philadelphia, PA
Haochang Shou
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
Philadelphia, PA
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
Philadelphia, PA
Russell Shinohara
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
Philadelphia, PA
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania
Philadelphia, PA
Introduction:
Spatial maps from brain-wide association studies are common in neuroimaging analyses investigating relationships between cortical arealization and behavior. However, rigorous statistical methods are needed for testing whether brain-phenotype associations are enriched (i.e., more extreme) in particular regions of the brain. Here, we propose an intersection-union test based on ordinal dominance curves (ODC) which improves upon previous methods by properly controlling for the false positive rate for null in-network associations while adapting the test statistic to the underlying data to increase statistical power.
Methods:
Ordinal dominance curves are a nonparametric framework for examining stochastic ordering between two distributions and can be used to perform two-sample hypothesis testing (Ramdas et al., 2017). Our method jointly tests whether in-network associations are significantly different from zero and stochastically greater than out-of-network associations by comparing the mean of the in-network associations and the shifted ordinal dominance curve to null distributions generated by a Smith permutation procedure (Winkler et al., 2014). We make our test statistic adaptive by multiplying the shifted ordinal dominance curve by a step function which can selectively upweight higher quantiles of the association distribution to better detect variance shifts between in-network and out-of-network locations. To validate our method, we use data from the baseline cohort of the Adolescent Brain Cognitive Development (ABCD) study, obtained using the NIMH Data Archive. Scores on three cognitive domains were derived from a Bayesian probabilistic principal component analysis model applied to nine tasks from a neurocognitive battery (Thompson et al., 2019). Plasmode simulations were conducted where simulated cortical thickness values were generated as a linear combination of two nuisance variables, a covariate of interest, and spatially correlated residuals derived from randomly sampling rescaled cortical thickness values in the ABCD dataset. We applied our method to the ABCD sample, where we tested for enrichment in the association between cortical thickness and three neurocognitive scores. Enrichment testing was performed in three functional networks which ranked highly on a sensorimotor-association axis (Keller et al., 2023), as we hypothesized that cortical thinning in association cortex would be correlated with higher neurocognitive scores.
Results:
Simulation studies confirmed that previous methods all suffered from significantly inflated false positive rates when in-network coefficients were zero or identically distributed to the out-of-network coefficients, while ODC controlled the type I error rate at the nominal level (Fig. 1a-b). ODC had lower power than other methods for detecting a mean shift between the in-network and out-of-network coefficients but demonstrated superior performance in detecting variance shifts (Fig. 1d-e). When applying ODC to the ABCD dataset, enrichment was present in the ventral attention network for all three neurocognitive domains (Fig. 2a-c). Enrichment tests were also significant in two cognitive domains – general cognition and learning/memory – for the frontoparietal network, while for the default mode network cortical thickness was only differentially associated with learning/memory scores. All significant tests detected enrichment in the negative direction.
Conclusions:
Previous methods for enrichment testing can lead to inflated false positive rates due to the spatial structure present in neuroimaging data. We proposed a nonparametric intersection-union test to address this issue, which correctly controlled the Type I error rate while maintaining high statistical power. Applying our method to the ABCD study, we found significant evidence of enrichment in association cortex for the relationship between cortical thickness and neurocognition.
Higher Cognitive Functions:
Executive Function, Cognitive Control and Decision Making
Reasoning and Problem Solving
Modeling and Analysis Methods:
Methods Development 2
Univariate Modeling 1
Keywords:
Cognition
Cortex
Data analysis
Modeling
MRI
Statistical Methods
STRUCTURAL MRI
Univariate
1|2Indicates the priority used for review
Abstract Information
By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.
The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was this research conducted in the United States?
Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.
Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using APA citation style.
2. Kim, SY (2005). PAGE: Parametric Analysis of Gene Set Enrichment. BMC Bioinformatics, 6(1), 144.
3. Koopmans, F. (2024). GOAT: Efficient and robust identification of gene set enrichment. Communications Biology, 7(1), 1–9.
4. Maleki, F. (2020). Gene Set Analysis: Challenges, Opportunities, and Future Research. Frontiers in Genetics, 11.
5. Ramdas, A. (2017). On Wasserstein Two-Sample Testing and Related Families of Nonparametric Tests. Entropy, 19(2), 47.
6. Thompson, W. K. (2019). The structure of cognition in 9 and 10 year-old children and associations with problem behaviors: Findings from the ABCD study’s baseline neurocognitive battery. Developmental Cognitive Neuroscience, 36, 100606.
7. Weinstein, S. M. (2024). Network enrichment significance testing in brain–phenotype association studies. Human Brain Mapping, 45(8), e26714.
8. Winkler, A. M. (2014). Permutation inference for the general linear model. Neuroimage, 92(100), 381-397.