Poster No:
544
Submission Type:
Abstract Submission
Authors:
Martin Gell1,2, Mauricio Hoffmann3,4, Tyler Moore5, Aki Nikolaidis6, Ruben Gur5, Giovanni Salum4,7,8, Michael Milham6,9, Simon Eickhoff10,2, Robert Langner10,2, Veronika Müller10,2, Theodore Satterthwaite5
Institutions:
1Department of Psychiatry, Psychotherapy, and Psychosomatics, RWTH Aachen University Hospital, Aachen, Germany, 2Research Centre Juelich, Institute of Neuroscience and Medicine (INM7), Juelich, Germany, 3Department of Neuropsychiatry, Universidade Federal de Santa Maria, Santa Maria, Brazil, 4Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, 5University of Pennsylvania, Philadelphia, PA, 6Child Mind Institute, New York, NY, 7National Institute of Developmental Psychiatry for Children and Adolescents (INCT-CNPq), São Paulo, Brazil, 8Child Mind Institute, New York, United States, 9Nathan S. Kline Institute for Psychiatric Research, Orangeburg, NY, 10Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf, Düsseldorf, North Rhine-Westphalia
First Author:
Martin Gell
Department of Psychiatry, Psychotherapy, and Psychosomatics, RWTH Aachen University Hospital|Research Centre Juelich, Institute of Neuroscience and Medicine (INM7)
Aachen, Germany|Juelich, Germany
Co-Author(s):
Mauricio Hoffmann
Department of Neuropsychiatry, Universidade Federal de Santa Maria|Universidade Federal do Rio Grande do Sul
Santa Maria, Brazil|Porto Alegre, Brazil
Tyler Moore
University of Pennsylvania
Philadelphia, PA
Ruben Gur
University of Pennsylvania
Philadelphia, PA
Giovanni Salum
Universidade Federal do Rio Grande do Sul|National Institute of Developmental Psychiatry for Children and Adolescents (INCT-CNPq)|Child Mind Institute
Porto Alegre, Brazil|São Paulo, Brazil|New York, United States
Michael Milham
Child Mind Institute|Nathan S. Kline Institute for Psychiatric Research
New York, NY|Orangeburg, NY
Simon Eickhoff
Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf|Research Centre Juelich, Institute of Neuroscience and Medicine (INM7)
Düsseldorf, North Rhine-Westphalia|Juelich, Germany
Robert Langner
Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf|Research Centre Juelich, Institute of Neuroscience and Medicine (INM7)
Düsseldorf, North Rhine-Westphalia|Juelich, Germany
Veronika Müller
Institute for Systems Neuroscience, Medical Faculty, Heinrich-Heine University Düsseldorf|Research Centre Juelich, Institute of Neuroscience and Medicine (INM7)
Düsseldorf, North Rhine-Westphalia|Juelich, Germany
Introduction:
Despite sustained efforts to uncover the neurobiological basis of mental health disorders, finding links to specific syndromes using neuroimaging has remained challenging. Factors contributing to this include both high comorbidity among disorders and heterogeneity within diagnostic categories1. In response, models that separate the shared (or general, transdiagnostic) and the unique (or specific) dimensions of psychopathology2 have become an important area of research. Such bi-factor models offer the possibility to disentangle the general and unique biological underpinnings of psychopathology. However, to investigate associations between individual differences in neurobiology and psychopathology such dimensions also require sufficient test-retest reliability, as it sets an upper bound on effect size3 and determines signal-to-noise ratio in machine learning analyses4. Here we evaluated the reliability of 11 published bi-factor models in two large developmental datasets from the global north and south and investigated the impact of reliability on their associations with brain structure and function.
Methods:
We used items from the Child Behaviour Checklist (CBCL) to fit 11 bi-factor models in the ABCD5 (n = 5526, ages 9-10 at baseline, mean retest interval = 12 months) and BHRC6 (n = 772, ages 6-14 at baseline, mean retest interval = 8 months) datasets. CBCL items were configured to load on a single general factor and a varying number of specific factors as defined in all published models7. Test-retest reliability was then assessed using correlation, after regressing out participant age, timepoint and their interaction. Sensitivity analyses were performed using alternative reliability metrics (hierarchical omega and factor determinacy) and shorter retest intervals. Next, we used resting-state functional connectivity (FC) and cortical thickness (CT) to predict general and specific factors as well as summary scores in the ABCD dataset. Predictions were performed with linear ridge regression within nested 2-fold cross-validation using matched discovery (n=3,525) and replication (n=3,447) samples8. Using Mplus, bi-factor models were fit separately in each sample to avoid data leakage and improve replicability. Preprocessed resting-state fMRI data and CT maps were acquired from the ABCD BIDS Community Collection9 and parcellated using the Glasser atlas10. FC was then calculated using Pearson correlation between the time series of all parcels.
Results:
For all 11 models, the general "P" factor captured most variance across CBCL items (ABCD = 57-78%; BHRC = 60-79%) and had generally higher reliability (test-retest rABCD = 0.7 - 0.76; rBHRC = 0.55 - 0.59) than specific factors (rABCD = 0.36 - 0.61; rBHRC = 0.14 - 0.47) in both datasets (Fig. 1). Despite the favourable psychometric properties, p-factors could be predicted using FC (Fig. 2A) or CT with a comparable prediction accuracy to externalising and attention factors. Notably, p-factors also showed equivalent prediction accuracy and reliability to many standard CBCL summary scores (esp. total problems, attention, internalising and rule-breaking), which combine all sources of variance and ignore the multidimensional structure parsed by the bi-factor models (Fig. 2B).
Conclusions:
Here we demonstrate that while capturing most of the reliable variance in CBCL, general psychopathology factors showed comparable associations with brain structure and function to specific factors and summary scores. Across all bi-factor models, many specific factors displayed low test-retest reliability that will substantially attenuate associations and make comparisons between factors hard to interpret. These results suggest that deeper phenotyping is necessary to better characterise the variance unique to specific dimensions. Finally, while bi-factor models are better suited to address phenotypic complexity and heterogeneity in psychopathology, general factors exhibit predictive utility comparable to that of the CBCL total summary score.
Disorders of the Nervous System:
Psychiatric (eg. Depression, Anxiety, Schizophrenia) 1
Modeling and Analysis Methods:
Classification and Predictive Modeling 2
Connectivity (eg. functional, effective, structural)
Keywords:
Data analysis
Design and Analysis
Development
FUNCTIONAL MRI
Machine Learning
Psychiatric
Psychiatric Disorders
Statistical Methods
STRUCTURAL MRI
Other - Reliability
1|2Indicates the priority used for review
Provide references using author date format
1. Feczko, E. & Fair, D. A. Methods and Challenges for Assessing Heterogeneity. Biol. Psychiatry 88, 9–17 (2020).
2. Kotov, R. et al. The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology 126, 454–477 (2017).
3. Spearman, C. Correlation calculated from faulty data. Br. J. Psychol. 3, 271–295 (1910).
4. Gell, M. et al. The Burden of Reliability: How Measurement Noise Limits Brain-Behaviour Predictions. 2023.02.09.527898 Preprint at https://doi.org/10.1101/2023.02.09.527898 (2023).
5. Volkow, N. D. et al. The conception of the ABCD study: From substance use to a broad NIH collaboration. Dev. Cogn. Neurosci. 32, 4–7 (2018).
6. Salum, G. A. et al. High risk cohort study for psychiatric disorders in childhood: rationale, design, methods and preliminary results. Int. J. Methods Psychiatr. Res. 24, 58–73 (2015).
7. Hoffmann, M. et al. Reliability and validity of bifactor models of dimensional psychopathology in youth. J. Psychopathol. Clin. Sci. 131, 407–421 (2022).
8. Feczko, E. et al. Adolescent Brain Cognitive Development (ABCD) Community MRI Collection and Utilities. 2021.07.09.451638 Preprint at https://doi.org/10.1101/2021.07.09.451638 (2021).
9. Fair, D. A., Dosenbach, N. U. F., Moore, A. H., Satterthwaite, T. D. & Milham, M. P. Developmental Cognitive Neuroscience in the Era of Networks and Big Data: Strengths, Weaknesses, Opportunities, and Threats. Annu. Rev. Dev. Psychol. 3, 249–275 (2021).
10. Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016).